Mercurial > hgsubversion

annotate tests/comprehensive/test_verify_and_startrev.py @ 899:7f90bb48c9de

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
svn verify: use a custom editor and get_revision() Previously, we would fetch each file in the revision/changeset individually. With this change, we fetch the entire revision in one request, and use a custom editor to verify its contents. This is quite a lot faster than the previous means when verifying over the internet. By an order of magnitude or two, in fact. As data is transfered in a single operation, verifying a revision from PyPy took 30 seconds rather than 30 minutes, and saturated my 10Mbps connection. Please note that the output ordering isn't stable between the two; output will appear in reverse order when using the fast verifier.
author Dan Villiom Podlaski Christiansen <danchr@gmail.com>
date Wed, 14 Dec 2011 00:07:58 +0100
parents 6bc8046e3d0a
children f9014e28721b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
1 import os
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
2 import pickle
766
124cd25de4ef tests: fix running the comprehensive tests using nose.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 499
diff changeset
3 import sys
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
4 import unittest
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
5
426
72e63999722f tests: make comprehensive tests work properly with nose and run.py
Augie Fackler <durin42@gmail.com>
parents: 395
diff changeset
6 # wrapped in a try/except because of weirdness in how
72e63999722f tests: make comprehensive tests work properly with nose and run.py
Augie Fackler <durin42@gmail.com>
parents: 395
diff changeset
7 # run.py works as compared to nose.
72e63999722f tests: make comprehensive tests work properly with nose and run.py
Augie Fackler <durin42@gmail.com>
parents: 395
diff changeset
8 try:
72e63999722f tests: make comprehensive tests work properly with nose and run.py
Augie Fackler <durin42@gmail.com>
parents: 395
diff changeset
9 import test_util
72e63999722f tests: make comprehensive tests work properly with nose and run.py
Augie Fackler <durin42@gmail.com>
parents: 395
diff changeset
10 except ImportError:
766
124cd25de4ef tests: fix running the comprehensive tests using nose.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 499
diff changeset
11 sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
124cd25de4ef tests: fix running the comprehensive tests using nose.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 499
diff changeset
12 import test_util
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
13
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
14 from mercurial import hg
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
15 from mercurial import ui
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
16
897
6bc8046e3d0a move verify to a file of its own
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 894
diff changeset
17 from hgsubversion import verify
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
18
886
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
19 # these fixtures contain no files at HEAD and would result in empty clones
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
20 _skipshallow = set([
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
21 'binaryfiles.svndump',
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
22 'binaryfiles-broken.svndump',
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
23 'emptyrepo.svndump',
892
3bfb7e985c47 svn verify: add a test for corrupt repositories.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 888
diff changeset
24 'correct.svndump',
3bfb7e985c47 svn verify: add a test for corrupt repositories.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 888
diff changeset
25 'corrupt.svndump',
886
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
26 ])
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
27
888
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
28 _skipall = set([
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
29 'project_root_not_repo_root.svndump',
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
30 ])
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
31
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
32 _skipstandard = set([
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
33 'subdir_is_file_prefix.svndump',
892
3bfb7e985c47 svn verify: add a test for corrupt repositories.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 888
diff changeset
34 'correct.svndump',
3bfb7e985c47 svn verify: add a test for corrupt repositories.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 888
diff changeset
35 'corrupt.svndump',
888
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
36 ])
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
37
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
38 def _do_case(self, name, stupid, layout):
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
39 subdir = test_util.subdir.get(name, '')
886
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
40 repo, svnpath = self.load_and_fetch(name, subdir=subdir, stupid=stupid,
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
41 layout=layout)
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
42 assert len(self.repo) > 0
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
43 for i in repo:
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
44 ctx = repo[i]
899
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
45 self.assertEqual(verify.verify(repo.ui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
46 stupid=True), 0)
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
47 self.assertEqual(verify.verify(repo.ui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
48 stupid=False), 0)
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
49
886
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
50 # check a startrev clone
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
51 if layout == 'single' and name not in _skipshallow:
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
52 self.wc_path += '_shallow'
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
53 shallowrepo = self.fetch(svnpath, subdir=subdir, stupid=stupid,
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
54 layout='single', startrev='HEAD')
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
55
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
56 self.assertEqual(len(shallowrepo), 1,
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
57 "shallow clone should have just one revision, not %d"
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
58 % len(shallowrepo))
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
59
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
60 fulltip = repo['tip']
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
61 shallowtip = shallowrepo['tip']
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
62
894
9562f606c4aa make assertion failures slightly more helpful by including the failing verification
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 892
diff changeset
63 repo.ui.pushbuffer()
897
6bc8046e3d0a move verify to a file of its own
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 894
diff changeset
64 self.assertEqual(0, verify.verify(repo.ui, shallowrepo,
899
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
65 rev=shallowtip.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
66 stupid=True))
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
67 self.assertEqual(0, verify.verify(repo.ui, shallowrepo,
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
68 rev=shallowtip.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
69 stupid=False))
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
70
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
71 stupidui = ui.ui(repo.ui)
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
72 stupidui.config('hgsubversion', 'stupid', True)
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
73 self.assertEqual(verify.verify(stupidui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
74 stupid=True), 0)
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
75 self.assertEqual(verify.verify(stupidui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision()
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 897
diff changeset
76 stupid=False), 0)
886
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
77
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
78 # viewing diff's of lists of files is easier on the eyes
894
9562f606c4aa make assertion failures slightly more helpful by including the failing verification
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 892
diff changeset
79 self.assertMultiLineEqual('\n'.join(fulltip), '\n'.join(shallowtip),
9562f606c4aa make assertion failures slightly more helpful by including the failing verification
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 892
diff changeset
80 repo.ui.popbuffer())
886
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
81
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
82 for f in fulltip:
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
83 self.assertMultiLineEqual(fulltip[f].data(), shallowtip[f].data())
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
84
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 833
diff changeset
85
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
86 def buildmethod(case, name, stupid, layout):
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
87 m = lambda self: self._do_case(case, stupid, layout)
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
88 m.__name__ = name
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
89 bits = case, stupid and 'stupid' or 'real', layout
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
90 m.__doc__ = 'Test verify on %s with %s replay. (%s)' % bits
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
91 return m
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
92
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
93 attrs = {'_do_case': _do_case}
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
94 fixtures = [f for f in os.listdir(test_util.FIXTURES) if f.endswith('.svndump')]
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
95 for case in fixtures:
888
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
96 if case in _skipall:
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
97 continue
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
98 bname = 'test_' + case[:-len('.svndump')]
888
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
99 if case not in _skipstandard:
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
100 attrs[bname] = buildmethod(case, bname, False, 'standard')
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
101 name = bname + '_stupid'
c6388ed0ec0a svnmeta: only remove directory components in normalize()
Dan Villiom Podlaski Christiansen <dan@cabo.dk>
parents: 886
diff changeset
102 attrs[name] = buildmethod(case, name, True, 'standard')
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
103 name = bname + '_single'
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
104 attrs[name] = buildmethod(case, name, False, 'single')
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
105 # Disabled because the "stupid and real are the same" tests
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
106 # verify this plus even more.
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
107 # name = bname + '_single_stupid'
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 441
diff changeset
108 # attrs[name] = buildmethod(case, name, True, 'single')
395
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
109
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
110 VerifyTests = type('VerifyTests', (test_util.TestBase,), attrs)
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
111
636e9bf5d49c svncommands: add verify command
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
112 def suite():
833
312b37bc5e20 tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents: 766
diff changeset
113 all_tests = [unittest.TestLoader().loadTestsFromTestCase(VerifyTests)]
312b37bc5e20 tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents: 766
diff changeset
114 return unittest.TestSuite(all_tests)