hgsubversion: tests/comprehensive/test_verify_and

annotate tests/comprehensive/test_verify_and_startrev.py @ 1096:691078c03ed9

verify: use mercurials worker API to speedup stupid verify We use mercurial internal worker api to distribute the workload during verify. This helps with larger repositories. with patch, intel i5 dualcore hyperthreading: $ time hg svn verify hg svn 169.47s user 21.27s system 373% cpu 51.050 total before: $ time hg svn verify hg svn 100.48s user 11.85s system 99% cpu 1:52.97 total This makes stupid verify as fast as subvertpy verify. We use \0 as a separator for serializing the data as it's reserved on common fs.

author	David Soria Parra <dsp@experimentalworks.net>
date	Tue, 01 Oct 2013 16:55:56 +0200
parents	cd0d14e25757
children	ff4e102932ed

rev	line source
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	1 import os
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	2 import pickle
766 124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	3 import sys
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	4 import unittest
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	5
426 72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	6 # wrapped in a try/except because of weirdness in how
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	7 # run.py works as compared to nose.
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	8 try:
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	9 import test_util
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	10 except ImportError:
766 124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	11 sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	12 import test_util
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	13
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	14 from mercurial import hg
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	15 from mercurial import ui
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	16
897 6bc8046e3d0a move verify to a file of its own Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 894 diff changeset	17 from hgsubversion import verify
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	18
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	19 # these fixtures contain no files at HEAD and would result in empty clones
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	20 _skipshallow = set([
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	21 'binaryfiles.svndump',
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	22 'binaryfiles-broken.svndump',
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	23 'emptyrepo.svndump',
892 3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	24 'correct.svndump',
3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	25 'corrupt.svndump',
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	26 ])
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	27
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	28 _skipall = set([
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	29 'project_root_not_repo_root.svndump',
961 b729909d3793 tests: add tests where files are moved from root to trunk Patrick Mezard <patrick@mezard.eu> parents: 938 diff changeset	30 'movetotrunk.svndump',
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	31 ])
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	32
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	33 _skipstandard = set([
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	34 'subdir_is_file_prefix.svndump',
892 3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	35 'correct.svndump',
3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	36 'corrupt.svndump',
938 f9014e28721b editor: start separating svn copies from open files Patrick Mezard <patrick@mezard.eu> parents: 899 diff changeset	37 'emptyrepo2.svndump',
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	38 ])
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	39
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	40 def _do_case(self, name, layout):
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	41 subdir = test_util.subdir.get(name, '')
1092 cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	42 config = {}
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	43 for branch, path in test_util.custom.get(name, {}).iteritems():
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	44 config['hgsubversionbranch.%s' % branch] = path
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	45 repo, svnpath = self.load_and_fetch(name,
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	46 subdir=subdir,
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	47 layout=layout,
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	48 config=config)
1048 903c9c9dfe6a tests: count revisions explicitly Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 1046 diff changeset	49 assert test_util.repolen(self.repo) > 0
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	50 for i in repo:
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	51 ctx = repo[i]
899 7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	52 self.assertEqual(verify.verify(repo.ui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	53 stupid=True), 0)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	54 self.assertEqual(verify.verify(repo.ui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	55 stupid=False), 0)
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	56
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	57 # check a startrev clone
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	58 if layout == 'single' and name not in _skipshallow:
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	59 self.wc_path += '_shallow'
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	60 shallowrepo = self.fetch(svnpath, subdir=subdir,
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	61 layout='single', startrev='HEAD')
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	62
1048 903c9c9dfe6a tests: count revisions explicitly Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 1046 diff changeset	63 self.assertEqual(test_util.repolen(shallowrepo), 1,
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	64 "shallow clone should have just one revision, not %d"
1048 903c9c9dfe6a tests: count revisions explicitly Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 1046 diff changeset	65 % test_util.repolen(shallowrepo))
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	66
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	67 fulltip = repo['tip']
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	68 shallowtip = shallowrepo['tip']
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	69
894 9562f606c4aa make assertion failures slightly more helpful by including the failing verification Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 892 diff changeset	70 repo.ui.pushbuffer()
897 6bc8046e3d0a move verify to a file of its own Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 894 diff changeset	71 self.assertEqual(0, verify.verify(repo.ui, shallowrepo,
899 7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	72 rev=shallowtip.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	73 stupid=True))
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	74 self.assertEqual(0, verify.verify(repo.ui, shallowrepo,
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	75 rev=shallowtip.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	76 stupid=False))
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	77
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	78 stupidui = ui.ui(repo.ui)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	79 stupidui.config('hgsubversion', 'stupid', True)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	80 self.assertEqual(verify.verify(stupidui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	81 stupid=True), 0)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	82 self.assertEqual(verify.verify(stupidui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	83 stupid=False), 0)
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	84
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	85 # viewing diff's of lists of files is easier on the eyes
894 9562f606c4aa make assertion failures slightly more helpful by including the failing verification Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 892 diff changeset	86 self.assertMultiLineEqual('\n'.join(fulltip), '\n'.join(shallowtip),
9562f606c4aa make assertion failures slightly more helpful by including the failing verification Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 892 diff changeset	87 repo.ui.popbuffer())
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	88
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	89 for f in fulltip:
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	90 self.assertMultiLineEqual(fulltip[f].data(), shallowtip[f].data())
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	91
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	92
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	93 def buildmethod(case, name, layout):
cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	94 m = lambda self: self._do_case(case, layout)
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	95 m.__name__ = name
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	96 m.__doc__ = 'Test verify on %s (%s)' % (case, layout)
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	97 return m
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	98
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	99 attrs = {'_do_case': _do_case, 'stupid_mode_tests': True}
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	100 fixtures = [f for f in os.listdir(test_util.FIXTURES) if f.endswith('.svndump')]
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	101 for case in fixtures:
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	102 if case in _skipall:
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	103 continue
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 441 diff changeset	104 bname = 'test_' + case[:-len('.svndump')]
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	105 if case not in _skipstandard:
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	106 attrs[bname] = buildmethod(case, bname, 'standard')
1092 cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	107 attrs[bname + '_single'] = buildmethod(case, bname + '_single', 'single')
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	108 if case in test_util.custom:
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	109 attrs[bname + '_custom'] = buildmethod(case, bname + '_custom', 'custom')
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	110
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	111 VerifyTests = type('VerifyTests', (test_util.TestBase,), attrs)

Mercurial > hgsubversion

annotate tests/comprehensive/test_verify_and_startrev.py @ 1096:691078c03ed9