hgsubversion: tests/comprehensive/test_verify_and

annotate tests/comprehensive/test_verify_and_startrev.py @ 1233:0d0132cba155

editor: fix edge case with in memory file-store size limit There are a few cases where we will set a single file into to the editor's FileStore object more than once. Notably, for copied and then modified files, we will set it at least twice. Three times if editing fails (which it can for symlinks). If we pass the in-memory storage limit in between the first (or second if editing fails) time we set the file and the last time we set the file, we will write the data to the in memory store the first time and the file store the last time. We didn't remove it form the in-memory store though, and we always prefer reading from the in-memory store. This means we can sometimes end up with the wrong version of a file. This is fairly unlikely to happen in normal use since you need to hit the memory limit between two writes to the store for the same file. We only write a file multiple times if a) the file (and not one of it's parent directories) is copied and then modified or b) editing fails. From what I can tell, it's only common for editing to fail for symlinks, and they ten to be relatively small data that is unlikely to push over the limit. Finally, the default limit is 100MB which I would expect to be most often either well over (source code) or well under (binaries or automated changes) the size of the changes files in a single commit. The easiest way to reproduce this is to set the in-memory cache size to 0 and then commit a copied and modified symlink. The empty-string version from the failed editing will be the one that persists. I happened to stumble upon this while trying (and failing) to test a bug-fix for a related bug with identical symptoms (empty simlink). I have seen this in the wild, once, but couldn't reproduce it at the time. The repo in question is quite large and quite active, so I am quite confident in my estimation that this is a real, but very rare, problem. The test changes attached to this was mneant to test a related bug, but turned out not to actually cover the bug in question. They did trigger this bug though, and are worthwhile to test, so I kept them.

author	David Schleimer <dschleimer@fb.com>
date	Mon, 07 Apr 2014 17:51:59 -0700
parents	ff4e102932ed
children	c6b01fd34694

rev	line source
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	1 import os
766 124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	2 import sys
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	3 import unittest
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	4
426 72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	5 # wrapped in a try/except because of weirdness in how
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	6 # run.py works as compared to nose.
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	7 try:
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	8 import test_util
72e63999722f tests: make comprehensive tests work properly with nose and run.py Augie Fackler <durin42@gmail.com> parents: 395 diff changeset	9 except ImportError:
766 124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	10 sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	11 import test_util
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	12
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	13 from mercurial import hg
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	14 from mercurial import ui
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	15
897 6bc8046e3d0a move verify to a file of its own Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 894 diff changeset	16 from hgsubversion import verify
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	17
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	18 # these fixtures contain no files at HEAD and would result in empty clones
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	19 _skipshallow = set([
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	20 'binaryfiles.svndump',
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	21 'binaryfiles-broken.svndump',
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	22 'emptyrepo.svndump',
892 3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	23 'correct.svndump',
3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	24 'corrupt.svndump',
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	25 ])
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	26
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	27 _skipall = set([
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	28 'project_root_not_repo_root.svndump',
961 b729909d3793 tests: add tests where files are moved from root to trunk Patrick Mezard <patrick@mezard.eu> parents: 938 diff changeset	29 'movetotrunk.svndump',
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	30 ])
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	31
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	32 _skipstandard = set([
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	33 'subdir_is_file_prefix.svndump',
892 3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	34 'correct.svndump',
3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 888 diff changeset	35 'corrupt.svndump',
938 f9014e28721b editor: start separating svn copies from open files Patrick Mezard <patrick@mezard.eu> parents: 899 diff changeset	36 'emptyrepo2.svndump',
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	37 ])
c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	38
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	39 def _do_case(self, name, layout):
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	40 subdir = test_util.subdir.get(name, '')
1092 cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	41 config = {}
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	42 for branch, path in test_util.custom.get(name, {}).iteritems():
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	43 config['hgsubversionbranch.%s' % branch] = path
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	44 repo, svnpath = self.load_and_fetch(name,
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	45 subdir=subdir,
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	46 layout=layout,
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	47 config=config)
1048 903c9c9dfe6a tests: count revisions explicitly Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 1046 diff changeset	48 assert test_util.repolen(self.repo) > 0
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	49 for i in repo:
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	50 ctx = repo[i]
899 7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	51 self.assertEqual(verify.verify(repo.ui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	52 stupid=True), 0)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	53 self.assertEqual(verify.verify(repo.ui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	54 stupid=False), 0)
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	55
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	56 # check a startrev clone
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	57 if layout == 'single' and name not in _skipshallow:
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	58 self.wc_path += '_shallow'
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	59 shallowrepo = self.fetch(svnpath, subdir=subdir,
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	60 layout='single', startrev='HEAD')
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	61
1048 903c9c9dfe6a tests: count revisions explicitly Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 1046 diff changeset	62 self.assertEqual(test_util.repolen(shallowrepo), 1,
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	63 "shallow clone should have just one revision, not %d"
1048 903c9c9dfe6a tests: count revisions explicitly Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 1046 diff changeset	64 % test_util.repolen(shallowrepo))
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	65
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	66 fulltip = repo['tip']
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	67 shallowtip = shallowrepo['tip']
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	68
894 9562f606c4aa make assertion failures slightly more helpful by including the failing verification Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 892 diff changeset	69 repo.ui.pushbuffer()
897 6bc8046e3d0a move verify to a file of its own Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 894 diff changeset	70 self.assertEqual(0, verify.verify(repo.ui, shallowrepo,
899 7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	71 rev=shallowtip.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	72 stupid=True))
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	73 self.assertEqual(0, verify.verify(repo.ui, shallowrepo,
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	74 rev=shallowtip.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	75 stupid=False))
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	76
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	77 stupidui = ui.ui(repo.ui)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	78 stupidui.config('hgsubversion', 'stupid', True)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	79 self.assertEqual(verify.verify(stupidui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	80 stupid=True), 0)
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	81 self.assertEqual(verify.verify(stupidui, repo, rev=ctx.node(),
7f90bb48c9de svn verify: use a custom editor and get_revision() Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 897 diff changeset	82 stupid=False), 0)
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	83
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	84 # viewing diff's of lists of files is easier on the eyes
894 9562f606c4aa make assertion failures slightly more helpful by including the failing verification Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 892 diff changeset	85 self.assertMultiLineEqual('\n'.join(fulltip), '\n'.join(shallowtip),
9562f606c4aa make assertion failures slightly more helpful by including the failing verification Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 892 diff changeset	86 repo.ui.popbuffer())
886 d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	87
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	88 for f in fulltip:
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	89 self.assertMultiLineEqual(fulltip[f].data(), shallowtip[f].data())
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	90
d3ff5807f1bd fold test_startrev and test_verify into a new test; test_verify_and_startrev Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 833 diff changeset	91
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	92 def buildmethod(case, name, layout):
cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	93 m = lambda self: self._do_case(case, layout)
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	94 m.__name__ = name
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	95 m.__doc__ = 'Test verify on %s (%s)' % (case, layout)
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	96 return m
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	97
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	98 attrs = {'_do_case': _do_case, 'stupid_mode_tests': True}
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	99 fixtures = [f for f in os.listdir(test_util.FIXTURES) if f.endswith('.svndump')]
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	100 for case in fixtures:
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	101 if case in _skipall:
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	102 continue
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 441 diff changeset	103 bname = 'test_' + case[:-len('.svndump')]
888 c6388ed0ec0a svnmeta: only remove directory components in normalize() Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 886 diff changeset	104 if case not in _skipstandard:
1057 cd256960b622 comprehensive tests: consolidate stupidity into test_util Dan Villiom Podlaski Christiansen <dan@cabo.dk> parents: 1048 diff changeset	105 attrs[bname] = buildmethod(case, bname, 'standard')
1092 cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	106 attrs[bname + '_single'] = buildmethod(case, bname + '_single', 'single')
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	107 if case in test_util.custom:
cd0d14e25757 layouts: add custom layout for those of us that need weird mappings David Schleimer <dschleimer@fb.com> parents: 1057 diff changeset	108 attrs[bname + '_custom'] = buildmethod(case, bname + '_custom', 'custom')
395 636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	109
636e9bf5d49c svncommands: add verify command Dirkjan Ochtman <dirkjan@ochtman.nl> parents: diff changeset	110 VerifyTests = type('VerifyTests', (test_util.TestBase,), attrs)

Mercurial > hgsubversion

annotate tests/comprehensive/test_verify_and_startrev.py @ 1233:0d0132cba155