Mercurial > hgsubversion

annotate tests/test_rebuildmeta.py @ 787:4bbc6bf947f5 1.2.1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
replay: fetch full revision at most once per run (issue252) Before this change, hgsubversion was fetching full revisions from the first revision the project was created to the first revision containing converted data. Unfortunately, some projects exhibits such spans longer than 500 revisions, during which hgsubversion was uselessly scanning the whole tree. The fix is not technically perfect, we could record somewhere that while no data was converted we scanned the project already, instead of scanning once at every hgsubversion run until a revision is converted. But it should be good enough unless someone runs hgsubversion once for every target revision. One repository exhibiting this behaviour: svn://svn.zankasoftware.com/zanka
author Patrick Mezard <pmezard@gmail.com>
date Sun, 13 Feb 2011 20:10:52 +0100
parents e1e2af66953d
children 86d124a8768e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
643
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 638
diff changeset
1 import test_util
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 638
diff changeset
2
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
3 import os
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
4 import pickle
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 import unittest
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6
638
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
7 from mercurial import context
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
8 from mercurial import extensions
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 from mercurial import hg
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
10 from mercurial import ui
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
11
337
46e69be8e2c8 Reorganize to have a more conventional module structure.
Augie Fackler <durin42@gmail.com>
parents: 331
diff changeset
12 from hgsubversion import svncommands
416
cd6317fe70be invert the svnmeta/editor relationship
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 414
diff changeset
13 from hgsubversion import svnmeta
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
14
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
15 def _do_case(self, name, stupid, single):
194
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing.
Augie Fackler <durin42@gmail.com>
parents: 182
diff changeset
16 subdir = test_util.subdir.get(name, '')
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
17 layout = 'auto'
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
18 if single:
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
19 layout = 'single'
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
20 self._load_fixture_and_fetch(name, subdir=subdir, stupid=stupid, layout=layout)
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
21 assert len(self.repo) > 0
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
22 wc2_path = self.wc_path + '_clone'
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
23 u = ui.ui()
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
24 src, dest = hg.clone(u, self.wc_path, wc2_path, update=False)
638
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
25
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
26 # insert a wrapper that prevents calling changectx.children()
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
27 def failfn(orig, ctx):
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
28 self.fail('calling %s is forbidden; it can cause massive slowdowns '
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
29 'when rebuilding large repositories' % orig)
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
30
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
31 origchildren = getattr(context.changectx, 'children')
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
32 extensions.wrapfunction(context.changectx, 'children', failfn)
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
33
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
34 try:
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
35 svncommands.rebuildmeta(u, dest,
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
36 args=[test_util.fileurl(self.repo_path +
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
37 subdir), ])
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
38 finally:
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
39 # remove the wrapper
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
40 context.changectx.children = origchildren
ea0f42e0004d tests: disallow calling changectx.children() during rebuildmeta
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 622
diff changeset
41
331
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
42 self.assertTrue(os.path.isdir(os.path.join(src.path, 'svn')),
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
43 'no .hg/svn directory in the source!')
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
44 self.assertTrue(os.path.isdir(os.path.join(src.path, 'svn')),
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
45 'no .hg/svn directory in the destination!')
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
46 dest = hg.repository(u, os.path.dirname(dest.path))
748
e1e2af66953d svnmeta: store subdir in a file, and verify it when loading.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 655
diff changeset
47 for tf in ('rev_map', 'uuid', 'tagmap', 'layout', 'subdir', ):
331
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
48 stf = os.path.join(src.path, 'svn', tf)
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
49 self.assertTrue(os.path.isfile(stf), '%r is missing!' % stf)
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
50 dtf = os.path.join(dest.path, 'svn', tf)
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 246
diff changeset
51 self.assertTrue(os.path.isfile(dtf), '%r is missing!' % tf)
447
0d3b5acb1d51 tags: handle edits to tags as gracefully as possible
Augie Fackler <durin42@gmail.com>
parents: 416
diff changeset
52 old, new = open(stf).read(), open(dtf).read()
622
25714b4954b7 test_rebuildmeta: use just added diff support when comparing files.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 573
diff changeset
53 self.assertMultiLineEqual(old, new)
447
0d3b5acb1d51 tags: handle edits to tags as gracefully as possible
Augie Fackler <durin42@gmail.com>
parents: 416
diff changeset
54 self.assertEqual(src.branchtags(), dest.branchtags())
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
55 srcbi = pickle.load(open(os.path.join(src.path, 'svn', 'branch_info')))
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
56 destbi = pickle.load(open(os.path.join(dest.path, 'svn', 'branch_info')))
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
57 self.assertEqual(sorted(srcbi.keys()), sorted(destbi.keys()))
416
cd6317fe70be invert the svnmeta/editor relationship
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 414
diff changeset
58 revkeys = svnmeta.SVNMeta(dest).revmap.keys()
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
59 for branch in destbi:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
60 srcinfo = srcbi[branch]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
61 destinfo = destbi[branch]
447
0d3b5acb1d51 tags: handle edits to tags as gracefully as possible
Augie Fackler <durin42@gmail.com>
parents: 416
diff changeset
62 if srcinfo[:2] == (None, 0) or destinfo[:2] == (None, 0):
655
dcac6dc1abe6 test_rebuildmeta: make an assertion more helpful.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
63 self.assertTrue(srcinfo[2] <= destinfo[2],
dcac6dc1abe6 test_rebuildmeta: make an assertion more helpful.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
64 'Latest revision for %s decreased from %d to %d!'
dcac6dc1abe6 test_rebuildmeta: make an assertion more helpful.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
65 % (branch or 'default', srcinfo[2], destinfo[2]))
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
66 self.assertEqual(srcinfo[0], destinfo[0])
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
67 else:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
68 pr = sorted(filter(lambda x: x[1] == srcinfo[0] and x[0] <= srcinfo[1],
405
a98b8d424221 editor: simplify HgChangeReceiver constructor signature
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 383
diff changeset
69 revkeys), reverse=True)[0][0]
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
70 self.assertEqual(pr, destinfo[1])
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
71 self.assertEqual(srcinfo[2], destinfo[2])
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
72
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
73
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
74 def buildmethod(case, name, stupid, single):
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
75 m = lambda self: self._do_case(case, stupid, single)
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
76 m.__name__ = name
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
77 m.__doc__ = ('Test rebuildmeta on %s with %s replay. (%s)' %
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
78 (case,
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
79 (stupid and 'stupid') or 'real',
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
80 (single and 'single') or 'standard',
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
81 )
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
82 )
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
83 return m
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
84
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
85
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
86 attrs = {'_do_case': _do_case,
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
87 }
182
47d25d61abfa remove generators (compat with python2.3)
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 155
diff changeset
88 for case in [f for f in os.listdir(test_util.FIXTURES) if f.endswith('.svndump')]:
347
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 337
diff changeset
89 # this fixture results in an empty repository, don't use it
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 337
diff changeset
90 if case == 'project_root_not_repo_root.svndump':
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 337
diff changeset
91 continue
499
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
92 bname = 'test_' + case[:-len('.svndump')]
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
93 attrs[bname] = buildmethod(case, bname, False, False)
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
94 name = bname + '_stupid'
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
95 attrs[name] = buildmethod(case, name, True, False)
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
96 name = bname + '_single'
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
97 attrs[name] = buildmethod(case, name, False, True)
1fd3cfa47c5e Support for single-directory clones.
Augie Fackler <durin42@gmail.com>
parents: 474
diff changeset
98
155
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
99 RebuildMetaTests = type('RebuildMetaTests', (test_util.TestBase, ), attrs)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
100
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
101
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
102 def suite():
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
103 all = [unittest.TestLoader().loadTestsFromTestCase(RebuildMetaTests),
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
104 ]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
105 return unittest.TestSuite(all)