Mercurial > hgsubversion

annotate tests/test_fetch_symlinks.py @ 931:e1dbd9646d6a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
svnwrap: use custom StringIO class in get_file() The wrappers were calling ra.get_file() with a cStringIO object. Empirically, svn 1.7.5 is writing 16kB blocks to the stream object, and cStringIO reallocates its internal buffer and doubles its size whenever it is filled. With large committed files this requires two large memory blocks at the same time. SimpleStringIO implements the mimimum StringIO interface used by ra.get_file() but instead stores all the blocks and "join" them at the end. It means more fragmentation but requires only one large block, without overallocation. Also, 16kB blocks should be friendly to most allocators. In practice, this simple change let me convert a revision containing multiple moderately large files, the largest being around 450MB, with a 32-bits Windows setup, python 2.7, swig svn 1.7.5, in stupid mode, while it was previously aborting with "not enough memory". The same revision still fails in replay mode.
author Patrick Mezard <patrick@mezard.eu>
date Sun, 16 Sep 2012 19:31:49 +0200
parents bd12a4da0f35
children d741f536f23a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
97
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
1 import test_util
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
2
643
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 145
diff changeset
3 import unittest
97
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
4
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
5 class TestFetchSymlinks(test_util.TestBase):
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
6 def test_symlinks(self, stupid=False):
865
04729f3a3d17 test_util: merge load_fixture_and_fetch() into TestBase method
Patrick Mezard <patrick@mezard.eu>
parents: 859
diff changeset
7 repo = self._load_fixture_and_fetch('symlinks.svndump', stupid=stupid)
741
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
8 # Check symlinks throughout history
97
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
9 links = {
126
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
10 0: {
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
11 'linka': 'a',
145
b37c401b7f92 hg_delta_editor: reset properties of deleted entries
Patrick Mezard <pmezard@gmail.com>
parents: 126
diff changeset
12 'linka2': 'a',
126
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
13 'd/linka': 'a',
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
14 },
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
15 1: {
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
16 'linkaa': 'a',
145
b37c401b7f92 hg_delta_editor: reset properties of deleted entries
Patrick Mezard <pmezard@gmail.com>
parents: 126
diff changeset
17 'linka2': 'a',
126
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
18 'd2/linka': 'a',
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
19 },
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
20 2: {
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
21 'linkaa': 'b',
145
b37c401b7f92 hg_delta_editor: reset properties of deleted entries
Patrick Mezard <pmezard@gmail.com>
parents: 126
diff changeset
22 'linka2': 'a',
126
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
23 'd2/linka': 'b',
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
24 },
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
25 3: {
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
26 },
741
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
27 4: {
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
28 'linka3': 'a',
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
29 },
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
30 5: {
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
31 'linka3': 'a',
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
32 },
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
33 6: {
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
34 'linka3': 'a',
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
35 'linka4': 'link to this',
b3128fec5d54 editor: handle property changes to links.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
36 },
97
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
37 }
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 741
diff changeset
38
126
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
39 for rev in repo:
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
40 ctx = repo[rev]
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
41 for f in ctx.manifest():
859
1d07e86f5797 stupid: handle changes in svn 1.7 diff format
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
42 l = 'l' in ctx[f].flags()
1d07e86f5797 stupid: handle changes in svn 1.7 diff format
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
43 lref = f in links[rev]
1d07e86f5797 stupid: handle changes in svn 1.7 diff format
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
44 self.assertEqual(lref, l, '%r != %r for %s@%r' % (lref, l, f, rev))
126
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
45 if f in links[rev]:
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
46 self.assertEqual(links[rev][f], ctx[f].data())
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
47 for f in links[rev]:
24a64fb0e74b run.py: register and develop test_fetch_symlinks
Patrick Mezard <pmezard@gmail.com>
parents: 101
diff changeset
48 self.assertTrue(f in ctx)
97
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
49
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
50 def test_symlinks_stupid(self):
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
51 self.test_symlinks(True)
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
52
901
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
53 class TestMergeSpecial(test_util.TestBase):
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
54 def test_special(self):
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
55 repo = self._load_fixture_and_fetch('addspecial.svndump',
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
56 subdir='trunk')
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
57 ctx = repo['tip']
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
58 self.assertEqual(ctx['fnord'].flags(), 'l')
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
59 self.assertEqual(ctx['exe'].flags(), 'x')
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
60
97
0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
61 def suite():
901
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
62 all_tests = [
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
63 unittest.TestLoader().loadTestsFromTestCase(TestFetchSymlinks),
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
64 unittest.TestLoader().loadTestsFromTestCase(TestMergeSpecial),
bd12a4da0f35 replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents: 865
diff changeset
65 ]
833
312b37bc5e20 tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents: 832
diff changeset
66 return unittest.TestSuite(all_tests)