annotate tests/test_diff.py @ 931:e1dbd9646d6a

svnwrap: use custom StringIO class in get_file() The wrappers were calling ra.get_file() with a cStringIO object. Empirically, svn 1.7.5 is writing 16kB blocks to the stream object, and cStringIO reallocates its internal buffer and doubles its size whenever it is filled. With large committed files this requires two large memory blocks at the same time. SimpleStringIO implements the mimimum StringIO interface used by ra.get_file() but instead stores all the blocks and "join" them at the end. It means more fragmentation but requires only one large block, without overallocation. Also, 16kB blocks should be friendly to most allocators. In practice, this simple change let me convert a revision containing multiple moderately large files, the largest being around 450MB, with a 32-bits Windows setup, python 2.7, swig svn 1.7.5, in stupid mode, while it was previously aborting with "not enough memory". The same revision still fails in replay mode.
author Patrick Mezard <patrick@mezard.eu>
date Sun, 16 Sep 2012 19:31:49 +0200
parents 312b37bc5e20
children d741f536f23a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
643
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 347
diff changeset
1 import test_util
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 347
diff changeset
2
153
46f6b872c988 tests: Fix some missing suite definitions so that running the full testsuite
Augie Fackler <durin42@gmail.com>
parents: 138
diff changeset
3 import unittest
46f6b872c988 tests: Fix some missing suite definitions so that running the full testsuite
Augie Fackler <durin42@gmail.com>
parents: 138
diff changeset
4
138
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
5 from mercurial import ui
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
6
337
46e69be8e2c8 Reorganize to have a more conventional module structure.
Augie Fackler <durin42@gmail.com>
parents: 331
diff changeset
7 from hgsubversion import wrappers
138
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
8
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
9 expected_diff_output = '''Index: alpha
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
10 ===================================================================
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
11 --- alpha\t(revision 3)
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
12 +++ alpha\t(working copy)
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
13 @@ -1,1 +1,3 @@
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
14 -file: alpha
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
15 +alpha
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
16 +
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
17 +added line
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
18 Index: foo
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
19 ===================================================================
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
20 new file mode 100644
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
21 --- foo\t(revision 0)
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
22 +++ foo\t(working copy)
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
23 @@ -0,0 +1,1 @@
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
24 +This is missing a newline.
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
25 \ No newline at end of file
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
26 '''
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
27
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
28 class DiffTests(test_util.TestBase):
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
29 def test_diff_output(self):
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
30 self._load_fixture_and_fetch('two_revs.svndump')
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
31 self.commitchanges([('foo', 'foo', 'This is missing a newline.'),
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
32 ('alpha', 'alpha', 'alpha\n\nadded line\n'),
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
33 ])
40474f6c1f84 diff_cmd: more robust, add tests.
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
34 u = ui.ui()
331
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 257
diff changeset
35 u.pushbuffer()
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 644
diff changeset
36 wrappers.diff(lambda x, y, z: None, u, self.repo, svn=True)
331
75f082b5897e Switch to using url scheme wrappers instead of duplicating each command we wrap.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 257
diff changeset
37 self.assertEqual(u.popbuffer(), expected_diff_output)
153
46f6b872c988 tests: Fix some missing suite definitions so that running the full testsuite
Augie Fackler <durin42@gmail.com>
parents: 138
diff changeset
38
46f6b872c988 tests: Fix some missing suite definitions so that running the full testsuite
Augie Fackler <durin42@gmail.com>
parents: 138
diff changeset
39
46f6b872c988 tests: Fix some missing suite definitions so that running the full testsuite
Augie Fackler <durin42@gmail.com>
parents: 138
diff changeset
40 def suite():
833
312b37bc5e20 tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents: 832
diff changeset
41 all_tests = [unittest.TestLoader().loadTestsFromTestCase(DiffTests),
153
46f6b872c988 tests: Fix some missing suite definitions so that running the full testsuite
Augie Fackler <durin42@gmail.com>
parents: 138
diff changeset
42 ]
833
312b37bc5e20 tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents: 832
diff changeset
43 return unittest.TestSuite(all_tests)