annotate tests/test_push_dirs.py @ 931:e1dbd9646d6a

svnwrap: use custom StringIO class in get_file() The wrappers were calling ra.get_file() with a cStringIO object. Empirically, svn 1.7.5 is writing 16kB blocks to the stream object, and cStringIO reallocates its internal buffer and doubles its size whenever it is filled. With large committed files this requires two large memory blocks at the same time. SimpleStringIO implements the mimimum StringIO interface used by ra.get_file() but instead stores all the blocks and "join" them at the end. It means more fragmentation but requires only one large block, without overallocation. Also, 16kB blocks should be friendly to most allocators. In practice, this simple change let me convert a revision containing multiple moderately large files, the largest being around 450MB, with a 32-bits Windows setup, python 2.7, swig svn 1.7.5, in stupid mode, while it was previously aborting with "not enough memory". The same revision still fails in replay mode.
author Patrick Mezard <patrick@mezard.eu>
date Sun, 16 Sep 2012 19:31:49 +0200
parents db3a651494f9
children d741f536f23a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
1 import test_util
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
2
643
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 323
diff changeset
3 import unittest
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 323
diff changeset
4
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
5 class TestPushDirectories(test_util.TestBase):
84
01e747937d35 test_util: add commitchanges() to TestBase
Patrick Mezard <pmezard@gmail.com>
parents: 83
diff changeset
6 def test_push_dirs(self):
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
7 repo_path = self.load_and_fetch('emptyrepo.svndump')[1]
700
04b3f476e2c3 test_push_dirs.py: activate tests that were previously overlooked
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 643
diff changeset
8
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
9 changes = [
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
10 # Single file in single directory
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
11 ('d1/a', 'd1/a', 'a\n'),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
12 # Two files in one directory
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
13 ('d2/a', 'd2/a', 'a\n'),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
14 ('d2/b', 'd2/b', 'a\n'),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
15 # Single file in empty directory hierarchy
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
16 ('d31/d32/d33/d34/a', 'd31/d32/d33/d34/a', 'a\n'),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
17 ('d31/d32/a', 'd31/d32/a', 'a\n'),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
18 ]
84
01e747937d35 test_util: add commitchanges() to TestBase
Patrick Mezard <pmezard@gmail.com>
parents: 83
diff changeset
19 self.commitchanges(changes)
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
20 self.pushrevisions()
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
21 self.assertEqual(test_util.svnls(repo_path, 'trunk'),
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
22 ['d1', 'd1/a', 'd2', 'd2/a', 'd2/b', 'd31',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
23 'd31/d32', 'd31/d32/a', 'd31/d32/d33',
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
24 'd31/d32/d33/d34', 'd31/d32/d33/d34/a'])
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
25
85
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
26 # Add one revision with changed files only, no directory addition
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
27 # or deletion.
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
28 changes = [
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
29 ('d1/a', 'd1/a', 'aa\n'),
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
30 ('d2/a', 'd2/a', 'aa\n'),
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
31 ]
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
32 self.commitchanges(changes)
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
33 self.pushrevisions()
05a0c4f6060f push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents: 84
diff changeset
34
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
35 changes = [
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
36 # Remove single file in single directory
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
37 ('d1/a', None, None),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
38 # Remove one file out of two
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
39 ('d2/a', None, None),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
40 # Removing this file should remove one empty parent dir too
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
41 ('d31/d32/d33/d34/a', None, None),
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
42 ]
84
01e747937d35 test_util: add commitchanges() to TestBase
Patrick Mezard <pmezard@gmail.com>
parents: 83
diff changeset
43 self.commitchanges(changes)
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
44 self.pushrevisions()
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
45 self.assertEqual(test_util.svnls(repo_path, 'trunk'),
323
067914ecb4eb push: Fix a bug in deletion of an entire tree.
Augie Fackler <durin42@gmail.com>
parents: 268
diff changeset
46 ['d2', 'd2/b', 'd31', 'd31/d32', 'd31/d32/a', ])
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
47
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
48 def test_push_new_dir_project_root_not_repo_root(self):
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
49 repo_path = self.load_and_fetch('fetch_missing_files_subdir.svndump',
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
50 subdir='foo')[1]
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 731
diff changeset
51 changes = [('magic_new/a', 'magic_new/a', 'ohai',),
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
52 ]
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
53 self.commitchanges(changes)
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
54 self.pushrevisions()
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
55 self.assertEqual(test_util.svnls(repo_path, 'foo/trunk'), ['bar',
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
56 'bar/alpha',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
57 'bar/beta',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
58 'bar/delta',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
59 'bar/gamma',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
60 'foo',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
61 'magic_new',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
62 'magic_new/a'])
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
63
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
64 def test_push_new_file_existing_dir_root_not_repo_root(self):
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
65 repo_path = self.load_and_fetch('empty_dir_in_trunk_not_repo_root.svndump',
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
66 subdir='project')[1]
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 731
diff changeset
67 changes = [('narf/a', 'narf/a', 'ohai',),
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
68 ]
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
69 self.commitchanges(changes)
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
70 self.assertEqual(test_util.svnls(repo_path, 'project/trunk'), ['a',
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 731
diff changeset
71 'narf',
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 731
diff changeset
72 ])
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
73 self.pushrevisions()
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
74 self.assertEqual(test_util.svnls(repo_path, 'project/trunk'), ['a',
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
75 'narf',
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
76 'narf/a'])
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 731
diff changeset
77 changes = [('narf/a', None, None,),
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
78 ]
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
79 self.commitchanges(changes)
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
80 self.pushrevisions()
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
81 self.assertEqual(test_util.svnls(repo_path, 'project/trunk'), ['a'])
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
82
730
efb87d5bb311 subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 700
diff changeset
83 def test_push_single_dir_change_in_subdir(self):
efb87d5bb311 subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 700
diff changeset
84 # Tests simple pushing from default branch to a single dir repo
efb87d5bb311 subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 700
diff changeset
85 # Changes a file in a subdir (regression).
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
86 repo, repo_path = self.load_and_fetch('branch_from_tag.svndump',
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
87 stupid=False,
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
88 layout='single',
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
89 subdir='tags')
731
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
90 changes = [('tag_r3/alpha', 'tag_r3/alpha', 'foo'),
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
91 ('tag_r3/new', 'tag_r3/new', 'foo'),
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
92 ('new_dir/new', 'new_dir/new', 'foo'),
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
93 ]
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
94 self.commitchanges(changes)
730
efb87d5bb311 subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 700
diff changeset
95 self.pushrevisions()
869
db3a651494f9 test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
96 self.assertEqual(test_util.svnls(repo_path, 'tags'),
731
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
97 ['copied_tag',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
98 'copied_tag/alpha',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
99 'copied_tag/beta',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
100 'new_dir',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
101 'new_dir/new',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
102 'tag_r3',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
103 'tag_r3/alpha',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
104 'tag_r3/beta',
de036c2cb36a test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents: 730
diff changeset
105 'tag_r3/new'])
268
6ec5b5fc5b4d Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents: 85
diff changeset
106
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
107 def suite():
833
312b37bc5e20 tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents: 832
diff changeset
108 all_tests = [unittest.TestLoader().loadTestsFromTestCase(TestPushDirectories),
83
6c9b7cf1c5aa push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff changeset
109 ]
833
312b37bc5e20 tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents: 832
diff changeset
110 return unittest.TestSuite(all_tests)