Mercurial > hgsubversion
annotate tests/test_push_dirs.py @ 890:78db88de9622
Partial metadata rebuilding
For highly active subversion repositories, it can be excruciatingly
slow to pull updates one at a time from subversion. One way around
this is to setup another mercurial repo that pulls new commits from
svn periodicly (say every 5 minutes). When you want to update your
repository, you can pull commits from this mercurial repository via
native mercurial protocols, which will be much faster than pulling
directly from svn.
Unfortunately, your metadata will be out of date after doing so.
Highly active repositories also tend to be very large, which means
that it takes a long time to rebuild your metadata from scratch. To
address this, this adds support to do a partial rebuild on the
metadata by processing only revisions that have been added to the
repository after the last revision we processed.
With the rev map 1k revisions (~2 days) behind tip updatemeta is
dramatically faster than rebuild meta:
$ hg --time svn updatemeta
Time: real 0.570 secs (user 0.480+0.000 sys 0.060+0.000)
$ hg --time svn rebuildmeta
Time: real 129.160 secs (user 128.570+0.000 sys 0.320+0.000)
author | David Schleimer <dschleimer@fb.com> |
---|---|
date | Sat, 12 May 2012 07:28:23 -0700 |
parents | db3a651494f9 |
children | d741f536f23a |
rev | line source |
---|---|
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
1 import test_util |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
2 |
643
d2ef7220a079
tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
323
diff
changeset
|
3 import unittest |
d2ef7220a079
tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
323
diff
changeset
|
4 |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
5 class TestPushDirectories(test_util.TestBase): |
84
01e747937d35
test_util: add commitchanges() to TestBase
Patrick Mezard <pmezard@gmail.com>
parents:
83
diff
changeset
|
6 def test_push_dirs(self): |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
7 repo_path = self.load_and_fetch('emptyrepo.svndump')[1] |
700
04b3f476e2c3
test_push_dirs.py: activate tests that were previously overlooked
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
643
diff
changeset
|
8 |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
9 changes = [ |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
10 # Single file in single directory |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
11 ('d1/a', 'd1/a', 'a\n'), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
12 # Two files in one directory |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
13 ('d2/a', 'd2/a', 'a\n'), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
14 ('d2/b', 'd2/b', 'a\n'), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
15 # Single file in empty directory hierarchy |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
16 ('d31/d32/d33/d34/a', 'd31/d32/d33/d34/a', 'a\n'), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
17 ('d31/d32/a', 'd31/d32/a', 'a\n'), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
18 ] |
84
01e747937d35
test_util: add commitchanges() to TestBase
Patrick Mezard <pmezard@gmail.com>
parents:
83
diff
changeset
|
19 self.commitchanges(changes) |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
20 self.pushrevisions() |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
21 self.assertEqual(test_util.svnls(repo_path, 'trunk'), |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
22 ['d1', 'd1/a', 'd2', 'd2/a', 'd2/b', 'd31', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
23 'd31/d32', 'd31/d32/a', 'd31/d32/d33', |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
24 'd31/d32/d33/d34', 'd31/d32/d33/d34/a']) |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
25 |
85
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
26 # Add one revision with changed files only, no directory addition |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
27 # or deletion. |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
28 changes = [ |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
29 ('d1/a', 'd1/a', 'aa\n'), |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
30 ('d2/a', 'd2/a', 'aa\n'), |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
31 ] |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
32 self.commitchanges(changes) |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
33 self.pushrevisions() |
05a0c4f6060f
push_cmd: consider only dirs with added/removed files for addition or deletion
Patrick Mezard <pmezard@gmail.com>
parents:
84
diff
changeset
|
34 |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
35 changes = [ |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
36 # Remove single file in single directory |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
37 ('d1/a', None, None), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
38 # Remove one file out of two |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
39 ('d2/a', None, None), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
40 # Removing this file should remove one empty parent dir too |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
41 ('d31/d32/d33/d34/a', None, None), |
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
42 ] |
84
01e747937d35
test_util: add commitchanges() to TestBase
Patrick Mezard <pmezard@gmail.com>
parents:
83
diff
changeset
|
43 self.commitchanges(changes) |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
44 self.pushrevisions() |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
45 self.assertEqual(test_util.svnls(repo_path, 'trunk'), |
323
067914ecb4eb
push: Fix a bug in deletion of an entire tree.
Augie Fackler <durin42@gmail.com>
parents:
268
diff
changeset
|
46 ['d2', 'd2/b', 'd31', 'd31/d32', 'd31/d32/a', ]) |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
47 |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
48 def test_push_new_dir_project_root_not_repo_root(self): |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
49 repo_path = self.load_and_fetch('fetch_missing_files_subdir.svndump', |
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
50 subdir='foo')[1] |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
731
diff
changeset
|
51 changes = [('magic_new/a', 'magic_new/a', 'ohai',), |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
52 ] |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
53 self.commitchanges(changes) |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
54 self.pushrevisions() |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
55 self.assertEqual(test_util.svnls(repo_path, 'foo/trunk'), ['bar', |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
56 'bar/alpha', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
57 'bar/beta', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
58 'bar/delta', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
59 'bar/gamma', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
60 'foo', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
61 'magic_new', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
62 'magic_new/a']) |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
63 |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
64 def test_push_new_file_existing_dir_root_not_repo_root(self): |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
65 repo_path = self.load_and_fetch('empty_dir_in_trunk_not_repo_root.svndump', |
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
66 subdir='project')[1] |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
731
diff
changeset
|
67 changes = [('narf/a', 'narf/a', 'ohai',), |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
68 ] |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
69 self.commitchanges(changes) |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
70 self.assertEqual(test_util.svnls(repo_path, 'project/trunk'), ['a', |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
731
diff
changeset
|
71 'narf', |
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
731
diff
changeset
|
72 ]) |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
73 self.pushrevisions() |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
74 self.assertEqual(test_util.svnls(repo_path, 'project/trunk'), ['a', |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
75 'narf', |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
76 'narf/a']) |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
731
diff
changeset
|
77 changes = [('narf/a', None, None,), |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
78 ] |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
79 self.commitchanges(changes) |
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
80 self.pushrevisions() |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
81 self.assertEqual(test_util.svnls(repo_path, 'project/trunk'), ['a']) |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
82 |
730
efb87d5bb311
subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
700
diff
changeset
|
83 def test_push_single_dir_change_in_subdir(self): |
efb87d5bb311
subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
700
diff
changeset
|
84 # Tests simple pushing from default branch to a single dir repo |
efb87d5bb311
subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
700
diff
changeset
|
85 # Changes a file in a subdir (regression). |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
86 repo, repo_path = self.load_and_fetch('branch_from_tag.svndump', |
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
87 stupid=False, |
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
88 layout='single', |
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
89 subdir='tags') |
731
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
90 changes = [('tag_r3/alpha', 'tag_r3/alpha', 'foo'), |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
91 ('tag_r3/new', 'tag_r3/new', 'foo'), |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
92 ('new_dir/new', 'new_dir/new', 'foo'), |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
93 ] |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
94 self.commitchanges(changes) |
730
efb87d5bb311
subvertpy: fix bug with pushing changes to stuff in subdirs
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents:
700
diff
changeset
|
95 self.pushrevisions() |
869
db3a651494f9
test_util: pass repo_path to svnls(), turn it into a function
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
96 self.assertEqual(test_util.svnls(repo_path, 'tags'), |
731
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
97 ['copied_tag', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
98 'copied_tag/alpha', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
99 'copied_tag/beta', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
100 'new_dir', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
101 'new_dir/new', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
102 'tag_r3', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
103 'tag_r3/alpha', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
104 'tag_r3/beta', |
de036c2cb36a
test_push_dirs: use common commit-building infrastructure
Augie Fackler <durin42@gmail.com>
parents:
730
diff
changeset
|
105 'tag_r3/new']) |
268
6ec5b5fc5b4d
Added a test case for directory-add behavior when repo root is not project root.
Augie Fackler <durin42@gmail.com>
parents:
85
diff
changeset
|
106 |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
107 def suite(): |
833
312b37bc5e20
tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents:
832
diff
changeset
|
108 all_tests = [unittest.TestLoader().loadTestsFromTestCase(TestPushDirectories), |
83
6c9b7cf1c5aa
push_cmd: delete empty svn directories, refactor directory creation
Patrick Mezard <pmezard@gmail.com>
parents:
diff
changeset
|
109 ] |
833
312b37bc5e20
tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents:
832
diff
changeset
|
110 return unittest.TestSuite(all_tests) |