Mercurial > hgsubversion
annotate tests/test_svnwrap.py @ 890:78db88de9622
Partial metadata rebuilding
For highly active subversion repositories, it can be excruciatingly
slow to pull updates one at a time from subversion. One way around
this is to setup another mercurial repo that pulls new commits from
svn periodicly (say every 5 minutes). When you want to update your
repository, you can pull commits from this mercurial repository via
native mercurial protocols, which will be much faster than pulling
directly from svn.
Unfortunately, your metadata will be out of date after doing so.
Highly active repositories also tend to be very large, which means
that it takes a long time to rebuild your metadata from scratch. To
address this, this adds support to do a partial rebuild on the
metadata by processing only revisions that have been added to the
repository after the last revision we processed.
With the rev map 1k revisions (~2 days) behind tip updatemeta is
dramatically faster than rebuild meta:
$ hg --time svn updatemeta
Time: real 0.570 secs (user 0.480+0.000 sys 0.060+0.000)
$ hg --time svn rebuildmeta
Time: real 129.160 secs (user 128.570+0.000 sys 0.320+0.000)
author | David Schleimer <dschleimer@fb.com> |
---|---|
date | Sat, 12 May 2012 07:28:23 -0700 |
parents | f2de043ac924 |
children | d741f536f23a |
rev | line source |
---|---|
643
d2ef7220a079
tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
642
diff
changeset
|
1 import test_util |
d2ef7220a079
tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
642
diff
changeset
|
2 |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
3 import os |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
4 import subprocess |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
5 import tempfile |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
6 import unittest |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
7 |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
8 from hgsubversion import svnwrap |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
9 |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
10 class TestBasicRepoLayout(unittest.TestCase): |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
11 def setUp(self): |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
12 self.tmpdir = tempfile.mkdtemp('svnwrap_test') |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
13 self.repo_path = '%s/testrepo' % self.tmpdir |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
666
diff
changeset
|
14 subprocess.call(['svnadmin', 'create', self.repo_path, ]) |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
15 inp = open(os.path.join(os.path.dirname(__file__), 'fixtures', |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
16 'project_root_at_repo_root.svndump')) |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
666
diff
changeset
|
17 proc = subprocess.call(['svnadmin', 'load', self.repo_path, ], |
479
83fcb1cf6d8f
Avoid 'ValueError: close_fds is not supported on Windows platforms' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
478
diff
changeset
|
18 stdin=inp, |
83fcb1cf6d8f
Avoid 'ValueError: close_fds is not supported on Windows platforms' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
478
diff
changeset
|
19 close_fds=test_util.canCloseFds, |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
20 stdout=subprocess.PIPE, |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
21 stderr=subprocess.STDOUT) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
22 assert proc == 0 |
480
7fa100ae1a11
Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
479
diff
changeset
|
23 self.repo = svnwrap.SubversionRepo(test_util.fileurl(self.repo_path)) |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
24 |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
25 def tearDown(self): |
482
a42fb4f1716a
Reclaim repository object to avoid tearDown() failing to remove the open file 'testrepo/db/rep-cache.db'
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
481
diff
changeset
|
26 del self.repo |
873
c58213aaf7c8
test_svnwrap: use test_util.rmtree() (or fix it)
Patrick Mezard <patrick@mezard.eu>
parents:
833
diff
changeset
|
27 test_util.rmtree(self.tmpdir) |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
28 |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
29 def test_num_revs(self): |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
30 revs = list(self.repo.revisions()) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
31 self.assertEqual(len(revs), 7) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
32 r = revs[1] |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
33 self.assertEqual(r.revnum, 2) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
34 self.assertEqual(sorted(r.paths.keys()), |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
35 ['trunk/alpha', 'trunk/beta', 'trunk/delta']) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
36 for r in revs: |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
37 for p in r.paths: |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
38 # make sure these paths are always non-absolute for sanity |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
39 if p: |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
40 assert p[0] != '/' |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
41 revs = list(self.repo.revisions(start=3)) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
42 self.assertEqual(len(revs), 4) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
43 |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
44 class TestRootAsSubdirOfRepo(TestBasicRepoLayout): |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
45 def setUp(self): |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
46 self.tmpdir = tempfile.mkdtemp('svnwrap_test') |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
47 self.repo_path = '%s/testrepo' % self.tmpdir |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
666
diff
changeset
|
48 subprocess.call(['svnadmin', 'create', self.repo_path, ]) |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
49 inp = open(os.path.join(os.path.dirname(__file__), 'fixtures', |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
50 'project_root_not_repo_root.svndump')) |
832
e9af7eba88db
globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents:
666
diff
changeset
|
51 ret = subprocess.call(['svnadmin', 'load', self.repo_path, ], |
479
83fcb1cf6d8f
Avoid 'ValueError: close_fds is not supported on Windows platforms' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
478
diff
changeset
|
52 stdin=inp, |
83fcb1cf6d8f
Avoid 'ValueError: close_fds is not supported on Windows platforms' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
478
diff
changeset
|
53 close_fds=test_util.canCloseFds, |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
54 stdout=subprocess.PIPE, |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
55 stderr=subprocess.STDOUT) |
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
56 assert ret == 0 |
480
7fa100ae1a11
Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
479
diff
changeset
|
57 self.repo = svnwrap.SubversionRepo(test_util.fileurl( |
7fa100ae1a11
Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
479
diff
changeset
|
58 self.repo_path + '/dummyproj' |
7fa100ae1a11
Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
479
diff
changeset
|
59 )) |
7fa100ae1a11
Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents:
479
diff
changeset
|
60 |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
61 def suite(): |
833
312b37bc5e20
tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents:
832
diff
changeset
|
62 all_tests = [unittest.TestLoader().loadTestsFromTestCase(TestBasicRepoLayout), |
347
537de0300510
Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff
changeset
|
63 unittest.TestLoader().loadTestsFromTestCase(TestRootAsSubdirOfRepo)] |
833
312b37bc5e20
tests: avoid shadowing Python builtin all()
Yonggang Luo <luoyonggang@gmail.com>
parents:
832
diff
changeset
|
64 return unittest.TestSuite(all_tests) |