annotate tests/test_svnwrap.py @ 1457:019c3e194fba

tests: optimise creating repositories and loading dumps Previously, we'd use svnadmin for creating repositories and loading dumps. That tends to be a bit slow, as it forks a new process and loads the Subversion libraries into it. Instead, we extend our existing Subversion wrappers and load the dumps using the API. This is a noticable speedup. The only downside is that we rely on Subversion and Subvertpy to correctly close all file descriptors; an assumption which hasn't always held in the past. I ran some benchmarks on my relatively slow Mac with $TMPDIR on a ramdisk, and they showed a significant change: I compared ten runs of each with Subvertpy: min: -18.8% (299.1s -> 243.0s) median: -20.0% (307.1s -> 245.6s) ...and three runs of each with SWIG: min: -22.8% (368.7s -> 284.7s) median: -25.7% (384.4s -> 285.5s) (Since the timing measures wall clock time, the minimum time is likely to be the most accurate and useful measurement.)
author Dan Villiom Podlaski Christiansen <danchr@gmail.com>
date Tue, 07 Jun 2016 09:15:53 +0200
parents d741f536f23a
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
643
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 642
diff changeset
1 import test_util
d2ef7220a079 tests: import test_util as the first module in all relevant tests
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 642
diff changeset
2
347
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
3 import os
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
4 import subprocess
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
5 import tempfile
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
6 import unittest
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
7
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
8 from hgsubversion import svnwrap
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
9
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
10 class TestBasicRepoLayout(unittest.TestCase):
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
11 def setUp(self):
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
12 self.tmpdir = tempfile.mkdtemp('svnwrap_test')
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
13 self.repo_path = '%s/testrepo' % self.tmpdir
1457
019c3e194fba tests: optimise creating repositories and loading dumps
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 1044
diff changeset
14
019c3e194fba tests: optimise creating repositories and loading dumps
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 1044
diff changeset
15 with open(os.path.join(test_util.FIXTURES,
019c3e194fba tests: optimise creating repositories and loading dumps
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 1044
diff changeset
16 'project_root_at_repo_root.svndump')) as fp:
019c3e194fba tests: optimise creating repositories and loading dumps
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 1044
diff changeset
17 svnwrap.create_and_load(self.repo_path, fp)
019c3e194fba tests: optimise creating repositories and loading dumps
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents: 1044
diff changeset
18
480
7fa100ae1a11 Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents: 479
diff changeset
19 self.repo = svnwrap.SubversionRepo(test_util.fileurl(self.repo_path))
347
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
20
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
21 def tearDown(self):
482
a42fb4f1716a Reclaim repository object to avoid tearDown() failing to remove the open file 'testrepo/db/rep-cache.db'
Risto Kankkunen <risto.kankkunen@iki.fi>
parents: 481
diff changeset
22 del self.repo
873
c58213aaf7c8 test_svnwrap: use test_util.rmtree() (or fix it)
Patrick Mezard <patrick@mezard.eu>
parents: 833
diff changeset
23 test_util.rmtree(self.tmpdir)
347
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
24
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
25 def test_num_revs(self):
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
26 revs = list(self.repo.revisions())
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
27 self.assertEqual(len(revs), 7)
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
28 r = revs[1]
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
29 self.assertEqual(r.revnum, 2)
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
30 self.assertEqual(sorted(r.paths.keys()),
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
31 ['trunk/alpha', 'trunk/beta', 'trunk/delta'])
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
32 for r in revs:
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
33 for p in r.paths:
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
34 # make sure these paths are always non-absolute for sanity
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
35 if p:
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
36 assert p[0] != '/'
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
37 revs = list(self.repo.revisions(start=3))
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
38 self.assertEqual(len(revs), 4)
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
39
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
40 class TestRootAsSubdirOfRepo(TestBasicRepoLayout):
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
41 def setUp(self):
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
42 self.tmpdir = tempfile.mkdtemp('svnwrap_test')
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
43 self.repo_path = '%s/testrepo' % self.tmpdir
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 666
diff changeset
44 subprocess.call(['svnadmin', 'create', self.repo_path, ])
347
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
45 inp = open(os.path.join(os.path.dirname(__file__), 'fixtures',
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
46 'project_root_not_repo_root.svndump'))
832
e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8
Yonggang Luo <luoyonggang@gmail.com>
parents: 666
diff changeset
47 ret = subprocess.call(['svnadmin', 'load', self.repo_path, ],
479
83fcb1cf6d8f Avoid 'ValueError: close_fds is not supported on Windows platforms' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents: 478
diff changeset
48 stdin=inp,
83fcb1cf6d8f Avoid 'ValueError: close_fds is not supported on Windows platforms' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents: 478
diff changeset
49 close_fds=test_util.canCloseFds,
347
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
50 stdout=subprocess.PIPE,
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
51 stderr=subprocess.STDOUT)
537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead.
Dan Villiom Podlaski Christiansen <danchr@gmail.com>
parents:
diff changeset
52 assert ret == 0
480
7fa100ae1a11 Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents: 479
diff changeset
53 self.repo = svnwrap.SubversionRepo(test_util.fileurl(
7fa100ae1a11 Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents: 479
diff changeset
54 self.repo_path + '/dummyproj'
7fa100ae1a11 Avoid 'Abort: Illegal repository URL' exception
Risto Kankkunen <risto.kankkunen@iki.fi>
parents: 479
diff changeset
55 ))