hgsubversion: tests/comprehensive/test_stupid

annotate tests/comprehensive/test_stupid_pull.py @ 931:e1dbd9646d6a

svnwrap: use custom StringIO class in get_file() The wrappers were calling ra.get_file() with a cStringIO object. Empirically, svn 1.7.5 is writing 16kB blocks to the stream object, and cStringIO reallocates its internal buffer and doubles its size whenever it is filled. With large committed files this requires two large memory blocks at the same time. SimpleStringIO implements the mimimum StringIO interface used by ra.get_file() but instead stores all the blocks and "join" them at the end. It means more fragmentation but requires only one large block, without overallocation. Also, 16kB blocks should be friendly to most allocators. In practice, this simple change let me convert a revision containing multiple moderately large files, the largest being around 450MB, with a 32-bits Windows setup, python 2.7, swig svn 1.7.5, in stupid mode, while it was previously aborting with "not enough memory". The same revision still fails in replay mode.

author	Patrick Mezard <patrick@mezard.eu>
date	Sun, 16 Sep 2012 19:31:49 +0200
parents	7d9cd708f412
children	d741f536f23a

rev	line source
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	1 import os
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	2 import pickle
766 124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 613 diff changeset	3 import sys
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	4 import unittest
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	5
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	6 from mercurial import hg
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	7 from mercurial import ui
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	8
613 58f397523604 testrunner: delay and `simplify' importing of tests Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	9 # wrapped in a try/except because of weirdness in how
58f397523604 testrunner: delay and `simplify' importing of tests Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	10 # run.py works as compared to nose.
58f397523604 testrunner: delay and `simplify' importing of tests Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	11 try:
58f397523604 testrunner: delay and `simplify' importing of tests Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	12 import test_util
58f397523604 testrunner: delay and `simplify' importing of tests Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	13 except ImportError:
766 124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 613 diff changeset	14 sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
124cd25de4ef tests: fix running the comprehensive tests using nose. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 613 diff changeset	15 import test_util
613 58f397523604 testrunner: delay and `simplify' importing of tests Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 499 diff changeset	16
354 36d26e158748 comprehensive: fix for new repo layout. Augie Fackler <durin42@gmail.com> parents: 347 diff changeset	17 from hgsubversion import wrappers
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	18
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	19
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	20 def _do_case(self, name, layout):
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	21 subdir = test_util.subdir.get(name, '')
867 50c13e01c7e3 test_util: add a load_and_fetch() returning the repo_path Patrick Mezard <patrick@mezard.eu> parents: 833 diff changeset	22 repo, repo_path = self.load_and_fetch(name, subdir=subdir, stupid=False,
50c13e01c7e3 test_util: add a load_and_fetch() returning the repo_path Patrick Mezard <patrick@mezard.eu> parents: 833 diff changeset	23 layout=layout)
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	24 assert len(self.repo) > 0, 'Repo had no changes, maybe you need to add a subdir entry in test_util?'
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	25 wc2_path = self.wc_path + '_stupid'
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	26 u = ui.ui()
867 50c13e01c7e3 test_util: add a load_and_fetch() returning the repo_path Patrick Mezard <patrick@mezard.eu> parents: 833 diff changeset	27 checkout_path = repo_path
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	28 if subdir:
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	29 checkout_path += '/' + subdir
347 537de0300510 Remove the 'outgoing' wrapper, and use the Mercurial infrastructure instead. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 257 diff changeset	30 u.setconfig('hgsubversion', 'stupid', '1')
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	31 u.setconfig('hgsubversion', 'layout', layout)
816 86d124a8768e Fix hg.clone() calls changed by d976542986d2 Patrick Mezard <pmezard@gmail.com> parents: 766 diff changeset	32 test_util.hgclone(u, test_util.fileurl(checkout_path), wc2_path, update=False)
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	33 if layout == 'single':
1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	34 self.assertEqual(len(self.repo.heads()), 1)
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	35 self.repo2 = hg.repository(ui.ui(), wc2_path)
445 24a8471069c0 tests: test for heads equality instead of branchtags Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 383 diff changeset	36 self.assertEqual(self.repo.heads(), self.repo2.heads())
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	37
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	38
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	39 def buildmethod(case, name, layout):
1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	40 m = lambda self: self._do_case(case, layout)
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	41 m.__name__ = name
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	42 m.__doc__ = 'Test stupid produces same as real on %s. (%s)' % (case, layout)
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	43 return m
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	44
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	45 attrs = {'_do_case': _do_case,
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	46 }
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	47 for case in (f for f in os.listdir(test_util.FIXTURES) if f.endswith('.svndump')):
892 3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 867 diff changeset	48 if case == 'corrupt.svndump':
3bfb7e985c47 svn verify: add a test for corrupt repositories. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 867 diff changeset	49 continue
902 7d9cd708f412 tests: fix automatically generated test names Bryan O'Sullivan <bryano@fb.com> parents: 892 diff changeset	50 name = 'test_' + case[:-len('.svndump')].replace('-', '_')
821 f28e0f54a6ef svnmeta: store youngest revision pulled from subversion Augie Fackler <durin42@gmail.com> parents: 816 diff changeset	51 # Automatic layout branchtag collision exposes a minor defect
f28e0f54a6ef svnmeta: store youngest revision pulled from subversion Augie Fackler <durin42@gmail.com> parents: 816 diff changeset	52 # here, but since it isn't a regression we suppress the test case.
f28e0f54a6ef svnmeta: store youngest revision pulled from subversion Augie Fackler <durin42@gmail.com> parents: 816 diff changeset	53 if case != 'branchtagcollision.svndump':
f28e0f54a6ef svnmeta: store youngest revision pulled from subversion Augie Fackler <durin42@gmail.com> parents: 816 diff changeset	54 attrs[name] = buildmethod(case, name, 'auto')
499 1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	55 name += '_single'
1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	56 attrs[name] = buildmethod(case, name, 'single')
1fd3cfa47c5e Support for single-directory clones. Augie Fackler <durin42@gmail.com> parents: 445 diff changeset	57
832 e9af7eba88db globally: clean up whitespace around operators and commas to conform with PEP8 Yonggang Luo <luoyonggang@gmail.com> parents: 821 diff changeset	58 StupidPullTests = type('StupidPullTests', (test_util.TestBase,), attrs)
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	59
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	60
13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	61 def suite():
833 312b37bc5e20 tests: avoid shadowing Python builtin all() Yonggang Luo <luoyonggang@gmail.com> parents: 832 diff changeset	62 all_tests = [unittest.TestLoader().loadTestsFromTestCase(StupidPullTests),
194 13ae1bded5e7 Add some comprehensive tests that can be run with nose in order to make it easier to verify stupid and real replay do the same thing. Augie Fackler <durin42@gmail.com> parents: diff changeset	63 ]
833 312b37bc5e20 tests: avoid shadowing Python builtin all() Yonggang Luo <luoyonggang@gmail.com> parents: 832 diff changeset	64 return unittest.TestSuite(all_tests)

Mercurial > hgsubversion

annotate tests/comprehensive/test_stupid_pull.py @ 931:e1dbd9646d6a