Mercurial > hgsubversion
view README @ 931:e1dbd9646d6a
svnwrap: use custom StringIO class in get_file()
The wrappers were calling ra.get_file() with a cStringIO object.
Empirically, svn 1.7.5 is writing 16kB blocks to the stream object, and
cStringIO reallocates its internal buffer and doubles its size whenever
it is filled. With large committed files this requires two large
memory blocks at the same time.
SimpleStringIO implements the mimimum StringIO interface used by
ra.get_file() but instead stores all the blocks and "join" them at the
end. It means more fragmentation but requires only one large block,
without overallocation. Also, 16kB blocks should be friendly to most
allocators.
In practice, this simple change let me convert a revision containing
multiple moderately large files, the largest being around 450MB, with a
32-bits Windows setup, python 2.7, swig svn 1.7.5, in stupid mode, while
it was previously aborting with "not enough memory". The same revision
still fails in replay mode.
| author | Patrick Mezard <patrick@mezard.eu> |
|---|---|
| date | Sun, 16 Sep 2012 19:31:49 +0200 |
| parents | 050f03a3bdf5 |
| children | 3df6ed4e7561 |
line wrap: on
line source
.. -*-restructuredtext-*- ============ hgsubversion ============ hgsubversion is an extension for Mercurial that allows using Mercurial as a Subversion client. At this point, hgsubversion is usable by users reasonably familiar with Mercurial as a VCS. It's not recommended to dive into hgsubversion as an introduction to Mercurial, since hgsubversion "bends the rules" a little and violates some of the typical assumptions of early Mercurial users. Installation ------------ You need to have either have Subversion 1.5 (or later) installed along with either Subvertpy 0.7.4 (or later) or the Subversion SWIG Python bindings. You need Mercurial 1.3 or later. .. _mercurial: http://selenic.com/repo/hg .. _mercurial-stable: http://selenic.com/repo/hg-stable .. _crew: http://hg.intevation.org/mercurial/crew .. _crew-stable: http://hg.intevation.org/mercurial/crew-stable If you are unfamiliar with installing Mercurial extensions, please see the UsingExtensions_ page in the Mercurial wiki. Look at the example for specifying an absolute path near the bottom of the page. You want to give the path to the top level of your clone of this repository. .. _UsingExtensions: http://mercurial.selenic.com/wiki/UsingExtensions Before using hgsubversion, I *strongly* encourage you to run the automated tests. Just use nose_ if you have it (or ``easy_install nose`` if you want it), or use ``python tests/run.py`` to run the suite with the conventional test runner. Note that because I use nose, there's a lot of stdout spew in the tests right now. The important part is that all the tests pass. .. _nose: http://code.google.com/p/python-nose/ You can check that hgsubversion is installed and properly activated using the following command:: $ hg version --svn Mercurial Distributed SCM (version ...) Copyright (C) 2005-2010 Matt Mackall <mpm@selenic.com> and others This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. hgsubversion: ... Subversion: ... bindings: Subvertpy ... If your bindings are listed as `SWIG`, please consider installing Subvertpy_. .. _Subvertpy: http://pypi.python.org/pypi/subvertpy Further Reading --------------- More information on how to use hgsubversion is available from within Mercurial in the `subversion` help topic. To view it, use:: $ hg help subversion The Restructured Text source for this topic is also available in the file ``hgsubverson/help/subversion.rst``.