svnwrap: use custom StringIO class in get_file() The wrappers were calling ra.get_file() with a cStringIO object. Empirically, svn 1.7.5 is writing 16kB blocks to the stream object, and cStringIO reallocates its internal buffer and doubles its size whenever it is filled. With large committed files this requires two large memory blocks at the same time. SimpleStringIO implements the mimimum StringIO interface used by ra.get_file() but instead stores all the blocks and "join" them at the end. It means more fragmentation but requires only one large block, without overallocation. Also, 16kB blocks should be friendly to most allocators. In practice, this simple change let me convert a revision containing multiple moderately large files, the largest being around 450MB, with a 32-bits Windows setup, python 2.7, swig svn 1.7.5, in stupid mode, while it was previously aborting with "not enough memory". The same revision still fails in replay mode.

#!/bin/sh

mkdir temp
cd temp

svnadmin create repo
svn co file://`pwd`/repo wc
cd wc

mkdir -p trunk branches
svn add trunk branches
svn ci -m'initial structure'
cd trunk
echo a>a
svn add a
svn ci -mci1 a
cd ..
svn up
svn cp trunk branches/foo
svn ci -m'branch foo'
cd branches/foo
ln -s a fnord
svn add fnord
svn ci -msymlink fnord
mkdir 'spacy name'
echo a > 'spacy name/spacy file'
svn add 'spacy name'
svn ci -mspacy 'spacy name'
svn up
echo b > 'spacy name/surprise ~'
svn add 'spacy name/surprise ~'
svn ci -mtilde 'spacy name'
svn up ../..
echo foo > exe
chmod +x exe
svn add exe
svn ci -mexecutable exe
svn up ../..
cd ../../trunk
svn merge ../branches/foo
svn ci -mmerge
svn up

pwd
cd ../../..
svnadmin dump temp/repo > addspecial.svndump
echo
echo 'Complete.'
echo 'You probably want to clean up temp now.'
echo 'Dump in addspecial.svndump'
exit 0