Mercurial > hgsubversion
annotate tests/fixtures/addspecial.sh @ 931:e1dbd9646d6a
svnwrap: use custom StringIO class in get_file()
The wrappers were calling ra.get_file() with a cStringIO object.
Empirically, svn 1.7.5 is writing 16kB blocks to the stream object, and
cStringIO reallocates its internal buffer and doubles its size whenever
it is filled. With large committed files this requires two large
memory blocks at the same time.
SimpleStringIO implements the mimimum StringIO interface used by
ra.get_file() but instead stores all the blocks and "join" them at the
end. It means more fragmentation but requires only one large block,
without overallocation. Also, 16kB blocks should be friendly to most
allocators.
In practice, this simple change let me convert a revision containing
multiple moderately large files, the largest being around 450MB, with a
32-bits Windows setup, python 2.7, swig svn 1.7.5, in stupid mode, while
it was previously aborting with "not enough memory". The same revision
still fails in replay mode.
author | Patrick Mezard <patrick@mezard.eu> |
---|---|
date | Sun, 16 Sep 2012 19:31:49 +0200 |
parents | b6b1365e3489 |
children |
rev | line source |
---|---|
901
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
1 #!/bin/sh |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
2 |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
3 mkdir temp |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
4 cd temp |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
5 |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
6 svnadmin create repo |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
7 svn co file://`pwd`/repo wc |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
8 cd wc |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
9 |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
10 mkdir -p trunk branches |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
11 svn add trunk branches |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
12 svn ci -m'initial structure' |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
13 cd trunk |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
14 echo a>a |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
15 svn add a |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
16 svn ci -mci1 a |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
17 cd .. |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
18 svn up |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
19 svn cp trunk branches/foo |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
20 svn ci -m'branch foo' |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
21 cd branches/foo |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
22 ln -s a fnord |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
23 svn add fnord |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
24 svn ci -msymlink fnord |
904
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
25 mkdir 'spacy name' |
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
26 echo a > 'spacy name/spacy file' |
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
27 svn add 'spacy name' |
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
28 svn ci -mspacy 'spacy name' |
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
29 svn up |
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
30 echo b > 'spacy name/surprise ~' |
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
31 svn add 'spacy name/surprise ~' |
b6b1365e3489
canonicalize svn paths even more awesomely
Bryan O'Sullivan <bryano@fb.com>
parents:
901
diff
changeset
|
32 svn ci -mtilde 'spacy name' |
901
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
33 svn up ../.. |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
34 echo foo > exe |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
35 chmod +x exe |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
36 svn add exe |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
37 svn ci -mexecutable exe |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
38 svn up ../.. |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
39 cd ../../trunk |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
40 svn merge ../branches/foo |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
41 svn ci -mmerge |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
42 svn up |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
43 |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
44 pwd |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
45 cd ../../.. |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
46 svnadmin dump temp/repo > addspecial.svndump |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
47 echo |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
48 echo 'Complete.' |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
49 echo 'You probably want to clean up temp now.' |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
50 echo 'Dump in addspecial.svndump' |
bd12a4da0f35
replay: workaround svn not telling us about x/l flags (issue346)
Bryan O'Sullivan <bryano@fb.com>
parents:
diff
changeset
|
51 exit 0 |