hgsubversion: fetch_command.py annotate

annotate fetch_command.py @ 239:e2214c8fc91f

Put all stupid stuff in it's own module (separate from fetch-command).

author	Dirkjan Ochtman <dirkjan@ochtman.nl>
date	Wed, 08 Apr 2009 17:34:01 +0200
parents	c90cfa665b81
children

rev	line source
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	1 import os
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	2
6 1a5bb173170b Fixes for win32 compatibility. Changes suggested by Shun-ichi GOTO, with some alterations by me. Augie Fackler <durin42@gmail.com> parents: 0 diff changeset	3 from mercurial import util as merc_util
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	4 from svn import core
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	5 from svn import delta
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	6
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	7 import hg_delta_editor
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	8 import svnwrap
239 e2214c8fc91f Put all stupid stuff in it's own module (separate from fetch-command). Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 237 diff changeset	9 import stupid as stupidmod
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	10 import util
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	11
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	12
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	13 def fetch_revisions(ui, svn_url, hg_repo_path, skipto_rev=0, stupid=None,
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	14 tag_locations='tags',
167 3cd6a7354207 fetch: Add support for an authormap which can rename authors, intended for Graham Booker <gbooker@cod3r.com> parents: 166 diff changeset	15 authors=None,
179 a336e3e82648 Fetch: add a filemap argument for use in converting old repositories to Graham Booker <gbooker@cod3r.com> parents: 176 diff changeset	16 filemap=None,
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	17 **opts):
185 57355b0e7bd1 Creating patch for documention messages. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 184 diff changeset	18 """pull new revisions from Subversion
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	19 """
140 9ffde8662967 util: Add a command to normalize svn urls and use it in a couple of places. Test that it works and prevents failed assertions. Augie Fackler <durin42@gmail.com> parents: 133 diff changeset	20 svn_url = util.normalize_url(svn_url)
223 330f0b15d417 issue67: mercurial 1.3 util incompatibility with encoding swap Luke Opperman <luke@loppear.com> parents: 218 diff changeset	21 old_encoding = util.swap_out_encoding()
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	22 skipto_rev=int(skipto_rev)
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	23 have_replay = not stupid
133 2242dd1163c6 hg_delta_editor: fix bad parent revision calculation in the case of a branch Augie Fackler <durin42@gmail.com> parents: 129 diff changeset	24 if have_replay and not callable(
2242dd1163c6 hg_delta_editor: fix bad parent revision calculation in the case of a branch Augie Fackler <durin42@gmail.com> parents: 129 diff changeset	25 delta.svn_txdelta_apply(None, None, None)[0]): #pragma: no cover
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	26 ui.status('You are using old Subversion SWIG bindings. Replay will not'
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	27 ' work until you upgrade to 1.5.0 or newer. Falling back to'
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	28 ' a slower method that may be buggier. Please upgrade, or'
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	29 ' contribute a patch to use the ctypes bindings instead'
29 575bd29bc1d8 Fix a missing newline. Augie Fackler <durin42@gmail.com> parents: 25 diff changeset	30 ' of SWIG.\n')
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	31 have_replay = False
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	32 initializing_repo = False
234 33e885f5f86a Add --username and --password options to all commands Daniel Tang <dytang@cs.purdue.edu> parents: 229 diff changeset	33 user = opts.get('username', merc_util.getuser())
33e885f5f86a Add --username and --password options to all commands Daniel Tang <dytang@cs.purdue.edu> parents: 229 diff changeset	34 passwd = opts.get('password', '')
33e885f5f86a Add --username and --password options to all commands Daniel Tang <dytang@cs.purdue.edu> parents: 229 diff changeset	35 svn = svnwrap.SubversionRepo(svn_url, user, passwd)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	36 author_host = "@%s" % svn.uuid
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	37 tag_locations = tag_locations.split(',')
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	38 hg_editor = hg_delta_editor.HgChangeReceiver(hg_repo_path,
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	39 ui_=ui,
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	40 subdir=svn.subdir,
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	41 author_host=author_host,
167 3cd6a7354207 fetch: Add support for an authormap which can rename authors, intended for Graham Booker <gbooker@cod3r.com> parents: 166 diff changeset	42 tag_locations=tag_locations,
179 a336e3e82648 Fetch: add a filemap argument for use in converting old repositories to Graham Booker <gbooker@cod3r.com> parents: 176 diff changeset	43 authors=authors,
a336e3e82648 Fetch: add a filemap argument for use in converting old repositories to Graham Booker <gbooker@cod3r.com> parents: 176 diff changeset	44 filemap=filemap)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	45 if os.path.exists(hg_editor.uuid_file):
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	46 uuid = open(hg_editor.uuid_file).read()
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	47 assert uuid == svn.uuid
225 2117cb0118fe Get rid of .hg/svn/last_rev: Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 224 diff changeset	48 start = hg_editor.last_known_revision()
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	49 else:
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	50 open(hg_editor.uuid_file, 'w').write(svn.uuid)
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	51 open(hg_editor.svn_url_file, 'w').write(svn_url)
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	52 initializing_repo = True
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	53 start = skipto_rev
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	54
197 43d56e973c3c Replace a few asserts with aborts. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 188 diff changeset	55 if initializing_repo and start > 0:
43d56e973c3c Replace a few asserts with aborts. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 188 diff changeset	56 raise merc_util.Abort('Revision skipping at repository initialization '
43d56e973c3c Replace a few asserts with aborts. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 188 diff changeset	57 'remains unimplemented.')
43d56e973c3c Replace a few asserts with aborts. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 188 diff changeset	58
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	59 # start converting revisions
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	60 for r in svn.revisions(start=start):
203 907c160c6289 Refactor branch handling to be much more dynamic (and hopefully robust). Augie Fackler <durin42@gmail.com> parents: 200 diff changeset	61 valid = True
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	62 hg_editor.update_branch_tag_map_for_rev(r)
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	63 for p in r.paths:
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	64 if hg_editor._is_path_valid(p):
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	65 valid = True
166 db88e528e8e6 Fixed typo: "continue" should have been "break". Martin Geisler <mg@daimi.au.dk> parents: 163 diff changeset	66 break
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	67 if valid:
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	68 # got a 502? Try more than once!
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	69 tries = 0
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	70 converted = False
229 7f20914e52e8 Better 502 logic that does not catch non-502 errors. Augie Fackler <durin42@gmail.com> parents: 228 diff changeset	71 while not converted:
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	72 try:
198 df4611050286 Output consolidation; decrease the ‘Fetching...’ message to debug level. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 197 diff changeset	73 util.describe_revision(ui, r)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	74 if have_replay:
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	75 try:
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	76 replay_convert_rev(hg_editor, svn, r)
63 2e30b59a9c19 Added some coverage pragmas to stop it from trying to cover things we can't test. Augie Fackler <durin42@gmail.com> parents: 59 diff changeset	77 except svnwrap.SubversionRepoCanNotReplay, e: #pragma: no cover
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	78 ui.status('%s\n' % e.message)
239 e2214c8fc91f Put all stupid stuff in it's own module (separate from fetch-command). Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 237 diff changeset	79 stupidmod.print_your_svn_is_old_message(ui)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	80 have_replay = False
239 e2214c8fc91f Put all stupid stuff in it's own module (separate from fetch-command). Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 237 diff changeset	81 stupidmod.svn_server_pull_rev(ui, svn, hg_editor, r)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	82 else:
239 e2214c8fc91f Put all stupid stuff in it's own module (separate from fetch-command). Dirkjan Ochtman <dirkjan@ochtman.nl> parents: 237 diff changeset	83 stupidmod.svn_server_pull_rev(ui, svn, hg_editor, r)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	84 converted = True
63 2e30b59a9c19 Added some coverage pragmas to stop it from trying to cover things we can't test. Augie Fackler <durin42@gmail.com> parents: 59 diff changeset	85 except core.SubversionException, e: #pragma: no cover
229 7f20914e52e8 Better 502 logic that does not catch non-502 errors. Augie Fackler <durin42@gmail.com> parents: 228 diff changeset	86 if (e.apr_err == core.SVN_ERR_RA_DAV_REQUEST_FAILED
7f20914e52e8 Better 502 logic that does not catch non-502 errors. Augie Fackler <durin42@gmail.com> parents: 228 diff changeset	87 and '502' in str(e)
7f20914e52e8 Better 502 logic that does not catch non-502 errors. Augie Fackler <durin42@gmail.com> parents: 228 diff changeset	88 and tries < 3):
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	89 tries += 1
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	90 ui.status('Got a 502, retrying (%s)\n' % tries)
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	91 else:
224 2165461d2dd8 Exception clean-ups; use symbolic names & avoid Python 2.6 deprecations. Dan Villiom Podlaski Christiansen <danchr@gmail.com> parents: 203 diff changeset	92 raise merc_util.Abort(*e.args)
223 330f0b15d417 issue67: mercurial 1.3 util incompatibility with encoding swap Luke Opperman <luke@loppear.com> parents: 218 diff changeset	93 util.swap_out_encoding(old_encoding)
330f0b15d417 issue67: mercurial 1.3 util incompatibility with encoding swap Luke Opperman <luke@loppear.com> parents: 218 diff changeset	94
181 e37f9d3fd5e7 remove decorators (compat with python2.3) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 179 diff changeset	95 fetch_revisions = util.register_subcommand('pull')(fetch_revisions)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	96
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	97
38 9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	98 def cleanup_file_handles(svn, count):
9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	99 if count % 50 == 0:
9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	100 svn.init_ra_and_client()
9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	101
218 a360ddc97719 branches: change handling again, but this time a little less magic. Augie Fackler <durin42@gmail.com> parents: 203 diff changeset	102
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	103 def replay_convert_rev(hg_editor, svn, r):
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	104 hg_editor.set_current_rev(r)
f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	105 svn.get_replay(r.revnum, hg_editor)
38 9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	106 i = 1
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	107 if hg_editor.missing_plaintexts:
198 df4611050286 Output consolidation; decrease the ‘Fetching...’ message to debug level. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 197 diff changeset	108 hg_editor.ui.debug('Fetching %s files that could not use replay.\n' %
df4611050286 Output consolidation; decrease the ‘Fetching...’ message to debug level. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 197 diff changeset	109 len(hg_editor.missing_plaintexts))
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	110 files_to_grab = set()
98 c7ac013cf7fd fetch_command: simplify replay_convert_rev() with svn.list_files() Patrick Mezard <pmezard@gmail.com> parents: 97 diff changeset	111 rootpath = svn.subdir and svn.subdir[1:] or ''
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	112 for p in hg_editor.missing_plaintexts:
186 6266ba36ee15 Create patch to make normal output much less verbose… Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 185 diff changeset	113 hg_editor.ui.note('.')
38 9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	114 hg_editor.ui.flush()
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	115 if p[-1] == '/':
98 c7ac013cf7fd fetch_command: simplify replay_convert_rev() with svn.list_files() Patrick Mezard <pmezard@gmail.com> parents: 97 diff changeset	116 dirpath = p[len(rootpath):]
182 47d25d61abfa remove generators (compat with python2.3) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 181 diff changeset	117 files_to_grab.update([dirpath + f for f,k in
133 2242dd1163c6 hg_delta_editor: fix bad parent revision calculation in the case of a branch Augie Fackler <durin42@gmail.com> parents: 129 diff changeset	118 svn.list_files(dirpath, r.revnum)
182 47d25d61abfa remove generators (compat with python2.3) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 181 diff changeset	119 if k == 'f'])
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	120 else:
98 c7ac013cf7fd fetch_command: simplify replay_convert_rev() with svn.list_files() Patrick Mezard <pmezard@gmail.com> parents: 97 diff changeset	121 files_to_grab.add(p[len(rootpath):])
186 6266ba36ee15 Create patch to make normal output much less verbose… Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 185 diff changeset	122 hg_editor.ui.note('\nFetching files...\n')
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	123 for p in files_to_grab:
186 6266ba36ee15 Create patch to make normal output much less verbose… Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 185 diff changeset	124 hg_editor.ui.note('.')
38 9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	125 hg_editor.ui.flush()
9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	126 cleanup_file_handles(svn, i)
9ee7ce0505eb Fixes so that I can clone the melange repository successfully. Fixes a bug that Augie Fackler <durin42@gmail.com> parents: 33 diff changeset	127 i += 1
98 c7ac013cf7fd fetch_command: simplify replay_convert_rev() with svn.list_files() Patrick Mezard <pmezard@gmail.com> parents: 97 diff changeset	128 data, mode = svn.get_file(p, r.revnum)
97 0d3a2a7cefa3 hg_delta_editor: fix symlink prefix confusion Patrick Mezard <pmezard@gmail.com> parents: 90 diff changeset	129 hg_editor.set_file(p, data, 'x' in mode, 'l' in mode)
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	130 hg_editor.missing_plaintexts = set()
186 6266ba36ee15 Create patch to make normal output much less verbose… Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 185 diff changeset	131 hg_editor.ui.note('\n')
0 f2636cfed115 Initial import of hgsubversion into a public repository. Augie Fackler <durin42@gmail.com> parents: diff changeset	132 hg_editor.commit_current_delta()

Mercurial > hgsubversion

annotate fetch_command.py @ 239:e2214c8fc91f