hgsubversion: rebuildmeta.py annotate

annotate rebuildmeta.py @ 225:2117cb0118fe

Get rid of .hg/svn/last_rev: We now calculate the last known revision by iterating over all known revisions and finding the highest number. Theoretically, we might be able to simply read the latest entry, but in practice, that's a bug waiting to happen. For instance, we might want to achieve compatibility with '.hg/shamap' as generated by the ConvertExtension, and it not only cannot offer a guarantee of linearity, but it also allows more than one conversion to source exists. I'd say we have other problems to care about until this turns up as a hotspot in profiling. Such as why we leak circa 100MB of memory per 1000 revisions converted ;)

author	Dan Villiom Podlaski Christiansen <danchr@gmail.com>
date	Fri, 27 Mar 2009 01:09:36 +0100
parents	30df375590d1
children	f71af18c4379

rev	line source
155 ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	1 import os
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	2 import pickle
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	3
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	4 from mercurial import node
205 30df375590d1 Less fail with rebuildmeta url argument Luke Opperman <luke@loppear.com> parents: 196 diff changeset	5 from mercurial import util as mutil
155 ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	6
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	7 import svnwrap
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	8 import util
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	9
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	10 def rebuildmeta(ui, repo, hg_repo_path, args, **opts):
185 57355b0e7bd1 Creating patch for documention messages. Dan Villiom Podlaski Christiansen <danchr@cs.au.dk> parents: 181 diff changeset	11 """rebuild hgsubversion metadata using values stored in revisions
155 ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	12 """
205 30df375590d1 Less fail with rebuildmeta url argument Luke Opperman <luke@loppear.com> parents: 196 diff changeset	13 if len(args) != 1:
30df375590d1 Less fail with rebuildmeta url argument Luke Opperman <luke@loppear.com> parents: 196 diff changeset	14 raise mutil.Abort('You must pass the svn URI used to create this repo.')
155 ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	15 uuid = None
205 30df375590d1 Less fail with rebuildmeta url argument Luke Opperman <luke@loppear.com> parents: 196 diff changeset	16 url = args[0].rstrip('/')
30df375590d1 Less fail with rebuildmeta url argument Luke Opperman <luke@loppear.com> parents: 196 diff changeset	17 svn = svnwrap.SubversionRepo(url=url)
155 ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	18 subdir = svn.subdir
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	19 svnmetadir = os.path.join(repo.path, 'svn')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	20 if not os.path.exists(svnmetadir):
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	21 os.makedirs(svnmetadir)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	22
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	23 revmap = open(os.path.join(svnmetadir, 'rev_map'), 'w')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	24 revmap.write('1\n')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	25 last_rev = -1
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	26 branchinfo = {}
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	27 noderevnums = {}
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	28 for rev in repo:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	29 ctx = repo[rev]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	30 convinfo = ctx.extra().get('convert_revision', None)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	31 if convinfo:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	32 assert convinfo.startswith('svn:')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	33 revpath, revision = convinfo[40:].split('@')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	34 if subdir and subdir[0] != '/':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	35 subdir = '/' + subdir
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	36 if subdir and subdir[-1] == '/':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	37 subdir = subdir[:-1]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	38 assert revpath.startswith(subdir), ('That does not look like the '
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	39 'right location in the repo.')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	40 if uuid is None:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	41 uuid = convinfo[4:40]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	42 assert uuid == svn.uuid, 'UUIDs did not match!'
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	43 urlfile = open(os.path.join(svnmetadir, 'url'), 'w')
205 30df375590d1 Less fail with rebuildmeta url argument Luke Opperman <luke@loppear.com> parents: 196 diff changeset	44 urlfile.write(url)
155 ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	45 urlfile.close()
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	46 uuidfile = open(os.path.join(svnmetadir, 'uuid'), 'w')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	47 uuidfile.write(uuid)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	48 uuidfile.close()
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	49 commitpath = revpath[len(subdir)+1:]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	50 if commitpath.startswith('branches'):
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	51 commitpath = commitpath[len('branches/'):]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	52 elif commitpath == 'trunk':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	53 commitpath = ''
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	54 else:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	55 assert False, 'Unhandled case in rebuildmeta'
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	56 revmap.write('%s %s %s\n' % (revision,
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	57 node.hex(ctx.node()),
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	58 commitpath))
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	59 revision = int(revision)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	60 noderevnums[ctx.node()] = revision
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	61 if revision > last_rev:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	62 last_rev = revision
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	63 branch = ctx.branch()
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	64 if branch == 'default':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	65 branch = None
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	66 if branch not in branchinfo:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	67 parent = ctx.parents()[0]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	68 if (parent.node() in noderevnums
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	69 and parent.branch() != ctx.branch()):
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	70 parentbranch = parent.branch()
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	71 if parentbranch == 'default':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	72 parentbranch = None
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	73 else:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	74 parentbranch = None
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	75 branchinfo[branch] = (parentbranch,
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	76 noderevnums.get(parent.node(), 0),
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	77 revision)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	78 for c in ctx.children():
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	79 if c.branch() == 'closed-branches':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	80 if branch in branchinfo:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	81 del branchinfo[branch]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	82 branchinfofile = open(os.path.join(svnmetadir, 'branch_info'), 'w')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	83 pickle.dump(branchinfo, branchinfofile)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	84 branchinfofile.close()
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	85 tagsinfo = {}
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	86 realtags = svn.tags
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	87 tagsleft = realtags.items()
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	88 while tagsleft:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	89 tag, tagparent = tagsleft.pop(0)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	90 source, rev = tagparent
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	91 if source.startswith('tags/'):
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	92 src = source[len('tags/'):]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	93 if src in tagsinfo:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	94 tagsinfo[tag] = tagsinfo[src]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	95 elif src in realtags:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	96 if (realtags[src][1] <= last_rev
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	97 or realtags[src][0].startswith('tags/')):
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	98 tagsleft.append(src)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	99 else:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	100 older_tags = svn.tags_at_rev(rev)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	101 newsrc, newrev = older_tags[src]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	102 tagsleft.append((tag, (newsrc, newrev)))
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	103 if source.startswith('branches/') or source == 'trunk':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	104 source = determinebranch(source)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	105 if rev <= last_rev:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	106 tagsinfo[tag] = source, rev
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	107 tagsinfofile = open(os.path.join(svnmetadir, 'tag_info'), 'w')
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	108 pickle.dump(tagsinfo, tagsinfofile)
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	109 tagsinfofile.close()
181 e37f9d3fd5e7 remove decorators (compat with python2.3) Benoit Boissinot <benoit.boissinot@ens-lyon.org> parents: 155 diff changeset	110 rebuildmeta = util.register_subcommand('rebuildmeta')(rebuildmeta)
196 77812f98e250 Add a naive hg svn version command that works as long as hgsubversion is run from a checkout. Augie Fackler <durin42@gmail.com> parents: 185 diff changeset	111 rebuildmeta = util.command_needs_no_url(rebuildmeta)
155 ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	112
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	113 def determinebranch(branch):
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	114 if branch.startswith('branches'):
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	115 branch = branch[len('branches/'):]
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	116 elif branch == 'trunk':
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	117 branch = None
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	118 else:
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	119 assert False, 'Unhandled case while regenerating metadata.'
ba801f44d240 utility_commands: Implement rebuildmeta so that metadata can be rebuilt. Augie Fackler <durin42@gmail.com> parents: diff changeset	120 return branch

Mercurial > hgsubversion

annotate rebuildmeta.py @ 225:2117cb0118fe