Strony

czwartek, 28 stycznia 2010

hacking cvs2svn ;)

Several months ago my BOSS asked me to transfer our ancient CVS repository to the brand new Subversion system. Not a big deal -I thought - there is lovely tool which does it for me: cvs2svn converter made by Tigris. But just after that I realized it wouldn't be such easy task, as our current repository is rather huge and holds several years of our projects history. Some fraction of packages was obsolete and additionally there was a request to split it to three different SVN repos and keep in a newly Subversion repos only recent year of history.

Not a big deal once again - I said after reading of cvs2svn documentation - we've got a database holding all the cvs tags used to built our software, so I'd make a few queries to know what to keep and that's all, except... OMG! There is no "include" pattern for tags!!!

You could in a easy way exclude symbols from cvs, which shouldn't be converted, but I've got completely opposite situation. Now, smart guys, please try to write me a reqexp which is nagative to a pattern... Oups!

What I did it was a little modification of cvs2svn tool in cvs2svn/cvs2svn_lib/symbol_strategy.py:

Then of course I wrote a script which:

  • asked our tag db for packages and their tags/branches that should be migrated
  • generated cvs2svn option file
  • and run cvs2svn tool


Several testing later a big migration day arrived on agenda. I've started to migrate about 9:00 AM and before 6:00 PM ca. 2k packages were moved generating initially about 190k revisions. And one day later we've opened new repos to the whole collaboration.

poniedziałek, 25 stycznia 2010

SVN hooks once again, this time tags, branches, log messages etc.

This one hook I wrote quite long ago is forcing users of Subversion repository to:

  • provide log message on every commit (class logChecker)
  • block removing of "/tags", "/branches" and "/trunk" directories from repo (class pathChecker)
  • forcing branch and tag naming convention
  • making subdirectories under /tags read-only


My company is running Subversion for storing all software packages and our repo hasn't got the default structure (/tags, /trunk and /branches as a top directories) — instead of that every package has got it's own /trunk, /tags and /branches subdirectories. So the repository structure is something like this:

/Path/
PackageA/
trunk/
src/
doc/
tags/
PackageA-00-00-00/
src/
doc/
...
branches/
PackageA-00-00-00-branch/
src/
doc/

PackageB/
...
/OtherPath/
PackageC/
...


Looks rather complicated, but it isn't. ;)

Also we've got tags and branches naming convention. Every tag should be made of package name, then a three (or four for tags made over a branch) groups of digits, e.g.: PackageName-ii-jj-kk or PackageName-ii-jj-kk-ll.
Every branch name is very similar to tag name, except there is additional (-branch) string at the end.

So here is the hook itself:


If you want to use this one, you have to modify it a little:

  • replace 'librarian' or/and 'root' account name with your own repository admin
  • put repository admin mail to block messages - replace this fake 'librarian@mail' address
  • change regexp ffor tags and branches to match your criteria
  • put svn-policy.py script into your repository hook directory ($REPO/hooks)
  • modify "$REPO/hooks/pre-commit" script to switch this one on

    #!/bin/sh
    REPOS="$1"
    TXN="$2"
    SVNPOLICY=/path/to/repo/hooks/svn-policy.py # modify this line
    #SVN policy
    $SVNPOLICY -t "$TXN" "$REPOS" || exit 1
    # All checks passed, so allow the commit.
    exit 0



Maybe the code needs some cleaning and better structure but anyway happy using, comments and questions are welcome!

Cheers,
Krzysztof

piątek, 22 stycznia 2010

rendering graphs in PHP using GraphViz

Well known and very popular graph rendering software Graphviz has got many different language bindings and is widely used in many different areas of maths and computing. I don't have to speak more. It just renders all those graphs in easy way using very simple but powerful language.

Amongst various tools which are using Graphviz there is also a PHP package in PEAR — Image_GraphViz, which has got only one pitfall — it cannot render cluster (a subgraph) inside a cluster (at least not in 1.2.1 version). So, I modified it a bit to make it possible:



Here is the results of running above example on apache (it renders image in svg format, so not all web browsers are able to display it correctly):



Test Graph in SVG

I was very happy when I'd figured out that my changes went into the official Image_GraphViz 1.3.0RC3. Good luck folks!

czwartek, 26 listopada 2009

eol once again

Here is another pre-commit hook I wrote for Subversion repositories. Let's assume many people (not always smart enough) accessing and committing to your Subversion repository. They don't know how to configure their svn client, but you want to be sure, that all text files have the same line endings (e.g. Unix = LF). So, to be sure that all line endings are consistent, you could run this hook, which blocks commit operation if svn:eol-style is not set to required one.

As always you should modify also your pre-commit file, usually stored in repo/hooks directory, e.g.:

And that's it! ;)

poniedziałek, 23 listopada 2009

subversion hook for "syncing" inside one repository

The old school CVS has a very nice ability to make an alias in one module, which points to another one. In Subversion the situation is different, as svn is using it's own "file-system". Of course you can use svn:externals property to pull to your working copy some external source tree required during compilation. But what if you just want for some reason store exactly the same file tree in two different locations in your repo?

For that one you can use post-commit hook, which checks the change list of transaction and if required merge the changes made in one location to another:


#!/usr/bin/env python
##
# @file sync.py
# @author Krzysztof Daniel Ciba (Krzysztof.Ciba@NOSPAMgmail.com)
# @date 06/05/2009
# @brief post-commit hook for syncing two locations in repository
import os, sys
import datetime
import getopt
try:
my_getopt = getopt.gnu_getopt
except AttributeError:
my_getopt = getopt.getopt
import StringIO
from subprocess import Popen
from subprocess import PIPE

twinPaths = { "/destpath/dest1" : "/some/path/src1" }

svnhome = "/PATH/TO/YOUR/REPO/"
svnroot = "file:///PATH/TO/YOUR/REPO"
svnCmd = "/usr/bin/svn"
svnlookCmd = "/usr/bin/svnlook"

## make sync between twin packages
# @param rev revision number
def sync(rev):

svnlook = svnlookCmd + " changed " + svnhome + " -r " + rev
changed = Popen( svnlook.split(), stdout=PIPE).communicate()[0]
changed = StringIO.StringIO( changed )
changed = changed.read()

changed = changed.split( "\n" )

twins = []
for line in changed:
if ( line != "" ):
change, where = line.split()
where = where.strip()
sys.stdout.write( change + " in " + where + "\n" )
for srcPath, dstPath in twinPaths.iteritems():
print where + " " + srcPath

if ( where.startswith( srcPath ) ):
twins.append( ( srcPath, dstPath ) )

noDupes = []
[ noDupes.append(i) for i in twins if not noDupes.count(i) ]
twins = noDupes
if ( len( twins ) ):

for ( srcPath, dstPath ) in twins:
srcURI = svnroot + "/" + srcPath + "@" + rev
dstURI = svnroot + "/" + dstPath

cmd = "#!/bin/sh\n"
cmd += svnCmd + " co " + dstURI + "\n"
cmd += svnCmd + " merge " + dstURI + " " + srcURI + " " + dstPath +"\n"
cmd += svnCmd + " ci " + dstPath + " -m 'syncing " + dstPath + " with " + srcPath + "'\n"
cmd += "rm -rf "+ dstPath +"\n"

sys.stdout.write( cmd )
stdin, stdout, stderr = os.popen3( cmd )
err = stderr.read()
stderr.close()
out = stdout.read()
if ( err != "" ): return False
return True

## write usage and exit
def usage_and_exit(error_msg=None):
import os.path
stream = error_msg and sys.stderr or sys.stdout
if error_msg:
stream.write("ERROR: %s\n\n" % error_msg)
stream.write("USAGE: %s -r REV REPOS\n"
% (os.path.basename(sys.argv[0])))
sys.exit(error_msg and 1 or 0)

## main processing
def main( argv ):
repos_path = None
rev = None
try:
opts, args = my_getopt(argv[1:], 'r:h?', ["help"])
except:
usage_and_exit("problem processing arguments/options")
for opt, value in opts:
if opt == '--help' or opt == '-h' or opt == '-?':
usage_and_exit()
elif opt == '-r':
rev = value
else:
usage_and_exit("unknown option '%s'" % opt)

if rev is None:
usage_and_exit("must provide -r argument")
if len(args) != 1:
usage_and_exit("only one argument allowed (the repository).")

if ( sync(rev) ): return 0
return 1

## start processing
if __name__ == '__main__':
sys.exit( main( sys.argv ) )


Then of course you should modify your post-commit file, in my case it is very simple:


#!/bin/sh
REPOS="$1"
REV="$2"
# synchronisation between packages
SYNC=/PATH/TO/REPO/hooks/sync.py
$SYNC -r "$REV" "$REPOS"


And that's it! Every change (commit) made in /some/path/src1 would be merged and committed to /destpath/dest1.

środa, 18 listopada 2009

No i proszę, założyłem swojego bloga... Kto by pomyślał.

A dzisiaj dzień nie wesoły: starszy syn choruje, awansu w pracy nie ma, żona zła, atmosfera pod zdechłym psem. I nawet sam blogger pyta mnie: "Czy jesteś już członkiem?". Hmmm... No właśnie się tak dobrze czuję...