Tracking MediaWiki in a Subversion vendor branch

Vendor branches are the proper™ way of merging upstream changes in your web application installations. In Subversion, managing vendor branches isn’t so easy as it is in Git. Still, vendor branches make it much easier to track upstream.

From before I first deployed the Omega Research Wiki, I already used svn to track changes to my MediaWiki installation. However, for upgrading from one upstream release to the next, I used diff and patch. This isn’t the most reliable of methods as is exemplified by doing a diff comparing a fresh MediaWiki download with the actual files in my repo (which are supposed to belong to the same version).

What’s basically a shortcoming of svn is that I can’t just say:

svn merge http://svn.wikimedia.org/svnroot/mediawiki/tags/REL1_15_0/phase3/ \
http://svn.wikimedia.org/svnroot/mediawiki/tags/REL1_15_1/phase3/ .

This would have been incredibly helpful, because now I’m keeping a vendor branch not because of local modifications to upstream, but just to be able to merge cleanly.

In Subversion, maintaining a vendor branch by hand is quite some work, because you need to do a checkout first before you can import each version. (My working copy is normally a checkout of /trunk, not of /vendor/mediawiki.) Luckily, Subversion is distributed with a handy Perl script, svn_load_dirs.pl, which can do most of the heavy lifting.

Still, I didn’t feel like having to do to many manual steps, such as typing in the painfully long URLs for merging, so I decided to wrap the whole process into a nice little Bash script:

cat upgrade-mediawiki.sh
#!/bin/bash

svn_repo_url=file:///var/svn/wiki.omega-research.org/vendor/mediawiki/

merge=1
while [ $# -gt 2 ] # Process extra options do case "$1" in
--no-merge)
merge=0
;;
esac
shift
done

if [ $# -ne 2 ]; then echo "Usage:$0 [--no-merge] version version"
exit 1
fi

last_version=$1 new_version=$2

tmp_dir=mktemp -d
cd $tmp_dir for version in$*; do
echo "Downloading and extracting MediaWiki version $version..." if [ -z svn ls$svn_repo_url|grep $version ]; then branch=echo$version |sed -e 's/\.[0-9]\+$//' download_file="mediawiki-$version.tar.gz"
download_url="http://download.wikimedia.org/mediawiki/$branch/$download_file"

wget $download_url || { echo "Downloading$download_file failed">&2; exit 1; }
tar --extract --ungzip --transform 's/^mediawiki-//' --file $download_file svn_load_dirs.pl$svn_repo_url -t $version current$version
fi
done

cd -
if [ $merge == '1' ]; then svn merge "$svn_repo_url$last_version" "$svn_repo_url$new_version" . fi exit 0 The script only downloads and imports each specified version if that version doesn’t already exist in /vendor/mediawiki/. Also, because it has a --no-merge option, you can download all the old versions of MediaWiki that you’ve ever used, so that you can go back and compare old versions of your installation with the factory version. Of course, this is only useful if you were already tracking your installation in svn at the time, and even then not really. 😉 Anyway, the important thing is that you can use the script to download the version you’re running now and the version you want to upgrade too. I wanted to make a big jump, from 1.11.1 to 1.15.0 (I had put of the upgrade for a long time, because I first wanted to learn more about vendor branches): Of course, as always, you need to check if the patch went well. My own results were a vivid demonstration of the unreliability of my previous method: svn st|grep '^C' C languages/messages/MessagesKrj.php C languages/messages/MessagesWar.php C languages/messages/MessagesSe.php C languages/messages/MessagesFrc.php  Luckily, these were all files I was sure I hadn’t modified: for i in svn st|grep '^C'|sed -e 's/^C //'; do mv$i.merge-right.r32 $i; svn resolved$i; done
Resolved conflicted state of 'languages/messages/MessagesKrj.php'
Resolved conflicted state of 'languages/messages/MessagesWar.php'
Resolved conflicted state of 'languages/messages/MessagesSe.php'
Resolved conflicted state of 'languages/messages/MessagesFrc.php'


Apparently, a previous upgrade hadn’t turned my MediaWiki installation exactly into 1.11.1.

1. Comment by Rowan Rodrik
On September 20, 2009 at 12:42

You can’t use svn:externals for vendor branches.

When I was figuring out how to do vendor branches in svn, I first thought that I could be really clever by simply adding upstream as an external in myrepo/vendor. I thought: SVN doesn’t let me diff/merge two revisions from repos that are not checked out in my working dir, I’ll just add the remote repo as an external and work around the limitation. Turns out that there’s another limitation as well: you can’t merge anything from an external to the external’s context.

2. Comment by Rowan Rodrik
On March 11, 2010 at 23:43

I just extracted this script from my private repo for the Omega Research Wiki and turned it into a separate GitHub project: svn-upgrade-mediawiki. The reason is simply that I needed to upgrade my hardwood wikis and I don’t want to have to maintain a handful of different versions of the same script.

I had actually used svn_load_dirs.pl directly for these two wikis, so I had to take a few additional steps because my script strips “mediawiki-” part in the extracted dir name from the generated tag name. I upgraded my Dutch hardwood investments wiki like this:

svn-upgrade-mediawiki 1.14.0 1.15.2
# There's a merge failure; let's find out why
svn list file:///var/svn/wiki.hardhout-investeringen.net/vendor/mediawiki

1.15.2/
current/
mediawiki-1.12.0/
mediawiki-1.14.0/


# Let's rename these mediawiki-$v tags vendor_base=file:///var/svn/wiki.hardhout-investeringen.net/vendor/mediawiki for v in 1.12.0 1.14.0 do svn mv$vendor_base/mediawiki-$v$vendor_base/$v -m "Renamed mw$v vendor tag."
done

# Now, let's do a manual merge
svn merge file:///var/svn/wiki.hardhout-investeringen.net/vendor/mediawiki/{1.14.0,1.15.2}