BigSmoke

Smokes your problems, coughs fresh air.

Month: July 2009

Tracking MediaWiki in a Subversion vendor branch

July 20, 2009 /

Vendor branches are the proper™ way of merging upstream changes in your web application installations. In Subversion, managing vendor branches isn’t so easy as it is in Git. Still, vendor branches make it much easier to track upstream.

From before I first deployed the Omega Research Wiki, I already used svn to track changes to my MediaWiki installation. However, for upgrading from one upstream release to the next, I used diff and patch. This isn’t the most reliable of methods as is exemplified by doing a diff comparing a fresh MediaWiki download with the actual files in my repo (which are supposed to belong to the same version).

What’s basically a shortcoming of svn is that I can’t just say:

svn merge http://svn.wikimedia.org/svnroot/mediawiki/tags/REL1_15_0/phase3/ \
http://svn.wikimedia.org/svnroot/mediawiki/tags/REL1_15_1/phase3/ .

This would have been incredibly helpful, because now I’m keeping a vendor branch not because of local modifications to upstream, but just to be able to merge cleanly.

In Subversion, maintaining a vendor branch by hand is quite some work, because you need to do a checkout first before you can import each version. (My working copy is normally a checkout of /trunk, not of /vendor/mediawiki.) Luckily, Subversion is distributed with a handy Perl script, svn_load_dirs.pl, which can do most of the heavy lifting.

Still, I didn’t feel like having to do to many manual steps, such as typing in the painfully long URLs for merging, so I decided to wrap the whole process into a nice little Bash script:

cat upgrade-mediawiki.sh
#!/bin/bash
 
svn_repo_url=file:///var/svn/wiki.omega-research.org/vendor/mediawiki/
 
merge=1 [ $# -gt 2 ] # Process extra options
    "$1"
        --no-merge)
            merge=0
            ;;
   
   
 [ $# -ne 2 ]; then
    "Usage: $0 [--no-merge] version version"
    1
 
last_version=$1
new_version=$2
 
tmp_dir=`mktemp -d` $tmp_dir
 version $*;
    "Downloading and extracting MediaWiki version $version..."
 
    [ -z `svn ls $svn_repo_url|grep $version` ];
        branch= $version |sed -e 's/\.[0-9]\+$//'`
        download_file="mediawiki-$version.tar.gz"
        download_url="http://download.wikimedia.org/mediawiki/$branch/$download_file"
 
        wget $download_url || { "Downloading $download_file failed">&2; 1; }
        tar --extract --ungzip --transform 's/^mediawiki-//' --file $download_file
        svn_load_dirs.pl $svn_repo_url -t $version current $version
   
 - [ $merge == '1' ];
    svn merge "$svn_repo_url$last_version" "$svn_repo_url$new_version" .
 0

The script only downloads and imports each specified version if that version doesn’t already exist in /vendor/mediawiki/. Also, because it has a --no-merge option, you can download all the old versions of MediaWiki that you’ve ever used, so that you can go back and compare old versions of your installation with the factory version. Of course, this is only useful if you were already tracking your installation in svn at the time, and even then not really. 😉

Anyway, the important thing is that you can use the script to download the version you’re running now and the version you want to upgrade too. I wanted to make a big jump, from 1.11.1 to 1.15.0 (I had put of the upgrade for a long time, because I first wanted to learn more about vendor branches):

Of course, as always, you need to check if the patch went well. My own results were a vivid demonstration of the unreliability of my previous method:

svn st|grep '^C'
C      languages/messages/MessagesKrj.php
C      languages/messages/MessagesWar.php
C      languages/messages/MessagesSe.php
C      languages/messages/MessagesFrc.php

Luckily, these were all files I was sure I hadn’t modified:

i `svn st|grep '^C'|sed -e 's/^C //'`; mv $i.merge-right.r32 $i; svn resolved $i;
Resolved conflicted state of 'languages/messages/MessagesKrj.php'
Resolved conflicted state of 'languages/messages/MessagesWar.php'
Resolved conflicted state of 'languages/messages/MessagesSe.php'
Resolved conflicted state of 'languages/messages/MessagesFrc.php'

Apparently, a previous upgrade hadn’t turned my MediaWiki installation exactly into 1.11.1.

TheSixtyOne, a better social music site

July 20, 2009 /

If you liked Last.fm, you’ll love TheSixtyOne. Discovering music is turned into a reputation game. Much like Last.fm, you can explore similar artists, genres, tags, moods and users. But, by doing so and hearting songs, you can earn reputation points and extra hearts. As your points increase, you level up, which gives you more authority. There’s an endless amount of quests to earn points faster and to make you discover all the facets of the application.

Just try it out. It’s fun and—unlike Last.fm these days–it’s free. You can choose when and how to help these starving artists! 😀

Setting up Rsync daemon

July 15, 2009 /

What can you do when you want to backup your entire machine to a remote location but only have non-root shell access? You use rsyncd.

On the server you need to configure a module in /etc/rsyncd.conf: 

[module_name]
uid=0
gid=0
path=/path
transfer logging=no
read only=no

Then you need to rsync using a destination like user@host::module_name/

To make it work, you first need to set up a tunnel using SSH, because the rsync port (873) is not open in the firewall, most likely. So, you do this: 

ssh -N -L 873:localhost:873 user@host

I wrote a script to automate this all. Perhaps I will follow-up on this post sometime and post it.

Blogging: quantity over quality

July 10, 2009 /

They say that blogging is a long way down from essaying. Articles were already better than essays, and books even more so. So they say. A lot is said these days. I say: the more, the merrier. I say: let us say! Let us say whatever we want. And let people read whatever they want, now that they can.

I try to blog as much as I can, not because I think that each post is a piece of poetry that needs to be enjoyed by as many as possible, but because each post represents a train of thought that I need to get rid of. The sooner, the better. Noise is not a problem. The Internet is filled with noise, but nevertheless we only read what we want to read and we still find what we need. We filter. If you don’t, good luck surviving the digital age.

There is a convergence movement towards media integration. Eventually, this will lead to the collapse of all these media, meaning that the need for distinct, specific media will be less and less. This is good. We need to be truly press-less before the original promise of the press can be fulfilled: information liberation.

What I mean by this is that, when we publish, we don’t need to make a clear distinction between the weighty and the weightless. After all, God Google defines the weight of our publications not by means of what we say about it, but by means of incoming link frequency and the relative weight of these links.

When I say something worthwhile, I first have to convince others that I did, but before I can do that, I have to convince them to come look. Then, after visiting me and having been convinced by my great content, they might decide to link to me. Only as soon as that happens, do I gain weight. (According to Technorati, the weight of this blog is pathetic.)

Not every post I write resembles even an essay. Most postst don’t even come close. That’s because blog posts don’t need a point. I’ve made this point before, but I can’t repeat it often enough, because every time that I’m thinking “Shit, what did I do then and what was I thinking?” I’m reminded of it. I’ll stop reiterating this no sooner than I stop forgetting to write these things down.

When a blog post does attempt to make a point, there are even stronger arguments for quantity over quality, because publishing—really publishing—something makes it possible for others to respond, be this in comments or in their own blog. There’s something concrete for them to cite and to reference. This process gives me valuable feedback and I can simply republish my thoughts (which have matured not only through time, but also through argument) in a new post. Voila! And maybe the old post is so shitty that nobody is interested… Who cares?! Obviously, they don’t! So go on and do something else with the time that you’ve freed by not having to polish a turd.

Remember that blog posts are not naturally organized into some sort of reading hierarchy. There’s no reason that the same subject can’t be treated twice. Just hit that “Publish” button, get it out of your head and get on with your life!

Editing Wikipedia/MediaWiki articles with VIM

July 10, 2009 /

Wikipedia has a number of tips for people who like to edit Wikipedia articles with VIM.

Of course, it all starts with opening an article in VIM. How I do this depends on my mood: I can open it by firing GVIM using Vimperator or I can mount the wiki with WikipediaFS. I prefer the latter.

Spell-checking

One tool that’s very convenient for editing large sections of prose is VIM’s built-in spell-checker. Example:

set textwidth=0
set spell spelllang=nl

Wrapping, matching and folding

There’s some more useful stuff that I copied to “ftplugin/mediawiki.vim” in “$HOME/.vim/“:

" Wikipedia articles often only have line-breaks at the end of each paragraph,
" a situation Vim by default doesn't handle as other text editors.
setlocal wrap linebreak
setlocal textwidth=0

" No auto-wrap at all
setlocal formatoptions-=tc formatoptions+=l
if v:version >= 602 | setlocal formatoptions-=a | endif

" Make navigation more amenable to the long wrapping lines.
noremap  k gk
noremap  j gj
noremap   gk
noremap   gj
noremap  0 g0
noremap  ^ g^
noremap  $ g$
noremap D dg$
noremap C cg$
noremap A g$a
inoremap   gk
inoremap   gj

" utf-8 should be set if not already done globally
setlocal fileencoding=utf-8
setlocal matchpairs+=<:>

" Treat lists, indented text and tables as comment lines and continue with the
" same formatting in the next line (i.e. insert the comment leader) when hitting
"  or using "o".
setlocal comments=n:#,n:*,n:\:,s:{\|,m:\|,ex:\|}
setlocal formatoptions+=roq

" match HTML tags (taken directly from $VIM/ftplugin/html.vim)
if exists("loaded_matchit")
    let b:match_ignorecase=0
    let b:match_skip = 's:Comment'
    let b:match_words = '<:>,' .
    \ '<\@<=[ou]l\>[^>]*\%(>\|$\):<\@<=li\>:<\@<=/[ou]l>,' .
    \ '<\@<=dl\>[^>]*\%(>\|$\):<\@<=d[td]\>:<\@<=/dl>,' .
    \ '<\@<=\([^/][^ \t>]*\)[^>]*\%(>\|$\):<\@<=/\1>'
endif

" Other useful mappings
" Insert a matching = automatically while starting a new header.
inoremap   = =(getline('.')==''\|\|getline('.')=~'^=\+$')?"==\Left>":"="

" Enable folding based on ==sections==
:set foldexpr=getline(v:lnum)=~'^\\(=\\+\\)[^=]\\+\\1\\(\\s*\\)\\=\\s*$'?\">\".(len(matchstr(getline(v:lnum),'^=\\+'))-1):\"=\"
:set fdm=expr

To have this automatically executed when opening files ending in .mw:

"autocmd BufRead,BufNewFile *.mw setfiletype mediawiki" > $HOME/.vim/ftdetect/mediawiki.vim

Syntax highlighting

Additionally, if you want to have syntax highlighting for MediaWiki articles, you can save this syntax file to $HOME/.vim/syntax/mediawiki.vim.

© 2026 BigSmoke

Theme by Anders Noren — Up ↑