Category: Technology (Page 45 of 47)

Converting Monkey Audio to WAV in Gentoo

July 9, 2008 / Rowan Rodrik

I had the bad luck that a CD image I acquired from the Internet consisted of a .ape file for the audio and a .cue file for the track offsets. cdrdao has no problem with .cue files but .ape is the file extension for Monkey Audio files. Monkey Audio is the awkwardly licensed monkey brother of Flac. (They’re both lossless audio encoding formats.)

This is one of those I-want-to-remember-this-for-later posts which I should really do more often if I wander about investigating various technicalities. Although such posts aren’t the most comprehensive and useful to the reader, they’re invaluable to myself six months from now.

mac is a tool capable of converting .ape files to .wav files. But first, .Ape’s awkward licensing problem meant I had to find an ebuild for mac to put in my Portage overlay in $PORTDIR_OVERLAY at $PORTDIR_OVERLAY/media-sound/mac. Through references in the Gentoo Forum and the Gentoo Wiki, I came across two different bug reports with ebuilds for mac attached. Bug 52882 had the newest ebuilds attached.

$ mkdir -p /usr/local/portage/media-sound/mac
$ cd /usr/local/portage/media-sound/mac
$ wget http://bugs.gentoo.org/attachment.cgi?id=154775
$ ebuild mac-3.99.4.5-r2.ebuild manifest
$ emerge -av mac

Now, I could convert any .ape files to WAV:

$ mac CDImage.ape CDImage.wav -d

Alternatively, I could have done this using shntool, a handy front-end for all sorts of lossless audio programs:

$ emerge -av shntool
$ shntool conv -o wav CDImage.ape

Now, I could have just gone into the CDImage.cue file and change the reference to the .ape file to a reference to the .wav file:

$ sed -i -e 's/.ape/.wav/' CDImage.cue
$ cdrdao write CDImage.cue

But, first I was sidetracked to splitting the tracks (until realizing that wodim wouldn’t burn the correct pregaps).

Just grepping for “INDEX 01” didn’t generate the indexes in a format that shntool split would eat, leading to the emergence of yet another package: cuetools.

$ emerge -av cuetools
$ cuebreakpoints CDImage.cue | shntool split CDImage.wav

Mounting partititions residing inside a disk image

July 9, 2008 / halfgaar

When you’ve read a disk with dd or ddrescue into an image file, one of the things you want to do is mount the partitions inside it. But, how does one do that? The answer is using losetup, with a bit of knowledge of partition offsets.

First, you have to map the disk image to a loopback device:

# losetup /dev/loop/0 disk_image_file

Then use fdisk to print the partition table, displaying offsets in sectors:

# fdisk -lu /dev/loop/0

For my disk, that results in:

Disk /dev/sda: 250.0 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63   482415884   241207911   fd  Linux raid autodetect
/dev/sda2       482415885   488392064     2988090   fd  Linux raid autodetect

You can see that the first partition begins at sector 63. Now we’re going to setup a new loop device, with the proper offset. A disk sector is 512 bytes long, so an offset of 63 sectors is 32256 bytes. Therefore, you setup the first partition with this command:

# losetup -o 32256 /dev/loop/1 disk_image_file

Then you can mount that loop device:

# mount /dev/loop/1 /mnt/mountpoint

Using diff and patch to upgrade web application installations

March 13, 2008 / Rowan Rodrik

Update (July 30, 2008): I added information about making sure that the patch was successful.

When you install a big-ass web application such as WordPress or MediaWiki, you usually end with a bunch of configuration files and customizations (skins/themes, extension/plugins, uploads, etc.). This makes upgrading the files that come with the application a bit tricky. There’s a simple solution, however, which work regardless of whether you use a revision control system or not.

First of all, you do, of course, always need a revision control system. I personally recommend Git or Subversion, which are both excellent tools. But, that’s not what this post is about. I’m going to use two simple tools which are uniformly available on all (Unixy) platforms: diff and patch.

The procedure is simple:

Download the version of the application which you’re currently running. For our example, we pretend that this version is extracted into the directory webapp-1.4.3.
Then, download the version to which you’d like to upgrade. (We’re assuming that this version is extracted into the webapp-1.6.2 directory.)

Compare the two versions to create a patch file:

$ diff --unified --recursive --new-file webapp-1.4.3 webapp-1.6.2 > webapp-upgrade.diff

Apply the patch to the installation of said web app:

$ cd webapp-installed
$ patch --strip=1 --remove-empty-files < ../webapp-upgrade.diff || echo "Some failures!"

Check if everything was patched perfectly

Now, if the patch command returned a non-zero status (printing Some failures! in the above example), it's time to check which chunks of which files failed. Get a summary by searching all files with an .rej or a .orig suffix:

$ find . -name "*.rej"

After manually applying any failed hunks, what's left is to compare your directory containing the patched application to the directory with the contents of the new application archive which you've used to create the patch:

$ cd ..
$ diff --unified --recursive --new-file webapp-1.6.2 webapp-installed

Version management

Your upgrade is done. Now, if your using a revision control system, you just need to check in new files and check out deleted files. In Subversion, I do this quickly using the following command sequence:

$ svn status|sed -e '/^\?/!d; s/^\?//'|xargs svn add
$ svn status|sed -e '/^\!/!d; s/^\!//'|xargs svn del

If you'd been using Git, you could do this all a little bit more sophisticatedly, but my Git skills are not advanced enough to go around giving others advice. Also, it's nice to learn a generic method before learning more specific tools.

Humanity Lobotomy – The Importance of Net Neutrality

March 7, 2008 / halfgaar

In the early days, radio used to be a two-way communication system. Radio communication played a part in social, religious and political dialogs between people. Later, big corporations like NBC and CBS produced 97% of nighttime broadcasting in the United States; all courtesy of the Federal Communications Commission. We cannot let them do the same to the internet.

Wilderness

January 15, 2008 / Rowan Rodrik

2018-01-03. The below was originally published by Myrna on www.worldwide-wilderness.com. I copied it here while discontinuing that site in its current form. The deepest love of my life is the World on which we live. She gave us all we have in her abundance, she will take us back into her bosom, into her breath, at the end of our lives and make us into yet another of her creations. She deserves our love and respect; our fear and disregard of her are clearly harmful to her and ourselves. The disconnect that is caused by our way of (not) looking at the World begets so many of the problems that we can observe around us. The fear of people for our Earth has its roots in a fundamental misunderstanding of her nature, mistakingly thinking of Wilderness as destructive, dangerous and brutal in her untamed state. The fear has scared out of our heads the understanding that this is just one side of the duality, overshadowing the positive and creative, which lies not only at the opposite of the destructive side, but is also to be found within it. My understanding of Wilderness is of resillience and strenght, because in that, both priciples are recognized. I often take pictures of mushrooms, flowers and insects on the roadside or in the middle of the city. It proves to me the power of Nature to incorporate and generate even in the most difficult circumstances. My preference for taking pictures of mushrooms also stems from this idea. A thing as beautiful and special as a mushroom (remember that it does not need it’s possible bright colours to attract insects for polination or anything else …) can only grow where there is dead material to feed on. This is also why I love taking pictures at my father’s place. Fifteen years ago it was a spotless garden and three meadows, and now, after much digging, piling, planting and pulling down trees, it has become a piece of Wilderness akin to my idea of paradise, where rare flowers and animals can be found. All that can be seen there depends on something else to die or live for its own journey through life, into death. Without use of massive amounts of dead plant and tree material it is a long and arduous task to stimulate the growth of new life, patiently waiting for Nature itself to undertake the task of accumulating the wealth of death on which to grow. Seeing these things, and taking part in helping them along, has taught me some of the most important things that I now know about the power of the Wilderness that brought fourth a species as strange as us humans. It has also given me the precious insight that this Wilderness lives on inside of us, its creatures, with both its destructive and creative sides showing in our actions. Accepting the dominance of Wilderness in our creation makes it easier to understand our dual nature, and steer away from the emphasis on our destructive side that is so prevalent in our current culture of fear. Recognizing, accepting and dealing with this fear should, in my view, be the main priority of our culture in the decades to come. Stimulating this in myself and others is the main motivation in everything that I do. The sight of the Earth and her human inhabitants recoiling from each other in horror is one that I long to replace by a rapt fascination for everything that is and shares our World, and through that, have all conciousness around us marvel at the beauty and wholeness of us as a part of this marvellous creation that is our World.

Moved from Mnemosyne to FlashcardDB

January 14, 2008 / Rowan Rodrik

When I was studying Spanish last year, I had to choose a flashcard program to memorize new words. At the time, I couldn’t find any on-line program that just did the job and did it well. In a comment on my blog post from last year, however, I was pointed by Jeff to his amazing FlashcardDB.

The program I ended up with last year was Mnemosyne. Mnemosyne is not based on your regular Leitner system, but rather on a concept where, after each card, you have to indicate yourself how well you have remembered it. I found that, in the end, having to tell the system in which box to put the card instead of just saying if my answer was right or wrong was taking me more effort than the actual recollection of the information. Also, as someone who rarely remains at one place for very long, a desktop program just isn’t as practical for me as an online program.

With Mnemosyne, I had to constantly remind myself of a complicated grading system.

Now to FlashcardDB. The site is pretty social, which means that you can study (and sometimes even edit) card sets made by other users. When you sign up, you can also create card sets yourself. Card sets can be tagged and you can study these tags instead of individual card sets if you wish. If you already have cards somewhere else, import is easy as well.

The user interface is very slick, especially for such a new program. Thoughtful usage of AJAX means that you’re never distracted by page reloads when this would interrupt your flow of thought. Simple key bindings making studying an easier affair than in most desktop programs. The right arrow is used to show the answer, the up arrow (thumbs up) to mark the answer as correct, the down arrow (thumbs down) to mark the answer incorrect and the left arrow to go back to the previous card. Also the interface for adding cards is very pleasant. It’s just a matter of filling in the front of the card, pressing Tab, filling in the back of the card, pressing Tab, then Enter and on the next card.

Before going on to the conclusion, I want to add that also the Leitner system is very well implemented in FlashcardDB, including pretty diagrams to make it instantly clear to everyone how the system works. Now for my conclusion: My advice if you ever need to make flashcards yourself is that you really should take a look at FlashcardDB before looking at anything else.

Finally, the following Ruby code is a quick hack I used to convert Mnemosyne’s XML export to CSV data which can be imported by FlashcardDB:

#!/usr/bin/ruby
 
require 'rexml/document'
require 'csv'
 
xmldoc = REXML::Document.new($stdin)
 
CSV::Writer.generate($stdout) do |csv|
  xmldoc.each_element('//item') do |el|
    csv << [  el.elements[1,'Q'].text, el.elements[1,'A'].text  ]
  end
end

Rolling back a failed rdiff-backup session on a full partition

January 13, 2008 / halfgaar

For those of you who use rdiff-backup, you may have run into the situation where the partition you’re backing up to has no more space available. This is a very annoying situation, because rdiff-backup needs space to be able to roll back the session, so you’re kind of stuck. You might think: “doesn’t a decent file system have a reserved amount of space for the super user?” Yes, decent file systems do, but you’ll often be running rdiff-backup as root, so that won’t do any good.

There is a very simple solution. So simple, you might not think of it, hence this post. Simply make a dummy file of, say, 50 MB on the target partition, which you can delete when you need the space for rolling back. To make such a file l33t-style:

dd if=/dev/zero of=/location/dummyfile bs=1048576 count=50

Getting your terminal unstuck

January 13, 2008 / halfgaar

Have you ever had your terminal freezing on you for a reason you can’t identify? Fret no more, because this post will save your future login session.

Your problems are most likely caused by sending an XOFF character to the server, by pressing CTRL-S. The XOFF character is a flow control character, which instructs the server to stop sending data. To undo this, you need to send an XON character, which you can do with CTRL-Q.

Misfortune would have it that CTRL-S is the forward search command of the readline library. This makes searching in your bash history, among other things, a little annoying. Luckily, you’ll mostly use CTRL-R, for backward search, but sometimes you need a forward search as well. I would welcome any suggestion as to how to use CTRL-S without having your terminal freezing on you. The terminal program I use, Putty, doesn’t seem to have an option for it.

Upgraded WordPress from 2.1 to 2.3.1

November 26, 2007 / Rowan Rodrik

I’m now on WordPress 2.3.1. It was about time too; I was still on 2.1.

Importing the tags from Ultimate Tag Warrior worked fine. Before upgrading and importing, I quickly patched my local version of WP with a little help from Subversion:

$ svn diff http://svn.automattic.com/wordpress/tags/2.1/ http://svn.automattic.com/wordpress/tags/2.3.1/ > wp.diff
$ blog.bigsmoke.us
$ patch --remove-empty-files -p0 < ../wp.diff
$ svn revert wp-config.php
$ svn add `svn status|grep '^?'|sed -e 's/\?//'`
$ svn rm `svn status|grep '^!'|sed -e 's/!//'`

Then, after a few changes to my template files to play nice with WP’s new built-in tagging system, everything was running again.

Native PostgreSQL authentication in Rails with rails-psql-auth

July 16, 2007 / Rowan Rodrik

A while ago, I wrote a PostgreSQL auth plugin for Rails. The plugin basically defers all authentication and authorization worries to the database layer where they are supposed to be taken care of anyway.

Using this plugin, the user is asked for his or her credentials using a HTTP Basic authentication challenge. (The code for this is adapted from Coda Hale‘s Basic HTTP authentication plugin.) It’s possible to specify a guest_username in the database.yml which will be used as a fall-back if no credentials are supplied. After successful login or if a guest user is found, the plugin will make sure that all database operations run as that user. If any operation fails due to insufficient user rights, the user will be prompted for a username/password pair again.

Detailed and up-to-date documentation for the plugin can always be found at the plugin’s homepage. Go to the plugin’s project page for getting help or for reporting issues with the plugin.