Smokes your problems, coughs fresh air.

Tag: PHP (Page 1 of 2)

The decade-old posts bug

I just noticed that none of my posts older than a decade could be listed:

The culprit was in the following function, where I had to add "century" and "10" to the lists of $periods and $lengths respectively.

function bigsmoke_ago($timestamp_gmt)                                                    
{                                                                                        
  $difference = current_time('timestamp', true) - $timestamp_gmt;                        
  $periods = array("second", "minute", "hour", "day", "week", "month", "year", "decade");
  $lengths = array("60","60","24","7","4.35","12","10");                                 
                                                                                         
  for($j = 0; $difference >= $lengths[$j]; $j++)                                         
    $difference /= $lengths[$j];                                                         
  $difference = round($difference);                                                      
                                                                                         
  if($difference != 1) $periods[$j].= "s";                                               
  $text = "$difference $periods[$j] ago";                                                
                                                                                         
  if ($j < 3) {                                                                          
    $current_day = date('j', current_time('timestamp', true));                           
    $arg_day = date('j', $timestamp_gmt);                                                
    $relative_day = ($current_day == $arg_day ? 'today' : 'yesterday');
                          
    $text = "$relative_day, $text";                                                      
  }                                                                                      
                                                                                         
  return $text;                                                                          
}

While add it, I also got rid of the useless Google ads.

Plus, I improved the link text replacement for posts without comments and commenting turned off.

NFSN PHP file write permissions in safe_mode

I’ve been causing some (security) concerns for myself by thoughtlessly using the dreaded 777 permissions for upload directories to allow the various PHP-based websites that I host at nearlyfreespeech.net to write files there. What this drastic anti-security measure didn’t allow me to is to manage these uploaded files through SSH (and SCP/Rsync). In the chroot jail which I’m allowed to enter through SSH, I am ‘me’, while the files created from PHP end up being owned by user ‘web’. However, for some reason these files didn’t get owned by group ‘web’ of which the ‘me’ user is a member. Also, I got into trouble with new directories that were being created by the upload scripts.

Writing files in PHP more securely

When I make a very basic test script in PHP, which writes a new file to a 777 directory owned by ‘me’, the files simply end up being owned by web:web with 775 permissions. To make this work a little more securely, I have to change the authorization for two entities:

  1. The directory to which I want to write needs to be group writable (775) and owned by group ‘web’.
  2. The PHP file that does the writing needs to also be owned by group ‘web’.

This will also allow the PHP file to write files in subdirectories that it creates. In the 777 scenario above, it would be possible to create these dirs, but not to create files within them. The PHP safe_mode restrictions in effect won’t allow a script owned by user ‘me’ and another group than ‘web’ to write files in a directory owned by ‘web:web’ and 775 permissions set.

All in accordance with the advice on writing files in PHP by the NFSN team.

Application headaches

There’s a little more to it, though. The stuff that I uploaded through MediaWiki and WordPress with my super-liberal 777 permission set on the upload dir (owned by ‘me’) somehow never ended up with the same group write permissions as the files in PHP test described above.

WordPress on NearlyFreeSpeech.Net

Uploads created by WordPress did end up with the permissions of the beast (666) set. However, directories created by WordPress (the year/month no. subdirectories) ended up with ‘web:web’ ownership, which regardless of their 777 mode, didn’t allow PHP in safe_mode to create any files within these directories. This is easy enough to solve by changing the ownership of the PHP files doing the writing to group ‘web’. Of course, this is best coupled with making the same changes to the upload directories and also changing the mode of these to 775.

The WordPress installation notes on the NFSN member wiki [for members only] has some more details.

I reviewed the code in WordPress responsible for writing files and I noticed that, whether creating files or directory, it actually looks at the permissions of the parent directory to decide on the mode of the newly created entity (using something along the lines of “$new_file_perms = $parent_perms & 0000666” for files and “$new_dir_perms = $parent_perms & 0007777” for directories).

MediaWiki on NearlyFreeSpeech.Net

The NFSN member wiki offers some NFSN-specific instruction for setting up MediaWiki [in their walled garden].

As with WordPress, I changed the ownership of the top-level PHP files and the upload directories to group ‘web’, as well as changing the permissions of the upload directories to 775.

However, uploaded files are being created with the mode 644 instead of 664. This is hugely annoying, because, still, I’m not allowed to access these files through SSH. I have yet to find out how I can best remedy this. Probably, I’ll end up with writing a simple PHP script that I can call just to chmod everything within the upload directory when the urge to manipulate these files strikes me.

Another beef with MediaWiki is that it creates subdirectories in the uploads directory with mode 777 instead of looking at the mode of the parent dir as WordPress does so neatly.

How to test payformystay.com

I haven’t got much experience when it comes to testing web applications. Instead (and more so out of apathy than belief), I’ve always adhered to the ad-hoc test approach. However, the usage of pure Posgres unit tests back when I worked on a complicated investment database with Halfgaar did teach me the advantages of test-driven development.

For payformystay, though, unit tests simply won’t cut it. The database design is quite straight-forward with not that many relationships and the schema’s only complexities arise from it being remarkably denormalized and full of duplication. Besides and contrary to mine and Halfgaar’s PostgreSQL project for Sicirec, the business logic doesn’t live all neatly and contained on the database level. And I’m not using a clean ORM wrapper either, which I could use as a unit test target. And what would be the point, since in typical MySQL/PHP fashion it would be much too easy to circumvent for a particular function.

What I want for this application is full functional test coverage so that I know that all parts of the website function correctly in different browser versions across operating systems. In other words: I want to know that the various parts are functioning correctly as implied by the fact that the whole is functioning correctly.

But how do you do automated tests from a browser?

At first, I thought I should probably cobble something together myself with jQuery, maybe even using a plugin such as Qunit with the composite addon.

But how was I going to run the tests for JavaScript independence then? Using curl/wget or one of these hip, headless browsers which seem to be bred for this purpose?

Choises, choises…

Selenium

Then, there’s Selenium which is a pretty comprehensive set of test tools, meant precisely for what I need. Sadly my wants weren’t easily aligned with my needs. Hence, it took me some time (months, actually) before I was sure that Selenium was right for me.

Selenium provides the WebDriver API—implemented in a number programming languages—that lets you steer all popular browsers either through the standalone Selenium Server or Selenium Grid. The server executes and controls the browser. Since Selenium 2, it doesn’t even need a JavaScript injection in the browser to do this, which is very interesting for my tests related to my desire to make my AJAX-heavy toy also available to browsers with JavaScript disabled for whatever reason.

Selenium versus my pipe dream

Selenium IDE is a Firefox extension which lets you develop Selenium scripts by recording your interactions with the browser. It stores its script in “Selenese”. This held quite some appeal to me, because my initial testing fantasy revolved around doing it all “client-side”, in the sense that I wouldn’t have to leave my browser to do the testing. I wanted to be able to steer any browser on any machine that I happened to stumble upon at my test site and fire those tests.

Well, Selenese can be interpreted by some WebDriver API implementations to remotely steer the browser, but it can’t be executed from within the browser, except by loading it into the Selenium IDE, which is a Firefox-only extension. Also, driving the browser through JavaScript has been abandoned by Selenium with the move away from Selenium-RC to WebDriver (which they’re currently trying to push through the W3C standardization process).

With everyone moving away from my precious pipe-dream, I remained clinging to some home-grown jQuery fantasy. But, how was I going to test my JavaScript-free version? Questions.

I had to eventually replace my pipe dream with the question of which WebDriver implementation to use and which testing framework to use to wrap around it.

PHPUnit

I thought PHPUnit had some serious traction, but seeing that it had “unit” in its name, I thought it might not be suitable for functional testing. The documentation being unit-test-centric, in the sense of recommending you to name you test cases “[ClassYouWannaTest]Test” didn’t help in clearing the confusion.

Luckily, I came across an article about acceptance testing using Selenium/PHPUnit [acceptence test = functional test].

I’ve since settled on PHPUnit by Sebastian Bergmann with the Selenium extension also by Bergmann. His Selenium extension provides two base TestCase classes: PHPUnit_Extensions_SeleniumTestCase and PHPUnit_Extensions_Selenium2TestCase. I chose to use the latter. I hope I won’t be sorry for it, since it uses Selenium 2’s Selenium 1 backward compatible API. Otherwise, they’ll probably have me running for Facebook’s PHP-WebDriver in the end. (PHP-Webdriver also has a nice feature that it allows you to distribute a FF profile to Selenium Server/Grid.)

But what about my pipe dream?

If only I’d be able to visit my test site from any browser, click a button and watch all the test scripts run, the failures being filed into the issue tracker (with error log + screenshot) and a unicorn flying over the rainbow…

Anyway, it’s a pipe dream and the best way to deal with it is probably to put the pipe away, smoothen the sore and scratch the itch.

PEAR pain

As customary for PEAR projects, PHPUnit and its Selenium extension have quite a number of dependencies, meaning that installing and maintaining them manually in my project repo would be quite a pain. I’ve used the pear command to install everything locally, but my hosting provider doesn’t have all these packages intalled, so if I want to run tests from there (calling Selenium Server here), I’ll have to manage all that pear pain along with my project files.

Doesn’t PEAR offer some way to manage packages in any odd location? I’m not interested in what’s in /usr/share/php/. I want my stuff in ~/php-project-of-the-day/libs/

Process pain

So far, I’ve remotely hosted both the production and the development version of payformystay, which is specially nice if you want to share development features with others. Now, it’s difficult to decide what’s more annoying:

  1. Creating a full-fledged, locally hosted version of the website, so that I can execute the tests locally as well as host the testing version (Apache+PHP+MySQL) locally. A lot of misleading positive test results assured due to guaranteed differences between software versions and configurations
  2. Installing all the PEAR packages remotely so that I can run the test from my hosting provider’s shell. This implies having to punch a hole through the NAT wall at home or anywhere I happen to be testing at any moment. Bad idea. I don’t even have the password to all the routers that I pass during the year.
  3. Running the development version of the website remotely, but running the tests locally so that there are no holes to punch, except that I’ll have to tunnel to my host’s MySQL process because my tests need to setup, look-up and check stuff in the database. At least, now I don’t have to install server software on my development machine and need only the php-cli stuff.

Executing system commands from PHP with SUID executable.

If you want to execute system commands from something like PHP, you need a SUID executable which you can call from your PHP scripts. This is such a script. It could be extended to support parameters for the commands you want to execute, but that would be an enormous security risk, because then anybody can execute any command. If you need something as flexible as that, you need to think about adding some kind of security restrictions, like a list of allowed commands.

When writing this, it occurred to me how unnecessary this all is. I will explain below. First, I will describe the old way.

Here is the c source code, as written for our backup script, bsbackup.sh:

// Wrapper for the bsbackup.sh shell script, to be able to run it as root when
// started from a webserver, for example. Set the resulting executable to SUID
// root.
 
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <error.h>
 
int main(int argc, char *argv[], char *envp[])
{
    int set_uid_result;
    int effective_user_id;
    int execute_script_error;
    char* script;
 
    effective_user_id = geteuid();
 
    // Set real and effective user ID
    set_uid_result = setreuid(effective_user_id, effective_user_id);
 
    if (set_uid_result != 0)
    {
        printf("Failed to set user id\n");
        return 1;
    }
 
    script = "/usr/local/sbin/bsbackup.sh";
 
    // This does not return on success.
    execve(script, argv, envp);
    execute_script_error = errno;
 
    // Show a fancy error message.
    error(execute_script_error, execute_script_error, script);
 
    // Shouldn't be necceary, but you never know.
    return 1;
}

To compile:

gcc -o bsbackup bsbackup.c

You can then run this inside PHP:

// The 2>&1 makes all error messages appear on stdout, for easy capturing.
passthru('/usr/local/sbin/bsbackup usb_backup 2>&1');

As I said, when writing this, it all became very clear to me that it is quite useless. One can also install sudo, run visudo and put this in (assuming your webserver runs as www-data, like on (Debian and Ubuntu):

www-data ALL = NOPASSWD: /usr/local/sbin/bsbackup.sh

Then in PHP, just run this:

passthru('sudo /usr/local/sbin/bsbackup.sh usb_backup 2>&1');

I haven’t tested whether specifying the parameters after the script in the passthru actually works, but I think so. If not, you can just write a wrapper script around the command you’re going to execute.

See what you like best 🙂

Taking control of the wpautop filter

WordPress does automatic paragraph formatting using the wpautop filter, some PHP code originally developed by Matt Mullenweg. For most of the time that this blog has existed, I’ve disabled the wpautop filter using the following two lines in my theme’s functions.php file:

remove_filter('the_content', 'wpautop');
remove_filter('the_excerpt', 'wpautop');

I’m not the only one to do this. But, I’m tired of having to manually type <p>s for every paragraph in every post that I write.

I’d like to be able have it back on, but preferably a bit smarter so that you don’t get all that crap (also common on bulletin boards with empty paragraphs around undetected block-level elements, especially if these are non-standard, related to plugins.

At the very least I want to be able to turn it off for when it does annoy me, on these moments that I don’t want auto-anything. Also, I need this if I don’t want to break the nearly 300 posts that I formatted manually. So, I want to use a custom field to turn the filter on or off.

I found an unpublished plugin (unpublished in the sense that the source isn’t hosted somewhere proper such as wordpress.org/extend/plugins/) which does some of what I want in a somewhat messy unmaintained manner. After years of just entering those damn <p>s (and over a year since this draft was in the making), I decided to do my own plugin for post-by-post (and global) control of the wpautop filter. It’s called wpautop-control:

<?php
/*
Plugin Name: wpautop-control
Plugin URI: http://blog.bigsmoke.us/tag/wpautop-control/
Description: This plugin allows you fine control of when and when not to enable the wpautop filter on posts.
Author: Rowan Rodrik van der Molen
Author URI: http://blog.bigsmoke.us/
Version: 1.0
*/
 
if ( is_admin() ) {
  add_action('admin_menu', 'wpautop_control_menu');
  add_action('admin_init', 'wpautop_control_settings');
 
  function wpautop_control_menu() {
    add_submenu_page('options-general.php', 'wpautop-control', 'wpautop control', 'manage_options', 'wpautop-control-menu', 'wpautop_control_options');
  }
 
  function wpautop_control_options() {
    if (!current_user_can('manage_options'))  {
      wp_die( __('You do not have sufficient permissions to access this page.') );
    }
 
  ?>
  <div class="wrap">
    <h2>wpautop control options</h2>
 
    <form method="post" action="options.php">
      <?php settings_fields('wpautop-control') ?>
      <table class="form-table">
        <tr valign="top">
          <th scope="row">wpautop filter on by default?</th>
          <td>
            <label><input type="radio" name="wpautop_on_by_default" value="1" <?php if ( get_option('wpautop_on_by_default') == '1' ) echo 'checked="1"' ?>> yes</label>
            <label><input type="radio" name="wpautop_on_by_default" value="0" <?php if ( get_option('wpautop_on_by_default') == '0' ) echo 'checked="1"' ?>> no</label>
          </td>
      </table>
 
      <p class="submit">
      <input type="submit" class="button-primary" value="Save Changes" />
      </p>
    </form>
  </div>
  <?php
  }
 
  function wpautop_control_settings() {
    register_setting('wpautop-control', 'wpautop_on_by_default', 'intval');
  }
}
else { // ! is_admin()
  add_filter('the_content', 'wpautop_control_filter', 9);
 
  function wpautop_control_filter($content) {
    global $post;
 
    // Get the keys and values of the custom fields:
    $post_wpautop_value = get_post_meta($post->ID, 'wpautop', true);
 
    $default_wpautop_value = get_option('wpautop_on_by_default');
 
    $remove_filter = false;
    if ( empty($post_wpautop_value) )
      $remove_filter = ! $default_wpautop_value;
    elseif ($post_wpautop_value == 'true')
      $remove_filter = false;
    elseif ($post_wpautop_value == 'false')
      $remove_filter = true;
 
    if ( $remove_filter ) {
      remove_filter('the_content', 'wpautop');
      remove_filter('the_excerpt', 'wpautop');
    }
 
    return $content;
  }
}
 
?>

(I’ve requested the plugin to be added to the WordPress plugin repository, so that it won’t have to be reinvented another 20 times in the next year or so.) 😉

After installing the plugin, you can choose whether to enable or disable wpautop by default. Then, for every post where you want to deviate from the default, you can set the wpautop custom field to ‘true’ or ‘false’.

I want the new default to be to enable the filter, but since all my old posts have been manually formatted, I want all these to have the wpautop field added and set to ‘false’.

Adding the appropriate custom field values to all existing posts is easy thanks to MySQL’s INSERT … SELECT syntax:

INSERT INTO wp_postmeta (post_id, meta_key, meta_value)
     SELECT wp_posts.ID, 'wpautop', 'false'
       FROM wp_posts
      WHERE post_type = 'post';

Ytec, WordPress and Aihato.nl

On Oktober, the 25th, in what will be known to future generations as a historical move, Wiebe changed the A record of www.aihato.nl to point to the new production site running at Ytec. The new site, a collaboration by Ytec and me, based on WordPress, has been in development since May. At least, that’s when I started taking notes. There had been some discussion, wire-framing and design done before that time.

The graphical design for the new Aihato website was created in Photoshop by a Ytec employee, building on a wire-frame created by Ying Hao (good friend and owner of Ytec). Another Ytec employee freed me of the burden of slicing the design into HTML/CSS, so that I could concentrate on the WordPress programming work involved. I liked not having to worry too much about design for once.

Comfortably Installed at Ytec

Comfortably Installed at Ytec

Initial development setup

Because I had decided to put WordPress in its own subdirectory to keep my custom stuff separate from the factory default stuff, I needed my own vhost at Ytec, something I had gotten used to with all my previous web development projects. Initially, I tried to make things work in my own ~subdirectory on a shared vhost, but this wreaked havoc with the rewrite voodoo that I needed to make WordPress live comfortably in its own subdir. Maybe, it would have been better to use vendor branches; but decisions, decisions…

A Makefile for deployment, sychronisation and backups

On many of my recent projects, I’ve used Rake instead of GNU Make. This time, I took it oldschool to pimp up my Make skills a bit. This proved pretty necessary, because I’ve spent ages on a bug in a previous version of the Makefile were I defined a variable after a make target without realizing that I had to put this in a separate rule from the instructions to make that target.

Why I even need a Makefile? Because when you’ve had your fair share of deployment, synchronisation and backup problems, you like to define rules to avoid these problems. Makefiles are ideal for that purpose, because they consist of rules.

I’m publishing the Makefile here because it’s one of the prettier Makefiles I’ve made and I like to brag and remember myself of some of the new things that I learned during its creation.

RSYNC_OPTIONS := --verbose --progress --recursive --delete --links --times --filter='merge ./rsync-upload-filters'
WORKING_COPY_ROOT := ${HOME}/www.aihato.nl/
LIVE_PRODUCTION_ROOT := ytec.nl:/too/much/info/aihato.nl/
LIVE_DEVELOPMENT_ROOT := ytec.nl:/too/much/info/aihato-dev/
MYSQL_LOGIN := --user=aihato --password=InYourDreamsIdForgetToChangeThis
 
 
deploy-production:
    # First, I sync everything except the symlink to the current WP version
    rsync $(RSYNC_OPTIONS) --filter="exclude /wp"  $(WORKING_COPY_ROOT) $(LIVE_PRODUCTION_ROOT)
    # Now, if the symlink's target has changed, we've atomically upgraded all WP files
    rsync $(RSYNC_OPTIONS) $(WORKING_COPY_ROOT) $(LIVE_PRODUCTION_ROOT)
 
backup-production: 
    rsync ${RSYNC_OPTIONS} $(LIVE_PRODUCTION_ROOT)uploads/ $(HOME)/aihato-uploads/
    ssh ytec.nl "mysqldump $(MYSQL_LOGIN) aihato" > aihato.sql
 
update-development: 
    rsync $(RSYNC_OPTIONS) $(WORKING_COPY_ROOT) $(LIVE_DEVELOPMENT_ROOT)
    rsync $(RSYNC_OPTIONS) $(LIVE_PRODUCTION_ROOT)uploads/ $(LIVE_DEVELOPMENT_ROOT)uploads/
    ssh ytec.nl "mysqldump $(MYSQL_LOGIN) aihato | mysql $(MYSQL_LOGIN) dev_aihato"
 
backup-development: mysql-dump-development
    rsync ${RSYNC_OPTIONS} $(LIVE_DEVELOPMENT_ROOT)uploads/ $(HOME)/aihato-uploads/
 
mysql-dump-development:
    ssh ytec.nl "mysqldump $(MYSQL_LOGIN) dev_aihato" > dev_aihato.sql
 
.PHONY: update-development backup-production deploy-production mysql-dump-development
www.aihato.nl – Front page – top portion

Top portion of the front page

www.aihato.nl – Settings – Reading

www.aihato.nl – Settings – Reading

Front page

The front page, after the header with the navigation and logo, starts with of a little snippet of text to welcome visitors. The rest of the page is filled with some selected stuff from the rest of the website: the latest news excerpts (plus a link to the full archive and the news feed and the Aihato hyve), clickable sponsor logos, some upcoming agenda items, a promotional movie clip, the latest video from the video gallery, a carousel with the latest photos, another carousel with all the fighter profiles and the latest fight results.

In WordPress, when you want the home page to be a static page, you have to change a setting in the Settings / Reading subpanel. You will then have to choose another page to be the “posts” page. The other page will than use the template hierarchy the same way the home page would without this setting. The only custom page template you can use for it is home.php, which might cause confusion with the actual home page.

Template entanglement

The start page is one of a number of pages for www.aihato.nl that needed a custom template. To associate a custom template with the start page, I had two choices: I could either name the template file page-3.php, according to the Template Hierarchy, or I could create a Page Template. The difference between the two options is that with the latter option, the association with the custom template happens from the Edit Page screen, whereas the first option relies on the naming of a template file in my theme. I chose the first option, which is a bit ugly, because after setting a page as start page, editing the page slug is no longer possible. (Normally you can name the template file page-<slug>.php, which is clearer and doesn’t depend on database state.) Both solutions are ugly in a sense because there’s just too much stuff in the database to my taste, but that’s another story which I’ll probably tell in reference to Drupal one day, since Drupal is way uglier than WordPress in this sense.

I’ve ended up with a bit of a random mix of page-targeted templates and templates targeted from pages. The highlight is a template which does both: page-sportschool.php targets the page with the “sportschool” slug, but also has the following comment so that I can select it from the Page Edit screen for the subpages of “sportschool”:

<?php
/*
Template Name: Sportschool 
 */
?>
Aihato Events mangement interface

Aihato Events mangement interface

Events

It was decided that the new website, like the old website, would have an agenda. The old website’s agenda was never up-to-date, so the new agenda should be easier to edit. To that end, I created an aihato-events plugin.

The plugin is quite simple. It adds two tables to the database – one to record (and announce) events and another to store fight results for these events (wins, losses, etc.). The second table links to a fighter profile by post ID (but more about that later).

Aihato Event Contestants

Aihato Event Contestants

Agenda page

Agenda page

The event management interface is pretty decent. It includes a few darlings, which I wouldn’t like to kill, except that I will probably overhaul the whole Aihato Events UI at some unspecified time in the future. The darlings are small touches such as the “Add new” buttons above and below the table which add a new row through AJAX at the top or the bottom of the table depending on which button is clicked. I’m also always a sucker for the in-place AJAX editing of the rows. The reason why I’ll probably still overhaul the UI at some future time is that I don’t like the same simple tabular interface for the Contestants panel. I had predicted that fight events would generally first be placed in the agenda before the event takes place, untill after the event, the results would be added. So far, nothing has been placed in the agenda before it takes place. Only after, to be able to link it to the results to be added. And even if this wasn’t true, the two screens should still become one I think.

The homepage contains the first few upcoming events. Sadly, there aren’t any yet. 😕 Below that short (and empty) list, there’s a big button which links to the complete agenda. This page has a design that somewhat deviates from the rest. Of course, it also has some custom template programming (in a template called page-agenda.php).

Page with fight results

Page with fight results

Event results

The homepage also contains all the fight results for the latest event in a nice little table at the bottom right. Consistent with all the other areas on the homepage, this one is also followed by a link to the results for all recorded events in the form of a nice big button. The page with the complete results is powered by page-uitslagen.php.

This is one of the templates which I should really clean up by moving some code into nice and clean helper functions that live in the theme’s functions.php instead of all over the place.

I18n

My interest in internationalization for this website extends only as far as that I want the visitor to be talked to in Dutch as much as possible. For the rest, I don’t really care. How much I don’t care can be summed up by the total absence of __()-encapsulated strings in my theme. What’s worse: my custom plugins also lack these l10n hooks, although, because I always feel like a sinner when working directly in what is considered a translation target by me and the rest of the English-oriented development world, the event management stuff that I added to the management interface is in English (although, again, without l10n hooks, so what’s the point?).

Aihato – Profile – Tobias

Fighter profile for Tobias

Aihato – Edit fighter profile

Editing a fighter profile now

Aihato – Profile – Djura

Fighter profile for Djura

Fighter profiles

Fighter profiles play a dominant role in the new design. Implementation took some time, and I’m still not entirely satisfied. During development, custom post types were introduced in WordPress. I had already implemented the fighter profiles using a page template and a whole heap of custom fields. Adding new profiles this way, however, is far from user-friendly. The user has to:

  • Set the page parent to “Vechters” (Dutch for “Fighters”);
  • set the page template to “Vechter”;
  • add new custom fields for Discipline, Fight record, Weight, Class, Age, Length and City while making sure that the values are entered correctly since these don’t have a type;
  • and set a featured image for display in the fighter carousel on the front page and above the profiles.

This is a lot of work, none of which is very obvious, so I hoped that custom post types would save the day. Theoretically they could have, but there were a few issues, some of which I only encountered when I was already quite far into the development of an aihato-profiles plugin which implemented the aihato_fighter custom post type.

I started out by fooling with some plugins to do some of the heavy lifting (such as Custom Post Type UI). I wasn’t particularly charmed by these for reasons which I’ve sadly forgotten because I haven’t commented on it at the time. One reason I can think of is that I never like defining stuff in the database which I feel belongs in a file.

There seemed to be a bug in the custom post type admin interface created by WordPress in that, even though I had enabled thumbnail support for my post type, the UI for this was lacking. Another bug related to images was that clicking the Insert image button replaced the current page with the upload dialog instead of loading it in a modal dialog through AJAX. These two bugs were show-stoppers. I won’t comment any further on the whole custom post type development process until I actually continue this process.

Anyway, it all works now and I don’t mind doing some work on new fighter profiles myself. Editing existing ones is easy enough, and at the visitor end, it all looks sexy enough. 😎

Guest-book

Implementing the guest book was pretty easy. What was less easy was importing all the entries from the old guest-book. Although, even that was incredibly easy compared with extracting (exporting is too expensive a verb) the entries from the old guest-book. The old guest-book was basically impossible to spider, because the pagination depended on POST. If it were only the page number in the POST request, it wouldn’t have been too bad (and quite hackable for my purpose), but there was all sorts of session-related crap and other ugly stuff that smelled like a bunch of Microsoft Monkeys had gone all out in a HTTP obfuscation contest.

My initial import strategy consisted of a simple PHP script (with a function adapted from some plugin) to be ran from the command-line, that accepted the author and date as arguments and the post body over STDIN.

<?php
 
require_once('wp-config.php');
 
function guestbook_new_comment ( $commentdata ) {
  $commentdata['comment_post_ID'] = 19 # This is the Aihato guestbook page
  $commentdata['user_ID']         = 0 # These people don't have accounts
 
  $commentdata['comment_author_IP'] = '127.0.0.1' $_SERVER['REMOTE_ADDR'];
  $commentdata['comment_agent']     = 'Hacked together import scripts (by BigSmoke)';
 
  // We want to use the original comment date, not the time now.
  //$commentdata['comment_date']     = current_time('mysql');
  //$commentdata['comment_date_gmt'] = current_time('mysql', 1);
 
  // Automatically approve these comments.
  $commentdata['comment_approved'] = 1;
 
  // Actually add to the database
  $comment_ID = wp_insert_comment($commentdata);
 
  do_action('comment_post', $comment_ID, $commentdata['comment_approved']);
 
  return $comment_ID;
}
 
$commentdata['comment_author'] = $ARGV[1];
$commentdata['comment_date'] = $commentdata['comment_date_gmt'] = $ARGV[2];
$commentdata['comment_content'] = trim(readfile(STDIN));
 
$new_comment_id = guestbook_new_comment($commentdata);
echo "Inserted new comment $new_comment_id to post 19.\n";
 
?>

The script would be called from a Ruby script that parsed the ugly-ass HTML-like tag soup also known as the old guest-book. I have to admit that the script is as ugly as the shit it’s supposed to make sense of. Fuck it! One-of scripts don’t need to look good; it’s already been deleted from svn 75 revisions ago.

However, I never could call the PHP script from the Ruby script because I couldn’t get the necessary gems to install on the development server where the import needed to happen, so I ran the script locally and modified it to use WordPress’ XML-RPC interface. To make this work, I only had to install a WordPress plugin to allow anonymous comments through XML-RPC. (See my previous notes on this subject, if you’re interested.)

[By the way, I just copied this script to the clipboard using “svn cat https://svn.ytec.nl/svn/aihato/trunk/import-guestbook.rb@37|xsel --clipboard”; see my post on xsel if you want to learn more.]

Aihato - Guestbook

The finished guestbook, complete with all the old and new enties

#!/usr/bin/ruby
 
require 'scrapi'
require 'open3'
require 'xmlrpc/client'
 
 
guestbook_entry = Scraper.define do
  process "td > div.GB_Head > div.GB_Date", :date => :text
  process "td > div.GB_Head > div.GB_Name", :name => :text
  process "td > div.GB_Body > div.GB_BodyText", :body => :element
 
  result :date, :name, :body
end
 
guestbook = Scraper.define do
  array :entries
 
  process "table.GB_MainGrid tr", :entries => guestbook_entry
 
  result :entries
end
 
# I need to do this because the document has at least 3 <html> tags,
# so it's impossible to parse, even for Tidy
fake_document = "<html><body>"
reading_guestbook_table = false
STDIN.readlines.each do |line|
  if line =~ /<table class="GB_MainGrid"/
    reading_guestbook_table = true
  end
  
  if reading_guestbook_table
    fake_document += line
    reading_guestbook_table = false if line =~ %r{</table>}
  end
end
fake_document += "</body></html>"
 
entries = guestbook.scrape(fake_document)
 
entries.delete_at(0)
 
entries.each do |entry|
  next unless entry['body']
 
  date_parts_in_proper_order = entry['date'].split(/-/).reverse
  date_string_with_proper_zeroes = "%d%02d%02dT00:00:00" % date_parts_in_proper_order
  entry['date'] = XMLRPC::Convert.dateTime( date_string_with_proper_zeroes )
 
  server = XMLRPC::Client.new("aihato.dev.ytec.nl", "/wp/xmlrpc.php")
 
  entry['body'] = entry['body'].to_s
  entry['body'].gsub!(%r{<div class='GB_BodyText'>(.*)</div>}m, '\1')
  entry['body'].gsub!(%r{</p>\s*<p>}, "\n\n")
  entry['body'].gsub!(%r{</?p>}, "")
  entry['body'].chomp!
 
  new_comment_id = server.call('wp.newComment', 1, '', '', 19, {'comment_parent' => 0, 'content' => entry['body'].to_s, 'author' => entry['name'], 'author_url' => '', 'author_email' => 'dummy@example.com'} )
 
  puts new_comment_id.inspect
  
  # Change date and approval status
  server.call('wp.editComment', 1, 'myuser', 'nottherealpassword', new_comment_id, {'status' => 'approve', 'date_created_gmt' => entry['date'], 'author' => entry['name'], 'author_email' => 'dummy@example.com'})
 
  #Open3.popen3("php -q import-guestbook.php '#{entry['name']}' #{entry['date']}") do |stdin, stdout, stderr|
  #  stdin << entry['body']
  #end
end

Because I was too stupid to write a spider function to download the old guest-book, I ended up simply clicking through all the pages and feeding the page source to my import script one page at the time.

The new guest book is the only page on the website with comments enabled. For the rest it’s like any other page with its own custom template (page-gastenboek.php).

Aihato – Contact

The contact form

Contact form

In my notes made during the development process, I have made a few comments (1, 2, 3, 4) about the troubles I had when looking for a simple plugin to create a simple contact form. I would have saved quite some time if I had skipped the search and wrote my own code to handle it. In the end I did use a plug-in. Well, I forked it, but that’s just another way of using it, isn’t it?

Aihato - News - 2010

The news archive

News

The actual news section (where I could use WordPress’ core strength – its blogging engine) is maybe the foremost reason why I let myself be suckered into another web project despite my many vows to never program for money again. (Well, this being a club project, means that I could somewhat sidestep my many promises to myself, because there was hardly money involved in the process. (I train for free for a year.))

The old website’s news page was just a very long list of all the news since 2003. This was pretty suck-ass. What was much worse, though, was that there was no RSS feed. This new website being WordPress based means that I have a whole slew of feeds to chose from. It gave me quite a kick when the first news item posted by someone else hit my feed reader. Now, there are no longer any sites left that I have to manually check for updates. Yay!

An interesting choice I made for the news archive is that I skipped pagination altogether and instead presented a list of years all the way back to 2003 where you’d normally expect to see pagination. Personally, I don’t mind long pages. In fact, I often find clicking “Next” and “Previous” infinitely much more annoying.

Commenting on the news isn’t allowed by request of the Aihato boys. They gave some pretty good reasons not to do this mostly related to the intentional abuse by club members and members of competing clubs that they’ve seen on the website of a friendly club.

The news section is just one of the many places where I’ve made thankful use of WordPress’ new Post Thumbnails feature. I like it when stuff that’s only available through clumsy hacks and plugins makes it into core. By the way: when working with post thumbnails, the regenerate-thumbnails plugin proved to be an enormous aid.

Aihato – Photo albums

Overview of all photo albums

Aihato – Photo album – Ede

Photo album of a grappling competition

Media gallery

Even on the old website, the foto gallery played an important role. Thinking of the best way how to do this in WordPress was quite a headache.

To start with, the design requirements were pretty steep. Ying had included a coverflow-like effect in his wire-frame for viewing individual albums. Luckily, the list of photo albums wasn’t too difficult (a simple grid-view) and made easier still by the HTML/CSS guy. I also skipped a few requirements such as highest rated photos and videos. (I skipped the rating feature altogether.) Still, I spent a lot of time looking through available plug-ins and into different ways to solve the most challenging requirement: there had to be a separate section for the photo albums and the videos, where intuitively I’d simply include it all in the news as is customary with a blog. In the end, I did exactly this but with a twist.

The process of publishing a new photo album has become extremely straight-forward: the user has to upload the images using the Add image link, insert the gallery in the post (if they want a clear link from the news item view to the gallery) and check the “Fotogalerij” (Dutch for “photo gallery”) category (if they want the album to appear in the list of albums).

Since I’ve chosen not to make photo albums a separate entity in the back-end, I had to work a little magic to make them appear as such to the visitor. But I didn’t want to make the separation go too far; I don’t like websites (such as the old Aihato website) where the photo gallery seems bolted on as an afterthought and the user has to upload an album and then create a link to the album in the news.

The gallery view

You know how WordPress makes a comments feed available for every post? It accomplishes this using something it calls a rewrite endpoint (“feed” for feeds). For example:

http://www.example.com/blog/2010/11/08/post-with-interesting-comments/feed/atom/
http://www.example.com/blog/2010/11/08/post-with-interesting-comments/feed/rss/ 

You can add such a rewrite endpoint yourself using the add_rewrite_endpoint() function. The code below shows how I created an alternative view for my posts and pages called “gallery”. It also shows what I need to do to make an extra query variable available with the name of the endpoint. The part after the slash after the endpoint in the URL become the new query variable’s value.

add_rewrite_endpoint('gallery', EP_PERMALINK | EP_PAGES);
$wp_rewrite->flush_rules();
 
add_filter('query_vars', 'aihato_queryvars');
add_action('template_redirect', 'aihato_special_gallery_template');
 
function aihato_special_gallery_template() {
  global $wp_query;
 
  if ( is_category('fotogalerij') or is_category('filmgalerij') ) {
    include(TEMPLATEPATH . '/galleries.php');
    exit;
  }
 
  if ( isset($wp_query->query_vars['gallery']) ) {
    include(TEMPLATEPATH . '/gallery.php');
    exit;
  }
}
 
function aihato_queryvars($qvars) {
  $qvars[] = 'gallery';
  return $qvars;
}

The code above creates an alternative “view” of posts that I can use to view all the images attached to that post. When the user inserts the gallery into a post, the following code makes it so that instead of the images, the visitor will see a link to the gallery view of that post.

add_filter('post_gallery', 'aihato_gallery_filter', 2);
 
/**
 * Modifies the behaviour of the [gallery] shortcode.
 */
function aihato_gallery_filter($null, $attr = array()) {
// Snipped: code to generate a nice link
}

ContentFlow / FancyBox integration

To make the gallery view look cool, I implemented the CoverFlow effect using the ContentFlow jQuery plugin. It’s pretty cool. It supports reflection, scrolling with a scroll wheel and it just feels right™. I hooked it up to FancyBox, a very slick Lightbox clone for jQuery. The result was, I must say, immensely pleasing. 🙂 Both effects support scrolling and the FancyBox effects make it look like the images in the ContentFlow are really blown up and shrunk. (I’ve made it so that the FancyBox appears when you click the active image in the ContentFlow.)

This is some of the spaghetti code that made the two effects play nicely together:

// Returns the offset of the item to start showing
function albumFlowStartItem() {
  var hashNumber = window.location.hash;
 
  if ( hashNumber && hashNumber.match(/^#\d+$/) ) {
    hashNumber = hashNumber.replace(/^#(\d+)$/, '$1');
    return jQuery('#album_flow a#attachment-'+hashNumber).prevAll().size();
  }
 
  return 'center';
}
 
// My own custom state variable
jQuery.fancybox.remainActiveUntilClosed = false;
 
jQuery(document).ready(function() {
  jQuery('#album_flow a').fancybox({
    transitionIn: 'elastic',
    transitionOut: 'elastic',
    speedIn: 600,
    speedOut: 200,
    overlayShow: false,
    cyclic: true,
    onStart: function(selectedArray, selectedIndex, selectedOpts) {
      element = selectedArray[selectedIndex];
 
      return jQuery.fancybox.remainActiveUntilClosed || element.hasClassName('active');
    },
    onComplete: function() {
      jQuery.fancybox.remainActiveUntilClosed = true;
    },
    onClosed: function() {
      jQuery.fancybox.remainActiveUntilClosed = false;
    }
  });
 
  var albumFlow = new ContentFlow('album_flow', {
    reflectionHeight: 0.3,
    flowSpeedFactor: 0.7,
    startItem: albumFlowStartItem(),
    onclickActiveItem: function(item) {
      var itemOffset = jQuery(item.element).prevAll().size();
      jQuery.fancybox.pos(itemOffset);
    },
  });
});

Categories for photo/video galleries

To make a post appear in the photo gallery, you just have to check that category. Making posts appear in the video gallery works the same. These listings are displayed using the galleries.php template thanks to a little bit of code in aihato_special_gallery_template(). I redirected these archive views to that template because otherwise I’d have had to make a symlink to use the same file for the video category and the photo category. (I’d have needed two files: category-fotogalerij.php and category-filmgalerij.php.)

I like how I simply used a custom view of both a post and of two different category archives to achieve all my media gallery requirements. There’s no wild database customizations or heavy plug-ins involved. It’s low-fat and carb-free.

Aihato - Film gallery

The film gallery

YouTube is king

Because I was too lazy to find a good playback solution and I’m a bit reluctant to self-host video files anyway, I decided to put together something that relies solely on embedding videos hosted elsewhere. To be completely honest, although WordPress is quite flexible in this sense, “elsewhere” means just YouTube here.

The idea is simple: WordPress already allows you to just paste a YouTube URL into the post editor and all the embedding code is created for you. Building on this, to show the latest video on the homepage, I just perform a search for posts which contain a YouTube URL. Then I parse the content a bit, and include the YouTube ID in my own low-res embed code. (The latest video area on the homepage is smaller than the default embed created by WordPress.)

When generating the film gallery overview, my theme goes through all the YouTube URLs in all posts categorized as “Filmgalerij”. For each of these URLs, it uses the YouTube API to retrieve the movie title and the URL of an adequately sized thumbnail. That means that, for thumbnails to appear in the gallery, the associated posts don’t need a featured image, just one or more YouTube URLs. This approach also makes it so that you can embed as much YouTube URLs in each post as you like, since the gallery will cope beautifully.

When a visitor clicks a movie thumbnail, a YouTube embed pops up using FancyBox. Did I mention how cool FancyBox is? Pretty damn cool:

jQuery('a.youtube').click(function(){
  jQuery.fancybox({
    'padding': 0,
    'autoScale': false,
    'transitionIn': 'none',
    'transitionOut': 'none',
    'title': this.title,
    'width': 680,
    'height': 495,
    'href': this.href.replace(new RegExp("watch\\?v=", "i"), 'v/'),
    'type': 'swf',
    'swf': {
      'wmode': 'transparent',
      'allowfullscreen': 'true'
    }
  });
 
  return false;
});

Menus and navigation

// This is the ultimate in ugly hacks. Enjoy! :-)
function aihato_main_menu_filter($items) {
  global $wp_query;
 
  // menu-item-639 = Nieuws
  // menu-item-643 = Foto/Video (connected to the fotogalerij category)
 
  // This conditional makes it the current-menu-item also when we're in the filmgalerij category,
  // and when we're looking at the gallery view of a post (through the gallery rewrite endpoint).
  if ( is_category('filmgalerij') or isset($wp_query->query_vars['gallery']) ) {
    $items = preg_replace('!(menu-item-643)!', '\\1 current-menu-item', $items);
  }
  // This conditional ensures that the Nieuws menu item is active when we don't want to be in the gallery.
  // At the same time, it makes sure that the the Foto/Video menu item is inactive.
  elseif ( !isset($wp_query->query_vars['gallery']) and (is_archive() or is_single()) ) {
    $items = preg_replace('!(menu-item-639)!', '\\1 current-menu-parent', $items);
    $items = preg_replace('!current-menu-parent current-post-parent (menu-item-643)!', '\\1', $items);
  }
 
  return $items;  
}

As soon as WordPress 3.0 was released somewhere during the development of this website, I started to use its new Custom Menu Management feature.

Before the change:

<?php wp_list_pages(array( 'depth' => 1, 'title_li' => '', 'sort_column' => 'menu_order, post_title' )) ?>

After the change:

<?php wp_nav_menu(array( 'menu' => 'main', 'depth' => 1 )) ?>

As you can see, the change wasn’t difficult, but, more importantly, it gave me some useful powers that I could use for good. For the main menu, I could include a category, which I used to add the Fotogalerij category. I could also change the label of that item to be different from the category name so that it also seems to apply to the Filmgalerij category. That, together with the ugly hack above, gave me my illusionary Photo/Video category.

Aihato – Sportschool

Putting custom menus to good use in this section

Another place where I could put the custom menus to good use was the Sportschool section. There I had to design a submenu, because designers always forget a few vital pieces in their design, such as how submenus should look. However, the submenu shouldn’t just include links to pages, but also links to two different subscription forms (uploads). The new menu system allows me to do this quite easily.

So, again, I could replace something that didn’t do exactly what I wanted:

<?php wp_list_pages(array( 'title_li' => get_the_title(11), 'child_of' => 11, 'include' => array(11) )); ?>

With something simpler that did:

<?php wp_nav_menu(array( 'menu' => 'school' )) ?>

It is a curious aspect of this website that every section has its own means of navigating within that section.

  1. The fighter profiles section uses a carousel at the top to select fighters. In the future, some form controls to filter the carousel will also be added.
  2. The Photo/Video gallery is divided into two subsections (one for photos and one for videos). These subsections are subsequently navigated using a grid view of the individual photo albums or videos. When viewing a photo album, navigation is further refined using the ContentFlow UI.
  3. The news section is subdivided in yearly archives which are presented as a sort of pagination interface.
  4. The Sportschool (“Over Aihato”) section sports a simple “submenu” in the left column. This is in fact a separate menu defined in the theme and managed using the new menu editor.
  5. Finally, the guestbook uses WordPress’ default comment pagination.

Conclusion

This turned out to be a pretty long post taking a ridiculous amount of time to write. But, hey, this way I have at least documented the project. I don’t think that such detailed documentation would have happened otherwise. In my experience, “in-house” documentation sucks donkey ass. It’s never complete. It’s never up-to-date and – worst of all – it doesn’t invite comments. It’s just not part of big WWW.

I’m glad that the new website is on-line. I love how it turned out (even though I still hate web development). The enthusiastic reception of this project even compensates for some of my previous web development traumas. 😉 I find myself quite enjoying the after-work because of the laid-back attitude of the guys. What’s worse: I’m actually looking forward to implementing some of the planned improvements. That’s strange. Maybe it’s the complete lack of hysterics about the shape of a particular icon (“I want the trash can back!”) or the phrasing of a particular sentence (“How could this have happened?! You should have quadruple-checked this first! Aaarggh! Now our company will die because we look unprofessional!”). Some people are just more fun to work forwith than other people I guess.

MediaWiki thumb.php and rewrite rules

May, last year, I created an empty draft for this post, because, around that time, I had gone through quite some effort before I got thumbnails for foreign file repos working just right. Now, I’m taking a dive into my MediaWiki working dirs in preparation of the creation of a separate development environment, so it’s a good moment to rehash the past experience (almost as good as when I’d have done it right away).

This is how I configured the foreign file repo to be able to use images uploaded to the English wiki from the Dutch wiki:

$wgHashedUploadDirectory = false;
 
$wgForeignFileRepos[] = array(
    'class' => 'ForeignDBRepo',
    'name' => 'en',
    'url' => "http://wiki.hardwood-investments.net/media",
    'hashLevels' => 0,
    //'thumbScriptUrl' => "http://wiki.hardwood-investments.net/thumb.php",
    'transformVia404' => true,//!$wgGenerateThumbnailOnParse,
    'dbType' => $wgDBtype,
    'dbServer' => $wgDBserver,
    'dbUser' => $wgDBuser,
    'dbPassword' => $wgDBpassword,
    'dbName' => 'hardwood',
    'tablePrefix' => 'mw_',
    'hasSharedCache' => false,
    'descBaseUrl' => 'http://wiki.hardwood-investments.net/Image:',
    'fetchDescription' => false
);

To make thumbnails be generated by thumb.php on request I added the following to my .htaccess at the other end (and visa versa, because the Dutch wiki actually contains most of the images):

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^media/thumb/([^/]+)/([0-9]+)px-.*$ /thumb.php?f=$1&width=$2 [L,QSA]

www.stichting-ecosafe.org

Stichting EcoSafe is a Dutch foundation for the safe-keeping of the funds that are necessary for the maintenance of hardwood plantations. In July of 2006, together with Johan Ockels, I created a website for the Foundation. Johan was responsible for the organization of the whole process. This went very smooth and the website ended up being an emblem of simplicity and clarity. That’s why I wanted to blog a bit about it now, even though there are a few things that I’d probably end up doing different if I were to start from scratch. [There’s actually a disturbing number of things for which this is true, I’m coming to notice.]

File structure

Like with most websites, I started with creating an SVN repo so that I wouldn’t have to be afraid of ever losing anything.

The file structure was pretty standard:

  • a css dir for stylesheets;
  • img for images;
  • inc for shared PHP and mod_include stuff and for AJAX partials;
  • jot for to-do’s and other notes;
  • and js for JavaScript files and libraries.

Possible file structure improvements

If I were to redesign this file structure, I’d collapse css, img and js into one directory called layout, because these are typically things that require the same robots.txt and caching policy. Also, it is meaningless to organize things by file extension. If you want to sort something by file extension, use ls -X (or ls --sort=extension if you’re on GNU).

Server-side includes

The site would be so simple that I felt that any type of CMS or content transformation would be completely unnecessary. Instead, I decided to rely on Apache’s mod_include and just use a few partials for repeating page elements such as the left sidebar containing the logo and the menu.

Also, because I didn’t need to transform the HTML files, I decided I could use good ol’ HTML 4 instead of XHTML 1 (which I’d have to send to the browser with the wrong mime-type anyway).

This is the HTML for contact.nl.shtml:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/loose.dtd">
 
<html lang="en">
  <head>
    <title>Contact EcoSafe</title>
 
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
 
    <link rel="stylesheet" type="text/css" href="/css/style.css"></link>
  </head>
 
  <body>
    <!--#include virtual="/inc/left-side.en.html"-->
 
    <!--#include virtual="/inc/alt-lang.phtml"-->
 
    <div id="content">
      <h1>Contact</h1>
 
      <p>Your email to EcoSafe kan be sent to the following address:
      <a href="mailto:service@stichting-ecosafe.org">service@stichting-ecosafe.org</a>.
      Or, alternatively, you can fax us at +31 50 - 309 66 58.</p>
 
      <h2>About this website</h2>
 
      <p>For comments and/or suggestions concerning this website,
      you can direct an email message at:
      <a href="mailto:webmaster@stichting-ecosafe.org">webmaster@stichting-ecosafe.org</a>.</p>
    </div>
  </body>
</html>
Alternative language selection

Alternative language selection

I use <!--#include virtual--> to include the repeating parts. <!--#include virtual--> has several advantages over <!--#include file--> in that it allows for content-negotiation, execution of dynamic content etc., but here the only place were it holds an advantage is in the inclusion of /inc/alt-lang.phtml. alt-lang.phtml is a messy PHP script that figures out which language variants of a page are available and displays a selection of alternative language versions (variants with a language different from the current).

SSI and caching

Without the XBitHack directive set to full, all content handled by mod_include is sent without a Last-Modified header. However, I don’t want to use XBitHack at all, because I don’t want just any executable file to be handled by mod_include; that just too much of a … hack.

If I were to do something similar now, I’d use some kind of (sed) substitution to pre-process the includes locally so that more of what I end up uploading is simple static content. The dynamic part of the included PHP script, I would simply replace with JavaScript.

Visual design

As you can see in the HTML example, there’s hardly anything layout oriented in the HTML source. This is good, and means that I have to touch only the CSS for most minor and major lay-out modifications. (It is a pipe-dream to think that you only need to change the CSS to make the same HTML page look however you want as long as that HTML is rich enough in meaning, but for a site with pages of such simple structure, it’s a dream that comes pretty close to reality.)

I’m not much of a designer, but I think design is overrated anyway. Actually, I think that most website suffer from too much design.

The EcoSafe logo

The EcoSafe logo

To start the design, I got a logo made by Huite Zijlstra. Because the logo was pretty big and didn’t look good scaled down, I decided to put it at the left of the content area instead of at the top. This would still leave enough room for the menu (which actually takes less space horizontally than the logo).

Colors

For the color scheme, I just picked a few colors from the logo. As always, the base of the scheme would be black text on a white background for maximum readability. The print version hardly uses any colors.

@media screen {
body            {:;  }
*               {:;             }
a:link          {: #585;              }
h1              {: #880;              }
h2              {: #888;              }
strong          {: #a62;              }
#menu li a      {: #660;              }
}

Underlines

I wanted an underline below the level 1 and 2 headings. Because I didn’t like the effect of text-decoration:underline (too thick for <h2>s, too dark for <h1>s and different from browser to browser) and because border-bottom was set too far from the text, I made two simple PNG images that I could repeat-x along the bottom edge.

@media screen {
h1 {:('/img/h1-border-bottom.png'); }
h2 {:('/img/hx-border-bottom.png'); }
}

The menu is very simple. The markup is part of inc/left-side.en.html for the English version and inc/left-side.nl.html for the Dutch version:

cat inc/left-side.en.html
<div id="left" lang="en">
  <a class="logo" href="/index.en"><img class="logo" alt="[Logo]" src="/img/logo.jpg"></img></a>
 
  <ul id="menu" class="menu">
    <li><a href="/index.en" rel="start">Start page</a></li>
    <li><a href="/plantations.en">For plantations</a></li>
    <li><a href="/investors.en">For investors</a></li>
    <li><a href="/history.en">History</a></li>
    <!--<li><a href="/goals">Goals</a></li>-->
    <li><a href="/methods.en">How it works</a></li>
    <li><a href="/cost-structure.en">Cost structure</a></li>
    <li><a href="/cost-calculator.en">Cost calculator</a></li>
    <!--<li><a href="/clients.en">Clients</a></li>-->
    <li><a href="/contact.en">Contact</a></li>
  </ul>
</div>
 
<script type="text/javascript" src="/js/menu.js"></script>
The EcoSafe menu (in English)

The EcoSafe menu (in English)

As is customary, I started by removing all the default list styles and made the anchors behave as block-level elements. I used the big O from the logo for bullets in the list (using background-image instead of list-style-image because the latter gives unpredictable cross-browser results and doesn’t make the bullet clickable).

#menu {
 : 2em;
 : 2em;
 :;
 : 0;
}
 
#menu li {
 : 0;
}
 
#menu li a {
 :;
 :('/img/o-21x16.png');
 :;
 :;
 : 30px;
 :;
 :;
 :;
 : #660;
}
 
#menu li a:hover,
#menu li.active a {
 :('/img/oSafe-21x16.png');
}
 
#menu a:hover {
 : #787800;
}

JavaScript menu item activation

To add the active class to the currently active list item (<li>), I used a client-side solution using JavaScript. After all, it’s proper use of JavaScript to enhance your user interface with it (as long as, as many would say, it isn’t required for the UI to function (as it is in the Cost Calculator)).

// menu.js
 
var menu = document.getElementById('menu');
var anchors = menu.getElementsByTagName('a');
var locationHref = window.location.pathname.toString();
  
for (i = anchors.length - 1; i >= 0; i--) {
  a = anchors[i];
  aHref = a.href;
    
  // Does this menu item link to the current page?
  // We find out by looking if the window location contains the URL in the anchor
  // or the other way arround. The reason to look at both is content-negotiation.
  // It's also true if the location is just '/' and we're looking at the anchor of
  // the 'start' page.
  if ( (locationHref === '/' && a.rel === 'start') ||
       (locationHref !== '/' && ( locationHref.indexOf(aHref) !== -1 ||
                                  aHref.indexOf(locationHref) !== -1 ) ) ) {
    a.parentNode.className = 'active';
    break;
  }
}

I actually just fixed a long-standing bug that was caused by me not being able to fully rely on HTTP language negotiation for the selection of the appropriate language variant, which made me change all links from being language-neutral to including the language in the link target (e.g.: http:///history became http:///history.en and http:///history.nl), the problem with this being that, instead of being able to link to link to http:/// (http://www.stichting-ecosafe.org/), I had to link to http:///index.en or http:///index.nl, making it more difficult to detect the active anchor if we’re requesting the home page through http:/// instead of on of its language-specific URLs.

The JavaScript rested on the assumption that by reverse iterating through all the anchors in the menu and thus processing the link to http:/// as last, I’d know that I had struck the home page and wouldn’t need to worry that any of the links contain a slash. (I don’t know if I intended it to work this way, but it sure seems to me now that the only way this could ever have worked was as an apparent side-effect of the looping order; the SVN logs seem to agree.)

I could have solved this by redirecting all requests for http:/// to the appropriate variant. Maybe I should have (to avoid duplicate content). Instead I chose to add a rel="start" attribute to the links to the home page, as can be deduced from the JavaScript above. (To resolve the duplicate content issue, I could also add a canonical link to the header of the two language variants.)

Anyway, all this brings me to the messy subject of content negotiation.

Content and language negotiation

The EcoSafe website would be bi-lingual (English and Dutch) from the onset. Initially, I wanted to use language negotiation to the extend of having completely language-neutral URLs. For example: http:///cost-calculator instead of http:///cost-calculator.en and http:///cost-calculator.nl. In the end, you can make this work properly in the browser with the help of a cookie, but it’s still a pipe-dream because nothing else will work if you do not also offer another navigational path to the different variants. Maybe, we’ll revisit this topic for a later experiment.

Content-type negotiation is almost effortless with Apache thanks to mod_negotiation. If, like me, you despise to have .html, .htm, .xhtml, .phtml, .pxhtml. .sxhtml, .php, .xml in your URL (I actually used all of these at some time or other), you only have to make sure that MultiViews is in your options.

I’ve configured SSI by means of the following instead of a “magic mime-type”:

AddType         text/html       .shtml
AddHandler      server-parsed   .shtml
AddCharset      UTF-8           .shtml
AddOutputFilter Includes        .shtml

For PHP I couldn’t do the same because my web host was still at Apache 1.3. Otherwise, the following should have worked equally well for PHP:

# This doesn't work with Apache 1.3
AddType        text/html       .phtml
AddHandler     php-script      .phtml
AddCharset     UTF-8           .phtml

Configuring language priority is easy with Apache:

Integrating PHP and SSI

The integration of PHP with all the weirdness that I had configured and created around SSI took some figuring out. Luckily, PHP offers a virtual() function that works roughly the same as mod_include's <!--#include virtual-->. Here’s an example:

<body>
  <?php virtual('/inc/left-side.en.html'); ?>
  <?php $uri = '/cost-calculator.en.phtml'; include('inc/alt-lang.phtml'); ?>

In retrospect, it’s pretty much bullshit to use it. I could have just as well require()d the partials (which I actually did for the alternate language selection), but I probably started out using virtual on a more generic URL without language and content-type selection in it.

406 handling

Because I deployed on Apache 1.3 and the ForceLanguagePriority directive was only introduced with Apache 2.0.30, I had to write an ugly hack to avoid visitors getting 406 errors. To that end, I added a 406 handler to my .htaccess file:

LanguagePriority en nl
ForceLanguagePriority Prefer Fallback # This doesn't work with 1.3
 
ErrorDocument 406 /error-406.php # Luckily, this does 

error-406.php is a PHP file that figures out the available variants based on $_SERVER['REQUEST_URI']. Then, it simply picks the first one (which works because, accidentally, that’s the one I’ve given priority using the LanguagePriority directive as well), outputs a 200 OK header instead of the 406, and virtual()s the file of the variant. The code looks somewhat like this:

<?php
chdir($_SERVER['DOCUMENT_ROOT']);
$filenames = glob(basename($_SERVER['REQUEST_URI']) . ".*");
 
$filename = $filenames[0];
 
apache_setenv('DOCUMENT_URI', "/$filename");
 
header('HTTP/1.1 200 OK');
virtual("$filename");
EcoSafe Cost Calculator

EcoSafe Cost Calculator

EcoSafe Cost Calculator results

EcoSafe Cost Calculator results

The Cost Calculator

The EcoSafe Cost Calculator is some of the least neatly contained and most procedurally oriented PHP code I’ve ever produced while knowing full well what I was doing. It does almost everything it does in global scope. Yet, it does it well.

The thing is designed as a dynamic web page rather than a web application. What I mean by this is that it’s simply two pages (one for English and one for Dutch) using PHP among a number of pages using SSI. In an application, it’s usual to have just one ‘view’ that is the same for all languages, but here I chose to put the different language versions in different language pages and then include everything reusable (and language-neutral) from within these files.

Most of the actual processing and calculating is done in inc/costs-functions.php. (The part about gotos is a joke. (Labeled blocks would have been quite sufficient. 😉 ))

<?php # costs-functions.php - Stuff that's includes by cost-calculator.{nl,en}.phtml
/**
 * Just remember that this code was never meant to be general purpose or anything.
 * So, relaxeeee and keep your OO-axe burried where it belongs.
 * Oh, if only PHP would support GOTO's ... Sigh ...
 */

The rest of the file is just a whole lot of processing of form data and turning it into something that can be easily traversed for display to the user. There are even the function calls without arguments doing all their work on globals. These are actually only added to make it clearer em a piece of code is doing. And—I must say—after a few years it’s still remarkably clear to me what each part of the code is doing. There’s no deep, confusing nesting structures or anything. There’s just a whole lot of very simple code.

Some simple AHAH increases form interactivity

Users of the calculator can add any number of plantings and locations. When the user decides to add a planting or a location, the onClick event triggers the execution of addExtraPlanting() or addExtraLocation(). Here’s how addExtraPlanting() looks:

function addExtraPlanting() {
  lang = document.documentElement.lang;
 
  new Ajax.Updater(
    'plantings', '/inc/planting.' + lang, {
      method: 'get',
      insertion: Insertion.Bottom
    }
  );
}

Ajax.Updater comes from the Prototype JavaScript framework.

Here’s what inc/planting.en.phtml looks like. The same file is also included in a loop to rebuild the form’s state after submitting.

<li>
  <input name="num_hectares[]" type="text" size="5" value="<?php echo $num_hectares ?>" />
 
  hectares have been planted in
 
  <select name="plant_years[]"><?php require('planting_options.php') ?></select>
 
  (<a title="Remove this planting" href="#" onclick="removePlanting(this); return false;">x</a>)
</li>

I think that I’ve gone into small enough detail by now to get to the conclusion. Also showing the contents of planting_options.php would be pushing it. Ah, well…

<?php
 
if ( !isset($this_year) ) $this_year = intval(date('Y'));
if ( !isset($plant_year) ) $plant_year = $this_year;
 
for ($i = $this_year; $i >= $this_year - 20; $i--)
  echo "<option" . ($i == $plant_year ? " selected='1'" : "") . ">$i</option>\n";

(Yesterday, I couldn’t resist the temptation of turning this into a simple file to require() instead of the function definition it was. I think it’s funny to refactor something to remove encapsulation.)

Conclusion

As is usual when looking at old code, I see many things that I’d do (even just a little) different today, but I saw a surprising number of solutions that I actually still like now that I see them back after three years. Removing some of the remaining warts probably won’t do much good besides the masturbatory satisfaction it could give me. (It’s likely that the website won’t live much longer, making such extra attention very undeserved.) But, nevertheless, I’ve enjoyed blogging about it now to recoup the whole experience and to at least look at what I’d do different now and what I learned in the meantime.

Some links

PHP fgetcsv() behavior on empty lines

The PHP documentation for fgetcsv() states that A blank line in a CSV file will be returned as an array comprising a single null field, and will not be treated as an error. Here’s a quick demonstration of this behavior.

fgetcsv.php:

<?php
 
while ($fields = fgetcsv(STDIN, 0, ';'))
  print_r($fields);
 
exit(0);

Execute the script and feed it some CSV with empty lines:

php -q fgetcsv.php
"Veld 1";"Veld 2";"Veld 3";;"Veld 5"
 
"Field 1";;"Field 3";"Field 4";
;;;;
;"Campo 2";;;"Campo 5"

After pressing Ctrl+D, I’m presented with the following output:

Array
(
    [0] => Veld 1
    [1] => Veld 2
    [2] => Veld 3
    [3] => 
    [4] => Veld 5
)
Array
(
    [0] => 
)
Array
(
    [0] => Field 1
    [1] => 
    [2] => Field 3
    [3] => Field 4
    [4] => 
)
Array
(
    [0] => 
    [1] => 
    [2] => 
    [3] => 
    [4] => 
)
Array
(
    [0] => 
    [1] => Campo 2
    [2] => 
    [3] => 
    [4] => Campo 5
)
Array
(
    [0] => 
)

This behaviour on empty lines is a little bit annoying if you want to test if the line is empty():

$a = array(null);
print_r($a);
 
if ( empty($a) )
  echo '$a is empty';
else
  echo '$a is not empty';
 
echo "\n";

This code will print:

Array
(
    [0] => 
)
$a is not empty

Hence, the following function:

/**
 * This function tests if the given array (as returned by fgetcsv())
 * is the result of an empty line in the CSV file.
 *
 * It does not work for lines that contain only delimiters.
 * From the POV of this function, these are simply records with
 * many empty fields.
 */
function fgetcsv_empty_line($row_array) {
  return ( !isset($row_array[1]) and empty($row_array[0]) );
}

Now, if I change the call to empty() in my test to a call to fgetcsv_empty_line():

$a is empty
« Older posts

© 2022 BigSmoke

Theme by Anders NorenUp ↑