Smokes your problems, coughs fresh air.

Tag: CSS (Page 2 of 2)

Jeroen Dekker (photography) on-line

Jeroen Dekker, a friend and photographer, has recently, on May the 5th, put his website on concert photography on-line. (Go check it out! He has some great pictures there.)

I was very flattered when I was asked by Jeroen to give some SEO advice in the test stage of his website. I was even happier when I saw how well he had implemented my suggestions. In his concert photography section, he now has links consisting of the event name and the band name and the number of the photo. An example URL: http://jeroen-dekker.com/concerts/noordschok-2007/prey-band/1/. Also his page titles follow the same structure. As is often the case with SEO, the best results are acquired by remembering that good URLs are URLs which are cool enough that you won’t want to change the in the future and that good titles are titles which look good anywhere, be it in a bookmark or a search result.

Jeroen Dekker concert photography Jeroen Dekker news

I also noticed that, following some evangelizing on semantics and CSS from me, he had greatly cleaned up the HTML markup. Some pages could still profit from some bettermore pedantic markup though. An example from the news section (cleaned up for readability):

<p> The following bands played:<br>
 - <a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/eluveitie-band/">Eluveitie</a><br>
 - <a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/thy-majestie-band/">Thy  Majestie</a><br>
 - <a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/drottnar-band/">Drottnar</a><br>
 
 - <a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/whispering-gallery-band/">Whispering Gallery</a><br>
</p>

In my opinion, the above is a very awkward way to define what is really an unordered list:

<p>The following bands played:</p>
<ul>
  <li><a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/eluveitie-band/">Eluveitie</a></li>
  <li><a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/thy-majestie-band/">Thy  Majestie</a></li>
  <li><a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/drottnar-band/">Drottnar</a></li>
  <li><a href="http://jeroen-dekker.com/concerts/fear-dark-festival-hedon-12-mei-2007/whispering-gallery-band/">Whispering Gallery</a></li>
</ul>

Finally, a nice touch that I noticed on his site is that he doesn’t have explicit pagination. By this I mean that clicking on the page 2 link simply takes you to the first photo on that page, so that he needs only an URL for each photo and not an URL for each page or even photoset.

Web scraping in Ruby: why I had to use scrAPI instead of WWW::Mechanize and Hpricot

Thursday evening: so, I had written myself a nice little script using Aaron Patterson’s WWW::Mechanize and why’s Hpricot to extract some data from a popular web-based airport directory.

Hpricot logo

I was warmed up for Hpricot by the promise of XPath and CSS selector support (and a very cool logo, of course). As a long time XPath user, I started banging out some crispy XPath expressions until I realized that XPath support was only very partial. I kept on trying expressions that would work, even bowing down to expressions that, according to the Wiki, would work, but differently. Come on guys, either support a standard or just plainly ignore it, please! 😡 Because I couldn’t figure out how I’d have to integrate why’s fork of the XPath spec in my expressions, I decided to stick with why’s fork of the CSS selectors instead.

Then, it became time to execute my code. I had estimated that it would take about two hours to finish downloading and parsing the approximately 10.000 pages which contained the data in which I was interested. So, I executed my script, detached my screen session and went to bed, trusting that I would find a nice, handy CSV file in the morning.

Friday morning, I was disappointed to find that my script had been killed. I was left wondering what could have killed the script. I decided to restart the script at the countries starting with the letter b (it had died somewhere halfway the list of countries starting with a b). Soon the script was happily appending data again to the existing CSV file.

Disclaimer: why is a much more prolific Ruby coder than I’ll ever be, so please take my comments with a grain of salt. No, actually, rather take them with a few spoonfuls of salt.

Later, I talked about the spontaneous death of the script with Wiebe. Curious, he looked at the memory usage of my script and saw that it was happily munching away hundreds of megs of memory on our server. And memory usage was growing! With crucial server processes at the risk of running out of memory and with me having to build a circumference around the vegetable garden to protect it from a bunch of brawling chickens, Wiebe was friendly enough to drop in and take a look at my spaghetti code to see if he could fix the leak. He couldn’t, because the leak didn’t appear to be in my code. I wasn’t the first to be bugged by a leak in Hpricot.

That news didn’t make me very happy, because it implied I had to redo the script using different tools. I knew that WWW::Mechanize had been inspired by the Perl package by the same name, so I started by looking at that. After installing WWW::Mechanize, I explored CPAN’s WWW namespace a bit further and noticed that the Perl crowd also had two other good scrapers at their fingertips: WWW::Extractor and WWW::Scraper. Once again I was reminded that Perl, despite its funky syntax, is still the king of all scripting languages when it comes to the availability of quality modules. 🙁 After a few deep breaths, I set my rusty Perl skill into (slow)motion. Hell, this was supposed to be a quick script. Why was this taking so much time? (Yeah, yeah; cue all the jokes about developer incompetence. 😕 )

I was almost stamped by a horde of camels, each with a name more syntactically confusing than the other. Just before I was crushed, I came across a reference to a Ruby scraper with decent support for CSS3 selectors: scrAPI. Credits for this discovery go to the documentors of scRUBYt, a featurefull scraper layered on top of WWW::Mechanize. The documentation writers of scRUBYt where friendly enough to help their users by including a link to the competition.

It took me some time to rewrite the script using scrAPI, partially because it was hard to find any documentation that was more comprehensive than a few blog posts and a cheat sheet and less of a hassle than reading the source. But, when Assaf answered my need by pointing me to the online API docs, I was happy.

Another reason why it was hard to migrate from WWW::Mechanize/Hpricot to scrAPI was that Hpricot starts element offsets for XPath predicates and CSS selectors at zero instead of one where they should start. And of course, I had to rid myself of the weird breed between CSS and XPath selectors.

I was surprised that the script using scrAPI ran about twice as fast as the Hpricot-based script. This was including a cumulative sleep() time between each request of almost an hour, because the speed during testing made me worry about over-exerting their web server. Knowing that one of the popular features of Hpricot is its speed, this was very unexpected, although I have to admit that Hpricot did fill my memory very quickly.

Using the standard HTML link colors

Occasionally, I have to foray into web design. From the look of this blog you can deduce that I’m not really into the design part of web design. First and foremostly this is because I suck at graphics and colors. But another reason is that I assume that people visit my blog to read some text and not to look at the fancy graphical borders and background of that text.

Recently I’ve gotten into the habit of simply using the default HTML link colors instead of trying to come up with a comprehensible color scheme for each design. This means that my links are blue, my visited links are purple and my active links are red as in the example CSS snippet below.

a:link {
 :;     /* #00f */
}
a:visited {
 :;   /* #800080 */
}
a:active {
 :;      /* #f00 */
}

Related links

Newer posts »

© 2022 BigSmoke

Theme by Anders NorenUp ↑