Main

August 5, 2007

UWA Directory OpenSearch plugin

I have written an OpenSearch plugin to simplify searching the people directory of The University of Western Australia. It works in most modern browsers (not six-year-old Internet Explorer 6).



Technical Notes

If you follow the instructions at the Mozilla Developer Center then you may not realise that the plugin can be installed with Javascript (such as with the button on this page) rather than autodiscovery. This is useful for a couple of reasons:
  • Autodiscovery (as demonstrated for the UWA Directory) relies on a link in the <head> element of the web page, which is inaccessible in many blog and content management systems. In those situations, people place the link within the body of the document. This may work, depending on browser, but is certainly invalid markup.
  • It easier for users to add the search engine when it's provided directly, rather than having to provide a whole page of illustrated instructions

The Mozilla instructions also do not indicate any option other than encoding the search icon within the xml file itself, using Data:URI. In fact, a URL to an external image file is perfectly acceptable, such as:
<Image height="16" width="16" type="image/vnd.microsoft.icon" >http://example.com/websearch.ico</Image>.
Whilst it's nice to have the plugin file completely self-contained, Internet Explorer, even up to version 7, does not support Data:URI. However, as far as I can tell, IE does not support OpenSearch image icons at all anyway, so this point is possibly moot.

April 9, 2007

Podcast support added to RealToMp3

I've added some very simple podcast support to my Real audio downloading script, realToMp3. It just downloads the first mp3 content that it finds in the given podcast feed, which works fine for weekly shows where you also have realToMp3.py running weekly too. I thought it'd be nice to have the same script doing the Real stream and the podcast, where the download type is determined automatically. As always, the script is uncompiled Python, so you can read and modify to suit your needs.

Update 10-Apr-2007: I've added an optional second parameter that lets you specify a filename to save to, which is useful if the stream you are downloading doesn't have a very good title to start with (which is what the script usually uses).

January 29, 2007

Microsummaries - the new RSS?

If you're using Firefox 2 then you can join the three of us using its great new Microsummaries feature. A website can provide a microsummary, in which case you can think of it as a short, single-item RSS feed. Indeed, some bloggers have already set them up to provide notification of new posts. However, what's more interesting is that you can get Firefox to parse a page to produce a microsummary itself. Thus, if a website has a snippet of information that you want constantly-updated access to, then you can have it.

I became interested in this topic because I found that the current temperature provided by the ForecastFox extension did not match the official current temperature for my area provided by the Bureau of Meteorology, and I thought about creating a bookmarklet that could screenscrape the temperature from the BOM site. Well, it turns out that microsummaries do exactly that, but with much less effort.

Although the production of a microsummary generator is relatively straightforward, there is some coding involved. Fortunately, there is an even easier way:

  • install the Microsummary Generator Builder Firefox extension
  • visit the BOM web page
  • right-mouse-click and choose Microsummary generator
  • move the mouse around to make a box surround the snippet of interest, then left-mouse-click
  • choose Install Generator
  • bookmark the page, but in the Add Bookmark dialog box, notice that the Name field is now a drop-down list (yes, it's subtle, but it's the only way you can tell if there's a microsummary for that page). From that list you can choose either the normal page title, or the snippet you just chose
  • you'll probably want to put that bookmark on your bookmarks toolbar so that it's easily visible

Installing the generator actually just creates an XML file in your profile's new microsummary-generators folder which describes how to parse the web page in question. If you manually edit that XML file then you have to go to that bookmark's properties, and change from the microsummary to the static title, then back to the microsummary, to get the changes to be noticed. Also, you can happily uninstall the Microsummary Generator Builder once it's done its job of creating that XML file, not that it'll cause much harm where it is.

Note that the Microsummary Generator Builder does not ask you to set the microsummary's update frequency. To configure this, add an update field with an interval in minutes to the xml, such as:

...
</transform>
</template>
<update interval="15" />
</generator>

Finally, note that microsummaries can't be created from pages on FTP sites. Despite every step of the process working, when you go to bookmark the page the microsummary drop-down won't appear.

January 27, 2007

Free off-site backups

Although I always keep a backup of every version of my website, it can be difficult to view previous versions of single pages. For example, to view a stored PHP page I'd need to install it on a web server to view the output, otherwise I'd just see the source code. I'd have to ensure that any external style sheets were in place where expected, and background images restored. All in all, it's quite a complicated process.

Luckily, there's a much simpler alternative. The Wayback Machine, from the Internet Archive has been busily copying websites into their archives for many years. Just enter a URL to see what archives they have for that site. For example, you can see the evolution of my fractals page (note that some images from the early years were not archived, so the page backgrounds were not quite as hideous as they now appear)

Although I built my first web site in 1994, the earliest of my work in the Wayback Machine is the aforementioned 1999 reference.

January 8, 2007

Explore the internet safely

The Washington Post took a look at how long web browser vendors took to fix security flaws in 2006, with the result being another good reason to switch from Internet Explorer to Mozilla Firefox.

October 28, 2006

Importing email into Gmail from Pegasus

It's easy to import your old emails from Pegasus mail into GMail, but it does have a few steps. I've used Mozilla Thunderbird as an intermediary, because it turned out to be good at massaging the data into a mutually acceptable form.

  1. Create new mailbox in Pegasus in Unix mailbox format (instead of the default Pegasus mail format)
  2. Move desired emails into this new mailbox
  3. In Windows Explorer find the name of this new mailbox file. It ends with .mbx. (More details in Mozillazine)
  4. Move the mailbox file into Thunderbird's Local Folders, in your profile directory.
  5. Start Thunderbird and click on this new folder. This is when the file is actually tidied up.
  6. Exit Thunderbird.
  7. Run Google GMail Loader (GML), which processes the mbx file, and sends each email one by one to Gmail.

If you skip the Thunderbird step, then GML complains about the mbx file, though it may still work.

October 25, 2006

Firefox 2 size

To coincide with the release of Firefox 2 I've updated my Firefox size over time chart. This new version has seen a small increase in size, but nothing too concerning.

October 16, 2006

Update to RealToMp3 script

I've just uploaded an updated version of the realToMp3 python script, which converts online Real audio streams to mp3 files. The latest change is to make it work with URLs that have querystrings.

September 22, 2006

Remote editing with Vim

The old-fashioned way to edit remote files locally was through a continual process of:

  1. use FTP program to download file
  2. use local editor program to edit file
  3. use FTP program to upload file
  4. test changes, and repeat from step 1

This was then refined through the use of more advanced FTP software, such as Filezilla.

  1. use FTP program to choose file to edit
  2. local editor is automatically opened by FTP program to edit file
  3. when editor is exited, FTP program takes over to upload changes
  4. test changes, repeat from step 1

The problem with this approach is that you have to exit your editor to upload and test changes, then open it again for further work. Luckily, for those people who use Vim, there is now an easier method:

  • edit in Vim

That's right. You can just specify a remote file to load, and Vim takes care of everything. For example, to start editing a remote file via FTP from the command line:
gvim ftp://username@ftp.servername.net/folder/filename.php

Whenever you do a save, it'll be uploaded via FTP transparently.

This magic is performed by the netrw module, which you can find more about by typing :help pi_netrw.txt into Vim.

I did have to do a bit of fiddling to get this working on Windows (with Vim 7.0), though. As also found by a couple of people in the Vim mail archive, I received a file not found message whenever Vim tried to upload my changes. As suggested by the author of this Vim module, installing a new version of the module fixed the problem. To achieve this:

  • download Vimball. This is an automatic installer for Vim modules. The version included with Vim 7.0 doesn't seem to work on Windows, so we'll get a newer version.
  • install Vimball. The official instructions didn't work for me on Windows, as noted in the previous point, so it can't install itself. Instead, for step 2, I had to unpack the vimball.tar.gz file with a separate archiver, and copy the resultant files into their correct folders.
  • download netrw
  • install netrw, this time following the official instructions. If you have problems, try unpacking the netrw.vba.gz file first.

September 11, 2006

Geocode this!

Google has added geocoding of Australian and New Zealand addresses to their mapping products Google Maps and Google Earth.

This means that you can type a street address into the search box and Google will translate that into latitude and longitude and show you the exact location. You don't even need the full address, as 'WACA, Perth' will find Perth's main cricket ground.

When I last worked with geocoded data five years ago the actual street address was calculated by interpolating from the house numbers at the end of road segments. This did not take into account differing block widths, which is especially important with the recent subdivision craze that has been occurring in Perth and elsewhere. In addition, the varying shapes of blocks were completely unknown, so finding the physical centre of a block was impossible. If you look at the area surrounding the WACA ground, for example, you can see the light grey areas denoting the actual blocks. That is, very precise geocoding is possible, and that's a great advance on just a few years ago.

July 6, 2006

IP Visualiser

Every computer on the internet has an IP address, which is a set of numbers that uniquely identify that computer. For example, google.com is 64.233.167.99. Each of the four number segments goes up to 255 (or 28-1).

The Acorn Electron let the user define characters, such as letters or sprites, using 8 numbers each up to 255, giving an 8x8 pixel square.

As an example, 56, 64, 128, 255, 255, 128, 64, 56 denotes an arrow. The most significant bit of the first number gives the bottom left pixel, the least significant bit gives the top left pixel, and the rest followed similarly. So, 56 = 0 * 128 + 0 * 64 + 1 * 32 + 1 * 16 + 1 * 8 + 0 * 4 + 0 * 2 + 0 * 1, which gives a first column of 00111000.

Combining these two facts I present the IP Visualiser, which displays a given IP address as a 4x8 pixel square. As you might suspect, this is quite limiting in artistic terms. However, there are probably some simple company logos that can be represented faithfully. I wonder if those companies would value "their" IP addresses.

An IPv6 version would give much more scope for creative pictures, as its greater range of numbers would allow for an 8x16 pixel area. I'll extend the IP Visualiser when IPv6 becomes more widespread.

July 5, 2006

Firefox size over time

When Firefox 0.9 was released in 2004 I had a quick look at the size of the downloads of Firefox and its ancestors. Two years on it's time to revisit the subject, and check whether the much dreaded bloat has arrived.

This chart comprises Phoenix, Firebird and Firefox, and where both an exe and a zip were offered the smaller of the two is used.

Win32 Firefox download size (MB)
0.1 8.3
0.2 7.4
0.3 7.0
0.4 7.1
0.5 6.1
0.6 6.7
0.7 6.0
0.8 6.2
0.9 4.7
1.0 4.8
1.0.8 4.7
1.5 5.1
1.5.0.4 5.0
2.0 5.6

In summary, there's no sign of the middle-aged beer belly yet.

Updated 25 Oct 2006: Added Firefox 2

June 21, 2006

Fractal zoom canvas demo

So far my Firefox canvas demos have only used a small subset of the available canvas commands. To learn about some of the other features I have added the zoom capability to my previous fractal offerings.

View the fractal zoom at your leisure.

The zooming was accomplished by displaying sliced images from an offscreen (hidden by CSS) buffer. This is an extension of the concise description from the Mozilla canvas tutorial. I have also tidied up the previous fractal code somewhat, removing hard-coded constants. It still needs a general reorganisation to remove global variables, but that shall wait for another time.

Because Javascript is not the fastest of languages you may see some delay in the drawing of the fractal. After an extended delay Firefox may pop up a message box asking if you wish to continue. If you say 'yes' then the fractal should get drawn eventually. It is this speed issue that has caused me to draw the fractal in lower than maximal resolution. If you want to zoom in to a high-resolution fractal then have a look at a desktop fractal zoomer such as my old Windows software JM's Mandelbrot Explorer.

June 16, 2006

Flame canvas demo

Following on from my successful canvas demos of fractals I thought I'd attempt the old staple of the demo scene, the flame effect.

Luckily it's very common and very simple. There are dozens of examples in Programmers Heaven - Graphics & Sound - Fire & Flame, coded in Java, C, C++, ASM, Pascal and even QBasic. All of those are old hat, as the cool stuff is now being done online, using the canvas tag supported by all modern web browsers. Coding and deployment is much simpler, though the results are usually a tad slower.

That's enough waffling. Here's the fire. Just view the source to see the code.

This time I did the animation using setInterval() rather than setTimeout(), as it's slightly less code. I expected from their definitions that they'd be virtually identical, but setInterval() seemed to run faster but be more CPU intensive, as if the delay value was reduced. As such I increased the delay to compensate.

In addition, I used the canvas scale(x,y) command instead of doing the scaling manually. Again it slightly simplfied the code in this example, but I imagine for more complex programs the benefit would be greater.

June 9, 2006

Let's bookmark bookmarklets

When I click on the URL of an mp3 file in Firefox I want that file to be streamed via Winamp. Instead, Firefox downloads the file in its entirety before passing the file on to Winamp. To get around this I have been copying the URL into the clipboard, opening Winamp, then telling Winamp to play that URL directly. Whilst this works, it is a little inconvenient. A much nicer solution is to use a bookmarklet, such as HubLog's client-side M3U generator. Just bookmark the generator, then when you want to play the mp3s on a webpage just click the bookmark. What magic!

As with all good things on the internet, the code is there for all to see:

javascript:
a=document.getElementsByTagName('a');
s=new Array();
for(i=0;i<a.length;i++){
    if((a[i].href)&&(a[i].href.match(/\.mp3$/))){
        s.push(encodeURIComponent(a[i].href));
    }
}
var newline=encodeURIComponent('\n');
var m3u=s.join(newline);
if(m3u){
    location.href='data:audio/x-mpegurl,'+encodeURIComponent('#EXTM3U\n')+m3u;
} else {
    alert('no links found');}

It works by grabbing all of the links to mp3 files, then uses the lovely (though underused) inline data facility to create an m3u playlist in situ to pass to Winamp. All hail bookmarklets!

March 7, 2006

Weather APIs

I built my Weather Guesser using the Services_Weather PEAR PHP module. Although the module can retrieve weather data from CapeScience's GlobalWeather, EJSE, Weather.com and NOAA, only the last of these was suitable for my simple needs. I'll explain why, and describe the other weather APIs that I have found.

  • NOAA - the USA's National Oceanic and Atmospheric Administration's National Weather Service provides METAR, or METeorological Aerodrome Reports. The reports are freely available, and cover airports throughout the world. The actual data returned is in a cryptic code, like YPPH 071430Z 10003KT CAVOK 22/14 Q1012, but Services_Weather helpfully translates it into something more user-friendly.
  • weather.com - has onerous terms of use, effectively saying that you can use their weather API as long as it is to produce weather pages that look and function identically to the Weather.com site. It even has a term stating that a person's location has to be stored as a permanent user preference unless the user alters it, which makes my application ineligible.
  • AccuWeather - I noticed this because it is being used by the Mozilla Firefox weather extension ForecastFox. However, I couldn't find any details of a free API.
  • GlobalWeather - this service has disappeared, but an answer in their FAQ promises it will return.
  • Yahoo Weather - data is provided by Weather.com, but seems to have much more liberal terms of use than them. The API is really easy to use, requiring just a simple GET, eg. http://xml.weather.yahoo.com/forecastrss?p=USCA1116, and returning some comprehensive XML. Unfortunately, it doesn't provide an API method of determing the ID for a particular location. For the US zip codes can be used, but for the rest of the world you have to determine the code via a manual search on the Yahoo Weather home page. I suppose this could be screen-scraped, but that really defeats the purpose of an API.
  • EJSE - US only, so not investigated.
  • WeatherBug - US only, so not investigated.

February 19, 2006

Embedding data within HTML

After creating the interesting, though perhaps ultimately slightly pointless, online Asciifier recently, I was intrigued to learn of another method of inserting image data directly into an HTML page. The underused data:URI scheme involves a straightforward encoding a small amounts of data. The smallest example I could come up with is this 1x1 white pixel.

<img src='data:image/gif;base64,R0lGODdhAQABAIAAAP///////yw
AAAAAAQABAAACAkQBADs='/>

Once again downtrodden users of the ancient Internet Explorer 6 are left out in the cold, as that browser is not in tune with such modern lingo as this. People viewing this page with less antiquated software will now see a delightful image from the great Henry Fuseli - Silence, 1799-1801.

Despite being around for a number of years, data:URI has not been extensively used. Is this because it's not well-known, or because it's not very useful? I'm afraid it may be the latter, and until Internet Explorer supports it, it will remain a toy for internet nerds alone.

February 9, 2006

Convert RealAudio to MP3

Download realToMp3.py

RealAudio streams are not very easy to convert to mp3 files. There are some shareware tools that claim to perform the task, but with a little bit of effort you can set up a really simple system yourself for free. Here I shall document how to set up a really easy to use converter for Windows.

The end result is this lovely simple dialog for interactive downloading, or you could set up a scheduled task to grab a RealAudio stream and save it as an MP3 with no user input at all.

There is a price to pay for getting all this for free, and that is that there is no simple one-step installer. Instead, we are going to set up a variety of free tools, and tie them all together with some scripting. Here's the menu:

  1. Retrieve stream with MPlayer
  2. Convert to MP3 with LAME
  3. Automate the previous steps using Python
  4. Add a nice interface with Windows Scripting Host

Part 1: Retrieve stream with MPlayer

MPlayer is a highly regarded Open Source audio and video player. Once you've downloaded and installed it you can test that it's working from the good old command prompt:

mplayer -cache 320 -ao pcm:waveheader:file=stream.wav rtsp://some.stream.ra

Note that you need to pass a .ra or .rm stream. If you have a .ram stream instead, then save that as a local file and have a look in it with a text editor such as notepad to see the actual stream name.

If this does not work then take careful note of the error messages being emitted. This was the point at which I learnt that I needed to download some RealAudio CODECS from the MPlayer site, which were then unzipped into MPlayer's CODEC's folder.

Once you've got this going you'll see the one unavoidable annoyance, that you can't download any faster than the normal stream listening time. Anyway, the result of this is that you have converted a RealAudio stream to a probably rather large .wav file. Now we need to shrink that to a reasonably-sized MP3 file.

Part 2: Convert to MP3 with LAME

LAME is a popular Open Source MP3 encoder. We need the standard windows LAME binaries, which aren't provide by the LAME site, but a quick Google search will provide places such as this. Install it, and test from the command line again:

lame -b 64 stream.wav stream.mp3

That's the hard stuff done. If you don't mind going to the command line and doing some manual labour each time you want to save a stream then you can stop here. However, if you want to make your life easier in future, then you may wish to script those commands.

Part 3: Join parts 1 and 2 with Python

Jkx wrote a little Python script in 2004 to automate the MPlayer and LAME processes, and to parse the stream from the .ram file. Due to bit rot this no longer works, so I made a few small fixes and minor improvements to create realToMp3.py.

To run this you'll need the Open Source scripting language Python installed. Unlike the previous two packages this comes with a nice easy installer. Note that the locations of MPlayer and LAME are hard-coded in the script, so you'll need to edit these. You can use any text editor, but make sure you only ever use spaces, not tabs, or Python will complain. You can also edit the folder in which the resultant mp3s are saved.

Again you can test this from the command line:

realToMp3.py http://location.of.stream.ram

Part 4: Add GUI with Windows Scripting Host

Finally, if you want a nice icon on your desktop, and an easy interface to use your python script, then you can use this very short Windows Script (the successor to batch files), realToMp3.vbs. [Aside: Visual Basic Script (.vbs) may be yucky, but it has the crucial InputBox() command that JScript (.js) lacks.] If it's in the same folder as the python script then it should work without alteration. Then just create a shortcut to that script on your desktop, and set its icon to something nice.

Conclusion

On reflection it's a little odd to use two scripting languages to accomplish this task. However, trying to do parts 3 and 4 in the same language presents some interesting difficulties. I leave this as homework for the reader. In any case, it's extremely satisfying setting up this system instead of paying US$20 for some easier-to-install but harder-to-configure all-in-one software.

Updated 25-Feb-2006 Python script: Destination folder now works properly.

Updated 15-Oct-2006 Python script: Now works for URLs with querystrings.

Updated 9-Apr-2007 Python script: Added podcast support (read more).

January 25, 2006

Google keyboard shortcuts cut

There's always lots of fuss when Google introduces a new service or feature, but sometimes not much fanfare when one leaves. After using GMail's keyboard shortcuts for a while I was reminded of the old keyboard shortcuts that Google had for their main search engine. It had a bit of coverage when it was released:

These nifty new controls allow you to navigate Google results using computer keys. You activate the function by pressing any key, which displays a small graphic of three colorful balls next to the first result. Move the balls around the page with the U (up), J (down), H (left) or K (right) keys and press enter to view the result.

There are several shortcuts, including C (view cached page), S (view similar page) and others. Very useful if you don't like using a mouse, or are working on a laptop.

Well, it turns out that it was removed from the Google labs in March 2005. The project's newsgroup is still up, though. Since it's taken me 9 months to notice that it's gone, I guess I can't complain that I'll miss it too much.

January 20, 2006

Firefox popularity increasing

One of the most popular blogs on the internet, Boing Boing, is now seeing noticably more visits from Mozilla Firefox than from Internet Explorer.

W3Schools shows the long term browser trends amongst the more technically literate people who visit their site. It indicates that Firefox use is still increasing sharply.

However, the general internet population still overwhelmingly uses the decrepit, pitifully archaic Internet Explorer that came installed with Windows. The latest version was released 5 years ago, and as stated by the Wikipedia: "A notable portion of the widespread promulgation of spam, spyware, adware, and computer viruses across the Internet is known to be facilitated by exploitable bugs and flaws in the security architecture of Internet Explorer."

There is really no reason for these people to suffer any longer. Firefox is quick to download, easy to install, and then updates itself with no fuss whenever it needs to. I believe it is the duty of all technically knowledgable people to assist their friends and family to browse the web in safety and comfort. It's free, and well worth at least a test run.

If you only care about security, but not the extra features offered by Firefox, then the switch from IE is pretty straightforward. On installation Firefox will offer to bring across your bookmarks, and then you can browse the internet just as you did before. Gradually, you may notice some pleasant new features that bring incremental improvements to your browsing experience.

Tabbed browsing is something that doesn't sound too interesting until you actually use it, and wonder how you ever did without. I find that it speeds up my browsing by allowing me to open new pages whilst remaining on my current page. For example, if I'm on a page of movies reviews, I can click with the middle mouse button (usually the mouse wheel) on links to have them open in another tab. I can then switch to the other tabs at my leisure, and close them when I'm done.

Another aspect of Firefox is its support of extensions, which are user-coded enhancements to the browser. I have added four extensions to my browser:

  • cardgames offers a wide range of games to fill in your spare time. If you've been using Windows Solitaire, then this could be considered a massive upgrade.
  • netusage. In Australia internet accounts have restrictive quotas. This tool displays a graphical bar of how much of this month's quota you have used.
  • bugmenot lets you access free newspaper sites that require you to register your details, but without having to enter you own details. If you visit such a site with this extension installed, then a suitable name and password will be prefilled for you.
  • Bookmarks Synchronizer lets you keep your bookmarks list on multiple computers synchronised. For example, if you have a main computer that you usually use, you could set the Bookmarks Synchronizer to automatically update its bookmarks list to a remote FTP site whenever the bookmarks are changed. Then, on a secondary computer with the Bookmarks Synchronizer extension installed, you could request that the bookmarks be updated from that remote site. Note: It's quite hard to find the latest version of this extension. A current version is apparently here.
A bonus for programmers is the code for existing extensions can easily be looked at and modified, as the .xpi file that they are installed from is just a renamed .zip file. So, you can unzip an extensions of interest and see how it works.

If you're a web developer then you've probably already moved to Firefox because of its great DOM Inspector and Javascript console, the latter inherited from Netscape. When I needed to debug my recently released AJAX application Instant Crossword Tools, the Javascript console proved invaluable.

If you already use Firefox and you've read this far, then you get the reward of a single titbit: middle-clicking on a tab title will close that tab. There, that should save you some mouse movements.

January 15, 2006

The capitalist in all of us

At the start of this year Neil Turner was earning US $150 per month from the Google advertisements on his site. That was part of the inspiration for me to try out Google Adsense on some parts of my sites.

I made my first web page in 1994, when Netscape 1 was released, but I've never earnt anything from my web properties. As I recently moved to paid web hosting for the first time, I thought it'd be nice if I could get ads to cover that monthly cost. To my surprise, the money I garnered in one day will cover the cost of hosting for two months. I guess now I'll have to set my goals higher.

I'll reevaluate the situation in a month or so, to consider whether the money I make is compensating for the loss of the pristine wholesomeness of my site. I wonder if in the meantime I'll start to emulate the amazing money-making machine Darren the ProBlogger.

December 5, 2005

Blogspot experiment

I am testing out Blogger with The Martian Wombat blog. Since it has 300MB of space to put pictures, I will mostly use it for that purpose, especially as my current host has never had enough space to include images.

November 21, 2005

Google conquest

Google have diversified from being just a search engine to now offer many services. Of their products that perform functions that I am interested in, these are the ones that I have switched to, and why I switched. They are roughly in order of when they were released by Google.

Have changed
ToFromWhy
GoogleAltavistaBetter results.
Google GroupsnnEasier to access than having to telnet to a unix box, though slower in use because nn used keyboard shortcuts for everything. Easier to find new groups.
Google NewsBBC NewsCustomisable to stories of interest to me. More news services to choose from.
PicasaThe GimpGreat user interface and digital-photo specific features make it much more suitable than the industrial-strength general purpose Gimp.
GmailPegasus MailThreading makes reading emails easier. I don't have to backup my data.
Google EarthWorldwindBetter user interface, such as not having to choose which data set to use. Google has better data for the places I am interested in.
May change
Google AnalyticsAXS Visitor TrackingDon't have to monitor the size of my AXS log file. Nicer presentation of results. However, Google is slower, and I haven't yet been able to get all the data I can with AXS.
Will not change soon
BloggerMovableTypeMovableType allows me to customise anything, because it's all written in Perl.
Google ReaderSharpreaderSharpreader fits more data on screen, and lets me read faster and easier.

October 14, 2005

Firefox canvas demos

Mozilla Firefox 1.5 introduces the <canvas> drawing tag to play with. Apple's Safari browser already supports it, but until the strangely popular Internet Explorer does too it will have only limited usefulness. Nevertheless, that's no reason not to experiment.

After a brief play, I came up with some fractal examples.

Perhaps with universal browser support this will replace Flash. It's certainly much easier for a competent HTMLer to work with, with a minimal learning curve compared to any of the Flash creation languages.

Update 16 June 2006: Flame demo added.

Update 21 June 2006: Fractal zoom demo added.

September 19, 2005

Laszlo is flash

There are a variety of ways of creating Flash applications. A free, open source solution that has been gaining popularity is OpenLaszlo. It uses Java to compile javascript and XML source files into Flash - enough buzzwords to interest even the most jaded programmer.

After installing Laszlo, the first thing I noticed was how developer-focused the system is. To compile and run your Laszlo source you just need to naviagate to it in a browser, where it appears in a Laszlo frame along with all the essential links to docs and resources that you could need. I used vim to edit the source, then just refreshed the browser whenever I made a change. It was a bit slow to compile even my very small program, but I don't know whether this increases much for larger programs.

The included documentation and tutorials were great, with the exception that a few new features, such as drawing, were only described in the tips section of the Laszlo web site.

For a very few features, such as SOAP, the running Laszlo code must interact with a server-based java servlet. However, mostly you can just do a solo (ie. flash only) deployment. The server is still required for compilation and development.

For my first Laszlo program I created the television sizer. I spent a while putting together horrendous, but working, code. After a few days I got a better idea of how things were meant to work, and easily fixed my earlier efforts. So, while it was a bit of a jump from languages such as PHP, Perl, Python and ASP, it was only a small jump. The use of tags for everything made it feel like Cold Fusion, though much simpler and, as you'd expect, more graphically focused.

It's difficult to imagine it scaling well, but then how big would anyone want a flash application. Overall, it's been well worth learning, and might come in handy one of these days.

September 7, 2005

Firefox view source oddity

For no reason that I can discern, when viewing the following html source in Mozilla Firefox 1.0.6.

<html>
 <body>
  <table>
   <tr>
    <td colspan = "2">hello</td>
   </tr>
  </table>
 </body>
</html>

it gets confused with the space next to the colspan, and displays it as:

<html>
 <body>
  <table>
   <tr>
    <td colspan =" 2">hello</td>
   </tr>
  </table>
 </body>
</html>

which to any perfectionist programmer could be quite an annoyance. Especially if that section of the page is being dynamically generated by a PHP script, and they waste many minutes trying to track a fault in their code when it turns out to be a browser quirk...

Update 12 Sep 2005: This is fixed in Firefox 1.5 beta 1.

July 21, 2005

Lesser-known satellite and aerial imagery

If you've been playing with Google Earth then you might have been disappointed with the resolution of images provided, especially if you're not in urban USA. Although they have by far the best interface, other lesser-known sites may provide greater detail.

For example, in Western Australia, the Government's Department of Land Administration has for many years provided the most detailed aerial photography. You have to use an Internet Explorer-only ActiveX plug-in, but it's worth the effort if you want to see you neighbours sun-bathing (disclaimer: neighbours must be elephants).

I have been looking for a similar service to look at my old residences in the UK, but I haven't found anything yet.

May 26, 2005

WikiTrivia - an automated quiz

A year ago today I stopped work on a web-based trivia quiz that used questions automatically generated from Wikipedia articles. Called WikiQuiz, it let the user choose from a carefully chosen selection of topics, and then would display the content of an article linked to by that topic, but with all references to the article name blanked out. The quiz taker then had to guess what the missing article name was.

The hardest task was to get reasonable articles, ones that were neither impossible nor ridiculously easy. I considered allowing users to rate the fairness of their question, and building up a system whereby the most unfair questions would no longer be asked.

Unfortunately, the WikiQuiz never got to experience life, due to technical restrictions at its live location made just prior to its launch.

Today, exactly one year to the day since I last touched any of the WikiQuiz code, I came across the announcement of a similar venture, WikiTrivia. At this early stage it appears to suffer the main problem that I came across, that it asks quite unfair (often impossible) questions. I hope that it continues to be developed, as I look forward to experiencing the final implementation of the idea I had so long ago, even if it has arrived by an unexpected route.

March 29, 2005

Are MP3 blogs legal?

The antics of the RIAA in attacking some of the millions of people using peer-to-peer software to download music are well known. The Village Voice gives the latest details on their success.

... of the 8,400 suits filed (8,100 of which were filed against John Does) there have been about 1,700 settlements to date.(...)Most settlements top $3,000, and according to Newitz, some can go as high as $7,000.
...
Frequent downloader Cecilia Gonzalez didn't settle against the RIAA, and on January 7, she received only a summary judgment in a U.S. District Court. Throwing out Gonzalez's claims that she was simply "sampling" songs to see if she wished to buy them and that she was an "innocent infringer" unaware that she was violating record company copyrights, the court ordered her to pay damages of $750 for each of 30 songs she was found to have downloaded illegally, for a total of $22,500. That's more than the poverty line income for a family of five in 2004 ($22,030), but it is worth pointing out that damages of $750 per infringement is the minimum the RIAA could have received, and that the original complaint filed against Gonzalez claimed that she had nearly 2,500 downloaded songs.

However, there are some sources of MP3s that are not being targeted by the recording industry - MP3 blogs.

You'd think a site like FluxBlog (fluxblog.org) would have shriveled up and blown away by now. With every daily helping of two or three tightly and insightfully annotated indie-pop MP3s he serves up, author Matthew Perpetua all but formally invites the RIAA to come down on him like a ton of unsold Ashlee Simpson CDs.

According to USA Today:

Even the most popular MP3 blogs have no more than a few thousand visitors per day. Perhaps because of their size, or because they don't tend to offer mainstream pop, they have mostly escaped the Recording Industry Association of America's crackdown on illicit downloading.

The lobbying group for the world's largest record labels, seeking to stop illicit file trading that it says has decimated the music industry, has filed thousands of lawsuits against people who use file-trading networks like Kazaa, but MP3 bloggers haven't been targeted — at least so far.

As the Philadelphia Inquirer says:

MP3 blogs, where fans post beloved niche songs and comments, are safe middle ground in the downloading wars. Some labels even see them as a marketing tool.

The Democrat and Chronicle chimes in with:

Many mp3 blogs are, technically speaking, probably breaking the law, offering up copyright tracks that are not theirs to give away free.

While the Recording Industry Association of America has been notoriously lawsuit-happy in recent years over copyright infringement, mp3 blogs so far ";are still off the radar," says Pavelich. "That's part of the reason I've been reticent to talk to (the media) about it. There is that slight fear.

"I sincerely doubt most of the bands I'm using care this one song is downloadable."

So far she has never heard from RIAA. "And I certainly hope that I don't."

RIAA spokesman Jonathan Lamy said the association "is aware of" mp3 blogs "and as necessary will take appropriate action if we find examples of piracy."

The Guardian adds:

Things do go wrong. When an unfinished version of a recent Coldplay track was leaked to an MP3 blog, the company concerned was really upset, says Sean O'Connell, reviews editor at Rip and Burn magazine. But he adds: "It just got more people talking. I don't think it's done any harm."

He calls MP3 blogs "legally grey", yet many bloggers still consider themselves too small to attract record companies' attention.

"I suspect MP3 blogs are responsi­ble for an almost insignificant percentage of the music being illegally circulated these days," says Morris. "Peer-to-peer networks are obviously a far greater threat as far as the labels are concerned. And the industry seems to be built on the backs of an ever-decreasing number of artists; music blogs are not the place to go if you're an 18-year-old high school kid looking for the latest Eminem record."

That view is only partially supported by the industry. "There's perhaps hundreds of thousands of infringements happening every day on the peer-to-peer network," says a spokesperson for the International Federation of the Phonographic Industry (IFPI), which represents record companies internationally, "so in comparison, one guy who puts an MP3 file on his blog is not going to attract a lot of attention — but it is still illegal," she stresses.

Bloggers who post many tracks without permission, she says, are likely to receive a cease-and-desist letter.

The London News Review notes that:

All the best blogs have been careful: pushing remixes, or tracks they've digitised themselves (love hearing vinyl scratches all over MP3s), or new releases by small bands. Even the flurry of Ray Charles tributes was driven by the hunt: "You think he's just 'Rainy Night in Georgia' and 'I got a Woman'? Listen, and appreciate the scope of the man's genius." You can learn more about Charles (or Robert Quine) by listening than you would by reading broadsheet obits.

The tracks come down after a week or so, and all the blogs are more than happy to comply with removal requests. Most of them have a 'buy this album' link. They're cautious and reasonable, as far as I can see.

Vue Weekly says:

Obviously, all this unauthorized posting and distributing of copyrighted music is bound to have its legal complications. In a cursory attempt to cover themselves, nearly all MP3Js run a disclaimer on their blogs proclaiming that the tracks are for evaluation purposes only and highly urging visitors to purchase the album if you enjoyed the tracks. As well, the files are always rotated and never kept on longer than a week—a move to discourage other sites from linking to the songs. But luckily, since the type of songs these blogs tend to post travel well below the radar of most big record labels, lawsuits have yet to reach the MP3Js. Michaels did once receive a cease and desist order through the server that hosted Said the Gramophone, but was never told which file was the offending one. On the other hand, he has been contacted by numerous artists delighted to find their songs on his weblog.

Meanwhile, Coolfer suggests (referring to an article in BizReport) :

The focus of the article is Fluxblog and its founder Matthew Perpetua. ""Most of the artists that I cover are pretty obscure, and I like to help them get a bit of publicity and grassroots support," he said. That's why the RIAA isn't suing the pants off of most MP3 blogs. Few indie labels are members of the RIAA, and MP3 blogs--at least the ones I check out from time to time--rarely post tracks by RIAA member labels.

So are MP3 blogs legal? Well, the answer is maybe. Some record labels release a few tracks from an album for free download as a teaser for the rest of the album. If an MP3 blog sticks with these free downloads, then it's completely legal. However, often the blogs comment on, and make available for download, tracks that haven't been officially sanctioned for release. Whilst the blog author knows this status, the casual visitor has no way of knowing if the track is fully legal or not. To be sure, they must obsessively visit all offical sites related to the band in question, and try to find the track for download there. Often the tracks will be legal, but not easy to find.

The Morning News had a roundtable discussion with some of the leading MP3 bloggers about this (and other topics):

TMN: Future of mp3 blogs: Are the record companies hounding your doors? Have musicians approached you to get their songs heard? Feel any responsibility, financially, to the artist if you’re giving their songs away for free?

David: I get at least one email a day from an artist who wants me to link to their tunes. I am always thrilled when my weblog can be a conduit between artist and potential fan. Of course, this is why I started my site, and I benefit by discovering great music as well. I’ve heard from several small labels, and am glad to link to their artists (if I enjoy their music). Unfortunately, there hasn’t been any payola sent my way, but most artists are kind enough to offer their CDs, buttons, stickers, and/or band paraphernalia. My favorite thank you was a box of Tastykakes from a band in Philly, my hometown.

So, will the RIAA attack MP3 blogs? Well, logic is no help here. The evidence points to P2P downloads aiding, or at worst having negligible effect on music sales, yet the RIAA is attacking P2P ferociously. So, the fact that (high-quality, well-written) MP3 blogs such as Fluxblog and Scenestars probably encourage music sales is no defence. However, MP3 blogs have been around for some years now, and the record companies have ignored them so far, when it would have been easy for them to force web sites operating illegally to shut down.

In the past, sites hosting huge archives of MP3s, or even just huge archives of links to MP3s have been forced to close, but users of these sites have not been followed up.

MP3 blogs are more like listening to your favourite DJ than warehouses of illegal music. It would be nice for record labels to officially acknowledge their existence, and endorse what they do, but I won't hold my breath waiting.

Updated 14th April - added comment from The Guardian.

March 16, 2005

Interactive name popularity checker

The Baby Name Wizard's NameVoyager is great fun not just for looking at the popularity of specific names, but for whole groups of names. Why were those starting with O so unpopular in the 60s, why has F now dropped off, and why has K become the 'in' initial?

February 22, 2005

Solving cryptic crosswords using the internet

There's a plethora of sites offering to aid the user in solving standard crossword puzzles, but if you've moved onto cryptic crosswords then the pickings are much scarcer. Even after you've read a good guide to the topic, you might still find it difficult to get a toe-hold into the seemingly elitist world of the cryptic setters and solvers.

If you can't find a person to lead you by the hand, explaining puzzles in detail when you ask, then you might find some computer tools of equivalent benefit. Crossword Maestro is Windows software that actually solves cryptic clues, and gives an explanation too. It's been well-reviewed, but it's not free. An online site claiming to solve crypic clues is Crossword Tools, which requires a monthly subscription. However, you came here looking for free online tools to help you, and neither fulfills that role. We have to be a bit less ambitious.

One step down from these cryptic solvers, and so less like cheating, are the many sites that return anagrams and complete words with gaps. A prime example of this is WordFun.

I previously liked the Internet Anagram Server, but by default it returns too many irrelevant answers. To improve this you need to click advanced, and set maximum number of words in each anagram to 1.

Potentially more useful, is Richard's Crossword Tools, an interesting collection of tools, including the quite useful give me a word that means both X and Y.

Finally, a more comprehensive (or at least, more commercial) version of Richard's site is OneLook. For example, if you know it's 6 letters, starting with 'c', meaning barrel maker, then enter c?????:barrel maker. It has a huge word list which includes lots of wacky words, so you'll want to select Common words only.

Naturally, using any of these tools is cheating. Don't expect to win a puzzle championship using your internet-enabled laptop. However, with practice, you should be able to solve puzzles using only what's in your own noggin.

Update 20 Jan 2006: I now heartily recommend my own Instant Crossword Tools as a pleasantly easy way to get assistance with anagrams and word completions.

February 20, 2005

How not to run mailing lists

It's inevitable that people will be annoyed when they receive emails in a mailing list that they haven't subscribed to, but companies still persist in sending them.

Ideally, people should have to opt-in before they become a member of a list. However, some companies talk themselves into believing that their customers would opt-in if they had the choice, because the list is so useful, so they remove the need to opt-in at all. For example, Legacy Family Tree is an excellent genealogy program, but whenever you download a program update (even if you've paid and registered your copy) you are automatically resubscribed to their mailing lists. There's no opt-out option, and it doesn't matter how many times you've unsubscribed in the past, they put you back on their list to receive regular advertisement emails.

When you go to their web page to unsubscribe, you have to select which mailing list to unsubscribe. Since you didn't subscribe in the first place, it's difficult to know which to pick, so I selected all of them. Then, bless 'em, they sent me an error email saying that I wasn't subscribed to one of the lists I tried to unsubscribe from. Maybe they'll resubscribe me automatically to fix that error.

I wonder what the developer was thinking when he created that email? Hmmm, this person tried to unsubscribe because he doesn't want to receive any more emails. But he can't unsubscribe because he's not subscribed to this list, so I'd better send an email to let him know why he's not been unsubscribed to this list...

Moral of this story: use opt-ins for mailing lists. If you can't then allow people to opt-out. If you can't do either, then don't do a mailing list.

October 23, 2004

Is FTP faster than HTTP?

A rule of thumb from my early days on the Internet (circa 1994) that I have believed to this day is that there is a small but noticable difference in the time taken to download via FTP versus HTTP, due to the extra overheads associated with the hypertext protocol. Being unable to find anything definitive on the Googlenet, I decided to do the experiment.

I placed a 10,122,564 byte file on my ISP's web server, and proceeded to download it using NetTransport via first HTTP then FTP. My connection is 256/256 kb/s DSL.

HTTP: 7m 12s
FTP: 7m 15s

(If you naively calculate how long it should take to send that file on that speed link you'll come up with a touch over 5 minutes, if you ignore the overhead associated with encoding data over DSL.)

I'd say that's close enough to be called a dead heat. From now on I'll be using whatever protocol's most convenient.

Addendum (29 Mar 2005): Technically, there shouldn't be much difference. FTP has a slightly more involved initialisation process, but after that they are both just transferring identical bytes.


Some notes regarding speed changes
This is as convenient a place as any to note any changes in my broadband speed.

15 Nov 2004: My ISP changed their ADSL configuration to turn interleaving on, and increased the data rate (to counteract the ATM overheads) to 320,000/320,000 bits per second. The time for this test has been reduced to 6m 25s (HTTP).

2 Feb 2005: My ISP increased my speed (for free) to 5,408,000/928,000 (ie. 5.5Mb/s down,1Mb/s up). The time for this test has been reduced to 0m 19s (HTTP). I guess I'll need to use a larger file if I want to compare FTP and HTTP again.

15 May 2005: After my ISP modified their DSLAM settings my speed was changed to 5,984,000/928,000 (ie. 6Mb/s down,1Mb/s up). The download time is now 0m 16s (HTTP).

August 20, 2004

Firefox hyperlink problem

I know that bug 246078, where Firefox doesn't set itself as the default browser properly, is fixed in the nightlies. However, until a release version is fixed, the way that I workaround the problem is to edit the N/A Hyperlink file type properties such that:

application: D:\NET\MOZILL~2\FIREFOX.EXE -url "%1"
--- DDE stuff ---
DDE message: "%1",,-1,0,,,,
application: Firefox
topic: WWW_OpenURL

If I cure it the way that other people suggest, then I seem to always open a new browser window when clicking a URL from another application, instead of reusing the active one.

July 9, 2004

Creating Firefox extensions

Due to its comparative youth, there is a decided lack of information about writing extensions for Mozilla Firefox. Until it reaches its version 1 release, the specs are expected to remain in flux, and this has probably discouraged a few tutorial writers from creating a soon-to-be-outdated series on Firefox.

However, if you're keen, and given the number of Firefox extensions there are lots of keen people around, there is some information scattered about.

  • Books (free)
  • Articles
    • The most complete Firefox extensions information page I've seen is at Roachfiend
    • The Future of Mozilla Application Development
    • - a tutorial-style article about packaging Firefox extensions.
    • Tutorial: Creating a Mozilla Extension - a comprehensive, but old, tutorial for creating extensions for the Mozilla suite. Some changes necessary for it to work for Firefox are mentioned at the end. Due to its completeness it's worth doing the tutorial, but it's inconvenient that it's not aimed at Firefox.
    • Packaging Firefox extensions - crucial documentation on how extensions must be packaged, from the main developer of Firefox
    • Jed Brown has written a quite comprehensive article on converting extensions to the latest packaging format.
    • XUL Planet has a great XUL reference and some great tutorials, that finish just a bit too soon. This is the place to go to learn about all the available XUL tags.
  • Forums
    • The extensions forum at Mozillazine seems pretty well-stocked with knowledgable people.

It's also highly recommended to pick apart a simple extension to see how it works. Knowing that .xpi and .jar files are just renamed .zip files means they can be easily unzipped. Something like Search Button, by Pike, might be a comfortable place to start.

Once you're actually writing code rather than fussing with getting your environment set up then you should find it plain sailing. Seeing as the code is in Javascript you can use the debugging tool of champions, alert("here I am"), liberally scattered throughout to see what's going on.

The DOM Inspector is also greatly helpful. You can go straight to your code by entering the location of its overlay, such as 'chrome://myextension/content/myextensionOverlay.xul' then hitting 'inspect'. Or check out how the browser is made up at 'chrome://browser/content/browser.xul'.

If your extension does not appear to be there, then force Firefox to rebuild its overlay cache by deleting the overlayinfo folder in the firefox/chrome folder. Then resave the installed-chrome.txt file, and restart the browser. You should do this anyway at least once to build up the confidence that you're not killing anything important.

Actually, that last paragraph really only applies to Firefox 0.8. With Firefox 0.9 extensions are installed in profile folders, and I'm not sure what's going on with overlays there.

June 29, 2004

Zero-day exploits

The descriptions of the recent Internet Explorer and Internet Information Systems security problems in many online stories include the zero-day designation, such as in The Register: Internet Explorer. Quick, call security!.

While some speculate that an IIS zero day was used to own the servers, my guess is that the hosting boxes were not patched against a recent vulnerability (something like MS04-11). I would normally say "Hey, you should have been patched" and gone about my business. But this event is a bit different.

A good definition of this term is given by About.Com in Zero-Day Exploits

The Holy Grail for malicious program and virus writers is the “zero-day exploit”. A zero-day exploit is when the exploit for the vulnerability is created before, or on the same day as the vulnerability is learned about by the vendor. By creating a virus or worm that takes advantage of a vulnerability the vendor is not yet aware of and for which there is not currently a patch available the attacker can wreak maximum havoc.

June 17, 2004

Firefox 0.9

The ever-shrinking browser Mozilla Firefox has now reached version 0.9. It's smaller and perkier than ever before. Here's a list showing how the size of the Windows version has been reduced (size shown is smaller of exe or zip):


  • 8.3MB: phoenix 0.1

  • 7.4MB: phoenix 0.2

  • 7.0MB: phoenix 0.3

  • 7.1MB: phoenix 0.4

  • 6.1MB: phoenix 0.5

  • 6.7MB: firebird 0.6

  • 6.0MB: firebird 0.7

  • 6.2MB: firefox 0.8

  • 4.7MB: firefox 0.9

Lots of major changes were introduced between 0.8 and 0.9 so that the run up to version 1.0 would be used for fine-tuning. As such, it seems a tad buggier than previous versions. The most irritating one for me is that it doesn't set itself as the default browser properly, leading to problems such as two Firefox windows opening at once, or Internet Explorer loading, or just errors on clicking hyperlinks in other programs. The Bugzilla entries related to this are: 246674, 246078 and 246164. There are also some Mozillazine forums threads, such as this one.

Finally, if your favourite Firefox extensions have not been updated to work in v0.9 then you may want to keep v0.8 installed too. I have done that for the sake of card games.

June 4, 2004

Blogspot corruption

If you've been noticing corruption on pages of blogs hosted by Blogspot then you're not alone. It is a result of Mozilla's (Firefox and suite) interaction with the Blogspot servers, and it is being worked on as we speak. Seems it's compression related (hence the weird characters):

So when Mozilla first requests the page, it gets a response 200 ok, with gzip-encoded content. When it then re-requests it with a range, it gets 206 partial content (which is fine so far), but this is no longer gzip-encoded.

June 1, 2004

Browser tricks

For web interface designers, check out the useful information at Seven tricks that Web users don't know.

Only techie people glean information about the structure of a site by examining the words between all those forward slashes. Most users ignore the URL field except for typing in a top-level domain. In testing hundreds of users over a period of several years, I've never seen a non-technical user navigate by modifying the URL. Not once.