from __future__ import *

A few weeks ago I began my move from New York to San Francisco by... moving to Hawaii. For the summer, anyhow. In shipping, my trusty old Klipsch computer speakers seem to have eaten it. I tried hooking them up, and after some really painful shrieking, they worked for a few minutes.. until they caught fire. Really. My ex-favorite music spewing devices have found themselves a nice retirement home in a landfill somewhere on Maui.

I tried playing iPod DJ with the stereo here, but that doesn't really work very well for long stretches. It was time to break down and finally get an Airport Express. I looked around online a bit, and shipping was just ridiculous to Hawaii from most places (e.g. Smalldog wanted $95 S+H to ship a $120 refurb Airport Express to Kihei!). Fortunately, there is a Mac store in town, so I was able to pick one up at a reasonable price.

Immediately after bringing it home I plugged it in via ethernet, and tried to update the firmware. A few hundred times. It just wouldn't take. The Airport Admin utility would not do it. From either of my Macs. It'd try, and it would fail after a minute or two without even an error code. Eventually I found a link to download a standalone firmware updater application, which updated the firmware first try. Go figure. Certainly not the typical Apple experience.

Once it was updated and hooked up, it worked great. 90% of the time, anyway. iTunes will only play files that it natively understands through the Airport Express. Apparently, iTunes has decided that some of my MP3s were QuickTime files, and refused to play them through Airport Express, which means they were coming out of the built-in speakers. That didn't work out so well. I guess that sucks for people who care about OGG and other formats supported only via QuickTime.

It turns out that the reason these files were picked up as QuickTime is that iTunes has MPEG type code hate. MacAMP and SoundJam of yesteryear used MPEG, yet iTunes wants to see MPG3. It took me a lunch break to come up with a good way to track down all of these files without writing a lot of code. It was actually rather simple: the iTunes Music Library XML dump.

~/Music/iTunes/iTunes Music Library.xml is a brain-dump of iTunes that it creates every so often (on quit, I believe). It keeps track of almost everything that iTunes knows, all of your file's URLs, their type and creator codes, comments, metadata, you name it. It's also in plist format, which is extremely easy to work with from Python.

Assuming you have PyObjC installed, you can break into a Python interpreter and screw around with this rather easily:

% python
Python 2.4.1 (#2, Mar 31 2005, 00:05:10)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from Foundation import *
>>> import os
>>> dbFile = os.path.expanduser("~/Music/iTunes/iTunes Music Library.xml")
>>> db = NSDictionary.dictionaryWithContentsOfFile_(dbFile)
>>> type(db)
<objective-c class NSCFDictionary at 0x30a4b0>

With the database in hand, we can start poking around:

>>> track = db[u'Tracks'].itervalues().next()
>>> print track.description()
{
    Album = "Live Code";
    Artist = "Front 242";
    "Artwork Count" = 1;
    "Bit Rate" = 160;
    ...

In these track dictionaries, there are two keys relevant to this exercise: u'File Type', and u'Location'. File type is the integer representation of the type code of the track. Location is the URL of the track. With this, it's pretty easy to pick out the locations of the tracks that are set to the MPEG type code:

>>> import struct
>>> badPaths = [
...     NSURL.URLWithString_(track[u'Location']).path()
...     for track in db[u'Tracks'].itervalues()
...     if track.get(u'File Type') == struct.unpack('>I', 'MPEG')[0]
... ]
>>> badPaths[0]
/Volumes/...

With this, all we need to do is change the type and creator. Fortunately, that's easy. We'll use the iTunes creator code hook, and the expected MPG3 type code:

>>> import MacOS
>>> for path in badPaths:
...     MacOS.SetCreatorAndType(path, 'hook', 'MPG3')
...

And with that, my Airport Express does just about everything I'd have expected it to out of the box ;)

For various reasons, I have a need to determine the version of an iPod. Previously, I only needed to know if an iPod was "firmware version 2" or later (aka "Dock-connector"), and there's a whole slew of obvious ways to determine that (presence of a Notes folder would be the most obvious). There are also ways to grab iPod information with SPI (via COM on win32 or the private iPod.framework on Mac OS X), but I like to avoid that whenever possible.

So far, the most reliable method I've found was with a little parsing of the SysInfo file. This file is located at "$(IPOD_VOLUME)/iPod_Control/Device/SysInfo". The iPod_Control folder is going to be hidden on either platform, so you may not have noticed it before. Many examples of SysInfo files can be found by googling for some of the keys, such as buildID, visibleBuildID, boardHwName, etc. The most complete resource I've found is the on the iPodLinux forums.

As-is, the SysInfo files don't make any of the useful information very obvious. However, with a little reverse-engineering of the iPod Updater for Mac OS X, it was relatively easy to figure out what it was using to determine the iPod model.

In the Resources folder of any iPod Updater, you'll find a lot of interesting things. The most interesting bits are UpdaterVersions.plist and the Updates folder.

UpdaterVersions.plist is a typical XML property list with a single root key: Versions. Under this key you'll find a dictionary that maps updaterFamily to a dictionary of information about the iPods in that updaterFamily, not unlike the information you will find in a SysInfo. The dicts with a displayInAbout key with a true value are the most interesting, all of the other entries are simply variations on the theme (i.e. the iPod U2 Special Edition or the various colors of iPod Mini). When using this filter, you should have exactly one entry per unique iPodFamily value.

The Updates folder is interesting because it contains icons for each iPod, as well as the firmware images. The icons are named either DeviceIcon-$(iPodFamily).icns or DeviceIcon-$(iPodFamily)-$(iPodColor).icns. I'm relatively certain that the color can be determined from the last three characters of the pszSerialNumber, but I don't have enough iPods around to test that theory.

With all of this information, it would almost be easy to determine the model of an iPod. However, it's not quite that simple. Not all SysInfo files contain an iPodFamily key, and some that do incorrectly report 0! The only reliable identifier for an iPod family is the following (given a dict from either the UpdaterVersions.plist or from a SysInfo file):

def buildPair(dct):
    buildID = dct.get('buildID', 0)
    visibleBuildID = dct.get('visibleBuildID') or dct.get('VisibleBuildID', 0)
    # grab these bits: 0xFF000000L
    return (buildID >> 24, visibleBuildID >> 24)

What I'm doing here is pulling the "major" version out of the version keys. It appears that the versioning convention is: 0x0ABC8000 where the firmware version is pretty-printed as "A.B.C", trimming "C" and possibly "B" if they are 0. This is similar to the hex-version-convention you see in gestaltSystemVersion ('sysv') and other places. It would be a little less ugly if the UpdaterVersions.plist used the same case as the SysInfo files for the visibileBuildID key! The reason that we need both the buildID and the visibleBuildID is that the iPod Mini and "3G iPod" both report a buildID major version of 2, however the iPod Mini has a visibleBuildID major version of 1.

Parsing the SysInfo file in a manner that will grab the information in the right way is pretty trivial, but perhaps not so obvious:

import os
def infoForMountedPod(path):
    path = os.path.join(path, u'iPod_Control', u'Device', u'SysInfo')
    lines = file(path).read().splitlines()
    d = {}
    for line in lines:
        try:
            key, value = line.split(': ', 1)
        except ValueError:
            continue
        try:
            value = long(value.split(None, 1)[0], 0)
        except (ValueError, IndexError):
            pass
        d[key] = value
    return d

infoForMountedPod simply looks for all lines that look like a key-value pair (containing a ": "). For each of these lines, it will set the key-value pair in the returned dictionary. For lines whose first "word" (split on whitespace) is parseable as an integer (typically as hex), it will be set as an integer. Otherwise, it is set to the string value of that line (i.e. pszSerialNumber).

Parsing UpdaterVersions.plist is pretty trivial too:

import plistlib
def parseUpdaterVersions(path):
    # this is Python 2.4 plistlib API
    # in 2.3 you can use plistlib.Plist.fromFile
    versions = plistlib.readPlist(path)
    buildPairs = {}
    families = {}
    for k,v in versions['Versions'].iteritems():
        # only pick out unique iPodFamily dicts
        if v['displayInAbout']:
            fam = v['iPodFamily']
            # skip pre-2.x iPods and iPod Shuffle.
            # This is specific to my use case, you may
            # not want this filter.
            if 1 < fam < 128:
                assert fam not in families
                families[fam] = v
                pair = buildPair(v)
                assert pair not in buildPairs
                buildPairs[pair] = v
    return buildPairs, families

Note that all of this code is cross-platform, but you'll need an UpdaterVersions.plist or equivalent from somewhere, and you'll need to pick up plistlib.py from Python's src/Lib/plat-mac dir if using it on some other platform. Alternatively, you could use ElementTree to parse the plist XML. Its iterparse documentation has an excellent example of how easily it can be used to parse these files.

We've (finally) updated talkingpanda.com today with a fresh new look and a new product: iBar.

iBar turns your iPod (any iPod with a screen and a dock connector) into an "ultimate bartending tool" with over a thousand drink recipes, mixing techniques, and even a couple history lessons. Like any good bar should be, it's stocked. Altogether, the Notes content adds up to over 3 megabytes (yes, that's text!) with more than 40 minutes of original, professionally recorded, audio.

Like its sibling iLingo, iBar ships with installers for Mac OS X and Windows XP/2000 (everywhere iTunes is supported) that make installation painless.

And for relevance, here's a little bit about how it's all put together:

• The Mac OS X installer was developed with Python 2.3, PyObjC 1.2, and bundled up with py2app.
• The Win32 installer was developed with Python 2.4, win32all, Tkinter, PIL and is bundled up with py2exe and then made self-contained with NSIS.
• The build scripts for the installer application actually use even more open source stuff. We use JExcelAPI to convert Excel spreadsheets to XML, and Win32::Exe to swap out the ICO resource out of win32 executables -- but no Java or Perl makes it into the redistributable :)

We're also blogging updates and podcasting some of our content, so point your NetNewsWire or Safari RSS over to Talking Panda News!

My first commercial consumer application, Talking Panda, has just been released! We offer a set of foreign language phrase books for iPods.

The installer is written in PyObjC, and I've described some of the development process in this pythonmac-sig message.