from __future__ import *

My PyCon proposal, 60 Minutes of MacPython, has been accepted. I'll be speaking about Darwin, Apple Events, Cocoa (PyObjC), and the MacPython community.

For those of you interested (and you should be, there are plenty of other good talks and tutorials this year), it's not too late to get yourself a ticket. Even though early-bird registration just ended, it's still pretty cheap for a conference (remember: 3 days + 4 days of optional sprints!). It was great last year, I'm sure it'll be even better this time around (word has it that someone is even planning an afterparty for the second night).

This is the second in a finite series of articles about the development of aeve.

...Continuing from where I left off:

aeve.util

I didn't quite finish this one out last time (I think I was tired). aeve.util is generic could-be-anywhere Python code.

aeve.util.ShortBitmask

This is probably the least generally useful module in aeve.util, but the code is portable and it doesn't say Apple anywhere, so I figured this would be the right place for it, but it could or perhaps should be in aeve.terminology. The aete resource format (yes, that documentation is really from 1996) uses these 2-byte "flags" everywhere. In AEUserTermTypes.r [1], they're arrays of boolean pairs, and that's the representation I chose to use. ShortBitmask is an int subclass, with a metaclass that does the dirty work. It's not meant to be used directly, but to be again subclassed and provided with a __pairs__ member that enumerate the properties. None is a placeholder property for a reserved bit. Here's what it looks like in action:

>>> from aeve.util.ShortBitmask import ShortBitmask
>>> class EventReplyFlags(ShortBitmask):
...   __pairs__ = (
...     ('replyRequired', 'replyOptional'),
...     ('singleItem', 'listOfItems'),
...     ('notEnumerated', 'enumerated'),
...   )
...
>>> erf = EventReplyFlags(1 << 15 | 1 << 13)
>>> erf.replyRequired
0
>>> erf.replyOptional
32768
>>> erf.singleItem
16384
>>> erf.enumerated
8192
>>> erf
40960

I'm using the masks themselves instead of booleans for a reason, I forget what it is at the moment, but I certainly had one. In any case, it's equivalent to what it would be as a boolean and it's faster to just return the integer without the conversion (not that I care). This odd hack is only so I don't have to carry around a bunch of per-flag-type constants later on (in aeve.runtime and aeve.compiler). It does make it harder (but still possible) to create runtime types by hand, as they expect ShortBitmask subclasses and not ints (the same goes for NamedTuple vs. tuple). I'm willing to trade that "convenience" for readable code.

aeve.constants

aeve.constants is probably one of the most and least useful parts of aeve. Basically what I did was take the AppleEvents constants from Python itself (bgen generated from the C header), converted it all over to aeve.util.Enumerations, and grouped it into classes that I could understand (for my reference, not used in code).

Before (these aren't marked as related.. they're mixed in with the other 958 lines of constants):

keyDirectObject = FOUR_CHAR_CODE('----')
keyErrorNumber = FOUR_CHAR_CODE('errn')
keyErrorString = FOUR_CHAR_CODE('errs')
keyProcessSerialNumber = FOUR_CHAR_CODE('psn ')
keyPreDispatch = FOUR_CHAR_CODE('phac')
keySelectProc = FOUR_CHAR_CODE('selh')
keyAERecorderCount = FOUR_CHAR_CODE('recr')
keyAEVersion = FOUR_CHAR_CODE('vers')

After:

class AEEventParameterKeywords(FourCharCodeEnumeration):
    """Keywords for Apple event parameters"""
    keyDirectObject = FourCharCode('----')
    keyErrorNumber = FourCharCode('errn')
    keyErrorString = FourCharCode('errs')
    keyProcessSerialNumber = FourCharCode('psn ')
    keyPreDispatch = FourCharCode('phac')
    keySelectProc = FourCharCode('selh')
    keyAERecorderCount = FourCharCode('recr')
    keyAEVersion = FourCharCode('vers')

Yes, my code does needlessly use FourCharCode (since it will get turned into the mixed-in FourCharCode __baseclass__ anyway when the metaclass creates it). My excuse is that I was using regular expressions to do all this, and didn't feel a need to chop it out. I also compared the standard MacPython version (it used an older version of Universal Headers) with the OS X headers and I actually had to add in some missing OS X related constants, such as the typeApplicationBundleID, typeKernelProcessID, and typeMachPort addressing modes.

... to be continued

I've been working on totally rewriting aeve the past well, too long. This week I've made some huge progress. This is the beginning of a series of blog entries where I'm going to talk about the different components of aeve. I will probably only talk about ones that are finished, or mostly finished :) Today I'm just going to stick to the ones that have nothing in particular to do with Apple Events.

aeve._hacks

aeve._hacks is a package where I store things I shouldn't have to do, and it is the source of any import time side-effects that aeve has. Currently the only things in there are the Python 2.4 CVS versions of applesingle, and pprint. applesingle was replaced because it throws a string exception and because it has warnings. pprint was replaced because the Python 2.3 version won't pretty-print subclasses of pretty-printable types. Both of these hacks happen immediately on import of the aeve package, but nothing happens if you happen to be running a build of Python 2.4.

aeve.util

aeve.util is my repository for generic usable-elsewhere cross-platform Python code. I am pretty sure that I'm done with all of the modules in here.

aeve.util.Enumerations

This is one of my favories, and is strangely enough inspired by a similar construct in C#. In C# you can add metadata to just about anything (I think it's called attributes). You can, of course, do this in Python, but it's less of a "meta-protocol" than it is in C# (which has syntax support for the feature). aeve carries around a lot of enumerations whose enumerators need to carry around metadata (such as enumeration it came from, or its name). It works by making a per-enumeration subclass of the desired base type that has a modified __new__ and __repr__, and carries around a reference to the parent enumeration. The base type is typically int but in aeve it's usually a FourCharCode. FourCharCode is a str subclass that will __new__ from an int by struct.pack. The reason for this (as opposed to regular str) is that new-style Apple header files use integers to represent four char codes that contain macroman characters, presumably so that the header files are unix-tool friendly (7-bit unambiguous good ol' american ASCII). Enough of that, this is how it looks:

>>> from aeve.util import Enumerations
>>> class MyEnum(Enumerations.Enumeration):
...   a = 1
...   b = 2
...
>>> MyEnum.a
MyEnum.a
>>> MyEnum.a + 4
5
>>> MyEnum.fromName('a')
MyEnum.a
>>> MyEnum.fromValue(1)
MyEnum.a
>>> MyEnum.fromValue(1).enumeration
<class '__main__.MyEnum'>

aeve.util.NamedTuple

I've talked about this one before, but I just want to let Just know that I took all the lambdas out ;) NamedTuple is used to build magically delicious tuple trees out of terminologies. I wrote this so that I could introspect the objects (used in aeteViewer: pdf screenshot), but primarily because using named attributes instead of arbitrary indexes is a whole hell of a lot nicer to look at. It's now relatively easy to tell what aeve.compiler.core is doing, for example.

aeve.util.descriptors

Here's where I'm going to stash all of the generic descriptors I need. So far, I've only used one generic descriptor: metamethod. This has been discussed previously, I used Phillip's implementation.

aeve.util.microfailure

microfailure is a homebrew version of twisted.python.failure. I just use it to capture exceptions and pass them along "safely" without ignoring them. It's used by aeve.terminology.resfinder and aeve.compiler.core.

aeve.util.namemangling

namemangling makes strings safe for python names/atributes by appending underscores and replacing unicode with ascii. This is so it doesn't clash with a keyword or builtin. It will do substitution of particular unicode characters such as u'N{DIVISION SIGN}' (aka ÷) with pythonic ascii names like 'div'. If it doesn't have a built-in translation it will use unicodedata.name, or at worst case, it will use the hex representation of the unichar.

... to be continued

I ended up doing a high level overview of Apple Events / AppleScript in a post on pythonmac-sig this morning. In response to Jack's suggestion on the MacPython channel to add this information to the FAQ, I've expanded, edited, and wikified it as AppleScript. I'll add the FAQ entry sometime later.

I just got back yesterday morning from my ~2.5 week stint in Tokyo. I didn't really do any coding (open source or otherwise) while I was away, but I'm pretty sure I still remember how ;)

The Mac OS X 10.3.2 updater upgrades the Darwin kernel to 7.2.0 from 7.0.0. This has some effect on my package repository (mostly an effect on the tools I use to build it), so I'll probably be sweeping through and doing package updates this week. It did have a short lived effect on the official database, but Jack fixed the issue on Dec 26th.

Here's my blog-relevant TODO for the next two weeks or so:

• Finish the next version of aeve (huge changes!)
• Update my package repository
• Start writing a to-be-named (mac specific) music management software program that makes managing large collections of music possible. It will be written in PyObjC and will be dependent on iTunes via aeve as a backend (at least at first, because it's a pretty good database that abstracts ID3 really well). Kevin pointed me at musicbrainz, which in theory could make this stuff a lot more useful and efficient so I hope to integrate that, possibly in the first version. The big idea is to make it easy to write regexes to generate ID3 tag info from existing ID3 tag info and filenames, as well as bringing in metadata from other sources (like cddb, gracenote, musicbrainz, etc.) when the existing information is not complete.

This is a pattern I've been playing with lately (to be used in the next version of aeve, in the aeut/aete decoder). Basically it's a factory for "tuples with a little metadata that can be used a little bit like dicts too". Yeah I know that dict-like methods will stomp on names if there's a clash, but I don't have any use cases where that happens so my implementation doesn't consider that.

def newNamedTupleType(typename, names):
    """Return a subtype of tuple with named accessors"""
    dct = dict([(_name, property(lambda self,_i=_i:self[_i], doc=_name)) for (_i, _name) in enumerate(names)])
    dct['__names__']    = names
    dct['keys']         = lambda self: self.__names__
    dct['items']        = lambda self: zip(self.__names__, self)
    dct['iteritems']    = lambda self: itertools.izip(self.__names__, self)
    dct['values']       = lambda self: self
    return type(typename, (tuple,), dct)

It's primarily useful in conjunction with the struct module, for example, to unpack a version record, which is a pair of 16bit words, you could use something like the following:

import struct
VersionRecord = newNamedTupleType('VersionRecord', ('major', 'minor'))

def decodeVersionRecordFromPackedString(s):
    return VersionRecord(struct.unpack('HH', s))

def prettyPrintVersionRecordFromPackedString(s):
    rec = decodeVersionRecordFromPackedString(s)
    print "%(major)s.%(minor)s" % dict(rec.iteritems())

Yes, the example is contrived. But it really does make things easier when you're passing non-trivial versions of these around. Instead of having to remember the index of a field, you can just remember the name. In my typical use case, these names come from C structures, so it makes the Python code closer to what a C implementation would be like (in a good way).

I have a more generic (but mutable and not nearly as lightweight) solution to this problem (making structures easier to work with), currently called "ptypes" since it is so much like ctypes Structures API-wise. It's currently being used in a pure python package (macholib, formerly potool) for reading/writing Mach-O headers (the OS X analog of Linux's ELF, or Win32's PE). macholib is useful for dependency walking, and automation of install_name_tool like functionality. However, since it also understands symbol tables, it would be useful for determining if a particular source file contains Objective C class definitions.. which is good to know because Objective C classes can not be unloaded at runtime. It could even be used to write a symbolic debugger if one were so inclined. I have written a largely untested symbolic PowerPC disassembler in Python, so it would be neat to hook the two together eventually.

I was originally going to write a little bit about the unicodedata module, but google tells me that Ludoo already did. So instead, here's a quick aeve script that will set your iChat status message to the current track in iTunes (make sure to run with pythonw, it needs WindowManager access).

import aeve
ichat = aeve.talkto('com.apple.iChat')
itunes = aeve.talkto('com.apple.iTunes')
ichat.status_message = u'\N{BEAMED EIGHTH NOTES} %s - %s' % (
    itunes.current_track.name,
    itunes.current_track.artist)

Also see Donovan Preston's iTunesStatus script (requires Twisted and Quotient) that makes the same information available from a web browser!