from __future__ import *

Kailash and Friends Kailash Kher Kaipa

There's a new reactor in Twisted svn: threadedselectreactor. This reactor blocks on select() in a worker thread, so that it is easy to integrate with a foreign event loop without having to completely separate your Twisted code into its own thread. This is awesome, and mostly eliminates the need to have specific reactors for each useful foreign event loop (cfreactor, wxreactor, etc.). Especially when the existing integration reactor sucks -- wxreactor is more or less a worst-possible implementation, and the others aren't a whole lot better.

In order to integrate with foreign event loops, threadedselectreactor sports an extension to the reactor interface: interleave(waker). waker is a callable that takes a callable as an argument. Its job is to call this callable from the thread of the main event loop. So, whatever you're integrating with needs to be able to send messages from an arbitrary thread to the main thread. All of the event loops worth the fairy dust their bits are encoded on are designed to do this easily.

Other than that, the only other thing that needs to be considered is shutdown (if it matters to shut down cleanly). To do that, you can simply replace your "quit" function with an implementation that:

• Sets up an "after shutdown" reactor system event that will call "quitRightNow" when the reactor is dead
• Calls reactor.stop() to commence the self destruct sequence

wxPython already has wxCallAfter that works perfectly well as the waker argument for interleave, so you can Twistify a wxPython application in about six lines. Here's a full demo: doc/examples/core/threadedselect/wxdemo.py.

PyObjC has a PyObjCTools.AppHelper.callAfter that also serves as a too-trivial-to-be-true waker (coincidence? no -- I just added it). You can check out the demo from: doc/examples/core/threadedselect/Cocoa/SimpleWebClient/. The examples in PyObjC's Examples/Twisted/ have also been refactored to use this (instead of cfreactor, which should be considered dead weight).

pygame is more low-level and doesn't provide any conveniences beyond getting and posting events. So your waker should be some function that posts a USEREVENT to pygame, which you need to pick up and do something with. A minimal example of this is here: doc/examples/core/threadedselect/pygamedemo.py.

This reactor should trivially integrate with just about any event loop, including PyQt, GTK, etc. However, the margins of my blog aren't big enough to fit them all.

I had spent the past few days on and off trying to help a py2app user with a very hairy problem: when wxPython 2.4.2.4 was bundled, the main menu didn't work. Running the application as an alias bundle or with pythonw worked just fine, but when built as standalone or semi-standalone, the menu no longer showed up.

Right off the bat I (correctly) assumed it was a problem with wxPython or his sample code, because py2app doesn't do anything that would cause this sort of behavior. It does link to Cocoa, but it doesn't call into any AppKit functionality unless it needs to display an error message. Somewhat trivially converting his source to work with wxPython 2.5 solved this issue, so I was rather stumped.

I had a hunch that perhaps there was something that wxPython 2.4.2.4 wants that didn't end up in the bundle, so I broke out ktrace. Using ktrace is rather simple:

% ktrace ./dist/test.app/Contents/MacOS/test

This will create a file in the current directory, ktrace.out, with a log of just about everything that happened. For efficiency, this file is in a binary format that you must process with kdump. The output of kdump is quite lengthy, but it looks like this:

15121 ktrace   CALL  execve(0xbffffd1b,0xbffffc84,0xbffffc8c)
15121 ktrace   NAMI  "dist/test.app/Contents/MacOS/test"
15121 ktrace   NAMI  "/usr/lib/dyld"
15121 test     RET   execve 0

CALL is the actual system call, NAMI is a name to inode translation that the system call used, and RET is the value returned to the application. If there was an error during the system call, kdump will gladly tell you everything you wanted to know:

15121 test     CALL  open(0xbfffe7f0,0,0x1b6)
15121 test     NAMI  "/Library/Preferences/org.pythonmac.unspecified.test.plist"
15121 test     RET   open -1 errno 2 No such file or directory

Since I suspected that wxPython was missing a file, I wanted to narrow down the output, so I naturally used grep on the kdump output to find errors:

% kdump | grep -B 2 errno | grep wx
15121 test     NAMI  "/Users/bob/Desktop/simple/dist/test.app/Contents/Resources/wxPython"
15121 test     NAMI  "/Users/bob/Desktop/simple/dist/test.app/Contents/Resources/wxPython.so"
15121 test     NAMI  "/Users/bob/Desktop/simple/dist/test.app/Contents/Resources/wxPythonmodule.so"
15121 test     NAMI  "/Users/bob/Desktop/simple/dist/test.app/Contents/Resources/wxPython.py"
....

Unfortunately, what we're looking at here (and for the next 150 or so lines) is primarily just Python doing its module import search. Since I knew that all of sys.path pointed to locations under Resources, I can just filter that out. It is extremely unlikely that wxPython wants anything from there:

% kdump | grep -B 2 errno | grep wx | grep -v Resources
15121 test     NAMI  "/Users/bob/Desktop/simple/dist/test.app/Contents/MacOS/../Frameworks/libwx_mac-2.4.0.rsrc"
15121 test     NAMI  "/libwx_mac-2.4.0.rsrc"

There it is! So now I just need to make the setup.py look like the following:

from distutils.core import setup
import py2app
setup(
    app = ['test.py'],
    data_files = [
        ('../Frameworks', ['/usr/local/lib/libwx_mac-2.4.0.rsrc']),
    ],
)

Now the application works as expected. py2app 0.1.9 will include a recipe for wxPython to make sure that this file ends up in the bundle automagically, among other new features.