from __future__ import *

buy music albums Silver Apples buy mp3 albums Tarrus Riley buy tracks mp3 Kravits buy Reaper albums mp3 buy Kravits albums music buy music Evita CD online albums mp3 Silver Apples download Madonna CD music buy tracks music Kravits download music albums Silver Apples

Xcode Tools has an optional component, CHUD Tools (Computer Hardware Understanding Development Tools), that consists of some useful performance tools and low-level hardware facilities. My Dual 2ghz G5 has been having some serious stability problems lately, with what I believe is a dying CPU or logic board. When I saw errant CPU messages in the system log after experiencing unexplicable kernel panics and crashes I decided to see what would happen if I toggled the second CPU off with the Hardware preference pane that ships with CHUD. It worked! My G5 is now usable (though I will of course still get it repaired, but it's not convenient to do so at this time).

Unfortunately, when I reboot the machine, this setting is lost and all bets are off as to whether I'll be able to disable the second CPU before the machine crashes, so I decided to look into what I could do. I opened up the Hardware preference pane nib with Interface Builder to see what message was sent to change the CPU count (unsurprisingly, setCPUCount:), then I used class-dump to find the implementation address of that message. I then did an otool disassembly of the Hardware preference pane (otool -tVv ...) so that I could see what the code looked like at that address. It called an unconspicuously named function chudSetNumProcessors from the CHUDCore.framework subframework of the umbrella CHUD.framework, which happens to ship with documented headers!

At first, I tried writing a simple C program that naively called right into chudSetNumProcessors, which returned an error code that I didn't expect (from the documentation): something about the kext not being loaded. I knew the kext was indeed loaded, because the Hardware preference pane works and I've used Shark recently, so I looked at the headers for initialization functions. Unsurprisingly, I needed to call chudInitialize before trying to talk to the CHUD kext, so I ended up with the following program:

/*
% cc -o setNumProcessors setNumProcessors.c -framework CHUD
*/

#include <unistd.h>
#include <CHUD/CHUD.h>

int main(int argc, char **argv) {
    int rval = 0;
    int status = chudInitialize();
    if (status != chudSuccess) {
        fprintf(stderr, "FATAL: Could not initialize chud, error %dn", chudInitialize());
        return -1;
    }
    if (argc == 2) {
        int cpuCount;
        int curCPUCount = chudProcessorCount();
        int physCPUCount = chudPhysicalProcessorCount();
        sscanf(argv[1], "%d", &#038;cpuCount);
        if (cpuCount < 1 || cpuCount > physCPUCount) {
            fprintf(stderr, "CPU count of %d not acceptable, expecting between 1 and %dn", cpuCount, physCPUCount);
            rval = -1;
        } else {
            int res;
            res = chudSetNumProcessors(cpuCount);
            if (res != chudSuccess) {
                fprintf(stderr, "Could not change CPU count to %d, error %dn", cpuCount, res);
                rval = -1;
            }
        }
    } else if (argc > 2) {
        fprintf(stderr, "Must take zero or one argumentsn");
        rval = -1;
    }
    printf("CPU Count: %d of %dn", chudProcessorCount(), chudPhysicalProcessorCount());
    chudCleanup();
    return -1;
}

Now I can call this setNumProcessors application early on in the boot process and increase my odds of being able to use my computer on reboot!

UPDATE: rentzsch commented with a better solution. It's also possible to disable multiprocessing even earlier by twiddling a setting in Open Firmware (QA1141).

I'm not sure where I saw this first, but it can be convenient to use Python's complex type to represent 2D points or vectors. Adding two complex numbers together is the same operation as adding vectors, and the absolute value of a complex number is equivalent to the norm of the vector (can be used to calculate distance or length).

# The norm of the vector (3.0, 4.0)
>>> abs(3 + 4j)
5.0

# The distance between (8.0, 10.0) and (-5.0, 2.0)
>>> abs((-5 + 2j) - (8 + 10j))
15.264337522473749

# Adding the vectors (5.0, 8.0) and (-3.0, 2.0) to get (2.0, 10.0)
>>> (5 + 8j) + (-3 + 2j)
(2+10j)

# Suggested by Max Thiercy
# Multiplication of a vector (2, 3) by a scalar (5):
>>> a = 2 + 3j
>>> b = 5 * a
(10+15j)

UPDATE: Frederik commented that he had also written on this topic in the context of Tkinter

I was thoroughly amused when I saw the Linear equations solver in 3 lines recipe in ASPN's Python Cookbook this morning. Since it's so short, I'll reproduce it here as a two line recipe ;)

def solve(eq, var='x'):
    """
    >>> solve("x - 2*x + 5*x - 46*(235 - 24) = x + 2")
    3236.0
    """
    c = eval(eq.replace("=", "-(") + ")", {var: 1j})
    return -c.real/c.imag

The way it works is really clever. First, it uses string replacement to turn it into a single expression rather than an equality. This is effectively the same as rewriting an equation in the form of x = 4 to x - 4 = 0.

x - 2*x + 5*x + 46*(235 - 24) = x + 2
x - 2*x + 5*x + 46*(253 - 24) - (x + 2)

(Please forgive me for using uncommon math expressions below, the standard notations are confusing given the way the equation above was written and I wanted to remain close to valid Python syntax.)

In order to solve such an equation, you would work through the simple math and end up with something in the form of a*x + b. Since this is for simple linear equations only, the author took the liberty of (ab)using the complex type to make this happen automagically.

Complex numbers are in the form of a + bj, where the a represents the "real" component and b represents the "imaginary", the j represents math.sqrt(-1). Python's complex type stores the a in the real attribute and the b in the imag attribute. In this example, we are pretending that the j really represents something other than math.sqrt(-1), namely, the value of x that solves our equation. In order to solve for x, we need to do the following transformation:

a + bj
bj = -a
j = (-a)/b

When the solve function evaluates the rewritten expression, it uses 1j (really, 0 + 1j -- or complex(0, 1)) for the value of x, which Python simply evaluates down to a single complex number. As stated above, when we divide the negation of the real part by the imaginary part, we get the desired solution. Neat!

UPDATE: maxim commented that a more comprehensive recipe by Rick Muller, Manipulate simple polynomials in Python, is also available in the Python Cookbook. It's not two lines, though :)

• PyObjCTools.AppHelper.runEventLoop() will now bring your application to the front at startup when using pdb mode for convenience.
• objc.loadBundle() no longer filters the class list. This solves a few potential issues and shaves off about 1/3rd of the overhead of python -c "import AppKit".
• PyObjCTools.AppHelper.runEventLoop() no longer breaks on pure Objective-C exceptions. Most exceptions of this variety are more like warnings, and there is nothing that can be done them anyway.
• PyObjCTools.AppHelper.runEventLoop() now installs the interrupt handler and verbose exception logging when using pdb, either explicitly or by the USE_PDB environment variable.

I've also made some more progress on the (still marked NonFunctional) RemotePyInterpreter example. It starts up and works, but stdin is not fully implemented and completion is not available. As Cocoa expects completion to be synchronous, completion will be a little bit tricky to implement. I'll probably have it return no completions, and then trigger a completion after the results are available (if they came back fast enough, I guess?). After the interpreter is done, I'll see about writing a remote object browser (you can probably guess where this is eventually leading...).

After doing some optimization of PyObjC's initialization I started thinking about what else can be done to increase performance and offer more flexibility. I think the key to this is making PyObjC as lazy as possible. This might mean the return of an objc.runtime equivalent? Also, I'm not so sure about creating all of these selector objects so eagerly. Why bother? pydoc support for Objective-C classes is basically worthless anyway because there are no doc strings. I suppose we could have the dict-proxy object do the eager lookup if someone tries to iterate over it (more for code completion than pydoc), but I don't see a good reason to bother until you at least poke the class. This kind of laziness may also be the key to solving the infamous bug where you can't reasonably use a class method on a class that has an instance method of the same selector. There is a way, but I think it might crash if the class and instance selectors have different type signatures (though this would be rare if it even exists).

I've just received official notice that both of my 30 minute PyCon proposals have been accepted for PyCon 2005! I took a look at the list of other accepted proposals, and there was definitely some stiff competition. If you're close enough to go, you should. I've been the past two years, and it's top notch and affordable. I highly recommend it, regardless of your Python experience or specialization.

#57. Introduction to PyObjC:

Introduction to PyObjC is a quick tour of PyObjC suitable for anyone with Python experience looking to get their feet wet with Cocoa development using their programming language of choice. Prior Objective-C knowledge or Interface Builder experience is not necessary.

#58. PyObjC Hacking:

PyObjC Hacking is a quick tour of advanced uses for PyObjC and related tools. This talk is geared toward those who have a working knowledge of PyObjC (or at least attended Introduction to PyObjC) and ideally understand data structures at the C level.

I've made several pretty cool enhancements to PyObjC and py2app this weekend:

• There is now a fast path for the NSString/unicode bridge when Py_UNICODE_SIZE is 2. This is the default setting for Python.
• The default selector signature will have a void return value unless a "return" statement with an argument is used in the bytecode. In that case, it will default to an object return value.

This means that objc.accessor(...) isn't necessary for most setters!

• __bundle_hack__ is no longer necessary, py2app now sets a different environment variable to the current plugin during execution, and a hack is installed to NSBundle so that classes may respond to requests for their bundle with the +bundleForClass method. The class builder adds a default implementation of this to Python classes if this environment variable is set.
• Added objc.currentBundle(), which is equivalent to NSBundle.mainBundle() except after loading a plug-in. Makes it easier to load nib files.
• PyObjCTools.NibClassBuilder.extractClasses() now uses objc.currentBundle() instead of NSBundle.mainBundle(). This makes plugins less of a hassle to develop and allows identical code to be used for application or plugin development.
• objc.registerPlugin() and objc.pluginBundle() are now deprecated as they are no longer useful.

These all VASTLY improve support for plugins. Additionally, I have made some changes to the py2app bootstrap to support these new features, and I've cleaned it up so that it loads plugins into their own modules rather than stuffing them into __main__.

I have also started work on a new RemotePyInterpreter example, which runs the interpreter in a subprocess and communicates with it asynchronously. This will be useful for IDE work when it's done. The IPC protocol it uses is a cute little repr-based hack, loosely based on py.execnet. I'll also use this to implement out-of-process debugging (think pdb attach) and other fun things, hopefully in the near future.

For quite some time I've been neglecting my packman repository. I find it hard to make time and the infrastructure for maintainers and users alike are quite archaic.

Ideally I would like to move everything over to using Apple's installation tools with .mpkg based distributions. That, along with replacing some ugly distribution scripts in PyObjC, were why I wrote py2app's bdist_mpkg command. This opens up several doors and has a few immediate advantages:

• Installer can authenticate. packman can't. Several packages need this, especially those requiring headers. The current PackageManager UI requires the user to drop to shell and change permissions on these folders in a very obnoxious and unintuitive way.
• Installer creates receipts.
• Installer packages are standard. Other things use them. For example, SDL, a dependency of pygame, is already available in .pkg format from the source. That means less maintenance and more accountability.
• Installer (as of Mac OS X 10.3, anyway) can safely detect certain kinds of dependencies, where packman runs arbitrary code (uh oh!).
• Some people don't like the PackageManager UI. These people could download the packages themselves and simply double-click to install (though they would have to manage dependencies themselves).
• Installer can allow the user to customize an installation (to disable installation of scripts or documentation, for example). bdist_mpkg already organizes packages in a manner suitable for this.
• Installer will show important information such as the License and ReadMe before installation.
• installer can alternatively be used (but this does require authentication) for a hassle-free installation, or when installing by command-line (think ssh).
• Installer is understood by Apple's tools. Think custom installation DVDs, Mac OS X Server, Apple Remote Desktop, Apple's internal build process, etc.
• Installer may be improved in future versions of Mac OS X, which means that the packman infrastructure would get an upgrade for free (well, cheap).
• Using bdist_mpkg, packman could "record" (not perform) an installation so that it may be performed later or elsewhere. For a given package, packman would download it and all of its dependencies and create a self-contained .mpkg for simple installation and transport. People who have broadband at work or school and dial-up at home would probably be particularly fond of this feature.

... Of course, even after this is developed, it's no promise that I'll have a long-term interest in maintaining my own repository. However, with this kind of infrastructure and the right tools, I think that someone else (hopefully several others) will!

It has recently [1] come to my attention that using versioned frameworks without binary compatible APIs simply doesn't work all too well. Essentially, you may have an older version installed, but without swapping symlinks around, you you won't be able to use the older version for anything except runtime compatibility with software that is already linked.

This isn't a problem with the dyld runtime, simply a problem with gcc's compiler and linker (and supposedly there may be runtime bugs with CFBundle and/or NSBundle in this arena as well).

Frameworks

Mac OS X frameworks are structured in such a way that allow for versioned code, data, and headers. A typical multi-versioned framework looks something like this:

Foo.framework/
    Foo -> Versions/Current/Foo
    Headers -> Versions/Current/Headers
    Resources -> Versions/Current/Resources
    Versions/
        Current -> 2.0
        2.0/
            Foo
            Headers/
            Resources/
        1.0/
            Foo
            Headers/
            Resources/

Mach-O, MH_DYLIB, and the dyld runtime

The MH_DYLIB object file, the part that you link to, is "Foo". If this were not a framework, it would have the following layout, and it would be application-specific to put the headers and resources in the right place:

libFoo.dylib -> libFoo.2.0.dylib
libFoo.1.0.dylib
libFoo.2.0.dylib

Due to the way that MH_DYLIB object files work, each of these dylibs know their own (supposed) location on the filesystem. This is called the "install name" or the "id" of the dylib. Technically, this is a LC_ID_DYLIB load command in the Mach-O header. When you create an Mach-O object file (executable, dylib, bundle, etc.) that depends on another, it will generate a load command (LC_LOAD_DYLIB) referencing the other dylib. The data associated with the LC_LOAD_DYLIB is exactly the same data that was provided by the linked-to MH_DYLIB in its LC_ID_DYLIB load command, unless explicitly overrided with the -dylib_file option to ld(1) or rewritten post-link with a tool such as install_name_tool(1). The otool(1) tool can be used to view these load commands in a Mach-O file, among other things. It's very important to have these values set correctly because they are used by the dyld runtime to locate the intended MH_DYLIB.

gcc, ld, and frameworks

Apple's GCC includes many changes to support Objective C/C++ and framework based development. Unfortunately, none of them have any explicit support for versioned frameworks. For example, gcc(1) states that it uses the following algorithm for finding frameworks in headers:

-Fdir
    In Apple's version of GCC only, add the directory dir to the head
    of the list of directories to be searched for frameworks.

    The framework search algorithm is, for an inclusion of
    <Fmwk/Header.h>, to look for files named path/Fmwk.framework/Head-
    ers/Header.h or path/Fmwk.framework/PrivateHeaders/Header.h where
    path includes /System/Library/Frameworks/ /Library/Frameworks/, and
    /Local/Library/Frameworks/, plus any additional paths specified by
    -F.

    All the -F options are also passed to the linker.

ld(1) has an similar option, -framework, which has a similar algorithm:

-framework name[,suffix]
       Specifies a framework to link against.  Frameworks  are  dynamic
       shared  libraries,  but  they are stored in different locations,
       and therefore must be searched for differently. When this option
       is  specified,  ld  searches for framework `name.framework/name'
       first in any directories specified with the -F option,  then  in
       the  standard  framework  directories /Library/Frameworks, /Net-
       work/Library/Frameworks,  and  /System/Library/Frameworks.   The
       placement  of the -framework option is significant, as it deter-
       mines when and how the framework is searched.  If  the  optional
       suffix is specified the framework is first searched for the name
       with the suffix and then without.

Note that neither of these options allow for any consideration for the Versions directory in a framework, and therefore only link to whichever version was installed last, because the installation process for a framework will create the symlinks that point to locations inside the Versions directly.

Workaround

Unfortunately, since there is no support or hook that will allow proper usage of versioned frameworks, the workarounds are all ugly. I think the following workaround is the most appropriate: Just Don't Use GCC's Search Algorithms.

• Always use the compiler option -I/Path/To/Foo.framework/Versions/IntendedVersion/Headers
• use #include "Foo.h" instead of #include <Foo/Foo.h> in your sources
• Always use the direct path to the MH_DYLIB rather than any combination of -framework and -F.
• Or, if you're building extension bundles that will be used by an executable that already has the correct version of Foo linked in, use the -undefined dynamic_lookup linker option

A minimal compiler/link line for an executable would look like the following:

env MACOSX_DEPLOYMENT_TARGET=10.3 cc 
    -I/Path/To/Foo.framework/Versions/IntendedVersion/Headers 
    -o usesFoo 
    usesFoo.m /Path/To/Foo.framework/IntendedVersion/Foo

And a minimal compiler/link line for an extension bundle would look like:

env MACOSX_DEPLOYMENT_TARGET=10.3 cc 
    -I/Path/To/Foo.framework/Versions/IntendedVersion/Headers 
    -o fooUsingExtension.bundle 
    fooUsingExtension.m 
    -bundle -undefined dynamic_lookup

Quite ugly, eh? At least it works as intended. Unfortunately you'll need to emulate enough of GCC's search algorithms to find the framework, which is probably quite problematic from Xcode, but shouldn't be too hard from say, distutils, SCons, autoconf, etc. Note that you should also probably consider the NEXT_ROOT environment variable when building against an SDK.

Apparently, Darwin's implementation of realloc never frees memory if you ask for a smaller size. Python, and probably plenty of other software, assumes that it does, because that's how it behaves on the other platforms (at least Linux and OpenBSD). Oops!