from __future__ import *

MochiKit Talk at Freedel 06

September 22, 2006 at 12:11 PM | categories: AJAX, javascript, MochiKit | View Comments

While I didn't personally attend Freedel 06, I recently came across the slides for Anant Narayanan's Web Application Development talk. Specifically it covers JavaScript web application development with MochiKit and Dojo, and it's split into four sets of slides. I've just had a chance to give them a look and they're quite good.

The slides are all done with S5, which is what I used for the MochiKit Intro talk I gave a few months ago.

Part 1 - JavaScript:
A general overview of JavaScript and AJAX
Part 2 - MochiKit:
An overview of what MochiKit is and why you'd want to use it. Lots of pointers to cool features such as DOM and Signal.
Part 3 - Dojo:
Covers a bunch of the great functionality that Dojo has that isn't available anywhere else such as widgets, layout, animation, and offline storage.
Part 4 - Case Studies:
A few very short case studies of some JavaScript web applications.
Read and Post Comments

Erlang Binary Performance

September 21, 2006 at 02:28 PM | categories: python, c, erlang | View Comments

I was benchmarking egeoip today, which is my from-scratch Erlang geolocation library. It uses the MaxMind GeoLite City database, which has implementations in a bunch of other languages so it's great to compare with. The results were rather surprising to me, because I hadn't previously done any benchmarking of Erlang performance.

The test environment is a MacBook Pro 2ghz, Mac OS X 10.4.7, Erlang R11B-1 w/ HiPE enabled, Python 2.4.3 (using their GeoIP Python API, which is written in C). I do have other processes running (namely iTunes), but the benchmark is fair because the background load is consistent throughout the tests.

Erlang, BEAM:
~13k geolocations/sec
Python/C:
~18k geolocations/sec
Erlang, HiPE:
~44k geolocations/sec

As you can see, Erlang holds it own against Python w/ C extensions, and it can mop the floor with it when using the HiPE compiler. Erlang clearly kicks some serious ass at working with binaries, both in syntax and performance. The only work I had to do to make it faster was c(egeoip, [native]).

Note that I've only been using Erlang for a few weeks and have not done any profiling or performance tuning at all beyond what I assumed would be the fastest way given the documentation I had read.

Update

After looking at Shark results across the two implementations, it seems that the GeoIP API default settings are pessimistic for benchmarking purposes and that most of the time was spent in syscalls (Erlang looked like its time was spent in GC). A fair comparison would be using the memory cache option, which gets even better performance.

Python/C (Memory Cache):
~117k geolocations/sec

This is a lot more in line with what I expected, but I'm still impressed that Erlang w/ HiPE can get nearly 40% of the speed of C when scanning through a 25MB array of bytes. I'm pretty sure I can make some algorithmic improvements to the code (which the C implementation may or may not do), so we'll see how close I can get.

Update

After spending a while with eprof doing some profile driven optimizations, I was able to considerably speed up the Erlang code. The biggest BEAM optimization was moving the giant tuples out of function bodies, apparently BEAM is rather naive about that and decides to actually create and garbage collect them on every call in certain cases. Some other optimizations were done to the way it looks for null terminators and a hyper-optimized fast-path for IPv4 string to long conversion.

Given the API I could cheat and parse out some of the data when the user asks for it, rather than at record fetch time. This would make the benchmark incredibly fast, but it would be an unfair comparison with the Python/C version. I'll probably end up doing that anyway, since I'm typically looking for just the country of an IP address.

I still haven't really done any algorithmic optimizations to the lookup, but here's the numbers:

Erlang, BEAM:
~44k geolocations/sec
Erlang, HiPE:
~64k geolocations/sec

This brings the BEAM performance up to about 38% of C/Python and HiPE up to 55%. Not bad!

Read and Post Comments

svnsync: mirror your svn repository

September 14, 2006 at 12:15 PM | categories: subversion, universal binaries, macosx, FreeBSD | View Comments

Since the svnsync tool in Subversion 1.4 still lacks documentation, here's a quick example to get you started with it:

$ svnadmin create svn.mochikit.com
$ echo '#!/bin/sh' > svn.mochikit.com/hooks/pre-revprop-change
$ chmod +x svn.mochikit.com/hooks/pre-revprop-change
$ svnsync init file://$PWD/svn.mochikit.com http://svn.mochikit.com/
$ svnsync sync file://$PWD/svn.mochikit.com

The example performs the following tasks:

  • Creates a new local repository named svn.mochikit.com
  • Adds a trivial pre-revprop-change hook to the repository (required for svnsync)
  • Initializes the svn.mochikit.com repository to be a mirror of http://svn.mochikit.com/
  • Synchronizes the svn.mochikit.com repository with its initialized source

Note that this is point-in-time synchronization. To keep an up to date mirror you'll have to run svnsync sync on a regular basis (e.g. by cron job or post-commit hook).

If you're looking for an especially easy to install set of svn binaries for Mac OS X, I recommend Martin Ott's (of SubEthaEdit fame) build. I've been using it on all of my Macs for quite some time with no issues. Much less hassle than building it myself or using MacPorts (previously known as DarwinPorts).

Read and Post Comments

nginx: reverse proxy panacea

September 13, 2006 at 01:17 PM | categories: nginx, FreeBSD | View Comments

A few weeks ago in Reverse Proxy Roundup I was evaluating reverse HTTP proxy solutions. At the time I had settled on Pound, but frankly it wasn't good enough for the load I expected it to handle. Additionally, spawning so many threads used far more RAM and CPU than it should've.

Not long after I wrote that post, a friend of mine pointed me at nginx. nginx is an extremely high performance HTTP server that offers reverse proxying, FastCGI, SSL, and even IMAP/POP3 proxying.

I have had precisely zero problems with nginx so far after about 5 weeks and hundreds of millions of proxied requests. Forget about lighttpd, nginx uses less memory, doesn't leak, and has higher performance (in my FreeBSD configuration, anyway).

The reason I didn't blog about it sooner is that the project is Russian and so was a large majority of its documentation at the time. With the 0.4 release, there's a much more complete draft of the English documentation available. Check it out; your high traffic servers will thank you for it.

Read and Post Comments