from __future__ import *

2008-03-08

Exploring Erlang @ C4[1] video

Filed under: erlang, mochiweb — bob @ 8:56 am

The video of the Exploring Erlang talk I gave at C4[1] last August is up on the interwebs! It only took seven months :)

Video:: http://rentzsch.com/c4/c41VideosAvailable
Slides and source:: http://bob.pythonmac.org/archives/2007/08/11/exploring-erlang-c41/

Comments (2)

2007-12-17

Printing floats with Erlang

Filed under: erlang, mochiweb — bob @ 10:37 pm

The float printing options that ship with Erlang all suck. You either don't have sufficient precision, or you have an extremely verbose representation:

1> io:format("~w ~.18g ~g ~s~n", [0.1234567, 0.1234567, 0.1234567, float_to_list(0.1234567)]).
0.123457 0.123456700000000003 0.123457 1.23456700000000002548e-01
2> io:format("~w ~.18g ~g ~s~n", [0.1, 0.1, 0.1, float_to_list(0.1)]).
0.100000 0.100000000000000006 0.100000 1.00000000000000005551e-01

mochiweb includes an implementation of the algorithm from the "Printing Floating-Point Numbers Quickly and Accurately" paper in the mochinum module (which is standalone, if you just want to use it without the rest of mochiweb):

3> io:format("~s~n", [mochinum:digits(0.1234567)]).
0.1234567
4> io:format("~s~n", [mochinum:digits(0.1)]).
0.1

Problem solved :)

Comments (0)

Using the mochiweb project skeleton

Filed under: erlang, mochiweb — bob @ 9:06 pm

A bit over a week ago we moved a slightly modified version of our project skeleton into the open source mochiweb repository, which gets you a start shell script, a skeleton application following (most of) the OTP paradigms, a Makefile, and a web server that serves static files out of priv/www.

The first thing you have to do is make sure mochiweb is built:

$ cd ~/src/mochiweb
$ make
(cd src;make)
make[1]: Nothing to be done for `all'.

After that you just run ./scripts/new_mochiweb.erl with the project name and destination (you can put a symlink of that script on your PATH somewhere).

$ ./scripts/new_mochiweb.erl mochidemo ~/src
...

Starting it is easy, just build with make and then run start.sh or start-dev.sh from ~/src/mochidemo. The only difference between the two at the moment is that start-dev.sh will poll for changes in ebin files and purge them automatically when they're updated. The default port is 8000 listening on all IPs and it's configured in src/PROJECT_sup.erl.

$ cd ~/src/mochidemo
$ make
...
$ ./start-dev.sh
...
=PROGRESS REPORT==== 17-Dec-2007::20:58:33 ===
         application: mochidemo
          started_at: nonode@nohost

1> http:request("http://127.0.0.1:8000/").
...
{ok,{{"HTTP/1.1",200,"OK"},
     [{"date","Tue, 18 Dec 2007 04:58:55 GMT"},
      {"server","MochiWeb/1.0 (Any of you quaids got a smint?)"},
      {"content-length","88"},
      {"content-type","text/html"}],
     "<html>\n<head>\n\n</head>\n<body>\nMochiWeb running.\n</body>\n</html>\n"}}

As far as deployment goes, it's a convenience that mochiweb makes a symlink to itself as PROJECT/deps/mochiweb-src so that the start scripts can find it and put them in your code path. You'll probably want to put a full copy of mochiweb in its place, use a svn:external (which is what we do right now), or manage the mochiweb dependency by some other means.

Comments (0)

2007-11-07

mochiweb - another faster web server

Filed under: erlang, mochiweb — bob @ 9:39 pm

mochiweb is finally well on its way to becoming a proper open source project, thanks to Matthew Dempsky. We've got a mochiweb project on google code and a mochiweb group on google groups.

Similarly interesting is that iserve is now turning into something more useful. iserve and mochiweb are pretty much the same thing under the hood, but mochiweb exposes a different API (which I find to be nicer, but I designed it) and supports more of the HTTP protocol. They both (ab)use inets' {packet, http} mode and they're both very minimal.

Here's a mochiweb version of the minimal example that Tobbe gave for iserve:

-module(mochiweb_demo).
-export([mochiweb_request/1, start/1]).

start(Port) ->
    mochiweb_http:start([{port, Port}, {loop, {?MODULE, mochiweb_request}}]).

mochiweb_request(Req) ->
    Req:ok({"text/html",
    <<"<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">
        <html>
        <head><title >Welcome to mochiweb</title></head>
        <body>
            Hello
        </body>
        </html>">>}).

Comments (1)

2007-08-11

Exploring Erlang @ C4[1]

Filed under: erlang, macosx — bob @ 2:46 pm

Slides from my Exploring Erlang talk today at C4[1] are available here: http://undefined.org/c4-1/

Yes, it is definitely caturday.

For those of you interested in learning more about Erlang, I highly recommend buying Joe Armstrong's Programming Erlang and/or read his 2003 ph.d thesis, which is kinda similar in content to his book but is more technical and is freely available: Making Reliable Distributed Systems in the Presence of Software Errors. The thesis is a bit more dense, but it was what really made Erlang "click" for me.

If you want to play with the examples, go to CEAN and download "Erlang/OTP Full (Standard)" at the bottom for you version of Mac OS X and follow the "How do I install CEAN to /usr/local/bin" FAQ entry. Alternatively, if you want to wait longer, you could use MacPorts to download and compile it. Once you've got that set up just run erl from the examples directory in the slides download and it should all work.

Comments (2)

2007-03-16

Distel and Erlang mode for Emacs, on Mac OS X

Filed under: erlang — bob @ 6:19 pm

In the comments of my post on Erlang Mode for Emacs Tobbe pointed me at the new Distel repository for Distel. I had originally tried to install Distel last year when I began with Erlang but it didn't work. It works now! Here's the deal.

Check out Distel. In my case I'm going to check it out to /Users/bob/src/distel:
```
$ cd /Users/bob/src
$ svn co http://distel.googlecode.com/svn/trunk distel
```
Configure and compile Distel:
```
$ ./configure
$ make
```

Configure Distel and configure Erlang mode, ensure that the inferior Erlang shell has an explicit node name of emacs so that Distel can connect to the. Here's my full .emacs:

(add-to-list 'load-path  "/usr/local/lib/erlang/lib/tools-2.5.3/emacs")
      (setq erlang-root-dir "/usr/local/lib/erlang")
      (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path))
      (require 'erlang-start)
(add-to-list 'load-path "/Users/bob/src/distel/elisp")
      (require 'distel)
      (distel-setup)

;; prevent annoying hang-on-compile
(defvar inferior-erlang-prompt-timeout t)
;; default node name to emacs@localhost
(setq inferior-erlang-machine-options '("-sname" "emacs"))
;; tell distel to default to that node
(setq erl-nodename-cache
      (make-symbol
       (concat
        "emacs@"
        ;; Mac OS X uses "name.local" instead of "name", this should work
        ;; pretty much anywhere without having to muck with NetInfo
        ;; ... but I only tested it on Mac OS X.
        (car (split-string (shell-command-to-string "hostname"))))))

Add Distel to your ~/.erlang so that it's on your Erlang code path:

$ echo 'code:add_pathsz(["/Users/bob/src/distel/ebin"]).' >> ~/.erlang

If the Distel installation was done properly your Emacs mode line will be displayed as (Erlang EXT) next time you're editing an Erlang module. If you get a "nodedown: ..." message when executing a Distel command that means you haven't started a shell yet (C-c C-z). A good test is to stick your insertion point at a function name and hit M-. to get to its definition. M-TAB is auto-completion. I haven't used too much else from it yet.

(Now here's to hoping I don't have to learn any more elisp, ever).

Comments (0)

2007-03-14

Erlang Mode for Emacs

Filed under: erlang — bob @ 4:51 pm

Erlang ships with a quite nice Erlang mode for Emacs. Editing Erlang code is actually the only thing I ever use Emacs for.

Setting it up is only slightly painful. For a default R11B-3 installation your .emacs will look like this:

(setq load-path (cons  "/usr/local/lib/erlang/lib/tools-2.5.3/emacs"
      load-path))
      (setq erlang-root-dir "/usr/local/lib/erlang")
      (setq exec-path (cons "/usr/local/lib/erlang/bin" exec-path))
      (require 'erlang-start)
(defvar inferior-erlang-prompt-timeout t)

The first set of expressions sets up Emacs to find erlang.el and the path to your Erlang installation.

The second expression tells the emacs mode not to wait for an Erlang prompt. This is my only annoyance with the mode. Without this defvar, when you send a command from Emacs to Erlang it'll hang for 60 seconds (or until you hit Ctrl-G) if there was any IO since the last prompt (e.g. an error_logger report or an io:format call). The caveat with this setting is that the first command you issue (the one that causes the inferior shell to get started) will get sent before Erlang is started and get lost. Issuing the first command twice is a small price to pay in this case, because the lock-up is annoying as all hell when you're in the middle of the compile/play cycle.

Using the Erlang mode is pretty straightforward. Tab does the right thing for indentation and is a good way to check to see if your code makes sense syntactically. If it doesn't indent to the right place, you probably screwed up. The electric stuff like comma and semicolon are great and save you a good deal of typing. The only command I really use is C-c C-k which compiles and reloads the current module into the running Erlang inferior shell. It will start one if one isn't already running (C-c C-z just starts the shell).

The other gripe I have is that when you get compiler errors sometimes clicking the line numbers takes you to the wrong line in the source file, but that's a relatively minor annoyance for me. Restarting seems to fix that (until it breaks again) and normally I'm not fighting too many compiler errors :)

Comments (6)

2007-03-04

Erlang in Print (almost)

Filed under: erlang — bob @ 1:34 pm

Erlang finally has a book out in print again (almost): Programming Erlang, written by Joe Armstrong. The best part of the story is that the publisher is allowing a combo purchase of a beta PDF plus the print book when it's available. How cool is that? I think this model makes a lot of sense with the speed that technologies move these days. Imagine how much they could have made riding the Ruby on Rails hype train a few months earlier than any other publisher with a 70% finished beta PDF plus a print book when available?

I've purchased the book and read through what's available in the beta PDF so far. It definitely does not disappoint: this is the Erlang book and it will be for some time to come.

If you're particularly interested I also highly recommend reading Joe Armstrong's doctoral thesis paper (Making Reliable Systems in the Presence of Software Errors), which has a lot of overlap with what the beta PDF covers and what the book will cover when it's finished. There's also a lot of interesting information in the thesis that may or may not end up in the book such as the history and evolution of Erlang and some interesting case studies.

Comments (2)

2006-11-21

MochiAds - Flash Game Ad Network

Filed under: Genshi, MTASC, Pylons, SQLAlchemy, actionscript, erlang, flash, nginx, python — bob @ 2:00 pm

MochiAds is finally out the door! There's a pretty good summary of what we're doing on TechCrunch.

For the technically inclined, the UI for MochiAds is built with Pylons, Genshi and SQLAlchemy. The secret sauce is a combination of Python and Erlang code, and we've got Nginx as the gatekeeper. I've got nothing but good things to say about this whole stack. Erlang's module reloading, concurrency oriented programming model, and pattern matching has really been a dream.

For Flash game developers, we support ActionScript 1, ActionScript 2, and MTASC. Flash movies must be published in Flash 6 or later. Currently we support two ad formats: preloaders and interstitial ads. Either way it's just one line of code to toss in the movie.

The really Big Deal is that Flash game developers can make money off of their work throughout the entire lifespan of the project, especially if it spreads virally. It's no longer a bad thing if a portal "steals" the game without paying a licensing fee, the developer still gets paid!

Comments (7)

2006-09-21

Erlang Binary Performance

Filed under: c, erlang, python — bob @ 2:28 pm

I was benchmarking egeoip today, which is my from-scratch Erlang geolocation library. It uses the MaxMind GeoLite City database, which has implementations in a bunch of other languages so it's great to compare with. The results were rather surprising to me, because I hadn't previously done any benchmarking of Erlang performance.

The test environment is a MacBook Pro 2ghz, Mac OS X 10.4.7, Erlang R11B-1 w/ HiPE enabled, Python 2.4.3 (using their GeoIP Python API, which is written in C). I do have other processes running (namely iTunes), but the benchmark is fair because the background load is consistent throughout the tests.

Erlang, BEAM:: ~13k geolocations/sec
Python/C:: ~18k geolocations/sec
Erlang, HiPE:: ~44k geolocations/sec

As you can see, Erlang holds it own against Python w/ C extensions, and it can mop the floor with it when using the HiPE compiler. Erlang clearly kicks some serious ass at working with binaries, both in syntax and performance. The only work I had to do to make it faster was c(egeoip, [native]).

Note that I've only been using Erlang for a few weeks and have not done any profiling or performance tuning at all beyond what I assumed would be the fastest way given the documentation I had read.

Update

After looking at Shark results across the two implementations, it seems that the GeoIP API default settings are pessimistic for benchmarking purposes and that most of the time was spent in syscalls (Erlang looked like its time was spent in GC). A fair comparison would be using the memory cache option, which gets even better performance.

Python/C (Memory Cache):: ~117k geolocations/sec

This is a lot more in line with what I expected, but I'm still impressed that Erlang w/ HiPE can get nearly 40% of the speed of C when scanning through a 25MB array of bytes. I'm pretty sure I can make some algorithmic improvements to the code (which the C implementation may or may not do), so we'll see how close I can get.

Update

After spending a while with eprof doing some profile driven optimizations, I was able to considerably speed up the Erlang code. The biggest BEAM optimization was moving the giant tuples out of function bodies, apparently BEAM is rather naive about that and decides to actually create and garbage collect them on every call in certain cases. Some other optimizations were done to the way it looks for null terminators and a hyper-optimized fast-path for IPv4 string to long conversion.

Given the API I could cheat and parse out some of the data when the user asks for it, rather than at record fetch time. This would make the benchmark incredibly fast, but it would be an unfair comparison with the Python/C version. I'll probably end up doing that anyway, since I'm typically looking for just the country of an IP address.

I still haven't really done any algorithmic optimizations to the lookup, but here's the numbers:

Erlang, BEAM:: ~44k geolocations/sec
Erlang, HiPE:: ~64k geolocations/sec

This brings the BEAM performance up to about 38% of C/Python and HiPE up to 55%. Not bad!

Comments (5)

from future import *
Kailash and Friends Kailash Kher Kaipa

2008-03-08

Exploring Erlang @ C4[1] video

2007-12-17

Printing floats with Erlang

Using the mochiweb project skeleton

2007-11-07

mochiweb - another faster web server

2007-08-11

Exploring Erlang @ C4[1]

2007-03-16

Distel and Erlang mode for Emacs, on Mac OS X

2007-03-14

Erlang Mode for Emacs

2007-03-04

Erlang in Print (almost)

2006-11-21

MochiAds - Flash Game Ad Network

2006-09-21

Erlang Binary Performance

Update

Update

Links