Python iterators and sentinel values
A little known feature of PEP 234: Iterators (implemented in Python 2.1 and later) is the alternate form:
iter(callable, sentinel)
This form creates an iterator out of any callable, and will stop when an exception is raised or the sentinel value is returned by the callable. Additionally, raising StopIteration will stop the iteration as with a normal iterator. A sentinel value is a special value that indicates the end of a sequence. This can be any value, but the most commonly useful values are likely to be None, 0, or ''.
I haven't seen much code in the wild that takes advantage of this form of iter, but an excellent usage would be to replace this common idiom:
while True: data = fileobj.read(BLOCKSIZE) if not data: break # do something with data here
Using the sentinel form of iter, it can be rewritten as an iterator with more obvious control flow:
for data in iter(lambda: fileobj.read(BLOCKSIZE), ''): # do something with data here
Of course, nobody likes a lambda, but you could wrap this up as a little utility function, or you could wait for PEP 309: Partial Function Application (on the standards track, will be in Python 2.5). With partial it would look like this:
for data in iter(partial(fileobj.read, BLOCKSIZE), ''): # do something with data here
Thanks for the post, very informative.
I still don’t have my head wrapped around Ruby’s code blocks, but this looks similar to some of the examples I’ve seen.
Comment by Matt — 2005-06-15 @ 11:52 am
Until PEP 309 is implemented, functions as the one in this message can be used:
Comment by Christos Georgiou — 2005-06-24 @ 4:28 am
This form creates an iterator out of any callable, and will stop when an exception is raised…
At first I was excited by this, but testing on 2.4 reveals that it’s not true, or at least that it’s misleading. Consider:
>>> tuple(iter(range(5).pop, None))“, line 1, in ? Traceback (most recent call last):
File "
IndexError: pop from empty list
Based on your advice, I had expected the IndexError to be caught by the iterator and reraised as a StopIteration. That would be convenient but perhaps in some cases confusing; that’s not how it works. I guess you really meant to say what you said in the next sentence, that raising StopIteration would stop the iteration. That’s true but a bit trivial. Cheers!
Comment by Jess Austin — 2005-07-12 @ 4:00 pm
I didn’t say that exceptions would be converted into StopIteration, I just said the iteration will stop
Comment by Bob Ippolito — 2005-07-12 @ 4:06 pm
As with any uncaught exception, all execution will stop, not just the iteration. (maybe that’s what you meant?) Only a StopIteration will be caught by the iteration context. Cheers!
Comment by Jess Austin — 2005-07-12 @ 4:32 pm