Monday, March 09, 2009

Doctest goes double-digits

Several months ago I happened upon the original posting by Tim Peters announcing doctest. Realizing that it would soon be the 10th anniversary of its creation, I did the only thing a doctest fan could do, I put the auspicious date on my calendar.

Friday was the big day. While musing as to the best way to mark the occasion (an ode to doctest? perhaps test-driven cake?), I was surprised to see Tim walk into the office. Surely he had come by celebrate in person!

Not quite. He just dropped in to see how everyone at ZC has been getting along without him for the last couple of years. He also did a good job reminding us of how funny he is. Oh well, there's always 2019.
Sunday, March 09, 2008

ICP for Faster Web Apps

ICP (Internet Cache Protocol, RFC 2186) is a simple protocol intended to let one cache server ask it's peers whether or not they have a non-stale copy of a particular object ("object" is cache-speak for the thing on the other end of a URL), but it can also be used to make the most of your web apps.

The RFC is pretty easy reading as such things go, but I'll summarize here: if a cache server doesn't have a local copy of the requested object it can be configured to send out a request (usually over UDP) to all of it's siblings. The request includes little more than the requested URL. All the sibling caches send back HIT, MISS, or MISS_NOFETCH (more or less, read the RFC for the details). Here are the definitions of those from the RFC:

ICP_OP_HIT

An ICP_OP_HIT response indicates that the requested URL exists in this cache and that the requester is allowed to retrieve it.

ICP_OP_MISS

An ICP_OP_MISS response indicates that the requested URL does not exist in this cache. The querying cache may still choose to fetch the URL from the replying cache.

ICP_OP_MISS_NOFETCH

An ICP_OP_MISS_NOFETCH response indicates that this cache is up, but is in a state where it does not want to handle cache misses.
The first server to come back with a HIT will be sent the request, if all servers reply with a MISS, the first server to respond with a MISS will get the request. Servers that send back a MISS_NOFETCH will not be sent this request. The response to any particular request has no bearing on any future request; for example, a server can reply with a MISS_NOFETCH for a URL one moment and a HIT for the same URL a split second later.

This is all very nice for cache servers, but how does this help speed up my dynamic web app? Well, forward proxies can also use ICP to query "origin servers" (non-cache web servers). Why is that interesting? Lets say you have multiple origin servers, being good app servers they each have local caches (for data they request from databases, computed page fragments, etc.). If requests are randomly sent to the various origin servers, the app server cache hit rate will be pretty abysmal. The negative affects are intensified if there are many back-end servers and if the working data set is much larger than what will fit in the caches, meaning you get a high rate of cache churn; otherwise useful data being evicted because there's not enough room to keep it.

It's likely that the total cache size is larger than the working set, but since the requests are randomly distributed, poor use is made of the caches. It would be nice if a request that needed a particular subset of the data went to a server that was likely to have the needed data already cached. In other words, you want some flavor of "request affinity".

How you decide to affinitize requests depends on how your data is structured. For the systems I'm working on, we're serving hundreds of web sites out of a single large database, so site affinity makes the most sense. This means that all requests for a particular site should tend to go to the same server or set of servers (if there is more load than a single origin server can handle).

The sequence of events goes something like this: a request comes in, an ICP request is sent to all the origin servers and the results are used to direct the request. If a server has handled a request for that site before, it sends back a HIT, if not, then a MISS.

If a server would otherwise send a HIT but is too busy at the moment, it'll send a MISS instead, hoping that another server HITs. If none do, then the fastest MISS will get the request, which may mean that a new server gains an affinity for that site.

When used like this ICP can provide dynamic load balancing and data set partitioning in a simple, scalable fashion. If you want to play with ICP you'll need a forward proxy that supports ICP, like Squid and an ICP server like my Python implementation: zc.icp (developed at Zope Corporation, but not Zope-specific).
Saturday, February 23, 2008

Programmable Python Syntax via Source Encodings

Because of my recent forays into Common Lisp and Haskell this recent recipe on ASPN piqued my interest. It demonstrates using Python source file encodings as a hook for introducing alternate Python syntax. Here's the simplest example of the technique I could come up with:

codec.py:
import codecs

class StreamReader(codecs.StreamReader):
def decode(self, input, errors='strict'):
output = input.replace('until ', 'while not ')
output = output.replace('++', '+= 1')
return unicode(output), len(input)

def get_my_codec(name):
if name == 'play-language':
return (codecs.utf_8_encode, None, StreamReader, None)

codecs.register(get_my_codec)

The above creates a codec called "play-language" and defines a very, very simple-minded source code transformation. For actual use, you'd need to work harder at parsing the code.

Now, if you create a file named play1.py with these contents:
# coding=play-language

c = 0
until c == 10:
print c
c++

And execute this from a command line:
python -c 'import codec1;import play1'

You'll get this output:
0
1
2
3
4
5
6
7
8
9
Monday, April 16, 2007

Sleep considered harmful

Even though I'm not a big fan of biological sleep, in this post I'm talking about the "sleep" function available in Python. While we've all been warned that sleep may take longer to return than requested, most people don't realize just how course-grained sleep is on their operating system.

Here's a quiz: What is the minimum sleep interval on your operating system (other than 0)?

a) more than 10ms
b) 10ms
c) 4ms
d) 1ms
e) 0.1ms
f) less than 0.1ms

The answer: it depends. :)

On most flavors of Windows the answer is probably "a" (see below of a script to measure this for yourself). On operating systems with the Linux kernel the answer is one of 10, 4, or 1ms.

Why the variability on Linux? Linux uses a programmable timer to tick at a certain frequency, and then performs periodic tasks on each tick. The more ticks the more often it can task switch between processes (among other things), but there's also overhead with handling the interrupt, so higher tick rates reduce throughput.

Some distributions think a "server" kernel should have a tick frequency of 100Hz to allow for more to be done between interrupts, others think the clock should tick at 1000Hz to allow for more rapidly switching between tasks. Then there's the middle ground of 250Hz that some distributions use for desktop machines (giving 4ms sleep granularity). This isn't a simple issue, much discussion has taken place over the default setting and what the consequences are.

This is all very interesting, but what does it have to do with sleep? Python (indirectly) implements sleep based on the OS tick frequency. So if your OS has a frequency of 100Hz and you ask for a 15ms sleep, on the first tick you still have 5ms left on your sleep so you won't get woken up until the next tick. You asked for 15ms and got 20. What do you think would have happened if you'd asked for 1 or even 0.01ms instead? That's right, you'd have gotten 10ms.

Now, if you're waiting for a reply from a socket and the average round-trip time is around 1ms and you sleep for 1ms after sending the request, you just made your network app 10 times slower than it needs to be. This is where everyone says "use select instead"; and they'd be right. "select" represents the event-based approach that is generally superior to a poll-based approach involving sleep (libevent for example).

What if you really do need to sleep for less time than the sleep will allow? On unix-like operating systems there's a function (normally) available that will do what you want: "nanosleep". Using nanosleep you can request (and get) much smaller sleep times. Windows also has APIs to do finer-grained sleeping. Either should be accesible through ctypes (included in Python 2.5).

Curious what your minimum sleep time is? Here's a little Python script that will tell you:


import time

LOOPS = 100

def do_sleep(min_sleep):
total = 0.0
for i in range(LOOPS):
c = time.time()
time.sleep(min_sleep)
x = time.time()
total = total + x - c
return total / LOOPS

min_sleep = 0.000001
result = None
while True:
result = do_sleep(min_sleep)
if result > 2 * min_sleep:
break
min_sleep *= 2

print result
Wednesday, February 21, 2007

Reverse Interviewing at PyCon

Other than participating in the Testing Tools Panel at PyCon I have a secret recruiting mission. But, instead of trying to get a bunch of resumes I'm going to try to get as many smart people as possible to interview me about whether or not they would like to work at Zope Corp.

So, if you're going to PyCon and you're a talented programmer, find me (Benji York) or Jim Fulton and tell us you're interested and a bit about yourself and we'll answer your questions about the positions and the company. Zope 3 knowledge isn't required (although it's a plus).

If you don't get to talk with us (or aren't going to PyCon this year) and are interested, there's always the Zope Corp careers page and the ever popular careers@zope.com.
Sunday, February 18, 2007

Testing Tools Panel at PyCon plus some Sahi

For those attending PyCon this year and interested in testing, there's the Testing Tools Panel. The panel is being moderated by Grig Gheorghiu, who's done a great job of getting a nice mixture of people with different takes on testing and was nice enough to invite me to participate. The panel info page also has an invitation for people to add their own questions, so feel free to contribute.

While on the subject of testing, a coworker of mine introduced me to Sahi recently. Sahi is a Selenium-like system for in-browser testing. It seems to have at least one advantage over Selenium: the tests are written in (a slightly preprocessed) JavaScript, but isn't quite as well-known yet. The site has a screencast which is a pretty good into to the system.
Saturday, March 25, 2006

Zope 3 Quick Start updated for Zope 3.2

I've (finally) uploaded a version of the Zope 3 quick start guide that's been updated to match Zope 3.2. This version includes pointers on installing from a release (as well as a checkout as before). I've also fleshed out and reworked some of the text a bit. As is customary, there are both the (source) ReST version and a (generated) HTML version available.

I'd like to write a follow-on to the quick start, but need to decide on one of two directions, so feedback would be nice. Would people more like a "show me how to do simple stuff" recipe-like collection, or more of a "how to do sophisticated stuff with Zope 3" document?

If the former, it would be a good exercise to see what is helpful and what needs to be tweaked in Zope 3 to make it easy for the "middle class" developers that aren't building highly complex apps.

If the latter, it'll be more of a theoretical discussion of interfaces, adapters, views, schemas, utilities, etc. and how all of those things fit together. It'd also probably talk about using (most) of those things outside of Zope 3, as several people are already.

So, what do you want? Maybe a bit of both?