Wide Finder in Python
October 7th, 2007 | Published in performance, python | Add to del.icio.us
Fredrick Lundh has posted some really nice code, along with detailed explanations, showing Tim’s Wide Finder implemented in various ways in Python, using both threads and processes. I ran his wf-6.py on an 8-core 2.33 GHz Intel Xeon Linux box with 8GB of RAM, and it ran best at 5 processes, clocking in at 0.336 sec. Another process-based approach, wf-5.py, executed best with 8 processes, presumably one per core, in 0.358 sec. The multithreaded approach, wf-4.py, ran best with 5 threads, at 1.402 sec (but also got the same result with 19 threads, go figure). Using the same dataset, I get 11.8 sec from my best Erlang effort, which is obviously considerably slower. I used the bash shell time built-in for all measurements, for consistency.
I’ve been coding in Python for years, but lately I’ve been using it a lot, and can’t get enough of it. It’s so clean, and as Fredrick’s code shows, it’s got some extremely powerful capabilities that are also extremely easy to use. It’s obviously smoking fast as well. Some folks who commented on some of my previous blog entries seem to equate programs written using dynamic languages like Python with “unmaintainable one-off solutions.” If that’s your experience with such languages, blame the programmer, not the language.