{"id":17,"date":"2007-10-07T13:29:40","date_gmt":"2007-10-07T18:29:40","guid":{"rendered":"http:\/\/steve.vinoski.net\/blog\/2007\/10\/07\/wide-finder-in-python\/"},"modified":"2007-10-10T22:41:37","modified_gmt":"2007-10-11T03:41:37","slug":"wide-finder-in-python","status":"publish","type":"post","link":"https:\/\/steve.vinoski.net\/blog\/2007\/10\/07\/wide-finder-in-python\/","title":{"rendered":"Wide Finder in Python"},"content":{"rendered":"<p><a href=\"http:\/\/effbot.org\/\">Fredrick Lundh<\/a> has posted <a href=\"http:\/\/effbot.org\/zone\/wide-finder.htm\">some really nice code<\/a>, along with detailed explanations, showing Tim&#8217;s <a href=\"http:\/\/www.tbray.org\/ongoing\/When\/200x\/2007\/09\/20\/Wide-Finder\">Wide Finder<\/a> implemented in various ways in Python, using both threads and processes. I ran his <a href=\"http:\/\/svn.effbot.org\/public\/stuff\/sandbox\/wide-finder\/wf-6.py\">wf-6.py<\/a> on an 8-core 2.33 GHz Intel Xeon Linux box with 8GB of RAM, and it ran best at 5 processes, clocking in at 0.336 sec. Another process-based approach, <a href=\"http:\/\/svn.effbot.org\/public\/stuff\/sandbox\/wide-finder\/wf-5.py\">wf-5.py<\/a>, executed best with 8 processes, presumably one per core, in 0.358 sec. The multithreaded approach, <a href=\"\">wf-4.py<\/a>, ran best with 5 threads, at 1.402 sec (but also got the same result with 19 threads, go figure). Using the same dataset, I get 11.8 sec from <a href=\"\/blog\/2007\/09\/29\/more-file-processing-with-erlang\/\">my best Erlang effort<\/a>, which is obviously considerably slower. I used the bash shell <code>time<\/code> built-in for all measurements, for consistency.<\/p>\n<p>I&#8217;ve been coding in Python for years, but lately I&#8217;ve been using it a lot, and can&#8217;t get enough of it. It&#8217;s <em>so<\/em> clean, and as Fredrick&#8217;s code shows, it&#8217;s got some extremely powerful capabilities that are also extremely easy to use. It&#8217;s obviously smoking fast as well. Some folks who commented on some of my previous blog entries seem to equate programs written using dynamic languages like Python with &#8220;unmaintainable one-off solutions.&#8221; If that&#8217;s your experience with such languages, blame the programmer, not the language.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Fredrick Lundh has posted some really nice code, along with detailed explanations, showing Tim&#8217;s Wide Finder implemented in various ways in Python, using both threads and processes. I ran his wf-6.py on an 8-core 2.33 GHz Intel Xeon Linux box with 8GB of RAM, and it ran best at 5 processes, clocking in at 0.336 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12,11],"tags":[],"class_list":["post-17","post","type-post","status-publish","format-standard","hentry","category-performance","category-python"],"_links":{"self":[{"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/posts\/17","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/comments?post=17"}],"version-history":[{"count":0,"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/posts\/17\/revisions"}],"wp:attachment":[{"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/media?parent=17"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/categories?post=17"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/steve.vinoski.net\/blog\/wp-json\/wp\/v2\/tags?post=17"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}