Steve Vinoski's Blog

IEEE Computing Now is now featuring a new podcast of my first Functional Web column from the March/April issue of Internet Computing. That was the first issue of this new column, so it’s an introduction to the main topic of the column and what future editions will cover. It describes why I think functional programming languages are making a comeback, briefly compares them to current popular languages, explains why I believe they work well for web development, and describes some of the technologies and ideas future columns will explore.

The May/June issue of my Internet Computing column, “The Functional Web,” is now available. This one’s called “Scala and Lift — Functional Recipes for the Web” (PDF) and I had the great fortune of writing it together with Debasish Ghosh.

I’ve admired Debasish’s blog for awhile now, as he’s always exploring interesting angles and features of different functional programming languages and techniques. I emailed Debasish completely out of the blue and asked if he’d be interested in writing something together, and he jumped right in.

If you ever get a chance to work with Debasish, I highly encourage it. He made the whole task quite easy because he cranked out the first draft so quickly, and as we iteratively edited the column, he turned each draft around literally overnight. He and I live on opposite sides of the world, which worked out well because it allowed one of us to write and edit while the other slept. Meeting deadlines is key to delivering this column on time for each issue of IC, and so I really appreciated Debasish’s attention to our schedule. The fact that he writes well, as you can see in his blog, didn’t hurt at all either. :-)

Given their collective richness, it’s hard to cover both Scala and Lift in a single column, but I think we managed to describe the important issues and features. As always, if you have any feedback on this column, post it here in a comment or email me.

You may have already seen this on InfoQ or on Stefan’s blog, but the video of my 2008 QCon London presentation “REST, Reuse, and Serendipity” is now available.

Here it is, just a little over a year after I gave that presentation, and REST continues to deliver extremely well for my work. For example, I just finished a meeting a couple hours ago where some client code needs to interact with a particular part of my system via HTTP but wants XML instead of the JSON currently provided. Simple — it’s just a different representation of the same resources, and of course it wasn’t hard to guess months ago that such a need would eventually come down the pike, so fitting it into the system will be trivial. Can you imagine the hoops one would have to jump through with typical RPC-oriented systems for this case, where the marshaling format is typically tied to the protocol and you can’t change either one? You’d have to write a new service interface with new verbs and new messages and get the client side to use it, or write client-side wrappers around whatever you already have and ask the client programmers to somehow incorporate those wrappers into their code. Either way, there’s simply no chance of reusing existing agreements; instead, both sides require non-trivial specialization.

One problem I noticed, though, was that the client developers asked for a “REST-like interface” and also for a document listing all resource URIs, and for each one, the HTTP verbs that apply to it, the representations available from it, and what status codes to expect from invoking operations on it. Those two requests are sort of mutually exclusive, depending on what “REST-like” means; for a proper RESTful system, you don’t need a document like that, at least not the type of document they’re asking for.

A couple months ago I wrote that my Internet Computing column, “Toward Integration,” was going to end. Indeed it did, but I’m very pleased to report that I’ve replaced it with a whole new column entitled “The Functional Web.”

The inaugural column is imaginatively titled “Welcome to ‘The Functional Web'” and it provides the background for what I plan to cover in the column going forward. All signs indicate that functional languages are garnering significant interest these days. At QCon last week, for example, it seemed like everyone was talking about them, and Real World Haskell didn’t just win a Jolt Award for nothing. Most of my work these days involves web development with Erlang, and given my general interest in functional programming, the combination of these two areas seemed like the perfect direction for a new column.

If you’re a functional programmer working in the web space and have a knack for writing, drop me an email. If you’ve got a good proposal, I’d be happy to either co-author something with you or have you serve as a guest columnist for an issue.

Finally, I’d like to thank the readers of “Toward Integration” very much for sticking with me for the past seven years. That’s a long time, but rest assured your many positive comments and your feedback kept me going — I deeply appreciate it. I really hope you’ll join me for “The Functional Web” and that you’ll keep that excellent feedback coming — it’s going to be interesting and I guarantee we’re all going to learn a few things along the way.

A few months back Klacke gave me committer rights for Yaws. I’ve made a few fixes here and there, including adding support for passing “*” as the request URI for OPTIONS, enabling OPTIONS requests to be dispatched to appmods and yapps, and completing a previously-submitted patch for configuring the listen backlog. Klacke has just started putting a test framework into the codebase and build system so that contributors can include tests with any patches or new code they submit, and I’ve contributed to that as well.

The biggest feature I’ve added to date, though, is a new linked-in driver that allows Yaws to use the sendfile system call on Linux, OS X, and FreeBSD. I never wrote a linked-in driver before, so I was happy and fortunate to have an Erlang expert like Klacke providing hints and reviewing my code.

I did some preliminary testing that showed that sendfile definitely improves CPU usage across the board but depending on file size, sometimes does so at the cost of increasing request times. I used my otherwise idle 2-core 2.4GHz Ubuntu 8.04.1 Dell box with 2 GB of RAM, and ran Apache Bench (ab) from another Linux host to simulate 50 concurrent clients downloading a 64k data file a total of 100000 times. I saw that user/system CPU on the web host tended to run around 33%/28% without sendfile, while with sendfile it dropped to 22%/17%. The trade-off was request time, though, where each request for the 64k file averaged 0.928ms with sendfile but 0.567ms without. With larger files, however, sendfile is slightly faster and still has better CPU usage. For example, with a 256k file, sendfile averaged 2.251ms per request with user/system CPU at 8%/16% whereas it was 2.255ms and 16%/27% CPU without sendfile, which makes me wonder if the figures for the 64k file are outliers for some reason. I performed these measurements fairly quickly, so while I believe they’re reasonably accurate, don’t take them as formal results.

On my MacBook Pro laptop running OS X 10.5.6, CPU usage didn’t seem to differ much whether I used sendfile or not, but requests across the board tended to be slightly faster with sendfile.

I ran FreeBSD 7.0.1 in a Parallels VM on my laptop, and there I saw significantly better system CPU usage with sendfile than without, sometimes as much as a 30% improvement. Requests were also noticeably faster with sendfile than without, sometimes by as much as 17%, and again depending on file size, with higher improvements for larger files. User CPU was not a factor. All in all, though, I don’t know how much the fact that I ran all this within a VM affected these numbers.

Given that Yaws is often used for delivering mainly dynamic content, sendfile won’t affect those cases. Still, I think it’s nice to have it available for the times when you do have to deliver file-based content, especially if the files are of the larger variety. Anyway, I committed this support to the Yaws svn repository back around December 21 or so. If you’d like to do your own testing, please feel free — I’d be interested in learning your results. Also, if you have ideas for further tests I might try, please leave a comment to let me know.