Steve Vinoski's Blog

You may have already seen this on InfoQ or on Stefan’s blog, but the video of my 2008 QCon London presentation “REST, Reuse, and Serendipity” is now available.

Here it is, just a little over a year after I gave that presentation, and REST continues to deliver extremely well for my work. For example, I just finished a meeting a couple hours ago where some client code needs to interact with a particular part of my system via HTTP but wants XML instead of the JSON currently provided. Simple — it’s just a different representation of the same resources, and of course it wasn’t hard to guess months ago that such a need would eventually come down the pike, so fitting it into the system will be trivial. Can you imagine the hoops one would have to jump through with typical RPC-oriented systems for this case, where the marshaling format is typically tied to the protocol and you can’t change either one? You’d have to write a new service interface with new verbs and new messages and get the client side to use it, or write client-side wrappers around whatever you already have and ask the client programmers to somehow incorporate those wrappers into their code. Either way, there’s simply no chance of reusing existing agreements; instead, both sides require non-trivial specialization.

One problem I noticed, though, was that the client developers asked for a “REST-like interface” and also for a document listing all resource URIs, and for each one, the HTTP verbs that apply to it, the representations available from it, and what status codes to expect from invoking operations on it. Those two requests are sort of mutually exclusive, depending on what “REST-like” means; for a proper RESTful system, you don’t need a document like that, at least not the type of document they’re asking for.

A couple months ago I wrote that my Internet Computing column, “Toward Integration,” was going to end. Indeed it did, but I’m very pleased to report that I’ve replaced it with a whole new column entitled “The Functional Web.”

The inaugural column is imaginatively titled “Welcome to ‘The Functional Web'” and it provides the background for what I plan to cover in the column going forward. All signs indicate that functional languages are garnering significant interest these days. At QCon last week, for example, it seemed like everyone was talking about them, and Real World Haskell didn’t just win a Jolt Award for nothing. Most of my work these days involves web development with Erlang, and given my general interest in functional programming, the combination of these two areas seemed like the perfect direction for a new column.

If you’re a functional programmer working in the web space and have a knack for writing, drop me an email. If you’ve got a good proposal, I’d be happy to either co-author something with you or have you serve as a guest columnist for an issue.

Finally, I’d like to thank the readers of “Toward Integration” very much for sticking with me for the past seven years. That’s a long time, but rest assured your many positive comments and your feedback kept me going — I deeply appreciate it. I really hope you’ll join me for “The Functional Web” and that you’ll keep that excellent feedback coming — it’s going to be interesting and I guarantee we’re all going to learn a few things along the way.

QCon is always very, very good, but QCon London 2009 last week was the best one yet. Highlights:

• Ola Bini‘s “Emerging Languages in the Enterprise” track on Wednesday had some great talks, especially Rich Hickey’s Clojure talk. Ola’s deep knowledge and love of programming languages made him the perfect host for this track, and he put together a brilliant lineup.
• Speaking of Rich, I consider myself very lucky to have gotten to meet and spend a fair amount of time with him. He’s very bright, talented, knowledgeable, and experienced, and both of his talks were outstanding.
• I introduced Rich to Joe Armstrong at the conference party Wednesday evening and they spent the next few hours talking at length about functional programming, their respective languages, VM implementation issues, concurrency issues, etc. Ola jumped in as well. They also continued the conversation the next day. I just sat back, listened, and learned.
• Getting to spend time again with Joe was excellent. He always has incredibly useful analyses and opinions to express, and in general is fun to be around and easy to learn from.
• I also finally got to meet Ulf Wiger, Erlang developer extraordinaire, in person. He’s a laid back guy, quite well-informed and a deep thinker who can cover a wide variety of topics in amazingly useful detail. His talk on multicore programming in Erlang covered cutting edge Erlang development and presented some very difficult concurrency issues.
• Ulf’s talk, as well as Rich’s second talk, which was on persistent data structures and managed references, were part of Francesco Cesarini‘s “Functional and Concurrent Programming Languages Applied” track on Thursday. I met Francesco, who like Ulf is one of the world’s top Erlang developers, at QCon London last year. He assembled a great track for this conference, with Rich’s and Ulf’s back-to-back talks being way more than enough to sober up any developer who thinks that multicore is not an issue and that today’s methods for dealing with concurrency will continue to work just fine. Best of luck with that!
• Sir Tony Hoare‘s talk about the null reference being his “billion dollar mistake” was great because of all the detail he recounted from some of the early days of computing. He was both informative and entertaining. I was also impressed with Ulf during this talk, whom Professor Sir Hoare invited to come up to the front and present what turned out to be a pretty convincing argument in favor of the null reference.
• Paul Downey‘s talk on the downsides of standardization was by far the most humorous talk I heard, perfect to close out the track, but it also presented a number of hard-won useful lessons about the perils of standardization efforts.

As with all QCon conferences, there were a bunch of interesting tracks running in parallel, and unfortunately I still haven’t figured out how to be in multiple places at once. I had to miss Michael Nygard‘s talk, for example, because my own talk got moved to the same time slot as his.

My talk (PDF) covered the history of RPC, why it got to be the way it was, and why the forces that created it really aren’t all that viable anymore.

The final conference panel was by far the most inventive panel I’ve ever been on. Modeled after the British game show “It’s a Bullseye!” and hosted by none other than Jim Webber, it had contestants from the audience throwing darts to become eligible for a prize. Once a contestant became eligible, Jim would ask the panel — Michael Nygard, Ian Robinson, Martin Fowler, and me — to answer a question submitted by conference attendees either earlier during the conference or live via Twitter. Based on our answers, audience members held up either a green card if they liked the answers or a red one if they didn’t, and if the majority was green, the contestant would win a book. The questions were hard! We had only two minutes each to answer, which for some questions seemed like an eternity but for most was way too short. Anyway, it was great fun, and given how many there were in the audience after three grueling conference days and how much they seemed to be enjoying themselves, it worked very, very well.

If you have any interest at all in leading edge software and computing topics being presented by the world’s most knowledgeable speakers in a fun atmosphere, go to QCon. I guarantee you won’t be disappointed.

As my bio explains, I’m a member of technical staff at Verivue. If you ever visited the old Verivue website or heard me speak at conferences over the past couple years, you’ll know we divulged no details of what we’re developing. Our old website showed just a curtain that was barely cracked open with a light shining through, and what little text the website presented was intentionally vague. If only I had a dollar for each time I found myself in a conversation where everyone was trying to guess what we were doing!

Well, guess no more — our new website and the following articles provide the details:

• Verivue Formed to Address the Rapid Growth of IP Video
• VOD Goes Super-Size
• Verivue Surfaces With Comcast Backing

Now you know. :-)

Erlang’s heart feature provides a heartbeat-based monitoring capability for Erlang runtime systems, with the ability to restart a runtime system if it fails. It works reasonably well, but one issue with it is that if an error occurs such that it causes repeated immediate runtime crashes, heart will happily keep restarting the runtime over and over again, ad infinitum.

For yaws 1.80, released a few days ago on Feb. 12, I added a check to the heart setup in the yaws startup script to prevent endless restarts. I thought I’d share it here because it’s useful for Erlang systems in general and is in no way specific to yaws. It works by passing startup information from one incarnation to the next, checking that information to detect multiple restarts within a given time period. We track both the startup time and the restart count, and if we detect 5 restarts within a 60 second period, we stop completely. This is not to say that yaws is in dire need of this capability — it’s extremely stable in general and 1.80 in particular is a very good release — but I added it mainly because other Erlang apps sharing the same runtime instance as yaws may not enjoy that same high level of stability, especially while they’re still under development.

The command heart runs to start a new instance is set in the HEART_COMMAND environment variable. For yaws, it’s set like this (I’ve split this over multiple lines for clarity, but it’s just one line in the actual script):

HEART_COMMAND="${ENV_PGM} \
  HEART=true \
  YAWS_HEART_RESTARTS=$restarts \
  YAWS_HEART_START=$starttime \
  $program "${1+"$@"}

where

• ${ENV_PGM} is /usr/bin/env, which allows us to set environment variables for the execution of a given command.
• HEART is an environment variable that we use to indicate the command was launched by heart.
• YAWS_HEART_RESTARTS is an environment variable that we use to track the number of restarts already seen. The yaws script initially sets this to 1 and increments it for each heart restart.
• YAWS_HEART_START is an environment variable that we use to track the time of the current round of restarts. This is tracked as UNIX time, obtained by the script via the “date -u +%s” command.
• $program is the yaws script itself, i.e., $0.
• ${1+"$@"} is a specific shell construct that passes all the original arguments of the script unchanged along to $program.

The yaws script looks for HEART set to true, indicating that it was launched by heart. For that case, it then checks YAWS_HEART_RESTARTS and YAWS_HEART_START to see how many restarts we’ve seen since the start time. We get the current UNIX time and subtract the YAWS_HEART_START time; if it’s less than or equal to 60 seconds and the restart count is 5, we exit completely without restarting the Erlang runtime. Otherwise we restart, first adjusting these environment variables. If the restart count is less than 5 within the 60 second window, we increment the restart count and set the new value into YAWS_HEART_RESTARTS but keep the same YAWS_HEART_START time. But if the current time is more than 60 seconds past the start time, we reset YAWS_HEART_RESTARTS to 1 and set a new start time for YAWS_HEART_START. Look at the yaws script to see the details of this logic — scroll down to the part starting with if [ "$HEART" = true ].

Note that this approach is much like the way Erlang receive loops generally track state, by recursively passing state information to themselves.