Steve Vinoski's Blog

My latest Internet Computing column has been available since last Friday. It’s entitled “RPC and REST: Dilemma, Disruption, and Displacement” (PDF, HTML) and like my previous 2008 columns, it explores another angle of the “RPC vs. REST” debate.

Since previous columns have covered many of the technical angles, this time I present the debate from the technology adoption angle. As the abstract for the column says, many technologists tend to treat such debates as if they’re purely technical, but of course they’re never that black-and-white. What’s often behind some of the raging “technical” debates we’ve all seen or experienced is simply the difference between the arguing parties in their relative positions along the Technology Adoption Lifecycle curve. Nobody would be surprised at a disagreement over technology between someone classified as an early adopter or visionary (from the far left of the curve) and someone classified as a technology skeptic (from the far right), yet we always seem surprised when two people whose preferences aren’t too far apart on the curve — from the opposite edges of the mainstream band in the middle of the bell curve, for example — don’t see eye to eye, despite the fact that this sort of scenario is quite common. Even small differences in goals for adopted technologies and desired risk/reward trade-offs, along with the inevitable hidden and unstated assumptions resulting from such factors, can cause vigorous debate about what technology or approach is best for a given situation.

When it comes to published explanations of how innovation works and how technologies move along the adoption curve, my favorite author by far is Clayton Christensen. IMO all developers should study and learn from his books, specifically The Innovator’s Dilemma, The Innovator’s Solution, and Seeing What’s Next. All are amazingly insightful works that will open your eyes to how real-life markets react to technological change and advancement.

In this column I try to view and classify the “RPC vs. REST” debate based on Christensen’s theories about innovation and technology adoption. I hope you find it interesting, and as always, I welcome all constructive comments.

What if you knew a person who clearly knew an awful lot about large-scale software systems? Someone who earned their way to their knowledge and wisdom by spending days and nights analyzing, measuring and debugging when everyone else had already given up and gone home? Someone considered the “go-to guy” who could always fix things if the system were to encounter a mysterious problem so serious that it resulted in very real revenue loss for the customer for every second of downtime? And what if you could rig up some sort of device to easily tap into the way that person thinks, thereby getting a clear understanding of the rules by which he or she designs, builds, deploys, analyzes, measures, debugs, operates, manages, and upgrades those large-scale systems?

Michael Nygard is such a person, and luckily for the rest of us, he’s already created such a device: his book, entitled Release It! Design and Deploy Production-Ready Software.

Success is a problem we’d like our systems to have. So, we focus our energies on building them as best as we can. Since we tend to focus there, books also tend target only that initial development phase. Most books focus on analysis and design issues, or on the methods and processes for doing analysis, design, implementation, and testing. Sometimes you find a book that focuses on debugging, and some even get into performance and scalability concerns.

But I’ve never seen a book like this one. It addresses the truly hard part of software development, which is running a successful large-scale system in production.

The book is a patterns book, but the patterns it presents are concrete. First, there are patterns and anti-patterns for stability and capacity, intermixed with war stories about real-life large-scale systems that failed hard for reasons that wouldn’t ever occur at smaller scales. Nygard’s war stories bring the patterns and anti-patterns into focus, providing very real reasons for their existence, and hard-won proof that they do indeed work.

Following the patterns, the second half of the book discusses general design issues and operations issues. These parts build on the patterns and focus on gotchas, large and small, that keep small systems from growing into large ones. The general design chapters provide numerous suggestions for eliminating seemingly innocuous mistakes that can kill a system as scale increases. Finally, the operations sections put you in the shoes of the people who have to keep the system running in production, describing what you as a developer can do to make their lives easier (thereby making your own life easier too). Along the way, Nygard keeps us grounded by occasionally presenting more stories and scenarios, some of which include actual dollar figures for the choices and trade-offs made. Engineering always comes down to building the best system possible within the allotted budget, but how many books targeting software developers do you know that talk in concrete terms of costs?

Nygard’s writing style is clear and concise. If you read his book and read his blog, you’ll find the styles to be identical. I’m guessing his book’s copy editor didn’t have a lot of work to do to get this book ready for production.

Bottom line: if you care at all about production software systems, you’ll want to read this book.

Don says he agrees with a lot of what James has to say about Stefan’s recaps of the QCon SOA Track, listed below:

• Dan’s talk
• Pete’s talk
• Sanjiva’s talk
• my talk

I too agree with many of James’s insights. If you don’t regularly read his blog, you should subscribe immediately. I especially like this quote:

When I consider SOA and REST today, I do not see two competing visions, I see an evolution that has brought the industry back to a core set of fundamental design principles that we seemed to have lost sight of for a while.

How very true. However, I disagree with just one small point: his critique of the part of my talk where I said SOA has no constraints (as Mark Baker has been pointing out for years now).

James says that SOA has constraints, it’s just they’re just not well-documented. I know exactly what he’s saying, but I have to stick with what I said. The only official definition of SOA that I know of is the OASIS SOA Reference Model, which mentions several desirable properties for SOA systems, but it documents no constraints for inducing them AFAICS. Later in my talk, as Stefan’s notes indicate, I did say that I was willing to concede that the OASIS SOA RM promotes the client-server constraint. However, I was being really, really generous, since all the specification really does is hint that SOA systems should be distributed, and “distributed” has a lot more meanings than just “client-server.”

Since there are no official standard SOA constraints, the only constraints you get are the ones that each vendor or supplier of SOA platforms and systems decides to give you. Naturally, each different solution in this space will differ in those constraints.

As the quote from James above implies, pitting SOA and REST against each other doesn’t make a lot of sense. The reason I went to the trouble of pointing out in my talk that SOA is missing critical architectural requirements like constraints was to help illustrate the significant differences between SOA and REST. Fundamentally, as I also said in my talk, SOA isn’t really about architecture in the way that REST is. Rather, it’s about IT culture, best practices, breaking down the enterprise organizational and political barriers that needlessly increase IT costs, and following good fundamental software engineering principles like reuse, minimizing coupling, and maximizing cohesion.

When you read stuff like this, you can’t help but feel that IT is, without a doubt, doomed.

The gist of the posting is

• REST is too hard for the average developer
• Dynamic languages are too hard for the average developer

Along these same lines, a couple of folks told me in person that my recent blog entries about REST, dynamic languages, and ESBs were misguided because today’s enterprises are interested only in approaches, frameworks and platforms that allow average developers to produce quality systems.

This all strikes me as nothing but wrong-headed thinking.

On the dynamic language front, the worst code I have seen in my career has always, always, always been in compiled imperative languages, most often Java and C++. I would much rather let an average developer loose with a dynamic language, because the surface area is smaller, there’s a lot less rope available for self-hanging, and if they’re going to fail, they’ll fail way faster and thus allow much more time for recovery. The fact that dynamic language programs are usually smaller than their compiled counterparts means that they’re easier to read and review, and statistically, they’re likely to have fewer bugs. Furthermore, counting on the static language compiler to save you is simply wishful thinking. To paraphrase Tim Ewald from a conversation he and I had during lunch a week or so ago, compilation really amounts to just another unit test.

On the REST front, if you’re claiming that it’s harder than the alternatives, to me that’s just a sign that you don’t understand it. Is REST simple? No, but neither is SOA. However, unlike SOA, which is fairly wishy-washy, noncommittal, and loose, REST’s constraints provide real, actual guidance for developers, and those same constraints also provide opportunities for significant flexibility, extensibility, performance, scalability, and serendipity. SOA’s contracts come with no rules or constraints, and thus can easily result in a system that’s extremely brittle, tightly-coupled, and virtually impossible to upgrade. SOA itself isn’t inherently bad, as it’s certainly a step above the “every application for itself” mode of development that’s so widely practiced. Unlike REST, though, SOA doesn’t go nearly far enough to provide real, useful guidance to the poor developer who has to actually write the stuff, make it work, and keep it running.

And finally, regarding the overall notion that enterprises cater only to average developers, I’m not sure I agree. In my former life I met countless enterprise developers who were extremely sharp. While I have no doubt that there are numerous bean-counting CIOs and middle IT managers out there who think they can build high-quality IT systems with low-quality developers, at the end of the day, businesses generally know better than to think they can get something for nothing. Or to put it another way, they know they get what they pay for, and if they pay only for average developers or worse, they’ll get only average software and average systems, or worse. That’s a no-brainer.

If you’re in a position of technical leadership or project management and you’re asked to come in ahead of schedule and under budget, my advice is that you’re generally more likely to succeed with REST and dynamic languages than with the alternatives because their inherent constraints allow for better focus. Also, if you find yourself in such a position, you owe it to yourself and your team to continually lobby your superiors to help them understand the very real costs of their budgetary stinginess.

In a comment to the many comments on my post entitled “The ESB Question,” Bill de hÓra gets it right:

I’m disappointed with the responses here. Steve’s an expert technologist in this domain. There’s a real opportunity for learning when the ESB approach is *appropriate* – instead we see the same old reactionary web v enterprise positioning and the usual suspect arguments being rolled out against REST style integration.

Bill’s right; the discussion has unfortunately mostly degenerated into the usual no-light-all-heat “us vs. them” argument. I especially object to the folks who twist what I say and accuse me of saying things that I never even remotely hinted at, but I guess that’s the price of blogging publicly.

I’m not going to try to address the comments individually, especially the ones from the guys who, amusingly, wrote lengthy diatribes explaining to me just what ESBs are, what they’re supposed to do, and why they’re beneficial. Yes, guys, I get all that. Perhaps you should go back and read some of my publications from 3, or 4, or 5 years ago? Funny how I don’t recall any of you being around back then, when I felt more positive about ESBs (hint, it was before that term was coined) and I was blogging and writing to that effect.

Arguing this issue on technical merits is rather pointless. One of the best comments in this thread was from Dan Hatfield:

Honestly, I see the ESB as primarily a political thing. It allows for a greater degree of control on delivered solutions. In large companies, we don’t often do architecture – we do politecture…The politics drive the architecture. Not the way it should be…but that’s the way it is.

So true, so true. Another non-technical way to look at it is from the viewpoint of Clayton Christensen’s classic book, The Innovator’s Dilemma. For quite a few years now, we’ve seen a series of sustaining innovations in the “object/service RPC” line of descent originally popularized by CORBA and COM, both of which built on earlier RPC, distributed object, and TP monitor technologies. RMI, EJB, SOAP, WS-*, and ESB are all offspring in that line, and there are surely more to come. I feel that REST, on the other hand, fits the definition of a disruptive innovation perfectly (and if you’re too lazy to read the book, then please at least follow the link, otherwise you won’t understand this at all). The proponents of the sustaining technologies look at REST and say, “well it can’t solve this and it can’t solve that” and voice numerous other complaints about it, precisely as Christensen predicts they would. But Chistensen also explains why, at the end of the day, any real or perceived technical shortcomings simply don’t matter (and in this case, they’re mostly perceived, not real). HTTP-based REST approaches have a lower barrier to entry and are less complex than anything the sustaining technologies have to offer, and REST is disrupting them, whether all the smart folks pushing ESBs like it or not. It’s not a technical issue, and there’s no amount of technology the non-REST tribe can throw at it to stop it because it’s based on how markets work, not on the technical specifics.

At the end of the day, if you don’t think REST is viable or you don’t like dynamic languages, then don’t use them. It just means that you’re at a different point than I am on the Technology Adoption Lifecycle curve. Like I already said in my first follow-up, I’m not in that business anymore, so it doesn’t matter to me at all. I’ll just keep using what I know to be generally superior to all the other approaches I’ve worked on over the years.