scalability

RESTful Data

February 28th, 2008  |  Published in REST, column, coupling, integration, scalability  |  Add to del.icio.us

In my Jan/Feb Internet Computing column, Serendipitous Reuse (PDF), I talked about interface coupling and the benefits of REST’s uniform interface constraint. I find that whenever you discuss that topic, though, REST detractors tend to say, “Well, you’re just pushing the coupling problems to the data.”

The problem with that assertion is that it assumes coupling is a fixed constant — if you eliminate it from one point, whatever you’ve gotten rid of just has to pop up somewhere elsewhere, like some sort of strange “Conservation of Coupling” law. Of course, that’s not true. In my latest column, Demystifying RESTful Data Coupling (PDF), I turn my attention to this claim and explain how RESTful data works, and why it too, like RESTful interfaces, reduces coupling when compared to WS-* and other similar approaches.

Constructive feedback welcomed, as always.

Internet Computing Call for Special Issue Proposals

January 22nd, 2008  |  Published in REST, distributed systems, integration, performance, publishing, reuse, scalability, services  |  Add to del.icio.us

As you may know, I’m a columnist for IEEE Internet Computing (IC), and I’m also on their editorial board. Our annual board meeting is coming up, so to help with planning, we’ve issued a call for special issue proposals.

The topics that typically come up in this blog and others it connects to are pretty much all fair game as special issue topics: REST and the programmatic web, service definition languages, scalability issues, intermediation, tools, reuse, development languages, back-end integration, etc. Putting together a special issue doesn’t take a lot of work, either. It requires you to find 3-4 authors each willing to contribute an article, reviewers to review those articles (and IC can help with that), and a couple others to work with you as editors. As editors you also have to write a brief introduction for the special issue. I’ve done a few special issues over the years and if you enlist the right authors, it’s a lot less work than you might think.

As far as technical magazines go, IC is typically one of the most cited, usually second only to IEEE Software, as measured by independent firms. I think one reason for this is that it has a nice balance of industry and academic articles, so its pages provide information relevant to both the practitioner and the researcher.

“Internet SOAP” vs. REST: Huh?

December 28th, 2007  |  Published in REST, SOA, distributed systems, scalability  |  Add to del.icio.us

Dilip Ranganathan pointed me to a long rant from Ganesh Prasad about using SOAP at Internet scale. I see that Stu Charlton already chimed in there with some good comments and analysis, but I think there’s still more to say.

Unless I’m missing something, Ganesh seems to be saying, “Hey, if we just stick SOAP directly onto TCP, we can scale beyond Web scale to Internet scale!” Oh, if it were only so easy. I would think that it’s fairly obvious that just because TCP scales well doesn’t mean that higher-level protocols sitting on top of it automatically scale to the same degree.

Why does the Web scale so well? Because of particular constraints deliberately imposed to induce specific architectural properties. The caching constraint contributes heavily to Web scalability, for example. Statelessness and the uniform interface also play a big role there. These constraints along with conditional GET allow messages to be significantly reduced in size or better yet, eliminated altogether. The resulting scalability impact is huge.

Ganesh talks about a lot of the things you’d have to add to the mix get a useful SOA ecosystem on top of SOAP/TCP, but nowhere does he talk about the specific architectural properties and constraints required to make it all scale. Without that, it just ain’t gonna happen. Furthermore, I don’t believe any system based either on interface specialization (i.e., the opposite of the uniform interface constraint) or on “processThis” can scale to Web scale. Interface specialization significantly increases coupling while reducing visibility and applicability, while “processThis” is so devoid of semantics that it offers nowhere to practically apply constraints like caching and statelessness that are so critical to scalability.

“Release It!” Is Truly Excellent

November 13th, 2007  |  Published in book, enterprise, performance, scalability  |  Add to del.icio.us

What if you knew a person who clearly knew an awful lot about large-scale software systems? Someone who earned their way to their knowledge and wisdom by spending days and nights analyzing, measuring and debugging when everyone else had already given up and gone home? Someone considered the “go-to guy” who could always fix things if the system were to encounter a mysterious problem so serious that it resulted in very real revenue loss for the customer for every second of downtime? And what if you could rig up some sort of device to easily tap into the way that person thinks, thereby getting a clear understanding of the rules by which he or she designs, builds, deploys, analyzes, measures, debugs, operates, manages, and upgrades those large-scale systems?

Michael Nygard is such a person, and luckily for the rest of us, he’s already created such a device: his book, entitled Release It! Design and Deploy Production-Ready Software.

Success is a problem we’d like our systems to have. So, we focus our energies on building them as best as we can. Since we tend to focus there, books also tend target only that initial development phase. Most books focus on analysis and design issues, or on the methods and processes for doing analysis, design, implementation, and testing. Sometimes you find a book that focuses on debugging, and some even get into performance and scalability concerns.

But I’ve never seen a book like this one. It addresses the truly hard part of software development, which is running a successful large-scale system in production.

The book is a patterns book, but the patterns it presents are concrete. First, there are patterns and anti-patterns for stability and capacity, intermixed with war stories about real-life large-scale systems that failed hard for reasons that wouldn’t ever occur at smaller scales. Nygard’s war stories bring the patterns and anti-patterns into focus, providing very real reasons for their existence, and hard-won proof that they do indeed work.

Following the patterns, the second half of the book discusses general design issues and operations issues. These parts build on the patterns and focus on gotchas, large and small, that keep small systems from growing into large ones. The general design chapters provide numerous suggestions for eliminating seemingly innocuous mistakes that can kill a system as scale increases. Finally, the operations sections put you in the shoes of the people who have to keep the system running in production, describing what you as a developer can do to make their lives easier (thereby making your own life easier too). Along the way, Nygard keeps us grounded by occasionally presenting more stories and scenarios, some of which include actual dollar figures for the choices and trade-offs made. Engineering always comes down to building the best system possible within the allotted budget, but how many books targeting software developers do you know that talk in concrete terms of costs?

Nygard’s writing style is clear and concise. If you read his book and read his blog, you’ll find the styles to be identical. I’m guessing his book’s copy editor didn’t have a lot of work to do to get this book ready for production.

Bottom line: if you care at all about production software systems, you’ll want to read this book.