Serendipitous Reuse
January 5th, 2008 | Published in column, objects, REST, reuse, services, SOA | 26 Comments | Bookmark on Pinboard.in
Reusability is often promoted not only as a goal but also as a feature of all kinds of software architectures, designs, and systems. For example, in the CORBA, WS-*, and SOA worlds I formerly haunted, everyone spoke nonchalantly of reuse as if it were a given. You were supposed to simply identify the objects or services required to support your business processes, and then specify their interfaces. Then, anyone wanting to provide one or more of those objects or services was merely supposed to follow the appropriate interfaces and write implementations for them. Applications were written to the interfaces and thus were automatically decoupled from the implementations. As a result of these reusable interfaces, you could also potentially reuse the objects and services that implemented them, as well as the applications that consumed them.
Problem is, it never seemed to work out as easily as that. Most of the time, the interfaces people came up with were just too specific, and nobody could agree to apply them widely. Think of all the time people spent over the years in OMG, JSR, and W3C WS meetings trying to agree on just the infrastructure interfaces and not always succeeding. It’s therefore not surprising that there was never much success at defining broadly-accepted standard interfaces up at the application level; the scope is simply far too wide up there.
So, with technologies that promote interface variability, planned reuse is pretty hard. Consequently, serendipitous reuse, where services and facilities can be combined and reused beneficially in unforeseen ways, is virtually out of the question.
Stu Charlton’s blog was where I first saw those terms used, and I found them very enlightening. So did Bob Warfield.
One of the first things that attracted me to REST was the uniform interface constraint. Mark Baker first brought it to my attention nearly 8 years ago, and before I looked at it, I thought it was just a bad idea, like a totally generic doIt() interface, devoid of any meaningful semantics. But of course, there’s much more to it than that. The HTTP interface, for example, strikes a great balance that allows it to be efficiently reused across a very wide variety of applications. And when such a uniform interface is reused, the applications that use it stand a much better chance of being reusable themselves than if they were written against a non-uniform application-specific interface.
My latest Internet Computing column, entitled Serendipitous Reuse (and here’s the pdf if you prefer), explores how the uniform interface contributes to reuse of both the planned and serendipitous kinds.
January 5th, 2008 at 3:02 pm (#)
Having read your linked article, you present a compelling argument. However, my discomfort ensues when you mention that REST could be the solution — if applied “correctly”.
The trouble with REST is that there is no definitive reference implementation foe the very reason that you cite. It is almost a BYO protocol beneath your pplication protocol.. Some use “REST” to mean an RPCish protocol defined by POSTs containing a “method” argument whereas others prefer letting the HTTP operation define the method and the accessed resource or route handle the context. Nevermind the added confusion that REST does not specify a data exchange format and is therefore agnostic to XML, CSV, JSON, text, and binary implemented CIA cats and dogs.
While I agree that specificty inhibits serendipitous reuse, a RESTful interface can be written to prevent serendipity just as easily as a Java, .NET or WSDL.
January 5th, 2008 at 5:40 pm (#)
Evan: yes, like anything else, REST can be abused. But I don’t think you need a definitive reference implementation to know what REST really is, because REST is pretty well defined. Fielding’s thesis introduced REST and is very readable. There’s also the excellent RESTful Web Services book. For more information you can visit the REST wiki and tune into the rest-discuss mailing list. There are also a variety of good REST tutorials around the web. You’ll find that these resources are quite consistent.
BTW, the fact that REST does not specify a data exchange format is a huge positive, not a negative. It allows different applications to use the media types that make the best sense for them, and allows resources to provide representations in multiple formats based on client expectations. After having worked for years with systems that hard-code the data exchange format specifically to the protocol in a take-it-or-leave-it approach, I’ve found the flexibility that REST offers here extremely useful for extensibility and for enhancing reusability.
January 6th, 2008 at 6:06 pm (#)
Great article! I think that enterprise architects/developers can leverage many REST ideas to build better systems. CRUD-based programming model, globally addressable resources, multiple resource representations are very powerful concepts.
I also found very useful to have one of the resource representations in a standard format such as RDF (W3C) and Topic Maps (ISO). Standard interface + standard representation (as one of the representations with well defined semantic commitments) provide great platform for serendipitous reuse. It is wonderful to have the generic GET interface, but it is even better to have an ability to GET representations using the same language (with the same semantic commitments) and to have an ability to merge information from multiple sources.
January 7th, 2008 at 12:08 pm (#)
@Steve: I’m on the REST bandwagon. I do believe that a uniform interface helps with re-use, but I’m skeptical on exactly how much. You still have to define the data format that you are exchanging which is really the bulk of the problems that are encountered with re-use, no? I’m thinking that XML schema does so much more here to encourage reuse.
@Evan: Although the data format, IMO, is at the heart of the re-use problem, I don’t think that supporting any data format (JSON, XML, CSV, etc…) creates any confusion. In fact, IMO, since the protocol and data format are decoupled it creates less confusion as I have a common, well-defined, ubiquitous, lightweight, stable, and standards-based way to transmit any type of data I want. The key terms here being *ubiquitous*, *lightweight*, and *stable*. Those adjectives just don’t describe WS-* or CORBA.
January 7th, 2008 at 12:14 pm (#)
@Evan: One more thing… I had the same hangups as you on the fact that REST does not define a data exchange format. I finally realized that RPC parameters and return values are actually undefined “data exchange formats” that must be defined by the application developer. So in that regard RPC and REST really aren’t much different.
January 7th, 2008 at 3:58 pm (#)
Bill: a typical response to the uniform interface constraint is that it just pushes the problem to the data, so nothing is gained. I’ve heard this every time I’ve talked about REST, most recently in a question from some WS-* proponents during my talk at QCon in San Francisco. I’m actually considering making it the topic of my next column.
One thing to consider about HTTP, for example, is that it’s not an exaggeration to consider it an application framework. It gives you the standard operations that as a client you can invoke or as a server you can respond to. It also provides the status codes required to convey success or failure conditions, so servers know how to respond to particular conditions, and clients know what happened with each operation they’ve invoked so they can react appropriately. This alone is an enormous step up from the ad-hoc interface specialization approach, which essentially forces you to develop your own ad hoc protocol.
With such an application framework in place, focus naturally shifts to the data that’s exchanged. Note that XML or XSD alone do not save you here, contrary to your comment. Every time you invent your own format you essentially decrease your chances for reuse, because special code is required to understand it. But the very separation that you and I both mentioned in our comments to Evan is a big win here. If you choose a well-known data format, you increase your chances for reuse over just inventing your own format, since more systems already know about it. And since protocol and data format are decoupled, the resource can support multiple formats simultaneously, thus allowing clients to sort-of negotiate for the content type they prefer. Note that formats can be separate, or there can be two or more formats combined, such as XHTML together with an embedded microformat. Either way, extensibility is greatly enhanced with this approach, and extensibility often leads to greater reuse.
January 7th, 2008 at 4:15 pm (#)
Steve:
if you were to read my post on “REST creates a strong coupling”, you probably would not write a paragraph such as this one:
“The more specific the service interface, the less likely it is to be reused, serendipitously or otherwise, because the likelihood that an interface will fit what a client application requires shrinks as the interface’s specificity increases. Highly specialized interfaces inhibit general reuse because only purpose-built clients can invoke them. Should requirements change — and they will — modifying such highly specialized services and clients to fulfill the new requirements can be costly because of the high degree of coupling between them.”
This is true in traditional programming model, I don’t believe it is true in the Web Services world. Technologies such as XML/XSD/SXLT/XPath, BPEL and SCA/SDO dramatically change the environment when compared to a Java / C# implementation.
JJ-
January 7th, 2008 at 5:38 pm (#)
@Steve: I’ve read most of the O’Reilly book — I have a bad habit of never quite finishing tech books. However, I didn’t know about the mailing list. I’ll have to check it out.
@Bill: No, I understand your RPC-to-REST explanation although it took my read of the O’Reilly book for it to click. Don’t get me wrong: I’m not anti-REST. I do see benefits. It’s just that the implementations of REST vary somewhat wildly (again see aforementioned REST book from O’Reilly). Now if everyone was using RESTful routes instead of RPC via REST then I would have less heartburn. However, what some people call RESTful just ain’t. That said, I’m right there with you on the data exchange issue.
January 7th, 2008 at 6:47 pm (#)
JJ: no, I’ll stand by my statement, thanks. Having an interface description language in the picture doesn’t somehow make for loose coupling, as you seem to claim, and not having one certainly doesn’t make for tight coupling and all the other erroneous claims about REST you’ve placed in bold font there. I can’t see how you could possibly reach such conclusions. But maybe I missed the real point you were trying to make as I struggled to filter out all the insults, snide remarks, and sweeping generalizations that your blog continually makes.
I will try to say this in the nicest way possible, by repeating some advice I gave you earlier: stop writing unfounded opinions about REST, and go try to use it with an open mind to write some real applications. If you’ve never used REST for real, how in the world can you criticize it with any authority? If you actually had some facts to work with, then you wouldn’t have to continually shout in bold font to try to assert your opinions, or try to build your argument up by putting detractors down and insulting them personally if they happen to disagree with you.
January 8th, 2008 at 10:45 am (#)
Steve:
Loose coupling comes from the ability to separate the view of the interaction from the implementation perspective from the communication interface, not from the fact that an interface is specific or not.
You are correct when you say that some people’s view of Web Services and some Service containers (WCF comes to mind) do not separate the “interface” from the implementation and that’s a problem, how ever, I am not sure you are correct in the root cause analysis and your recommendation using REST.
Loose coupling can only be achieved if two components communicate via 4 interfaces:
C – Ci XXX Ce -> Pe XXX Pi – P
i = internal
e = external
Loose coupling is defined by how easy it is to bridge the internal interface with the external interface, i.e. how big and how easy it is to write the XXX code.
In REST Ce and Pe are the uniform interface. Everything connects to everything. IMHO, when you argue to use a data contract, you also argue that we should embed the semantics of Pi and Ci in it. Would we agree on that? Or are you arguing that semantics such as action, inter-actions, trans-actions, events… don’t exist anymore.
(BTW, I am trying collect the facts, but this is not easy)
thanks,
JJ-
January 8th, 2008 at 11:12 am (#)
JJ: didn’t Subbu Allamaraju already answer this?
January 8th, 2008 at 1:11 pm (#)
Steve:
actually, this post from Subbu is an answer to my post about SimpleDB and not about “Rest Creates a Strong Coupling”.
I think Subbu responded to this post with this one: http://www.subbu.org/weblogs/main/2007/12/on_hypermedia.html
After reading Subbu’s response multiple times, I don’t see any disagreement between he and I. Have you read this one? I think he and I say essentially the same thing.
JJ-
January 9th, 2008 at 12:44 am (#)
JJ: as I said in a previous comment, not having an IDL does not make coupling worse, nor does having an IDL somehow make it better. Also, in your diagrams, the internal implementation details and how they’re hooked up to what you call the external interface (the XXX code you mention in a comment above) don’t matter, since you can’t tell anything about that hookup with any of the technologies you’ve mentioned.
I sincerely hope that the “January article” you keep mentioning is devoid of he inflammatory language that occupies the majority of your blog text. I mean, honestly. I hate to keep harping about this, but you keep asking me to read your blog, so I have to keep wading through all the abuse you’re trying to heap on people, and I grew tired of it the first time I saw it, let alone the third or fourth time. Your “strong coupling” posting says you have no emotional attachment to WS-*, but you’re quite honestly the worst offender I know of when it comes to being unable to discuss these technical issues without losing your temper, flaming or insulting someone, or making numerous unhelpful snide remarks. I didn’t see anything nearly as bad when I was attending W3C meetings for years where the two sides were locked in constant battle. I mean, look at that “strong coupling” posting — it’s about 80% pointless flames and 20% content, and I think that’s being generous. What’s worse though is after having to read through that 80% prattle again is finding that the technical viability of the %20 content, as I point out in the previous paragraph, is quite questionable. In all honesty, I simply don’t have time to waste on such things — I work for a startup, and time is scarce.
So, assuming this fabled January article you keep mentioning is being published in a real journal with real reviewers/referees and technical editors, it won’t contain any of your usual flames and verbal abuse. If so, rather than continuing to have to suffer through your blog postings, I’d much rather wait to get a copy of or pointer to the article before commenting any further.
January 9th, 2008 at 10:38 am (#)
JJ and Steve: I’m all for losing your temper, flaming, insulting, and baiting in between valid arguments. It gets people all riled up and the troops marshalled on both sides of the argument. It brings out peoples passion.
Although I don’t agree with it JJ, I enjoyed your blog both from a technical perspective and an entertaining one as well. I think it was more 99% content in the blog and 50/50 flaim/content in the comments.
January 9th, 2008 at 1:03 pm (#)
Steve:
hopefully it will be published on InfoQ, Stu has already started to review the article and gave a great set of comments to chew on. I hope Stefan will review it too and maybe a couple of other people. If you want to review it too, I’d be honored.
Frankly, I am not trying to make any false claims against REST, I am simply an architect with a lot of questions. If you catch me saying something false and it can be substantiated, I’d be the first one to say I was wrong. Now I tend to be skeptical of things I hear in general, and I teach my children to be that way too, even what I tell them (minus being skeptical).
Now, loose coupling is becoming a very important question. It the reuse that you are talking about that real? I just reported on a couple of articles that I found interesting from IBM and SAP: http://www.infoq.com/news/2008/01/sap-ibm-sca-composition
I think REST is leading the way, there is no question about it.
I am simply trying to establish a model for understanding loose coupling. First people thought it was about a common syntax (XML) and channel independence, but obviously that’s far from enough. We also evolved from RPC to coarse grain interactions, but it does not seem enough too.
All I am asking is, is this model for loose coupling valid? (considering it is completely technology neutral) and if it is how does REST play there. Neither Stu, Stefan or yourself (in your latest article) have explained how you write the XXX code in REST.
Now, if it is not the right model for loose coupling, what is your model? You talk about not tying the implementation to the interface, do you mean the internal one or the external one, in your model, are you tying directly the implementation to the data contract, and therefore the REST interface?
Steve, sorry for the flame, this is really the by-product of ten years of discussions, nothing personal. You have to also understand that I have been a fairly lonely voice across the industry in the WS-* space, utilizing the stack in ways that pretty much nobody has been using up until now. Frankly, I consider it a miracle that the stack turned out to be what it is today. I know you hate it, but believe me it could have been much worse. So I tend to be a bit more vocal, because no one else will pick up this debate. Most standards architects will be ducking because they use WS-* like CORBA and RPC, some tried to lock it to do just that, but somehow it did not work.
In my view, ROA and EDA are essential elements of an SOA (not separate models), so I am a bit upset at how possessive the REST camp is about Resources, nothing personal again.
Bill thanks for your comments, I also enjoy your blog (I guess I could call my blog Hangry JJ). It is perfectly ok to disagree, I live in a certain context and I am not trying to claim that everything I say applies to everything under the sun. I am all for always defining the context in which you use a technology.
thanks,
JJ-
January 9th, 2008 at 7:54 pm (#)
Steve,
I still have unanswered questions about the above and I am eagerly awaiting your column on this. I think it’s an important topic worth highlighting.
Here is one issue that I am still struggling with.
I undestand that if I use a uniform interface and use a standard format for my representation(s) then more clients will be able to interact with me.
However if I change the format of my representation then all clients will now have to change their code.
Ex. If my representation contained a String and I change it to a number and a String.
In the WS-* world this would be an interface (IDL) change. In the REST world this is a change in the format of my representation. Either way I end up forcing my clients to change their code. So in WS-* it’s dealt at the interface level and in REST the burden is shifted to the data. The problem still exists in both models, but at different levels. Having a uniform interface doesn’t really help in this case.
However I disagree when people say “nothing is gained”. There are advantages. But in some cases the uniform interface constraint just pushes the problem to the data.
January 9th, 2008 at 10:34 pm (#)
Rajith: forget REST, SOA, WS-*, HTTP, CORBA, JMS, etc. for the moment. Now ask yourself: under what circumstances would you have to make a change like the one you describe? Are you changing a widely-deployed and widely-used data record in a service that’s already been available in production forever and a day? Or are you still developing the service, and still settling on how clients interact with it and use it? Was any thought given to versioning when the service was deployed and put into use, such that you can make the change while retaining compatibility? Or have you already considered a bunch of different approaches and don’t see any way of preserving compatibility at all?
Fact is, your question is orthogonal to the opinion that Bill expressed that the uniform interface just moves the problem to the data. Regardless of the technology you’re using, versioning requires design and forethought. It’s not something you can push off until later with total confidence of being able to fully and easily address it when the need arises. That said, some technologies are better than others when it comes to versioning. Messaging systems often have an advantage, since they often just send bytes from here to there. I do think RESTful web services have an advantage here as well, since a resource can support multiple representation formats of itself, and the flexibility and applicability of URIs can be quite helpful too. CORBA, on the other hand, has long suffered from versioning issues, which we tried for quite awhile in the OMG to solve but never really got there, so the sorts of versioning you can do in CORBA are pretty limited. Ultimately, you can do versioning poorly with pretty much any technology, but your ability to do it well is sometimes limited by the technologies you choose.
January 10th, 2008 at 11:34 am (#)
JJ: regarding the XXX code, I guess I still don’t see what it has to do with coupling. I have an internal implementation that I present to the world through some external interface. XXX represents part of the internal implementation, and so it’s mostly invisible to users of the external interface.
When I speak of not tying an implementation to the interface, I’m speaking about the implementation of a client to the interface of a server.
I actually don’t know any raving REST lunatics who make wild claims about REST being the only answer, or about REST having invented the notion of resources, or anything like that. The REST people I know, both those I know personally and those I know only through the Web or email, are all very thoughtful experienced folks who have seen and worked with a number of approaches in their careers, and have arrived at REST based on their knowledge and on experience. None would claim it to be the answer to everything. I guess that’s why I don’t understand the animosity towards those of us who know REST has significant merits. I guess there are fanboys or whatever you want to call them in every community, but I personally don’t know any, and those who’ve been at the receiving end of some of the unfortunate remarks in your blog certainly don’t fall into that category.
January 10th, 2008 at 1:34 pm (#)
Steve,
My examples was a bit extreme and if it gets to a point of changing a widely deployed service in such a manner(without considering versioning), then before anything it highlights an issue with your planing/engineering process. But the point is that the problem doesn’t go away. It just manifest at a different level.
If something like that happens (well we always see badly designed software), then REST is no better than anything else. What I wanted to highlight is that simply bcos you use a uniform interface it’s not going to help you if you fail in your planning process.In other words versioning is still an issue even if you use a uniform interface.
You highlighted the point I was looking for.
If you use WS-* in a message oriented manner, instead of the RPC crap, then I claim you still have more or less the same advantages you described above. You can still use URI’s to your benifit and you could have your service return a different representation accordingly.
The problem I see is that some REST folks (not you) claims that REST is the best (and sometimes the only sane) way of managing change due to Uniform interface etc, which I think is not entirely true.
January 11th, 2008 at 3:52 pm (#)
Steve:
I think this is where our disconnect comes from. Sure the XXX code is private since it sits behind the external/communication interface.
However the way you design the external interface influences the amount of code you have to write at the XXX level (again, assuming you never want to go back and change the implementation behind the internal interface).
All I argue is that if you directly create your component implementation behind the external interface you can’t really achieve loose coupling, even with REST because this implementation is going to expect specific data contracts (or service contract with WS-*).
Do you think what I am saying is correct? I think Rajith is expressing the same thing.
Again, IMHO, the question is really how do you minimize the effort to build the XXX code, this is really how you achieve loose coupling.
JJ-
January 12th, 2008 at 11:34 am (#)
JJ: because you’ve chosen to go back into personal attack mode in your blog, you’re now completely banned from commenting here, and I will interact with you no further. My advice to you, in all seriousness, is to seek professional help for your problem. I sincerely hope you don’t treat your family or friends this way.
REST proponents can easily answer your questions, as I’ve said repeatedly, but we choose not to because you’ve proven that you’re simply not worth the trouble. I thought we were making some progress here, since your recent comments were civil, but then you reverted to business as usual and are again calling people names, putting words in their mouths, and personally attacking them. Why would we bother wasting our valuable time trying to have an adult discussion with someone who’s proven time and again that he’s incapable of such? If you’d like to choose to grow up and act civilized, then perhaps you’d get your answers. But if not, then keep writing similar posts in your blog, because they let the world know precisely the type of immature person you choose to be. The only person who looks foolish in such posts is you.
January 18th, 2008 at 1:17 pm (#)
Comments on Serendipitous Reuse…
I just finished reading an article in IEEE Computer titled "Serendipitous Reuse" (pdf here) by Steve Vinoski. There is also some interesting discussion on this article at Steve Vinoski’s blog. The central premise of this article is that unif…
January 18th, 2008 at 1:34 pm (#)
Hi Steve,
I just had a chance to read your IEEE paper. Congrats, it is well-written. However, I do have some comment on your analysis.
I don’t think the point that specialized interfaces inhibit reuse does not necessarily lead to the conclusion that “reducing interface differentiation” promotes reuse. Uniformity of interfaces lowers the barrier of entry for application development, but what promotes reuse is, in my opinion, the nature of the problem being solved by an interface. The more specific the problem is, the less likely it is to be reused. The most popular REST/HTTP APIs publicly available today are also solving hard problems that are expensive and time-consuming for everyone to replicate.
Secondly, uniform interface are not necessarily semantically richer than “as generic as possible” interfaces or even “specialized” interfaces. For instance, a RESTful version of your example “BagOfBytes processThis(BagOfBytes)” would not provide all the semantics of the data and context for a client application to use it.
I have a longish comment at my blog.
Subbu
January 21st, 2008 at 5:44 pm (#)
[…] on how some think REST requires a definition language like WSDL or IDL. He first mentions it within Serendipitous Reuse, and further expands on it in the blogs Lying Through Their Teeth: Easy vs. Simple, IDL vs. Human […]
January 24th, 2008 at 11:51 pm (#)
[…] to read an interesting article by Steve Vinoski. After graduating from the school of SOAP I’ve always wondered what was so […]
January 28th, 2008 at 11:03 pm (#)
[…] is really the key, that REST has going for it is… Adoption! What Steve Vinoski terms “Serendipitous reuse“. Developing RESTful applications is agile and allows for iterative development; to a large […]