The ESB Question

October 4th, 2007  |  Published in enterprise, integration, REST, services, WS-*  |  55 Comments  |  Bookmark on Pinboard.in

The other day, the always-insightful Patrick Logan questioned whether ESBs are worth using. It’s definitely a good question.

In a previous life, I helped develop ESBs. I’ve written about them and I’ve promoted them. But somewhere along the way, I lost the religion.

It happened over a period of years as I learned more about REST, and as I learned how simple it was to develop integration systems using dynamic languages, which I’ve always been a fan of (for example, you can still find my name in the Perl source distribution, from about 20 years back when I ported it to Apollo Domain/OS). Despite the fact that I did it for years, I can’t understand anymore why anyone would subject themselves to writing integration systems or frameworks in imperative compiled languages like Java or C++. Doing it from scratch in such languages means that you’re definitely staring at a multi-year project, if you want to do it right. Or, if you’d rather use some sort of ESB platform or middleware framework underneath, you’re still looking at a number of months just to get anything nontrivial fully written, tested, and deployed.

Some of the mindset behind ESB-type middleware is the desire to do everything in Java or C++. Many developers just want to write some code and plug it into a magical framework that transparently handles all the distribution, persistence, security, transactions, and reliability underneath. Chuckle. Underlying frameworks just grow and grow as they try to provide all this, and so they develop more bugs, more inconsistencies, more special cases, less flexibility, and less reliability as time goes on, not to mention foisting on the unwary the XML configuration hell that Patrick alluded to.

Eventually, these mono-language programmers end up completely incapable of addressing a problem unless they can see how to implement it in their One True Language™. Too many programmers I’ve encountered over the decades know only a single programming language, with many of them even lacking in that one language. They therefore tend to settle into areas that let them use that language no matter what, all day, every day, even if it means writing hundreds of lines of code that could easily be replaced by just a few lines of a language better suited to the problem.

Another mindset behind the ESB is that of the enterprise architect. Large enterprises believe they can save themselves a lot of money and trouble if they can just get the whole enterprise to agree on a single integration architecture. Unfortunately, that’s pretty much impossible because of autonomous divisions independently building/buying and deploying different solutions, because of acquisitions and mergers that bring in different technologies and tools, and because of legacy systems that never seem to go away. The ESB appeals to such architects because it’s touted as being so flexible that it can accommodate everything in the enterprise and then some. This flexibility allows the architect to mandate it as “the standard,” and then proceed to beat up the ESB vendor to add this and that, while also beating up teams within the enterprise to use the ESB for everything whether it really fits or not. The ESB becomes like one of those tools you see on late-night TV, where it’s a screwdriver and a hammer and a wrench and fishing reel and a paint brush, and plus you can flip it over and it can make you a grilled cheese sandwich. Of course, such tools always end up not doing any of those things particularly well.

So, why not just do things the way Patrick suggested: when faced with an integration problem, just grab the nearest dynamic language, such as Ruby or Python, grab the modules they provide that surely cover whatever protocols and formats are needed for the integration, and code it up in just a day or two? Well, ESB proponents would say that this would never do because you end up with way too many one-off solutions, while mono-compiled-language proponents would natter on pointlessly about type safety, compilers preventing errors, and performance. An assumption behind these arguments is that the integration problem being solved requires high performance, and that the solution will live for a long time. In my experience, these assumptions are rarely correct, especially the second one, since business rules change so rapidly nowadays that the integrations required to implement them change all the time as well.

Frankly, if I were an enterprise architect today, and I were genuinely concerned about development costs, agility, and extensibility, I’d be looking to solve everything I possibly could with dynamic languages and REST, and specifically the HTTP variety of REST. I’d avoid ESBs and the typical enterprise middleware frameworks unless I had a problem that really required them (see below). I’d also try to totally avoid SOAP and WS-*.

The only place I can imagine using an ESB would be if I had some legacy systems that just couldn’t be replaced, had to be made to talk to each other, and there were significant measured (as opposed to imagined) performance issues to be addressed within the integration. Under such circumstances, the right ESB could shine; I’ve seen those circumstances, and I’ve seen an ESB solve exactly that problem on a number of occasions, and solve it well. But on the whole, such scenarios don’t really arise that often.

Responses

  1. Pete Lacey says:

    October 4th, 2007 at 9:03 am (#)

    Yes, those are tears of joy streaming down my cheeks. But not just for the SOAP v. REST thing. It’s this: “Unfortunately, [getting the whole enterprise to agree on a single integration architecture is] pretty much impossible because of autonomous divisions independently building/buying and deploying different solutions…” Brilliant!

  2. Web Things, by Mark Baker » Blog Archive » Vinoski recommends says:

    October 4th, 2007 at 9:39 am (#)

    [...] to listen very carefully to regarding distributed computing and enterprise integration, but even more so now that he’s left the employ of a company with skin in that game. That message isn’t [...]

  3. Chuck Adams says:

    October 4th, 2007 at 1:27 pm (#)

    So the answer to ESB and application integration is … REST?

    So basically, the existing knowledge base and, oh, salesforce.com should just be completely rewritten as a REST app? I mean, is there even a single case study where this fantastigasmic REST panacea has ever addressed a problem beyond the simple CRUD form apps that one might slap together with Rails?

    I wonder if there’s some kind of application that would let you expose a REST interface to an older application that has a different kind of service interface, without the client having to translate the messaging itself…

  4. Laurent Szyster says:

    October 4th, 2007 at 1:52 pm (#)

    Hi Steve,

    What you wrote are pearls of wisdom.

    So, why do entreprises continue to look for “industry standard” service bus instead of focusing on de-facto standard network protocols?

    Because, well, because on one hand there are not enough people who can understand why those large “integrated” systems suck and on the other hand there’s “a sucker born every minute”.

    Pointy-haired bosses don’t want to rely on good network application developers because they are too rare to hire and too difficult to “manage”. So, they prefer to plan their software infrastructure for incompetent and/or bored people.

    Some consultants understand that very well and to maximize their profits they favor complicated, slow, buggy, expensive and inflexible tools that creative programmers hate but which can be applied by any dull enabler.

    Of course you cannot fool all people all the time.

    When application development grind to a halt and major projects fail in chain, then there will be no money left for premium priced ESB consultants and software licenses.

    Until then and as long as fear and money talk louder in the meeting rooms than facts on the ground, ESB will continue to walk.

  5. Arnon Rotem-Gal-Oz says:

    October 4th, 2007 at 2:12 pm (#)

    Hi Steve,
    Sure believing the the enterprise can have a single solution is not smart – but I think that the ESB (or at least the SB part of it) still has merit. REST is not the end-all solution just like ESBs or the next technology/architecture

    I wrote a little be more on this on my site http://www.rgoarchitects.com/nblog/2007/10/04/TheTimesTheyAreAchaginSteveVinoskiAndTheESBQuestion.aspx

  6. Marcus says:

    October 4th, 2007 at 3:34 pm (#)

    I think that this is interesting, but isn’t relevant for any use of ESB or ESB-like software that I consider. In fact, I’d go so far as to say that this sort of approach really fits into the imagined “zero-sum-game” world that some REST proponents seem to live in.

    Its funny but so few people who also use ESB and WS-* style protocols and tools frequently operate with such disdain for other ways of doing things.

    The other thing that strikes me is that this essentially seems to boil down to Not Invented Here syndrome. Me, I’m very happy if in a diverse set of groups within a company of any size more than a few people that there can generally be a standard approach to incoming and outgoing data to fit *most* cases.

    I guess the next time you talk to some dweeb who thinks their one true ESB will be the one true data transfer bus for the enterprise you should hand them this article to help swing the pendulum the other way. I just don’t hang around with people who think that way (or read their blogs), but not because I avoid them on purpose, I just haven’t come across it much.

    Are you really trying to suggest that one-off ruby scripts are better for partner integration? I hope you’ve got a good code-review policy in place, cause you’ll need it a whole lot more than my enterprise does.

    :-)

  7. Marcus says:

    October 4th, 2007 at 3:43 pm (#)

    Programmer 1: Lots of tools seem over-engineered, let’s get back to Shangri-La where we all live happy lives with lotus petals and drink fresh water from the stream. Besides, I’ve got my trusty saw, and check it out, it’s cool, one of those neato Japanese traditional types that cuts on the backstroke. I never need anything else.

    Programmer 2: Yeah, I love those saws. I use them all the time because I can carry it around on the roof with me while I’m working. When I’m back on the ground I use the band-saw and buzz saw more often though. You don’t have any of those?

  8. Peter Walker says:

    October 4th, 2007 at 4:07 pm (#)

    I share Steve’s weariness concerning “silver bullets” :-)

    I am surprised though at the assumption that developers are obliged to use a single language if they adopt an ESB. Quite the contrary – an ESB should allow for the construction of Composite Applications using many other technologies, the ones most appropriate to the job. Ours for example allows the use of Java, BPEL, XSLT, Scripting, SQL (just to name a few) with many bindings to non-Java technologies.

  9. ESBs Enable Real-World SOA - Sam Gentile says:

    October 4th, 2007 at 11:11 pm (#)

    [...] the ESB is a fundamental part of SOA, which is what we believe as well obviously, in contrast to Steve Vinoski, who now has sworn them off for the whole RESTafarian thing. You know, I have a lot of respect for [...]

  10. C# database application says:

    October 5th, 2007 at 12:18 am (#)

    Hi,

    When i read all the Hype regarding SOAP and WS – i cannot help recall the hype which went on about OOPs about 10 yrs back.

    We had purely academic professors lecture about OOPs to hands on programmers not understanding a thing about what this prof. was talking about.

    There is SOA conference slated for Nov. 21-23 in India at Bangalore.

    Your opinions expressed here (if i understand it right) could be valuable

  11. Curt says:

    October 5th, 2007 at 12:38 am (#)

    Interesting… However, real world research, and factual evidence suggest your view is skewed.

    Sonic Software
    BEA
    IBM
    IONA
    TIBCO
    webMethods
    Cape Clear
    Fiorano
    Oracle
    Software AG
    Neudesic
    And how many open source projects????

    All deliver ESB’s. It seems highly unlikely that all of these organizations, with plenty of smart folks, could be incorrect. Moreover, Gartner, Forester, ZapThink, and Burton, all speak loudly about ESB’s being an enabling technology on the road to building workable SOA’s. In fact just last week, InfoWeek published an article “What’s Next For SOA” http://www.tmcnet.com/usubmit/2007/10/01/2980218.htm that describes today’s ESB’s, and the features they provide. There is little doubt that any decent ESB can in fact provide an excellent framework for pluggin in services, and leaving the worry of governance to the framework.

    I’d love to hear the specifics behind why you believe that ESB’s are not valuable.

  12. steve says:

    October 5th, 2007 at 1:09 am (#)

    To Chuck Adams: ever used the web, Chuck? REST assured that it’s not implemented on top of an ESB.

    To Curt: I see that Neudesic is in your list, and your email address coincidentally is from the same place. You obviously have a vested interest in making sure that ESBs are viewed positively, so let’s be sure to be clear on that.

    Making a list of smart companies to prove a point, well, doesn’t actually prove anything. I bet there are some actual smart people out there who have locked themselves out of their car, for example, or deleted whole directories they really wanted to keep but never backed up, or even paid actual money for a Zune. What does that prove?

    Also, since you’re talking about analysts and specifically Burton, you might want to look at the first comment at the top of this list, right from a Burton analyst. Who should I believe, you with your second-hand information from InfoWeek, or comments straight from an analyst?

    Rather than asking me for specifics, Curt, why don’t you give me some instead? Specifically, please give me an actual real-world scenario that you know for a fact can’t be solved, or would cost a lot more to solve, with REST and dynamic languages, rather than an ESB. I know of some, given that I was developing ESBs before the term “ESB” was even coined. Unlike most of the commenters on this posting, I’ve personally built at least one of everything I’m talking about, on the ESB side and on the REST side, so I’m extremely confident in my opinion. I’d therefore much rather get specifics from you.

  13. Dave says:

    October 5th, 2007 at 2:29 am (#)

    “… so I’m extremely confident in my opinion.”

    As confident as you were back when you were pushing CORBA? ;)

    Agree with a lot of your points Steve, but your degree of confidence (or otherwise) is not one of them!

    Dave

  14. David Illsley says:

    October 5th, 2007 at 4:02 am (#)

    (aside: I work in an ‘ESB Foundation Technologies’ department, so I have some kind of vested interest in seeing people buy ESBs.)

    I think i’d reduce your argument to Build vs Buy. Yes, for some organisations with the right application lifecycles and employees, building integration fabric when needed (from whatever tools in whatever style) will be the right answer.

    I’d argue that there are a great many organisations which want their SOA to consist of as little custom code as possible. They don’t have the staff, and its not their core competency. Using pre-built integration middleware, serviced by the vendor, and understood by a wide number of consultants/contractors makes sense for them.

    I’m intrigued to know what you’d think of an ESB which deals with some of: “distribution, persistence, security, transactions, and reliability”, and lets you write services/mediations in dynamic languages? Or are you dead set against anything that is SOA rather than REST?

  15. steve says:

    October 5th, 2007 at 7:40 am (#)

    Well, Dave, I don’t see how you can agree or disagree with my confidence. My confidence is what it is, and it’s based on actually building and running stuff, so whether you believe it or not is immaterial.

    As for CORBA, yes, I am that confident. I stand by, and in fact am rightfully very proud of, all the CORBA work I was involved in. We did some really great things, and there are still a lot of CORBA-based systems out there today churning away, doing precisely what they were designed to do, day in and day out. And because many of them are part of critical systems, they’ll continue to be around for years to come. I started working on CORBA in 1991, and here we are, over 16 years later, and those systems are still in use and going strong. Sounds pretty successful to me.

    But CORBA isn’t perfect, as we all know. If I hadn’t learned anything in those 16 years that have passed about its flaws and better ways to do things, I’d certainly be in the wrong business.

  16. Jonas Ekstrom says:

    October 5th, 2007 at 9:43 am (#)

    Your problem is the way you approach your choices in life. There is no single path or a single answer for the future. It’s not about losing a religion or having a religion. It’s about being part of the evolution.

    Religions are based on beliefs and hopes, but there is no single answer to the future, to the past or to why we’re here. You and your initiatives are just part of the natural selection. CORBA, ESB, J2EE, REST, SOAP, all of them plays a role in the presence, but none will be ultimate choice for the future.

    CORBA is just like the Coelacanth, a product more fit than its predecessors but weaker than its successors. Natural selection strikes and the organism become extinct or exterminated. But in the big picture, it played a great part.

    Religions give easy answers, but there are no easy answers. Natural selection will strike again and again until we have reached the perfect SOA.

    Be more agnostic (or even better – an atheist) in the relation to technologies. Dogmas (like I’m going REST, f* the rest) are destructive …

    Hallelujah!

  17. Jim says:

    October 5th, 2007 at 10:46 am (#)

    As a true enterprise architect I welcome diverse opinions from all sides and balance those opinions against the enterprise requirements as a whole. I do honestly thank you for your clear and thought provoking piece.

    But I need to get this clear in my head… The answer to drinking the cool-aid of proprietary vendor ESB solutions on top of WS-* protocols is to build “a magical framework that transparently handles all the distribution, persistence, security, transactions, and reliability underneath” using the “One True Language™” of Python and REST? Wouldn’t this mean “doing it from scratch in such languages means that you’re definitely staring at a multi-year project, if you want to do it right?”

    I understand that REST has it’s place in the Internet scale application where security and transactions are secondary to performance. But what I am responsible for are Enterprise applications where security, transactionality and reliability are primary requirements. They can surely be built on top of REST, but they will not be implemented in an industry standard way and will require lots of effort to “do it right.” My experience is that there are about 1000 or so developers in the entire world that can get security, transactionality and reliability right and outside of you and your close friends, most of them work for the ESB vendors.

  18. Web Things, by Mark Baker » Blog Archive » Only the best, detest says:

    October 5th, 2007 at 11:50 am (#)

    [...] reading the comments in Steve’s initial ESB post, I spotted a paragraph that nailed the elephant-in-the-room meta issue on the head; Its funny but [...]

  19. Brad Williams says:

    October 5th, 2007 at 11:59 am (#)

    I don’t see the connection (besides sociological) between typeless languages and not-using-bloatware. Don’t Java and C# have libraries too?

  20. Dan Hatfield says:

    October 5th, 2007 at 12:09 pm (#)

    I’ve built both types of systems.
    Honestly, I see the ESB as primarily a political thing. It allows for a greater degree of control on delivered solutions.
    In large companies, we don’t often do architecture – we do politecture…The politics drive the architecture.
    Not the way it should be…but that’s the way it is.

  21. David Pallmann says:

    October 5th, 2007 at 12:51 pm (#)

    Disclosure: My company provides an ESB.

    Human beings tend to overreact to things they don’t like. I’ve witnessed this my entire life, in every area from religion to technology.

    The problem is, we tend to overreact to something we dislike and the pendulum swings too far the other way — often becoming equally inappropriate.

    When people have an experience with too much infrastructure, the natural thing to do is want to avoid infrastructure. But surely having no infrastructure at all is just as bad. The two extremes are to be avoided.

    I believe the debates currently raging about REST / SOA / ESB boil down to the simple question of how much infrastructure is appropriate and what shape / capabilities should it provide. I’ve blogged on this here:

    http://davidpallmann.spaces.live.com/blog/cns!E95EF9DC3FDB978E!266.entry

    For those who think no infrastructure is realistic in an enterprise setting, I’d be very interested to hear what your philosophy is on change management.

  22. Curt says:

    October 5th, 2007 at 1:11 pm (#)

    Steve,
    I’m happy to provide specifics, and address a number of the points in your post.

    “some of the mindset behind ESB-type middleware is the desire to do everything in Java or C++.”

    Really? How so? Actually, the opposite is true, ESB’s are intended to embrace the use of a large variety of technologies, and the better ESB’s do not constrain you to any specific language, transport protocol (within reasonable limits), interface type, or architectural style. A good ESB will in fact help decouple the implementation of the service interfaces from the goal of transporting messages. Thus, a developer no longer needs to consider all of the details and intricacies that we used to with Corba, COM, DCOM, etc. Sure, we no provide tools so that metadata can be imported from service endpoints, or repositories, and that’s good, who wants to deal with all the details when you have business features as your driving concern?

    You continue to stress the single minded programmer concept, “Too many programmers I’ve encountered over the decades know only a single programming language,…” How does the language used have anything to do with the concepts? Aren’t you are falling into your own trap here, by stating “I’d also try to totally avoid SOAP and WS-*.” You’ve just reduced the number of tools in your toolbox considerably, as both of these technologies are extremely valuable, and powerful, if applied correctly. Why would you purposely reduce your toolset? Why don’t you just stick with one language then, how is this different? Sure languages can help support concepts (like Eiffel and Spec# do for contract based development), but seldom is the solution dependent in any way on the language. As a result of dealing with traditional application development, integration issues, Web development, and a constantly changing technical community, most developers today have much more breadth than they ever have. Most know a variety of languages, and many can effectively code services, data components, build databases, and construct UI’s. My experience with developers is very different from what you’ve stated yours to be.

    Looking at the enterprise, you state that they want to, “…save themselves a lot of money and trouble if they can just get the whole enterprise to agree on a single integration architecture…”. I’m sure we’d all like to save money, but in my experience very few people actually go down this path for any significant period of time, as they realize it is futile to expect all parties to agree on anything of a singular nature. In addition, most technical and business decision makers realize their organizations are, and will continue to be heterogeneous, and therefore they must support a variety of interfaces, transports, messaging schemes, and other technological factors. All commercial ESB’s offer features to address these needs; adaptation, mediation, and transformation. You go on to mention a way that you would address this, “…grab the modules they provide that surely cover whatever protocols and formats are needed for the integration.”

    Ruby and Python have integration modules for all of my LOB applications? No they don’t, and rarely have I found myself in a situation where I get to design and construct everything from a whiteboard state, let alone actually write the code instead of buying a product. Adaptation is a critical feature in any SOA, and all commercially available ESB’s today have strong support for adaptation. In addition, most have some form of extensible framework that provides for the creation of new adapters, to increase reuse at the adapter level.

    Based on your closing statements you agree that legacy systems might be a place to use an ESB. It is fair to say that mainframes, mini’s, and micro’s do not have outstanding support for Web based protocols, as many are using Cobol, RPG, and are designed to connect directly to terminals. Given this scenario, how might one go about interconnecting these systems with REST, HTTP, and your choice of dynamic language? And since, I’m pretty sure it’s a safe bet to assume that heterogeneity is not going away anytime soon (in fact there is an argument for it growing), that embracing differences is critical to success, and ESB’s are designed to do exactly this.

    It is widely accepted that there is significant value in developing common semantic models for business entities. A different interpretation of identical entities is a common problem, and can lead to all sorts of less than ideal decisions. REST does not cure this problem, reduce its complexity, or change the facts here. Creating reusable components requires commonality, and the process of analyzing a business process, and factoring out the common elements is difficult (for a variety of reasons, many non-technical), but these common elements are critical to ensuring any architecture is extensible and flexible, including RESTful elements. You still need common entities to make any of this work effectively, and achieve any notion of reusability, no matter what the implementation technology.

    The raging debate between SOA, or ESB’s, and REST has centered largely on the specific implementation of the interfaces, specifically those interfaces that use RPC style operations. I think we’d all agree that RPC style interfaces are often less ideal than document based interfaces, but REST isn’t the only way to implement a resource centric, document based interface with endpoint transparency. Good designs are most often the result of good architectural discipline, rarely are implementations dependent on the languages, and technologies used. In fact, that’s precisely why many of these ideas have been with use so long; they endure the ever changing technical landscape because the concepts outlive the specific tools.

    Contract based development has proven to work well on a number of projects that I’ve been involved in, and many of my colleagues have had similarly pleasant experiences with this process. Working with REST is a challenge in this type of an environment because everything in REST is implied; there is no explicit contract, or definition of behavior. This certainly complicates communicating to other parties exactly what is going to happen with your submission, or request, this lack of semantics often leads to errors in the use of these services, or incorrect use of the data by the consuming service. Aside from the advantages of contract first development, a strong SOA principle is to enhance flexibility, and agility so that the business processes can be as fluid as required (we all know they change constantly); addressing only resources (as REST does), and not a combination of resources, operations, and composition does not provide a complete solution. The ability to quickly create new applications or to change existing applications is made possible by many of the features inherent in today’s ESB’s http://davidpallmann.spaces.live.com/blog/cns!E95EF9DC3FDB978E!266.entry.

    In general, I’m not a technology worshiper, although I’ve been in the industry for 25 years, but instead I look for the best tools, best practices, and best disciplines to get the job done. REST is most assuredly a value tool, and its use will continue to grow, and I expect that many ESB’s will soon offer strong support for REST.

  23. Bill de hOra says:

    October 5th, 2007 at 3:26 pm (#)

    “Its funny but so few people who also use ESB and WS-* style protocols and tools frequently operate with such disdain for other ways of doing things”

    @Marcus: My experience is the polar opposite – the enterprise has looked down on REST for years; most recently you see it described it as a some kind of cult. Ironic beyond words given that that REST has objective technical principles that let you evaluate it, whereas the others, especially SOA, cannot be so evaluated (some WS-* specs excepted, and most all the security stuff is worthy), aka “Null Style”. There are good reasons for REST proponents to be skeptical.

    I’m disappointed with the responses here. Steve’s an expert technologist in this domain. There’s a real opportunity for learning when the ESB approach is *appropriate* – instead we see the same old reactionary web v enterprise positioning and the usual suspect arguments being rolled out against REST style integration.

  24. Bill de hOra says:

    October 5th, 2007 at 3:34 pm (#)

    “It is widely accepted that there is significant value in developing common semantic models for business entities. A different interpretation of identical entities is a common problem, and can lead to all sorts of less than ideal decisions. REST does not cure this problem, reduce its complexity, or change the facts here.”

    REST enables common semantic models in ways that the WS-* canon do not support. It does this by making layering distinctions around communications models, especially between message content and protocol operations. If you can *actually* obtain a common semantic model – and I mean real one like KIF or RDF, or FIPA ACL, not some hazy XML based pseudo-language, then REST is ideal for conveying that information.

  25. Marcus says:

    October 5th, 2007 at 7:26 pm (#)

    My random lame prediction for today (sorry, I know this is only partially topical to ESBs): At some point OWL, RDF, Semantic Web, or some such related technology will be viable for daily technology problems. Probably in many ways that we don’t fully picture yet. I can’t wait, its pretty exciting stuff.

    At that point, basic, vanilla RESTful machine to machine services will be *really* exciting, and I’ll venture to predict that SOAP/WS-* will be shockingly uninteresting. Until then, RESTful it’s just another great way to integrate things.

    More topical though: note to self, it’s more and more clear to me every day that I want to stay far away from the painful painful shops that have to live with these craptacular one-offs every day. It might be great for some script writer who gets to gallivant off into the sunset on some new mission, but I sure as hell am not gonna maintain that crap. This is spoken with way too much experience dealing with these disgusting little itch scratchers.

  26. Stan says:

    October 6th, 2007 at 4:49 pm (#)

    The painful thing about the “enterprise standard ESB” is that there are so many of them. My company states a new standard, builds a handful of new systems to the standard, adds adapters to a couple legacy systems, and then suddenly it’s not cool or funded any more. Time for a new one! We get a handful of new systems on that one and, well, you can see it coming a mile away. In the prior millennium, I actually wrote one of these stayed on the charts a month or two.

    For about the third time in more recent memory, somebody is now proposing to build the one abstract API for all new services with the Enterprise providing convenient concrete implementations in all languages to completely hide the underlying transportation and envelope protocols. Not holding my breath.

  27. Ben Kloosterman says:

    October 7th, 2007 at 3:51 am (#)

    Interesting comments

    SOAP vs REST

    Since performance is not a factor in most cases SOAP is better than REST as ypou can do every thing you can with RESt simpler, The problem is the SOAP toolkits for python etc are primative.

    Personally i think things will go more towards EDA type architectures with workflows after all

    SOA –
    Get some Data
    Do something

    EDA
    When this occurs do something.

    Is far more natural in the long run for business processes and espcecially business to business process.

    It is also worth noting the latest C# has a lot of features of the dynamic languages…

    You are mad to write business processes in C++ , C++ should be used for OS and drivers ( not even games) . I wrote in C/C++ for 15 years but now im truely frightened ( memory leaks , difficult to detect memoey corruption etc etc ) .

    Regards,

    Ben

  28. Aehso’s Output » ESBs and REST. says:

    October 7th, 2007 at 7:20 am (#)

    [...] post (and followup) by Steve Vinoski, spawning lots of other discussions.  I should be surprised by [...]

  29. Dmitry says:

    October 7th, 2007 at 4:08 pm (#)

    I think that Resource-oriented architecture (ROA) will play more and more important role in enterprise. ROA brings back attention to data as a foundation of enterprise computing. It emphasizes that after all these years we do not have good support for distributed data in programming languages/infrastructure. REST brings back attention to data/information modeling. It provides complementary view on information resources in enterprise, helps to understand and implement a concept of enterprise information/knowledge grid. ESB can help with low level integration, but there are more complex (and interesting) problems in enterprise computing and ROA is in a better position to influence solutions for those problems. ESBs are available now and they work, but ROA has more potential for the future

  30. John Davies says:

    October 7th, 2007 at 6:42 pm (#)

    In my experience, too many people tend to think ESBs are the silver bullet. We should remember that most of the ESB vendors were selling MOMs, EAI and databases before ESB became the mode and very few of them have changed their products since. ESB is a very lose term that’s grossly over used like SOA, in fact it’s like calling all food “fusion”.

    My background is investment banking and I’ve been around for about at long as Steve, in fact we used to frequent the same circles back in the mid 80s in the days of C, PERL, early C++ etc. Anyway the most successful integration implementations (I include EAI, ESB, SOA in here) have used a complete mixture of technology from CORBA to Jelly through PERL and SQL. While most pet stores can be implemented in Java it is very naive to think that one language alone can solve so many problems. If you’re going to chose any technology as being better your best option is shell scripts (awk, grep, cut, tail etc.) and PERL, one very large US bank famously implemented their entire integration strategy on this just a few years ago and it’s already out-lived a good half-dozen Java based efforts since.

    I’m a fan of REST but there are many areas in my business, batch processing for example, where it doesn’t fit too well. It is though, just another way of solving a problem and a pretty good one in this domain.

    The list of analysts in Curt’s post (“Gartner, Forester, ZapThink, and Burton”), are paid to write about what people want to read, they have few if any fresh ideas of their own, in fact if they did they wouldn’t be working for analysts. Anyone with brains is out there using them not writing about what the clever people are doing, most of what they say is obvious and out of date.

    At the heart of ESB and SOA is EAI. EAI is a payload on a transport, solve these issues and you’re sell on your way to acronym Nirvana.

    -John-

  31. Labnotes » Rounded Corners - 159 (As seen on TV) says:

    October 8th, 2007 at 5:08 pm (#)

    [...] seen on TV. Steve Vinoski on the similarities between ESB and the bargain multi-tools they sell you on late-night TV: Frankly, if I were an enterprise architect today, and I were genuinely concerned about development [...]

  32. Paul H. says:

    October 9th, 2007 at 9:47 am (#)

    There’s something in the middle and nobody can agree on what to call it … is it an “ESB”? is it “REST + Dynamic Languages”? … I can’t help but think that these same type of arguments arose during the infancy of data comms (should I use a Cisco “switch” or a Novell “switch”? is a “switch” really necessary, or can I get away with PPP?) … nowadays, the thing in the middle is colloquially referred to as “The Network”, and switches/routers/etc. are the generally accepted, standardized, manageable, and self-federating mediation technologies … I look forward to the day when we’re in a similar stage of maturity at the level of EAI – a day when there’s a colloquial name for the thing in the middle (“The Interop Network”?) and there exists generally accepted, standardized, manageable, and self-federating mediation technologies … until then, we’ll all have to suffer through endless arguments along the lines of “should I use a BEA ESB or a Sonic ESB? is an ESB really necessary, or can I get away with REST + Dynamic Languages?”

    As far as the points being argued, here’s my two cents: the ESB *concept* embraces heterogeneity at two important levels – i.e., the architectural style level (SOA/EDA/ROA/NBT) and the application protocol level (CORBA/WS-*/REST/NBT); whereas REST + Dynamic Languages requires singular enterprise commitments at the architectural style level (ROA) as well as the application protocol level (REST), only allowing flexibility at the implementation level (Java/C#/Perl/Ruby/NBT). And as someone who has suffered through the effects of singular commitments to an application protocol (CORBA) and an architectural style (EDA), it’s probably easy to guess my preference between the two approaches.

    Paul H.

  33. Brennan Stehling says:

    October 9th, 2007 at 7:23 pm (#)

    No type safety? I am sure you still want reliable services, but compiler protected type safety is one thing but with dynamic languages you really push the type safety to unit and integration tests.

    Overall I want to agree with you that ESBs from corporations trying to solve the worlds IT problems with a single solution should not be trusted. But suggesting that you can only achieve the benefits from dynamic languages with REST undercuts your points for me. Today I dealt with problems due to inheritance and generics and maybe a dynamic solution would have solved my problem sooner, but every approach has some bad mixed in with the good.

    I see many comments do not like WS-* standards, but I have been seeing advantages of a formal agreement about how XML communications will work unattached from a specific programming language or platform. REST is simply a very easy to use solution, but it does not have all of the features of a full WS-* implementation. (the ever evolving standard)

    And with .NET 3.5 I am glad to see that REST is being included as a new feature for WCF which implements many of the WS-* standards. I could quickly expose features of a .NET application/service to languages like Ruby using REST to ease that integration while exposing the same features with SOAP for other .NET/WCF systems. I like that sort of flexibility. The on-the-wire compatibility is what appeals most to me.

  34. steve says:

    October 9th, 2007 at 8:40 pm (#)

    Sigh. Brennan, you say (added emphasis mine):

    Overall I want to agree with you that ESBs from corporations trying to solve the worlds IT problems with a single solution should not be trusted. But suggesting that you can only achieve the benefits from dynamic languages with REST undercuts your points for me.

    Brennan, can you show me where I say that REST and dynamic languages are the only way to do anything? Did I say anything even remotely like that?

    I don’t mean to pick on Brennan, but I really wish the ESB proponents would stop twisting my words and misinterpreting what I actually said just to try to prove their points.

    One other thing: claiming that “REST is not as capable as WS-*” is a sign that you deeply misunderstand REST.

  35. Paul H. says:

    October 10th, 2007 at 8:37 am (#)

    >>Did I say anything even remotely like that?

    … of course you did – paraphrasing, you said “In a perfect world, REST and dynamic languages are the *only* way I would do anything.”

    Here’s the exact quote: “Frankly, if I were an enterprise architect today, and I were genuinely concerned about development costs, agility, and extensibility, I’d be looking to solve everything I possibly could with dynamic languages and REST, and specifically the HTTP variety of REST.”

  36. Architect’s Linkblog » Blog Archive » 8 Links for 10/9/07 says:

    October 10th, 2007 at 8:53 am (#)

    [...] deployment, configuration, disaster recovery, etc. through hardware and storage virtualization.The ESB Question – Steve Vinoski is against ESB’s for most uses and describes why here.  Plenty of [...]

  37. Brennan Stehling says:

    October 10th, 2007 at 8:57 am (#)

    I guess I am mixing what you wrote with another blog.

    http://devhawk.net/2007/10/09/Throwing+Gasoline+On+The+Fire.aspx

    But what you wrote about Java and C++ being incapable while dynamic languages are the right approach is what lead me the impression that dynamic languages are the only solution for you.

    And there is nothing wrong with REST having limitations. I do not think one approach has to do everything, but handling 90% of your needs is a perfectly good solution if you can mix it with another solution to fill in the last 10%. If the REST approach takes much less effort and time then it is a good solution.

    Here is a great example of using REST.

    http://www.subsonicproject.com/view/rest-linq-and-you.aspx

    And I would not say I am an ESB proponent. I am happy creating services and provisioning them myself so that they work well with each other. I do not need some fancy ESB solution to orchestrate the services for me. To me, an ESB is just bloat for most shops, just like EJB for Java was this past decade. And for the big shops that may have needed it, they may have been able to get by better using a simpler solution.

  38. steve says:

    October 10th, 2007 at 10:06 am (#)

    Paul H: hmm, I hope this isn’t a sign that reading comprehension has become a problem in the ESB community.

    Let’s focus on my exact quote, specifically this part: “I’d be looking to solve everything I possibly could with dynamic languages and REST.” A more verbose way to say that is, “I’d always consider dynamic languages and REST as a possible solution for a given problem, but if they proved to be the wrong fit, I’d use another solution.” Turning that around makes it, “If dynamic languages and REST proved to be the right fit for a given problem, then I’d use them to solve it, otherwise I’d use something else.” Your paraphrased interpretation, OTOH, does not follow from what I said because its “perfect world” inference ignores important parts of my original statement. For example, the middle part of my original sentence was “[if] I were genuinely concerned about development costs, agility, and extensibility,” which completely invalidates your use of the phrase “in a perfect world” because in a perfect world, you’d have no such concerns. Also, the fact that I mentioned in my original posting some of the circumstances under which I’d actually use an ESB further invalidates your (mis)interpretation.

    I don’t talk about perfect worlds, and none of what I’ve ever published focuses on perfect worlds, because I write real software for real systems every day, and so I prefer to think about and focus on the actual world.

    On the plus side, though, I do thank you for helping prove my point about being misinterpreted.

  39. steve says:

    October 10th, 2007 at 10:20 am (#)

    Brennan: thanks for responding, and I’m sorry for mistakenly painting you as an ESB proponent. Based on your latest comments, it seems we’re pretty much in agreement.

    My comments about Java and C++, BTW, are mostly based on the fact that they’re much more verbose than dynamic languages, and more lines of code generally means more bugs. It has always killed me to watch mono-lingual developers waste days trying to solve a problem in their one language that could be solved in just a few minutes in the right language. I use C++ pretty much every single day, and have done so since 1988. I’ve written my share of Java, and though I’m not a big fan and my current work doesn’t require it, I’m sure I’ll be dealing with it again at some point. However, I also use Python, Ruby, Perl, and Erlang daily, and I’m well aware of the trade-offs that come along with each language. At the end of the day, I believe that using multiple languages every day makes you better at all of them, and a better developer overall.

  40. Tom says:

    October 10th, 2007 at 10:48 am (#)

    Halleleua

    A voice for truth and experience

  41. Paul H. says:

    October 10th, 2007 at 6:31 pm (#)

    … sounds like misinterpretation isn’t just my problem – who said I was an “ESB proponent”? ;-) I’m just a guy who’s waiting for the day when there’s some sort of consensus about what to call that thing in the middle so we can get past arguments such as “To ESB or to REST+DL?” … I only said that if given a choice between the ESB concept and REST+DL that I’d pick the ESB concept.

    Anyhoo, I apologize if I misinterpreted your meaning – but the English language is like that (almost any given sentence can have multiple interpretations) … thanks for elucidating.

  42. steve says:

    October 10th, 2007 at 8:32 pm (#)

    Paul H: thanks for being a good sport, and sorry for calling you an ESB proponent. :-)

  43. Dilip Ranganathan says:

    October 10th, 2007 at 10:37 pm (#)

    “It has always killed me to watch mono-lingual developers waste days trying to solve a problem in their one language that could be solved in just a few minutes in the right language”

    Steve, I believe this is a very naive position. Most of us just do not have the luxury to use a different language to solve a problem even if it means getting the job done quicker simply because the company you work for will not allow it. I worked for a financial firm a few months ago and we wrote a really high performance application that needs to constantly poll market data vendors for the latest stock price for a particular security. I had to write the entire thing in C++ and I couldn’t do simple things like serializing an object graph to XML without going absolutely nuts (I tried using Boost Serialization library in this case). I begged my boss to do it in C#/.NET where the programming experience is a lot more pleasant (and considering we are writing this product for the Windows platform). He simply blew me off citing some obscure company policy and some claptrap about not being able to find a developer that can maintain a newer language. If this happen for C# imagine what would’ve happened had I suggested parallelizing some CPU intensive work using Erlang!

    You see, left to our own devices programmers are generally smart enough to figure out what gets the result faster (because we are lazy by default, remember?). The problem is with the so called “corporate policy”. It has managed to stump me many many times in my career.

  44. steve says:

    October 10th, 2007 at 10:47 pm (#)

    Hi Dilip: I don’t think “naive” is a fair description. You’re assuming the only reason developers do what I described is because of corporate policies like the one you described. I don’t work in such places, and never will (in fact part of the reason I left IONA is because they weren’t supportive of using Erlang), so most of my experiences with the mono-lingual developers I’ve encountered have been in places that allowed and even encouraged developers to use multiple languages. What I described is unfortunately all too real and was not caused in any way by corporate policies, but rather by developers sporting only a single hammer for that nail.

  45. SnapLogic Blog » ESBs and the Last Mile of Integration says:

    October 11th, 2007 at 11:57 am (#)

    [...] I’ve been following the “what is the value of ESBs” discussion kicked off by Patrick Logan, which then migrated over to Steve Vionski’s blog. [...]

  46. Dilip says:

    October 12th, 2007 at 1:34 pm (#)

    “I don’t think “naive” is a fair description”

    Its not. Sorry! Wrong choice of words — I should’ve said “its not always possible to branch off to another language”.

    BTW, any chance you can provide a comments feed?

  47. SnapLogic Blog » More on ESBs and the Last Mile… says:

    October 12th, 2007 at 6:26 pm (#)

    [...] been following the thread that Mike blogged and a comment from Paul H on Steve’s original post stated: There’s something in the middle and nobody [...]

  48. Ron says:

    October 13th, 2007 at 9:53 am (#)

    The poster (Curt) who says that ZapThink says that ESBs are an enabling technology on the road to SOA has mischaracterized our position. Speaking from ZapThink’s perspective, we don’t believe that ESBs are neither necessary nor sufficient to enable SOA. In fact, we’ve seen plenty of SOA solutions that leverage a wide variety of non-ESB infrastructure. To be as unambiguous as possible: ESB is vendor marketing spin. True, there is certainly capabilities within an ESB that *might* enable companies to produce truly loosely coupled, composite, and heterogeneous Services in an environment of continuous change, but you can just as easily build tightly-coupled, proprietary, point-to-point Service integration with ESBs. There’s nothing about an ESB that substitutes for the need to do architecture. And there’s nothing about architecture that requires the adherence to a particular technological infrastructure.

    If you want to make SOA work in a heterogeneous environment, why would you want to limit yourself to one technology, one approach? You’re buying right into their strategy of locking you into a platform. That’s only good if you sell platforms. Wake up folks – architecture is YOUR responsibility, not that of some vendors hawking middleware!

    So, don’t put ZapThink in the camp of the ESB bigots. We certainly are not. Implement SOA with intermediaries and REST. Why not?

  49. steve says:

    October 15th, 2007 at 10:00 am (#)

    Dilip: each blog entry already has a comments feed. Here is the feed for this entry. Or did you mean a feed for all comments across all entries? I guess if there’s a WordPress plugin for that, I could deploy it. Still learning my way around WP.

  50. The Cave » Blog Archive » ESB is Passè? says:

    October 17th, 2007 at 2:36 pm (#)

    [...] Die ESB Die? Interesting. [...]

  51. Dilip Ranganathan says:

    October 24th, 2007 at 9:25 pm (#)

    Steve

    I meant a feed for all comments across all entries. Like the way Sam Ruby has it today. That way I wouldn’t have missed your reply to my request for comment feed :-)

  52. Chui Tey says:

    November 6th, 2007 at 7:22 pm (#)

    I’ve written enough hastily put together integration scripts to observe that most of the code added later were for runtime monitoring and debugging support – start, stop, pause, logging, reestablishing database connections.

    Dynamic languages hold some promise, but more important is a runtime which makes it easy to debug. Shouldn’t ESBs be more like VMs which make it easy to inspect and debug?

  53. Jumping Through Hoops – REST links says:

    December 7th, 2007 at 5:28 am (#)

    [...] Vinoski – The ESB Question includes a great quote: Frankly, if I were an enterprise architect today, and I were genuinely [...]

  54. Paul Fremantle says:

    December 10th, 2007 at 11:44 am (#)

    I’ve recently written an article which responds in some ways to your blog entry. Its entitled “Reclaiming the ESB”. You can read it here: http://wso2.org/library/2913

    Paul Fremantle

  55. Johnny G says:

    March 4th, 2008 at 4:25 pm (#)

    Sonic Software
    BEA
    IBM
    IONA
    TIBCO
    webMethods
    Cape Clear
    Fiorano
    Oracle
    Software AG
    Neudesic

    Didn’t many of these companies also rally behind Java EJB’s and RMI? Of course there is no way they could have been wrong about that one either.