Steve Vinoski's Blog

The other day, the always-insightful Patrick Logan questioned whether ESBs are worth using. It’s definitely a good question.

In a previous life, I helped develop ESBs. I’ve written about them and I’ve promoted them. But somewhere along the way, I lost the religion.

It happened over a period of years as I learned more about REST, and as I learned how simple it was to develop integration systems using dynamic languages, which I’ve always been a fan of (for example, you can still find my name in the Perl source distribution, from about 20 years back when I ported it to Apollo Domain/OS). Despite the fact that I did it for years, I can’t understand anymore why anyone would subject themselves to writing integration systems or frameworks in imperative compiled languages like Java or C++. Doing it from scratch in such languages means that you’re definitely staring at a multi-year project, if you want to do it right. Or, if you’d rather use some sort of ESB platform or middleware framework underneath, you’re still looking at a number of months just to get anything nontrivial fully written, tested, and deployed.

Some of the mindset behind ESB-type middleware is the desire to do everything in Java or C++. Many developers just want to write some code and plug it into a magical framework that transparently handles all the distribution, persistence, security, transactions, and reliability underneath. Chuckle. Underlying frameworks just grow and grow as they try to provide all this, and so they develop more bugs, more inconsistencies, more special cases, less flexibility, and less reliability as time goes on, not to mention foisting on the unwary the XML configuration hell that Patrick alluded to.

Eventually, these mono-language programmers end up completely incapable of addressing a problem unless they can see how to implement it in their One True Language™. Too many programmers I’ve encountered over the decades know only a single programming language, with many of them even lacking in that one language. They therefore tend to settle into areas that let them use that language no matter what, all day, every day, even if it means writing hundreds of lines of code that could easily be replaced by just a few lines of a language better suited to the problem.

Another mindset behind the ESB is that of the enterprise architect. Large enterprises believe they can save themselves a lot of money and trouble if they can just get the whole enterprise to agree on a single integration architecture. Unfortunately, that’s pretty much impossible because of autonomous divisions independently building/buying and deploying different solutions, because of acquisitions and mergers that bring in different technologies and tools, and because of legacy systems that never seem to go away. The ESB appeals to such architects because it’s touted as being so flexible that it can accommodate everything in the enterprise and then some. This flexibility allows the architect to mandate it as “the standard,” and then proceed to beat up the ESB vendor to add this and that, while also beating up teams within the enterprise to use the ESB for everything whether it really fits or not. The ESB becomes like one of those tools you see on late-night TV, where it’s a screwdriver and a hammer and a wrench and fishing reel and a paint brush, and plus you can flip it over and it can make you a grilled cheese sandwich. Of course, such tools always end up not doing any of those things particularly well.

So, why not just do things the way Patrick suggested: when faced with an integration problem, just grab the nearest dynamic language, such as Ruby or Python, grab the modules they provide that surely cover whatever protocols and formats are needed for the integration, and code it up in just a day or two? Well, ESB proponents would say that this would never do because you end up with way too many one-off solutions, while mono-compiled-language proponents would natter on pointlessly about type safety, compilers preventing errors, and performance. An assumption behind these arguments is that the integration problem being solved requires high performance, and that the solution will live for a long time. In my experience, these assumptions are rarely correct, especially the second one, since business rules change so rapidly nowadays that the integrations required to implement them change all the time as well.

Frankly, if I were an enterprise architect today, and I were genuinely concerned about development costs, agility, and extensibility, I’d be looking to solve everything I possibly could with dynamic languages and REST, and specifically the HTTP variety of REST. I’d avoid ESBs and the typical enterprise middleware frameworks unless I had a problem that really required them (see below). I’d also try to totally avoid SOAP and WS-*.

The only place I can imagine using an ESB would be if I had some legacy systems that just couldn’t be replaced, had to be made to talk to each other, and there were significant measured (as opposed to imagined) performance issues to be addressed within the integration. Under such circumstances, the right ESB could shine; I’ve seen those circumstances, and I’ve seen an ESB solve exactly that problem on a number of occasions, and solve it well. But on the whole, such scenarios don’t really arise that often.