SOA: Dead or Alive?

It's been a year since Burton analyst Anne Thomas Manes launched a bombshell when she pronounced SOA dead on her burtongroup.com blog on January 5, 2009.

"Once thought to be the savior of IT," said Manes, "SOA instead turned into a great failed experiment-at least for most organizations."

Manes painted a dreary picture of SOA failure: "After investing millions, IT systems are no better than before. In many organizations, things are worse: costs are higher, projects take longer, and systems are more fragile than ever."

Companies had seen the writing on the wall, said Manes, and were abandoning SOA: "The people holding the purse strings have had enough. With the tight budgets of 2009, most organizations have cut funding for their SOA initiatives."

The recession economy, said Manes, put a nail in SOA's coffin. To make her point, a cartoon on her blog shows "the economy" as a meteor crashing into a dinosaur labeled "SOAsaurus" to cause its extinction.

"It's time to accept reality," said Manes. "SOA fatigue has turned into SOA disillusionment. Business people no longer believe that SOA will deliver spectacular benefits."

While pronouncing SOA dead, Manes declared the underlying concept of improving IT through services alive and well. She saw the SOA torch being passed to "the offspring" of SOA, the newer service-oriented technologies such as mashups, software as a service (SaaS), and cloud computing.

As Mark Little noted on markclittle.blogspot.com on January 7, 2009: "For the umpteenth time SOA is dead apparently. It's been shot so many times over the past year or so that you'd think it had a role in a Sam Peckinpah movie! However, maybe it's got 9 lives or you need to use a silver bullet, because each time it dies it keeps coming back."

Likewise, Joe McKendrick on zdnet.com on January 6, 2009, said that since he began writing his column in 2004 he has "heard SOA declared dead over and over again."

Manes' pronouncement ignited a storm of debate across the blogosphere. While many disagreed, a large number of analysts, SOA specialists, enterprise architects, and other IT practitioners chimed in their support for her view that SOA was dead or dying. Manes seemed to have opened an SOA wound in which all the accumulated ills poured out. Tales, opinions, and anecdotes were aired. The trails of blood indicated that SOA, if not dead, was seriously wounded.

For example, said Craig Cameron from Web and Flo on Manes' blog on January 7, 2009: "I would say that SOA might not be dead yet but I am hearing of a lot of SOA projects being delayed or killed off. It might be more appropriate to say that SOA may be going into hibernation for a bit."

Said Francis Carden on Manes' blog on January 6, 2009: "SOA is about IT doing it right and IT should be doing it right whatever the acronym. However, doing it right takes too long, costs too much, risks complete failure and time has proven, can't always be done at all."

In a BriefingsDirect debate hosted by Dana Gardner on "How Dead SOA Really Is" on January 26, 2009, the transcript of which is on briefingsdirect.blogspot.com, Manes says:

"Certainly, lots of people have refuted my claim. At the same time, I've had at least as many people, and probably more, I am dead-on right. My goal with the blog post was to at least get the conversation going, and I think I managed to do that effectively."

Among those glad to see SOA go was Kurt Cagle, who in "SOA is Dead? It's About Time!" on January 13, 2009, wrote:

"No doubt that SOA vendors will continue to try to prop up their particular dead parrot for a while, even as businesses axe SOA project after SOA project as being complex, unworkable and too fragile-just as no doubt, there will be those (few) business projects that will be held up as being successful because they happened to hit the (small) sweet spot where SOA as a model actually works reasonably well, but I for one am just as happy to see this rather ripe smelling bird now pushing up the daisies."

Likewise, said Jim Kita on his MS Architects blog on geekswithblogs.net on January 7, 2009: "It will not come as a great loss for me, as I have had a difficult time explaining SOA to technologists and making the business case to financial stakeholders."

Moreover, said Kita: "I will not mourn the loss of the hype surrounding the term SOA, because I will now be free to explain all of the misunderstanding and confusion away with a curt 'Oh, SOA died in '09.' "

A prevailing view among the "SOA is dead" camp was that SOA as practiced was vast, complex, expensive, and required an inordinate amount of time and attention.

Articulating this view was Microsoft Architect Evangelist Denny Boynton on blog.dennyboynton.com on January 15, 2009, who in his SOA "Autopsy" wrote:

"SOA is the ubiquitous panacea presented and re-presented over the course of the last ten or so years, a business cure-all that has generated a lot of hype and attention, but few tangible, measureable results. At some point in the middle part of this decade, SOA extended far beyond simply building a solid service-oriented infrastructure. It began to involve things like governance councils, enterprise service busses (ESBs) and a myriad other things. It was a simple, eloquent concept made huge and untenable."

Said a member of The Process Ninja on Manes' blog on June 28, 2009: "SOA is dead? Was it ever alive in the first place? SOA, in my humble opinion is the stuff of pipe dreams. For the vast majority of companies it is not achievable or realistic, rather it being the new plaything of the super-rich...and who is super rich anymore."

Even those who disagreed that SOA was dead supported the view that SOA as practiced was too large, complex, and expensive.

Said SOA consultant Jordan Braunstein on soa-today.blogspot.com on February 23, 2009: "I think BPM, BAM, and ESB initiatives (i.e. Big SOA) are suffering from being large, complex, and somewhat bloated efforts to gaining SOA momentum."

Similarly, said Beth Gold-Berstein on ebizq.net on February 19, 2009: "I agree that to date, the lack of experience, methodologies and skills has made SOA too complex for the average organization."

Said Joe McKendrick on his zdnet.com blog on January 15, 2010: "As it first developed, SOA was a luxury for the well-heeled. To put SOA into action, you needed robust tools, a proprietary application server that handled all the plumbing and protocols underneath, and expensive consultants to make it all happen. SOA was a nice high-margin business of vendors."

Signs that SOA was in serious trouble were seen in August 2007, when a study by Nuclear Research showed that only 37 percent of companies surveyed had achieved a positive return on investment for their SOA deployments. David O'Connell, senior analyst at Nucleus Research, was quoted as saying, "There are a handful of SOA success stories out there-but not many."

Supporting Nuclear Research's findings was an influential Gartner study released in November 2008 that showed interest in SOA falling dramatically. "Overall," said Gartner, "the two major reasons that organizations choose for not pursuing SOA are a lack of skills and expertise, and no viable business case."

Gartner's report caused observers to say that "SOA fatigue" had set in and that SOA had sunk into the "trough of disillusionment" of Gartner's hype cycles.

ZapThink analyst Jason Bloomberg on October 1, 2007, put out a Zapflash entitled "Who's Killing SOA?" Bloomberg's view was that virtually everyone was killing SOA. His list included integration platform vendors, enterprise architects, industry analysts, CIOs, business executives, and large consulting firms. Bloomberg described a host of SOA malpractices that members in each of these categories were guilty of committing.

A common knock against vendors is that they have been re-labeling products that are not truly SOA in nature and selling them as SOA enablers. Bloomberg described a typical integration platform vendor whose "proprietary, tightly-coupled integration middleware" did not lend itself "to SOA best practices."

In mimicking the vendors' SOA re-labeling approach, Bloomberg said: "Reinventing your company and rearchitecting your software from the ground up is far too expensive and time-consuming, and after all, you already have the older middleware in the can. The only option is to slap Web Services interfaces on your stuff, call it an Enterprise Service Bus (ESB), and sell it as SOA middleware. Hopefully your customers won't notice the old wine in new bottles. After all, that's what marketing is for!"

Similarly, an informationage.com report entitled "SOA Wars" on October 21, 2006, said that: "Sensing a boom market in the making, established enterprise software vendors are dressing established products in new SOA clothing, and new players are touting innovative 'pure play' SOA products."

Supporting this view was analyst David Lithicum, who on www.sys-con.com on September 9, 2008, wrote: "The fact is that dozens of vendors built integration tools when integration was hot. As the world moved toward SOA, driven by the hype, vendors just relabeled their tools 'SOA,' even though they still address integration and not architecture."

Moreover, said Linthicum: "We've all experienced the hype: 'We're a SOA tool, and we're here to help!' However, most SOA vendors out there don't understand the value of SOA, or even how to approach SOA. They focus on the tactical and not the strategic. There is not as much value to the business there."

The results were dire, according to Linthicum, as he explained: "I'm surprised I have to keep explaining this, but there are those out there who think that tactical selling of this type of technology is the most productive approach. It's clearly not if you've seen the results. VDA, or Vendor Driven Architecture, is killing SOA."

Dennis Boynton on blog.dennyboynton.com relates a tale in his role at a previous company in which "a major software vendor" explained to his SOA team that they were there in the hopes of "selling us SOA."

Said Boynton: "After I finished my fit of laughing and asked for clarification, they stated that, by buying their suite of SOA software, we would have everything we needed to implement an SOA. This wasn't unique to that vendor either. Many big name companies were actively engaging customers purporting that implementing a SOA was a matter of buying software licenses and/or hardware from them."

Carl Parziale on Manes' blog on January 7, 2009, tells of the wastefulness he experienced in vetting vendors' SOA products:

"Anne, as a Service Owner, I will offer you my support of your premise. I run a service owning team in a large enterprise and have spent a lot of wasted time evaluating, reevaluating, and re-reevaluating SOA-branded products that solve many imaginary problems. It's cost my team thousands of hours of distractions to work with the SOA-driven outside influences, usually ignoring the context and purpose of the actual Services. Many of the products add dubious value but cost significant time and money."

There is a widespread feeling that SOA was oversold by vendors and analysts. Mike Alvarez summed up the feeling when on Manes' blog on January 15, 2009, saying, "The term 'SOA' ended up being an overly-hyped term created by software and analyst firms to get some air time and sell products."

In the same vein, software architect Salam Khan on his blog on aslamkhan.net on March 16, 2009, wrote: "Is SOA dead? Not yet but the vendors are doing a great job of killing it with implementations."

Moreover, he said: "Is SOA heavyweight? No. But the vendors make it very, very heavyweight because that is the core of their economic model."

In "A SOA PostMortem" on Application Development Trends magazine's adtmag.com site on March 3, 2009, author John K. Waters reports Manes as saying that, unsurprisingly, most of the negative responses to her blog came from vendors.

However, as Waters tells us, one vendor executive who agreed with Manes was Mulesource CTO and co-founder Ross Mason. As Waters relates:

"The reason that SOA has become almost a dirty word, Mason says, is that too many vendors have picked it up as a marketing tool to sell products. 'They've diluted the message for short-term gain,' Mason says. "' SOA has nothing about product. It has nothing to do with technology, really, though technology is an enabler. But the promise of SOA is founded on sound principles.' "

"The truth is," Mason is quoted as saying, "the people who failed to deliver on the promise of SOA were the vendors."

Sharing this view was Philip Stander, who on Manes' blog on September 3, 2009, wrote:

"My opinion: The major product vendors once again got in the way with overcomplicated SOA stacks and got in with so much focus on technology (product) implementation that budgets were spent before legacy business functions could be migrated to a SOA-compliant architecture."

The causes attributed to SOA's death are many. Among them is the view that the association of ESBs with SOA was a major factor in undermining SOA's success.

Said Dr. Yigal Gur on Manes' blog on January 15, 2009: "It is quite unfortunate that the SOA - which is the pattern of EA (enterprise architecture), became a synonym of ESB in the heads of CIOs."

Supporting this view was Jason Bloomberg, who in his Zapflash in October 2007 said: "Not every integration vendor has taken this route, but many have. You can identify them when their sales people make statements like "ESBs are necessary for SOA."

Similarly, said Jim Kita on geekswithblogs.net on January 6, 2009: "If anyone has heard me speak, you will understand, though, that SOA is not a product (specific application server) or a specific technology (like ESB)."

Said Piet Jan Baarda in his whitepaper "Your SOA Needs a Business Case" in November 2008: "On one hand some software suppliers will lead you to believe that the acquisition of an
Enterprise Service Bus or the development of a number of Web Services is sufficient to be able to say 'Yes, we do SOA!.' Many customers happily join them and count on the next miracle tool to make all their information problems go away without the need to do complex things like changing the organization or the current way of working."

Poor data management also is seen as an SOA killer. A major promoter of this view is David Linthicum, who on ebizq.net on July 24, 2009, said: "What's missing within most typical SOA projects is the focus on the data, and that is killing SOA."

Linthicum reiterated this view on ebizq.net on August 16, 2009, declaring that: "Data and SOA go hand-in-hand. Those that neglect the data, data integration, and data quality are doomed. Trust me on that one."

Henrik Liliendahl Sørensen agreed, replying on Linthicum's ebizq.net blog on July 27, 2009: "The optimal Data Architecture is of great importance. But that said, even with the finest Data Architecture the Data Quality contained within may kill SOA and any other architecture. Only the killing is so brutal with SOA."

Likewise, said Ash Parikh on his blog on blogs.informatica.com on September 28, 2009: "I couldn't agree with Dave more - although SOA is about 'architecture,' most of us get caught up in handling the 'service' piece of it and forget that the key to a good architecture is its foundation - the 'data.' "

Lack of commitment, poor communication, and lack of cooperation within organizations also are seen as SOA killers.

Said Kerry on Manes' blog on January, 11, 2009: "Anne is correct about SOA being dead. It failed for the same reason that DCE failed-and the reasons are not technical. The reason for its failure is based on internal politics, culture and business rules complexity."

Mike Kavis on it.toolbox.com on January, 6, 2009, cited "lack leadership skills and emotional intelligence" as among the leading causes of SOA failure, explaining: "When was the last time you saw IT leadership who excelled in Technology, Business, and People Skills? You are lucky if they are good at two of these but to deliver transformational technology based initiatives they need to be skilled in all three."

Similarly, said Dan Zrobok on Manes' blog on January 9, 2009: "Anyway, the real reason that projects (SOA/Non-SOA) fail is that IT consistently fails to talk about projects in a manner that the business can understand."

In her original blog, Manes argued that SOA success required a massive effort, explaining: "Successful SOA (i.e., application re-architecture) requires disruption to the status quo. SOA is not simply a matter of deploying new technology and building service interfaces to existing applications; it requires redesign of the application portfolio. And it requires a massive shift in the way IT operates."

In the efforts that succeeded, she said, "SOA was just one aspect of the transformation effort. And here's the secret to success: SOA needs to be part of something bigger."

In "A SOA PostMortem" on adtmag.com on March 3, 2009, Manes is quoted as saying that "the success stories are few and far between, because most organizations are simply not willing to make a big enough commitment to SOA." Manes, says the report, "points to Bechtel, British Telecom, and MassMutual as examples of organizations that took the necessary plunge."

In the BriefingDirect debate on the SOA hosted by zdnet's Dana Gardner, Manes says that, "One of the primary reasons that a lot of SOA initiatives are failing is because people don't actually do the architecture."

Analyst David Linthicum, a member of the debate panel, argues that the problem is that smart architects are in short supply, explaining that "the majority of people out there who are wrestling around with architecture are ill-equipped to solve some of the issues. They have a tendency to focus in wrong areas. Anne hit this in her blog as well. It was brilliant."

Moreover, Linthicum is quoted as saying: "I wish there were something you could buy in a box or something you could download or some cloud you can connect to, but at the end of the day it's the talent of the people who are doing the job. That's where people have been falling down."

In SOA discussions, you sometimes hear practitioners use the term "sweet spot." Piet Jan Baarden, for example, says his aim is to "show the sweet spot is, of course, somewhere in the middle." Baarden argues that the middle is a delicate balance between simply deploying technology and tools and implementing services "across the board as the cure-all paradigm."

"There is no miracle tool and blind implementation of services is not a good idea," he says.

Even when the architectural concepts of SOA are understood, "the actual application is not easy," says Baarda. Like Manes, Baarda says he believes SOA fails when "organizations are not able to create the enterprise architecture and governance competence that is needed to achieve the benefits of the application of these concepts."

A lack of understanding SOA also is cited as a cause of failure. That SOA has been the subject of a vast amount of confusion is seen in the myriad conflicting interpretations, descriptions, pronouncements, and prescriptions.

Said Scott Francis on Manes' blog on January 5, 2009: "I've actually had conversations with IT professionals about their SOA initiatives where they had trouble defining what SOA stood for, and not understanding that one could have a service-oriented-architecture without buying a 'SOA' stack of software."

Similarly, said Mitch Barnett on Manes' blog on January 19, 2009:"What is a SOA? If you ask 10 technologists, you will get 10 different answers. Worse yet, how do you ever explain it to a business person (i.e. the one with the money) and more importantly, why should they care? If you look at the Wikipedia definition of a SOA, it is some 10 pages long - ridiculous!"

Said Mike Kavis on it.toolbox.com on January 6, 2009: "I am a member of many LinkedIn groups that focus on architecture. There is so much discussion on what the value of EA is and hundreds of different answers. If we can't agree on the value of EA amongst ourselves, how in the world is the CEO or CFO going to support an EA initiative?"

The efforts to establish SOA standards have not helped, rather hindered, the quest to bring order and consensus to SOA.

As Loraine Lawson noted on itbusinessedge.com on October 5, 2009, CDBI SOA consultant Lawrence Wilkes and his colleague, David Sprott, "are taking aim at the competing SOA standards, which they say use different definitions for the same terms and offer conflicting methods so that it's impossible to develop a common approach for SOA's life cycle."

David Sprott in "The Failure of SOA Standards!" on davidsprottsblog.blogspot.com on October 1, 2009, wrote that: "You will have heard the joke before: 'The nice thing about standards is that there are so many to choose from.' Sadly in SOA land this is reality."

The three main SOA standards groups (OASIS, The Open Group and OMG), said Sprott, "have each followed their own path, ignoring the blindingly obvious need to have consistency across their work."

Moreover, said Sprott, "The situation is so ridiculous that the three organizations resorted recently to publishing a document to explain how to navigate around the SOA architecture standards."

Lawrence Wilkes in "A Unified SOA" on lwsoa.blogspot.com on October 2, 2009: "Time after time we see that the lack of a common SOA framework is hindering SOA adoption in organizations."

Starkly contrasting views of the state of SOA and its future are seen in the views of Forrester colleagues Randy Heffner and Jim Kobielus.

Heffner, who has made a long-running practice out of promoting and explicating SOA, says in the executive summary of his report entitled "SOA Is Far from Dead-But It Should Be Buried" on May 11, 2009:

"Sparked by a tinderbox of economic jitters and technology backlash, a recent thread of industry discussion cries out, 'SOA is dead!' Although many have had fun with the discussion, it is in fact quite misguided. No prior industry initiative for IT architecture has had an impact as positive and broad-reaching as service-oriented architecture (SOA). But SOA's impact is only part of the story: You have many more technology initiatives besides SOA. You need a bigger architectural vision that encompasses SOA, business process management, event processing, Web 2.0, and much more besides. Although SOA is far from dead, it should be buried inside a larger vision. Forrester set the foundation of such a vision in 2005 with Digital Business Architecture."

In contrast to Heffner's sunny outlook, Kobielus painted a grim picture. In the January 26, 2009, BriefingsDirect debate hosted by zdnet's Dana Gardner, Kobielus said:

"The whole 'SOA is dead' theme struck a responsive chord in the industry, because there's a lot of fatigue, not only with the buzzword, but the very concept. It's been too grandiose and nebulous. It's been oversold, expectations have been built up too high, and governance is a bear."

Moreover, said Kobielus: "We all know the real-world implementation problems with SOA, the way it's been developed and presented and discussed in the industry. The core of it is services. As Anne indicated, services are the unit of governance that SOA begot.

We all now focus on services. Now, we're moving into the world of cloud computing and you know what? A nebulous environment has gotten even more nebulous. The issues with governance, services, and the cloud - everything is a service in the cloud. So, how do you govern everything? How do you federate public and private clouds? How do you control mashups and so forth? How do you deal with the issues like virtual machine sprawl?

The range of issues now being thrown into the big SOA hopper under the cloud paradigm is just growing, and the fatigue is going to grow, and the disillusionment is going to grow with the very concept of SOA. I just want to point that out as a background condition that I'm sensing everywhere."

The SOA failure rates that Manes describes lend credence to Kobielus's view. As Piet Jan Baarda related, a Burton Group study in June 2008 showed that "50% of the investigated 20 companies classify their SOA initiatives as a complete failure. 30% classifies it as neither successful or failure. Only 20% of the companies were prepared to call their SOA initiatives successful."

In March 2009, John K. Waters reported on adtmag.com that: "Last year, Manes and her Burton Group colleagues embarked on an extensive research effort to find out how companies in the midst of SOA initiatives were faring. "

The results, Manes is quoted as saying, showed that: "None of these companies, except for the very few success stories I mentioned, have been able to deliver on the promise of SOA, which is cost savings and increased business agility. In fact, in a lot of organizations, agility has been decreased, because they now have more moving parts, but don't have the appropriate management infrastructure in place and coordinated."

In his report from the International SOA Symposium on October 23, 2009, Joe Mckendrick on zdnet.com relates Manes saying that, "Business wasn't really interested in buying something called 'SOA," adding that in her own research, "fewer than 10% of companies have seen significant business value in their efforts."

The consensus among the "SOA is dead" camp is that SOA has been superseded by newer service-oriented technologies like cloud computing and Software as a Service (SaaS). Many see the future in Web-oriented technologies like REST.

However, as Forrester analyst Kobielus noted, there is so much confusion and uneasiness surrounding SOA and cloud computing that companies seem to be in a holding pattern as they attempt to sort out the direction in which things are moving.

In the BriefingDirect debate in January 2009, Manes says, "My core recommendation is to think big and take small steps."

In his report from the International SOA Symposium, McKendrick says the prevailing theme was "Next-Gen" SOA. He describes Manes' talk, in which, he said, she picked up the trail of her pronouncement that "SOA was dead, long live services." Manes is quoted as saying, "We have an opportunity at this point to resurrect SOA. We need a different approach, one based on architectural principles."

At a panel discussion later in the day, McKendrick says: "Anne further elaborated on her thinking behind the 'Dead' post, emphasizing her point that both end-user organizations and vendors are still too wrapped up in the idea of delivering some type of 'SOA' package, versus delivering agility and flexibility."

Loraine Lawson on itbusinessedge.com on October 9, 2009, points to the views of Dion Hinchcliffe, who she calls "one of the more significant voices in the Web 2.0 world." Hinchcliffe, she says, is arguably the first to expand upon the concept of Web-oriented architecture "in his much-referenced 2008 post, "What Is WOA? It's The Future of Service-Oriented Architecture (SOA)."

In his post, Hinchcliffe writes that SOA has failed, explaining that: "Unfortunately, despite two decades of experiments in heavyweight software engineering (the alphabet soup of EAI, SOA, ESB) for solving these types integration problems, we've seen relatively marginal improvements for most implementers despite heavy investments by businesses large and small."

Moreover, he says: "Even though Service-Oriented Architecture (SOA) initiatives around the world have the right goals, most efforts have fallen profoundly short of our desired levels of integration and improved business agility."

The promising answer to SOA's shortcomings, says Hinchcliffe, is seen in the model of the World Wide Web. The Web model, he says "provides a single, open, and unified information architecture that is consistent, easily consumed, extremely scalable, securable, very reusable, resilient, and highly federated."

"Traditional SOA is facing a crises of identity at this point," says Hinchcliffe, "particularly given fairly lackluster results for most, and WOA may just be the prescription we need to make SOA deliver the robust outcomes that we were formerly expecting of it."

It may be that the goal of achieving a simpler, easier, cheaper, and flexible IT infrastructure will remain elusive. In a March 2007 whitepaper from Equifax entitled "Why Business Leaders Should Care About SOA," the authors wrote:

"Over the years, developers have tried a variety of approaches to make development projects less grandiose and expensive. However, in spite of a plethora of technologies, methodologies and standards, nothing has proved to be the silver bullet that reduces waste, inflexibility and complexity. Each new trend seems to solve one set of problems only to create a host of new ones. Now the buzz is service oriented architecture, or SOA. It promises to have staying power, experts say."

	1. 03-09-2010 16:43
	Well, I don’t my own blog or anything, but I am not sure if I agree with everything said here. SOA definitely did not have the impact that we hoped it would, nor did it ever live up to technological expectations that we had, but I am more of an optimist in that the efforts that took SOA-initiators (standard committees, products, etc) can be re-used by general cloud vendors. I have said this, and will say it again, cloud is not the be all and end all to all problems – but for cloud to actually “succeed”, it needs to learn from SOA’s mistakes. One of which: too many standards. One last comment: one of the problems that cloud faces already is that we are not really sure what constitutes “success” for a cloud provider or a user. Registered

Art Sedighi