Adopting, designing, and governing SOA well

SOA Best Practices Digest

Subscribe to SOA Best Practices Digest: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get SOA Best Practices Digest: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

SOA Best Practices Authors: Jason Bloomberg, Andy Thurai, Charles Araujo, Hollis Tibbetts, Pat Romanski

Related Topics: Cloud Computing, Agile Software Development, Change Leadership Journal, CIO, SOA Best Practices Digest, Microservices Journal

Agile Development: Blog Post

Enterprise Architecture: Ripe for Digital Disruption | @CloudExpo #Microservices

The secret to being an agile architect? Not architecting. The secret to managing an agile organization? Not managing.

Ever since I published The Agile Architecture Revolution, people have been confused by what I mean by Agile Architecture. The crux of the confusion: the difference between architecting a software system and architecting a human/software system.

If our goal of following Agile is to build good software, the theory goes, then we should ask ourselves what kind of architecture our software requires, and by definition, such architecture is Agile Architecture. To this day, if you Google "Agile Architecture," you're likely to uncover discussions that presuppose that definition - unless, of course, your search turns up something I've written.

When I use the phrase Agile Architecture, in contrast, I'm talking about a style of Enterprise Architecture whose primary goal is to make our organizations more agile - in other words, better able to deal with change, and to leverage change for competitive advantage.

To accomplish that enterprisewide goal, we must architect the organization itself - and what is an enterprise but a human/software system?

Emergence and Architecting the Enterprise
The key to Agile Architecture is emergence. In fact, business agility is the emergent property we seek from the Complex Adaptive System (CAS) we call the enterprise. (See my recent Cortex newsletter for a discussion of emergence as it relates to architecture).

Agile Architecture is a set of intentional acts we as individuals should take in order to get our enterprises to exhibit this most important of emergent properties. The question of the day, therefore, is what are these intentional acts? How do we actually go about architecting an enterprise to be agile?

At this point many of the enterprise architects reading this will want to argue over whether the Agile Architecture I'm discussing is actually Enterprise Architecture (EA). Frankly, I don't give a damn what you call it.

Arguing over what is or is not EA - or even worse, what EA should or should not be - is a complete waste of time, and happens to be one of the reasons executives wonder why they're spending so much money on EA in the first place.

For the sake of argument, therefore, let's just say that Agile Architecture is a reinvention of EA, which you can call EA if you want. But whatever you call it, it's essential to understand the difference between architecting a software system and architecting a human/software system, in particular at the enterprise level.

City Planning: The Wrong Metaphor for EA
To make this distinction, let's take a common metaphor for EA - the metaphor of city planning. Cities are made up of city blocks connected by streets, and within each block are buildings that contain homes, offices, etc. Those homes and offices are analogs for various software systems and applications. We might consider the blocks to represent the systems in a particular department or line of business.

City planners deal with city-wide issues like traffic, utilities, and the like, just as EAs should deal with enterprisewide issues like business/IT alignment, efficient business processes, etc. The tools planners use to influence their cities, including zoning regulations, public works investments, etc., are analogs for the tools of the traditional EA, namely the various artifacts and governance policies that are the EA's stock in trade. So, is city planning a useful metaphor for EA?

EAs who appreciate the city planning metaphor will point out that there are plenty of people in the city to be sure, and many of their activities influence or otherwise deal with the citizens in their enterprise. They will also rightly claim that city planning focuses on how cities deal with change, rather than how to assemble a static system like a model railroad layout.

But EA-as-city-planning is not Agile Architecture. In fact, it's just the opposite. The more planned a city is, the less agile it becomes. Why? Because city planning allows for change but not for emergence. The question we should be asking instead is: how do we produce the results we want from an unplanned city?

If we take the complicated problems we have today and seek to instill some sense of order and planning in order to achieve a particular final state, we're heading in the wrong direction. Even if we were able to accomplish this Sisyphean task, we'd be no more agile than when we started.

Self-Organization: The Most Important Tool in the Agile Architect's Tool Belt
If instilling order and planning is the wrong approach to EA, then clearly we must rethink our entire notion of EA. Once again, we can find the answer in complex systems theory, and the principle of self-organization.

My earlier Cortex also discussed the importance of self-organizing teams to achieving desirable emergent properties, with the important caveat that emergence won't appear at the two-pizza team size favored by Agile-centric organizations. Nevertheless, self-organization is the key to emergence, just not at the two-pizza level.

In fact, in the context of the organizations in which they participate, the behavior of individual humans is never emergent. If we focus on influencing individual human behavior, therefore, we're focusing on the wrong thing.

For example, if we can craft a test for the behavior we think we want and select for people who can pass the test, then we are selecting for non-emergent behavior. The better we get at selecting people who pass the test, the less agile our organization becomes.

Because every human being acts autonomously and is thus inherently unpredictable, the emergent properties of human/software systems are what CAS theorists refer to as strongly emergent, because you can't derive the emergent behavior by more careful analysis or control over subsystem behavior.

For this reason the iteration central to applying Agile Architecture is absolutely essential, because you can influence (but not control) the emergent behavior by iterating the initial conditions or other constraints that lead to effective self-organizing teams.

In fact, there's no way to know for sure ahead of time if some policy or process we might put in place to aid our self-organizing teams will actually result in better agility overall. Instead, we must try different things, see what emergent properties result, and feed back that information to improve our policies and processes.

The better and faster our organizations can gather the necessary business insight, feed it back to the decision making processes, and make the decisions that will drive business agility, the more agile our enterprises become.

The Intellyx Take: Is It Architecture?
In my opinion, the iteration of constraints and initial conditions that drive and influence self-organization within the enterprise is the actual role of an architect who is architecting emergent behavior - in particular, business agility.

You may call such activities something else - management practice or some such - and to be sure, we must reinvent management practice along the same lines as EA. But whatever we call it, there needs to be an understanding that creating the conditions that lead to effective self-organizing teams is itself an architectural activity, an activity separate from the architectural activities such teams undertake when their goal is to implement a software system.

Furthermore, self-organization at the team level is insufficient. Emergent patterns never appear at the team level, after all. We must also architect self-organization across teams, remembering all the while that the people within the teams are making their decisions about how they should behave and interact.

Managers cannot manage this self-organization from outside the self-organizing teams - either at the team level or across teams. The reason for this impossibility is brutally obvious once you see it: managing a team from outside is part of organizing that team - and if an external party takes that role, then the team is no longer self-organized.

If you're a manager and you think you'll be out of a job as a result, not to worry. Managers can still be on the teams as participants. Even outside the teams, executives have three important roles: communicate the strategic goals of the organization, delineate the constraints, and get out of the way.

The secret to being an agile architect? Not architecting. The secret to managing an agile organization? Not managing. At least, in any traditional sense of architecting or managing.

The good news is that many organizations are already well on their way to implementing this vision of emergent business agility - enough of them, in fact, that the ones who aren't with the program are increasingly at a competitive disadvantage.

This shift, in fact, is at the heart of digital disruption. Agile Architecture is the secret to weathering the storm. Disrupt or be disrupted - your choice.

Intellyx advises companies on their digital transformation initiatives and helps vendors communicate their agility stories. As of the time of writing, none of the organizations mentioned in this article are Intellyx customers. Image credit: Leeann Cafferata.

More Stories By Jason Bloomberg

Jason Bloomberg is a leading IT industry analyst, Forbes contributor, keynote speaker, and globally recognized expert on multiple disruptive trends in enterprise technology and digital transformation. He is ranked #5 on Onalytica’s list of top Digital Transformation influencers for 2018 and #15 on Jax’s list of top DevOps influencers for 2017, the only person to appear on both lists.

As founder and president of Agile Digital Transformation analyst firm Intellyx, he advises, writes, and speaks on a diverse set of topics, including digital transformation, artificial intelligence, cloud computing, devops, big data/analytics, cybersecurity, blockchain/bitcoin/cryptocurrency, no-code/low-code platforms and tools, organizational transformation, internet of things, enterprise architecture, SD-WAN/SDX, mainframes, hybrid IT, and legacy transformation, among other topics.

Mr. Bloomberg’s articles in Forbes are often viewed by more than 100,000 readers. During his career, he has published over 1,200 articles (over 200 for Forbes alone), spoken at over 400 conferences and webinars, and he has been quoted in the press and blogosphere over 2,000 times.

Mr. Bloomberg is the author or coauthor of four books: The Agile Architecture Revolution (Wiley, 2013), Service Orient or Be Doomed! How Service Orientation Will Change Your Business (Wiley, 2006), XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996). His next book, Agile Digital Transformation, is due within the next year.

At SOA-focused industry analyst firm ZapThink from 2001 to 2013, Mr. Bloomberg created and delivered the Licensed ZapThink Architect (LZA) Service-Oriented Architecture (SOA) course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, which was acquired by Dovel Technologies in 2011.

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting), and several software and web development positions.