I have just realised a major error that I have been making when I have been talking to folks about SOA, I have not appeared excited about SOA and as a result I think I might have come off as negative or disapproving of the approach. I only had this cognition yesterday when I was asked advice on a couple of SOA issues.
I think what has happened is that I have been banging on for so many years now about elevating good Object Oriented design principals beyond a single application to level where it could apply to a collection of applications I have gotten sick of my own voice on the issue. Since 1998 when I started to playing with CORBA and model driven architecture I realized that the future was in items like components, application as services and distributed application where the boundary lines where greyed.
So what I plan to do in this post is describe to you what it is like to live in the promised land of SOA, what the SOA landscape actually looks like and most importantly how to get to this promised land. Just to cover the above three areas one could fill a book or two so I am going to distill thing down.
The promise of SOA is simply vastly increased agility to respond to organisational demands without paying more. This means that IT can develop new application quicker and integrate existing applications in new ways to meet changing business demands. It would be wildly irresponsible for me to say SOA is a silver bullet but it is very close.
Just as a side point there has been an attempt to define a maturity model for SOA which appear to based on the CMMI from Carnegie-Mellon Software Engineering Institute. I offer a slightly different take on this model, this is slightly more result orientated.
- Level 1 - Web enabled Applications (CRM - ERP - Productivity Applications)
- Internal & External Users
- Securer but course Access Control
- Connected & Discounted Operations
- Unified User Experienced
- Reliability & Scalability
- Platform Centric
- Level 2 - Composite, Mash-up & Portals
- Various internal & external data-sources
- Internal & External Users
- Personalisation & Roles Base Security
- Platform Agnostic
- Level 3 - Business Process Driven
- Systems and Application are described in terms of services they offer
- Low level of interdependency between services
- Services can be easily brought together to automate business processes
- High levels of standardisation across all systems yet allowing for localization
- Zero function duplication in systems
We know SOA works and we know it can drive down operating and development cost by about 20%, so what is the catch? If one exist it is that it is not easy, by my last count there where at least 50 standards group work on various aspects of SOA and the majority of these groups are not working together, anyhow more on this issues when we talk about how to get to the promised land.
This lead to what a SOA implementation looks like, instead of quoting the reference model here (which by the way is worth reading) I am going to share an image from Sun which I think is the best diagram I have seen that explains SOA from a before and after perspective. Unfortunately it lacks the rich details and references to technologies so I shall elaborate further on these.
The benefit of the above diagram is that it shows a before and view of an architecture and via this contrast we can see how different things are. For example within the SOA above when a new application needs to created the designers would be asking themselves how do they use the existing component's/services to stitch together a new application. They are composing a application rather then retro fitting or running pipes everywhere and affect data states in non confined or defined ways.
Within the level labeled as Data Repository in the before/after diagram we would expect to see our core corporate applications. Certain place our data warehouses and marts here, the interface at this level does not necessary have to direct access to the database, API's are equally acceptable this level. The other thing that is commonly found here are integration buses and wrapper type systems that expose the underlying infrastructure.
My general rule of thumb for a demarcation line at this level is that the interfaces exposed here should not reflect how the business works but how the infrastructure works, put in other words an engineer should able to map in his or her mind how a particular database or message bus participants in the interface. At the same time a business unit manager should not need to understand what is happening at this level and these interfaces should not map to any particular business function.
"Business Services" are designed around and built upon the interfaces that are exposed at the Repository level. The requirements for a business service should be mapped to a self contained function that ideally is performed across the organisation. In the before/after diagram 'Check Customer Status' is used as a example. There would by a number of different situations in the course of business that would required that status of a customer to be check and depending upon the outcome of that check certain actions would be allowed or not allowed.
It is the job of these business services to deliver clearly defined actions, other example of business services not included in the diagram would include, Ship Product, Validate Credit Card, Request Inventory, Search (Free Form & Structured) etc. Coupled with these business type services, there are other supporting services that help the architecture work, these are items like, Data Services which house dedupers, parers, verifiers etc, Security Services which cover such things as single-sign-on and authentication of users and applications.
All of these services are contained within registry that catalogs and describes each of the services. It is import to point out here even tho the registry is in essence a directory it sits insides a structure that provides a number of base function to the services, like version control, monitoring, common user interface templates, data dictionary etc, basically these are common services which can be consumed by services or are used to operate and monitor the services.
Now that we have a tangible collection of services an analyst can work with a business unit manager to string together the 'business services' into a business process, these processes sit within the composite application level. An example of a process could be insurance claim, this shown below.
Within the level labeled as Data Repository in the before/after diagram we would expect to see our core corporate applications. Certain place our data warehouses and marts here, the interface at this level does not necessary have to direct access to the database, API's are equally acceptable this level. The other thing that is commonly found here are integration buses and wrapper type systems that expose the underlying infrastructure.
My general rule of thumb for a demarcation line at this level is that the interfaces exposed here should not reflect how the business works but how the infrastructure works, put in other words an engineer should able to map in his or her mind how a particular database or message bus participants in the interface. At the same time a business unit manager should not need to understand what is happening at this level and these interfaces should not map to any particular business function.
"Business Services" are designed around and built upon the interfaces that are exposed at the Repository level. The requirements for a business service should be mapped to a self contained function that ideally is performed across the organisation. In the before/after diagram 'Check Customer Status' is used as a example. There would by a number of different situations in the course of business that would required that status of a customer to be check and depending upon the outcome of that check certain actions would be allowed or not allowed.
It is the job of these business services to deliver clearly defined actions, other example of business services not included in the diagram would include, Ship Product, Validate Credit Card, Request Inventory, Search (Free Form & Structured) etc. Coupled with these business type services, there are other supporting services that help the architecture work, these are items like, Data Services which house dedupers, parers, verifiers etc, Security Services which cover such things as single-sign-on and authentication of users and applications.
All of these services are contained within registry that catalogs and describes each of the services. It is import to point out here even tho the registry is in essence a directory it sits insides a structure that provides a number of base function to the services, like version control, monitoring, common user interface templates, data dictionary etc, basically these are common services which can be consumed by services or are used to operate and monitor the services.
Now that we have a tangible collection of services an analyst can work with a business unit manager to string together the 'business services' into a business process, these processes sit within the composite application level. An example of a process could be insurance claim, this shown below.
There a also a number of emerging technologies in the process construction area and they warrant a posting by themselves, but for now let just look at the key concept in this process area with is orchestration and choreography. We over complicate things sometimes, the basic difference between the two is that one's macro and the other micro. Orchestration is said to micro, focusing on one process and controls the messages that send and received by the process to services, in this area languages like BPEL are starting to develop and likely to become the future if it is not already the standard.
The macro or choreography side of things is not that clean, firstly choreography deals with the conversations that services have amongst themselves and how this conversation is defined, it protocol is not as clear as that of a application talking to a service (Micro - Orchestration). I personally am punting on WS-Choreography but due to huge vendor interest in this area there is competition and proprietary approaches like BPMN, BPSS, BPEL4WS, WSFL and XLANG.
Our user facing applications sit at the top of architecture levering services via processes, here is one of the key pay offs of SOA, these applications are composites they don't need to reinvent the wheel and there is uniform that comes as part of the structure, rather then being dependant on guidelines and good practice.
Which way to the promised land, I am sure everyone can see the benefit of SOA, but the question is how do we achieve it, how do you get to this place, where applications can put together quickly and easily that results in flexible structure?
Ones needs a map or at lease a general outline which will help avoid shallow waters and sea monsters, in reality the technical challenges, will fade into insignificance compared to the organisational, commercial and business issues. Therefore these need to faced upfront and addressed, the map below attempts to pull forward all the sea monsters and rough patches yet spaces them out so that you are not dealing with too many twisters at once.
To achieve a service orientated architecture and service oriented applications is journey that does not have an end point where all the benefits are deliver, there is no pot at the end of the rainbow, rather benefits come with each step that the organisation takes. I can hear the relief that previous statements brings, I think everyone is happy with instead gratification.
This is underpinned and is only made possible by an system architectural constitution, or what some call a governance. The Architectural Constitution is a manifestation of the commitment by key organisational members to a service orientated information technology approach. The constitution is a published document that sets out the best practices for the organisation, description of services and their suggested reuses and standards and process that can be leveraged to achieve reuse.
Hanging off this constitution are the standard practices of requirements gathering, problem solving and prototyping.
