One huge problem with building microservices is how they are used in new solutions without gradually turning the shiny, separated, autonomous services into slow-to-change, hard-to-regression test, inter-dependent big balls of mud
There is this temptation to wire things up and make changes to expand our services to fit a new solution, however this can quickly lead to accidental complexity
Using Domain Driven Design (DDD) principles specifically the concept of a bounded-context we can navigate these challenges and create solutions that delight our customers while keeping our engineering teams automous and happy
After Strategic DDD and Organisational Context-Mapping Comes …
The outcome of a strategic domain driven design initiative is a map of bounded-contexts within an organisation. This is the current state of your socio-technical landscape with teams, services, dependencies
Once we have this map and boundaries, teams can start to work on solutions within their contexts. Once we build this culture of “as-a-service” interaction model between teams in those boundaries the next question we hear from clients is “how do we build a solution that cuts across one or more bounded-contexts?”
Note at this point, there should be domain services and events that can be orchestrated into a working solution. However, there are key choices around how we thread the services and where we thread them to ensure the boundaries remain such that teams stay autonomous (and can independently deploy frequently to production, regardless of what this solution does with them)
The tricky part
The tricky part about building a new solutions is determining context – are they part of an existing context or are they part of a new context? Can this solution team exist isolation, using external services or are they an existing team implementing a new solution?
A solution that orchestrates can create dependency (accidentally or by design) through the endpoints and the model it consumes from the different bounded-contexts. An event-driven choreography based solution can be marginally better and introduce loose-coupling between the solution components and external services however it may introduce data coupling or worse create pockets of duplicate data which evolve independently overtime
It is important to be aware of these choices and understand service orchestration, choreography, consistency vs availability, coupling vs cohesion and more importantly the context in which you are building a solution
Solving with Orchestration
One option for building a solution is to pull services and orchestrates them to build a high value service of services. Key here is to recognise if this is part of an existing context if this is new bounded-context with a unique team. We see this in practise when a “Customer Services context” is extracted out from other contexts because it is considered fluid and more rapidly changing than core domains or because there is a different team in the organisation that handles this type of “front office” functionality. In such scenarios, we see application flows orchestrate calls to downstream core domain services to build high-value chain solutions
Solving with Choreography
Event based choreography is the other choice when building solutions. This is when instead of a solution coordinating services, listens passively to existing business events to react and take action. Event based choreography is seen in solutions when systems need to listen and synchronise information from existing systems across different business contexts and do not care about the information being slightly stale (consistency). These solutions need availability because the need is to have information close to the users in this context
We see this in Customer Management context where events from different transactional contexts are consumed to build a view of customer-related activities. The needs in this context are to serve customers in real-time, fast over a phone line and fetching data from downstream systems (orchestrating) can be delayed by using a local store
Using stateful streams
One challenge we find after DDD and microservices is the autonomous solution contexts tend to create their own model or replica of data and increasingly rely on customisation and synchronisation. The ubiquitous language that builds the model for the bounded-context can also introduce a fragment of an enterprise data entity which now lives across many systems in different contexts. This is why systems synchronise from a master system periodically, looking for the latest attributes they care about in their context. The master systems, over time, needs to facilitate higher frequency and volume of data exchange increasing the cost of ownership over time
One approach to improve data quality and reduce callbacks to core system of record is to use an event stream. This stream is populated by the various contexts with their data and acts like a central log of changes. New solutions would simply consume the streaming API to read from a stateful stream on events and event start writing to them to build extensibility
Another reason to use event streams is to synchronise larger volume of data than what the light-weight event pattern provides. We have had solutions that used streams over event/enrich patterns due to message ordering, size of events supported by messaging infrastructure and the ability to replay messages or process in a time window
Summary
Building new services using existing microservices can look easy however if we are to continue to maintain autonomy and not introduce coupling/dependency across teams then we need to respect the existing boundaries
It is therefore key for solution designers to understand if they are operating within an existing context or outside of it. Then they must decide if they need to simply orchestrate services or pick something with slightly delayed consistency model like event choreography. Finally they must look at the data impact and volume/size to see if the solution would benefit from a stateful stream.
Alok Mishra 