Journal of Distributed Software Engineering, Architecture and Design
Modernising Systems: Applying Domain Driven Design (DDD) for modern software re-achitecture
<div class="cs-rating pd-rating" id="pd_rating_holder_1819065_post_2579"></div>
<p class="wp-block-paragraph">In this post we focus on modernisation focussed on monolithic databases</p>
<p class="wp-block-paragraph">Data stores within an organisation can over time grow in complexity especially as engineers/database admins and business look to use the same data store in multiple ways for cost-effective solutions. We find that organisations optimise for cost by using common data stores to add on new extensions in the form of new tables, relationships, attributes etc. to serve new solution needs leading to growth inherent complexity. Interestingly, this measurable explicit early cost optimisation impacts agility and change leading to lost revenue which is implicit and hard to measure – which is why we feel it before we see it! </p>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/shared-database-many-contexts.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/shared-database-many-contexts.png?w=1024" alt="" class="wp-image-2584" width="545" height="357" /></a><figcaption class="wp-element-caption">Single Database, different contexts </figcaption></figure>
</div>
<p class="wp-block-paragraph">While the single data store paradigm can serve many solution contexts, it can become a bottleneck especially when rest of the enterprise is rapidly delivering change. Change as small as trying to change a few attribute types on a few tables can cascade massive programs of work due to the hidden impact to various consumers, this is why many organisations want to orient to more autonomous models with solution context specific data stores</p>
<p class="wp-block-paragraph">The key to achieving this state is going from a monolithic data store to a more microservice architecture and the way to doing this is via techniques like Domain Driven Design (DDD) which starts with business and business capability led change vs a technology led decomposition</p>
<h2 class="wp-block-heading">Approaching Monolith Decomposition</h2>
<p class="wp-block-paragraph">Where staring at a monolithic database decomposition problem there are two approaches to unravelling the complexity to start decomposing the database – the first approach is driven by technologist close to the beating heart of the problem and is motivated by finding technical decomposition solutions (SQL vs NoSQL, partitioning by region, microservices over a single database but presenting different services etc). This approach can render quick initial proof-of-concepts given the team is close to the monolith but it may struggle in the long run as it appears to be another “IT led refactoring exercise”</p>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/bottom-up.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/bottom-up.png?w=772" alt="" class="wp-image-2586" width="448" height="396" /></a><figcaption class="wp-element-caption">Option 1: Start with re-engineering the database knowing what you know and use technology led decomposition</figcaption></figure>
</div>
<p class="wp-block-paragraph">The second approach to this decomposition problem is to start with business capabilities with data needed to service those capabilities in specific solution contexts. We take this information model by context and break the monolith along business needs. Technology implementation comes later!</p>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/top-down.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/top-down.png?w=766" alt="" class="wp-image-2588" width="446" height="397" /></a><figcaption class="wp-element-caption">Option 2: Model data on Business capabilities and needs, then map this to the existing monolith </figcaption></figure>
</div>
<h2 class="wp-block-heading">Using DDD to lead Monolith Decomposition Programs</h2>
<p class="wp-block-paragraph">We can approach monolith decomposition using Domain Driven Design (DDD) which is a fantastic design technique for orienting software engineering to business domains. How do we do this then? The steps for doing this is the same as doing strategic and tactical DDD</p>
<p class="wp-block-paragraph">Some of the steps of this process are: discover and classify top-level business domains, finding capabilities for each of the business domains, mapping business capabilities to business processes and for each processes capturing the specific data requirements and actions (commands, queries). This process leads to a rich set of bounded contexts </p>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/contexts.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/contexts.png?w=984" alt="" class="wp-image-2590" width="410" height="245" /></a><figcaption class="wp-element-caption">Find the business contexts</figcaption></figure>
</div>
<p class="wp-block-paragraph">The key to the business led decomposing is building context specific domain model which contain details about what information management happens in a specific solution context</p>
<figure class="wp-block-image size-large"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/business-domain-model.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/business-domain-model.png?w=1024" alt="" class="wp-image-2592" /></a></figure>
<p class="wp-block-paragraph">This can look quite interesting as we look across business domains where some of the generic domain services like Customer, Document etc emerge as repeating themes. Lets take a note of these common services</p>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/domain-a.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/domain-a.png?w=1024" alt="" class="wp-image-2605" width="-79" height="-47" /></a><figcaption class="wp-element-caption">Domain Model: We build the model for each domain and each context </figcaption></figure>
</div>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/domain-models.jpeg"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/domain-models.jpeg?w=1024" alt="" class="wp-image-2595" width="786" height="243" /></a><figcaption class="wp-element-caption">Our top-down analysis leads to distinct business domains with different bounded contexts and models. Our next steps would be to map this to the monolith</figcaption></figure>
</div>
<p class="wp-block-paragraph">Once we have a domain model to support business domains and contexts within domains then we can start to map this back to the monolith which requires working with an ERD or database model to map the domain models to. Notice how this is “cloning” and “mapping” vs breaking an existing database</p>
<figure class="wp-block-image size-large"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/map-contexts-to-monolith.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/map-contexts-to-monolith.png?w=1024" alt="" class="wp-image-2598" /></a><figcaption class="wp-element-caption">Mapping the context back to the monolith to copy specific areas of existing monolith to the new data models</figcaption></figure>
<p class="wp-block-paragraph">The mapping of the domain model to the data model requires looking at the various contexts, domain entities and attributes and finding where they are managed in the monolithic database</p>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/mapping-to-data-model.jpeg"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/mapping-to-data-model.jpeg?w=850" alt="" class="wp-image-2601" width="509" height="614" /></a><figcaption class="wp-element-caption">Mapping to the ERD!</figcaption></figure>
</div>
<h2 class="wp-block-heading">What comes after domain model and mapping?</h2>
<p class="wp-block-paragraph">Our monolith decomposition journey has just started as we have a good design to build from. We can now take this design and start implementing changes incrementally using patterns like Strangler Fig pattern. All change should be test driven and we ought to be executing tests before starting to break things apart! </p>
<p class="wp-block-paragraph">So my last note on monolith decomposition is we need to follow Domain Driven Design (DDD) with domain driven tests which help us ensure we can test for consistency while we make fundamental changes</p>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><a href="https://alok-mishra.com/wp-content/uploads/2022/09/testing-is-key.png"><img src="https://alok-mishra.com/wp-content/uploads/2022/09/testing-is-key.png?w=1024" alt="" class="wp-image-2603" width="660" height="362" /></a><figcaption class="wp-element-caption">Testing is key! </figcaption></figure>
</div>
<h2 class="wp-block-heading">Summary</h2>
<p class="wp-block-paragraph">Monolith to microservices is a journey of software re-engineering motivated by accelerated change with autonomy to engineering and business teams. Monolithic databases can drive interesting band-aid solutions which add to complexity, slow down business agility and lead to large complex transformations</p>
<p class="wp-block-paragraph">The key to escaping this cycle is to start top-down, from first principles and lead with business capabilities along business lines. I like to say “<em>what if this as a 3rd party software which we consumed to run the business? how would that engineering team model its data store and integrate ?”</em></p>
<p class="wp-block-paragraph">Hope this post helps you with your journey with refactoring your monolithic database. I am keen to hear about your stories in this space</p>
In this post we focus on modernisation focussed on monolithic databases
Data stores within an organisation can over time grow in complexity especially as engineers/database admins and business look to use the same data store in multiple ways for cost-effective solutions. We find that organisations optimise for cost by using common data stores to add on new extensions in the form of new tables, relationships, attributes etc. to serve new solution needs leading to growth inherent complexity. Interestingly, this measurable explicit early cost optimisation impacts agility and change leading to lost revenue which is implicit and hard to measure – which is why we feel it before we see it!
Single Database, different contexts
While the single data store paradigm can serve many solution contexts, it can become a bottleneck especially when rest of the enterprise is rapidly delivering change. Change as small as trying to change a few attribute types on a few tables can cascade massive programs of work due to the hidden impact to various consumers, this is why many organisations want to orient to more autonomous models with solution context specific data stores
The key to achieving this state is going from a monolithic data store to a more microservice architecture and the way to doing this is via techniques like Domain Driven Design (DDD) which starts with business and business capability led change vs a technology led decomposition
Approaching Monolith Decomposition
Where staring at a monolithic database decomposition problem there are two approaches to unravelling the complexity to start decomposing the database – the first approach is driven by technologist close to the beating heart of the problem and is motivated by finding technical decomposition solutions (SQL vs NoSQL, partitioning by region, microservices over a single database but presenting different services etc). This approach can render quick initial proof-of-concepts given the team is close to the monolith but it may struggle in the long run as it appears to be another “IT led refactoring exercise”
Option 1: Start with re-engineering the database knowing what you know and use technology led decomposition
The second approach to this decomposition problem is to start with business capabilities with data needed to service those capabilities in specific solution contexts. We take this information model by context and break the monolith along business needs. Technology implementation comes later!
Option 2: Model data on Business capabilities and needs, then map this to the existing monolith
Using DDD to lead Monolith Decomposition Programs
We can approach monolith decomposition using Domain Driven Design (DDD) which is a fantastic design technique for orienting software engineering to business domains. How do we do this then? The steps for doing this is the same as doing strategic and tactical DDD
Some of the steps of this process are: discover and classify top-level business domains, finding capabilities for each of the business domains, mapping business capabilities to business processes and for each processes capturing the specific data requirements and actions (commands, queries). This process leads to a rich set of bounded contexts
Find the business contexts
The key to the business led decomposing is building context specific domain model which contain details about what information management happens in a specific solution context
This can look quite interesting as we look across business domains where some of the generic domain services like Customer, Document etc emerge as repeating themes. Lets take a note of these common services
Domain Model: We build the model for each domain and each context
Our top-down analysis leads to distinct business domains with different bounded contexts and models. Our next steps would be to map this to the monolith
Once we have a domain model to support business domains and contexts within domains then we can start to map this back to the monolith which requires working with an ERD or database model to map the domain models to. Notice how this is “cloning” and “mapping” vs breaking an existing database
Mapping the context back to the monolith to copy specific areas of existing monolith to the new data models
The mapping of the domain model to the data model requires looking at the various contexts, domain entities and attributes and finding where they are managed in the monolithic database
Mapping to the ERD!
What comes after domain model and mapping?
Our monolith decomposition journey has just started as we have a good design to build from. We can now take this design and start implementing changes incrementally using patterns like Strangler Fig pattern. All change should be test driven and we ought to be executing tests before starting to break things apart!
So my last note on monolith decomposition is we need to follow Domain Driven Design (DDD) with domain driven tests which help us ensure we can test for consistency while we make fundamental changes
Testing is key!
Summary
Monolith to microservices is a journey of software re-engineering motivated by accelerated change with autonomy to engineering and business teams. Monolithic databases can drive interesting band-aid solutions which add to complexity, slow down business agility and lead to large complex transformations
The key to escaping this cycle is to start top-down, from first principles and lead with business capabilities along business lines. I like to say “what if this as a 3rd party software which we consumed to run the business? how would that engineering team model its data store and integrate ?”
Hope this post helps you with your journey with refactoring your monolithic database. I am keen to hear about your stories in this space
Alok brings experience in engineering and architecting distributed software systems from over 20 years across industry and consulting. His posts focus on Systems Integration, API design, Microservices and Event driven systems, Modern Enterprise Architecture and other related topics
View all posts by alokmishra
Good article. I feel like a bit more details could be provided on the actual migration journey after the DDD model and mapping (including options of breaking up the database into smaller partitions)
Thank you Alex, great topic for the next post! I will say the migration journey starts with tests & lots of fun workshops to orient ourselves to the outside-in!
![https://alok-mishra.com/wp-content/uploads/2022/09/shared-database-many-contexts.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/shared-database-many-contexts.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/shared-database-many-contexts.png?w=1024"](https://alok-mishra.com/wp-content/uploads/2022/09/shared-database-many-contexts.png?w=1024"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/bottom-up.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/bottom-up.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/bottom-up.png?w=772"](https://alok-mishra.com/wp-content/uploads/2022/09/bottom-up.png?w=772"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/top-down.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/top-down.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/top-down.png?w=766"](https://alok-mishra.com/wp-content/uploads/2022/09/top-down.png?w=766"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/contexts.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/contexts.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/contexts.png?w=984"](https://alok-mishra.com/wp-content/uploads/2022/09/contexts.png?w=984"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/business-domain-model.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/business-domain-model.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/business-domain-model.png?w=1024"](https://alok-mishra.com/wp-content/uploads/2022/09/business-domain-model.png?w=1024"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/domain-a.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/domain-a.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/domain-a.png?w=1024"](https://alok-mishra.com/wp-content/uploads/2022/09/domain-a.png?w=1024"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/domain-models.jpeg"><img](https://alok-mishra.com/wp-content/uploads/2022/09/domain-models.jpeg"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/domain-models.jpeg?w=1024"](https://alok-mishra.com/wp-content/uploads/2022/09/domain-models.jpeg?w=1024"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/map-contexts-to-monolith.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/map-contexts-to-monolith.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/map-contexts-to-monolith.png?w=1024"](https://alok-mishra.com/wp-content/uploads/2022/09/map-contexts-to-monolith.png?w=1024"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/mapping-to-data-model.jpeg"><img](https://alok-mishra.com/wp-content/uploads/2022/09/mapping-to-data-model.jpeg"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/mapping-to-data-model.jpeg?w=850"](https://alok-mishra.com/wp-content/uploads/2022/09/mapping-to-data-model.jpeg?w=850"){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/testing-is-key.png"><img](https://alok-mishra.com/wp-content/uploads/2022/09/testing-is-key.png"><img){kind=link}
![https://alok-mishra.com/wp-content/uploads/2022/09/testing-is-key.png?w=1024"](https://alok-mishra.com/wp-content/uploads/2022/09/testing-is-key.png?w=1024"){kind=link}
Good article. I feel like a bit more details could be provided on the actual migration journey after the DDD model and mapping (including options of breaking up the database into smaller partitions)
Thank you Alex, great topic for the next post! I will say the migration journey starts with tests & lots of fun workshops to orient ourselves to the outside-in!