Rethinking Microservice Decomposition: A Team-Driven Approach
Integrating project management and technical trade-offs for agile system evolution
Recently, I have had some insights about microservice decomposition.
In the past, when discussing microservices decomposition, we usually start from the technical perspective, identify the domain boundary and then cut into the problem from the technical perspective to find out the appropriate decomposition solution.
The book Monolith to Microservices describes a number of technical “patterns” that cover most of the scenarios at a high level.
In general, there are three approaches:
- Split the database first, then the code
- Split the code first, then the database
- Split them both at once
However, the third approach is highly discouraged in the book, and in my personal experience as well.
For example, in the following diagram, when we want to separate module C into a microservice, two new components are created, microservice C and a new database.
Under the first approach, we would have the monolith begin to double-write the two databases and try to make the two databases consistent.
Once we can ensure that the data is consistent, we will start to develop Microservice C and then transfer some functions from the monolith to the new service.
On the other hand, in the second approach, we will start implementing Microservice C directly, but the new service will still use the original database.
When the new service is developed and the functionality of the monolith is transferred to the new service, then we will start to handle the database split.
There is a lot of technical selection in the book about which approach to take, and there are advantages and disadvantages to both approaches. Therefore, I highly recommend that book, and I believe that anyone who needs to implement a microservice decomposition needs to read it.
Nevertheless, I want to approach this issue from a different perspective.
Team Topology Matters
The choice should be based on team topology and project planning.
If the team topology follows Conway’s Law, i.e., Module C or Microservice C is governed by an independent team, then the second approach is much better.
The main reason is dealing with database split in the first place will face a lot of real-world challenges, which will inevitably slow down the project tempo and lengthen the development cycle.
When a system is no down time, it means a continuous write request, and how to keep the database consistent and up-to-date in such a scenario is a difficult problem.
There are many solutions, but none of them are silver bullets, and all of them need to be designed for a specific scenario.
For example,
- Use a strong consistency-guaranteed solution such as 2PC or XA. In practice, this is seldom used, because of the performance loss and the limitations.
- Double-write on the write side with continuous compensation. This is because the application’s double-write cannot be guaranteed to be completely consistent, and may be partially successful and partially failure due to an exception or scaling, so a compensation mechanism is needed to correct this on a regular basis.
- Use outbox pattern or event driven. By converting all write requests into events and ensuring event process is idempotent, we can ensure all databases are updated correctly. However, this is a huge change, and the real-time nature of the writes will be degraded.
The above three solutions are common, but they all have their limitations, and the amount of development is extremely large. Moreover, in order not to affect the data structure, we usually suspend the feature iteration until the database split is finished.
And that’s not the worst part. The database split is not the end of the story because it still takes time to develop the new service and the original monolith needs time to migrate the functionality.
In other words, when we do a database split, it will affect the project’s progress, and for a long time.
So what are the benefits of splitting the code first?
First of all, when the new service exposes the interfaces, the old monolith will be able to migrate functionality based on these new interfaces. Once the migration is complete, the old monolith can start running subsequent feature iterations without having to wait for the database split.
The impact of the database split is therefore limited to the microservices team and does not extend to other teams.
Conclusion
Decomposing microservices in a monolith is always a tough challenge.
Because there is no one-size-fits-all solution, most of them need to be customized based on the scenarios. The only way to become an expert in decomposing microservices is to experience more scenarios.
Most books only look at this matter from a technical point of view, but as I gain more and more experience, I realize that the technical aspect of decomposing microservices only takes up a small part of the picture, and it’s more about compromises and trade-offs between various realities.
I’m trying to approach this topic from a project management perspective because I’ve had the experience.
Originally published on Medium