Monolith to API Gateway – how we migrated our API

The rusty past

Back in 2014-2015 there were no APIs for common usage in Home24. The first, important API consumer, native apps, arrived at a time when we had no support of any API and thus, we had to build it on top of existing systems. AT that time, the whole platform was a monolith with all the dependencies and applications acting as a whole. So the first challenge was to build a RESTful API on top of something that was never intended to be used for that.

Here’s where our old good friend ACL (Anti-Corruption Layer) came in handy:

Screen Shot 2017-08-29 at 17.44.06

In our case, ACL took care of all inconsistencies, provided additional business rules – a checking and enforcing layer, and was also used to signal unrecoverable issues in order to fix them in the current monolith production systems.

Heavens of the cloud

This approach was working… for a while… until we needed to release a feature that affected the holy “monolith” and the realization came that we would need two weeks to do it! Why? Because one small change in the part concerning us would cascade changes across the whole monolith and require QA & release for each part of it.

At this time, the proper solution hadn’t even been suggested, let alone developed. Everybody was reading about shiny microservices and how they make life easier allowing separate domains and dependencies – yet the whole infrastructure was just a bunch of physically reserved servers somewhere on a data centre. It would be a nightmare to try and reproduce the separated services approach there because while we would gain benefits on the software and development side, we would pay a heavy price on the infrastructure side.

And then came the obvious learning – we cannot take care of both infrastructure needs and the system architecture restructure. Fortunately, a nice and clean solution to the infrastructure issue arrived just in time – we decided to go the AWS path and this solved most of the issues with scaling fleets, providing secure and automatic ways to handle our infrastructure.

Having cloud at our back helped a lot – we could focus on the things we really needed without spending much time dealing with questions like “how can we fit this multi-service approach to our static physical machines clusters?”. This step was a necessity in order to proceed to the next stages of our migration.

The cross(road) of changes

With the powers the new infrastructure provided, we had almost unlimited possibilities to create an architecture which would be granular, scalable and so on. So, the next question was how should we do it? We had many domains to take care of but we also needed to integrate them into our API. Having a service is one thing, having it integrated, however, is quite different. Add the necessity of production systems, where everything should be in place from the first shot, and you can start to understand the challenge: all the services must be implemented and integrated in a way that conforms to the existing contracts.

So the migration of our APIs, as well as other products, was gradual. The less shifting of parts there is, the less error-prone the migration is. One step at a time, one service at a go. It is as if we were taking some parts of the monolith and slicing it out:

Screen Shot 2017-08-29 at 17.44.16

This transition phase is what takes the most time. The dependencies start to go to the microservices yet the process of integration is slow as there are many consumer applications and not only the API to consider.

But microservices bring another challenge to the game: too many pieces. For example – to implement a checkout, we would need services for accounts, authentication, cart, stocks, orders, catalog, and many others. And then there are problems for the consumer – now that the monolith is gone, the business logic of the relationship between the sliced parts is gone as well.

Key to the gate(way)

The problem for all the applications which now need to talk to many microservices instead of one monolith is obvious – it’s a complex thing and it’s not clear how to communicate properly with all of the parts and fulfill the business logic constraints, building a desired domain out of small micro-services blocks. Orchestration and abstraction – that’s what we lost with the monolith. So we started to think towards the solution and came up with an idea – what if we add a system part responsible for doing exactly that?

And here’s how our API Facade was born. Indeed, if we have all the micro-services we can just build an orchestration layer between them and the consuming application. And then we would have only strength and no weaknesses. It is also known as an API Gateway approach.

Screen Shot 2017-08-29 at 17.44.25

How does it work? Well, as we form domains out of our building blocks, micro-services, we shape these domains under the governance of API Facade as endpoints. Each endpoint stands for finished and defined domain and aggregates / orchestrates underlying microservices. But, truth be told, it’s a bold statement to assume there are no weaknesses..

.. because the API Facade is now a single point of truth (read: single point of failure). If something goes wrong with one endpoint, it will go wrong with all the others. The fleet is holding them all together. That’s why we know what has to be done in the future.

Perspectives and learnings

The API Gateway approach serves us extremely well and we now need to do the final step on our way – to separate our API Facade infrastructure so that all the endpoints will have their own independent stacks.

Screen Shot 2017-08-29 at 17.44.34

Each endpoint will be an independent module that can be deployed, maintained and developed separately, yet still being a part of the API Facade project while having all the benefits.

Conclusion

To conclude, let me wrap up our learnings:

  • Solve one problem at a time
  • Prepare for long-term migration
  • Be ready for compromises. You will live with your legacy for long time. Treat it well
  • Explore the market for ready solutions. AWS didn’t just bring all the keys to all the doors. We explored and found the right solutions ourselves.
  • Reiterate your decisions. Develop things knowing you will revisit them.

In the next posts of the cycle we will revisit the most significant and challenge-filled parts of our journey, dive more into details regarding specific topics of migration and/or technical solutions involved there. Stay tuned!

References list

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s