The Thing No One Tells You About Microservices

Microservices are a powerful and effective approach to designing large systems, but they are not simple and the approach is often misunderstood to the point that I and most experts I know would probably agree that most teams that claim to be practicing

microservices

. Focus, right? I was discussing this with someone recently in the comments of one of my old videos and I thought it was worth exploring this idea a little further, so that's our topic today: what is the real value of

microservices

and why is the value missing? value of your microservices in your microservices design. Hi, I'm Dave Farley from Continuous Delivery and welcome to my channel.

If he hasn't been here before, hit subscribe and if you enjoy today's content, hit like too. I have talked several times here before about different aspects of microphone services as a focus of their advantages and challenges, in the conversation that I was referring to earlier and that prompted me to make this video, there were two ideas that I really wanted to discuss. The first is a broad definitional problem, that is, do the names we use for

thing

s really matter? Does it matter if what you call a micros service is different from what I call a micros service?

I think so. Because unless we can agree on the terminology, we really can't. communicate effectively and perhaps even more importantly, we can't really learn and progress as an industry with new ideas if the micro service is used only as a label and we ignore all the attributes of that microservice that define it, what does it mean? Let's say we build a microservices system if we ignore the definitions, we could also say we build a naming system without that basic level of understanding and agreement of the definitions, these

thing

s don't make any sense, the labels we apply are arbitrary and useless. unless we have that understanding, and yet I see more and more people making what seems to me to be exactly that argument: you're being too pedantic in your microservice definition of continuous integration, tdd, continuous delivery, or whatever else, well, no. as far as I can see.

I'm using the definition that was typically defined by the creators of the concept and here's the tricky part that we're so used to missing definitions for ideas like these in the software world that there are often many deceptively different definitions out there, so It is a reasonable criticism of me that I have chosen the definitions that I like to talk with, but generally, as I said, I try to use the definitions that were the definitions made by the creators of the idea, where I can find it for the microservices that I tend. to use this list of doio microservices as a shorthand for the more comprehensive description, which is quite defining of Martin Fowler's website, written by James Lewis and Martin together, so microservices are small, they focus on one task aligned with a limited, autonomous, independently implementable and loosely coupled context.

For other services, this is a fairly accepted definition, so if your microservices are not small, focus on a task aligned with the bounded context, are autonomous, can be employed independently, and are loosely coupled, then they are not microservices. , whatever you have, it may be useful, maybe even. nice design, but don't call them microservices, it's too confusing. The most important but also the most challenging of these attributes in terms of impact on development and ability to scale systems and teams is that they are independently deployable. I think it can be done. It's a good case, and I've seen it in the past, that all of these other things are primarily important because they help us maintain services independently.

Deployable microservices are independent. Implementation is really the whole game here because that's what allows us to organize development into many. Small autonomous teams and autonomy are important because that is one of the most important predictors of success from the dorom metrics findings. Smaller autonomous teams build better software faster. Let me pause and thank our sponsors. We are lucky to have sponsorship. Equal and transic experts, these companies offer products and services that are extremely well aligned with the topics we discuss here each week, so if you're looking for excellence in continuous delivery and software engineering, click the links in the description below to verify.

This may all sound great, but standalone implementation is actually setting a pretty high bar for design and requires a level of design sophistication that many teams struggle to achieve. Certainly, most so-called microservices systems I see do not have services that are Independently Deployable, that is, services that can be deployed without testing them together with any other surface before release. I've talked about it before about another aspect of this design problem, although it's something that came up in my discussion with that viewer in the comments, he said that the vast majority of microservices I've seen actually share a common database and a behind-the-scenes data model.

I agree that this is a common pattern and problem, but again, they are not microservices because they are not autonomous, we cannot change one independently of the other. or at least there are some types of changes that we cannot make without affecting both: we cannot change the storage schema in one service and not update and deploy the other service if they both depend on the same schema and the same storage my interlocutor gave an example processing an order for a customer where this can pose some design challenges. We could imagine two services processing orders and customer details or something similar.

The lure here for more traditional thinkers. More traditional, that is, those more accustomed to standardized databases. It would be to build something like this so that when an order comes in for something for a customer, the shared data store links the order to the correct customer account record, but that's why bounded context alignment is important here, fundamentally , there are some pretty strong limits to the scalability of normalized data. Data normalization is good because it means that there is a version of any fact, but it has a disadvantage is that interactions with those facts represent a form of coupling, so a Distributed systems where data is normalized are often much more difficult.

For a change, I talked about a topic related to this in this episode by joining our services through their data storage and representation. We are leaking information here and increasing the coupling between them, which makes it more difficult to change them. Microservice is a distributed systems model and in distributed systems, data synchronization problems can be very complicated, so it is worth avoiding them as much as possible and that is largely what the advice on Aligning with limited context and not sharing data if we limit ourselves to only sharing information through messages that travel between services has many advantages, it means that conversations are clear and well defined, there are no back doors, all interaction is done through the services APIs, but it also makes it clearer what the problem is.

What we have to solve is to define a conversation, an interaction protocol between the services that addresses the cases that interest us. Be lazy and resort to the now oversimplified model of normalized data to keep our services up to date. between us, we are breaking the microservice model in a pretty deep way, we have coupled two different bounded contexts for technical reasons, that's a bad idea, so now it's harder to work on the code for both Services, harder to test , more difficult to maintain. and if it is kept in separate repositories, it is also slow and annoying to change it because we have to jump between different repositories every time we want to do something completely opposite to the advantages that microservices should offer.

I would say that this design approach is poorly abstracted to the service level, although at the surface level using normalized data makes it easier to get started. It will be much more difficult to maintain in the future. This is a common problem with the design. I think the tactical solution often feels naive. It's easier at first, when you model the real problem you're interested in, and trying to solve it requires a little more thought. Let's imagine some of the operations we would like these services to support for the client. The Detail Service must be able to perform. create and register new customers in some way and retain their relative contact details, perhaps account ID, name, address, email, perhaps a telephone number, what aspects of this information are important in the context of placing an order , although almost none of them, all of them, we need to know if When placing an order, it is for which customer it is for, so all we need is the account ID.

The weakest relationship between customer details and an order is the account ID. I talk about three levels of modeling for services like these in this video, so our order processing service only needs to store the account ID of an order if this or some other service needs to access more customer details, such as the customer's email address, perhaps to let you know when the order was shipped, you should ask the customer details service to give us those details at the time we need them, you don't need to access them directly, we may We're worried that our very simple approach to order processing means that people can now place orders for non-existent accounts and we don't really want to overwhelm the system with useless orders this way.

We could solve this problem by having the Customer Details Service notify the Order Processing Service with new customers when they are added, so that now the Order Processing Service can maintain its own list. of valid account IDs and verify that the account ID attached to an order is in that list, although the data is no longer normalized, there are duplicate lists of account IDs in both services, this is not a bad thing, although it is a common strategy . To maintain relationships between data contained in different Services, yes, you have to do a little more housekeeping work to keep everything up to date and tidy, but the result is significant decoupling between services, making it easier to work with. in this entire system, for example.

I could write an order processing service test without the need for a Customer Service Detail because all I would need to do for my test is notify the order processing service that account 77 was added and then place the order for account 77. The big difference I see here is that instead of relying on generic technical levels of integration between the parts of our system, we make these conversations much more explicit, we abstract them to the level of the business problem we are trying to solve. solve, not at the level of The level of the technologies we are using means that conversations are simpler and if we follow the good advice of always translating information that transits between limited contexts, these conversations are now dramatically more loosely coupled and are more explicit and inherent as part of the design. of the system that makes sense to everyone, not just the technologists on the team, these conversations become important integration points in the system, more obvious and much more manageable, as a result, more defensible against change, this also makes things much easier when I want to add new features because the conversations that are already supported are now explicitly defined clearly through the services APIs and make much more obvious sense in the context of any service.

Let's imagine adding a customer accounting service to our design that listens to customers. are added and also for orders that are added and stores a customer's order list. We could imagine that the service keeps a running total of the amount spent by the customer on each order and perhaps sends a notification when the customer spent more than a certain threshold amount which means they now got a more privileged status or something. The real trick here is that all of these conversations take place at the level of abstraction that represents the problem domain, not the technicalities that we are in the language of placing orders and creating customers instead of adding records and defining select statements, this It is a much healthier level of coupling and I recommend it.

Thanks so much for looking. I would like to thank our sponsors again for supporting us and If you are interested in joining, check out the links in the description below.Thank God.