6 min read

Adventures in Kubernetes (Part 6)

Featured Image

Slow and Steady - Winning the Kubernetes Marathon

Recently, a big cloud provider closed the accounts of an online platform using their services. In response, the company claimed that they will move their entire infrastructure and be "back up and running" in a short time. (Some reports claim 1 week, others claim 12 hours). Understandably, most responses from people working in the software industry who know something about hosting services found this claim difficult to swallow, and as was seen, a false claim. Moving over infrastructure, even if you believe you are running a cloud agnostic system using all open source software such as Kubernetes and Containers is a daunting task. There are many tiny dependencies and assumptions that get baked into a system over the months or years of development that only become obvious one you actually attempt to move. Moving from a monolithic application to a microservice architecture is a similar experience. It's not something you can do in a week or even a few months. To win the Kubernetes marathon you must take a slow and steady pace. In this last installment of our Kubernetes adventure story, we will explain why this is, and what one could do if time is of the essence.

The first thing we have to ask is why do we want to go slow? Microservices are a means to make the product agile and flexible, responding quickly to change. It's all about going to market faster, outperforming the competition, and being more competitive. Surely, we can't do that by going slow! There is an old adage, "Slow and steady win the race", or as it has been popularized by the US Navy, "Slow is smooth, and smooth is fast." and it's nice and catchy, but how does it apply to migrating to microservices and an orchestration system such as Kubernetes? To best understand why going slow is necessary we need to do a short dive into three concepts underlying a well designed microservice architecture and how they differ from components in a monolith.

  1. Domain Driven Design
  2. Service Oriented Architecture
  3. Single Source of Responsibility

Domain driven design, tells us that our application is made for a specific area of domain knowledge, and that we need a clear model for that domain, to allow for smooth changes to our software. That model of the domain will continually evolve as we gain better understanding and add more features to our products. In many companies, that model is more of a mental model, shared across different people in the organization, and rarely well understood by any one person. That is the model that our application uses as events and messages are sent across our system, and that is the model that changes when we alter how our system works. When you are migrating your system into something like Kubernetes, that model will have to change to make full use of the new technical realities you are introducing into the system. Because of this, it will take time for the new model to percolate through the development teams. Knowledge can only be gained so fast within a company, and if due care is not given to making sure there is alignment, it can cause more problems than it solves. In order to have a proper functioning model of our application and products, we need time to digest the new realities that we find ourselves in, and we need to be patient in making sure a solid core of developers who will be maintaining and deploying the system, understand it correctly. As long as that communication flow is open, and people are patient, creating the new functional model with Kubernetes in mind, will go well. If you rush it, there will be confusion and very possibly the wrong lessons will be learned going forward.

But, even if we devoted time to training everyone in Kubernetes for a couple weeks, and carefully laid out our plans for the new model in working with Kubernetes, we would still have reason not to rush. In the late 90s, many companies created monolithic enterprise applications centered around a single decoupled messaging service, called the Enterprise Service Bus (ESB). This was the backbone of Service Oriented Architecture (SOA) which also often ended up as a single point of failure within the system. For some, Kubernetes, reminds them of the ESB. A single point of failure which all services need to be tied to, creating what is often called a "distributed monolith". However, we can avoid this by moving slowly into Kubernetes, learning each building block, and making sure our services work independent of the rest of our infrastructure and other services. By its nature, Kubernetes is many independent pieces, different apis, which use a declarative approach to defining the environment. Yes, when doing a quick migration it's tempting to just throw everything into the same helm chart and create dependencies where none are needed. At the same time, by moving slowly we can discover dependencies that we wrongly assumed were independent of each other. We can also learn from SOA, in how we manage communication between the different parts of our enterprise, seeing each group of microservices as their own independent applications, all communicating through Kubernetes internal DNS based message system. Like any SOA transformation, we make our progress, one service at a time, redefining responsibilities along the way.

Distributed systems tend to have a few common complexities and design issues that need to be tackled differently than they would be in a traditional monolithic architecture, or dedicated VMs. A split brain problem, is a fun description for issues associated with a cluster of databases and deciding which version of the data is canon. In teams working with microservices and making deployment changes, a similar issue arises. If your teams are not ideal, you may have problems not knowing which team's version of the deployment is the correct one. Or worse, you may not know which teams' configuration details are the right ones for production. You can solve this by having a dedicated production team, or a devops team, or you can solve this by having each team be given sole responsibility for specific services, or you can create a system which is automated, leaving control to a bureaucratic process rather than any specific group of individuals. However you decide to solve these sorts of problems in distributed systems, the best way to go about it is slowly. Experiment which method works best for your context, and transition to it piece by piece. Odds are, at some point your company will go through all four methods, and will even find two other ways of doing it. However, if you rush, you'll go "all in" in one method and likely do it poorly, missing critical details until it becomes too late.

But what are you to do if time is of the essence, and you really do need to move everything over to Kubernetes in just a short amount of time? The answer to that is simple. Take your current environments, and find existing docker files or images that replicate the VMs you already use. Slowly, try to compile or run your software in the container, and respond to each error message and missing dependency. Deploy one container per VM, and add each VM to your Kubernetes cluster. Create the necessary ingress rules, or open up each VM as a load balancer with it's own IP and run it the same way you currently run your VMs. This will allow you to have a Kubernetes cluster, and some pods and containers, making waste with a lot of hardware. But it will be relatively quick. You won't be able to get any real benefit from Kubernetes, but at least you'll make the least of it, and have something to build from later. Having something is better than nothing, and you can always go slower, and smoother and thus get faster valuable results at a future time.

We hope sharing our adventure makes your adventure easier, with less hurdles and more realistic expectations of what's possible and what's valuable. This concludes our series on Kubernetes. Come and try our new plans, and ChatOnce product which wouldn't have been possible, without the adventure we embarked on.

Avi Kessner, Software Architect

Avi started as a Flash Animator in 1999, moving into programming as Action Script evolved. He’s worked as a Software Architect at OnceHub since September 2019, where he focuses on Cloud Native Automated workflows and improving team practices. In his free time, he enjoys playing games, Dungeons & Dragons, Brazilian jujitsu, and historical European sword fighting.