Pre-requisites
Some familiarity with Docker and Docker Compose is assumed. If you’re not familiar with these tools, I recommend you read Docker in Action and Docker Compose in Action. Altenatively, find your closest SysAdmin or DevOps friend and bother them until they help you out. A nice way to “motivate” them is to just start saying random buzzwords vaguely related to the concept of “containers” until they stop you and explain everything in detail :)
Why would anyone do this to themselves?
A sane mind is usually confronted with this question when it comes across its first kubernetes (or k8s as the cool kids call it) in the wild.
In an ideal world, one in which we aren’t bound by silly things like the laws of physics, just running a couple docker containers would be more than enough. Sadly, as we do not live in an ideal world, we have to deal with some small problems that come along with the impurity of the real world.
Lets imagine that we’re working on a small web app. Call it y.com. The application consists of five containers - one for each microservice - and we’ve deployed it on our very own server using a small docker compose file. Our server lives in a data center where we co-host with a small local company in northern Virginia.
A couple people found it interesting so over time we’ve garnered 3 million users worldwide. That many users means we encounter a couple small problems:
- Can one server handle all of the traffic?
- Can it do it without degrading service quality?
- What happens when we need to deploy a new version of the app?
- What happens when our server’s CPU randomly dies?
- What’s the user experience from people in Perth, Australia aka the other end of the world?
Points 1 and 2 can be somewhat solved by using a beefier server: one with a better CPU, more and faster RAM, and heaps of NVME SSD storage. This approach is called vertical scaling and its drawbacks are well known - it’s very expensive, it’s hard to manage, and it has a hard limit on how big you can make just one server. The other problems simply cannot be solved with vertical scaling. Clearly, running the whole app on a single server is not an option.
The generally accepted solution to these problems is to horizontally scale the app. This means that we run multiple servers connected into clusters, each of which is responsible for a subset of the traffic. The servers are then connected together using a load balancer. This approach is called horizontal scaling and it’s the most common way to scale a web app. With it we get some serious benefits:
- The app is distributed across multiple servers, so any one server can be weaker (cheaper) than our starting server.
- We can update the app on a per-server basis, with the rest of the cluster being unaffected.
- We introduce redundancy - with the failure of one machine, the rest pick up the slack. This protects us from the CPU failure scenario, but not from the whole building losing power. To guard against that, we can introduce a higher level of redundancy by having multiple clusters in different locations.
- Having multiple clusters world-wide means we can reduce the distance between the user and the server, which means the user experience is better.
This sounds excellent… in theory. In practice we have to actually manage all those servers. How many servers are you willing to SSH into and run a couple commands? What if the servers have different requirements so the compose files differ between them? And what happens with all of that when our app grows from 3 to 300 million users?
This is where we have to face our destiny.
Conducting the orchestra
Turns out orchestration doesn’t have anything to do with orchestras. Or maybe it does.
Orchestration is a collective name for the different methodologies with which we make administration of very large systems easier or even possible. We can imagine any number of different systems, both homogenous and heterogenous, upon which we want to automate some process. There exist many tools for orchestrating systems, mostly differing in their philosophy. The more famous ones are Ansible, Pulumi, Terraform, OpenShift, Nomad, and Kubernetes.
The first three focus on infrastructure as code, with the latter three being container management tools. The first group and second group can (and frequently are) used together to manage infrastructure.
For the purpose of this discussion, we’ll focus on just one of the tools. The biggest and buzzwordiest of them all.
Kubernetes
Kubernetes is an open source tool for initializing and managing containers. It is an exceptionally generic tool which means that you have to specify and worry about more things than you would have to do with a tool like Docker Compose. Its main components are:
- Container distribution