What Is Container Orchestration? Why Do We Need Kubernetes?

In the previous guide we learned: Docker packages an application together with its dependencies, so the application runs the same way in development, testing and production. Read Part 1 →

But packaging an application into a container is only the first step.

Then comes the real question:

In production, simply running

docker run

is not enough. A production system also needs to:

Manage many containers
Connect containers to each other
Scale when traffic goes up
Recover when a container crashes
Keep the application running even if a server fails
Manage updates

A system that does all of this is called a container orchestration platform.

1. What is the problem once containers exist?

Say your application is already packaged in a Docker container. Now a lot of questions come up:

How do I run the container in production?
What if the application also needs a database container?
What if it needs Redis, a messaging service or a backend API?
How will the containers communicate with each other?
How do I add more containers when traffic suddenly goes up?
How do I remove the extra containers when traffic drops?
Who restarts a container when it crashes?
What happens if an entire server fails?

Running one container by hand is easy:

1 Container
    ↓
Easy to manage

But as the number of containers grows, managing them by hand quickly becomes hard:

100 Containers
       ↓
1,000 Containers
       ↓
10,000 Containers
       ↓
Manual management becomes impractical

This is exactly why we need container orchestration.

2. What is container orchestration?

Container orchestration means automatically deploying, connecting, scaling, restarting, distributing and managing containers.

An orchestration platform usually handles:

Starting and stopping containers
Restarting failed containers
Connecting containers
Scaling up and scaling down
Load balancing traffic
Distributing containers across multiple servers
Managing application updates
Keeping the application highly available

In other words, you don’t have to manage every container by hand. You tell the orchestrator:

The orchestrator then works to maintain that state. This is called the desired state approach.

3. What is Kubernetes?

Kubernetes is a container orchestration platform.

It helps manage containerized applications across multiple machines. When multiple machines are managed together, that group is called a cluster.

Server 1
Server 2
Server 3
Server 4
    ↓
Kubernetes manages them
    ↓
Cluster

A simple definition:

A more complete definition:

Kubernetes is a container orchestration platform that can deploy, scale, load balance and manage hundreds or thousands of containers across a cluster.

4. Other Container Orchestration Tools

Kubernetes is not the only orchestration technology. Several other tools have existed, and some still do:

Tool	Created by	Main idea
Docker Swarm	Docker	Simple setup, fewer advanced features
Kubernetes	Google / Open Source	Powerful, flexible, widely used
Apache Mesos	Apache ecosystem	Advanced, but comparatively complex

Docker Swarm

Docker Swarm is relatively easy to get started with. If you know Docker, learning Swarm is fairly straightforward. But for complex production environments it usually has fewer advanced features and a smaller ecosystem than Kubernetes.

Docker
   ↓
Docker Swarm
   ↓
Multiple Docker hosts managed together

Apache Mesos

Apache Mesos can manage advanced distributed workloads. But the setup is complex, the learning curve is steep and configuration is relatively difficult. That is why Kubernetes has become more popular for general application orchestration.

Kubernetes

Kubernetes can feel a bit complicated at first. But its advantages are:

Powerful
Flexible
Highly configurable
Large ecosystem
Production-ready
Supports complex architectures

That is why Kubernetes is so common in modern infrastructure.

5. Why is Kubernetes so popular?

Kubernetes strikes a strong balance between power and flexibility. All the major cloud providers support Kubernetes:

Google Cloud
Microsoft Azure
Amazon Web Services

You can install and manage Kubernetes yourself, by hand. Or you can use a cloud provider’s managed Kubernetes service:

Google Cloud → GKE
Azure        → AKS
AWS          → EKS

GKE: Google Kubernetes Engine, Google Cloud’s managed Kubernetes service.
AKS: Azure Kubernetes Service, Microsoft Azure’s managed Kubernetes service.
EKS: Elastic Kubernetes Service, AWS’s managed Kubernetes service.

With a managed service, the cloud provider handles much of the operational work of running the Kubernetes infrastructure, and you can focus on deploying and managing your application.

6. The main benefits of container orchestration

Now the most important part: what does orchestration actually give a production system?

A. High Availability

High availability means the application stays available as much as possible, even when part of it fails. Without Kubernetes, if a single server fails, the application can go down:

Application
    ↓
Server 1 ❌
    ↓
Application down ❌

In a Kubernetes cluster, multiple instances of the application can run on different nodes:

App 1 → Node 1 ✅
App 2 → Node 2 ❌
App 3 → Node 3 ✅
             ↓
Kubernetes detects the failure
             ↓
New instance starts
             ↓
Application stays available

B. Load Balancing

Say your application has three instances. If all the traffic goes to one of them, that container can get overloaded. So the traffic needs to be distributed:

             Users
               ↓
         Load Balancer
        /      |      \
       ↓       ↓       ↓
    App 1    App 2    App 3

With Kubernetes Services and related networking mechanisms, traffic can be distributed across multiple application instances. This improves performance and reliability and reduces the load on any single container.

C. Scaling Up

Under normal traffic, three containers might be enough for the application. When traffic spikes, at times like Black Friday, you need more capacity:

Traffic ↑
   ↓
3 replicas → 10 replicas
   ↓
More capacity

Increasing the number of copies, or replicas, of an application is broadly called scaling up / scaling out.

D. Scaling Down

Traffic isn’t always high. At night, when there are fewer users, the extra containers can be removed:

Traffic ↓
   ↓
10 replicas → 3 replicas
   ↓
CPU, RAM and cost saved

E. Node Scaling

Here it helps to understand the difference between application scaling and node scaling. A node is a machine in the Kubernetes cluster: a physical server, a virtual machine or a cloud instance.

Scaling	Meaning	Example
Application scaling	Adding or removing app instances or replicas	3 Pods → 10 Pods
Node scaling	Adding or removing machines in the cluster	3 Servers → 5 Servers

When the existing nodes run out of CPU or RAM, there is no room to run new containers. Then a new node has to be added to the cluster:

Cluster
├── Node 1 — Full
├── Node 2 — Full
├── Node 3 — Full
└── Node 4 — New

7. Declarative Configuration

This is one of the most important concepts in Kubernetes. Declarative means telling Kubernetes the final desired state instead of giving it step-by-step commands.

Say your application needs three copies. With a manual approach, you would have to start each container and restart it yourself whenever it crashes. With Kubernetes, you say:

replicas: 3

Desired State = 3
Actual State  = 3
Status        = Matched ✅

If one instance crashes:

Desired State = 3
Actual State  = 2
        ↓
Kubernetes creates a new instance
        ↓
Desired State = 3
Actual State  = 3

This continuous process relates to the Kubernetes concept of reconciliation.

8. Simple real-world example

Say you have an online shopping application. The architecture:

Frontend
Backend API
Database
Redis
Payment Service
Notification Service

Each of these can be a separate containerized service. Without Kubernetes, you would have to start the frontend, backend, database and Redis by hand; configure the networking and ports; monitor the containers; restart them when they crash; create more containers when traffic goes up; and update deployments yourself.

With Kubernetes, you can define in configuration:

How many copies of the frontend and backend to run
How the services communicate
Where traffic goes
What happens when a container fails
How updates and scaling happen

Kubernetes then acts like the control layer of the whole system:

              Kubernetes
                  ↓
       ┌──────────┼──────────┐
       ↓          ↓          ↓
    Frontend    Backend     Redis
       ↓          ↓
     Users      Database
                  ↓
             Payment Service

9. Important terms

Container: An isolated environment where an application runs.
Docker: The technology/platform for building and running containers.
Orchestration: The automatic management of multiple containers.
Kubernetes: A container orchestration platform.
Cluster: A group of machines managed together by Kubernetes.
Node: A physical server, VM or cloud machine in the cluster.
Load Balancing: Distributing user traffic across multiple instances.
High Availability: Keeping the application running and reachable even when something fails.
Declarative Configuration: Defining the desired final state instead of giving step-by-step instructions.

10. Docker vs Kubernetes

What does Docker solve? How do I package an application and its dependencies consistently?

Application + Dependencies
           ↓
      Docker Image
           ↓
        Container

What does Kubernetes solve? How do I reliably manage hundreds or thousands of containers in production?

Containers
    ↓
Kubernetes
    ↓
Deploy · Manage · Scale
Restart · Load Balance · Recover

11. Complete flow

Developer
    ↓
Application Code
    ↓
Dockerfile
    ↓
Docker Image
    ↓
Container
    ↓
Many Containers
    ↓
Kubernetes
    ↓
Cluster
    ↓
Production

In production, Kubernetes manages deploying, scheduling, connecting, load balancing, monitoring, restarting, scaling and updating.

12. Restaurant analogy

A restaurant chain is a useful analogy for understanding container orchestration:

Docker Image = Recipe: A repeatable instruction for making a burger from bread, meat, cheese and sauce.
Container = Actual Burger: Burger 1, Burger 2 and Burger 3 can all be made from the same recipe.
Node = Kitchen: Each kitchen has a limited capacity.
Cluster = Kitchens: A group of restaurant kitchens managed together.
Kubernetes = Manager: Decides which kitchen gets the work, how many burgers get made and where more capacity is needed.

13. Mapping it to your own Kubernetes infrastructure

Say your Kubernetes cluster has three control plane nodes and two worker nodes. Together, they make up the Kubernetes cluster.

Control Plane Nodes
├── Server 1
├── Server 2
└── Server 3

Worker Nodes
├── Worker 1
└── Worker 2

When you deploy an app, Kubernetes can decide which worker node it runs on:

Kubernetes Cluster
│
├── Worker Node 1
│   ├── Your App
│   └── Backend
│
└── Worker Node 2
    ├── PostgreSQL
    └── Other Application

If the application crashes, Kubernetes can detect the failure and restart or replace the workload. This is the practical version of the concepts from the tutorial.

14. Does Kubernetes manage containers directly?

When we start learning, we say: Kubernetes manages containers. Conceptually, that is correct. But Kubernetes usually does not treat an individual container as its primary deployment unit. Instead, there is a concept called a Pod.

Kubernetes
    ↓
Pod
    ↓
Container

In a typical simple application, one Pod can contain one container. So the Pod will become very important later on.

15. The 7 most important points

Docker solves packaging: App + Dependencies → Container Image.
Managing production needs orchestration: Many Containers → Complex Management → Orchestration.
Kubernetes is an orchestration platform: it manages containers automatically.
A cluster is a group of machines: each machine is a node.
Kubernetes helps provide high availability: Failure → Recover → Application remains available.
Kubernetes can scale: more instances when traffic goes up, fewer when it drops.
Kubernetes is declarative: you state the desired state; Kubernetes maintains it.

Final simplified explanation

Docker packages an application into a container. But when a production environment has many containers and servers, managing everything by hand becomes very difficult. That is why we need container orchestration.

Container orchestration automatically deploys, connects, scales, restarts and load balances containers, distributes them across multiple servers and recovers them from failures.

Kubernetes is a container orchestration platform. Kubernetes manages many machines as a single cluster. Each machine in the cluster is called a node.

A very important philosophy in Kubernetes is the declarative desired state. You say, “I want 3 instances.” Kubernetes continuously works to keep three instances running.

One-line summary

Docker
↓
Container
↓
Many Containers
↓
Kubernetes
↓
Cluster
↓
Reliable Production