Why Every Software Engineer Needs to Understand Containers
Containers have become the standard unit of deployment across cloud platforms. Whether you are pushing to Kubernetes on AWS, GCP, or Azure, deploying a Vercel/Fly.io/Railway app, or working on a microservices architecture, understanding Docker and Kubernetes is no longer optional infrastructure knowledge — it is core engineering knowledge. This guide covers the practical fundamentals that get you to productive quickly.
Docker: Building and Running Containers
A Production-Ready Dockerfile
Most "hello world" Dockerfiles use a single-stage build that ships all development dependencies to production. Multi-stage builds solve this — the build stage has all your compilers and dev tools, while the final stage has only what the app needs to run:
# Dockerfile for a Node.js / Next.js application
# Stage 1: Install dependencies
FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --only=production
# Stage 2: Build the application
FROM node:20-alpine AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build
# Stage 3: Production image
FROM node:20-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
# Security: create a non-root user
RUN addgroup --system --gid 1001 nodejs && adduser --system --uid 1001 nextjs
# Copy only what's needed to run
COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
# Run as non-root user
USER nextjs
EXPOSE 3000
ENV PORT=3000
CMD ["node", "server.js"]
Key practices in this Dockerfile:
- Multi-stage build — The final image is Alpine-based (~50 MB vs ~1 GB for the build stage). Dev dependencies never reach production.
- Non-root user — Running as root inside a container is a security vulnerability. If an attacker breaks out of the app, they have root access to the container filesystem. The non-root user limits the blast radius.
- COPY package.json first, then source — Docker caches each layer. If you copy source first and source changes frequently, the
npm cilayer invalidates on every build. Copying package.json first means npm install only re-runs when dependencies actually change.
The .dockerignore File
# .dockerignore — files Docker should never copy into the build context
node_modules
.next
.git
.env
.env.local
.env.*.local
README.md
Dockerfile
.dockerignore
*.log
coverage
.nyc_output
Without .dockerignore, Docker sends your entire node_modules (often 200+ MB) to the build daemon on every docker build. This makes builds slow and can accidentally include local .env files with secrets in the image layer cache.
Essential Docker Commands
# Build an image
docker build -t myapp:latest .
# Build for a specific platform (e.g., for ARM vs x86 servers)
docker build --platform linux/amd64 -t myapp:latest .
# Run a container with port mapping and env vars
docker run -p 3000:3000 -e DATABASE_URL=postgresql://... myapp:latest
# Run with .env file
docker run -p 3000:3000 --env-file .env.local myapp:latest
# Shell into a running container for debugging
docker exec -it <container_id> sh
# View logs
docker logs -f <container_id>
# Remove all stopped containers and unused images
docker system prune -f
Docker Compose: Local Development Multi-Service Setup
# docker-compose.yml — local development with app + Postgres + Redis
version: '3.9'
services:
app:
build:
context: .
target: builder # Use the builder stage (with devDependencies)
ports:
- '3000:3000'
environment:
DATABASE_URL: postgresql://postgres:password@db:5432/myapp
REDIS_URL: redis://redis:6379
volumes:
- .:/app # Mount source for hot reload
- /app/node_modules # Anonymous volume: don't override container's node_modules
depends_on:
db:
condition: service_healthy
redis:
condition: service_started
command: npm run dev
db:
image: postgres:16-alpine
environment:
POSTGRES_PASSWORD: password
POSTGRES_DB: myapp
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ['CMD', 'pg_isready', '-U', 'postgres']
interval: 5s
timeout: 5s
retries: 5
redis:
image: redis:7-alpine
volumes:
- redis_data:/data
volumes:
postgres_data:
redis_data:
# Start all services
docker compose up -d
# View logs for all services
docker compose logs -f
# Run a one-off command in a service container
docker compose exec app npm run db:migrate
# Stop and remove containers (preserve volumes)
docker compose down
# Stop and remove everything including volumes (destructive)
docker compose down -v
Kubernetes: Container Orchestration
Kubernetes (K8s) manages the deployment, scaling, and operation of containerized applications across a cluster of machines. The core mental model: you declare desired state in YAML manifests, and Kubernetes continuously reconciles the actual state to match it.
Core Concepts
- Pod — The smallest deployable unit. Usually one container, sometimes a main container + sidecar containers (log collectors, proxies). Pods are ephemeral — they can be killed and recreated at any time.
- Deployment — Manages a set of identical Pods. You specify replicas, image, and update strategy. Kubernetes ensures the desired number of healthy Pods is always running.
- Service — A stable network endpoint for a set of Pods. Pods have dynamic IPs; Services have stable IPs and DNS names. Types: ClusterIP (internal), NodePort (external via node), LoadBalancer (cloud load balancer).
- Ingress — Routes HTTP/HTTPS traffic from outside the cluster to Services based on hostnames and paths. Requires an Ingress controller (nginx-ingress is most common).
- ConfigMap — Non-sensitive configuration data (env vars, config files) separate from the container image.
- Secret — Base64-encoded sensitive data (passwords, API keys, TLS certs). Mounted as env vars or files.
Deployment Manifest
# k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myregistry/myapp:v1.2.3 # Always use specific tags, never :latest in production
ports:
- containerPort: 3000
env:
- name: NODE_ENV
value: production
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: myapp-secrets
key: database-url
resources:
requests:
memory: '128Mi'
cpu: '100m'
limits:
memory: '512Mi'
cpu: '500m'
livenessProbe:
httpGet:
path: /api/health
port: 3000
initialDelaySeconds: 10
periodSeconds: 30
readinessProbe:
httpGet:
path: /api/ready
port: 3000
initialDelaySeconds: 5
periodSeconds: 10
Service and Ingress
# k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp-svc
namespace: production
spec:
selector:
app: myapp
ports:
- port: 80
targetPort: 3000
type: ClusterIP # Internal only; Ingress handles external traffic
---
# k8s/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-ingress
namespace: production
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: 'true'
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
rules:
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-svc
port:
number: 80
tls:
- hosts:
- app.example.com
secretName: myapp-tls
Essential kubectl Commands
# Apply all manifests in a directory
kubectl apply -f k8s/
# Check deployment status
kubectl rollout status deployment/myapp -n production
# List pods
kubectl get pods -n production
# View logs for a pod
kubectl logs -f deployment/myapp -n production
# Shell into a running pod
kubectl exec -it deployment/myapp -n production -- sh
# View recent events (great for debugging)
kubectl describe pod <pod-name> -n production | tail -30
# Roll back to the previous deployment
kubectl rollout undo deployment/myapp -n production
# Scale up manually
kubectl scale deployment/myapp --replicas=5 -n production
# Port-forward for local debugging without Ingress
kubectl port-forward svc/myapp-svc 3000:80 -n production