Chapter 4: Docker Compose

Authored by syscook.dev

What is Docker Compose?

Docker Compose is a tool for defining and running multi-container Docker applications. It uses YAML files to configure your application's services, networks, and volumes, making it easy to manage complex applications with multiple containers.

Key Concepts:

Service: A container that runs as part of your application
Network: Communication between services
Volume: Persistent data storage
Environment: Configuration through environment variables
Dependencies: Service startup order and relationships
Scaling: Running multiple instances of a service

Why Use Docker Compose?

1. Simplify Multi-Container Management

Compose makes it easy to manage applications with multiple services.

# docker-compose.yml
version: '3.8'
services:
  web:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      - db
      - redis
  db:
    image: postgres:13
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
  redis:
    image: redis:alpine

Benefits:

Single command to start all services
Automatic service discovery
Shared networking and volumes
Easy scaling and updates

2. Environment Consistency

Compose ensures consistent environments across development, staging, and production.

# Development environment
version: '3.8'
services:
  app:
    build: .
    volumes:
      - .:/app
    environment:
      - NODE_ENV=development
    ports:
      - "3000:3000"

3. Easy Service Orchestration

Compose handles service dependencies, networking, and data persistence automatically.

# Production environment
version: '3.8'
services:
  web:
    image: myapp:latest
    depends_on:
      - db
      - cache
    environment:
      - NODE_ENV=production
  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data
  cache:
    image: redis:alpine
    volumes:
      - redis_data:/data

volumes:
  postgres_data:
  redis_data:

Docker Compose File Structure

1. Basic Compose File

Minimal Configuration

# docker-compose.yml
version: '3.8'
services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"

Service Configuration Options

version: '3.8'
services:
  web:
    # Image or build
    image: nginx:alpine
    # build: .
    
    # Container name
    container_name: my-web
    
    # Port mapping
    ports:
      - "80:80"
      - "443:443"
    
    # Environment variables
    environment:
      - NGINX_HOST=localhost
      - NGINX_PORT=80
    # env_file: .env
    
    # Volume mounts
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./html:/usr/share/nginx/html
    
    # Network configuration
    networks:
      - frontend
    
    # Dependencies
    depends_on:
      - api
    
    # Restart policy
    restart: unless-stopped
    
    # Resource limits
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: '0.5'
        reservations:
          memory: 256M
          cpus: '0.25'

2. Advanced Configuration

Multi-Environment Setup

# docker-compose.yml (base)
version: '3.8'
services:
  web:
    image: ${IMAGE_NAME:-myapp}:${IMAGE_TAG:-latest}
    ports:
      - "${WEB_PORT:-3000}:3000"
    environment:
      - NODE_ENV=${NODE_ENV:-development}
    depends_on:
      - db
  db:
    image: postgres:${POSTGRES_VERSION:-13}
    environment:
      - POSTGRES_DB=${POSTGRES_DB:-myapp}
      - POSTGRES_USER=${POSTGRES_USER:-user}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-password}
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

Environment-Specific Files

# docker-compose.override.yml (development)
version: '3.8'
services:
  web:
    build: .
    volumes:
      - .:/app
    environment:
      - NODE_ENV=development
      - DEBUG=true
    ports:
      - "3000:3000"
      - "9229:9229"  # Debug port

# docker-compose.prod.yml (production)
version: '3.8'
services:
  web:
    image: myapp:latest
    environment:
      - NODE_ENV=production
    deploy:
      replicas: 3
      resources:
        limits:
          memory: 512M
          cpus: '0.5'
    restart: unless-stopped

Service Configuration Deep Dive

1. Build Configuration

Build from Dockerfile

version: '3.8'
services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
      args:
        - NODE_ENV=production
      target: production
    ports:
      - "3000:3000"

Build with Multiple Dockerfiles

version: '3.8'
services:
  web:
    build:
      context: .
      dockerfile: Dockerfile.web
  api:
    build:
      context: .
      dockerfile: Dockerfile.api

2. Networking Configuration

Custom Networks

version: '3.8'
services:
  web:
    image: nginx:alpine
    networks:
      - frontend
  api:
    image: myapp:latest
    networks:
      - frontend
      - backend
  db:
    image: postgres:13
    networks:
      - backend

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true

External Networks

version: '3.8'
services:
  web:
    image: nginx:alpine
    networks:
      - external_network

networks:
  external_network:
    external: true
    name: my_external_network

3. Volume Configuration

Named Volumes

version: '3.8'
services:
  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
  web:
    image: nginx:alpine
    volumes:
      - static_files:/usr/share/nginx/html

volumes:
  postgres_data:
    driver: local
  static_files:
    driver: local

Bind Mounts

version: '3.8'
services:
  web:
    image: nginx:alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./html:/usr/share/nginx/html
      - /var/log/nginx:/var/log/nginx

4. Environment Configuration

Environment Variables

version: '3.8'
services:
  web:
    image: myapp:latest
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://user:password@db:5432/myapp
      - REDIS_URL=redis://redis:6379
    env_file:
      - .env
      - .env.production

Environment File

# .env
NODE_ENV=production
DATABASE_URL=postgresql://user:password@db:5430/myapp
REDIS_URL=redis://redis:6379
WEB_PORT=3000
POSTGRES_DB=myapp
POSTGRES_USER=user
POSTGRES_PASSWORD=password

Advanced Compose Features

1. Service Dependencies

Startup Order Control

version: '3.8'
services:
  web:
    image: myapp:latest
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
  db:
    image: postgres:13
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d myapp"]
      interval: 30s
      timeout: 10s
      retries: 3
  redis:
    image: redis:alpine

Health Checks

version: '3.8'
services:
  web:
    image: myapp:latest
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

2. Service Scaling

Scale Services

# Scale web service to 3 instances
docker compose up --scale web=3

# Scale multiple services
docker compose up --scale web=3 --scale api=2

Load Balancing with Nginx

version: '3.8'
services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - web
  web:
    image: myapp:latest
    deploy:
      replicas: 3

3. Secrets Management

Docker Secrets

version: '3.8'
services:
  web:
    image: myapp:latest
    secrets:
      - db_password
      - api_key
  db:
    image: postgres:13
    secrets:
      - db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt
  api_key:
    external: true

External Secrets

version: '3.8'
services:
  web:
    image: myapp:latest
    secrets:
      - db_password
    environment:
      - DB_PASSWORD_FILE=/run/secrets/db_password

secrets:
  db_password:
    external: true
    name: myapp_db_password

Production Deployment

1. Production Compose File

Production-Ready Configuration

# docker-compose.prod.yml
version: '3.8'
services:
  web:
    image: myapp:latest
    deploy:
      replicas: 3
      resources:
        limits:
          memory: 512M
          cpus: '0.5'
        reservations:
          memory: 256M
          cpus: '0.25'
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
    environment:
      - NODE_ENV=production
    networks:
      - frontend
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./ssl:/etc/nginx/ssl:ro
    depends_on:
      - web
    networks:
      - frontend
    restart: unless-stopped

  db:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD_FILE=/run/secrets/db_password
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./backups:/backups
    networks:
      - backend
    secrets:
      - db_password
    restart: unless-stopped

  redis:
    image: redis:alpine
    volumes:
      - redis_data:/data
    networks:
      - backend
    restart: unless-stopped

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true

volumes:
  postgres_data:
    driver: local
  redis_data:
    driver: local

secrets:
  db_password:
    external: true

2. Deployment Strategies

Blue-Green Deployment

# docker-compose.blue.yml
version: '3.8'
services:
  web-blue:
    image: myapp:v1.0
    ports:
      - "3000:3000"
    networks:
      - blue

networks:
  blue:
    driver: bridge

# docker-compose.green.yml
version: '3.8'
services:
  web-green:
    image: myapp:v1.1
    ports:
      - "3001:3000"
    networks:
      - green

networks:
  green:
    driver: bridge

Rolling Updates

# Update service with rolling restart
docker compose up -d --no-deps web

# Update with new image
docker compose pull web
docker compose up -d web

Common Use Cases

1. Full-Stack Application

React + Node.js + PostgreSQL

version: '3.8'
services:
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    environment:
      - REACT_APP_API_URL=http://localhost:5000
    depends_on:
      - backend

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    ports:
      - "5000:5000"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://user:password@db:5432/myapp
    depends_on:
      - db
      - redis

  db:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - postgres_data:/var/lib/postgresql/data

  redis:
    image: redis:alpine
    volumes:
      - redis_data:/data

volumes:
  postgres_data:
  redis_data:

2. Microservices Architecture

API Gateway + Services

version: '3.8'
services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - api-gateway

  api-gateway:
    image: myapp/api-gateway:latest
    environment:
      - USER_SERVICE_URL=http://user-service:3001
      - ORDER_SERVICE_URL=http://order-service:3002
    depends_on:
      - user-service
      - order-service

  user-service:
    image: myapp/user-service:latest
    environment:
      - DATABASE_URL=postgresql://user:password@user-db:5432/users
    depends_on:
      - user-db

  order-service:
    image: myapp/order-service:latest
    environment:
      - DATABASE_URL=postgresql://user:password@order-db:5432/orders
    depends_on:
      - order-db

  user-db:
    image: postgres:13
    environment:
      - POSTGRES_DB=users
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - user_data:/var/lib/postgresql/data

  order-db:
    image: postgres:13
    environment:
      - POSTGRES_DB=orders
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - order_data:/var/lib/postgresql/data

volumes:
  user_data:
  order_data:

3. Development Environment

Hot Reload Development

version: '3.8'
services:
  web:
    build: .
    ports:
      - "3000:3000"
    volumes:
      - .:/app
      - /app/node_modules
    environment:
      - NODE_ENV=development
      - DEBUG=true
    command: npm run dev

  db:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp_dev
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    ports:
      - "5432:5432"
    volumes:
      - postgres_dev_data:/var/lib/postgresql/data

volumes:
  postgres_dev_data:

Compose Commands

1. Basic Commands

Service Management

# Start services
docker compose up

# Start in background
docker compose up -d

# Start specific services
docker compose up web db

# Stop services
docker compose down

# Stop and remove volumes
docker compose down -v

# Restart services
docker compose restart

# Restart specific service
docker compose restart web

Service Information

# List services
docker compose ps

# View logs
docker compose logs

# View logs for specific service
docker compose logs web

# Follow logs
docker compose logs -f web

# View service status
docker compose top

2. Advanced Commands

Service Scaling

# Scale services
docker compose up --scale web=3

# Scale specific service
docker compose up --scale web=3 --scale api=2

Service Updates

# Pull latest images
docker compose pull

# Rebuild and start
docker compose up --build

# Force recreate containers
docker compose up --force-recreate

Service Execution

# Execute command in service
docker compose exec web bash

# Execute command as root
docker compose exec --user root web bash

# Run one-time command
docker compose run web npm test

Troubleshooting

1. Common Issues

Service Dependencies

# Check service status
docker compose ps

# View service logs
docker compose logs web

# Check service health
docker compose exec web curl -f http://localhost:3000/health

Network Issues

# List networks
docker network ls

# Inspect network
docker network inspect myapp_default

# Test connectivity
docker compose exec web ping db

Volume Issues

# List volumes
docker volume ls

# Inspect volume
docker volume inspect myapp_postgres_data

# Remove volumes
docker compose down -v

2. Debugging Commands

Service Debugging

# View service configuration
docker compose config

# Validate compose file
docker compose config --quiet

# View service logs with timestamps
docker compose logs -t web

# View service resource usage
docker compose top

Summary

Docker Compose is a powerful tool for managing multi-container applications:

Simplifies orchestration of complex applications with multiple services
Provides consistent environments across development, staging, and production
Enables easy scaling and service management
Handles networking and storage automatically
Supports advanced features like health checks, secrets, and scaling
Integrates with Docker Swarm for production orchestration

By mastering Docker Compose, you can efficiently manage complex applications and deploy them consistently across different environments.

This tutorial is part of the SysCook DevOps series. Continue to the next chapter to learn about Docker networking and storage.

What is Docker Compose?​

Key Concepts:​

Why Use Docker Compose?​

1. Simplify Multi-Container Management​

2. Environment Consistency​

3. Easy Service Orchestration​

Docker Compose File Structure​

1. Basic Compose File​

Minimal Configuration​

Service Configuration Options​

2. Advanced Configuration​

Multi-Environment Setup​

Environment-Specific Files​

Service Configuration Deep Dive​

1. Build Configuration​

Build from Dockerfile​

Build with Multiple Dockerfiles​

2. Networking Configuration​

Custom Networks​

External Networks​

3. Volume Configuration​

Named Volumes​

Bind Mounts​

4. Environment Configuration​

Environment Variables​

Environment File​

Advanced Compose Features​

1. Service Dependencies​

Startup Order Control​

Health Checks​

2. Service Scaling​

Scale Services​

Load Balancing with Nginx​

3. Secrets Management​

Docker Secrets​

External Secrets​

Production Deployment​

1. Production Compose File​

Production-Ready Configuration​

2. Deployment Strategies​

Blue-Green Deployment​

Rolling Updates​

Common Use Cases​

1. Full-Stack Application​

React + Node.js + PostgreSQL​

2. Microservices Architecture​

API Gateway + Services​

3. Development Environment​

Hot Reload Development​

Compose Commands​

1. Basic Commands​

Service Management​

Service Information​

2. Advanced Commands​

Service Scaling​

Service Updates​

Service Execution​

Troubleshooting​

1. Common Issues​

Service Dependencies​

Network Issues​

Volume Issues​

2. Debugging Commands​

Service Debugging​

Summary​