Skip to main content

Chapter 5: Docker Networking & Storage

Authored by syscook.dev

What are Docker Networking and Storage?​

Docker networking enables communication between containers, hosts, and external networks. Docker storage provides persistent data storage for containers using volumes and bind mounts. Together, they form the foundation for containerized application data flow and persistence.

Key Concepts:​

  • Network: Communication layer between containers and external systems
  • Volume: Named storage managed by Docker
  • Bind Mount: Direct mapping of host directory to container
  • Network Driver: How containers communicate (bridge, host, overlay)
  • Volume Driver: How data is stored (local, nfs, cloud)
  • Data Persistence: Ensuring data survives container lifecycle

Why Use Docker Networking and Storage?​

1. Enable Container Communication​

Networking allows containers to communicate with each other and external services.

# Create custom network
docker network create my-network

# Run containers on same network
docker run -d --name web --network my-network nginx
docker run -d --name db --network my-network postgres

# Containers can communicate using names
docker exec web curl http://db:5432

Benefits:

  • Service discovery using container names
  • Isolated network segments
  • Load balancing and service mesh capabilities

2. Persist Application Data​

Storage ensures data survives container restarts and updates.

# Create named volume
docker volume create my-data

# Mount volume to container
docker run -d --name db \
  -v my-data:/var/lib/postgresql/data \
  postgres:13

# Data persists even if container is removed
docker rm db
docker run -d --name db-new \
  -v my-data:/var/lib/postgresql/data \
  postgres:13

3. Secure and Isolate Applications​

Networking and storage provide security boundaries and data isolation.

# Isolated network for sensitive services
docker network create --internal secure-network

# Encrypted volume for sensitive data
docker volume create --driver local \
  --opt type=tmpfs \
  --opt device=tmpfs \
  --opt o=size=1g,uid=1000 \
  secure-data

Docker Networking Deep Dive​

1. Network Types and Drivers​

Bridge Network (Default)​

# Create bridge network
docker network create --driver bridge my-bridge

# Run containers on bridge network
docker run -d --name web --network my-bridge nginx
docker run -d --name api --network my-bridge node:18

# Containers can communicate using names
docker exec web curl http://api:3000

Bridge Network Features:

  • Default network driver
  • Containers can communicate by name
  • Isolated from host network
  • Automatic DNS resolution

Host Network​

# Use host network
docker run -d --name web --network host nginx

# Container uses host's network directly
# No port mapping needed
curl http://localhost:80

Host Network Features:

  • Container shares host's network stack
  • No network isolation
  • Better performance
  • Direct access to host ports

Overlay Network (Swarm)​

# Create overlay network for swarm
docker network create --driver overlay my-overlay

# Deploy service on overlay network
docker service create --name web \
  --network my-overlay \
  --replicas 3 \
  nginx

Overlay Network Features:

  • Multi-host networking
  • Encrypted communication
  • Service discovery across hosts
  • Load balancing

2. Network Configuration​

Custom Network with Options​

# Create network with custom subnet
docker network create \
  --driver bridge \
  --subnet=172.20.0.0/16 \
  --ip-range=172.20.240.0/20 \
  --gateway=172.20.0.1 \
  my-custom-network

# Run container with specific IP
docker run -d --name web \
  --network my-custom-network \
  --ip 172.20.0.10 \
  nginx

Network with External Connectivity​

# Create network with external access
docker network create \
  --driver bridge \
  --opt com.docker.network.bridge.enable_icc=true \
  --opt com.docker.network.bridge.enable_ip_masquerade=true \
  --opt com.docker.network.bridge.host_binding_ipv4=0.0.0.0 \
  my-external-network

3. Service Discovery and Load Balancing​

DNS-Based Service Discovery​

# Create network
docker network create app-network

# Run multiple web services
docker run -d --name web1 --network app-network nginx
docker run -d --name web2 --network app-network nginx
docker run -d --name web3 --network app-network nginx

# Run load balancer
docker run -d --name lb --network app-network \
  -p 80:80 \
  nginx:alpine

# Configure load balancer
docker exec lb sh -c 'cat > /etc/nginx/nginx.conf << EOF
events {}
http {
    upstream backend {
        server web1:80;
        server web2:80;
        server web3:80;
    }
    server {
        listen 80;
        location / {
            proxy_pass http://backend;
        }
    }
}
EOF'

Health Checks and Service Discovery​

# Run service with health check
docker run -d --name web \
  --network app-network \
  --health-cmd="curl -f http://localhost:80 || exit 1" \
  --health-interval=30s \
  --health-timeout=10s \
  --health-retries=3 \
  nginx

Docker Storage Deep Dive​

1. Volume Types and Management​

Named Volumes​

# Create named volume
docker volume create my-data

# List volumes
docker volume ls

# Inspect volume
docker volume inspect my-data

# Use volume in container
docker run -d --name db \
  -v my-data:/var/lib/postgresql/data \
  postgres:13

Volume with Specific Driver​

# Create volume with local driver
docker volume create --driver local \
  --opt type=none \
  --opt o=bind \
  --opt device=/host/path \
  my-bind-volume

# Create volume with NFS driver
docker volume create --driver local \
  --opt type=nfs \
  --opt o=addr=192.168.1.100,rw \
  --opt device=:/path/to/nfs \
  my-nfs-volume

2. Bind Mounts and tmpfs​

Bind Mounts​

# Mount host directory
docker run -d --name web \
  -v /host/html:/usr/share/nginx/html:ro \
  nginx

# Mount with specific options
docker run -d --name web \
  -v /host/html:/usr/share/nginx/html:ro,Z \
  nginx

Bind Mount Options:

  • ro: Read-only mount
  • rw: Read-write mount (default)
  • Z: SELinux labeling
  • z: Shared SELinux labeling

tmpfs Mounts​

# Mount tmpfs for temporary data
docker run -d --name web \
  --tmpfs /tmp:rw,size=100m \
  nginx

# Mount tmpfs with specific options
docker run -d --name web \
  --tmpfs /tmp:rw,size=100m,uid=1000,gid=1000 \
  nginx

3. Volume Backup and Migration​

Volume Backup​

# Backup volume data
docker run --rm \
  -v my-data:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/my-data-backup.tar.gz -C /data .

# Restore volume data
docker run --rm \
  -v my-data:/data \
  -v $(pwd):/backup \
  alpine tar xzf /backup/my-data-backup.tar.gz -C /data

Volume Migration​

# Export volume
docker run --rm \
  -v my-data:/data \
  alpine tar czf - -C /data . > my-data.tar.gz

# Import volume
docker run --rm \
  -v my-data:/data \
  -i alpine tar xzf - -C /data < my-data.tar.gz

Advanced Networking Patterns​

1. Microservices Networking​

Service Mesh with Consul​

# docker-compose.yml
version: '3.8'
services:
  consul:
    image: consul:latest
    ports:
      - "8500:8500"
    command: agent -server -bootstrap-expect=1 -client=0.0.0.0 -ui

  web:
    image: nginx:alpine
    depends_on:
      - consul
    environment:
      - CONSUL_HOST=consul:8500

  api:
    image: myapp:latest
    depends_on:
      - consul
    environment:
      - CONSUL_HOST=consul:8500

API Gateway Pattern​

version: '3.8'
services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - api-gateway

  api-gateway:
    image: myapp/api-gateway:latest
    environment:
      - USER_SERVICE_URL=http://user-service:3001
      - ORDER_SERVICE_URL=http://order-service:3002
    depends_on:
      - user-service
      - order-service

  user-service:
    image: myapp/user-service:latest
    environment:
      - DATABASE_URL=postgresql://user:password@user-db:5432/users

  order-service:
    image: myapp/order-service:latest
    environment:
      - DATABASE_URL=postgresql://user:password@order-db:5432/orders

2. Database Clustering​

PostgreSQL Cluster​

version: '3.8'
services:
  postgres-master:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - postgres_master_data:/var/lib/postgresql/data
    networks:
      - db-network

  postgres-slave:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - postgres_slave_data:/var/lib/postgresql/data
    networks:
      - db-network
    depends_on:
      - postgres-master

  pgpool:
    image: pgpool/pgpool:latest
    environment:
      - PGPOOL_BACKEND_HOSTNAME0=postgres-master
      - PGPOOL_BACKEND_PORT0=5432
      - PGPOOL_BACKEND_HOSTNAME1=postgres-slave
      - PGPOOL_BACKEND_PORT1=5432
    ports:
      - "5432:5432"
    networks:
      - db-network
    depends_on:
      - postgres-master
      - postgres-slave

networks:
  db-network:
    driver: bridge

volumes:
  postgres_master_data:
  postgres_slave_data:

3. Load Balancing and High Availability​

HAProxy Load Balancer​

version: '3.8'
services:
  haproxy:
    image: haproxy:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    depends_on:
      - web1
      - web2
      - web3

  web1:
    image: nginx:alpine
    networks:
      - app-network

  web2:
    image: nginx:alpine
    networks:
      - app-network

  web3:
    image: nginx:alpine
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

Advanced Storage Patterns​

1. Distributed Storage​

GlusterFS Cluster​

version: '3.8'
services:
  gluster1:
    image: gluster/gluster-centos:latest
    volumes:
      - gluster1_data:/var/lib/glusterd
      - gluster1_brick:/brick
    command: glusterd --log-level=DEBUG

  gluster2:
    image: gluster/gluster-centos:latest
    volumes:
      - gluster2_data:/var/lib/glusterd
      - gluster2_brick:/brick
    command: glusterd --log-level=DEBUG

  gluster3:
    image: gluster/gluster-centos:latest
    volumes:
      - gluster3_data:/var/lib/glusterd
      - gluster3_brick:/brick
    command: glusterd --log-level=DEBUG

volumes:
  gluster1_data:
  gluster1_brick:
  gluster2_data:
  gluster2_brick:
  gluster3_data:
  gluster3_brick:

Ceph Storage​

version: '3.8'
services:
  ceph-mon:
    image: ceph/ceph:latest
    command: ceph-mon --cluster=ceph --mon-data=/var/lib/ceph/mon/ceph-mon --public-addr=0.0.0.0
    volumes:
      - ceph_mon_data:/var/lib/ceph/mon
    networks:
      - ceph-network

  ceph-osd:
    image: ceph/ceph:latest
    command: ceph-osd --cluster=ceph --osd-data=/var/lib/ceph/osd --public-addr=0.0.0.0
    volumes:
      - ceph_osd_data:/var/lib/ceph/osd
    networks:
      - ceph-network
    depends_on:
      - ceph-mon

volumes:
  ceph_mon_data:
  ceph_osd_data:

networks:
  ceph-network:
    driver: bridge

2. Data Persistence Strategies​

Database with Replication​

version: '3.8'
services:
  postgres-master:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - postgres_master_data:/var/lib/postgresql/data
      - ./postgres-master.conf:/etc/postgresql/postgresql.conf:ro
    command: postgres -c config_file=/etc/postgresql/postgresql.conf

  postgres-slave:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - postgres_slave_data:/var/lib/postgresql/data
      - ./postgres-slave.conf:/etc/postgresql/postgresql.conf:ro
    command: postgres -c config_file=/etc/postgresql/postgresql.conf
    depends_on:
      - postgres-master

volumes:
  postgres_master_data:
  postgres_slave_data:

Application with Data Backup​

version: '3.8'
services:
  app:
    image: myapp:latest
    volumes:
      - app_data:/app/data
      - ./backup:/backup
    environment:
      - BACKUP_SCHEDULE=0 2 * * *
    command: sh -c 'cron && node app.js'

  backup:
    image: alpine:latest
    volumes:
      - app_data:/data
      - ./backup:/backup
    command: sh -c 'while true; do tar czf /backup/app-$(date +%Y%m%d).tar.gz -C /data .; sleep 86400; done'

volumes:
  app_data:

Security Considerations​

1. Network Security​

Network Isolation​

# Create isolated network
docker network create --internal secure-network

# Run sensitive service on isolated network
docker run -d --name db \
  --network secure-network \
  postgres:13

Network Encryption​

# Create encrypted overlay network
docker network create \
  --driver overlay \
  --opt encrypted \
  secure-overlay

2. Storage Security​

Encrypted Volumes​

# Create encrypted volume
docker volume create \
  --driver local \
  --opt type=tmpfs \
  --opt device=tmpfs \
  --opt o=size=1g,uid=1000,nodev,nosuid,noexec \
  secure-volume

Volume Access Control​

# Create volume with specific permissions
docker volume create \
  --driver local \
  --opt type=none \
  --opt o=bind,uid=1000,gid=1000,ro \
  --opt device=/host/secure/path \
  secure-bind-volume

Monitoring and Troubleshooting​

1. Network Monitoring​

Network Inspection​

# List networks
docker network ls

# Inspect network
docker network inspect my-network

# Check network connectivity
docker exec web ping db

# Monitor network traffic
docker exec web netstat -tulpn

Network Performance Testing​

# Test network bandwidth
docker run --rm --network my-network \
  nicolaka/netshoot iperf3 -c db

# Test network latency
docker run --rm --network my-network \
  nicolaka/netshoot ping -c 10 db

2. Storage Monitoring​

Volume Inspection​

# List volumes
docker volume ls

# Inspect volume
docker volume inspect my-data

# Check volume usage
docker system df -v

Storage Performance Testing​

# Test storage performance
docker run --rm -v my-data:/data \
  nicolaka/netshoot dd if=/dev/zero of=/data/test bs=1M count=1000

Best Practices​

1. Networking Best Practices​

  • Use custom networks for application isolation
  • Implement service discovery with DNS
  • Use load balancers for high availability
  • Encrypt sensitive network traffic
  • Monitor network performance and security

2. Storage Best Practices​

  • Use named volumes for persistent data
  • Implement regular backups
  • Use appropriate volume drivers for your use case
  • Monitor storage usage and performance
  • Implement data encryption for sensitive information

3. Security Best Practices​

  • Isolate sensitive services on private networks
  • Use encrypted volumes for sensitive data
  • Implement proper access controls
  • Regular security audits and updates
  • Monitor for security vulnerabilities

Summary​

Docker networking and storage are fundamental components of containerized applications:

  • Networking enables communication between containers and external systems
  • Storage provides persistent data storage and sharing
  • Security is essential for protecting data and communications
  • Monitoring helps maintain performance and reliability
  • Best practices ensure efficient and secure operations

By mastering these concepts, you can build robust, scalable, and secure containerized applications.

This tutorial is part of the SysCook DevOps series. Continue to the next chapter to learn about Docker production deployment.