Environment Management

Master environment management strategies to ensure consistency, reliability, and security across all deployment environments.

Environment Management Overview

Effective environment management ensures that applications run consistently across development, staging, and production environments.

Environment Strategy

Infrastructure as Code (IaC)

IaC enables you to define and provision infrastructure using code, ensuring reproducibility and version control.

Terraform Implementation

1. Terraform Project Structure

terraform/
├── modules/
│   ├── networking/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── kubernetes/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   └── database/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
├── environments/
│   ├── dev/
│   │   ├── main.tf
│   │   ├── terraform.tfvars
│   │   └── backend.tf
│   ├── staging/
│   │   ├── main.tf
│   │   ├── terraform.tfvars
│   │   └── backend.tf
│   └── production/
│       ├── main.tf
│       ├── terraform.tfvars
│       └── backend.tf
└── terraform.tfstate

2. AWS Infrastructure Module

# modules/networking/main.tf
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-vpc"
    }
  )
}

resource "aws_subnet" "public" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone = var.availability_zones[count.index]
  
  map_public_ip_on_launch = true
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-public-subnet-${count.index + 1}"
      Type = "public"
    }
  )
}

resource "aws_subnet" "private" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index + length(var.availability_zones))
  availability_zone = var.availability_zones[count.index]
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-private-subnet-${count.index + 1}"
      Type = "private"
    }
  )
}

resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-igw"
    }
  )
}

resource "aws_eip" "nat" {
  count  = length(var.availability_zones)
  domain = "vpc"
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-nat-eip-${count.index + 1}"
    }
  )
  
  depends_on = [aws_internet_gateway.main]
}

resource "aws_nat_gateway" "main" {
  count         = length(var.availability_zones)
  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index].id
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-nat-${count.index + 1}"
    }
  )
}

# modules/networking/variables.tf
variable "environment" {
  description = "Environment name"
  type        = string
}

variable "vpc_cidr" {
  description = "CIDR block for VPC"
  type        = string
}

variable "availability_zones" {
  description = "List of availability zones"
  type        = list(string)
}

variable "common_tags" {
  description = "Common tags for all resources"
  type        = map(string)
  default     = {}
}

# modules/networking/outputs.tf
output "vpc_id" {
  description = "VPC ID"
  value       = aws_vpc.main.id
}

output "public_subnet_ids" {
  description = "Public subnet IDs"
  value       = aws_subnet.public[*].id
}

output "private_subnet_ids" {
  description = "Private subnet IDs"
  value       = aws_subnet.private[*].id
}

3. Kubernetes Cluster Module

# modules/kubernetes/main.tf
resource "aws_eks_cluster" "main" {
  name     = "${var.environment}-cluster"
  role_arn = aws_iam_role.cluster.arn
  version  = var.kubernetes_version
  
  vpc_config {
    subnet_ids              = var.subnet_ids
    endpoint_private_access = true
    endpoint_public_access  = true
    public_access_cidrs     = var.allowed_cidr_blocks
    security_group_ids      = [aws_security_group.cluster.id]
  }
  
  enabled_cluster_log_types = [
    "api",
    "audit",
    "authenticator",
    "controllerManager",
    "scheduler"
  ]
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-eks-cluster"
    }
  )
  
  depends_on = [
    aws_iam_role_policy_attachment.cluster_policy,
    aws_iam_role_policy_attachment.vpc_resource_controller
  ]
}

resource "aws_eks_node_group" "main" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "${var.environment}-node-group"
  node_role_arn   = aws_iam_role.node.arn
  subnet_ids      = var.subnet_ids
  
  instance_types = var.node_instance_types
  
  scaling_config {
    desired_size = var.desired_nodes
    max_size     = var.max_nodes
    min_size     = var.min_nodes
  }
  
  update_config {
    max_unavailable = 1
  }
  
  labels = {
    Environment = var.environment
  }
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-node-group"
    }
  )
  
  depends_on = [
    aws_iam_role_policy_attachment.node_policy,
    aws_iam_role_policy_attachment.cni_policy,
    aws_iam_role_policy_attachment.container_registry
  ]
}

# IAM Role for EKS Cluster
resource "aws_iam_role" "cluster" {
  name = "${var.environment}-eks-cluster-role"
  
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = {
        Service = "eks.amazonaws.com"
      }
      Action = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy_attachment" "cluster_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.cluster.name
}

resource "aws_iam_role_policy_attachment" "vpc_resource_controller" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.cluster.name
}

# Security Group for EKS Cluster
resource "aws_security_group" "cluster" {
  name        = "${var.environment}-eks-cluster-sg"
  description = "Security group for EKS cluster"
  vpc_id      = var.vpc_id
  
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  
  tags = merge(
    var.common_tags,
    {
      Name = "${var.environment}-eks-cluster-sg"
    }
  )
}

4. Environment-Specific Configuration

# environments/production/main.tf
terraform {
  required_version = ">= 1.0"
  
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  
  backend "s3" {
    bucket         = "myapp-terraform-state"
    key            = "production/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "terraform-state-lock"
  }
}

provider "aws" {
  region = var.aws_region
  
  default_tags {
    tags = {
      Environment = "production"
      ManagedBy   = "Terraform"
      Project     = "myapp"
    }
  }
}

module "networking" {
  source = "../../modules/networking"
  
  environment        = "production"
  vpc_cidr          = "10.0.0.0/16"
  availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
  
  common_tags = {
    Environment = "production"
    Project     = "myapp"
  }
}

module "kubernetes" {
  source = "../../modules/kubernetes"
  
  environment          = "production"
  vpc_id              = module.networking.vpc_id
  subnet_ids          = module.networking.private_subnet_ids
  kubernetes_version  = "1.28"
  node_instance_types = ["t3.large"]
  desired_nodes       = 3
  min_nodes          = 3
  max_nodes          = 10
  
  common_tags = {
    Environment = "production"
    Project     = "myapp"
  }
}

module "database" {
  source = "../../modules/database"
  
  environment       = "production"
  vpc_id           = module.networking.vpc_id
  subnet_ids       = module.networking.private_subnet_ids
  instance_class   = "db.r5.xlarge"
  allocated_storage = 100
  multi_az         = true
  backup_retention = 30
  
  common_tags = {
    Environment = "production"
    Project     = "myapp"
  }
}

# environments/production/terraform.tfvars
aws_region = "us-west-2"

# Network configuration
vpc_cidr = "10.0.0.0/16"
availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]

# Kubernetes configuration
kubernetes_version = "1.28"
node_instance_types = ["t3.large"]
desired_nodes = 3
min_nodes = 3
max_nodes = 10

# Database configuration
db_instance_class = "db.r5.xlarge"
db_allocated_storage = 100
db_multi_az = true
db_backup_retention_period = 30

Terraform CI/CD Pipeline

pipeline {
    agent any
    
    environment {
        AWS_CREDENTIALS = credentials('aws-credentials')
        ENVIRONMENT = "${params.ENVIRONMENT}"
    }
    
    parameters {
        choice(
            name: 'ENVIRONMENT',
            choices: ['dev', 'staging', 'production'],
            description: 'Target environment'
        )
        choice(
            name: 'ACTION',
            choices: ['plan', 'apply', 'destroy'],
            description: 'Terraform action'
        )
    }
    
    stages {
        stage('Terraform Init') {
            steps {
                dir("terraform/environments/${ENVIRONMENT}") {
                    sh '''
                        terraform init \
                            -backend-config="bucket=myapp-terraform-state" \
                            -backend-config="key=${ENVIRONMENT}/terraform.tfstate"
                    '''
                }
            }
        }
        
        stage('Terraform Validate') {
            steps {
                dir("terraform/environments/${ENVIRONMENT}") {
                    sh 'terraform validate'
                }
            }
        }
        
        stage('Terraform Plan') {
            steps {
                dir("terraform/environments/${ENVIRONMENT}") {
                    sh '''
                        terraform plan \
                            -out=tfplan \
                            -var-file=terraform.tfvars
                    '''
                }
            }
        }
        
        stage('Approval') {
            when {
                expression { params.ACTION == 'apply' || params.ACTION == 'destroy' }
            }
            steps {
                script {
                    def plan = readFile("terraform/environments/${ENVIRONMENT}/tfplan")
                    input message: "Review Terraform plan for ${ENVIRONMENT}. Proceed?",
                          parameters: [text(name: 'PLAN', description: 'Terraform Plan', defaultValue: plan)]
                }
            }
        }
        
        stage('Terraform Apply') {
            when {
                expression { params.ACTION == 'apply' }
            }
            steps {
                dir("terraform/environments/${ENVIRONMENT}") {
                    sh 'terraform apply -auto-approve tfplan'
                }
            }
        }
        
        stage('Terraform Destroy') {
            when {
                expression { params.ACTION == 'destroy' }
            }
            steps {
                dir("terraform/environments/${ENVIRONMENT}") {
                    sh 'terraform destroy -auto-approve -var-file=terraform.tfvars'
                }
            }
        }
    }
    
    post {
        always {
            archiveArtifacts artifacts: 'terraform/environments/**/*.tfplan', allowEmptyArchive: true
        }
        success {
            slackSend(
                channel: '#infrastructure',
                color: 'good',
                message: "✅ Terraform ${params.ACTION} completed for ${ENVIRONMENT}"
            )
        }
        failure {
            slackSend(
                channel: '#infrastructure',
                color: 'danger',
                message: "❌ Terraform ${params.ACTION} failed for ${ENVIRONMENT}"
            )
        }
    }
}

Configuration Management

Configuration management ensures consistent configuration across environments while allowing environment-specific overrides.

Environment Configuration Strategy

1. Configuration Hierarchy

# config/default.yml
application:
  name: myapp
  version: 1.0.0
  
server:
  port: 8080
  timeout: 30000
  
database:
  pool:
    min: 2
    max: 10
  timeout: 5000
  
logging:
  level: info
  format: json
  
features:
  new_ui: false
  beta_features: false

# config/production.yml
server:
  port: 80
  timeout: 60000
  workers: 4
  
database:
  pool:
    min: 10
    max: 50
  ssl: true
  replication:
    enabled: true
    read_replicas: 3
  
logging:
  level: warn
  
features:
  new_ui: true
  beta_features: false
  
monitoring:
  enabled: true
  sampling_rate: 0.1

2. Configuration Loading

// config-loader.js
const fs = require('fs');
const yaml = require('js-yaml');
const _ = require('lodash');

class ConfigLoader {
  constructor() {
    this.config = {};
    this.environment = process.env.NODE_ENV || 'development';
  }
  
  load() {
    // Load default configuration
    const defaultConfig = this.loadFile('config/default.yml');
    
    // Load environment-specific configuration
    const envConfig = this.loadFile(`config/${this.environment}.yml`);
    
    // Merge configurations (environment overrides default)
    this.config = _.merge({}, defaultConfig, envConfig);
    
    // Override with environment variables
    this.applyEnvironmentVariables();
    
    // Validate configuration
    this.validate();
    
    return this.config;
  }
  
  loadFile(filepath) {
    try {
      const content = fs.readFileSync(filepath, 'utf8');
      return yaml.load(content);
    } catch (error) {
      console.error(`Failed to load config file ${filepath}:`, error);
      return {};
    }
  }
  
  applyEnvironmentVariables() {
    // Override specific values with environment variables
    const envMappings = {
      'DATABASE_URL': 'database.url',
      'DATABASE_PASSWORD': 'database.password',
      'REDIS_URL': 'cache.url',
      'LOG_LEVEL': 'logging.level',
      'FEATURE_NEW_UI': 'features.new_ui'
    };
    
    Object.entries(envMappings).forEach(([envVar, configPath]) => {
      if (process.env[envVar]) {
        _.set(this.config, configPath, this.parseValue(process.env[envVar]));
      }
    });
  }
  
  parseValue(value) {
    // Parse boolean strings
    if (value === 'true') return true;
    if (value === 'false') return false;
    
    // Parse numbers
    if (/^\d+$/.test(value)) return parseInt(value, 10);
    if (/^\d+\.\d+$/.test(value)) return parseFloat(value);
    
    return value;
  }
  
  validate() {
    const required = [
      'application.name',
      'server.port',
      'database.url'
    ];
    
    const missing = required.filter(path => !_.has(this.config, path));
    
    if (missing.length > 0) {
      throw new Error(`Missing required configuration: ${missing.join(', ')}`);
    }
  }
  
  get(path, defaultValue = null) {
    return _.get(this.config, path, defaultValue);
  }
}

module.exports = new ConfigLoader();

Secrets Management

1. HashiCorp Vault Integration

// vault-client.js
const vault = require('node-vault');

class VaultClient {
  constructor() {
    this.client = vault({
      apiVersion: 'v1',
      endpoint: process.env.VAULT_ADDR,
      token: process.env.VAULT_TOKEN
    });
  }
  
  async getSecret(path) {
    try {
      const response = await this.client.read(path);
      return response.data;
    } catch (error) {
      console.error(`Failed to read secret from ${path}:`, error);
      throw error;
    }
  }
  
  async getDatabaseCredentials(environment) {
    const path = `secret/data/${environment}/database`;
    const secret = await this.getSecret(path);
    
    return {
      host: secret.host,
      port: secret.port,
      username: secret.username,
      password: secret.password,
      database: secret.database
    };
  }
  
  async getAPIKeys(environment) {
    const path = `secret/data/${environment}/api-keys`;
    return await this.getSecret(path);
  }
  
  async rotateSecret(path, newValue) {
    try {
      await this.client.write(path, { data: newValue });
      console.log(`Secret rotated successfully at ${path}`);
    } catch (error) {
      console.error(`Failed to rotate secret at ${path}:`, error);
      throw error;
    }
  }
}

module.exports = new VaultClient();

2. AWS Secrets Manager Integration

// aws-secrets-manager.js
const AWS = require('aws-sdk');

class SecretsManager {
  constructor() {
    this.client = new AWS.SecretsManager({
      region: process.env.AWS_REGION || 'us-west-2'
    });
  }
  
  async getSecret(secretName) {
    try {
      const response = await this.client.getSecretValue({
        SecretId: secretName
      }).promise();
      
      if ('SecretString' in response) {
        return JSON.parse(response.SecretString);
      } else {
        const buff = Buffer.from(response.SecretBinary, 'base64');
        return buff.toString('ascii');
      }
    } catch (error) {
      console.error(`Failed to retrieve secret ${secretName}:`, error);
      throw error;
    }
  }
  
  async createSecret(secretName, secretValue) {
    try {
      await this.client.createSecret({
        Name: secretName,
        SecretString: JSON.stringify(secretValue)
      }).promise();
      
      console.log(`Secret ${secretName} created successfully`);
    } catch (error) {
      console.error(`Failed to create secret ${secretName}:`, error);
      throw error;
    }
  }
  
  async updateSecret(secretName, newValue) {
    try {
      await this.client.updateSecret({
        SecretId: secretName,
        SecretString: JSON.stringify(newValue)
      }).promise();
      
      console.log(`Secret ${secretName} updated successfully`);
    } catch (error) {
      console.error(`Failed to update secret ${secretName}:`, error);
      throw error;
    }
  }
  
  async rotateSecret(secretName) {
    try {
      await this.client.rotateSecret({
        SecretId: secretName,
        RotationLambdaARN: process.env.ROTATION_LAMBDA_ARN
      }).promise();
      
      console.log(`Secret rotation initiated for ${secretName}`);
    } catch (error) {
      console.error(`Failed to rotate secret ${secretName}:`, error);
      throw error;
    }
  }
}

module.exports = new SecretsManager();

Kubernetes ConfigMaps and Secrets

1. ConfigMap Management

# configmap-dev.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: myapp-config
  namespace: dev
data:
  app.config: |
    {
      "environment": "dev",
      "logLevel": "debug",
      "features": {
        "newUI": true,
        "betaFeatures": true
      },
      "monitoring": {
        "enabled": true,
        "samplingRate": 1.0
      }
    }
  nginx.conf: |
    server {
      listen 80;
      server_name localhost;
      
      location / {
        proxy_pass http://app:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
      }
      
      location /health {
        access_log off;
        return 200 "healthy\n";
      }
    }

# secret-dev.yaml
apiVersion: v1
kind: Secret
metadata:
  name: myapp-secrets
  namespace: dev
type: Opaque
stringData:
  database-url: "postgresql://user:password@postgres:5432/myapp_dev"
  redis-url: "redis://redis:6379"
  api-key: "dev-api-key-12345"
  jwt-secret: "dev-jwt-secret-67890"

2. Using ConfigMaps and Secrets in Deployments

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: dev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: app
        image: myapp:latest
        ports:
        - containerPort: 8080
        
        # Environment variables from ConfigMap
        env:
        - name: NODE_ENV
          value: "dev"
        - name: LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: myapp-config
              key: logLevel
        
        # Environment variables from Secrets
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: myapp-secrets
              key: database-url
        - name: API_KEY
          valueFrom:
            secretKeyRef:
              name: myapp-secrets
              key: api-key
        
        # Mount ConfigMap as volume
        volumeMounts:
        - name: config
          mountPath: /etc/config
          readOnly: true
        - name: nginx-config
          mountPath: /etc/nginx
          readOnly: true
      
      volumes:
      - name: config
        configMap:
          name: myapp-config
          items:
          - key: app.config
            path: config.json
      - name: nginx-config
        configMap:
          name: myapp-config
          items:
          - key: nginx.conf
            path: nginx.conf

Environment Consistency

Docker Multi-Stage Builds

# Dockerfile with consistent environments
# Build stage
FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm ci --only=production

# Copy source code
COPY . .

# Build application
RUN npm run build

# Production stage
FROM node:18-alpine

WORKDIR /app

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001

# Copy built artifacts
COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist
COPY --from=builder --chown=nodejs:nodejs /app/node_modules ./node_modules
COPY --from=builder --chown=nodejs:nodejs /app/package*.json ./

# Set user
USER nodejs

# Expose port
EXPOSE 8080

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD node healthcheck.js

# Start application
CMD ["node", "dist/index.js"]

Environment Parity Checklist

# environment-parity-checklist.yml
development:
  infrastructure:
    - Docker Compose for local services
    - Minikube for Kubernetes testing
    - LocalStack for AWS services
  
  configuration:
    - Same configuration structure as production
    - Development-specific values
    - Debug logging enabled
  
  dependencies:
    - Same versions as production
    - Additional development tools
  
  data:
    - Anonymized production data
    - Seed data for testing

staging:
  infrastructure:
    - Mirror of production infrastructure
    - Scaled down resources
    - Separate network/VPC
  
  configuration:
    - Production-like configuration
    - Staging-specific endpoints
    - Enhanced logging
  
  dependencies:
    - Exact production versions
    - No development dependencies
  
  data:
    - Sanitized production data
    - Recent data snapshots

production:
  infrastructure:
    - Full production setup
    - High availability configuration
    - Auto-scaling enabled
  
  configuration:
    - Production configuration
    - Optimized settings
    - Error logging only
  
  dependencies:
    - Locked versions
    - Security-hardened
  
  data:
    - Live production data
    - Regular backups

Key Takeaways

Environment Management Best Practices

1. Infrastructure as Code: Version control all infrastructure
2. Configuration Management: Centralize and standardize configuration
3. Secrets Management: Secure sensitive data with proper tools
4. Environment Parity: Maintain consistency across environments
5. Automation: Automate environment provisioning and updates

Implementation Strategy

• Start with IaC: Define infrastructure as code first
• Centralize Configuration: Use configuration management tools
• Secure Secrets: Implement proper secrets management
• Test Environments: Validate changes in lower environments
• Document Everything: Maintain clear documentation

Common Patterns

• Environment-Specific Overrides: Base config + environment overrides
• Secrets Rotation: Regular rotation of sensitive credentials
• Infrastructure Testing: Test infrastructure changes before applying
• Environment Isolation: Strong network and access isolation
• Configuration Validation: Validate configs before deployment

---

Next Steps: Ready to master release management? Continue to Section 5.3: Release Management to learn about version control, feature flags, and release coordination.

---

Effective environment management ensures consistency, security, and reliability across all stages of the software delivery pipeline.