Advanced GitHub Actions Workflows

Master advanced patterns and optimization techniques for sophisticated GitHub Actions workflows.

Matrix Builds

Matrix builds allow you to run the same job across multiple configurations simultaneously, enabling comprehensive testing across different environments, versions, and configurations.

Basic Matrix Configuration

strategy:
  matrix:
    node-version: [16, 18, 20]
    os: [ubuntu-latest, windows-latest, macos-latest]

jobs:
  test:
    runs-on: ${{ matrix.os }}
    steps:
    - uses: actions/checkout@v3
    - name: Setup Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node-version }}
    - name: Install dependencies
      run: npm ci
    - name: Run tests
      run: npm test

Complex Matrix Strategies

Multi-Dimensional Matrix

strategy:
  matrix:
    node-version: [16, 18, 20]
    os: [ubuntu-latest, windows-latest]
    database: [postgresql, mysql, sqlite]
    include:
      # Include specific combinations
      - node-version: 18
        os: macos-latest
        database: postgresql
    exclude:
      # Exclude specific combinations
      - node-version: 16
        os: windows-latest
        database: mysql

Dynamic Matrix Generation

jobs:
  generate-matrix:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
    - id: set-matrix
      run: |
        # Generate matrix dynamically based on conditions
        MATRIX='{"node-version":["16","18"]}'
        if [ "${{ github.event_name }}" == "pull_request" ]; then
          MATRIX='{"node-version":["18"]}'
        fi
        echo "matrix=$MATRIX" >> $GITHUB_OUTPUT

  test:
    needs: generate-matrix
    runs-on: ubuntu-latest
    strategy:
      matrix: ${{ fromJSON(needs.generate-matrix.outputs.matrix) }}
    steps:
    - uses: actions/checkout@v3
    - name: Setup Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node-version }}

Matrix Optimization Techniques

Fail-Fast Strategy

strategy:
  fail-fast: true  # Stop all matrix jobs if one fails
  matrix:
    node-version: [16, 18, 20]
    os: [ubuntu-latest, windows-latest, macos-latest]

# Or disable fail-fast for independent testing
strategy:
  fail-fast: false  # Continue all matrix jobs even if one fails
  matrix:
    node-version: [16, 18, 20]
    os: [ubuntu-latest, windows-latest, macos-latest]

Maximum Parallel Jobs

strategy:
  max-parallel: 3  # Limit concurrent matrix jobs
  matrix:
    node-version: [16, 18, 20]
    os: [ubuntu-latest, windows-latest, macos-latest]

Conditional Execution

Advanced conditional logic enables smart workflow behavior based on various conditions and contexts.

Conditional Jobs

Branch-Based Conditions

jobs:
  test:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
    - run: echo "Running tests on main branch"

  deploy-staging:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/develop'
    steps:
    - run: echo "Deploying to staging"

  deploy-production:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
    - run: echo "Deploying to production"

Event-Based Conditions

jobs:
  ci:
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request'
    steps:
    - run: echo "Running CI for pull request"

  release:
    runs-on: ubuntu-latest
    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
    steps:
    - run: echo "Creating release"

Complex Conditions

jobs:
  deploy:
    runs-on: ubuntu-latest
    if: |
      github.ref == 'refs/heads/main' &&
      github.event_name == 'push' &&
      contains(github.event.head_commit.message, '[deploy]')
    steps:
    - run: echo "Deploying to production"

Conditional Steps

Step-Level Conditions

steps:
- name: Install dependencies
  run: npm ci
  if: github.event_name == 'pull_request'

- name: Run tests
  run: npm test
  if: always() && steps.install-deps.outcome == 'success'

- name: Deploy
  run: npm run deploy
  if: github.ref == 'refs/heads/main' && steps.test.outcome == 'success'

Environment-Based Conditions

steps:
- name: Deploy to staging
  run: npm run deploy:staging
  if: github.ref == 'refs/heads/develop'
  env:
    DEPLOY_ENV: staging

- name: Deploy to production
  run: npm run deploy:production
  if: github.ref == 'refs/heads/main'
  env:
    DEPLOY_ENV: production

Advanced Conditional Logic

Multiple Conditions

steps:
- name: Security scan
  run: npm run security:scan
  if: |
    (github.event_name == 'pull_request' && github.actor != 'dependabot[bot]') ||
    (github.event_name == 'push' && github.ref == 'refs/heads/main')

Function-Based Conditions

steps:
- name: Build
  run: npm run build
  if: contains(github.event.head_commit.modified, 'src/') || contains(github.event.head_commit.added, 'src/')

- name: Test
  run: npm test
  if: contains(github.event.head_commit.modified, 'src/') || contains(github.event.head_commit.added, 'tests/')

Parallel vs Serial Execution

Understanding when to use parallel or serial execution is crucial for workflow optimization.

Parallel Job Execution

Independent Jobs

jobs:
  test-unit:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Run unit tests
      run: npm run test:unit

  test-integration:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Run integration tests
      run: npm run test:integration

  lint:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Run linting
      run: npm run lint

  security-scan:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Run security scan
      run: npm audit

Parallel with Dependencies

jobs:
  setup:
    runs-on: ubuntu-latest
    steps:
    - name: Setup
      run: echo "Setup complete"

  test:
    runs-on: ubuntu-latest
    needs: setup
    steps:
    - name: Test
      run: echo "Testing"

  build:
    runs-on: ubuntu-latest
    needs: setup
    steps:
    - name: Build
      run: echo "Building"

  deploy:
    runs-on: ubuntu-latest
    needs: [test, build]
    steps:
    - name: Deploy
      run: echo "Deploying"

Serial Job Execution

Sequential Dependencies

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Build
      run: echo "Building"

  test:
    runs-on: ubuntu-latest
    needs: build
    steps:
    - name: Test
      run: echo "Testing"

  deploy-staging:
    runs-on: ubuntu-latest
    needs: test
    steps:
    - name: Deploy to staging
      run: echo "Deploying to staging"

  e2e-test:
    runs-on: ubuntu-latest
    needs: deploy-staging
    steps:
    - name: E2E tests
      run: echo "Running E2E tests"

  deploy-production:
    runs-on: ubuntu-latest
    needs: e2e-test
    steps:
    - name: Deploy to production
      run: echo "Deploying to production"

Hybrid Execution Patterns

Conditional Parallel Execution

jobs:
  ci:
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request'
    steps:
    - name: CI tests
      run: npm test

  cd:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    needs: ci
    steps:
    - name: CD deployment
      run: npm run deploy

Caching Strategies

Effective caching significantly improves workflow performance by reducing build times and resource usage.

Basic Caching

Node.js Dependencies

- name: Cache node modules
  uses: actions/cache@v3
  with:
    path: |
      ~/.npm
      node_modules
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

- name: Install dependencies
  run: npm ci

Python Dependencies

- name: Cache pip dependencies
  uses: actions/cache@v3
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

- name: Install dependencies
  run: pip install -r requirements.txt

Advanced Caching Patterns

Multi-Level Caching

- name: Cache dependencies
  uses: actions/cache@v3
  with:
    path: |
      ~/.npm
      node_modules
    key: ${{ runner.os }}-deps-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-deps-
      ${{ runner.os }}-

- name: Cache build artifacts
  uses: actions/cache@v3
  with:
    path: |
      dist/
      build/
    key: ${{ runner.os }}-build-${{ hashFiles('**/src/**') }}
    restore-keys: |
      ${{ runner.os }}-build-
      ${{ runner.os }}-

Conditional Caching

- name: Cache dependencies
  uses: actions/cache@v3
  if: github.event_name == 'pull_request'
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}

- name: Cache build artifacts
  uses: actions/cache@v3
  if: github.ref == 'refs/heads/main'
  with:
    path: dist/
    key: ${{ runner.os }}-build-${{ github.sha }}

Custom Caching Logic

Dynamic Cache Keys

- name: Generate cache key
  id: cache-key
  run: |
    if [ "${{ github.event_name }}" == "pull_request" ]; then
      CACHE_KEY="${{ runner.os }}-pr-${{ github.event.pull_request.number }}"
    else
      CACHE_KEY="${{ runner.os }}-${{ github.ref }}"
    fi
    echo "key=$CACHE_KEY" >> $GITHUB_OUTPUT

- name: Cache artifacts
  uses: actions/cache@v3
  with:
    path: dist/
    key: ${{ steps.cache-key.outputs.key }}-${{ hashFiles('**/src/**') }}

Cache Invalidation

- name: Invalidate cache
  run: |
    # Invalidate cache when dependencies change
    if [ -f "package-lock.json" ]; then
      echo "CACHE_VERSION=$(date +%s)" >> $GITHUB_ENV
    fi

- name: Cache with version
  uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}-${{ env.CACHE_VERSION }}

Performance Optimization

Workflow Optimization Techniques

Job Optimization

jobs:
  fast-checks:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Quick lint
      run: npm run lint:quick
    - name: Quick test
      run: npm run test:unit

  comprehensive-tests:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
    - uses: actions/checkout@v3
    - name: Full test suite
      run: npm run test
    - name: Integration tests
      run: npm run test:integration

Step Optimization

steps:
- name: Setup environment
  run: |
    # Combine multiple commands in single step
    npm ci --prefer-offline --no-audit
    npm run build
    npm run test:unit

- name: Parallel operations
  run: |
    # Run independent operations in parallel
    npm run lint &
    npm run type-check &
    npm run security-scan &
    wait

Resource Optimization

Runner Selection

jobs:
  lightweight-tests:
    runs-on: ubuntu-latest
    steps:
    - name: Quick tests
      run: npm run test:unit

  heavy-builds:
    runs-on: ubuntu-latest-4-cores  # Use larger runners for heavy operations
    steps:
    - name: Build and test
      run: npm run build:all

Timeout Configuration

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10  # Set job timeout
    steps:
    - name: Long-running test
      run: npm run test:integration
      timeout-minutes: 5  # Set step timeout

Build Optimization

Incremental Builds

- name: Cache build artifacts
  uses: actions/cache@v3
  with:
    path: |
      .next/cache
      node_modules/.cache
    key: ${{ runner.os }}-build-${{ hashFiles('**/src/**') }}

- name: Incremental build
  run: |
    # Only build changed files
    npm run build:incremental

Parallel Builds

strategy:
  matrix:
    package: [frontend, backend, shared]
    
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Build ${{ matrix.package }}
      run: npm run build:${{ matrix.package }}

Advanced Patterns

Workflow Composition

Reusable Workflows

# .github/workflows/ci.yml
name: CI
on:
  pull_request:
    branches: [ main ]

jobs:
  test:
    uses: ./.github/workflows/test.yml
    secrets: inherit

# .github/workflows/test.yml
name: Test
on:
  workflow_call:
    inputs:
      node-version:
        required: false
        type: string
        default: '18'

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Setup Node.js ${{ inputs.node-version }}
      uses: actions/setup-node@v3
      with:
        node-version: ${{ inputs.node-version }}
    - name: Run tests
      run: npm test

Workflow Dependencies

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      image-tag: ${{ steps.build.outputs.tag }}
    steps:
    - name: Build image
      id: build
      run: |
        TAG="app:${{ github.sha }}"
        docker build -t $TAG .
        echo "tag=$TAG" >> $GITHUB_OUTPUT

  test:
    needs: build
    runs-on: ubuntu-latest
    steps:
    - name: Test image
      run: docker run --rm ${{ needs.build.outputs.image-tag }} npm test

  deploy:
    needs: [build, test]
    runs-on: ubuntu-latest
    steps:
    - name: Deploy
      run: docker push ${{ needs.build.outputs.image-tag }}

Error Handling

Retry Logic

steps:
- name: Flaky test with retry
  uses: nick-invision/retry@v2
  with:
    timeout_minutes: 10
    max_attempts: 3
    command: npm run test:flaky

- name: Network operation with retry
  run: |
    for i in {1..3}; do
      npm run deploy && break
      echo "Attempt $i failed, retrying..."
      sleep 5
    done

Graceful Degradation

steps:
- name: Optional test
  run: npm run test:optional
  continue-on-error: true

- name: Check optional test result
  run: |
    if [ "${{ steps.optional-test.outcome }}" == "failure" ]; then
      echo "Optional test failed, but continuing..."
    fi

Key Takeaways

Advanced Workflow Principles

Matrix Builds: Use for comprehensive testing across multiple configurations
Conditional Execution: Implement smart logic based on context and events
Parallel Processing: Maximize efficiency with parallel job execution
Intelligent Caching: Reduce build times with strategic caching
Performance Optimization: Continuously optimize for speed and efficiency

Best Practices Summary

Use Matrix Builds: For testing across multiple environments and versions
Implement Conditions: Smart workflow behavior based on context
Optimize Caching: Strategic caching for dependencies and artifacts
Parallel Execution: Run independent jobs simultaneously
Monitor Performance: Track and optimize workflow execution times

Common Patterns

Fail-Fast vs Continue: Choose appropriate strategy for your use case
Dynamic Matrix: Generate matrix configurations based on conditions
Multi-Level Caching: Cache dependencies and build artifacts separately
Conditional Steps: Skip unnecessary steps based on context
Error Recovery: Implement retry logic and graceful degradation

Next Steps: Ready to create custom actions? Continue to Section 2.3: Custom Actions Development to learn how to build reusable workflow components.

Mastering these advanced patterns will help you create sophisticated, efficient, and maintainable GitHub Actions workflows. In the next section, we'll explore how to build custom actions for reusable automation.

Matrix Builds​

Basic Matrix Configuration​

Complex Matrix Strategies​

Multi-Dimensional Matrix​

Dynamic Matrix Generation​

Matrix Optimization Techniques​

Fail-Fast Strategy​

Maximum Parallel Jobs​

Conditional Execution​

Conditional Jobs​

Branch-Based Conditions​

Event-Based Conditions​

Complex Conditions​

Conditional Steps​

Step-Level Conditions​

Environment-Based Conditions​

Advanced Conditional Logic​

Multiple Conditions​

Function-Based Conditions​

Parallel vs Serial Execution​

Parallel Job Execution​

Independent Jobs​

Parallel with Dependencies​

Serial Job Execution​

Sequential Dependencies​

Hybrid Execution Patterns​

Conditional Parallel Execution​

Caching Strategies​

Basic Caching​

Node.js Dependencies​

Python Dependencies​

Advanced Caching Patterns​

Multi-Level Caching​

Conditional Caching​

Custom Caching Logic​

Dynamic Cache Keys​

Cache Invalidation​

Performance Optimization​

Workflow Optimization Techniques​

Job Optimization​

Step Optimization​

Resource Optimization​

Runner Selection​

Timeout Configuration​

Build Optimization​

Incremental Builds​

Parallel Builds​

Advanced Patterns​

Workflow Composition​

Reusable Workflows​

Workflow Dependencies​

Error Handling​

Retry Logic​

Graceful Degradation​

Key Takeaways​

Advanced Workflow Principles​

Best Practices Summary​

Common Patterns​