Integrating Terraform into CI/CD Pipelines - DevOps Preparation

Terraform CI/CD Jenkins GitHub Actions GitLab CI Azure DevOps Automation

Integrating Terraform into CI/CD Pipelines

Published on: November 17, 2023 | Author: DevOps Engineering Team

Learn to automate Terraform deployments with robust CI/CD pipelines. Implement security, testing, and promotion workflows across multiple platforms.

Pipeline Design Patterns

Choose the right pipeline architecture for your Terraform workflows.

Code Commit

→

Validation

→

Plan

→

Security Scan

→

Apply (Manual)

→

Verify

🚀 Trunk-based Development

Ideal for: Small teams, rapid iteration

main branch → Auto-plan
PR merge → Auto-apply (staging)
Manual promotion to production

🏢 GitFlow Strategy

Ideal for: Enterprise, multiple environments

feature/* → Develop → Plan
develop → Staging → Apply
main → Production → Apply

Pipeline Design Principles

✅ Separate plan and apply stages ✅ Manual approval for production ✅ Comprehensive testing ✅ Security scanning ✅ Rollback capabilities

CI Platform Implementations

Terraform integration examples for popular CI/CD platforms.

🐙 GitHub Actions

# .github/workflows/terraform.yml
name: 'Terraform'
on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - uses: hashicorp/setup-terraform@v2
        with:
          terraform_version: 1.5.0
      
      - name: Terraform Format
        run: terraform fmt -check
      
      - name: Terraform Init
        run: terraform init
      
      - name: Terraform Validate
        run: terraform validate
      
      - name: Terraform Plan
        run: terraform plan
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

🦊 GitLab CI

# .gitlab-ci.yml
stages:
  - validate
  - plan
  - apply

terraform:validate:
  stage: validate
  image: hashicorp/terraform:latest
  script:
    - terraform init
    - terraform validate
    - terraform fmt -check

terraform:plan:
  stage: plan
  image: hashicorp/terraform:latest
  script:
    - terraform init
    - terraform plan -out=planfile
  artifacts:
    paths:
      - planfile
  only:
    - merge_requests

terraform:apply:
  stage: apply
  image: hashicorp/terraform:latest
  script:
    - terraform init
    - terraform apply -auto-approve planfile
  when: manual
  only:
    - main

🔷 Azure DevOps

# azure-pipelines.yml
trigger:
  branches:
    include:
    - main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: TerraformInstaller@1
  inputs:
    terraformVersion: '1.5.0'

- task: TerraformTaskV4@4
  inputs:
    provider: 'azurerm'
    command: 'init'
    workingDirectory: '$(System.DefaultWorkingDirectory)'

- task: TerraformTaskV4@4
  inputs:
    provider: 'azurerm'
    command: 'validate'
    workingDirectory: '$(System.DefaultWorkingDirectory)'

- task: TerraformTaskV4@4
  inputs:
    provider: 'azurerm'
    command: 'plan'
    workingDirectory: '$(System.DefaultWorkingDirectory)'
    environmentServiceName: 'Azure-Service-Connection'

⚙️ Jenkins Pipeline

// Jenkinsfile
pipeline {
    agent any
    environment {
        AWS_ACCESS_KEY_ID = credentials('aws-access-key')
        AWS_SECRET_ACCESS_KEY = credentials('aws-secret-key')
    }
    stages {
        stage('Checkout') {
            steps { checkout scm }
        }
        stage('Terraform Init') {
            steps {
                sh 'terraform init'
            }
        }
        stage('Terraform Validate') {
            steps {
                sh 'terraform validate'
            }
        }
        stage('Terraform Plan') {
            steps {
                sh 'terraform plan -out=tfplan'
            }
        }
        stage('Terraform Apply') {
            when {
                branch 'main'
            }
            steps {
                input message: 'Apply Terraform?', ok: 'Apply'
                sh 'terraform apply tfplan'
            }
        }
    }
}

Security Best Practices

Secure your Terraform pipelines and infrastructure.

🔐 Secrets Management

# Never store secrets in code
# Use CI/CD secret stores:

# GitHub Actions:
env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY }}

# GitLab CI:
variables:
  AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY}

# Jenkins:
environment {
  AWS_ACCESS_KEY = credentials('aws-key')
}

# Azure DevOps:
- task: AzureKeyVault@1
  inputs:
    azureSubscription: '$(azureSubscription)'
    KeyVaultName: '$(keyVaultName)'

🛡️ Infrastructure Security

# Integrate security scanning
- name: TFSec Security Scan
  run: |
    docker run --rm -v "$(pwd):/src" \
    aquasec/tfsec /src

- name: Checkov Scan
  run: |
    pip install checkov
    checkov -d .

- name: Terraform Compliance
  run: |
    docker run --rm -v "$(pwd):/target" \
    eerkunt/terraform-compliance

Security Considerations

🔒 Use minimal privilege IAM roles 🔒 Enable state encryption 🔒 Scan for secrets in code 🔒 Implement branch protection 🔒 Require code reviews

Automated Testing

Implement comprehensive testing in your Terraform pipelines.

Syntax and Validation Testing

# Pre-commit hooks or pipeline steps
terraform validate
terraform fmt -check -recursive
tflint --enable-rule=aws_instance_invalid_type
tfsec --exclude-downloaded-modules

Unit Testing with Terratest

// terraform_test.go
package test

import (
    "testing"
    "github.com/gruntwork-io/terratest/modules/terraform"
)

func TestTerraform(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "../",
    }
    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)
    
    // Add assertions here
    instanceID := terraform.Output(t, terraformOptions, "instance_id")
    assert.NotEmpty(t, instanceID)
}

Integration Testing

# kitchen-terraform example
# .kitchen.yml
---
driver:
  name: terraform

provisioner:
  name: terraform

platforms:
  - name: aws

suites:
  - name: default
    verifier:
      name: terraform
      systems:
        - name: basic
          controls:
            - state_file
            - output_values

Environment Promotion

Manage infrastructure changes across multiple environments.

Environment Promotion Workflow

Select an environment to see promotion strategy:

Select an environment to see promotion strategy...

Environment	Automation Level	Approval Required	Testing Requirements
Development	Full Auto-apply	None	Basic validation
Staging	Auto-plan, Manual apply	Team Lead	Integration tests
Production	Manual plan and apply	Multiple approvers	Full test suite + security scan

Troubleshooting Common Issues

Solve frequent problems in Terraform CI/CD pipelines.

State Locking Issues

# In CI/CD, handle state locks gracefully
- name: Terraform Apply with retry
  run: |
    max_retries=3
    count=0
    until terraform apply -auto-approve; do
        count=$((count + 1))
        if [ $count -eq $max_retries ]; then
            echo "Max retries reached"
            exit 1
        fi
        echo "Retrying in 10 seconds..."
        sleep 10
    done

Timeout Handling

# Set appropriate timeouts
# GitHub Actions
timeout-minutes: 30

# GitLab CI
extends: .terraform_timeout

.terraform_timeout:
  timeout: 2h

# Jenkins
timeout(time: 30, unit: 'MINUTES') {
    sh 'terraform apply'
}

# Azure DevOps
timeoutInMinutes: 60

Pipeline Optimization Tips

Cache Terraform providers and plugins
Use parallel stages for independent modules
Implement pipeline templates for consistency
Set resource limits to prevent cost overruns
Use matrix builds for multiple Terraform versions

SKY Tech – Explore Technology!

Saturday, November 29, 2025