Terraform Modules Reusability Infrastructure as Code DevOps

How to Create and Use Terraform Modules for Reusable Code

Published on: November 9, 2023 | Author: DevOps Engineering Team

Terraform Modules: Build Once, Use Everywhere

Welcome to Part 6 of our Terraform Mastery Series! You've learned about variables and outputs. Now it's time to level up your infrastructure code with modules. Terraform modules let you package and reuse configurations across projects and teams, making your infrastructure code more maintainable, scalable, and consistent.

What You'll Learn

• What are Modules and Why Use Them?
• Understanding the Root Module
• Calling Public Modules from Terraform Registry
• Creating Your Own Local Modules
• Module Structure and Best Practices
• Passing Inputs to Modules
• Using Module Outputs
• Module Versioning with Git

What are Modules and Why Use Them?

Modules are containers for multiple Terraform resources that are used together. Think of them as functions in programming - they encapsulate logic and can be reused with different inputs.

Reusability

Write once, use everywhere. Create standardized infrastructure patterns that can be reused across projects and teams.

Abstraction

Hide complexity behind simple interfaces. Consumers don't need to understand the implementation details.

Organization

Break down complex infrastructure into logical, manageable components that are easier to understand and maintain.

Module Analogy

Think of modules like LEGO blocks. You can create standardized components (modules) and combine them in different ways to build complex structures (infrastructure) without reinventing the wheel each time.

Understanding the Root Module

Every Terraform configuration has at least one module - the root module. This is the directory where you run terraform apply.

my-terraform-project/
├── main.tf          # Primary resources
├── variables.tf     # Input variables
├── outputs.tf       # Output values
├── terraform.tfvars # Variable values
└── terraform.tfstate # State file (generated)

The root module can call other modules (child modules) to compose more complex infrastructure:

my-terraform-project/
├── main.tf
├── variables.tf
├── outputs.tf
└── modules/
    ├── networking/
    │   ├── main.tf
    │   ├── variables.tf
    │   └── outputs.tf
    └── compute/
        ├── main.tf
        ├── variables.tf
        └── outputs.tf

Calling Public Modules from Terraform Registry

The Terraform Registry hosts thousands of pre-built modules you can use in your configurations.

1

Find a Module

Browse the Terraform Registry for modules. For example, the AWS VPC module:

terraform-aws-modules/vpc/aws

2

Call the Module

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"
  version = "~> 3.0"
  
  name = "my-vpc"
  cidr = "10.0.0.0/16"
  
  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
  
  enable_nat_gateway = true
  enable_vpn_gateway = true
  
  tags = {
    Terraform = "true"
    Environment = "dev"
  }
}

3

Use Module Outputs

output "vpc_id" {
  value = module.vpc.vpc_id
}

output "private_subnets" {
  value = module.vpc.private_subnets
}

Benefits of Public Modules

• Community tested: Used and validated by thousands of users
• Best practices: Implement AWS and Terraform best practices
• Time saving: Avoid reinventing common patterns
• Versioned: Pin to specific versions for stability

Creating Your Own Local Modules

Creating your own modules helps standardize infrastructure patterns within your organization.

1

Create Module Directory Structure

my-project/
├── main.tf
├── variables.tf
├── outputs.tf
└── modules/
    └── web-server/
        ├── main.tf
        ├── variables.tf
        └── outputs.tf

2

Define Module Interface (Variables)

variable "name" {
  type        = string
  description = "Name prefix for resources"
}

variable "instance_type" {
  type        = string
  description = "EC2 instance type"
  default     = "t3.micro"
}

variable "instance_count" {
  type        = number
  description = "Number of instances to create"
  default     = 1
}

variable "vpc_id" {
  type        = string
  description = "VPC ID where instances will be created"
}

variable "subnet_ids" {
  type        = list(string)
  description = "List of subnet IDs for instances"
}

3

Implement Module Logic

resource "aws_security_group" "web" {
  name_prefix = "${var.name}-web-"
  vpc_id      = var.vpc_id
  
  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  
  tags = {
    Name = "${var.name}-web-sg"
  }
}

resource "aws_instance" "web" {
  count = var.instance_count
  
  ami           = "ami-0c02fb55956c7d316" # Amazon Linux 2
  instance_type = var.instance_type
  subnet_id     = var.subnet_ids[count.index % length(var.subnet_ids)]
  
  vpc_security_group_ids = [aws_security_group.web.id]
  
  user_data = file("${path.module}/user-data.sh")
  
  tags = {
    Name = "${var.name}-web-${count.index}"
  }
}

4

Define Module Outputs

output "instance_ids" {
  value = aws_instance.web.*.id
}

output "instance_private_ips" {
  value = aws_instance.web.*.private_ip
}

output "security_group_id" {
  value = aws_security_group.web.id
}

Module Structure and Best Practices

A well-structured module follows consistent patterns that make it easy to use and maintain.

main.tf

Purpose: Contains the primary resource definitions

Contains: AWS resources, data sources, providers

Best Practice: Keep it focused on the module's core purpose

variables.tf

Purpose: Defines the module's input interface

Contains: Input variable declarations with types and descriptions

Best Practice: Provide sensible defaults when possible

outputs.tf

Purpose: Defines what the module exposes to callers

Contains: Output values for important resources and computed values

Best Practice: Document outputs with descriptions

README.md

Purpose: Documentation for module users

Contains: Usage examples, input/output reference, requirements

Best Practice: Include complete usage examples

Module Design Principles

• Single Responsibility: Each module should do one thing well
• Clear Interface: Well-defined inputs and outputs
• Documentation: Clear usage instructions and examples
• Testing: Include examples and tests for different use cases

Passing Inputs to Modules

Modules accept inputs through variables, allowing customization for different use cases.

module "web_servers" {
  source = "./modules/web-server"
  
  name           = "production"
  instance_type = "t3.large"
  instance_count = 3
  vpc_id         = module.vpc.vpc_id
  subnet_ids     = module.vpc.private_subnets
}

You can use the same module with different inputs for different environments:

# Development environment
module "web_dev" {
  source = "./modules/web-server"
  
  name           = "dev"
  instance_type = "t3.micro"
  instance_count = 1
  vpc_id         = module.vpc.vpc_id
  subnet_ids     = module.vpc.private_subnets
}

# Production environment
module "web_prod" {
  source = "./modules/web-server"
  
  name           = "prod"
  instance_type = "t3.large"
  instance_count = 3
  vpc_id         = module.vpc.vpc_id
  subnet_ids     = module.vpc.private_subnets
}

Using Module Outputs

Module outputs allow you to pass information from child modules to the root module or between modules.

# Reference outputs from the web-server module
output "web_instance_ids" {
  value = module.web_servers.instance_ids
}

output "web_private_ips" {
  value = module.web_servers.instance_private_ips
}

# Pass outputs between modules
module "load_balancer" {
  source = "./modules/load-balancer"
  
  name               = "web"
  vpc_id             = module.vpc.vpc_id
  subnet_ids         = module.vpc.public_subnets
  target_instance_ids = module.web_servers.instance_ids
  security_group_id  = module.web_servers.security_group_id
}

Module Versioning with Git

Versioning your modules ensures stability and allows controlled updates.

1. Store Modules in Git Repositories

Create separate repositories for your modules:

# Module repository structure
terraform-aws-web-server/
├── main.tf
├── variables.tf
├── outputs.tf
├── README.md
└── versions.tf

2. Tag Releases with Semantic Versioning

git tag v1.0.0
git tag v1.1.0
git tag v2.0.0
git push --tags

3. Reference Versioned Modules

module "web_server" {
  source = "git::https://github.com/my-org/terraform-aws-web-server.git?ref=v1.2.0"
  
  name           = "production"
  instance_type = "t3.large"
  instance_count = 3
  vpc_id         = module.vpc.vpc_id
  subnet_ids     = module.vpc.private_subnets
}

Versioning Best Practices

• Use semantic versioning: MAJOR.MINOR.PATCH
• Tag releases: Create tags for stable versions
• Pin versions: Always specify version constraints
• Test upgrades: Test minor and patch versions before applying
• Maintain changelog: Document changes between versions

Complete Module Example

Here's a complete example of a reusable web server cluster module:

# modules/web-cluster/variables.tf
variable "name" { type = string }
variable "environment" { type = string }
variable "instance_type" { type = string; default = "t3.micro" }
variable "min_size" { type = number; default = 1 }
variable "max_size" { type = number; default = 3 }
variable "vpc_id" { type = string }
variable "subnet_ids" { type = list(string) }

# modules/web-cluster/main.tf
resource "aws_launch_template" "web" {
  name_prefix   = "${var.name}-${var.environment}-"
  image_id      = "ami-0c02fb55956c7d316"
  instance_type = var.instance_type
  vpc_security_group_ids = [aws_security_group.web.id]
  
  user_data = base64encode(file("${path.module}/user-data.sh"))
}

resource "aws_autoscaling_group" "web" {
  name               = "${var.name}-${var.environment}-asg"
  min_size           = var.min_size
  max_size           = var.max_size
  vpc_zone_identifier = var.subnet_ids
  
  launch_template {
    id      = aws_launch_template.web.id
    version = "$Latest"
  }
  
  tag {
    key                 = "Name"
    value               = "${var.name}-${var.environment}"
    propagate_at_launch = true
  }
}

# modules/web-cluster/outputs.tf
output "asg_name" {
  value = aws_autoscaling_group.web.name
}

output "security_group_id" {
  value = aws_security_group.web.id
}

Key Takeaways

• Modules promote reusability: Write once, use across projects
• Standardize infrastructure: Enforce best practices organization-wide
• Abstract complexity: Hide implementation details behind clean interfaces
• Version control modules: Use Git tags for stable releases
• Compose complex systems: Build complex infrastructure from simple modules
• Leverage public modules: Save time with community-tested patterns

In our next tutorial, we'll dive into Terraform Workspaces and Remote State, where you'll learn how to manage multiple environments and collaborate effectively with your team.

---

← Previous: Terraform Variables and Outputs

This is Part 6 of The Ultimate Terraform Mastery Series.

Next: Terraform Workspaces and Remote State →

A Guide to Terraform Variables, Outputs, and Best Practices

Terraform Variables Outputs Configuration DevOps

Mastering Terraform Variables and Outputs

Published on: November 2, 2023 | Author: DevOps Engineering Team

Dynamic Terraform Configurations

Welcome to Part 5 of our Terraform Mastery Series! So far, you've learned HCL syntax and state management. Now it's time to make your configurations dynamic and reusable. Variables and outputs are the key to creating flexible, maintainable Terraform code that works across different environments and use cases.

What You'll Learn

• Why Variables Matter
• Variable Types and Validation
Variable Input Methods
• Variable Precedence Rules
• Sensitive Variables
• Output Values Deep Dive
• Real-World Examples
• Variables and Outputs Best Practices

Why Variables Matter

Variables transform static configurations into dynamic, reusable templates. Here's what they enable:

Environment Flexibility

Use the same configuration for dev, staging, and production with different variable values.

Team Collaboration

Different team members can use the same code with their own settings.

Security

Keep sensitive values out of your codebase and inject them at runtime.

Variables vs. Hardcoded Values

Hardcoded (inflexible): instance_type = "t3.micro"

Variable (flexible): instance_type = var.instance_size

Variables make your code adaptable to different requirements without changing the core logic.

Variable Types and Validation

Terraform supports several variable types with built-in validation:

Basic Types

Complex Types

Validation

Sensitive Data

Basic Variable Types

variable "instance_type" {
  type        = string
  description = "EC2 instance type"
  default     = "t3.micro"
}

variable "instance_count" {
  type        = number
  description = "Number of instances to create"
  default     = 1
}

variable "enable_monitoring" {
  type        = bool
  description = "Enable detailed monitoring"
  default     = false
}

Complex Variable Types

variable "availability_zones" {
  type        = list(string)
  description = "List of availability zones"
  default     = ["us-east-1a", "us-east-1b"]
}

variable "tags" {
  type        = map(string)
  description = "Resource tags"
  default     = {
    Environment = "dev"
    Project     = "terraform-learning"
  }
}

variable "network_config" {
  type = object({
    vpc_cidr    = string
    subnet_count = number
    enable_nat   = bool
  })
  default = {
    vpc_cidr    = "10.0.0.0/16"
    subnet_count = 2
    enable_nat   = true
  }
}

Variable Validation

variable "environment" {
  type        = string
  description = "Deployment environment"
  
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be dev, staging, or prod."
  }
}

variable "instance_size" {
  type        = string
  description = "EC2 instance size"
  
  validation {
    condition     = can(regex("^[tcm][23a]?\\.[a-z]+$", var.instance_size))
    error_message = "Instance size must be a valid AWS instance type."
  }
}

Sensitive Variables

variable "database_password" {
  type        = string
  description = "Database administrator password"
  sensitive   = true
}

variable "api_keys" {
  type        = map(string)
  description = "API keys for external services"
  sensitive   = true
}

Sensitive Variable Behavior

Terraform will redact sensitive variable values from console output and logs, but they are still stored in state files in plain text. Always encrypt your state files!

Variable Input Methods

Terraform provides multiple ways to set variable values, each with different use cases:

1. terraform.tfvars Files

Recommended for most projects. Create environment-specific files:

instance_type = "t3.large"
instance_count = 3
environment = "prod"

tags = {
  Environment = "production"
  CostCenter  = "infrastructure"
}

Usage: terraform apply (automatically loaded)

2. Command Line (-var)

Quick overrides for individual variables:

terraform apply -var="instance_type=t3.large" \
                -var="instance_count=5"

Best for: Quick testing and debugging

3. Environment Variables (TF_VAR_)

Great for CI/CD pipelines and automation:

export TF_VAR_instance_type="t3.large"
export TF_VAR_database_password="secret123"
terraform apply

Best for: Automated deployments and secrets

4. Variable Definition Files (.auto.tfvars)

Automatically loaded files for different environments:

environment = "prod"
instance_count = 5
enable_monitoring = true

Usage: terraform apply (automatically loaded)

Variable Precedence Rules

When the same variable is set multiple ways, Terraform follows specific precedence rules:

1

Environment variables (TF_VAR_name)

Highest priority

2

terraform.tfvars files

3

*.auto.tfvars files

4

-var and -var-file flags

5

Variable defaults

Lowest priority

Precedence Example

If you have default = "t3.micro" but set TF_VAR_instance_type=t3.large, Terraform will use t3.large because environment variables have highest precedence.

Working with Sensitive Variables

Sensitive variables require special handling to protect your secrets:

1

Mark Variables as Sensitive

variable "db_password" {
  type        = string
  sensitive   = true
  description = "Database password"
}

2

Use Environment Variables for Secrets

export TF_VAR_db_password="my-secret-password"
terraform apply

This keeps secrets out of your code and version control.

3

Never Commit Secrets

Add sensitive files to .gitignore:

# Terraform sensitive files
*.tfvars
*.tfstate
*.tfstate.backup
.terraform/

Output Values Deep Dive

Outputs expose information from your infrastructure for other configurations or users:

Basic Outputs

output "instance_ip" {
  value       = aws_instance.web.public_ip
  description = "Public IP of the web server"
}

output "load_balancer_dns" {
  value       = aws_lb.web.dns_name
  description = "DNS name of the load balancer"
}

Complex Outputs

output "all_instance_ips" {
  value       = aws_instance.web.*.public_ip
  description = "List of all instance IPs"
}

output "security_group_rules" {
  value = {
    for sg in aws_security_group.web :
    sg.name => sg.ingress[*].from_port
  }
  description = "Security group ingress rules"
}

Sensitive Outputs

output "database_password" {
  value       = aws_db_instance.main.password
  sensitive   = true
  description = "Database administrator password"
}

Output Security Note

Even with sensitive = true, outputs are stored in plain text in state files. Always encrypt your state backend!

Real-World Examples

Let's see variables and outputs in action with a complete example:

variable "environment" {
  type        = string
  description = "Deployment environment"
  
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be dev, staging, or prod."
  }
}

variable "instance_config" {
  type = map(object({
    instance_type = string
    instance_count = number
  }))
  default = {
    dev = {
      instance_type  = "t3.micro"
      instance_count = 1
    }
    staging = {
      instance_type  = "t3.medium"
      instance_count = 2
    }
    prod = {
      instance_type  = "t3.large"
      instance_count = 3
    }
  }
}

resource "aws_instance" "web" {
  count = var.instance_config[var.environment].instance_count
  
  ami           = "ami-0c02fb55956c7d316"
  instance_type = var.instance_config[var.environment].instance_type
  
  tags = {
    Name        = "web-server-${count.index}"
    Environment = var.environment
  }
}

output "web_instance_ips" {
  value = aws_instance.web.*.public_ip
}

output "environment_summary" {
  value = "Deployed ${var.instance_config[var.environment].instance_count} instances of type ${var.instance_config[var.environment].instance_type} in ${var.environment} environment"
}

Variables and Outputs Best Practices

Follow these guidelines for maintainable and secure configurations:

Use Descriptive Names

Choose clear, meaningful names for variables and outputs:

# Good
variable "database_instance_type" {}

# Avoid
variable "db_type" {}

Provide Defaults When Possible

Default values make configurations easier to use:

variable "instance_type" {
  type    = string
  default = "t3.micro"
}

Add Descriptions

Document the purpose and expected values:

variable "environment" {
  description = "Deployment environment (dev, staging, prod)"
  type        = string
}

Use Validation

Catch errors early with input validation:

variable "port" {
  type        = number
  validation {
    condition = var.port > 0 && var.port < 65536
    error_message = "Port must be between 1 and 65535"
  }
}

Key Takeaways

• Variables make configurations reusable across environments
• Use appropriate types and validation for safety
• Follow precedence rules for variable assignment
• Mark sensitive data to protect secrets
• Outputs expose important information for other systems
• Never commit secrets to version control

In our next tutorial, we'll explore Terraform Modules, where you'll learn how to create reusable, composable infrastructure components that can be shared across your organization.

---

← Previous: Managing Terraform State

This is Part 5 of The Ultimate Terraform Mastery Series.

Next: Terraform Modules Deep Dive →