Shell Scripting: The DevOps Automation Powerhouse
Master Bash scripting to automate tasks, manage systems, and become a more efficient DevOps engineer.
📅 Published: Feb 2026
⏱️ Estimated Reading Time: 22 minutes
🏷️ Tags: Bash Scripting, Shell Scripting, Automation, DevOps, Cron, Scripting
🐚 Introduction to Bash Scripting: Your Automation Superpower
What is Shell Scripting?
Shell scripting is like writing recipes for your computer. Instead of manually typing commands one by one, you write them in a file, and the computer executes them automatically. It's the secret weapon of every efficient system administrator and DevOps engineer.
Think of it this way:
Manual commands = Cooking by following a recipe step-by-step each time
Shell script = Having a robot chef that knows the entire recipe and can cook it perfectly every time
Why Learn Shell Scripting?
Automation - Make repetitive tasks happen automatically
Consistency - Same commands run the same way every time
Time-saving - Write once, run infinitely
Documentation - Scripts document how things are done
Error reduction - Fewer manual typing errors
Your First Shell Script
Let's create the classic "Hello World" script:
#!/bin/bash # This is a comment - it's not executed # File: hello.sh echo "Hello, World!" echo "Today is: $(date)" echo "You are: $(whoami)"
Breaking it down:
#!/bin/bash- This is called a shebang. It tells the system to use Bash to run this script.#- Anything after # is a comment (ignored by the computer)echo- Prints text to the screen$(command)- Runs the command inside and uses its output
Making and Running Scripts
# 1. Create the script file nano hello.sh # 2. Add the content above # 3. Make it executable chmod +x hello.sh # 4. Run it ./hello.sh # Output: # Hello, World! # Today is: Mon Feb 10 10:30:00 UTC 2026 # You are: yourusername # Alternative ways to run: bash hello.sh sh hello.sh source hello.sh # Runs in current shell (for functions/variables)
Important: The ./ before the filename tells Linux "look in the current directory for this program." Without it, Linux looks in system directories like /bin/ and /usr/bin/.
📦 Variables, Loops, and Conditionals: The Building Blocks
Variables: Storing Information
Variables are like labeled boxes where you store information to use later.
#!/bin/bash # File: variables.sh # Define variables NAME="Alice" AGE=30 SERVER="web01.example.com" PATH="/home/user/documents" # Use variables echo "Hello, $NAME" echo "You are $AGE years old" # Quotes matter! echo "Server: $SERVER" # Works echo 'Server: $SERVER' # Doesn't expand variable echo "Path: $PATH" # Uses system PATH variable (be careful!) echo "Path: ${PATH}" # Same as above echo "My path: $MY_PATH" # Better: Use different variable names # Command output in variables CURRENT_DATE=$(date) UPTIME=$(uptime -p) USER_COUNT=$(who | wc -l) echo "Date: $CURRENT_DATE" echo "System uptime: $UPTIME" echo "Users logged in: $USER_COUNT" # Arithmetic operations COUNT=10 COUNT=$((COUNT + 1)) echo "Count: $COUNT" # Increment shortcut ((COUNT++)) echo "Count after increment: $COUNT"
User Input: Making Scripts Interactive
#!/bin/bash # File: input.sh # Ask for user input read -p "What is your name? " USERNAME read -sp "Enter password: " PASSWORD # -s = silent (no echo) echo read -p "Enter age: " AGE echo "Hello $USERNAME, you are $AGE years old" echo "Password length: ${#PASSWORD} characters" # ${#VAR} = length # Read multiple values read -p "Enter three numbers: " NUM1 NUM2 NUM3 echo "You entered: $NUM1, $NUM2, $NUM3" # Read entire line read -p "Enter a sentence: " SENTENCE echo "You said: $SENTENCE"
Conditionals: Making Decisions
Conditionals let your script make decisions based on conditions.
#!/bin/bash # File: conditions.sh # Simple if statement read -p "Enter a number: " NUMBER if [ $NUMBER -gt 10 ]; then echo "$NUMBER is greater than 10" fi # If-else if [ $NUMBER -gt 10 ]; then echo "$NUMBER is greater than 10" else echo "$NUMBER is 10 or less" fi # If-elif-else if [ $NUMBER -gt 100 ]; then echo "$NUMBER is greater than 100" elif [ $NUMBER -gt 50 ]; then echo "$NUMBER is between 51 and 100" elif [ $NUMBER -gt 10 ]; then echo "$NUMBER is between 11 and 50" else echo "$NUMBER is 10 or less" fi # File tests FILE="/etc/passwd" if [ -f "$FILE" ]; then echo "$FILE exists and is a regular file" fi if [ -d "/etc" ]; then echo "/etc is a directory" fi if [ -r "$FILE" ]; then echo "You can read $FILE" fi if [ -w "$FILE" ]; then echo "You can write to $FILE" fi if [ -x "/bin/bash" ]; then echo "/bin/bash is executable" fi
Common Test Operators
# Numeric comparisons [ $A -eq $B ] # Equal [ $A -ne $B ] # Not equal [ $A -lt $B ] # Less than [ $A -le $B ] # Less than or equal [ $A -gt $B ] # Greater than [ $A -ge $B ] # Greater than or equal # String comparisons [ "$STR1" = "$STR2" ] # Equal [ "$STR1" != "$STR2" ] # Not equal [ -z "$STR" ] # String is empty [ -n "$STR" ] # String is not empty # File tests [ -f "file" ] # Is a regular file [ -d "dir" ] # Is a directory [ -e "file" ] # Exists [ -r "file" ] # Readable [ -w "file" ] # Writable [ -x "file" ] # Executable [ -s "file" ] # Size > 0
Loops: Repeating Actions
Loops let you repeat actions multiple times.
#!/bin/bash # File: loops.sh # For loop - through a list echo "Counting to 5:" for i in 1 2 3 4 5; do echo "Number: $i" done # For loop - through a range echo "Counting 1 to 10:" for i in {1..10}; do echo "Number: $i" done # For loop - with step echo "Even numbers 0 to 10:" for i in {0..10..2}; do echo "Number: $i" done # For loop - through files echo "Files in current directory:" for file in *; do echo "Found: $file" done # While loop - until condition is false echo "Counting down from 5:" COUNT=5 while [ $COUNT -gt 0 ]; do echo "Count: $COUNT" ((COUNT--)) done # Until loop - until condition is true echo "Waiting for file to appear..." until [ -f "/tmp/ready.flag" ]; do echo "File not found, waiting..." sleep 2 done echo "File found!" # Infinite loop (with break) echo "Press Ctrl+C to stop this loop" while true; do echo "Looping..." sleep 1 done
Real Example: Processing Files
#!/bin/bash # File: process-logs.sh LOG_DIR="/var/log" BACKUP_DIR="/backup/logs" # Create backup directory if it doesn't exist mkdir -p "$BACKUP_DIR" # Process each .log file for logfile in "$LOG_DIR"/*.log; do if [ -f "$logfile" ]; then # Get just the filename (without path) filename=$(basename "$logfile") echo "Processing: $filename" # Count lines line_count=$(wc -l < "$logfile") # Backup if large if [ $line_count -gt 1000 ]; then echo " Backing up (lines: $line_count)" cp "$logfile" "$BACKUP_DIR/$filename.backup" fi # Check for errors error_count=$(grep -c -i "error\|fail" "$logfile") if [ $error_count -gt 0 ]; then echo " WARNING: Found $error_count errors" fi fi done echo "Processing complete!"
🔧 Functions & Arguments: Creating Reusable Code
Functions: Your Code Building Blocks
Functions are like mini-scripts within your script. They let you reuse code without repeating yourself.
#!/bin/bash # File: functions.sh # Define a simple function say_hello() { echo "Hello from a function!" echo "The time is: $(date)" } # Call the function say_hello # Function with parameters greet_person() { local name=$1 # local makes it only available in function local time_of_day=$2 echo "Good $time_of_day, $name!" } # Call with arguments greet_person "Alice" "morning" greet_person "Bob" "evening" # Function that returns a value (exit code) is_file_exists() { local filename=$1 if [ -f "$filename" ]; then return 0 # Success else return 1 # Failure fi } # Use the function if is_file_exists "/etc/passwd"; then echo "File exists" else echo "File does not exist" fi # Function that "returns" output get_file_size() { local filename=$1 if [ -f "$filename" ]; then stat -c%s "$filename" else echo "0" fi } # Capture function output SIZE=$(get_file_size "/etc/passwd") echo "File size: $SIZE bytes"
Script Arguments: Making Scripts Flexible
Script arguments let users pass information to your script when they run it.
#!/bin/bash # File: arguments.sh # Special variables for arguments echo "Script name: $0" echo "First argument: $1" echo "Second argument: $2" echo "Third argument: $3" # All arguments echo "All arguments: $@" echo "Number of arguments: $#" # Loop through all arguments echo "Processing arguments:" for arg in "$@"; do echo " Argument: $arg" done # Shift arguments (remove first one) echo "Before shift: $@" shift echo "After shift: $@" # Practical example: Process files process_files() { for file in "$@"; do if [ -f "$file" ]; then echo "Processing: $file" # Do something with the file else echo "Error: $file not found" >&2 fi done } # Call with arguments process_files "$@"
Option Parsing: Professional Scripts
For more complex scripts, use getopts:
#!/bin/bash # File: options.sh # Default values VERBOSE=false OUTPUT_FILE="" INPUT_FILES=() # Parse options while getopts ":vf:o:" opt; do case $opt in v) VERBOSE=true ;; f) INPUT_FILES+=("$OPTARG") ;; o) OUTPUT_FILE="$OPTARG" ;; \?) echo "Invalid option: -$OPTARG" >&2 exit 1 ;; :) echo "Option -$OPTARG requires an argument." >&2 exit 1 ;; esac done # Shift processed options shift $((OPTIND-1)) # Remaining arguments OTHER_ARGS=("$@") # Show what we got if $VERBOSE; then echo "Verbose mode enabled" fi if [ -n "$OUTPUT_FILE" ]; then echo "Output file: $OUTPUT_FILE" fi echo "Input files: ${INPUT_FILES[@]}" echo "Other arguments: ${OTHER_ARGS[@]}" # Usage: ./options.sh -v -f file1.txt -f file2.txt -o output.txt arg1 arg2
🚨 Error Handling: Making Scripts Robust
Exit Codes: Success or Failure?
Every command returns an exit code (0 = success, non-zero = failure).
#!/bin/bash # File: exit-codes.sh # Check exit code of last command ls /tmp echo "Exit code: $?" ls /nonexistent 2>/dev/null echo "Exit code: $?" # Non-zero means error # Use exit codes in conditions if ls /tmp >/dev/null 2>&1; then echo "/tmp exists" else echo "/tmp doesn't exist" fi # Set your script's exit code if [ ! -f "/etc/passwd" ]; then echo "Critical file missing!" >&2 exit 1 # Exit with error fi echo "Everything is fine" exit 0 # Exit with success
Trapping Errors and Signals
#!/bin/bash # File: error-handling.sh # Trap errors set -e # Exit immediately if any command fails # set -u # Exit if using undefined variable # set -o pipefail # Fail if any command in pipeline fails # Trap specific signals cleanup() { echo "Cleaning up..." rm -f /tmp/tempfile.$$ echo "Cleanup complete" } trap cleanup EXIT # Run cleanup on exit trap 'echo "Interrupted!"; exit 1' INT TERM # Handle Ctrl+C # Example with error handling backup_file() { local source=$1 local destination=$2 if [ ! -f "$source" ]; then echo "Error: Source file not found: $source" >&2 return 1 fi if ! cp "$source" "$destination"; then echo "Error: Failed to copy $source to $destination" >&2 return 1 fi echo "Backup successful: $source -> $destination" return 0 } # Try the backup if backup_file "/etc/passwd" "/tmp/backup.passwd"; then echo "Backup succeeded" else echo "Backup failed" >&2 exit 1 fi # Create temp file echo "Creating temp file..." echo "temp data" > /tmp/tempfile.$$ # This will trigger the cleanup on exit
Logging: Keeping Records
#!/bin/bash # File: logging.sh LOG_FILE="/var/log/myscript.log" DEBUG=false # Logging function log() { local level=$1 local message=$2 local timestamp=$(date '+%Y-%m-%d %H:%M:%S') echo "[$timestamp] [$level] $message" >> "$LOG_FILE" # Also print to console if debug mode if [ "$level" = "ERROR" ] || [ "$DEBUG" = true ]; then echo "[$level] $message" fi } # Usage log "INFO" "Script started" log "DEBUG" "Debug information" log "WARNING" "Something might be wrong" log "ERROR" "Something went wrong!" # Example with error checking check_disk_space() { local usage=$(df / | awk 'NR==2{print $5}' | sed 's/%//') if [ $usage -gt 90 ]; then log "ERROR" "Disk usage critical: ${usage}%" return 1 elif [ $usage -gt 80 ]; then log "WARNING" "Disk usage high: ${usage}%" else log "INFO" "Disk usage normal: ${usage}%" fi return 0 } check_disk_space
⚙️ Automating System Tasks
System Information Script
#!/bin/bash # File: system-info.sh # Colors for output RED='\033[0;31m' GREEN='\033[0;32m' YELLOW='\033[1;33m' NC='\033[0m' # No Color echo -e "${GREEN}=== System Information ===${NC}" echo # Host information echo -e "${YELLOW}Host Information:${NC}" echo "Hostname: $(hostname)" echo "Kernel: $(uname -r)" echo "Uptime: $(uptime -p | sed 's/up //')" # CPU information echo -e "\n${YELLOW}CPU Information:${NC}" echo "CPU Model: $(grep "model name" /proc/cpuinfo | head -1 | cut -d: -f2 | xargs)" echo "CPU Cores: $(nproc)" echo "Load Average: $(uptime | awk -F'load average:' '{print $2}' | xargs)" # Memory information echo -e "\n${YELLOW}Memory Information:${NC}" TOTAL_MEM=$(free -h | awk '/^Mem:/ {print $2}') USED_MEM=$(free -h | awk '/^Mem:/ {print $3}') PERCENT=$(free | awk '/^Mem:/ {printf("%.0f"), $3/$2*100}') echo "Total: $TOTAL_MEM" echo "Used: $USED_MEM" echo "Usage: $PERCENT%" # Color code memory usage if [ $PERCENT -gt 90 ]; then echo -e "Status: ${RED}CRITICAL${NC}" elif [ $PERCENT -gt 70 ]; then echo -e "Status: ${YELLOW}WARNING${NC}" else echo -e "Status: ${GREEN}OK${NC}" fi # Disk information echo -e "\n${YELLOW}Disk Information:${NC}" df -h / | awk 'NR==2 {printf("Mount: %s\nTotal: %s\nUsed: %s\nFree: %s\nUsage: %s\n", $6, $2, $3, $4, $5)}' # Network information echo -e "\n${YELLOW}Network Information:${NC}" IP_ADDR=$(hostname -I | awk '{print $1}') PUBLIC_IP=$(curl -s ifconfig.me 2>/dev/null || echo "Not available") echo "Local IP: $IP_ADDR" echo "Public IP: $PUBLIC_IP" # Services status echo -e "\n${YELLOW}Services Status:${NC}" for service in ssh nginx mysql docker; do if systemctl is-active --quiet $service 2>/dev/null; then echo -e "$service: ${GREEN}Running${NC}" else echo -e "$service: ${RED}Stopped${NC}" fi done echo -e "\n${GREEN}Report generated at: $(date)${NC}"
User Management Automation
#!/bin/bash # File: user-manager.sh # Configuration USER_LIST_FILE="users.txt" DEFAULT_SHELL="/bin/bash" DEFAULT_GROUP="developers" # Check if running as root if [ "$(id -u)" -ne 0 ]; then echo "This script must be run as root" >&2 exit 1 fi # Function to create a user create_user() { local username=$1 local fullname=$2 # Check if user exists if id "$username" &>/dev/null; then echo "User $username already exists, skipping..." return 1 fi # Create user with home directory useradd -m -s "$DEFAULT_SHELL" -c "$fullname" "$username" if [ $? -eq 0 ]; then echo "User $username created successfully" # Set random password password=$(openssl rand -base64 12) echo "$username:$password" | chpasswd # Force password change on first login chage -d 0 "$username" # Add to default group usermod -aG "$DEFAULT_GROUP" "$username" # Create .ssh directory mkdir -p "/home/$username/.ssh" chmod 700 "/home/$username/.ssh" chown "$username:$username" "/home/$username/.ssh" # Save credentials (in real script, send via secure method) echo "Username: $username" >> /tmp/user_credentials.txt echo "Password: $password" >> /tmp/user_credentials.txt echo "---" >> /tmp/user_credentials.txt return 0 else echo "Failed to create user $username" >&2 return 1 fi } # Main script echo "Starting user creation..." # Check if user list file exists if [ ! -f "$USER_LIST_FILE" ]; then echo "Error: $USER_LIST_FILE not found" >&2 echo "Create a file with format: username:Full Name" exit 1 fi # Read file and create users while IFS=':' read -r username fullname; do # Skip empty lines and comments [[ -z "$username" || "$username" =~ ^# ]] && continue echo "Creating user: $username ($fullname)" if create_user "$username" "$fullname"; then echo "✓ Success" else echo "✗ Failed" fi echo done < "$USER_LIST_FILE" echo "User creation complete!" echo "Credentials saved to: /tmp/user_credentials.txt" echo "Please distribute credentials securely and delete the file."
⏰ Script Scheduling with Cron
Understanding Cron
Cron is Linux's task scheduler. It runs commands at specific times automatically. Think of it as an alarm clock for your scripts.
Cron Syntax
* * * * * command_to_execute │ │ │ │ │ │ │ │ │ └── Day of week (0-7, 0=Sunday) │ │ │ └──── Month (1-12) │ │ └────── Day of month (1-31) │ └──────── Hour (0-23) └────────── Minute (0-59)
Managing Cron Jobs
# Edit your user's cron jobs crontab -e # List your cron jobs crontab -l # Remove all your cron jobs crontab -r # Edit another user's cron jobs (as root) sudo crontab -u username -e # System-wide cron directories ls /etc/cron.hourly/ ls /etc/cron.daily/ ls /etc/cron.weekly/ ls /etc/cron.monthly/
Common Cron Examples
# In crontab -e: # Run every minute * * * * * /path/to/script.sh # Run at 2:30 AM every day 30 2 * * * /backup/daily-backup.sh # Run every Monday at 4 PM 0 16 * * 1 /reports/weekly-report.sh # Run every 5 minutes */5 * * * * /monitoring/check-status.sh # Run at 10:15 on the 1st of each month 15 10 1 * * /billing/monthly-invoice.sh # Run every hour from 9 AM to 5 PM, Monday-Friday 0 9-17 * * 1-5 /monitoring/hourly-check.sh # Run at reboot @reboot /scripts/startup.sh # Special strings @yearly /scripts/yearly-maintenance.sh # Same as 0 0 1 1 * @monthly /scripts/monthly-backup.sh # Same as 0 0 1 * * @weekly /scripts/weekly-cleanup.sh # Same as 0 0 * * 0 @daily /scripts/daily-report.sh # Same as 0 0 * * * @hourly /scripts/hourly-monitor.sh # Same as 0 * * * *
Cron Best Practices
# 1. Always use full paths # BAD: script.sh # GOOD: /home/user/scripts/script.sh # 2. Redirect output to logs * * * * * /script.sh >> /var/log/script.log 2>&1 # 3. Set proper environment * * * * * cd /path && ./script.sh # 4. Use locking to prevent overlapping runs * * * * * /usr/bin/flock -n /tmp/script.lock /script.sh # 5. Test commands manually first # Example with all best practices: */5 * * * * cd /opt/app && /usr/bin/flock -n /tmp/backup.lock /opt/app/backup.sh >> /var/log/backup.log 2>&1
🎯 Real-world Examples
Example 1: Automated Backup Script
#!/bin/bash # File: backup-system.sh # Automated backup script with rotation and notifications # Configuration BACKUP_DIR="/backup" SOURCE_DIRS="/home /etc /var/www" DB_NAME="mydatabase" DB_USER="root" DB_PASS="password" RETENTION_DAYS=7 LOG_FILE="/var/log/backup.log" EMAIL="admin@example.com" # Log function log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE" } # Error handling set -e trap 'log "ERROR: Backup failed!"; exit 1' ERR # Check if backup directory exists mkdir -p "$BACKUP_DIR" # Start backup log "Starting backup..." # Create timestamp TIMESTAMP=$(date '+%Y%m%d_%H%M%S') BACKUP_PATH="$BACKUP_DIR/backup_$TIMESTAMP" # Create backup directory mkdir -p "$BACKUP_PATH" # Backup files log "Backing up files..." for dir in $SOURCE_DIRS; do if [ -d "$dir" ]; then log " Backing up $dir" tar -czf "$BACKUP_PATH/$(basename $dir).tar.gz" "$dir" 2>/dev/null || log " Warning: Failed to backup $dir" fi done # Backup database log "Backing up database..." mysqldump -u"$DB_USER" -p"$DB_PASS" "$DB_NAME" > "$BACKUP_PATH/database.sql" 2>/dev/null || log " Warning: Database backup failed" # Create backup manifest log "Creating manifest..." ls -lh "$BACKUP_PATH" > "$BACKUP_PATH/manifest.txt" du -sh "$BACKUP_PATH" >> "$BACKUP_PATH/manifest.txt" # Compress backup log "Compressing backup..." tar -czf "$BACKUP_PATH.tar.gz" -C "$BACKUP_DIR" "backup_$TIMESTAMP" rm -rf "$BACKUP_PATH" # Clean old backups log "Cleaning old backups..." find "$BACKUP_DIR" -name "backup_*.tar.gz" -mtime +$RETENTION_DAYS -delete # Calculate backup size BACKUP_SIZE=$(du -h "$BACKUP_PATH.tar.gz" | cut -f1) # Send notification log "Backup completed successfully!" log "Backup size: $BACKUP_SIZE" log "Location: $BACKUP_PATH.tar.gz" # Send email (if mail command is configured) echo "Backup completed at $(date)" | mail -s "Backup Successful" "$EMAIL" exit 0
Example 2: System Monitoring Script
#!/bin/bash # File: system-monitor.sh # Monitors system resources and sends alerts # Thresholds CPU_THRESHOLD=80 MEM_THRESHOLD=85 DISK_THRESHOLD=90 LOAD_THRESHOLD=$(nproc) # Number of CPU cores # Alert file (prevents duplicate alerts) ALERT_FILE="/tmp/monitor.alerts" LOG_FILE="/var/log/system-monitor.log" # Initialize alert file touch "$ALERT_FILE" # Log function log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE" } # Check CPU usage check_cpu() { local cpu_usage=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d'%' -f1) local load=$(uptime | awk -F'load average:' '{print $2}' | awk '{print $1}' | tr -d ',') log "CPU Usage: ${cpu_usage}%, Load: $load" if (( $(echo "$cpu_usage > $CPU_THRESHOLD" | bc -l) )); then send_alert "CPU" "CPU usage is ${cpu_usage}% (threshold: ${CPU_THRESHOLD}%)" fi if (( $(echo "$load > $LOAD_THRESHOLD" | bc -l) )); then send_alert "LOAD" "Load average is $load (threshold: ${LOAD_THRESHOLD})" fi } # Check memory usage check_memory() { local mem_usage=$(free | awk '/Mem/{printf("%.0f"), $3/$2*100}') local swap_usage=$(free | awk '/Swap/{printf("%.0f"), $3/$2*100}') log "Memory Usage: ${mem_usage}%, Swap: ${swap_usage}%" if [ $mem_usage -gt $MEM_THRESHOLD ]; then send_alert "MEMORY" "Memory usage is ${mem_usage}% (threshold: ${MEM_THRESHOLD}%)" fi if [ $swap_usage -gt 50 ]; then send_alert "SWAP" "High swap usage: ${swap_usage}%" fi } # Check disk usage check_disk() { local disk_usage=$(df / | awk 'NR==2{print $5}' | sed 's/%//') local inode_usage=$(df -i / | awk 'NR==2{print $5}' | sed 's/%//') log "Disk Usage: ${disk_usage}%, Inodes: ${inode_usage}%" if [ $disk_usage -gt $DISK_THRESHOLD ]; then send_alert "DISK" "Disk usage is ${disk_usage}% (threshold: ${DISK_THRESHOLD}%)" fi if [ $inode_usage -gt 90 ]; then send_alert "INODE" "Inode usage is ${inode_usage}%" fi } # Check services check_services() { local services="ssh nginx mysql docker" for service in $services; do if systemctl is-active --quiet "$service"; then log "Service $service: RUNNING" else send_alert "SERVICE" "Service $service is DOWN" fi done } # Send alert (prevents duplicates) send_alert() { local type=$1 local message=$2 local alert_key="${type}_${message}" # Check if already alerted if ! grep -q "$alert_key" "$ALERT_FILE"; then log "ALERT: $message" echo "$alert_key" >> "$ALERT_FILE" # Send email alert echo "Alert: $message at $(date)" | mail -s "System Alert: $type" "admin@example.com" # Could also send to Slack/Teams/etc. # curl -X POST -H 'Content-type: application/json' \ # --data "{\"text\":\"$message\"}" \ # https://hooks.slack.com/services/... fi } # Clear old alerts (older than 1 hour) clear_old_alerts() { if [ -f "$ALERT_FILE" ]; then # In real implementation, you'd track timestamps # For simplicity, we clear if file is old if [ $(find "$ALERT_FILE" -mmin +60) ]; then > "$ALERT_FILE" fi fi } # Main monitoring function main() { log "=== Starting system monitor ===" clear_old_alerts check_cpu check_memory check_disk check_services log "=== Monitor completed ===" echo } # Run main function main
Example 3: Log Cleanup Automation
#!/bin/bash # File: log-cleanup.sh # Automatically cleans up old logs and archives important ones # Configuration LOG_DIRS="/var/log /var/log/nginx /var/log/mysql /opt/app/logs" ARCHIVE_DIR="/backup/logs" RETENTION_DAYS=30 COMPRESS_DAYS=7 MAX_LOG_SIZE="100M" LOG_FILE="/var/log/log-cleanup.log" # Colors for output GREEN='\033[0;32m' YELLOW='\033[1;33m' RED='\033[0;31m' NC='\033[0m' # No Color # Log function log() { local level=$1 local message=$2 local color=$NC case $level in INFO) color=$GREEN ;; WARN) color=$YELLOW ;; ERROR) color=$RED ;; esac echo -e "[$(date '+%Y-%m-%d %H:%M:%S')] ${color}[$level]${NC} $message" | tee -a "$LOG_FILE" } # Create archive directory mkdir -p "$ARCHIVE_DIR" # Main cleanup function cleanup_logs() { log "INFO" "Starting log cleanup..." total_saved=0 for log_dir in $LOG_DIRS; do if [ ! -d "$log_dir" ]; then log "WARN" "Directory not found: $log_dir" continue fi log "INFO" "Processing: $log_dir" # Archive logs older than COMPRESS_DAYS find "$log_dir" -name "*.log" -type f -mtime +$COMPRESS_DAYS 2>/dev/null | while read logfile; do if [ -s "$logfile" ]; then filename=$(basename "$logfile") archive_name="${filename}.$(date '+%Y%m%d').tar.gz" log "INFO" " Archiving: $filename" # Archive the log tar -czf "$ARCHIVE_DIR/$archive_name" -C "$(dirname "$logfile")" "$filename" # Clear the original log (keep last 1000 lines) tail -1000 "$logfile" > "${logfile}.tmp" mv "${logfile}.tmp" "$logfile" saved=$(stat -c%s "$ARCHIVE_DIR/$archive_name") total_saved=$((total_saved + saved)) fi done # Delete archived logs older than RETENTION_DAYS deleted_count=$(find "$ARCHIVE_DIR" -name "*.tar.gz" -type f -mtime +$RETENTION_DAYS -delete -print | wc -l) if [ $deleted_count -gt 0 ]; then log "INFO" " Deleted $deleted_count old archives" fi # Truncate large log files find "$log_dir" -name "*.log" -type f -size +$MAX_LOG_SIZE 2>/dev/null | while read logfile; do log "WARN" " Truncating large log: $(basename "$logfile")" tail -10000 "$logfile" > "${logfile}.tmp" mv "${logfile}.tmp" "$logfile" done done # Cleanup empty files find $LOG_DIRS -name "*.log" -type f -empty -delete 2>/dev/null | wc -l | read empty_deleted if [ $empty_deleted -gt 0 ]; then log "INFO" "Deleted $empty_deleted empty log files" fi # Calculate space saved in human readable format if [ $total_saved -gt 0 ]; then saved_human=$(numfmt --to=iec --suffix=B $total_saved) log "INFO" "Total space saved: $saved_human" fi log "INFO" "Log cleanup completed!" } # Run with error handling set -e trap 'log "ERROR" "Log cleanup failed!"; exit 1' ERR cleanup_logs
📋 Quick Reference Cheat Sheet
| Task | Code Example | Description |
|---|---|---|
| Shebang | #!/bin/bash | First line of script |
| Variable | NAME="Alice" | Define variable |
| Use variable | echo "$NAME" | Print variable |
| Command output | DATE=$(date) | Store command output |
| User input | read -p "Name: " NAME | Get user input |
| If condition | if [ $A -gt $B ]; then | Compare numbers |
| If file exists | if [ -f "$FILE" ]; then | Check if file exists |
| For loop | for i in {1..5}; do | Loop through range |
| While loop | while [ $COUNT -gt 0 ]; do | Loop while true |
| Function | myfunc() { echo "Hello"; } | Define function |
| Function call | myfunc | Call function |
| Arguments | $1, $2, $@ | Script arguments |
| Exit code | exit 0 or exit 1 | Exit script |
| Error check | if command; then | Check if command succeeded |
| Logging | echo "msg" >> logfile | Append to log |
| Cron job | * * * * * /script.sh | Run every minute |
| Cron with log | * * * * * /script.sh >> log 2>&1 | Redirect output |
🚀 Practice Exercises
Exercise 1: Create a System Health Dashboard
#!/bin/bash # health-dashboard.sh # Function to print section header header() { echo "========================================" echo " $1" echo "========================================" } # System info header "System Information" echo "Hostname: $(hostname)" echo "Kernel: $(uname -r)" echo "Uptime: $(uptime -p)" # CPU header "CPU Information" echo "Cores: $(nproc)" echo "Load: $(uptime | awk -F'load average:' '{print $2}')" # Memory header "Memory Information" free -h | awk '/^Mem:/ {print "Total: "$2, "Used: "$3, "Free: "$4, "Usage: "$3/$2*100"%" }' # Disk header "Disk Information" df -h / | awk 'NR==2 {print "Total: "$2, "Used: "$3, "Free: "$4, "Usage: "$5}' # Services header "Services Status" services=("ssh" "nginx" "mysql" "docker") for service in "${services[@]}"; do if systemctl is-active --quiet $service; then echo "✓ $service: Running" else echo "✗ $service: Stopped" fi done
Exercise 2: File Organizer Script
#!/bin/bash # organize-files.sh TARGET_DIR="$1" if [ -z "$TARGET_DIR" ]; then echo "Usage: $0 <directory>" exit 1 fi if [ ! -d "$TARGET_DIR" ]; then echo "Error: Directory not found" exit 1 fi cd "$TARGET_DIR" # Create category directories mkdir -p images documents archives scripts others # Move files by extension for file in *; do if [ -f "$file" ]; then case "${file##*.}" in jpg|jpeg|png|gif|bmp) mv "$file" images/ echo "Moved $file to images/" ;; pdf|doc|docx|txt|md) mv "$file" documents/ echo "Moved $file to documents/" ;; zip|tar|gz|bz2) mv "$file" archives/ echo "Moved $file to archives/" ;; sh|py|js|php) mv "$file" scripts/ echo "Moved $file to scripts/" ;; *) mv "$file" others/ echo "Moved $file to others/" ;; esac fi done echo "Organization complete!"
Exercise 3: Password Generator
#!/bin/bash # password-generator.sh # Default length LENGTH=12 # Parse arguments while getopts "l:h" opt; do case $opt in l) LENGTH=$OPTARG ;; h) echo "Usage: $0 [-l length]"; exit 0 ;; *) echo "Invalid option"; exit 1 ;; esac done # Character sets UPPER="ABCDEFGHIJKLMNOPQRSTUVWXYZ" LOWER="abcdefghijklmnopqrstuvwxyz" DIGITS="0123456789" SPECIAL="!@#$%^&*()_+-=[]{}|;:,.<>?" # Combine all characters ALL_CHARS="${UPPER}${LOWER}${DIGITS}${SPECIAL}" echo "Generating $LENGTH character password..." # Generate password PASSWORD="" for i in $(seq 1 $LENGTH); do # Get random character CHAR=${ALL_CHARS:$((RANDOM % ${#ALL_CHARS})):1} PASSWORD="${PASSWORD}${CHAR}" done echo "Password: $PASSWORD" # Strength check STRENGTH=0 [[ $PASSWORD =~ [A-Z] ]] && ((STRENGTH++)) [[ $PASSWORD =~ [a-z] ]] && ((STRENGTH++)) [[ $PASSWORD =~ [0-9] ]] && ((STRENGTH++)) [[ $PASSWORD =~ [!@#\$%^\&*()_+-=[]{}|;:,.<>?] ]] && ((STRENGTH++)) echo -n "Strength: " case $STRENGTH in 4) echo "Strong" ;; 3) echo "Good" ;; 2) echo "Weak" ;; *) echo "Very Weak" ;; esac
🔗 Master Shell Scripting with Hands-on Labs
Shell scripting is the most practical skill for DevOps automation. The best way to learn is through hands-on practice with real-world scenarios.
👉 Practice shell scripting with guided exercises and real projects at:
https://devops.trainwithsky.com/
Our platform provides:
Step-by-step scripting tutorials
Real automation challenges
Code review and feedback
Production-ready script templates
Progress from beginner to advanced scripting
Frequently Asked Questions
Q: Should I use Bash or Python for scripting?
A: Use Bash for system tasks (file operations, process management). Use Python for complex logic, data processing, or when you need libraries.
Q: How do I debug a shell script?
A: Use set -x to see each command executed, or add echo statements to trace execution.
Q: What's the difference between $VAR and ${VAR}?
A: They're usually the same. Use ${VAR} when concatenating: echo "${VAR}text" vs echo "$VARtext" (which looks for variable VARtext).
Q: How do I make my script run faster?
A: Avoid unnecessary subshells $(command), use built-in shell features, and minimize file operations in loops.
Q: Should I always use #!/bin/bash?
A: Yes for Bash scripts. For maximum compatibility, use #!/usr/bin/env bash.
Q: How do I handle spaces in filenames?
A: Always quote variables: "$filename". Use find -print0 | xargs -0 for commands that don't handle spaces well.
Q: What's the best way to learn shell scripting?
A: 1) Start with small scripts 2) Read others' scripts 3) Practice daily 4) Automate your own tasks.
Have scripting questions or need help with a script? Share your code in the comments below! 💬
Comments
Post a Comment