/shell-scripting

name: shell-scripting
description: Write robust, portable shell scripts with proper error handling, argument parsing, and testing. Use when automating system tasks, building CI/CD scripts, or creating container entrypoints.

Shell Scripting

Purpose

Provides patterns and best practices for writing maintainable shell scripts with error handling, argument parsing, and portability considerations. Covers POSIX sh vs Bash decision-making, parameter expansion, integration with common utilities (jq, yq, awk), and testing with ShellCheck and Bats.

When to Use This Skill

Use shell scripting when:

• Orchestrating existing command-line tools and system utilities
• Writing CI/CD pipeline scripts (GitHub Actions, GitLab CI)
• Creating container entrypoints and initialization scripts
• Automating system administration tasks (backups, log rotation)
• Building development tooling (build scripts, test runners)

Consider Python/Go instead when:

• Complex business logic or data structures required
• Cross-platform GUI needed
• Heavy API integration (REST, gRPC)
• Script exceeds 200 lines with significant logic complexity

POSIX sh vs Bash

**Use POSIX sh (#!/bin/sh) when:**

• Maximum portability required (Linux, macOS, BSD, Alpine)
• Minimal container images needed
• Embedded systems or unknown target environments

**Use Bash (#!/bin/bash) when:**

• Controlled environment (specific OS, container)
• Arrays or associative arrays needed
• Advanced parameter expansion beneficial
• Process substitution `<(cmd)` useful

For detailed comparison and testing strategies, see `references/portability-guide.md`.

Essential Error Handling

Fail-Fast Pattern

#!/bin/bash
set -euo pipefail

# -e: Exit on error
# -u: Exit on undefined variable
# -o pipefail: Pipeline fails if any command fails

Use for production automation, CI/CD scripts, and critical operations.

Explicit Exit Code Checking

#!/bin/bash

if ! command_that_might_fail; then
    echo "Error: Command failed" >&2
    exit 1
fi

Use for custom error messages and interactive scripts.

Trap Handlers for Cleanup

#!/bin/bash
set -euo pipefail

TEMP_FILE=$(mktemp)

cleanup() {
    rm -f "$TEMP_FILE"
}

trap cleanup EXIT

Use for guaranteed cleanup of temporary files, locks, and resources.

For comprehensive error patterns, see `references/error-handling.md`.

Argument Parsing

Short Options with getopts (POSIX)

#!/bin/bash

while getopts "hvf:o:" opt; do
    case "$opt" in
        h) usage ;;
        v) VERBOSE=true ;;
        f) INPUT_FILE="$OPTARG" ;;
        o) OUTPUT_FILE="$OPTARG" ;;
        *) usage ;;
    esac
done

shift $((OPTIND - 1))

Long Options (Manual Parsing)

#!/bin/bash

while [[ $# -gt 0 ]]; do
    case "$1" in
        --help) usage ;;
        --verbose) VERBOSE=true; shift ;;
        --file) INPUT_FILE="$2"; shift 2 ;;
        --file=*) INPUT_FILE="${1#*=}"; shift ;;
        *) break ;;
    esac
done

For hybrid approaches and validation patterns, see `references/argument-parsing.md`.

Parameter Expansion Quick Reference

# Default values
${var:-default}              # Use default if unset
${var:=default}              # Assign default if unset
: "${API_KEY:?Error: required}"  # Error if unset

# String manipulation
${#var}                      # String length
${var:offset:length}         # Substring
${var%.txt}                  # Remove suffix
${var##*/}                   # Basename
${var/old/new}               # Replace first
${var//old/new}              # Replace all

# Case conversion (Bash 4+)
${var^^}                     # Uppercase
${var,,}                     # Lowercase

For complete expansion patterns and array handling, see `references/parameter-expansion.md`.

Common Utilities Integration

JSON with jq

# Extract field
name=$(curl -sSL https://api.example.com/user | jq -r '.name')

# Filter array
active=$(jq '.users[] | select(.active) | .name' data.json)

# Check existence
if ! echo "$json" | jq -e '.field' >/dev/null; then
    echo "Error: Field missing" >&2
fi

YAML with yq

# Read value (yq v4)
host=$(yq eval '.database.host' config.yaml)

# Update in-place
yq eval '.port = 5432' -i config.yaml

# Convert to JSON
yq eval -o=json config.yaml

Text Processing

# awk: Extract columns
awk -F',' '{print $1, $3}' data.csv

# sed: Replace text
sed 's/old/new/g' file.txt

# grep: Pattern match
grep -E "ERROR|WARN" logfile.txt

For detailed examples and best practices, see `references/common-utilities.md`.

Testing and Validation

ShellCheck: Static Analysis

# Check script
shellcheck script.sh

# POSIX compliance
shellcheck --shell=sh script.sh

# Exclude warnings
shellcheck --exclude=SC2086 script.sh

Bats: Automated Testing

#!/usr/bin/env bats

@test "script runs successfully" {
    run ./script.sh --help
    [ "$status" -eq 0 ]
    [ "${lines[0]}" = "Usage: script.sh [OPTIONS]" ]
}

@test "handles missing argument" {
    run ./script.sh
    [ "$status" -eq 1 ]
    [[ "$output" =~ "Error" ]]
}

Run tests:

bats test/

For CI/CD integration and debugging techniques, see `references/testing-guide.md`.

Defensive Programming Checklist

#!/bin/bash
set -euo pipefail

# Check required commands
command -v jq >/dev/null 2>&1 || {
    echo "Error: jq required" >&2
    exit 1
}

# Check environment variables
: "${API_KEY:?Error: API_KEY required}"

# Check files
[ -f "$CONFIG_FILE" ] || {
    echo "Error: Config not found: $CONFIG_FILE" >&2
    exit 1
}

# Quote all variables
echo "Processing: $file"        # ❌ Unquoted
echo "Processing: \"$file\""    # ✅ Quoted

Platform Considerations

macOS vs Linux Differences

# sed in-place
sed -i '' 's/old/new/g' file.txt    # macOS
sed -i 's/old/new/g' file.txt       # Linux

# Portable: Use temp file
sed 's/old/new/g' file.txt > file.txt.tmp
mv file.txt.t