/secret-management

name: managing-secrets
description: Managing secrets (API keys, database credentials, certificates) with Vault, cloud providers, and Kubernetes. Use when storing sensitive data, rotating credentials, syncing secrets to Kubernetes, implementing dynamic secrets, or scanning code for leaked secrets.

Managing Secrets

Secure storage, rotation, and delivery of secrets (API keys, database credentials, TLS certificates) for applications and infrastructure.

When to Use This Skill

Use when:

• Storing API keys, database credentials, or encryption keys
• Implementing secret rotation (manual or automatic)
• Syncing secrets from external stores to Kubernetes
• Setting up dynamic secrets (database, cloud providers)
• Scanning code for leaked secrets
• Implementing zero-knowledge patterns
• Meeting compliance requirements (SOC 2, ISO 27001, PCI DSS)

Quick Decision Frameworks

Framework 1: Choosing a Secret Store

| Scenario | Primary Choice | Alternative | |----------|----------------|-------------| | Kubernetes + Multi-Cloud | Vault + ESO | Cloud Secret Manager + ESO | | Kubernetes + Single Cloud | Cloud Secret Manager + ESO | Vault + ESO | | Serverless (AWS Lambda) | AWS Secrets Manager | AWS Parameter Store | | Multi-Cloud Enterprise | HashiCorp Vault | Doppler (SaaS) | | Small Team (<10 apps) | Doppler, Infisical | 1Password Secrets Automation | | GitOps-Centric | SOPS (git-encrypted) | Sealed Secrets (K8s-only) |

**Decision Tree:**

• Kubernetes? → External Secrets Operator (ESO) with chosen backend
• Single cloud? → Cloud-native (AWS/GCP/Azure)
• Multi-cloud/on-prem? → HashiCorp Vault
• GitOps? → SOPS or Sealed Secrets

Framework 2: Static vs. Dynamic Secrets

| Secret Type | Use Dynamic? | TTL | Solution | |-------------|-------------|-----|----------| | Database credentials | YES | 1 hour | Vault DB engine | | Cloud IAM (AWS/GCP) | YES | 15 min | Vault cloud engine | | SSH/RDP access | YES | 5 min | Vault SSH engine | | TLS certificates | YES | 24 hours | Vault PKI / cert-manager | | Third-party API keys | NO | Quarterly | Vault KV v2 (manual rotation) |

Framework 3: Kubernetes Secret Delivery

| Method | Use Case | Rotation | Restart Required | |--------|----------|----------|------------------| | **External Secrets Operator** | Static secrets, periodic sync | Polling (1h) | Yes | | **Secrets Store CSI Driver** | File-based, watch rotation | inotify | No | | **Vault Secrets Operator** | Vault-specific, dynamic | Automatic renewal | Optional |

HashiCorp Vault Fundamentals

Core Components

• **Secrets Engines**: KV v2 (static), Database (dynamic), AWS, PKI, SSH
• **Auth Methods**: Kubernetes, JWT/OIDC, AppRole, LDAP
• **Policies**: HCL-based access control (least privilege)
• **Leases**: TTL for secrets, auto-renewal, auto-revocation

Static Secrets (KV v2)

# Create secret
vault kv put secret/myapp/config api_key=sk_live_EXAMPLE

# Read secret
vault kv get secret/myapp/config

# List versions
vault kv metadata get secret/myapp/config

Dynamic Database Credentials

# Configure PostgreSQL
vault write database/config/postgres \
  plugin_name=postgresql-database-plugin \
  connection_url="postgresql://{{username}}:{{password}}@postgres:5432/mydb"

# Create role
vault write database/roles/app-role \
  db_name=postgres \
  creation_statements="CREATE ROLE \"{{name}}\"..." \
  default_ttl="1h"

# Generate credentials
vault read database/creds/app-role

For detailed Vault architecture, see `references/vault-architecture.md`.

Kubernetes Integration

External Secrets Operator (ESO)

Syncs secrets from 30+ providers to Kubernetes Secrets.

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-backend
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      auth:
        kubernetes:
          role: "app-role"

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: database-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
  target:
    name: db-credentials
  data:
  - secretKey: password
    remoteRef:
      key: secret/data/database/config

Vault Secrets Operator (VSO)

Kubernetes-native Vault integration with automatic lease renewal.

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  name: postgres-creds
spec:
  vaultAuthRef: vault-auth
  mount: database
  path: creds/app-role
  renewalPercent: 67  # Renew at 67% of TTL
  destination:
    name: dynamic-db-creds

For ESO vs CSI vs VSO comparison, see `references/kubernetes-integration.md`.

Secret Rotation Patterns

Pattern 1: Versioned Static Secrets (Blue/Green)

1. Create new secret version in Vault 2. Update staging environment 3. Monitor for errors (24-48 hours) 4. Gradual production rollout (10% → 50% → 100%) 5. Revoke old secret (after 7 days)

Pattern 2: Dynamic Database Credentials

Vault auto-generates credentials with short TTL:

• App fetches credentials from Vault
• Vault automatically renews lease (at 67% of TTL)
• On expiration, Vault revokes access
• On renewal failure, app requests new credentials

Pattern 3: TLS Certificate Rotation

Using cert-manager + Vault PKI:

• cert-manager requests certificate from Vault
• Automatically renews before expiration (default: 67% of duration)
• Updates Kubernetes Secret on renewal
• Optional pod restart (via Reloader)

For detailed rotation workflows, see `references/rotation-patterns.md`.

Multi-Language Integration

Python (hvac)

import hvac

client = hvac.Client(url='https://vault.example.com')
client.auth.kubernetes(role='app-role', jwt=jwt)

# Fetch dynamic credentials
response = client.secrets.database.generate_credentials(name='postgres-role')
username = response['data']['username']
password = response['data']['password']

Go (Vault API)

import vault "github.com/hashicorp/vault/api"

client, _ := va