/devops-storage-backup

name: devops-storage-backup
description: Use when choosing, running or repairing storage and backups - block vs file vs object, ZFS (pools, vdev layouts, datasets, snapshots, send/receive, scrub, ARC), Ceph (cephadm, MON/MGR/OSD/MDS, pools, CRUSH, RBD/CephFS/RGW, health, OSD replacement, capacity), NFS and iSCSI basics, S3-compatible object storage (MinIO status, Garage, SeaweedFS, Ceph RGW), Kubernetes storage pointers (CSI, Rook-Ceph, Longhorn), backup strategy (3-2-1-1-0, RPO/RTO, retention, immutability, encryption, off-site), restic, BorgBackup/borgmatic, Velero, rclone, Proxmox Backup Server pointer, restore testing, disaster-recovery runbooks, capacity monitoring and SMART. Not for LVM and single-host filesystems (devops-linux), database dumps and PITR (devops-databases), Proxmox PBS jobs and vzdump (devops-proxmox), PV/PVC basics (devops-kubernetes) or public-cloud storage services.
versions:
  openzfs: "2.4.4 (GitHub release zfs-2.4.4, 2026-08-21)"
  ceph: "21.2.0 Umbrella (2026-10-28), 20.2.5 Tentacle, 19.2.7 Squid (EOL 2026-12-02)"
  restic: "0.19.1 (2026-07-05)"
  borgbackup: "1.4.5 stable (Borg 2 is in beta)"
  borgmatic: "2.2.0"
  velero: "1.18.4 (repository velero-io/velero)"
  rclone: "1.76.0"
  rook: "1.21.2"
  longhorn: "1.13.0"
  seaweedfs: "4.53 (Apache-2.0)"
  minio: "community repository no longer maintained"
user-invocable: true
references:
  - references/storage-selection.md
  - references/nfs-iscsi.md
  - references/object-storage-s3.md
  - references/kubernetes-storage-pointers.md
  - references/zfs-pools-datasets.md
  - references/zfs-snapshots-replication.md
  - references/zfs-destructive-ops.md
  - references/ceph-architecture-deploy.md
  - references/ceph-operations-health.md
  - references/ceph-destructive-ops.md
  - references/backup-strategy.md
  - references/restic.md
  - references/borg-borgmatic.md
  - references/velero-kubernetes-backup.md
  - references/rclone-pbs-pointers.md
  - references/backup-destructive-ops.md
  - references/restore-testing-dr.md
  - references/capacity-smart-monitoring.md
  - references/templates/restic-backup.sh
  - references/templates/restic-backup.service
  - references/templates/restic-backup.timer
  - references/templates/restic-maintenance.sh
  - references/templates/borgmatic-config.yaml
  - references/templates/zfs-snapshot-replicate.sh
  - references/templates/zfs-snapshot-policy.md
  - references/templates/velero-schedule.yaml
  - references/templates/restore-test-checklist.md
  - references/templates/dr-runbook.md
  - references/templates/backup-policy-table.md
related-skills:
  - devops-linux
  - devops-databases
  - devops-proxmox
  - devops-kubernetes
  - devops-k8s-cluster
  - devops-openstack
  - devops-observability
  - devops-secrets
  - devops-sre-ops

<objective> Covers where data lives and how it is protected: choosing a storage type, ZFS, Ceph, NFS/iSCSI, S3-compatible object storage, backup strategy and the tools that implement it (restic, Borg/borgmatic, Velero, rclone), restore testing, disaster-recovery runbooks and capacity/SMART monitoring. Does not cover LVM and single-host filesystems (devops-linux), database dumps and point-in-time recovery (devops-databases), Proxmox vzdump and PBS jobs (devops-proxmox), PV/PVC mechanics (devops-kubernetes) or public-cloud storage services. </objective>

DevOps Storage and Backup

Agent Workflow (MANDATORY)

Before ANY storage or backup change, spawn in parallel, one `Agent` call each with a `name`:

1. **fuse-ai-pilot:explore-codebase** - find existing backup scripts, timers, pool/dataset layout files, Velero manifests, inventories 2. **fuse-ai-pilot:research-expert** - verify the flag or option for the installed version at openzfs.github.io, docs.ceph.com, restic.readthedocs.io, borgbackup.readthedocs.io, velero.io 3. **mcp__context7__query-docs** - `/websites/openzfs_github_io_openzfs-docs`, `/ceph/ceph`, `/restic/restic`, `/websites/borgbackup_readthedocs_io_en_stable`, `/velero-io/velero`

Then read the system read-only (`zpool status`, `zfs list`, `ceph -s`, `restic snapshots`, `velero backup get`). After file changes run **fuse-ai-pilot:sniper**.

Versions (checked 2026-11-10)

OpenZFS zfs-2.4.4, Ceph 21.2.0 / 20.2.5 / 19.2.7, restic 0.19.1, Borg 1.4.5, borgmatic 2.2.0, Velero 1.18.4, rclone 1.76.0, Rook 1.21.2, Longhorn 1.13.0 (GitHub release pages and docs.ceph.com/releases, read today). Proxmox VE 9.2 ships Ceph Squid and Tentacle, not Umbrella: do not recommend Umbrella for PVE (devops-proxmox). Always compare with the installed version before using a flag.

Overview

| Area | Decision in one line | |------|----------------------| | **Type** | Block for one VM/DB, file for shared POSIX, object for blobs and backups ([storage-selection.md](references/storage-selection.md)) | | **ZFS** | Mirrors or raidz2 vdevs, `ashift` fixed at vdev creation, scrub monthly, snapshots are not backups | | **Ceph** | 3 MON, replicated size 3 / min_size 2, failure domain host, never run above 85 percent | | **Backup** | 3-2-1-1-0 policy, one immutable or offline copy, encryption on, restore test on a schedule | | **Tools** | restic or Borg for hosts, Velero for Kubernetes objects, ZFS send for datasets, PBS for Proxmox guests | | **Proof** | A backup that was never restored is a hope, not a backup ([restore-testing-dr.md](references/restore-testing-dr.md)) |

Critical Rules

1. **Read-only first** - `zpool status`, `zfs list -t all`, `ceph -s`, `restic snapshots`, `velero backup get` before any change 2. **Dry-run before every destroy** - `zfs destroy -nv`, `restic forget --dry-run`, `borg prune --dry-run`, `ceph orch apply ... --dry-run` 3. **Destructive commands are proposed, not run** - ACTION / EFFECT / PROOF / WAIT-FOR-GO ([zfs-destructive-ops.md](references/zfs-destructive-ops.md), [ceph-destructive-ops.md](references/ceph-destructive-ops.md), [backup-destructive-ops.md](references/backup-destructive-ops.md)) 4. **A snapshot on the same pool i