/activity-monitor

name: activity-monitor
description: Guardian service that monitors the active runtime agent's state and automatically restarts it if stopped. Use when checking agent liveness state or understanding the auto-restart mechanism.
user-invocable: false

Activity Monitor Skill

PM2 guardian service that monitors the active runtime agent's (Claude or Codex) activity state and ensures it's always running.

When to Use

This is a **PM2 service** (not directly invoked by Claude). It runs continuously in the background.

What It Does

1. **Activity Monitoring**: Tracks the agent's busy/idle state every second 2. **Status File**: Writes `~/zylos/activity-monitor/agent-status.json` with current state (busy/idle, idle_seconds, health) 3. **Guardian Mode**: Automatically restarts the agent if it stops or crashes 4. **Maintenance Awareness**: Waits for restart/upgrade scripts to complete before starting the agent 5. **Recovery Liveness Detection**: Sends C4 heartbeat probes during recovery and after restarts; healthy sessions have no periodic primary heartbeat 6. **Health Check**: Periodically enqueues system health checks (PM2, disk, memory) via the C4 control queue 7. **Daily Upgrade**: Enqueues a Claude Code upgrade via the C4 control queue at 5:00 AM local time daily (disabled by default; enable with `zylos config set daily_upgrade_enabled true`) 8. **Context Monitoring**: Receives context usage data after every turn; triggers early memory sync and new-session handoff at thresholds read from `config.json` (`new_session_threshold` for Claude, `codex_new_session_threshold` for Codex), each with a built-in runtime fallback if the key is unset

Status File Format

{
  "state": "idle",
  "last_activity": 1738675200,
  "last_check": 1738675210,
  "last_check_human": "2026-02-04 20:30:10",
  "idle_seconds": 5,
  "inactive_seconds": 10,
  "source": "conv_file",
  "health": "ok"
}

• `state`: "busy" | "idle" | "stopped" | "offline"
• `idle_seconds`: Time since entering idle state (0 when busy)
• `source`: "conv_file" (reliable) | "tmux_activity" (fallback)
• `health`: "ok" | "unavailable" | "rate_limited" | "auth_failed" — current public health; legacy `recovering` / `down` states remain readable

PM2 Management

# Start
pm2 start ~/zylos/.claude/skills/activity-monitor/scripts/activity-monitor.js --name activity-monitor

# Restart
pm2 restart activity-monitor

# View logs
pm2 logs activity-monitor

# Check status
pm2 list

Guardian Behavior

• **Detection**: Checks if the agent is running every second
• **Restart Delay**: Waits 5 seconds of continuous "not running" before restarting
• **Maintenance Wait**: Detects restart/upgrade scripts and waits for completion
• **Recovery Prompt**: Sends catch-up message via C4 after restart

How It Works

1. **Monitor Loop** (every 1s):

• Check if tmux session exists
• Check if agent process is running
• Detect activity (conversation file mtime for Claude; tmux activity for Codex)
• Calculate idle/busy state
• Write status to ~/zylos/activity-monitor/agent-status.json

2. **Guardian Logic**:

• If agent not running for 5+ seconds → restart
• Wait for maintenance scripts to complete
• Send recovery prompt via C4 after successful restart

3. **Activity Detection**:

• Claude primary: Conversation file modification time (reliable)
• Codex / fallback: tmux window activity
• Threshold: 3 seconds without activity = idle

Heartbeat Liveness Detection

The health engine uses the C4 control queue to verify runtime responsiveness during recovery. Healthy sessions do not receive periodic `primary` probes, and there is no heartbeat enable/interval setting. The internal maintenance timer still processes pending probe results and recovery scheduling.

Recovery Probes

| Phase | Trigger | On Success | On Failure | |-------|---------|------------|------------| | `recovery` | Unavailable/recovering maintenance retry or user-message recovery | Return to `ok` | Apply existing rate-limit detection or recovery handling | | `post_restart` | Restart notification or process-running signal after its grace period | Return to `ok` | Apply existing recovery handling | | `down-check` | Legacy `down` state retry interval | Return to `ok` | Stay `down` and retry later |

Retries in unavailable/recovering use exponential backoff (1m, 5m, 25m, then 60m). Rate-limit cooldown expiry only clears the wait; a later recovery trigger verifies responsiveness. Failed or expired pending probes still run their existing failure handling, including API-error fast scanning while waiting.

Recovery and post-restart probes require end-to-end acknowledgement. The dispatcher retains confirmed-active busy auto-ack for other heartbeat phases; idle heartbeat auto-ack has been removed.

See [Recovery and maintenance behavior](references/health-engine.md) for timing and compatibility details.

Unhealthy Message Routing

When health is not `ok`, C4 intake still records incoming messages as delivered. The MessageRouter returns an immediate status response for the current message when replies are enabled; `--no-reply` messages are accepted silently. If the router IPC is unavailable, `c4-receive.js` falls back to the status file and follows the same delivered/current-message behavior.

Health Check

The activity monitor periodically enqueues system health checks via the C4 control queue.

• **Enabled**: On by default. Disable with `zylos config set health_check_enabled false`
• **Interval**: Every 24 hours (86400 seconds)
• **Persisted state**: `~/zylos/activity-monitor/health-check-state.json` (survives restarts)
• **Priority**: 3 (normal)
• **Gated by health**: Only enqueued when `health === 'ok'`
• **Gated by agent**: Only enqueued when the agent process is running

The health check control message instructs Claude to: 1. Check PM2 services via the health-check skill's sanitized PM2 snapshot helper 2. Check disk space via `df -h`