/agentsop-selfhost-decision

name: agentsop-selfhost-decision
version: 0.1.0
description: >-
  Project-kickoff rubric for the self-host vs managed-cloud decision — when is running your
  own inference engine / LLM platform worth the ops cost vs paying per-token for a managed
  API? Decide on two axes — VOLUME (a cost-crossover slider) and COMPLIANCE (a hard gate).
  Use at kickoff when choosing where to run inference, or when cost / data-residency
  pressure forces a re-evaluation.
domain: deployment-decision / infrastructure-economics
kind: enhancement-overlay (project-kickoff rubric)
phase: D-enhance (D6)
cross_links:
  - llm-engine-selection
  - dify-sop
trigger_keywords:
  - "self-host vs cloud"
  - "self-host vs API"
  - "managed API vs run our own"
  - "is it cheaper to self-host the model"
  - "data residency LLM"
  - "air-gapped LLM"
  - "on-prem inference"
  - "GPU cost vs API cost"
  - "Dify cloud vs self-host"
  - "when to self-host inference"
when_to_use:
  - "at project kickoff, deciding where an LLM / platform runs: managed API/cloud vs self-hosted GPU"
  - "cost pressure: monthly API spend is climbing and someone asks 'should we just run our own?'"
  - "compliance pressure: data-residency / air-gap / regulated-data requirement appears mid-project"
  - "evaluating a self-hostable platform's paid tiers (Dify Cloud vs Docker; managed vLLM vs your own GPUs)"
  - "designing a hybrid (self-host baseline + managed burst) topology"
when_not_to_use:
  - "WHICH inference engine to run once you've decided to self-host — that's [[agentsop-llm-engine-selection]]"
  - "HOW to build the Dify app once you've decided to self-host it — that's [[agentsop-dify]]"
  - "single-call / hobby / one-user workloads where the answer is trivially 'just call the API'"
  - "training / fine-tuning siting (different cost structure: burst GPU, not steady serving)"

Self-host vs Managed-cloud Decision — A Project-Kickoff Rubric

> **Overlay, not a deep dive.** This skill answers *where to run* (self-host vs managed), not *which engine* ([[agentsop-llm-engine-selection]]) or *how to build the app* ([[agentsop-dify]]). It fires first, at kickoff, and hands off to those once the side is chosen.

---

1. 何时激活 (When to Activate)

1.1 直接信号 (Direct triggers)

• At kickoff you must decide **where an LLM or LLM platform runs**: a managed API/cloud (OpenAI / Anthropic / Bedrock / Dify Cloud) vs **your own GPUs / your own Docker** (vLLM, self-hosted Dify).
• **Cost pressure**: monthly managed spend is climbing; someone says *"should we just run our own and stop paying per token?"*
• **Compliance pressure**: a **data-residency / air-gap / regulated-data** requirement (finance, medical, gov, GDPR region-lock) appears and the managed path is suddenly in question.
• You're comparing a self-hostable platform's tiers — e.g. **Dify Cloud Pro ($59) vs self-deployed Docker** [architjn.com/blog/dify-cloud-pricing-plans], or managed-vLLM-as-a-service vs your own H100s.

1.2 反向信号 (Skip this rubric when)

• The decision is already *self-host*, and the open question is **which engine** → go to **[[agentsop-llm-engine-selection]]** (vLLM vs TGI vs SGLang vs TensorRT-LLM vs llama.cpp).
• The decision is already *self-host Dify*, and the open question is **how to build/operate it** → go to **[[agentsop-dify]]**.
• **Single user / hobby / one stream** — the answer is "just call the managed API"; no rubric needed.
• **Training / fine-tuning** siting — different cost structure (burst GPU, spot, not steady-state serving).

1.3 心智门槛 (Mental check)

> This rubric exists because the loud reflex — *"running our own is cheaper / more serious"* — is **true only above a volume crossover, and only if you have the ops capacity, and only if compliance hasn't already forced your hand.** The job is to evaluate the **gate before the slider**, and to cost the **ops burden**, not just the GPU.

判断公式:

• 如果你的瓶颈是 **per-token spend at high, predictable volume** → self-host *may* win (run OP-1).
• 如果你的瓶颈是 **data can't leave our boundary** → compliance gate decides (run OP-2), cost is secondary.
• 如果你的瓶颈是 **we don't have anyone to run a GPU box at 3am** → managed wins regardless of the GPU math (run OP-3).

---

2. 核心心智模型 (Core Mental Model)

2.1 The trade, in one line

> **Self-host trades ops burden for control + unit-cost-at-scale. Managed trades $/token for zero ops. The crossover is a function of two axes: volume and compliance.**

• **Managed**: you pay **$/token**, marginal, no fixed cost, no ops. Cost scales *linearly* with usage and never sleeps.
• **Self-host**: you pay a **fixed floor** (GPU + ops labor + infra) plus a tiny marginal cost. Cost is *flat-then-cheap-per-unit* — but only if utilization stays high.

2.2 Two axes, not one — and they are not symmetric

                COMPLIANCE  (a GATE — binary, evaluated FIRST)
                     │
   managed FORBIDDEN │  self-host (or in-region managed) MANDATORY
   ──────────────────┼──────────────────────────────────────────►  VOLUME
                     │                                    (a SLIDER —
   managed allowed   │   below V*: managed cheaper          continuous
                     │   above V*: self-host cheaper         crossover)
                     │   (IF utilization high + ops capacity exists)

• **Compliance is a GATE**: binary, evaluated **before** cost. A hard data-residency / air-gap NO **fails the managed path regardless of volume** [dify Case 3]. (Caveat: a managed in-region / VPC / BAA tier can re-open the gate.)
• **Volume is a SLIDER**: a continuous cost crossover `V*`. Below it managed wins; above it self-host wins — *conditionally*.

2.3 The crossover formula (OP-1)

managed_cost(V)   = V × $/token_managed
selfhost_cost(V)  = (GPU + ops_labor + infra_fixed) + V × $/token_marginal

V*  =  (GPU + ops_labor + infra_fixed) / ($/token_managed − $/token_marginal)

• **Below V***: managed total cost is lower → use managed.
• **Above V***: self-host total c