/hf-cloud-sagemaker-deployment-planner

name: hf-cloud-sagemaker-deployment-planner
description: 'Plan and coordinate the deployment of a model to Amazon SageMaker AI. Use this skill whenever the user wants to deploy, host, serve, or expose a model on SageMaker or AWS — including phrases like "deploy a model", "host this LLM on AWS", "serve this embedding model", "deploy a reranker", "deploy a text-to-image / diffusion model", "host this for async inference", "create an endpoint", "serve my fine-tuned model", or any request that involves making a model available for inference on AWS. Use this even when the user is vague (e.g. "I just want to get this running on AWS, you figure it out"). Works for text-generation LLMs, embedding models, rerankers, classifiers, text-to-image / diffusion models — picks the right serving stack and chooses between real-time and async inference. This is the entry-point skill for SageMaker deployment work — it asks clarifying questions, picks a deployment pathway, and coordinates the other deployment skills.'

SageMaker Deployment Planner

You are helping a user deploy a model to Amazon SageMaker. Most users invoking this skill want the model deployed with reasonable defaults, in as few questions as possible. Ask only what you need, recommend a pathway honestly, and hand off to the specialized skills.

Workflow phases

1. **Discovery** — what is being deployed and what are the constraints (this skill) 2. **Pathway selection** — real-time / serverless / async / batch / Bedrock CMI (this skill) 3. **Context preflight** — `hf-cloud-aws-context-discovery`, then `hf-cloud-python-env-setup` 4. **IAM preflight** — `hf-cloud-sagemaker-iam-preflight` 5. **Image selection** — `hf-cloud-serving-image-selection` 6. **Deployment** — `hf-cloud-sagemaker-production-defaults`

Phases 1–2 are this skill's job. The others activate when their patterns match.

Discovery: ask only what you need

You will eventually need to know:

• **What model**: HuggingFace ID, S3 path to artifacts, or model name. If the user is vague ("the model I fine-tuned"), ask for the artifact location.
• **Model type**: text-generation LLM, embedding/reranker, or other (classifier, NER, etc.). This determines the serving stack — usually inferable from the model name (anything ending in `-embed-*`, starting with `BAAI/bge-`, `sentence-transformers/*` etc. is embeddings; chat/instruct models are LLMs). Only ask if it's genuinely ambiguous.
• **Traffic shape**: roughly how often will this be called?
• **Latency tolerance**: interactive, near-real-time, or async?
• **Cost sensitivity**: ask only if the user signals it or the traffic pattern is ambiguous.

Region comes from `hf-cloud-aws-context-discovery` — don't ask unless the user volunteers it.

Do **not** front-load all of these. A common minimal set is just: *what model, and roughly how often will it be called?* The model name usually settles the model-type question. That alone is often enough to narrow the pathway to two candidates. If the user already told you something, don't ask again.

Pathway selection

| Pathway | When it fits | When it does not | |---|---|---| | **Real-time endpoint** | Steady traffic, sub-second to few-second latency, always-on | Very spiky or very sparse traffic (wastes money on idle) | | **Real-time, scale to zero** | Sparse or scheduled traffic, dev/test endpoints, and a client that tolerates a ~9 min first request after idle | Any interactive SLA: every request during the wake fails with a 400 | | **Serverless inference** | Spiky/intermittent, tolerates cold starts (~10s+), simpler models | LLMs above a few B params (memory/cold-start limits), strict SLAs | | **Async inference** | Long inference (>60s), large payloads, queue-friendly | Interactive synchronous calls | | **Batch transform** | Offline scoring over a dataset | Anything online or interactive | | **Bedrock Custom Model Import** | Wants Bedrock-compatible API, supported base family, weights only | Custom inference logic, unsupported architectures |

For LLMs, **real-time endpoints are the default** unless traffic is explicitly spiky/sparse or inference is long-running. Serverless looks attractive for "low traffic" cases but most LLMs exceed its memory limits.

For **embeddings**, real-time is again the default — but CPU instances are usually the right choice (much cheaper, fast enough for most embedding workloads). Don't reflexively recommend GPU instances for embedding models; ask `hf-cloud-serving-image-selection` to consider CPU variants if the model is small (<1B params) and traffic is moderate.

For **text-to-image, video generation, or other long-inference workloads** (>30s per request) where traffic is also bursty: async inference is the right answer. It supports genuine scale-to-zero between batches and queues requests via S3, so you don't pay for idle GPU. `hf-cloud-sagemaker-production-defaults` has a dedicated `deploy_async.py` for this.

Real-time, real-time scale-to-zero, and async are the three scripted pathways (`deploy.py`, `deploy_ic.py`, `deploy_async.py` in `hf-cloud-sagemaker-production-defaults`). Serverless, batch transform, and Bedrock Custom Model Import are not currently scripted — for those, hand the user off with a brief explanation rather than trying to deploy them through this workflow.

**Scale to zero, real-time or async?** Both reach zero and both make the first request after idle slow. Pick async when one inference can exceed the 60s `InvokeEndpoint` limit, when payloads are large, or when the client can accept an S3 result instead of a synchronous response. Pick real-time scale-to-zero when the client needs a normal synchronous HTTP response and can retry through the wake. Real-time scale-to-zero needs inference components; the plain real-time pathway cannot go below one instance.

If two pathways are both reasonable, say so in one sentence each and pick one. Don't bury the recommendation in options.

Instance selection: check quota before recommendi