/model-serving-security

name: model-serving-security
description: >-
  Harden a deployed model inference endpoint — authn/authz, rate limiting and abuse
  control, input validation, and exposure to model-extraction / inversion /
  membership-inference attacks. Use when reviewing how a model is served (REST/gRPC,
  Triton/TorchServe/KServe, or a hosted inference API), not how it was trained.

Goal

An inference endpoint that's authenticated, abuse-resistant, and not leaking the model or its training data through its outputs — covering the serving layer beneath LLM/app-level concerns.

What to review

• **AuthN/Z & exposure** — is the endpoint authenticated and authorized per caller?

Not publicly reachable without controls? Tenant isolation for multi-tenant serving?

• **Rate limiting & abuse / cost** — per-caller quotas to blunt **model extraction**

(stealing the model by querying it) and to cap inference cost/DoS.

• **Model extraction** — high-volume systematic querying to clone the model; mitigate

with rate limits, anomaly detection, output granularity limits, and monitoring.

• **Model inversion / membership inference** — outputs (esp. confidence scores/logits)

that leak training data or reveal whether a record was in the training set; limit exposed confidence detail.

• **Input validation & resource safety** — malformed/adversarial/oversized inputs;

tensor/shape validation; timeouts and resource caps to prevent inference DoS.

• **Serving infra** — the model server itself (Triton/TorchServe/KServe) patched and

hardened; management/metrics endpoints not exposed; no unsafe model auto-loading.

Steps

1. Identify the serving stack, exposure, and who can call it. 2. Check authn/z, rate limiting/quotas, and input validation at the endpoint. 3. Assess extraction/inversion/membership-inference exposure — especially what confidence/score detail is returned. 4. Review the model server's own hardening and management-surface exposure. 5. Recommend controls prioritized by exposure (unauthenticated/public first).

Output

A findings list (surface · risk · evidence · fix) plus controls: authn/z, rate limits/quotas, input validation, output-granularity limits, and monitoring. For LLM-specific output risks (prompt injection, jailbreaks) defer to `llm-security`; for the app around it, the appsec plugins.

Notes

The model is intellectual property and the training data is a confidentiality boundary — both can leak through a perfectly "working" endpoint via extraction and inversion. Returning raw confidence scores/logits is the quiet enabler of inversion and membership inference; expose the minimum the use case needs. Rate limiting is the single highest-leverage control here.