/running-openmed-ondevice

name: running-openmed-ondevice
description: "Run OpenMed models fully on-device with the MLX (Apple Silicon), CoreML (iOS/macOS), or ONNX/WebGPU (cross-platform/browser) backends, including convert-quantize-run workflows. Use when the user wants to deploy OpenMed at the edge, run NER/de-id on Apple Silicon, target iPhone/iPad/Mac, export to ONNX or WebGPU, quantize a clinical model to int8/4-bit, run with no network, or pick between MLX/CoreML/ONNX. Covers the mlx/coreml/onnx extras, the convert() functions and python -m convert CLIs, quantization, loading a local MLX artifact through analyze_text, OpenMedMLXLanguageModel/generate_text, and the on-device-only PHI guarantee (nothing leaves the host)."
license: Apache-2.0
metadata:
  project: OpenMed
  category: deployment-ops
  pairs: adjacent
  version: "1.0"

Running OpenMed on-device

OpenMed runs **fully on-device** by design. These three backends let you take it further at the edge: **MLX** (Apple Silicon acceleration), **CoreML** (iOS/macOS / Neural Engine), and **ONNX / WebGPU** (cross-platform and in-browser). The flow is the same: **convert → (quantize) → run locally**. Because inference is local, raw PHI never leaves the device — the strongest privacy posture OpenMed offers.

When to use this skill

When you need OpenMed where there is no server: an iOS/macOS app (CoreML), fast NER/de-id on an Apple Silicon Mac (MLX), or a portable/browser deployment (ONNX/WebGPU). For a hosted endpoint use `serving-openmed-rest-api`; for an agent tool use `deploying-openmed-mcp`; for corpora use `batch-processing-clinical-text`.

Pick a backend

| Backend | Extra | Best for | Quantization | | --- | --- | --- | --- | | **MLX** | `openmed[mlx]` | Apple Silicon Macs; fastest local NER/de-id; on-device LLMs | 4-bit / 8-bit weights | | **CoreML** | `openmed[coreml]` | iOS/iPadOS/macOS apps, Neural Engine | int8 palettization | | **ONNX / WebGPU** | `openmed[onnx]` | cross-platform runtimes, browser (transformers.js) | fp16 (WebGPU); int8 via ORT |

Quick start — MLX (Apple Silicon)

pip install "openmed[mlx]"

# Convert a HF token-classification model to an OpenMed MLX artifact, 8-bit:
python -m openmed.mlx.convert --model OpenMed/<some-ner-model> --output ./mlx_ner --quantize 8

import openmed

# Run NER/de-id through the normal API — pass the local artifact dir as model_name.
# The loader auto-detects the MLX backend from the artifact (or set backend explicitly).
result = openmed.analyze_text(
    "Patient received 75mg clopidogrel for NSTEMI.",
    model_name="./mlx_ner",          # local MLX artifact directory
    output_format="dict",
)

# Force MLX via config if you prefer to be explicit:
from openmed.core.config import OpenMedConfig
cfg = OpenMedConfig(backend="mlx")   # None=auto-detect, "mlx", or "hf"

`convert()` is also importable: `openmed.mlx.convert.convert(model_id, output_dir, quantize_bits=8)`. The CLI accepts `--quantize {4,8}`, `--quantize-group-size`, `--cache-dir`, and an optional `--eval-suite` to **certify quantized recall** against the full-precision parent (recommended for clinical models — quantization can drop recall on rare entities).

On-device LLM generation (MLX)

from openmed.mlx.lm import generate_text, OpenMedMLXLanguageModel

text = generate_text(
    messages=[{"role": "user", "content": "Summarize: chest pain, troponin elevated."}],
    model_name="OpenMed/laneformer-2b-it-q4-mlx",   # resolves to a local MLX-LM artifact
    max_tokens=128,
)

llm = OpenMedMLXLanguageModel("OpenMed/laneformer-2b-it-q4-mlx")
out = llm.generate(prompt="...", max_tokens=64, temp=0.0)

Quick start — CoreML (iOS/macOS)

pip install "openmed[coreml]"
python -m openmed.coreml.convert --model OpenMed/<some-ner-model> --output model.mlpackage --quantize int8

from openmed.coreml.convert import convert
convert(
    "OpenMed/<some-ner-model>",
    "model.mlpackage",
    compute_units="cpuAndNeuralEngine",   # "all" | "cpuAndNeuralEngine" | "cpuOnly"
    compute_precision="float16",          # float16 for Neural Engine, float32 for CPU
    quantize="int8",                      # emits an int8-palettized sibling .mlpackage
)

Bundle the `.mlpackage` in your Xcode app and run it with Core ML; the converter writes the `id2label` map so your app can decode token labels. Use `float16` + `cpuAndNeuralEngine` for the Neural Engine; `int8` shrinks the model for storage-constrained devices.

Quick start — ONNX / WebGPU

pip install "openmed[onnx]"
python -m openmed.onnx.convert --model OpenMed/<some-ner-model> --output ./onnx_out

from openmed.onnx.convert import convert
res = convert("OpenMed/<some-ner-model>", "./onnx_out", include_webgpu=True, opset=18)
# Emits model.onnx (fp32) and model.webgpu.onnx (fp16) + an export manifest.

Run `model.onnx` with ONNX Runtime on any platform, or ship `model.webgpu.onnx` to the browser via transformers.js for in-page, zero-upload inference. Use `--no-webgpu` to skip the fp16 artifact.

Workflow

1. **Pick the backend** for the target (table above). 2. **Convert** the HF/OpenMed model with the matching `convert()` / `python -m openmed.<backend>.convert`. 3. **Quantize** if size/latency demands it (MLX 4/8-bit, CoreML int8, WebGPU fp16). For clinical de-id/NER, **certify recall** — MLX's `--eval-suite` writes a recall-delta report so you don't silently lose rare entities. 4. **Run locally:** MLX artifacts go straight through `analyze_text` / `deidentify`; CoreML/ONNX artifacts run in their native runtimes (Core ML, ONNX Runtime, transformers.js). 5. **Verify** outputs against the full-precision model before shipping (`evaluating-with-leakage-gates` for de-id).

Hand-off to / from OpenMed

• **Same API surface:** an MLX artifact path is a drop-in `model_name` for

`openmed.analyze_text` / `deidentify` — downstream skills (`building-patient-timelines`, `exporting-to-fh