/deidentifying-multilingual-text

name: deidentifying-multilingual-text
description: "De-identify non-English clinical text on-device with OpenMed by passing lang= and locale= to deidentify(). Use when the user has Spanish, German, French, Italian, Portuguese, Dutch, Hindi, Telugu, Arabic, Japanese, or Turkish medical notes, needs locale-aware fake surrogates, must handle language-specific national IDs (DNI, NIR, Steuer-ID, codice fiscale, BSN, CPF, TCKN, Aadhaar), or asks which languages OpenMed PII supports. Covers SUPPORTED_LANGUAGES, get_pii_models_by_language, get_patterns_for_language, LANG_TO_LOCALE, and accent normalization. Pairs with OpenMed deidentifying-clinical-text and generating-synthetic-surrogates."
license: Apache-2.0
metadata:
  project: OpenMed
  category: de-identification
  pairs: adjacent
  version: "1.0"

De-identifying multilingual text

OpenMed de-identifies clinical text in many languages, each with a dedicated PII model, language-specific regex patterns (national IDs, phone formats), and a locale-aware surrogate generator. Pass `lang=` to `deidentify` / `extract_pii` and the right model, patterns, and fake-data tables are selected automatically. Everything runs **on-device**.

When to use this skill

Use it whenever the source text is not English, or when surrogates must look native to the locale (a German note should get German-looking fake names and a valid-format Steuer-ID surrogate, not a US SSN).

Discover supported languages at runtime — don't hardcode

import openmed
from openmed.core.pii_i18n import SUPPORTED_LANGUAGES, get_patterns_for_language

print(sorted(SUPPORTED_LANGUAGES))   # query it; the set is the source of truth
# Language-appropriate default model for a code:
models = openmed.get_pii_models_by_language("es")
# Language-specific regex patterns (national IDs, phones, etc.):
patterns = get_patterns_for_language("de")

The set currently spans English plus European, South Asian, Middle Eastern, and East Asian languages — but **always read `SUPPORTED_LANGUAGES`** rather than trusting a number, since it changes as models ship. MCP exposes the same list via `openmed_list_pii_languages`.

Quick start (Spanish)

import openmed

nota = (
    "El paciente Carlos Hernández (DNI 12345678Z), nacido el 11/04/1979, "
    "vive en Calle Mayor 5, Madrid. Teléfono 612 345 678."
)

result = openmed.deidentify(
    nota,
    lang="es",                # selects the Spanish PII model + ES patterns
    method="replace",         # locale-native fake values
    locale="es_ES",           # Faker locale (defaults from lang via LANG_TO_LOCALE)
)
print(result.deidentified_text)
# El paciente [surrogate name] (DNI [surrogate]), nacido el [date], ...

For German, just switch the code:

befund = "Patientin Anna Müller, geb. 11.04.1979, Steuer-ID 12 345 678 901."
result = openmed.deidentify(befund, lang="de", method="replace")

Workflow

1. **Confirm the language is supported** by checking `SUPPORTED_LANGUAGES`. 2. **Pass `lang=`** to `deidentify`/`extract_pii`. This selects the language-specific model (via `get_pii_models_by_language`) *and* the regex pattern set (via `get_patterns_for_language`) for national IDs and formats. 3. **Set `locale=` for surrogates** when `method="replace"`. If omitted, the locale is derived from `lang` through `LANG_TO_LOCALE` (e.g. `pt`→`pt_PT`). Override for regional variants (`pt_BR`, `en_GB`, Gulf/Levant Arabic). 4. **Let accent normalization happen.** For models trained on accent-free text (Spanish), `deidentify` auto-strips diacritics before inference and maps spans back to the *original* accented text. You normally do not set `normalize_accents` yourself. 5. **Keep surrogates stable** across a document with `consistent=True, seed=...`.

Language-specific national IDs

The pattern sets encode and the validators check real national identifier formats and checksums, so structured IDs are caught even when the model is unsure. Examples available in `openmed.core.pii_i18n`:

| Language | Identifier | Validator | | --- | --- | --- | | French | NIR / INSEE | `validate_french_nir` | | German | Steuer-ID | `validate_german_steuer_id` | | Italian | Codice Fiscale | `validate_italian_codice_fiscale` | | Spanish | DNI / NIE | `validate_spanish_dni`, `validate_spanish_nie` | | Dutch | BSN | `validate_dutch_bsn` | | Hindi | Aadhaar | `validate_aadhaar` | | Portuguese | CPF / CNPJ | `validate_portuguese_cpf`, `validate_portuguese_cnpj` | | Turkish | TCKN | `validate_turkish_tckn` |

These map to OpenMed `CANONICAL_LABELS` (`ID_NUM`, `SSN`) and are redacted by the same policy actions as any other identifier.

Hand-off to / from OpenMed

• **Core de-id:** `deidentifying-clinical-text` — methods, thresholds,

`keep_mapping`, policies (all accept `lang`/`locale`).

• **Surrogates:** `generating-synthetic-surrogates` — locale-native fakes and

custom providers per language.

• **Policies:** `configuring-privacy-policies` — `policy=` works with any `lang`.
• **Audit:** `auditing-deidentification-runs` records the model and language in

the no-PHI report.

• **Other surfaces:** MCP `openmed_deidentify` / `openmed_list_pii_languages`;

REST `POST /pii/deidentify` (both take a language parameter).

Edge cases & gotchas

• **Never run the English model on other languages.** Recall collapses. Always

pass `lang=`; the default model is English only.

• **`lang` ≠ `locale`.** `lang` picks the *detection* model and patterns;

`locale` shapes the *replacement* fakes. Set both when surrogate realism matters (e.g. `lang="pt", locale="pt_BR"`).

• **Some locales are approximations.** Faker has no Telugu locale, so OpenMed

maps `te`→`en_IN` and warns once. Override `locale=` if you need closer regional surrogates.

• **Date order varies.** Day-first languages (fr, de, it, es, nl, pt, …) parse

`11/04/1979` as 11 April; the date logic is language-aware — keep `lang` set when shifting dates.

• **Mixed-lan