/cellwhisperer

name: cellwhisperer
description: |
  Use when the user needs to work with single-cell RNA-seq data using CellWhisperer: processing datasets for the interactive web app, scoring/annotating cells' types and states with free-text queries, or loading CellWhisperer as a Python library for local inference. Triggers on: "cellwhisperer", "cell type annotation", "scRNA-seq analysis", "single-cell scoring", "cellxgene", or any request involving transcriptome-to-text similarity scoring (e.g. when the user asks for specific attributes/properties of cells, including their type and state).

CellWhisperer

CellWhisperer is a multimodal AI model combining transcriptomics with natural language to enable intuitive interaction with scRNA-seq datasets. Published in [Nature Biotechnology](https://doi.org/10.1038/s41587-025-02857-9).

This skill provides three capabilities:

1. **End-to-end analysis** — Prepare an h5ad dataset, process it through the CellWhisperer pipeline, and launch the interactive cellxgene web app. 2. **API-based cell scoring** — Query the hosted CellWhisperer API at `cellwhisperer.bocklab.org` to embed texts and score/annotate cells on demand, without local model installation. 3. **Local library usage** — Install CellWhisperer as a Python library and use it programmatically for model loading, embedding, and scoring.

Installation (for Claude Code users)

# generally prevent auto-update for your safety
claude plugin marketplace add epigen/cellwhisperer@v0.1.0
claude plugin install cellwhisperer@cellwhisperer

After installing, restart Claude Code or run `/reload-plugins`. The skill becomes available as `/cellwhisperer` or is invoked automatically when CellWhisperer-related tasks are detected.

Project setup

CellWhisperer uses [pixi](https://pixi.sh) for environment management.

git clone git@github.com:epigen/cellwhisperer.git --recurse-submodules
cd cellwhisperer

All commands below should be run from the CellWhisperer project root using `pixi run`.

Before starting, read the project README for full context:

• `README.md` — installation, dataset format, web app launch, paper reproduction

---

Feature 1: End-to-End scRNA-seq Analysis

Goal: take a user's h5ad file from raw counts to an interactive CellWhisperer-powered cellxgene browser.

Step 1: Prepare the dataset

Place the h5ad file at `resources/<dataset_name>/read_count_table.h5ad`.

**Requirements** (validate before proceeding):

• Raw integer read counts in `.X` or `.layers["counts"]` (int32, no NaN)
• `.var` must have a unique index and a `gene_name` column with gene symbols
• Recommended: provide `ensembl_id` in `.var` (computed if missing)
• Recommended: filter cells with <100 genes expressed
• Use `categorical` dtype for categorical `.obs` columns
• 2D embeddings in `.obsm` must be `np.ndarray` (not DataFrame), dtype float/int, shape `(n_obs, >=2)`, no Inf values

Write a validation script if the user's data needs checking. Common issues:

• Normalized counts instead of raw → check `.layers["counts"]`
• Gene symbols in index but no `gene_name` column → copy index to `gene_name`
• Object dtype obs columns → convert to categorical

Step 2: Run the processing pipeline

cd src/cellxgene_preprocessing
pixi run snakemake --cores 8 --config 'datasets=["<dataset_name>"]'

Key notes:

• GPU accelerates processing (4GB VRAM sufficient). Set `CUDA_VISIBLE_DEVICES` to select GPU.
• Without GPU, increase `--cores` (e.g. 32).
• Memory: allocate ~2x the h5ad file size.
• Cluster captions use GPT-4 API by default (`OPENAI_API_KEY` env var). Without it, falls back to a local Mixtral model (requires 40GB VRAM GPU).
• Output lands in `results/<dataset_name>/`.

Step 3: Launch cellxgene

pixi run cellxgene launch -p 5005 --host 0.0.0.0 --max-category-items 500 \
  --var-names gene_name \
  results/<dataset_name>/cellwhisperer_clip_v1/cellxgene.h5ad

Access at `http://localhost:5005`. The web app connects to the hosted CellWhisperer API at `cellwhisperer.bocklab.org` for AI features (search, chat).

To self-host the embedding model (4GB VRAM), add:

--cellwhisperer-clip-model results/models/jointemb/cellwhisperer_clip_v1.ckpt

---

Feature 2: API-Based Cell Scoring

Use the hosted CellWhisperer API to embed text queries and score cells without installing the full model locally. This is useful when an agent or script needs quick cell-type annotations or text-transcriptome similarity scores.

API endpoints

Base URL: `https://cellwhisperer.bocklab.org/clip/api`

Get logit scale (learned CLIP temperature)

import requests
response = requests.get("https://cellwhisperer.bocklab.org/clip/api/logit_scale")
logit_scale = float(response.content)

Embed text queries

import pickle
import torch
import requests

texts = ["T cell", "B cell", "monocyte"]
response = requests.post(
    "https://cellwhisperer.bocklab.org/clip/api/text_embedding",
    json=texts,
)
text_embeds = torch.from_numpy(pickle.loads(response.content))
# Shape: (len(texts), embedding_dim)

Score cells against text

Once you have text embeddings and precomputed transcriptome embeddings (from `adata.obsm["transcriptome_embeds"]` in a processed dataset), compute similarity:

import torch

# text_embeds: (n_texts, embedding_dim) from API
# transcriptome_embeds: (n_cells, embedding_dim) from adata.obsm["transcriptome_embeds"]
transcriptome_embeds = torch.from_numpy(adata.obsm["transcriptome_embeds"])

scores = torch.matmul(text_embeds, transcriptome_embeds.t()) * logit_scale
# Shape: (n_texts, n_cells) - higher score = stronger match

Standalone scoring recipe (no CellWhisperer install needed)

For quick annotation of cells that already have precomputed transcriptome embeddings:

import pickle
import requests
import torch
import numpy as np
import anndata

# Load a CellWhisperer-processed dataset
adata = anndata.read_h5ad("results/<d