/sc-preprocessing

name: sc-preprocessing
description: Load when normalising QC'd scRNA into a PCA-ready AnnData via scanpy / Seurat / SCTransform
  / Pearson residuals. Skip when QC thresholds are still undecided (use sc-qc); batch correction across
  samples (use sc-batch-integration).
trigger: single cell preprocess, scRNA preprocessing, normalize hvg pca, base preprocessing, Seurat preprocessing, SCTransform preprocessing
tags:
- singlecell
- scrna
- preprocessing
- normalization
- hvg
- pca
- scanpy
- seurat
- sctransform
- pearson_residuals

sc-preprocessing

When to use

The user has a QC'd AnnData and wants the standard "filter → normalise → HVG → PCA" pipeline before clustering or batch integration. Four interchangeable backends are available: `scanpy` (default; CP10k log + HVG seurat flavour), `seurat` (R-backed LogNormalize / CLR / RC), `sctransform` (R-backed regularised NB), and `pearson_residuals` (raw-count HVG selection plus Pearson residual transformation). The skill stops at PCA — UMAP / clustering live in `sc-clustering`, multi-sample correction in `sc-batch-integration`.

Use from a step

pre = load_skill("sc-preprocessing")
adata = read_input("results/01_qc/intermediate/adata_qc.h5ad")
# max_mt_pct 10: PBMC cells above 10% mitochondrial reads are mostly damaged (agreed with the user).
adata = pre.preprocess(adata, method="scanpy", max_mt_pct=10.0)
write_output(pre.run_info(adata)["filter_summary"], "tables/filter_summary.json")
write_output(pre.pca_variance_table(adata), "tables/pca_variance_ratio.csv")
write_output(adata, "intermediate/adata_preprocessed.h5ad")

A complete step that runs on demo data: `examples/example_step.py`.

If the input already passed through `sc-filter`, use `preprocess(adata, apply_filters=False)`. This preserves that step's retained cells and genes, including its doublet decision, while normalization, HVG selection and PCA still run.

API

<!-- api:begin generated from _api.py; regenerate with run.py api <skill dir> --write -->

`preprocess(adata, *, method: str='scanpy', apply_filters: bool=True, min_genes: int=200, min_cells: int=3, max_mt_pct: float=20.0, n_top_hvg: int | None=None, n_pcs: int=50, normalization_target_sum: float=10000.0, scanpy_hvg_flavor: str='seurat', pearson_hvg_flavor: str='seurat_v3', pearson_theta: float=100.0, seurat_normalize_method: str='LogNormalize', seurat_scale_factor: float=10000.0, seurat_hvg_method: str='vst', sctransform_regress_mt: bool=True, remove_doublets: bool=True, doublet_score_threshold: float=0.25, preserve_var_names: bool=False)`

Filter cells and genes, normalise, select highly variable genes and run PCA.

The input is first brought into the OmicsClaw scRNA contract (counts in ``layers['counts']``, a counts snapshot in ``raw``) and QC metrics are added when missing. With apply_filters=True, cells and genes are filtered by the thresholds below, and doublets are dropped when ``predicted_doublet`` or ``doublet_score`` from sc-doublet-detection are in ``obs``. Finally the chosen method normalises ``X``, flags ``var['highly_variable']`` and writes ``obsm['X_pca']``. What was filtered is recorded for :func:`run_info`.

:param method: ``"scanpy"`` (normalize_total, log1p, HVG, PCA; the default), ``"pearson_residuals"`` (analytic Pearson residuals for HVG and PCA; ``X`` stays log-normalised), ``"seurat"`` (Seurat LogNormalize in R) or ``"sctransform"`` (Seurat SCTransform in R). :param apply_filters: Apply cell, gene and doublet filtering. Default True. Set False after sc-filter to preserve its retained cells and genes. :param min_genes: Drop cells with fewer detected genes. Default 200, the scanpy and Seurat tutorial value; lower it for low-depth data. :param min_cells: Drop genes detected in fewer cells. Default 3, the tutorial value. :param max_mt_pct: Drop cells above this mitochondrial percentage. Default 20.0, a permissive threshold; PBMC data usually uses 5 to 10. Agree it with the user. :param n_top_hvg: Number of highly variable genes. Default ``None``: 3000 for ``sctransform``, 2000 for the other methods (their tutorials' values). :param n_pcs: Principal components to compute. Default 50. :param normalization_target_sum: Counts per cell after ``normalize_total`` (``scanpy`` only). Default 10000. :param scanpy_hvg_flavor: HVG flavor for ``scanpy``: ``"seurat"`` (default), ``"cell_ranger"`` or ``"seurat_v3"``. :param pearson_hvg_flavor: HVG flavor for ``pearson_residuals``. Default ``"seurat_v3"``. :param pearson_theta: Overdispersion for Pearson residuals. Default 100, scanpy's default. :param seurat_normalize_method: Seurat ``NormalizeData`` method. Default ``"LogNormalize"``. :param seurat_scale_factor: Seurat scale factor. Default 10000. :param seurat_hvg_method: Seurat ``FindVariableFeatures`` method. Default ``"vst"``. :param sctransform_regress_mt: Regress out mitochondrial percentage in SCTransform. Default ``True``. :param remove_doublets: Drop cells flagged as doublets when the flags are present. Default ``True``. :param doublet_score_threshold: Score above which a cell counts as a doublet when only ``doublet_score`` is present. Default 0.25. :param preserve_var_names: Keep the input's gene identifiers instead of the symbols chosen during standardisation. Default ``False``. :returns: A new AnnData with normalised ``X``, ``layers['counts']``, ``raw`` counts snapshot, ``var['highly_variable']``, ``obsm['X_pca']``, ``uns['pca']``. :raises ValueError: an unknown method, or input with no count-like matrix. :raises RuntimeError: the R methods fail, or their R packages are missing.

`run_info(adata, *, keep: bool=True) -> dict`

What :func:`preprocess` recorded about its run.

Keys: ``method``; ``filter_summary`` (cells and genes before and after filtering, cells removed by each threshold under ``filter_stats``, whether QC metrics were reused, and ``input_preparation``: the matrix used as counts, gene-name source, warning