/cellxgene-census

name: cellxgene-census
description: Query the CZ CELLxGENE Census programmatically for versioned public single-cell and spatial transcriptomics data. Use when you need population-scale cell metadata, gene expression slices, Census summary counts, source H5AD URIs/downloads, embeddings, spatial Census data, or reference atlas comparisons across organisms, tissues, diseases, assays, and cell types. For analyzing your own local single-cell data use scanpy, anndata, or scvi-tools.
allowed-tools: Read Write Edit Bash
license: MIT
compatibility: Requires Python >=3.10,<3.13. Examples target cellxgene-census 1.17.x and the 2025-11-08 stable LTS Census; spatial workflows need the spatial extra and TileDB-SOMA >=1.15.5. No authentication is required for public Census data.
metadata:
  version: "1.3"
  skill-author: K-Dense Inc.

CZ CELLxGENE Census

Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.

The Census includes:

• **217+ million total cells** and **125+ million unique cells** in the 2025-11-08 stable LTS release
• **1,845 datasets** in the 2025-11-08 stable LTS release
• **Human, mouse, marmoset, rhesus macaque, and chimpanzee** data in the current schema
• **Standardized metadata** (cell types, tissues, diseases, donors)
• **Raw gene expression** matrices and source H5AD lookup/download helpers
• **Pre-calculated summary counts, embeddings, and spatial data**
• **Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools**

When to Use This Skill

This skill should be used when:

• Querying single-cell expression data by cell type, tissue, or disease
• Exploring available single-cell datasets and metadata
• Training machine learning models on single-cell data
• Performing large-scale cross-dataset analyses
• Integrating Census data with scanpy or other analysis frameworks
• Computing statistics across millions of cells
• Accessing pre-calculated embeddings or model predictions

Installation and Setup

Install the Census API:

uv pip install "cellxgene-census==1.17.*"

For spatial workflows:

uv pip install "cellxgene-census[spatial]==1.17.*" "spatialdata[extra]>=0.2.5"

For PyTorch model training, use TileDB-SOMA-ML. The old `cellxgene_census.experimental.ml` loaders are deprecated:

uv pip install "cellxgene-census==1.17.*" tiledbsoma-ml

Core Workflow Patterns

Eight patterns, each with code, are in [references/core_workflow_patterns.md](references/core_workflow_patterns.md):

1. **Opening the Census** — always pin `census_version` so an analysis stays reproducible. 2. **Exploring Census information** — available datasets, cell counts, and summary tables. 3. **Querying expression data** — small to medium scale into an `AnnData`. 4. **Large-scale queries** — out-of-core processing when the slice will not fit in memory. 5. **Machine learning with PyTorch** — the Census data loaders. 6. **Spatial Census data** — accessing spatial assays. 7. **Integration with Scanpy** — handing a Census slice to a standard Scanpy workflow. 8. **Multi-dataset integration** — combining datasets and handling batch effects.

Key Concepts and Best Practices

Always Filter for Primary Data

Unless analyzing duplicates, always include `is_primary_data == True` in queries to avoid counting cells multiple times:

obs_value_filter="cell_type == 'B cell' and is_primary_data == True"

Specify Census Version for Reproducibility

Always specify the Census version in production analyses:

census = cellxgene_census.open_soma(census_version="2025-11-08")

Estimate Query Size Before Loading

For large queries, first check the number of cells to avoid memory issues:

# Get cell count
metadata = cellxgene_census.get_obs(
    census, "homo_sapiens",
    value_filter="tissue_general == 'brain' and is_primary_data == True",
    column_names=["soma_joinid"]
)
n_cells = len(metadata)
print(f"Query will return {n_cells:,} cells")

# If too large (>100k), use out-of-core processing

Use tissue_general for Broader Groupings

The `tissue_general` field provides coarser categories than `tissue`, useful for cross-tissue analyses:

# Broader grouping
obs_value_filter="tissue_general == 'immune system'"

# Specific tissue
obs_value_filter="tissue == 'peripheral blood mononuclear cell'"

Select Only Needed Columns

Minimize data transfer by specifying only required metadata columns:

obs_column_names=["cell_type", "tissue_general", "disease"]  # Not all columns

Check Dataset Presence for Gene-Specific Queries

When analyzing specific genes, verify which datasets measured them:

presence = cellxgene_census.get_presence_matrix(
    census,
    "homo_sapiens",
    var_value_filter="feature_name in ['CD4', 'CD8A']"
)

Two-Step Workflow: Explore Then Query

First explore metadata to understand available data, then query expression:

# Step 1: Explore what's available
metadata = cellxgene_census.get_obs(
    census, "homo_sapiens",
    value_filter="disease == 'COVID-19' and is_primary_data == True",
    column_names=["cell_type", "tissue_general"]
)
print(metadata.value_counts())

# Step 2: Query based on findings
adata = cellxgene_census.get_anndata(
    census=census,
    organism="Homo sapiens",
    obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True",
)

Available Metadata Fields

Cell Metadata (obs)

Key fields for filtering:

• `cell_type`, `cell_type_ontology_term_id`
• `tissue`, `tissue_general`, `tissue_ontology_term_id`
• `disease`, `disease_ontology_term_id`
• `assay`, `assay_ontology_term_id`
• `donor_id`, `sex`, `self_reported_ethn