{"id":"cellxgene-census","name":"cellxgene-census","summary":"CZ CELLxGENEセンサスをプログラム的に検索し、バージョン管理された公開単細胞および空間トランスクリプトミクスデータを検索できます。","body":"# CZ CELLxGENE Census\n\n## Overview\n\nThe CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.\n\nThe Census includes:\n- **217+ million total cells** and **125+ million unique cells** in the 2025-11-08 stable LTS release\n- **1,845 datasets** in the 2025-11-08 stable LTS release\n- **Human, mouse, marmoset, rhesus macaque, and chimpanzee** data in the current schema\n- **Standardized metadata** (cell types, tissues, diseases, donors)\n- **Raw gene expression** matrices and source H5AD lookup/download helpers\n- **Pre-calculated summary counts, embeddings, and spatial data**\n- **Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools**\n\n## When to Use This Skill\n\nThis skill should be used when:\n- Querying single-cell expression data by cell type, tissue, or disease\n- Exploring available single-cell datasets and metadata\n- Training machine learning models on single-cell data\n- Performing large-scale cross-dataset analyses\n- Integrating Census data with scanpy or other analysis frameworks\n- Computing statistics across millions of cells\n- Accessing pre-calculated embeddings or model predictions\n\n## Installation and Setup\n\nInstall the Census API:\n```bash\nuv pip install \"cellxgene-census==1.17.*\"\n```\n\nFor spatial workflows:\n```bash\nuv pip install \"cellxgene-census[spatial]==1.17.*\" \"spatialdata[extra]>=0.2.5\"\n```\n\nFor PyTorch model training, use TileDB-SOMA-ML. The old `cellxgene_census.experimental.ml` loaders are deprecated:\n\n```bash\nuv pip install \"cellxgene-census==1.17.*\" tiledbsoma-ml\n```\n\n## Core Workflow Patterns\n\nEight patterns, each with code, are in\n[references/core_workflow_patterns.md](references/core_workflow_patterns.md):\n\n1. **Opening the Census** — always pin `census_version` so an analysis stays reproducible.\n2. **Exploring Census information** — available datasets, cell counts, and summary tables.\n3. **Querying expression data** — small to medium scale into an `AnnData`.\n4. **Large-scale queries** — out-of-core processing when the slice will not fit in memory.\n5. **Machine learning with PyTorch** — the Census data loaders.\n6. **Spatial Census data** — accessing spatial assays.\n7. **Integration with Scanpy** — handing a Census slice to a standard Scanpy workflow.\n8. **Multi-dataset integration** — combining datasets and handling batch effects.\n\n## Key Concepts and Best Practices\n\n### Always Filter for Primary Data\nUnless analyzing duplicates, always include `is_primary_data == True` in queries to avoid counting cells multiple times:\n```python\nobs_value_filter=\"cell_type == 'B cell' and is_primary_data == True\"\n```\n\n### Specify Census Version for Reproducibility\nAlways specify the Census version in production analyses:\n```python\ncensus = cellxgene_census.open_soma(census_version=\"2025-11-08\")\n```\n\n### Estimate Query Size Before Loading\nFor large queries, first check the number of cells to avoid memory issues:\n```python\n# Get cell count\nmetadata = cellxgene_census.get_obs(\n    census, \"homo_sapiens\",\n    value_filter=\"tissue_general == 'brain' and is_primary_data == True\",\n    column_names=[\"soma_joinid\"]\n)\nn_cells = len(metadata)\nprint(f\"Query will return {n_cells:,} cells\")\n\n# If too large (>100k), use out-of-core processing\n```\n\n### Use tissue_general for Broader Groupings\nThe `tissue_general` field provides coarser categories than `tissue`, useful for cross-tissue analyses:\n```python\n# Broader grouping\nobs_value_filter=\"tissue_general == 'immune system'\"\n\n# Specific tissue\nobs_value_filter=\"tissue == 'peripheral blood mononuclear cell'\"\n```\n\n### Select Only Needed Columns\nMinimize data transfer by specifying only required metadata columns:\n```python\nobs_column_names=[\"cell_type\", \"tissue_general\", \"disease\"]  # Not all columns\n```\n\n### Check Dataset Presence for Gene-Specific Queries\nWhen analyzing specific genes, verify which datasets measured them:\n```python\npresence = cellxgene_census.get_presence_matrix(\n    census,\n    \"homo_sapiens\",\n    var_value_filter=\"feature_name in ['CD4', 'CD8A']\"\n)\n```\n\n### Two-Step Workflow: Explore Then Query\nFirst explore metadata to understand available data, then query expression:\n```python\n# Step 1: Explore what's available\nmetadata = cellxgene_census.get_obs(\n    census, \"homo_sapiens\",\n    value_filter=\"disease == 'COVID-19' and is_primary_data == True\",\n    column_names=[\"cell_type\", \"tissue_general\"]\n)\nprint(metadata.value_counts())\n\n# Step 2: Query based on findings\nadata = cellxgene_census.get_anndata(\n    census=census,\n    organism=\"Homo sapiens\",\n    obs_value_filter=\"disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True\",\n)\n```\n\n## Available Metadata Fields\n\n### Cell Metadata (obs)\nKey fields for filtering:\n- `cell_type`, `cell_type_ontology_term_id`\n- `tissue`, `tissue_general`, `tissue_ontology_term_id`\n- `disease`, `disease_ontology_term_id`\n- `assay`, `assay_ontology_term_id`\n- `donor_id`, `sex`, `self_reported_ethnicity`\n- `development_stage`, `development_stage_ontology_term_id`\n- `dataset_id`\n- `is_primary_data` (Boolean: True = unique cell)\n\nThe current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with `list(census[\"census_data\"].keys())`.\n\n### Gene Metadata (var)\n- `feature_id` (Ensembl gene ID, e.g., \"ENSG00000161798\")\n- `feature_name` (Gene symbol, e.g., \"FOXP2\")\n- `feature_type`\n- `feature_length` (Gene length in base pairs)\n- `nnz`, `n_measured_obs` (availability summaries useful for checking sparsity and coverage)\n\n## Reference Documentation\n\nThis skill includes detailed reference documentation:\n\n### references/census_schema.md\nComprehensive documentation of:\n- Census data structure and organization\n- All available metadata fields\n- Value filter syntax and operators\n- SOMA object types\n- Data inclusion criteria\n\n**When to read:** When you need detailed schema information, full list of metadata fields, or complex filter syntax.\n\n### references/common_patterns.md\nExamples and patterns for:\n- Exploratory queries (metadata only)\n- Small-to-medium queries (AnnData)\n- Large queries (out-of-core processing)\n- PyTorch integration\n- Spatial Census access patterns\n- Scanpy integration workflows\n- Multi-dataset integration\n- Best practices and common pitfalls\n\n**When to read:** When implementing specific query patterns, looking for code examples, or troubleshooting common issues.\n\n## Common Use Cases\n\n### Use Case 1: Explore Cell Types in a Tissue\n```python\nwith cellxgene_census.open_soma() as census:\n    cells = cellxgene_census.get_obs(\n        census, \"homo_sapiens\",\n        value_filter=\"tissue_general == 'lung' and is_primary_data == True\",\n        column_names=[\"cell_type\"]\n    )\n    print(cells[\"cell_type\"].value_counts())\n```\n\n### Use Case 2: Query Marker Gene Expression\n```python\nwith cellxgene_census.open_soma() as census:\n    adata = cellxgene_census.get_anndata(\n        census=census,\n        organism=\"Homo sapiens\",\n        var_value_filter=\"feature_name in ['CD4', 'CD8A', 'CD19']\",\n        obs_value_filter=\"cell_type in ['T cell', 'B cell'] and is_primary_data == True\",\n    )\n```\n\n### Use Case 3: Train Cell Type Classifier\n```python\nimport tiledbsoma as soma\nfrom tiledbsoma_ml import ExperimentDataset, experiment_dataloader\n\nwith cellxgene_census.open_soma() as census:\n    experiment = census[\"census_data\"][\"homo_sapiens\"]\n    with experiment.axis_query(\n        measurement_name=\"RNA\",\n        obs_query=soma.AxisQuery(value_filter=\"is_primary_data == True\"),\n    ) as query:\n        dataset = ExperimentDataset(\n            query=query,\n            layer_name=\"raw\",\n            obs_column_names=[\"cell_type\"],\n            batch_size=128,\n            shuffle=True,\n        )\n        dataloader = experiment_dataloader(dataset)\n\n        for X, obs in dataloader:\n            labels = obs[\"cell_type\"]\n            # Training logic\n            pass\n```\n\n### Use Case 4: Cross-Tissue Analysis\n```python\nwith cellxgene_census.open_soma() as census:\n    adata = cellxgene_census.get_anndata(\n        census=census,\n        organism=\"Homo sapiens\",\n        obs_value_filter=\"cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True\",\n    )\n\n    # Analyze macrophage differences across tissues\n    sc.tl.rank_genes_groups(adata, groupby=\"tissue_general\")\n```\n\n## Troubleshooting\n\n### Query Returns Too Many Cells\n- Add more specific filters to reduce scope\n- Use `tissue` instead of `tissue_general` for finer granularity\n- Filter by specific `dataset_id` if known\n- Switch to out-of-core processing for large queries\n\n### Memory Errors\n- Reduce query scope with more restrictive filters\n- Select fewer genes with `var_value_filter`\n- Use out-of-core processing with `axis_query()`\n- Process data in batches\n\n### Duplicate Cells in Results\n- Always include `is_primary_data == True` in filters\n- Check if intentionally querying across multiple datasets\n\n### Gene Not Found\n- Verify gene name spelling (case-sensitive)\n- Try Ensembl ID with `feature_id` instead of `feature_name`\n- Check dataset presence matrix to see if gene was measured\n- Some genes may have been filtered during Census construction\n\n### Version Inconsistencies\n- Always specify `census_version` explicitly\n- Use same version across all analyses\n- Check release notes for version-specific changes","author":"@K-Dense-AI","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/K-Dense-AI/scientific-agent-skills/tree/main/skills/cellxgene-census","license":"MIT","category":"document","lang":"en","tokens":2269,"stars":0,"calls30d":1,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/census_schema.md","size":7910,"sha256":"5869d53bf34906e9cf45d5c12ed0b5a8225d32936f5b5ece7c08e07e5e167688"},{"path":"references/common_patterns.md","size":11033,"sha256":"0801f85e8facffef18f3e8bfda2f26d1f2cae2763fe844c2531d292aa9a39928"},{"path":"references/core_workflow_patterns.md","size":9796,"sha256":"067be2dcce03ba2fbe59d25a3c0c231ef7b3abcd12cafdd0cd85dd73fc037b90"}],"requires":{"mcp":[],"tools":["Read Write Edit Bash"]},"safety":{"flags":[],"scannedAt":"2026-08-22","hasScripts":false,"networkEndpoints":[]}}