{"id":"anndata","name":"anndata","summary":"単一セル解析における注釈付き行列のデータ構造。.h5adファイルを扱う場合やscverseエコシステムとの統合時に使います。","body":"# AnnData\n\n## Overview\n\nAnnData is a Python package for handling annotated data matrices, storing experimental measurements (X) alongside observation metadata (obs), variable metadata (var), and multi-dimensional annotations (obsm, varm, obsp, varp, uns). Originally designed for single-cell genomics through Scanpy, it now serves as a general-purpose framework for any annotated data requiring efficient storage, manipulation, and analysis.\n\n## When to Use This Skill\n\nUse this skill when:\n- Creating, reading, or writing AnnData objects\n- Working with h5ad, zarr, or other genomics data formats\n- Performing single-cell RNA-seq analysis\n- Managing large datasets with sparse matrices or backed mode\n- Concatenating multiple datasets or experimental batches\n- Subsetting, filtering, or transforming annotated data\n- Integrating with scanpy, scvi-tools, or other scverse ecosystem tools\n\n## Installation\n\nRequires Python 3.11+. Current stable release: 0.12.16 (released 2026-05-18).\n\n```bash\nuv pip install \"anndata==0.12.16\"\n\n# Lazy I/O and dask-backed operations\nuv pip install \"anndata[dask,lazy]==0.12.16\"\n\n# Development / docs (contributors)\nuv pip install \"anndata[dev,test,doc]==0.12.16\"\n```\n\nUse unpinned installs only when intentionally tracking the latest compatible release.\n\nCurrent API notes:\n- Use `anndata.io` for non-native `read_*` and `write_*` helpers. Top-level `anndata.read_h5ad` and `anndata.read_zarr` remain supported.\n- Avoid deprecated APIs: `ad.read`, `AnnData.concatenate()`, `AnnData.*_keys()`, and `anndata.__version__`. Prefer `ad.read_h5ad`, `ad.concat`, mapping `.keys()`, and `importlib.metadata.version(\"anndata\")`.\n- Treat `anndata.experimental` APIs as useful but unstable. Prefer them for large-data workflows only when their current caveats are acceptable.\n\n## Quick Start\n\n### Creating an AnnData object\n```python\nimport anndata as ad\nimport numpy as np\nimport pandas as pd\n\n# Minimal creation\nX = np.random.rand(100, 2000)  # 100 cells × 2000 genes\nadata = ad.AnnData(X)\n\n# With metadata\nobs = pd.DataFrame({\n    'cell_type': ['T cell', 'B cell'] * 50,\n    'sample': ['A', 'B'] * 50\n}, index=[f'cell_{i}' for i in range(100)])\n\nvar = pd.DataFrame({\n    'gene_name': [f'Gene_{i}' for i in range(2000)]\n}, index=[f'ENSG{i:05d}' for i in range(2000)])\n\nadata = ad.AnnData(X=X, obs=obs, var=var)\n```\n\n### Reading data\n```python\n# Native formats (read_h5ad/read_zarr remain at top-level)\nadata = ad.read_h5ad('data.h5ad')\nadata = ad.read_h5ad('large_data.h5ad', backed='r')  # lazy load for large files\nadata = ad.read_zarr('data.zarr')\n\n# Other formats: prefer anndata.io (top-level imports are deprecated)\nfrom anndata.io import read_csv, read_loom, read_mtx\n\nadata = read_csv('data.csv')\nadata = read_loom('data.loom')\n\n# 10X Genomics: use scanpy (not anndata) — see scanpy skill\nimport scanpy as sc\nadata = sc.read_10x_h5('filtered_feature_bc_matrix.h5')\nadata = sc.read_10x_mtx('filtered_feature_bc_matrix/')\n```\n\n### Writing data\n```python\n# Write h5ad file\nadata.write_h5ad('output.h5ad')\n\n# Write with compression\nadata.write_h5ad('output.h5ad', compression='gzip')\n\n# Write other formats\nadata.write_zarr('output.zarr')\nadata.write_csvs('output_dir/')\n```\n\n### Basic operations\n```python\n# Subset by conditions\nt_cells = adata[adata.obs['cell_type'] == 'T cell']\n\n# Subset by indices\nsubset = adata[0:50, 0:100]\n\n# Add metadata\nadata.obs['quality_score'] = np.random.rand(adata.n_obs)\nadata.var['highly_variable'] = np.random.rand(adata.n_vars) > 0.8\n\n# Access dimensions\nprint(f\"{adata.n_obs} observations × {adata.n_vars} variables\")\n```\n\n## Core Capabilities\n\n### 1. Data Structure\n\nUnderstand the AnnData object structure including X, obs, var, layers, obsm, varm, obsp, varp, uns, and raw components.\n\n**See**: `references/data_structure.md` for comprehensive information on:\n- Core components (X, obs, var, layers, obsm, varm, obsp, varp, uns, raw)\n- Creating AnnData objects from various sources\n- Accessing and manipulating data components\n- Memory-efficient practices\n\n### 2. Input/Output Operations\n\nRead and write data in various formats with support for compression, backed mode, and cloud storage.\n\n**See**: `references/io_operations.md` for details on:\n- Native formats (h5ad, zarr)\n- Alternative formats (CSV, MTX, Loom, 10X, Excel)\n- Backed mode for large datasets\n- Remote data access\n- Format conversion\n- Performance optimization\n\nCommon commands:\n```python\nfrom anndata.io import read_mtx\n\n# Read/write h5ad\nadata = ad.read_h5ad('data.h5ad', backed='r')\nadata.write_h5ad('output.h5ad', compression='gzip')\n\n# 10X Genomics (via scanpy)\nimport scanpy as sc\nadata = sc.read_10x_h5('filtered_feature_bc_matrix.h5')\n\n# Read MTX format\nadata = read_mtx('matrix.mtx').T\n```\n\n### 3. Concatenation\n\nCombine multiple AnnData objects along observations or variables with flexible join strategies.\n\n**See**: `references/concatenation.md` for comprehensive coverage of:\n- Basic concatenation (axis=0 for observations, axis=1 for variables)\n- Join types (inner, outer)\n- Merge strategies (same, unique, first, only)\n- Tracking data sources with labels\n- Lazy concatenation (AnnCollection)\n- On-disk concatenation for large datasets\n\nCommon commands:\n```python\n# Concatenate observations (combine samples)\nadata = ad.concat(\n    [adata1, adata2, adata3],\n    axis=0,\n    join='inner',\n    label='batch',\n    keys=['batch1', 'batch2', 'batch3']\n)\n\n# Concatenate variables (combine modalities)\nadata = ad.concat([adata_rna, adata_protein], axis=1)\n\n# Lazy collection over backed AnnData objects (experimental)\nfrom anndata.experimental import AnnCollection\n\nbacked_adatas = [\n    ad.read_h5ad(path, backed='r')\n    for path in ['data1.h5ad', 'data2.h5ad']\n]\ncollection = AnnCollection(\n    backed_adatas,\n    join_obs='outer',\n    join_vars='inner',\n    label='dataset'\n)\n```\n\n### 4. Data Manipulation\n\nTransform, subset, filter, and reorganize data efficiently.\n\n**See**: `references/manipulation.md` for detailed guidance on:\n- Subsetting (by indices, names, boolean masks, metadata conditions)\n- Transposition\n- Copying (full copies vs views)\n- Renaming (observations, variables, categories)\n- Type conversions (strings to categoricals, sparse/dense)\n- Adding/removing data components\n- Reordering\n- Quality control filtering\n\nCommon commands:\n```python\n# Subset by metadata\nfiltered = adata[adata.obs['quality_score'] > 0.8]\nhv_genes = adata[:, adata.var['highly_variable']]\n\n# Transpose\nadata_T = adata.T\n\n# Copy vs view\nview = adata[0:100, :]  # View (lightweight reference)\ncopy = adata[0:100, :].copy()  # Independent copy\n\n# Convert strings to categoricals\nadata.strings_to_categoricals()\n```\n\n### 5. Best Practices\n\nFollow recommended patterns for memory efficiency, performance, and reproducibility.\n\n**See**: `references/best_practices.md` for guidelines on:\n- Memory management (sparse matrices, categoricals, backed mode)\n- Views vs copies\n- Data storage optimization\n- Performance optimization\n- Working with raw data\n- Metadata management\n- Reproducibility\n- Error handling\n- Integration with other tools\n- Common pitfalls and solutions\n\nKey recommendations:\n```python\n# Use sparse matrices for sparse data\nfrom scipy.sparse import csr_matrix\nadata.X = csr_matrix(adata.X)\n\n# Convert strings to categoricals\nadata.strings_to_categoricals()\n\n# Use backed mode for large files\nadata = ad.read_h5ad('large.h5ad', backed='r')\n\n# Store raw before filtering\nadata.raw = adata.copy()\nadata = adata[:, adata.var['highly_variable']]\n```\n\n## Integration with Scverse Ecosystem\n\nAnnData serves as the foundational data structure for the scverse ecosystem:\n\n### Scanpy (Single-cell analysis)\n```python\nimport scanpy as sc\n\n# Preprocessing\nsc.pp.filter_cells(adata, min_genes=200)\nsc.pp.normalize_total(adata, target_sum=1e4)\nsc.pp.log1p(adata)\nsc.pp.highly_variable_genes(adata, n_top_genes=2000)\n\n# Dimensionality reduction\nsc.pp.pca(adata, n_comps=50)\nsc.pp.neighbors(adata, n_neighbors=15)\nsc.tl.umap(adata)\nsc.tl.leiden(adata)\n\n# Visualization\nsc.pl.umap(adata, color=['cell_type', 'leiden'])\n```\n\n### Muon (Multimodal data)\n```python\nimport muon as mu\n\n# Combine RNA and protein data\nmdata = mu.MuData({'rna': adata_rna, 'protein': adata_protein})\n```\n\n### PyTorch integration\n```python\nfrom anndata.experimental import AnnLoader\n\n# Create DataLoader for deep learning\ndataloader = AnnLoader(adata, batch_size=128, shuffle=True)\n\nfor batch in dataloader:\n    X = batch.X\n    # Train model\n```\n\n## Common Workflows\n\n### Single-cell RNA-seq analysis\n```python\nimport anndata as ad\nimport scanpy as sc\n\n# 1. Load data (10X via scanpy; anndata handles h5ad/zarr natively)\nadata = sc.read_10x_h5('filtered_feature_bc_matrix.h5')\n\n# 2. Quality control\nadata.obs['n_genes'] = (adata.X > 0).sum(axis=1)\nadata.obs['n_counts'] = adata.X.sum(axis=1)\nadata = adata[adata.obs['n_genes'] > 200]\nadata = adata[adata.obs['n_counts'] < 50000]\n\n# 3. Store raw\nadata.raw = adata.copy()\n\n# 4. Normalize and filter\nsc.pp.normalize_total(adata, target_sum=1e4)\nsc.pp.log1p(adata)\nsc.pp.highly_variable_genes(adata, n_top_genes=2000)\nadata = adata[:, adata.var['highly_variable']]\n\n# 5. Save processed data\nadata.write_h5ad('processed.h5ad')\n```\n\n### Batch integration\n```python\n# Load multiple batches\nadata1 = ad.read_h5ad('batch1.h5ad')\nadata2 = ad.read_h5ad('batch2.h5ad')\nadata3 = ad.read_h5ad('batch3.h5ad')\n\n# Concatenate with batch labels\nadata = ad.concat(\n    [adata1, adata2, adata3],\n    label='batch',\n    keys=['batch1', 'batch2', 'batch3'],\n    join='inner'\n)\n\n# Apply batch correction\nimport scanpy as sc\nsc.pp.combat(adata, key='batch')\n\n# Continue analysis\nsc.pp.pca(adata)\nsc.pp.neighbors(adata)\nsc.tl.umap(adata)\n```\n\n### Working with large datasets\n```python\n# Open in backed mode\nadata = ad.read_h5ad('100GB_dataset.h5ad', backed='r')\n\n# Filter based on metadata (no data loading)\nhigh_quality = adata[adata.obs['quality_score'] > 0.8]\n\n# Load filtered subset\nadata_subset = high_quality.to_memory()\n\n# Process subset\nprocess(adata_subset)\n\n# Or process in chunks\nchunk_size = 1000\nfor i in range(0, adata.n_obs, chunk_size):\n    chunk = adata[i:i+chunk_size, :].to_memory()\n    process(chunk)\n```\n\n## Troubleshooting\n\n### Out of memory errors\nUse backed mode or convert to sparse matrices:\n```python\n# Backed mode\nadata = ad.read_h5ad('file.h5ad', backed='r')\n\n# Sparse matrices\nfrom scipy.sparse import csr_matrix\nadata.X = csr_matrix(adata.X)\n```\n\n### Slow file reading\nUse compression and appropriate formats:\n```python\n# Optimize for storage\nadata.strings_to_categoricals()\nadata.write_h5ad('file.h5ad', compression='gzip')\n\n# Use Zarr for cloud storage; v3 writes are opt-in in anndata 0.12\nimport anndata as ad\n\nad.settings.zarr_write_format = 3\nad.settings.auto_shard_zarr_v3 = True  # experimental; independent of zarr_write_format\nadata.write_zarr('file.zarr', chunks=(1000, 1000))\n```\n\n### Index alignment issues\nAlways align external data on index:\n```python\n# Wrong\nadata.obs['new_col'] = external_data['values']\n\n# Correct\nadata.obs['new_col'] = external_data.set_index('cell_id').loc[adata.obs_names, 'values']\n```\n\n## Additional Resources\n\n- **Official documentation**: https://anndata.readthedocs.io/\n- **Scanpy tutorials**: https://scanpy.readthedocs.io/\n- **Scverse ecosystem**: https://scverse.org/\n- **GitHub repository**: https://github.com/scverse/anndata","author":"@K-Dense-AI","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/K-Dense-AI/scientific-agent-skills/tree/main/skills/anndata","license":"MIT","category":"writing","lang":"en","tokens":3098,"stars":0,"calls30d":1,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/best_practices.md","size":12643,"sha256":"0fc0cbfc4159febf3d1fe1d0d3b3511ee19278979c0ad7b162ba1a3685a42f0c"},{"path":"references/concatenation.md","size":10699,"sha256":"71dd714baa2736e4baaa1a4703a8d30de5dfe246f35275811005fbefe6b2c30e"},{"path":"references/data_structure.md","size":8744,"sha256":"41d00139454a6d5fa5f8071583d9074effd1af9ff6658034274398939ca7fb3d"},{"path":"references/io_operations.md","size":11989,"sha256":"300e15e8c97cf98af9d79acc71d41d04a549f1b8c4dac4bb1669bcb5ddbf2747"},{"path":"references/manipulation.md","size":11848,"sha256":"279a3b9c42760ec8fd1c3d19e3cb87692bd09920695498d3ee1bea9054385d66"}],"requires":{"mcp":[],"tools":["Read Write Edit Bash"]},"safety":{"flags":[],"scannedAt":"2026-08-22","hasScripts":false,"networkEndpoints":["anndata.readthedocs.io","example.org","scanpy.readthedocs.io","scverse.org"]}}