{"id":"geniml","name":"geniml","summary":"Genimlは監査された局所ゲノミック間隔ワークフローに使います。BEDやユニバース契約の検証、Region2VecやscEmbedの計画、モデル/トークナイザーの互換性の検査、コンセンサスユニバースの評価などです。","body":"# Geniml\n\nUse Geniml for machine learning and statistical workflows over genomic interval\nsets. Treat coordinates, assemblies, token vocabularies, model artifacts, and\nsample grouping as explicit contracts. The bundled scripts validate or plan;\nthey do not import Geniml, contact services, deserialize models, or execute\ntraining.\n\n`Bash` is declared only for explicit, user-approved `uv`, Python, Geniml,\nGtars, Git, and native CLI commands shown in this guide; bundled Python helpers\ndo not spawn subprocesses. Example paths under `data/`, `refs/`, `work/`, and\n`models/` are user-provided project placeholders, not missing bundled files.\n\n## Verified release snapshot\n\n- Latest stable PyPI release on 2026-07-23: `geniml==0.8.4` (2026-01-14).\n- PyPI does not declare `Requires-Python`; its classifiers list Python\n  3.10-3.14. Prefer Python 3.11 or 3.12 where all native/ML wheels resolve.\n- `geniml==0.8.4` accepts `gtars>=0.2.5`; the verified base smoke used current\n  `gtars==0.9.2` (2026-06-17, Python >=3.10).\n- Extras are `ml` and `test`. The base install omits Torch, Gensim, Scanpy,\n  Hugging Face Hub, pyBigWig, and HMM dependencies.\n- Upstream documentation contains stale examples. Release source and installed\n  `--help` output take precedence where they conflict.\n\n## Install reproducibly\n\nUse a project environment and commit its generated lockfile:\n\n```bash\nuv venv --python 3.12\nuv pip install \"geniml==0.8.4\" \"gtars==0.9.2\"\n```\n\nFor Region2Vec, scEmbed, evaluation, or universe methods needing ML libraries:\n\n```bash\nuv pip install \"geniml[ml]==0.8.4\" \"gtars==0.9.2\"\n```\n\nFor a durable project, prefer:\n\n```bash\nuv add \"geniml[ml]==0.8.4\" \"gtars==0.9.2\"\nuv lock\n```\n\nDo not install an unpinned Git branch. Record Python, OS/architecture, the\nresolved lockfile, and the PyPI artifact digest. Geniml itself is BSD-2-Clause;\nthe `MIT` frontmatter value licenses this skill's content.\n\n## Start with the safety gate\n\nBefore importing Geniml or running an external binary:\n\n1. Work only with explicit local regular files. Reject URLs, FIFOs, devices,\n   and symlinks unless the user deliberately changes that policy.\n2. Validate BED structure and the declared assembly against a trusted local\n   chromosome-sizes file.\n3. Bound file count, bytes, rows, workers, epochs, and output size.\n4. Separate train/validation/test by patient, donor, biological replicate, or\n   other independent unit—not by BED row or cell alone.\n5. Inventory and checksum the universe, tokenizer, model, config, inputs,\n   metadata manifest, and native binaries.\n6. Obtain explicit approval before any BEDbase or Hugging Face download. Never\n   infer approval from a model ID or BEDbase identifier.\n7. Keep logs aggregate and bounded. BED filenames, sample IDs, phenotypes,\n   labels, barcodes, and genomic intervals may be sensitive.\n\n## Coordinate and assembly contract\n\nBED intervals are normally **0-based, half-open** `[start, end)`: start is\nincluded, end is excluded, and length is `end - start`. Do not mix them with\n1-based closed coordinates from VCF/GFF or user-facing genome browsers.\n\nFor every corpus and artifact, record:\n\n- assembly and patch/accession where possible (for example GRCh38 versus\n  GRCh38.p14), plus the chromosome-sizes checksum;\n- contig naming convention (`chr1` versus `1`), alt/random/decoy policy, and\n  mitochondrial naming;\n- coordinate convention, sorting order, duplicate/overlap policy, and whether\n  BED strand is meaningful;\n- liftover tool, chain digest, source/target assemblies, unmapped fraction, and\n  post-liftover validation.\n\nReject negative coordinates, `end <= start`, integer overflow, unknown\ncontigs, ends beyond contig length, malformed columns, mixed assemblies, and\nsilent contig renaming. Sorting and normalization never repair an assembly\nmismatch. BED3 has no strand; when column 6 is present, preserve `+`, `-`, or\n`.` unless the assay contract says otherwise.\n\nRun a bounded validation and normalization **plan** before analysis:\n\n```bash\npython skills/geniml/scripts/bed_validator.py \\\n  --input data/peaks.bed \\\n  --assembly GRCh38 \\\n  --chrom-sizes refs/GRCh38.chrom.sizes\n```\n\nThe validator reports proposed actions but never rewrites the BED file.\n\n## Current API map\n\n### Region and tokenizer I/O\n\nPrefer Gtars for new interval/tokenizer code:\n\n```python\nfrom gtars.models import Region, RegionSet\nfrom gtars.tokenizers import Tokenizer\n\nregions = RegionSet(\"data/peaks.bed\")\ntokenizer = Tokenizer.from_bed(\"refs/universe.bed\")\nencoded = tokenizer(regions)\ninput_ids = encoded[\"input_ids\"]\n```\n\n`RegionSet` and `Tokenizer` also accept remote inputs in some constructors;\nthis skill permits local paths only unless network access is explicitly\napproved. `geniml.io.RegionSet(regions, backed=False)` remains available as a\nlegacy Python implementation; backed sets are iterable but not indexable.\n`geniml.io.Region` uses `stop`, while `gtars.models.Region` uses `end`.\n\nWith gtars 0.9.2, seven special tokens are added to a BED vocabulary. Therefore\n`len(tokenizer)` is not simply the number of universe rows. Preserve universe\nrow order and the exact special-token map.\n\n### Region2Vec\n\nThe modern class lives at a concrete module path:\n\n```python\nfrom geniml.region2vec.main import Region2VecExModel\nfrom geniml.region2vec.utils import Region2VecDataset\nfrom gtars.tokenizers import Tokenizer\n\ntokenizer = Tokenizer.from_bed(\"refs/universe.bed\")\ndataset = Region2VecDataset(\"work/tokens.parquet\", shuffle=True)\nmodel = Region2VecExModel(tokenizer=tokenizer, embedding_dim=100)\nmodel.train(dataset, epochs=10, window_size=5, num_cpus=4, seed=42)\n```\n\nThe Parquet input must contain one list-valued `tokens` column, one document\nper row. See [references/region2vec.md](references/region2vec.md) for export,\nencoding, legacy CLI, and evaluation details.\n\n### scEmbed\n\nImport `ScEmbed` from `geniml.scembed.main`. AnnData `.var` must contain\n`chr`, `start`, and `end`; rows are cells and nonzero features identify\naccessible regions. Pre-tokenize to a Parquet `tokens` column and use the same\nTokenizer for training and inference. See\n[references/scembed.md](references/scembed.md).\n\n### BEDspace\n\nBEDspace remains in 0.8.4 and invokes an external StarSpace executable.\nStarSpace is archived and upstream Geniml does not pin a compatible revision.\nTreat BEDspace as a legacy reproduction path, not the default for new systems.\nSee [references/bedspace.md](references/bedspace.md) for the exact stable CLI\nspelling and an immutable, explicitly unverified build baseline.\n\n### Consensus universes and assessment\n\nThe installed 0.8.4 CLI uses:\n\n```text\ngeniml build-universe {cc,ccf,ml,hmm} ...\ngeniml assess-universe ...\ngeniml eval {gdst,npt,ctt,rct,bin-gen} ...\n```\n\nCC/CCF/ML/HMM consume precomputed coverage bigWigs. Do not concatenate or\ngenerate coverage until all BED files pass the same assembly contract.\nAssessment and embedding metrics are distinct: `assess-universe` measures fit\nof a universe to interval collections, while `eval` implements CTT, RCT, GDST,\nand NPT for embeddings. See\n[references/consensus_peaks.md](references/consensus_peaks.md) and\n[references/utilities.md](references/utilities.md).\n\n## Important 0.8.4 migration notes\n\n- The 0.7.0 changelog moved new RegionSet/tokenizer work toward Gtars.\n- The 0.4.0 names `TreeTokenizer` and `AnnDataTokenizer` are historical; the\n  current Gtars API exposes `Tokenizer`.\n- In the 0.8.4 wheel, `geniml.region2vec` and `geniml.scembed` do not re-export\n  their modern classes/functions. Use the concrete module paths above.\n- `geniml tokenize` and `geniml region2vec` call names no longer exported by\n  their package `__init__` files; do not build new workflows around those CLI\n  paths without an installed-version smoke test.\n- `geniml scembed` parses legacy MatrixMarket options but its command body is a\n  no-op in 0.8.4. Use `geniml.scembed.main.ScEmbed`.\n- Official pages still show `geniml assess`; the release command is\n  `geniml assess-universe`.\n- `.gtok` remains present in legacy datasets, but upstream issue #14 proposes\n  deprecating many-file `.gtok` workflows. Prefer one bounded Parquet corpus.\n- Config key `embedding_size` is accepted only for backward compatibility;\n  use `embedding_dim`.\n\n## Model and universe compatibility\n\nA Region2Vec/scEmbed inference bundle is valid only when these agree:\n\n- model `config.yaml` `vocab_size` and `embedding_dim`;\n- exact `universe.bed` bytes/order and assembly;\n- tokenizer implementation/version and special-token IDs;\n- checkpoint tensor shapes and pooling policy;\n- Geniml/Gtars versions and any tokenization parameters.\n\nGeniml 0.8.4 defaults to `checkpoint.pt`, `config.yaml`, and `universe.bed`.\nIts loader uses `torch.load(..., weights_only=True)`, but `.pt`, Gensim\n`.model`, pickle, joblib, and native binaries remain untrusted inputs. Inspect\nand checksum artifacts before loading; use an isolated environment and never\nload a checkpoint merely to discover its metadata.\n\n```bash\npython skills/geniml/scripts/model_artifact_inspector.py \\\n  --model-dir models/region2vec\n\npython skills/geniml/scripts/tokenizer_compatibility.py \\\n  --model-dir models/region2vec \\\n  --universe refs/universe.bed \\\n  --assembly GRCh38\n```\n\n`Region2VecExModel(model_path=\"org/repo\")`, `ScEmbed(model_path=\"org/repo\")`,\nand Gtars `Tokenizer.from_pretrained(...)` can download from Hugging Face.\nLocal `from_pretrained(\"models/local\")` loads a local bundle. Pin Hub revision\nand expected hashes when a user approves download; then work offline from the\nverified cache.\n\n## BEDbase downloads and caches\n\n`BBClient.load_bed`, `load_bedset`, and token-cache operations may contact\n`https://api.bedbase.org`. The default cache is\n`$BBCLIENT_CACHE` or `~/.bbcache`; `BEDBASE_API` changes the endpoint. Do not\nread unrelated environment variables. Set an explicit project cache, estimate\nsize, approve identifiers/endpoints, and verify returned checksums before use.\n\nLocal inspection commands are safer:\n\n```text\ngeniml bbclient seek ID --cache-folder /absolute/project/cache\ngeniml bbclient inspect-bedfiles --cache-folder /absolute/project/cache\ngeniml bbclient inspect-bedsets --cache-folder /absolute/project/cache\n```\n\nThe `cache-bed`, `cache-bedset`, and `cache-tokens` subcommands may use the\nnetwork. Do not run them implicitly or include sensitive local BED files in an\nupload/cache workflow.\n\n## Local audit and planning CLIs\n\nAll scripts are standard-library-only and default to redacted JSON:\n\n```bash\n# Audit manifest paths, checksums, assemblies, and patient/donor leakage\npython skills/geniml/scripts/corpus_auditor.py \\\n  --manifest data/manifest.tsv --assembly-column assembly \\\n  --group-column patient_id --split-column split\n\n# Plan tokenizer/model compatibility checks\npython skills/geniml/scripts/tokenizer_compatibility.py \\\n  --model-dir models/r2v --universe refs/universe.bed --assembly GRCh38\n\n# Plan consensus construction; does not execute Geniml or coverage tools\npython skills/geniml/scripts/consensus_plan.py \\\n  --manifest data/manifest.tsv --chrom-sizes refs/GRCh38.chrom.sizes \\\n  --assembly GRCh38 --method cc --output-dir work/consensus\n\n# Plan an embedding run; does not import ML libraries\npython skills/geniml/scripts/embedding_plan.py \\\n  --mode region2vec --data work/tokens.parquet \\\n  --universe refs/universe.bed --output-dir work/r2v \\\n  --assembly GRCh38\n```\n\nUse `--help` for resource limits and explicit path-disclosure controls.\n\n## References\n\n- [Region2Vec](references/region2vec.md): modern API, artifacts, CLI drift,\n  training, encoding, and evaluation.\n- [scEmbed](references/scembed.md): AnnData/token preparation, training,\n  inference, annotation, privacy, and leakage.\n- [BEDspace](references/bedspace.md): metadata schema, exact legacy CLI,\n  StarSpace status, artifacts, and retrieval.\n- [Consensus peaks](references/consensus_peaks.md): coverage prerequisites,\n  CC/CCF/ML/HMM, assessment, and assembly safeguards.\n- [Utilities](references/utilities.md): I/O, Gtars tokenizers, BBClient,\n  evaluation, model safety, migration, and dated sources.\n\nSource snapshot and primary-paper links are dated in\n[references/utilities.md](references/utilities.md). Re-check release metadata\nand installed signatures before changing the pinned versions.","author":"@K-Dense-AI","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/K-Dense-AI/scientific-agent-skills/tree/main/skills/geniml","license":"MIT","category":"coding","lang":"en","tokens":3056,"stars":0,"calls30d":2,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/bedspace.md","size":9265,"sha256":"138c78745f65f2e494850350b1e8ddd43398597d7e0c89e5bf53bc34a3b0f7cb"},{"path":"references/consensus_peaks.md","size":10178,"sha256":"ca68e384aec11780cf6fa0c42c638c8cec370bf15fdd1182769147ca7c75eb31"},{"path":"references/region2vec.md","size":10135,"sha256":"333ed22da3fed1f0c1fbfd6868f59e8c1e498b2ae590ff955425f6b11fb8301e"},{"path":"references/scembed.md","size":10139,"sha256":"5b964c3f0a71dd025a0c6aaf3a11b1f6e9e039db0ae78472b5bcdf39cc82959c"},{"path":"references/utilities.md","size":13553,"sha256":"9f5929c2edcbc91c76a039ba28c6a7b6f81e4664c9412931da73961cd949bcac"},{"path":"scripts/bed_validator.py","size":11186,"sha256":"cc5769dcab0faa3958d0cacd8feb5ec4f413591f58df4d3dc5c3ee6b9ff7f2b4"},{"path":"scripts/_common.py","size":13387,"sha256":"201a17965976ab138f8a3ef32d2d9fae8c2a51e907fe8f04e1d76a15033e548f"},{"path":"scripts/consensus_plan.py","size":14245,"sha256":"c39bda5fed2a977d0b79aac1daed5489bea2756b028add861d3055c28f80eef8"},{"path":"scripts/corpus_auditor.py","size":10637,"sha256":"c21b1c6031f077acbac5a65c36e4b1ae443b9683b7fd65517a7ea22b4ba83e55"},{"path":"scripts/embedding_plan.py","size":17044,"sha256":"5f91fba98d07606d682e2a20b2535ca1a4bce3af969b6835e9e1a53c4c2ff78c"},{"path":"scripts/__init__.py","size":58,"sha256":"7bf3af913d4e1ac2707799e67f39fcc384945ffdeb8c8eca6aad7282179a2fc9"},{"path":"scripts/model_artifact_inspector.py","size":12111,"sha256":"0eabc02c435bd2afb92075d153059928db421f318191e8e236797abd122d1f1a"},{"path":"scripts/tokenizer_compatibility.py","size":10994,"sha256":"1eaaf8ae3bfa2db1ad4573f90f7c194a408307bd7c293ca5f128d6d177a667ef"}],"requires":{"mcp":[],"tools":["Read Write Edit Bash Glob"]},"safety":{"flags":[{"code":"net.endpoints","kind":"exfiltration","excerpt":"api.bedbase.org, docs.bedbase.org, doi.org","message":"bundled scripts reach 3 external host(s)","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":true,"networkEndpoints":["api.bedbase.org","docs.bedbase.org","doi.org"]}}