{"id":"autoresearch","name":"lab:autoresearch","summary":"プラグインスキルのための自己改善ループ。program.md を読み、1回の反復で1つの変異を提案し、決定的なスコアラーに対して評価し、gitで改善を保持し、失敗を元に戻します。","body":"# Autoresearch — Plugin Skill Self-Improvement\n\nIteratively improve plugin skills via the autoresearch pattern:\npropose one mutation -> eval -> keep/revert -> repeat.\n\n## Usage\n\n```\n/lab:autoresearch                           # Targeted: attack weakest skill+dimension\n/lab:autoresearch --skill review            # Focus on one skill\n/lab:autoresearch --strategy sweep          # Process all skills alphabetically\n/lab:autoresearch --dry-run                 # Show what would change, don't commit\n```\n\nFor overnight runs:\n\n```\n/loop 5m /lab:autoresearch --strategy sweep --max-iterations 200\n```\n\n## Iron Laws\n\n1. **ONE mutation per iteration** — if description needs \"and\", split into two\n2. **NEVER mutate read-only files** — check program.md before every write\n3. **EVAL is deterministic** — always use the wrapper script, never LLM-judge\n4. **REVERT on regression OR checks failure** — no exceptions\n5. **LOG every iteration** — use `keep` or `revert` command (never skip)\n6. **CHECK ideas.md before proposing** — don't rediscover known optimizations\n\n## Wrapper Script Commands\n\nAll eval/git/journal operations go through ONE script. Do NOT run these manually.\n\n```bash\n# Find the weakest skill+dimension\npython3 lab/autoresearch/scripts/run-iteration.py target --strategy targeted\n\n# Score a skill (before mutation, to get baseline)\npython3 lab/autoresearch/scripts/run-iteration.py score <skill-name>\n\n# After mutation: score + checks + compare → verdict (KEEP or REVERT)\npython3 lab/autoresearch/scripts/run-iteration.py eval <skill-name>\n\n# Act on verdict:\npython3 lab/autoresearch/scripts/run-iteration.py keep <skill> <dim> <old> <new> \\\n  --desc \"what changed\" --asi '{\"hypothesis\": \"why\", \"mechanism\": \"how\"}'\n\npython3 lab/autoresearch/scripts/run-iteration.py revert <skill> <dim> <old> <new> \\\n  --desc \"what was attempted\" --asi '{\"hypothesis\": \"why\", \"regression\": \"what broke\", \"avoid\": \"do not retry this\"}'\n\n# Check overall progress\npython3 lab/autoresearch/scripts/run-iteration.py status\n```\n\n## Core Loop (ONE iteration)\n\n### Step 1: Read State\n\n1. Read `lab/autoresearch/program.md` (goals, mutable surface, rules)\n2. Read `lab/autoresearch/ideas.md` if it exists (deferred optimizations)\n3. Run: `python3 lab/autoresearch/scripts/run-iteration.py status`\n\n### Step 2: Select Target\n\nRun: `python3 lab/autoresearch/scripts/run-iteration.py target --strategy targeted`\n\nParse the JSON: `skill`, `dimension`, `failing_checks`. If `all_perfect` → STOP.\n\n### Step 3: Read + Propose\n\n1. Read target SKILL.md and its references/ listing\n2. Read eval definition from `lab/eval/evals/{skill}.json`\n3. Check `ideas.md` for deferred ideas about this skill\n4. Check recent journal entries for prior failures on this skill (avoid repeats)\n5. Consult `${CLAUDE_SKILL_DIR}/references/mutation-strategies.md`\n6. Propose exactly ONE change targeting the failing checks\n\n### Step 4: Apply + Evaluate\n\n1. Apply the mutation via Edit tool\n2. Run: `python3 lab/autoresearch/scripts/run-iteration.py eval <skill-name>`\n3. Parse JSON → check `verdict` field\n\n### Step 5: Keep or Revert\n\n**If verdict is KEEP**:\n\n```bash\npython3 lab/autoresearch/scripts/run-iteration.py keep <skill> <dim> <old> <new> \\\n  --desc \"...\" --asi '{\"hypothesis\": \"...\", \"mechanism\": \"...\"}'\n```\n\n**If verdict is REVERT**:\n\n```bash\npython3 lab/autoresearch/scripts/run-iteration.py revert <skill> <dim> <old> <new> \\\n  --desc \"...\" --asi '{\"hypothesis\": \"...\", \"regression\": \"...\", \"avoid\": \"...\"}'\n```\n\n### Step 6: Ideas Backlog\n\nIf during analysis you discovered a promising optimization you can't act on now:\n\n- Append it to `lab/autoresearch/ideas.md` as a bullet\n- On next resume: prune stale/tried ideas, experiment with the rest\n\n### Step 7: Continue or Stop\n\n- All targets >= 0.95? Print \"AUTORESEARCH_COMPLETE\"\n- Max iterations reached? Print \"AUTORESEARCH_COMPLETE\"\n- 50 consecutive discards? Print \"AUTORESEARCH_STUCK\"\n- Otherwise: immediately start Step 1 again\n\n## References\n\n- `${CLAUDE_SKILL_DIR}/references/mutation-strategies.md` — mutation type catalog\n- `${CLAUDE_SKILL_DIR}/references/state-management.md` — git protocol, journaling\n- `lab/autoresearch/program.md` — research agenda (read every iteration)","author":"@oliver-kriska","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/oliver-kriska/claude-elixir-phoenix/tree/main/lab/autoresearch","license":"MIT","category":"writing","lang":"en","tokens":1087,"stars":0,"calls30d":1,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":".gitignore","size":130,"sha256":"9080da282fcb9210b964dddf40bc5b07a45971e709f05d366ac1b722430c0e02"},{"path":"program.md","size":4037,"sha256":"b5ebced72826145ee030859bddca95201e25fa5c78341daaa702eb9f2437f670"},{"path":"references/mutation-strategies.md","size":2468,"sha256":"9dda8315581f06f069e1d964d65637f4ae97fcd6515d1ae935ffe7ae5cf3c079"},{"path":"references/state-management.md","size":1125,"sha256":"1481eda4f918687971fececedfb8e9f251572f9aaa94e02d224a45a1183b0d03"},{"path":"retention.py","size":5821,"sha256":"dff8b5046e460b4b4c3851ad50db4c4414ba058bf674e1e25f04f4e1fe248ab0"},{"path":"scripts/checks.sh","size":3300,"sha256":"3451c8e37f8eab24b69a9f0194835137275962f64aeb5060cb89d8c42be47d2b"},{"path":"scripts/protected_sections.py","size":3476,"sha256":"d5294fe09f283ec6c38d3f676cd773588e16e5c301317ad9c9602a96425dcb41"},{"path":"scripts/run-iteration.py","size":25608,"sha256":"d7b9f6d4b9748514e4852e814002eda52dc20b55b5b85d2ab321fec12f7827b1"},{"path":"scripts/score-skill.py","size":1271,"sha256":"8f9cddfe41f5f7b5b3c53a693fce057b6396f361ee32f28a58465367d1045ae0"},{"path":"tests/__init__.py","size":0,"sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},{"path":"tests/test_deviation_dispatch.py","size":3152,"sha256":"b935345df561f07fa56b2842446d0c3f158f175a7f35335c50312969e7a1aa73"},{"path":"tests/test_protected_sections.py","size":3422,"sha256":"fbb25f9776618de0eb5a117449c51b1825c9627c47ccfb334532b1e4d9e84d21"}],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[{"code":"code.eval","kind":"dangerous-code","where":"scripts/run-iteration.py:525","excerpt":"eval (","message":"evaluates code at runtime","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":true,"networkEndpoints":[]}}