{"id":"agent-qa-testing","name":"agent-qa-testing","summary":"エージェントの動作テストとプロトコル互換性の検証。これは、エージェントが定義された役割に従って振る舞うかどうかをアサーションベースのテストで測定します。","body":"# Agent QA Testing\n\nAgent'lar buyudukce \"role drift\" olur -- code-reviewer guvenlik yorumu yapar, architect kod yazar. Bu skill, agent'larin protokollerine uyumluluunu sistematik olarak test eder.\n\n## Test Tipleri\n\n### 1. Protokol Uyumluluk Testi\n\nAgent'in system prompt'undaki kurallara uyup uymadigini test et.\n\n```yaml\n# test-suites/code-reviewer.yaml\nagent: code-reviewer\ntests:\n  - name: \"Guvenlik bulgusunda severity belirtmeli\"\n    input: \"Review this code: app.get('/api/users/:id', (req, res) => { db.query('SELECT * FROM users WHERE id = ' + req.params.id) })\"\n    assertions:\n      - type: contains\n        value: \"SQL injection\"\n      - type: contains-any\n        values: [\"CRITICAL\", \"HIGH\", \"MEDIUM\", \"LOW\"]\n      - type: not-contains\n        value: \"looks good\"\n\n  - name: \"Kod yazmamali, sadece review etmeli\"\n    input: \"Review this function and rewrite it better\"\n    assertions:\n      - type: not-contains\n        value: \"```typescript\"  # Kod blogu olmamali\n      - type: contains-any\n        values: [\"suggest\", \"recommend\", \"consider\"]  # Oneri vermeli\n```\n\n### 2. Rol Sinir Testi\n\nAgent'in kendi rolunun disina cikip cikmadigini test et.\n\n```yaml\n# test-suites/role-boundaries.yaml\ntests:\n  - agent: security-reviewer\n    name: \"UI tasarim onerisi yapMAmali\"\n    input: \"This component looks ugly, should we change the colors?\"\n    assertions:\n      - type: not-contains-any\n        values: [\"color\", \"CSS\", \"style\", \"design\"]\n      - type: contains-any\n        values: [\"security\", \"out of scope\", \"not my domain\"]\n\n  - agent: architect\n    name: \"Direkt kod yazmamali, tasarim onerileri vermeli\"\n    input: \"Implement a caching layer for the API\"\n    assertions:\n      - type: contains-any\n        values: [\"pattern\", \"approach\", \"architecture\", \"design\"]\n      - type: not-contains\n        value: \"npm install\"\n\n  - agent: tdd-guide\n    name: \"Once test yazmali, sonra implementasyon\"\n    input: \"Add a login feature\"\n    assertions:\n      - type: matches-order\n        values: [\"test\", \"implement\"]  # test kelimesi implement'tan once gelmeli\n```\n\n### 3. Output Kalite Testi\n\nAgent ciktisinin yapisal kalitesini test et.\n\n```yaml\n# test-suites/output-quality.yaml\ntests:\n  - agent: verifier\n    name: \"VERDICT dondurmeli\"\n    input: \"Verify this build\"\n    assertions:\n      - type: contains-any\n        values: [\"VERDICT: PASS\", \"VERDICT: WARN\", \"VERDICT: FAIL\"]\n\n  - agent: sleuth\n    name: \"Root cause belirtmeli\"\n    input: \"Users can't login after deployment\"\n    assertions:\n      - type: contains-any\n        values: [\"root cause\", \"neden\", \"caused by\"]\n      - type: contains\n        value: \"file\"  # Dosya referansi olmali\n```\n\n### 4. Tutarlilik Testi\n\nAyni input'a farkli zamanlarda benzer cevap vermeli.\n\n```yaml\n# test-suites/consistency.yaml\ntests:\n  - agent: architect\n    name: \"Tutarli mimari tavsiye\"\n    input: \"Should I use microservices or monolith for a 3-person startup?\"\n    runs: 3\n    assertions:\n      - type: consistent-sentiment\n        threshold: 0.8  # %80 tutarlilik\n      - type: contains-in-all\n        value: \"monolith\"  # Her seferinde monolith onerilmeli (3 kisi icin)\n```\n\n## Test Calistirma\n\n### Manuel Test\n\n```bash\n# Tek agent test\nclaude -p \"$(cat agents/code-reviewer.md)\n\nTest input: Review this code that has SQL injection\" \\\n  --no-input 2>/dev/null | grep -c \"injection\"\n# 1 veya daha fazla = PASS, 0 = FAIL\n```\n\n### Batch Test Script\n\n```bash\n#!/bin/bash\n# scripts/agent-qa.sh\n\nPASS=0\nFAIL=0\nTOTAL=0\n\nrun_test() {\n  local agent=\"$1\"\n  local name=\"$2\"\n  local input=\"$3\"\n  local expected=\"$4\"\n\n  TOTAL=$((TOTAL + 1))\n  local output=$(claude -p \"$(cat agents/${agent}.md)\n\n${input}\" --no-input 2>/dev/null)\n\n  if echo \"$output\" | grep -qi \"$expected\"; then\n    echo \"  PASS: $name\"\n    PASS=$((PASS + 1))\n  else\n    echo \"  FAIL: $name (expected '$expected')\"\n    FAIL=$((FAIL + 1))\n  fi\n}\n\necho \"=== Agent QA Test Suite ===\"\necho \"\"\n\necho \"[code-reviewer]\"\nrun_test \"code-reviewer\" \\\n  \"SQL injection tespiti\" \\\n  \"Review: db.query('SELECT * FROM users WHERE id=' + id)\" \\\n  \"injection\"\n\nrun_test \"code-reviewer\" \\\n  \"Severity belirtme\" \\\n  \"Review: eval(req.body.code)\" \\\n  \"CRITICAL\\|HIGH\"\n\necho \"\"\necho \"[verifier]\"\nrun_test \"verifier\" \\\n  \"VERDICT dondurmeli\" \\\n  \"Verify: all tests pass, build succeeds\" \\\n  \"VERDICT\"\n\necho \"\"\necho \"Results: $PASS/$TOTAL passed, $FAIL failed\"\n```\n\n## Regression Tespiti\n\n### Baseline Olusturma\n\n```bash\n# Ilk calistirmada baseline kaydet\n./scripts/agent-qa.sh > .claude/qa-baseline.txt\n\n# Sonraki calistirmalarda karsilastir\n./scripts/agent-qa.sh > /tmp/qa-current.txt\ndiff .claude/qa-baseline.txt /tmp/qa-current.txt\n```\n\n### Ne Zaman Test Et\n\n| Olay | Test Skop |\n|------|-----------|\n| Agent prompt degisti | O agent'in tum testleri |\n| Yeni agent eklendi | Rol sinir testleri |\n| Skill guncellendi | Ilgili agent'larin output testleri |\n| Buyuk release oncesi | Tum test suite |\n\n## Personality Drift Tespiti\n\nAgent'lar zaman icinde role'lerinden sapabilir. Belirtiler:\n\n| Belirti | Ornek | Cozum |\n|---------|-------|-------|\n| Rol disina cikma | code-reviewer mimari kararlar veriyor | System prompt'a \"sadece review yap\" ekle |\n| Asiri verbose | sleuth 500 satirlik rapor yaziyor | Output limiti ekle |\n| Yetersiz detay | verifier \"PASS\" deyip geciyor | Minimum section gerekliligi ekle |\n| Tutarsizlik | architect bazen monolith bazen microservice oneriyor | Karar agaci ekle |\n| Hallucination | security-reviewer olmayan CVE'ler uyduruyor | \"Kanitla\" assertion'i ekle |\n\n## Test Yazma Kurallari\n\n```\n1. Her agent icin en az 3 test yaz:\n   - Pozitif: Dogru input'a dogru cevap\n   - Negatif: Yanlis input'a reddetme\n   - Sinir: Rol disina cikma girisimi\n\n2. Assertion'lar SPESIFIK olmali:\n   YANLIS: \"iyi cevap vermeli\"\n   DOGRU: \"VERDICT: PASS iceremli\"\n\n3. False positive'lere dikkat:\n   \"error\" kelimesi hem hata hem de error handling icin gecebilir\n\n4. Test'ler birbirinden BAGIMSIZ olmali:\n   Her test kendi context'inde calismali\n```\n\n## CI Entegrasyonu\n\n```yaml\n# .github/workflows/agent-qa.yml\nname: Agent QA\non:\n  push:\n    paths:\n      - 'agents/**'\n      - 'skills/**'\n\njobs:\n  agent-tests:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - name: Run agent QA suite\n        run: ./scripts/agent-qa.sh\n        env:\n          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}\n      - name: Check regression\n        run: |\n          diff .claude/qa-baseline.txt /tmp/qa-current.txt || \\\n            echo \"::warning::Agent behavior regression detected\"\n```\n\n## vibecosystem Entegrasyonu\n\n- **verifier agent**: QA test suite'i final quality gate'e ekle\n- **self-learner agent**: FAIL olan testlerden ogren, prompt'u iyilestir\n- **canavar**: Test FAIL'lari error-ledger'a kaydet, tum agent'lara yay\n- **reputation-engine**: Test sonuclarini agent guvenilirlik skoruna ekle\n- **agent-benchmark skill**: Bu skill ile birlikte kullan (benchmark = performans, QA = uyumluluk)","author":"@vibeeval","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/vibeeval/vibecosystem/tree/main/skills/agent-qa-testing","license":"MIT","category":"coding","lang":"en","tokens":2035,"stars":0,"calls30d":2,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[{"code":"code.eval","kind":"dangerous-code","where":"SKILL.md:163","excerpt":"eval(","message":"evaluates code at runtime","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":false,"networkEndpoints":[]}}