{"id":"baoyu-url-to-markdown","name":"baoyu-url-to-markdown","summary":"任意のURLを取得し、baoyu-fetch CLI(サイトスペシフィックアダプター付きChromeのCDP)を使ってマークダウンに変換します。","body":"# URL to Markdown\n\nFetches any URL via `baoyu-fetch` CLI (Chrome CDP + site-specific adapters) and converts it to clean markdown.\n\n## User Input Tools\n\nWhen this skill prompts the user, follow this tool-selection rule (priority order):\n\n1. **Prefer built-in user-input tools** exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.\n2. **Fallback**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.\n3. **Batching**: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.\n\nConcrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes.\n\n## CLI Setup\n\n**Important**: The CLI source is vendored in `{baseDir}/scripts/lib`. `scripts/package.json` installs only third-party runtime dependencies.\n\n**Agent Execution Instructions**:\n1. Determine this SKILL.md file's directory path as `{baseDir}`\n2. Resolve `${BUN}` runtime: if `bun` installed → `bun`; else suggest installing Bun\n3. If `{baseDir}/scripts/node_modules` does not exist, run `${BUN} install --cwd {baseDir}/scripts`\n4. `${READER}` = `{baseDir}/scripts/baoyu-fetch`\n5. Replace all `${READER}` in this document with the resolved value\n\n## Preferences (EXTEND.md)\n\nCheck EXTEND.md in priority order — the first one found wins:\n\n| Priority | Path | Scope |\n|----------|------|-------|\n| 1 | `.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | Project |\n| 2 | `${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | XDG |\n| 3 | `$HOME/.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | User home |\n\n| Result | Action |\n|--------|--------|\n| Found | Read, parse, apply settings |\n| Not found | **MUST** run first-time setup (see below) — do NOT silently create defaults |\n\n**EXTEND.md supports**: download media by default, default output directory.\n\n### First-Time Setup ⛔ BLOCKING\n\nWhen EXTEND.md is not found, you **MUST** use `AskUserQuestion` to gather preferences before creating EXTEND.md. **NEVER** create EXTEND.md with silent defaults. Generation is BLOCKED until setup completes. Batch all three questions into a single call:\n\n- **Q1 — Media** (header \"Media\"): \"How to handle images and videos in pages?\"\n  - \"Ask each time (Recommended)\" — Prompt after each save\n  - \"Always download\" — Download to local `imgs/` and `videos/`\n  - \"Never download\" — Keep remote URLs\n- **Q2 — Output** (header \"Output\"): \"Default output directory?\"\n  - \"url-to-markdown (Recommended)\" — Save to `./url-to-markdown/{domain}/{slug}.md`\n  - User may pick \"Other\" and type a custom path\n- **Q3 — Save** (header \"Save\"): \"Where to save preferences?\"\n  - \"User (Recommended)\" — `~/.baoyu-skills/` (all projects)\n  - \"Project\" — `.baoyu-skills/` (this project only)\n\nAfter answers, write EXTEND.md, confirm \"Preferences saved to [path]\", then continue.\n\nFull template: [references/config/first-time-setup.md](references/config/first-time-setup.md).\n\n### Supported Keys\n\n| Key | Default | Values | Description |\n|-----|---------|--------|-------------|\n| `download_media` | `ask` | `ask` / `1` / `0` | `ask` = prompt each time, `1` = always, `0` = never |\n| `default_output_dir` | empty | path or empty | Default output directory (empty = `./url-to-markdown/`) |\n\n**EXTEND.md → CLI mapping**:\n\n| EXTEND.md key | CLI argument | Notes |\n|---------------|-------------|-------|\n| `download_media: 1` | `--download-media` | Requires `--output` to be set |\n| `default_output_dir: ./posts/` | Agent constructs `--output ./posts/{domain}/{slug}.md` | Agent generates path, not a direct flag |\n\n**Value priority**: CLI arguments → EXTEND.md → skill defaults.\n\n## Usage\n\n```bash\n# Default: headless capture, markdown to stdout\n${READER} <url>\n\n# Save to file\n${READER} <url> --output article.md\n\n# Save with media download\n${READER} <url> --output article.md --download-media\n\n# Wait for interaction (login/CAPTCHA) — auto-detect and continue\n${READER} <url> --wait-for interaction --output article.md\n\n# Wait for interaction — manual control (Enter to continue)\n${READER} <url> --wait-for force --output article.md\n\n# JSON output\n${READER} <url> --format json --output article.json\n\n# Force specific adapter\n${READER} <url> --adapter youtube --output transcript.md\n```\n\n## Options\n\n| Option | Description |\n|--------|-------------|\n| `<url>` | URL to fetch |\n| `--output <path>` | Output file path (default: stdout) |\n| `--format <type>` | Output format: `markdown` (default) or `json` |\n| `--json` | Shorthand for `--format json` |\n| `--adapter <name>` | Force adapter: `x`, `youtube`, `hn`, or `generic` (default: auto-detect) |\n| `--headless` | Force headless Chrome (no visible window) |\n| `--wait-for <mode>` | Interaction wait mode: `none` (default), `interaction`, or `force` |\n| `--wait-for-interaction` | Alias for `--wait-for interaction` |\n| `--wait-for-login` | Alias for `--wait-for interaction` |\n| `--timeout <ms>` | Page load timeout (default: 30000) |\n| `--interaction-timeout <ms>` | Login/CAPTCHA wait timeout (default: 600000 = 10 min) |\n| `--interaction-poll-interval <ms>` | Poll interval for interaction checks (default: 1500) |\n| `--download-media` | Download images/videos to local `imgs/` and `videos/`, rewrite markdown links. Requires `--output` |\n| `--media-dir <dir>` | Base directory for downloaded media (default: same as `--output` directory) |\n| `--cdp-url <url>` | Reuse existing Chrome DevTools Protocol endpoint |\n| `--browser-path <path>` | Custom Chrome/Chromium binary path |\n| `--chrome-profile-dir <path>` | Chrome user data directory (default: `BAOYU_CHROME_PROFILE_DIR` env or `./baoyu-skills/chrome-profile`) |\n| `--debug-dir <dir>` | Write debug artifacts (document.json, markdown.md, page.html, network.json) |\n\n## Agent Quality Gate\n\n**CRITICAL**: treat default headless capture as provisional. Some sites render differently in headless mode and can silently return low-quality content without failing the CLI.\n\nAfter every headless run, inspect the saved markdown. See [references/quality-gate.md](references/quality-gate.md) for the full checklist, recovery workflow, and capture-mode table. Read it whenever a run looks suspicious or the user asks about login/CAPTCHA handling.\n\n## Output Path Generation\n\nThe agent must construct the output file path — `baoyu-fetch` does not auto-generate paths.\n\n**Algorithm**:\n1. Determine base directory from EXTEND.md `default_output_dir` or default `./url-to-markdown/`\n2. Extract domain from URL (e.g., `example.com`)\n3. Generate slug from URL path or page title (kebab-case, 2-6 words)\n4. Construct: `{base_dir}/{domain}/{slug}/{slug}.md` — each URL gets its own directory so media files stay isolated\n5. Conflict resolution: append timestamp `{slug}-YYYYMMDD-HHMMSS/{slug}-YYYYMMDD-HHMMSS.md`\n\nPass the constructed path to `--output`. Media files (`--download-media`) are saved into subdirectories next to the markdown file, keeping each URL's assets self-contained.\n\n## Adapters & Media\n\nSee [references/adapters.md](references/adapters.md) for the adapter catalog (X, YouTube, Hacker News, generic), per-adapter notes, the media download flow (`ask` / always / never), and the JSON output schema. Read it before answering adapter-specific questions or handling media prompts.\n\n## Environment Variables\n\n| Variable | Description |\n|----------|-------------|\n| `BAOYU_CHROME_PROFILE_DIR` | Chrome user data directory (can also use `--chrome-profile-dir`) |\n\n**Troubleshooting**: Chrome not found → use `--browser-path`. Timeout → increase `--timeout`. Login/CAPTCHA → `--wait-for interaction`. Debug → `--debug-dir` to inspect captured HTML and network logs.\n\n## Extension Support\n\nCustom configurations via EXTEND.md. See **Preferences** section above for paths and supported keys.","author":"@JimLiu","ownerProfile":null,"authorContacts":null,"sourceUrl":"https://github.com/JimLiu/baoyu-skills/tree/main/skills/baoyu-url-to-markdown","license":"MIT","category":"document","lang":"en","tokens":2031,"stars":0,"calls30d":1,"claimed":false,"visibility":"public","origin":"crawler","version":"0.1.0","createdAt":"2026-08-22","updatedAt":"2026-08-22","files":[{"path":"references/adapters.md","size":3231,"sha256":"98ae3d1444cd681fbcd472f2839ae071ae008b8f90a781c6295f94e166e7f606"},{"path":"references/config/first-time-setup.md","size":2459,"sha256":"571620de72538e8b4f247a63e6db27ce5a27495d1283015ec16c1076d91811cc"},{"path":"references/quality-gate.md","size":2490,"sha256":"a14c1055be7b0c18ac6229cf4ab8d20cfbfdf0a8a8543d484fc39a83ea3972aa"},{"path":"scripts/baoyu-fetch","size":122,"sha256":"fdc95a6483e0df17fd012bcd3756b2da158128bd0b1d5363093ea4ae446bc8db"},{"path":"scripts/bun.lock","size":35037,"sha256":"eb1032693007cf617c585920cf4895d47f43b823880cdae800045783bf60a188"},{"path":"scripts/lib/adapters/generic/index.ts","size":2451,"sha256":"2dc64acdceb2d406c864ab28ccbcb52348c39a5c7ceba9928814d6af93ceb11e"},{"path":"scripts/lib/adapters/hn/index.ts","size":10454,"sha256":"354bbae8600eeffb1d814eddb5f6452a5676a98a311b48dc19f6f85575f5213e"},{"path":"scripts/lib/adapters/index.ts","size":834,"sha256":"328c022b8ad10df88b7fae9aab8fda55cbecc7dec3bacd5a061591dc4d4ce20a"},{"path":"scripts/lib/adapters/types.ts","size":2100,"sha256":"f8ff9d5762c0d2c2beb6bb66d299020d20cf7135bdfd420cf168f66b9fac20ba"},{"path":"scripts/lib/adapters/x/article.ts","size":12677,"sha256":"fee60a0b6ea1e361a20316632d24b846586d649e494b7ff28a2ab1a12ef4d38c"},{"path":"scripts/lib/adapters/x/index.ts","size":4048,"sha256":"2c3eb04c68c1f393a60a6755f7d4ad568f76af657556293b9063fe569a055564"},{"path":"scripts/lib/adapters/x/login.ts","size":2258,"sha256":"2256b982fc94599655b2ebb29674fb3843c66692d457201d81e1d97b81355de7"},{"path":"scripts/lib/adapters/x/match.ts","size":297,"sha256":"644a44aecba218030e177202b404dcea5ea3c33aad4143a2d69ceb79875151f1"},{"path":"scripts/lib/adapters/x/payloads.ts","size":1681,"sha256":"7fc49b04d45b3806338fe4ac20d444516a8a6b797dc661cd04bd49536ddc6bfd"},{"path":"scripts/lib/adapters/x/session.ts","size":1404,"sha256":"e98f0d690b77464c57486f517d9aa5c8ce6a7827adfed3aa9be9aa93757866bb"},{"path":"scripts/lib/adapters/x/shared.ts","size":12980,"sha256":"6c1bc42db39dc240c8baf4db126b4aab6279c66df941ee93df4ff29cede44a1d"},{"path":"scripts/lib/adapters/x/single.ts","size":2583,"sha256":"6aad7745fb3c223c71564e87912958e5ae2f2f285d2fcc9ffc1d6de168f8f032"},{"path":"scripts/lib/adapters/x/thread-loader.ts","size":9078,"sha256":"3205301481abec9662ce0e38ac8c9dafb7550b937cc12174a4786ab6be30869e"},{"path":"scripts/lib/adapters/x/thread.ts","size":9196,"sha256":"69f9465ee13c8020d21dbac8c57c24b85ae7bba1cd5bc4cfa6eea73def87822d"},{"path":"scripts/lib/adapters/x/types.ts","size":600,"sha256":"961679524761c8efe956435f5b18b522c779dd17ea20e8ba33ca6d73a6c04ba2"},{"path":"scripts/lib/adapters/youtube/index.ts","size":898,"sha256":"518cb4f8e1a3ac831dad87d49c8ce7e02d7484f5a45085b8094da6a3f412cd13"},{"path":"scripts/lib/adapters/youtube/transcript.ts","size":13301,"sha256":"45562c9d50a25a08405833c28c4a508505f00a11b4d9250600d0b1676777a28a"},{"path":"scripts/lib/adapters/youtube/utils.ts","size":6523,"sha256":"d7b4a517d435f4d29bb89eba19eba94f1025cb64323d9bd0e3c5efe15ca18e42"},{"path":"scripts/lib/browser/cdp-client.ts","size":6879,"sha256":"c218de5b4adc3456da77ea299e3151b7060bbc5a02d82b2242f79be8f5aa4a3f"}],"requires":{"mcp":[],"tools":[]},"safety":{"flags":[{"code":"net.endpoints","kind":"exfiltration","excerpt":"i.ytimg.com, news.ycombinator.com, twitter.com, www.youtube.com, x.com","message":"bundled scripts reach 5 external host(s)","severity":"warn"}],"scannedAt":"2026-08-22","hasScripts":true,"networkEndpoints":["i.ytimg.com","news.ycombinator.com","twitter.com","www.youtube.com","x.com"]}}