`,\n `${esc(s.title)}

`,\n `${esc(meta(s))} · thread

${esc(s.summary)}

`,\n `

What HN is saying

`,\n );\n\n if (s.discussion.lead) {\n parts.push(`

${esc(s.discussion.lead)}

`);\n }\n\n parts.push(`

${esc(p.label)}

${label}${esc(p.text)}

`);\n\n parts.push(\n `

from ${s.commentsRead} comments and replies

/, else the whole document. */\nfunction mainContent(html: string): string {\n for (const t of [\"article\", \"main\"]) {\n const blocks = [...html.matchAll(new RegExp(`<${t}\\\\b[^>]*>([\\\\s\\\\S]*?)`, \"gi\"))]\n .map((m) => m[1])\n .sort((a, b) => b.length - a.length);\n if (blocks[0] && blocks[0].length > 500) return blocks[0];\n }\n return html.match(/]*>([\\s\\S]*?)<\\/body>/i)?.[1] ?? html;\n}\n\nexport async function fetchArticleText(url: string, maxChars: number): Promise {\n try {\n const res = await fetch(url, {\n headers: { \"user-agent\": USER_AGENT, accept: \"text/html,application/xhtml+xml\" },\n redirect: \"follow\",\n signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),\n });\n if (!res.ok) return null;\n\n const type = res.headers.get(\"content-type\") ?? \"\";\n if (!/text\\/html|application\\/xhtml/i.test(type)) {\n await res.body?.cancel();\n return null;\n }\n const length = Number(res.headers.get(\"content-length\"));\n if (Number.isFinite(length) && length > MAX_BYTES) {\n await res.body?.cancel();\n return null;\n }\n\n const html = (await res.text()).slice(0, MAX_BYTES);\n const text = htmlToText(mainContent(html.replace(BOILERPLATE, \" \")));\n\n // Too short to be a real article — cookie wall, JS-only page, paywall stub\n if (text.length < 400) return null;\n return text.slice(0, maxChars);\n } catch {\n return null;\n }\n}\n",[228],"045b5582-2a10-4bdf-ba81-b4c97533cf6e","README.md","6dfc4c34-a257-11f1-9a7b-1607ee4eb77e",65,["D",1787865023038],8,"# hn-digest\n\nA daily email of the **top 5** Hacker News front-page stories, each with a summary of the\nlinked page and a summary of what the top comments are arguing about.\n\nSource is the official [HN Firebase API](https://github.com/HackerNews/API) — keyless and\nunauthenticated. Two cron jobs:\n\n| File | Schedule | Job |\n| --- | --- | --- |\n| `collect.ts` | hourly, `:07` | Record what's on the front page |\n| `main.ts` | `07:00 UTC` | Pick the top 5 and email them |\n\n## Why two jobs\n\n`topstories.json` is a live snapshot, so a single daily poll would miss anything that\npeaked overnight and fell off. The collector records every front-page story hourly into\n`hn-digest:pool`, keeping high-water marks for score and comment count. The digest then\npicks the highest **peak** score of the past 36 hours — a genuine top-of-day, not\nwhatever happened to be up at 07:00.\n\n## How the digest run works\n\n1. `pick()` takes the 5 highest-peak-score pool entries not already emailed.\n2. Each story is refetched for its live score and its `kids` list. **`kids` is in HN's own\n rank order** — comments carry no score in the API, so this is the only signal for \"top\n comment\". Algolia's `children` is chronological and `hnrss.org/item?id=` is\n reverse-chronological, so neither can rank.\n3. **Replies are fetched too** — 10 top-level comments, each with up to 3 replies (~25-35\n comments per story). This is not optional polish. Reply count tracks contention: on the\n Nvidia/Hugging Face thread the three most-replied top comments were \"what are they\n buying exactly?\" (11 replies), \"what is the business model?\" (12) and a prediction about\n banning quantized models (19). Top-level alone made the digest report *questions HN\n asked* instead of *what HN concluded* — the answers were all one level down. The prompt\n marks replies as replies so they aren't read as independent opinions.\n4. `article.ts` fetches the linked page and extracts text. Failures (paywalls, JS-only\n pages, video, PDFs) leave the summary leaning on the comments, and it says so. Ask/Show\n HN self-posts use the post body instead.\n5. `summarize.ts` makes one Claude call per story. No comments means no discussion section\n rather than an invented one. The discussion comes back as a framing line plus labelled\n bullets, parsed into `Discussion { lead, points[] }` — the shape follows the thread, so\n a genuine disagreement gets opposing bullets while a pile of corrections doesn't get\n forced into a fake two-sided split. `parseDiscussion` falls back to keeping the prose as\n a single unlabelled point if the model returns no bullets.\n6. `render.ts` builds the HTML + plain-text email, sent via `std/email`.\n7. The seen-set is written **after** the send succeeds, and only for stories that made it\n into the email. Anything that failed is retried on the next run. A blob read failure\n aborts the run rather than re-sending everything.\n\n## Env vars\n\n| Key | Required | Purpose |\n| --- | --- | --- |\n| `ANTHROPIC_API_KEY` | yes | Claude API |\n| `DIGEST_TO` | no | Recipient. Defaults to the val owner's address. |\n| `DIGEST_DRY_RUN` | no | `1` logs the email instead of sending it. |\n\n## Cost\n\nFive Claude calls a day on `claude-sonnet-5` — roughly **$2/month**.\n\n`claude-haiku-4-5` was measured against Sonnet on identical input for the same five\nstories and lost on specifics: it missed the Microduck's motor count, loop rate and\nlicence, missed three of the six argument threads on the Cloudflare post, and got the\nReachy Mini licence wrong. Haiku costs about $1/month less. Not worth it here, but the\n`model` argument to `summarize()` makes the comparison easy to re-run.\n\nThe collector makes ~31 HN API calls an hour; the digest makes ~5 × 31. All free.\n\n## Tuning\n\nKnobs at the top of `main.ts`: `DIGEST_SIZE`, `MIN_SCORE`, `MAX_AGE_HOURS`,\n`TOP_COMMENTS`, `REPLIES_PER_COMMENT`, `ARTICLE_MAX_CHARS`. `WATCH_TOP` and `POOL_TTL_DAYS` are in\n`collect.ts`. Prompt and tone live in `SYSTEM` in `summarize.ts`.\n",[235],"pulls","branches","forks","envVars",{"_249":187,"_251":26},"connections",{"_249":26,"_250":26},"count","unfilledCount","emptyCount",{"_33":234,"_13":235,"_64":236,"_43":-5,"_35":36,"_167":237,"_169":80,"_37":290,"_39":239,"_66":21,"_171":240,"_68":24,"_173":-5,"_263":291,"_17":292},{"_33":227,"_13":228,"_64":229,"_43":-5,"_35":36,"_167":230,"_169":80,"_37":287,"_39":195,"_66":67,"_171":232,"_68":24,"_173":-5,"_263":288,"_17":289},{"_33":220,"_13":221,"_64":222,"_43":-5,"_35":36,"_167":223,"_169":80,"_37":284,"_39":26,"_66":203,"_171":225,"_68":24,"_173":-5,"_263":285,"_17":286},{"_33":213,"_13":214,"_64":215,"_43":-5,"_35":36,"_167":216,"_169":80,"_37":281,"_39":187,"_66":67,"_171":218,"_68":24,"_173":-5,"_263":282,"_17":283},{"_33":206,"_13":207,"_64":208,"_43":-5,"_35":36,"_167":209,"_169":80,"_37":278,"_39":26,"_66":67,"_171":211,"_68":24,"_173":-5,"_263":279,"_17":280},{"_33":198,"_13":199,"_64":200,"_43":-5,"_35":36,"_167":201,"_169":80,"_37":275,"_39":74,"_66":203,"_171":204,"_68":24,"_173":-5,"_263":276,"_17":277},{"_33":190,"_13":191,"_64":192,"_43":-5,"_35":36,"_167":193,"_169":80,"_37":272,"_39":195,"_66":67,"_171":196,"_68":24,"_173":-5,"_263":273,"_17":274},{"_33":182,"_13":183,"_64":184,"_43":-5,"_35":36,"_167":185,"_169":80,"_37":269,"_39":187,"_66":67,"_171":188,"_68":24,"_173":-5,"_263":270,"_17":271},{"_33":175,"_13":176,"_64":177,"_43":-5,"_35":36,"_167":119,"_169":80,"_37":266,"_39":179,"_66":67,"_171":180,"_68":24,"_173":-5,"_263":267,"_17":268},{"_33":164,"_13":165,"_64":166,"_43":-5,"_35":36,"_167":168,"_169":80,"_37":262,"_39":26,"_66":67,"_171":172,"_68":24,"_173":-5,"_263":264,"_17":265},["D",1787863854757],"children",[],[165],["D",1788205851780],[],[176],["D",1787863898404],[],[183],["D",1787864951620],[],[191],["D",1787864990917],[],[199],["D",1787863849452],[],[207],["D",1787864744015],[],[214],["D",1787863922615],[],[221],["D",1787863924903],[],[228],["D",1787865023038],[],[235],["D",1787862662938],["D",1789545036283],{"_11":12,"_133":71},{"_11":12,"_128":129,"_70":71,"_30":69,"_130":131,"_132":24},{"_39":119,"_13":16,"_33":36,"_120":82,"_121":-5,"_122":-5,"_123":-5,"_124":-5,"_125":-5,"_126":14,"_127":12},"ctx",{"_54":-5,"_151":331,"_332":333,"_334":335,"_336":24,"_337":24,"_338":24,"_339":80,"_340":-5,"_341":-7,"_342":-7,"_343":26,"_344":24,"_345":24,"_346":-5,"_347":24,"_348":24,"_349":24,"_350":-5},"ENV",{"_302":303,"_304":305,"_306":16,"_307":308,"_309":310,"_311":312,"_313":314,"_315":316,"_317":318,"_319":320,"_321":322,"_323":324,"_325":326,"_327":328,"_329":330},"REACT_APP_SENTRY_DSN","https://586020d86f05489db63a554690acf1e9@o1398359.ingest.sentry.io/6724499","RENDER_GIT_COMMIT","811aa22498c9b0b5a47e8d778de579cdb990f332","RENDER_GIT_BRANCH","POSTHOG_API_TOKEN","phc_nmoyuAkStEZuUp69rYlg1DZgrIt19Gx0Y4Hwf8tGgJP","POSTHOG_API_HOST","https://wart.val.town","WEB_API_TEMPLATE","https://{val}.web.val.run","APP_URL","https://www.val.town","MODULE_URL","https://esm.town","VANITY_WEB_API_TEMPLATE","https://{label}.val.run/","DENOLS_URL","https://lsp.val.town","NODE_ENV","production","CLOUDFLARE_TURNSTILE_SITE_KEY","0x4AAAAAACtaIZJH7s9rBMZ6","GOOGLE_CLIENT_ID","875989696899-jtsqkhlbdv5jrlsk8631p7ipjg3c6o62.apps.googleusercontent.com","STRIPE_PUBLISHABLE_KEY","pk_live_51N2zvWEpZTTLipIkVEwXrUG9AT0yeOOEYxszarZT1jpsZV58Ly6wlg3SltZpHN26CUalOBmUSY7adQRQiSkCZz2500qSXNkf8Y","API_URL","https://api.val.town",[],"preferencesKeybindings","default","preferencesTheme","light","preferencesFixers","preferencesCopilot","preferencesSelectionAi","preferencesCloseBrackets","preferencesChatPanelState","preferencesDprint","preferencesDenofmt","notificationCount","flagDenols","isBusinessOrgMember","businessOrgHandle","isBusinessOrgAdmin","isHigherTownieTier","navCollapsed","currentOrgId"]

History · _probe.ts