\n {title === \"Val Town Blog\" ? title : `${title}

[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6,"_7":8,"_9":10},"actionData","errors","root",{"_1626":1627,"_1628":1629},"routes/_app.x.$handle.$name",{"_103":102,"_32":46,"_112":93,"_27":113,"_13":14,"_114":-5,"_11":115,"_116":117,"_118":119,"_120":121,"_122":123,"_20":-5,"_124":-5,"_125":-5,"_126":25,"_127":128,"_129":61,"_130":-5,"_131":-5,"_132":133},"routes/_app.x.$handle.$name.branch.$branch.version.$version.diff",{"_11":12,"_13":14,"_15":16,"_17":-5,"_18":-5},"project",{"_44":45,"_32":46,"_39":47,"_48":-5,"_49":25,"_50":51,"_52":51,"_53":54,"_55":56,"_18":57,"_58":59,"_60":61,"_62":-5,"_63":64,"_65":66,"_67":68,"_69":70,"_71":72,"_73":74,"_75":-5,"_76":77,"_78":77,"_79":80,"_81":80,"_82":80,"_83":77,"_84":77,"_85":86,"_87":88,"_89":90,"_91":25},"version",824,"changes",[19],"aiChatInfo","userId",{"_20":21,"_22":23,"_24":25,"_26":25,"_27":28,"_29":28},"file",{"_30":31,"_32":33,"_34":35,"_36":38,"_39":40,"_41":14,"_42":43,"_24":25},"previous",{"_30":31,"_32":33,"_34":35,"_36":37},"deleted",false,"deletedHere","path","components/Head.tsx","latestPath","fileId","d8758240-119b-11f0-b6de-569c3dd06744","name","Head.tsx","fileType","script","content","/** @jsxImportSource https://esm.sh/react@18.2.0 */\nimport { readFile } from \"https://esm.town/v/std/utils/index.ts\";\nimport { BLOG_URL, SITE_DESCRIPTION, SITE_TITLE } from \"../utils/constants.ts\";\nimport type { BlogPost } from \"../utils/types.ts\";\nimport { BLOG_DID } from \"../atproto/constants.ts\";\nimport { makeTID } from \"../atproto/lib.ts\";\n\nconst css = await readFile(\"styles/index.css\", import.meta.url);\nconst hlcss = await readFile(\"styles/highlight.css\", import.meta.url);\n\nexport default function Head({ post }: { post?: BlogPost }) {\n const title = post?.title ?? SITE_TITLE;\n const description = post?.description ?? SITE_DESCRIPTION;\n let ogImage: string | URL;\n if (post?.thumbnail) {\n ogImage = post.thumbnail;\n } else {\n ogImage = new URL(\"/og-image.png\", BLOG_URL);\n ogImage.searchParams.append(\"title\", title);\n }\n\n const canonicalURL = post?.link\n ? new URL(post.link, BLOG_URL).href\n : BLOG_URL;\n const author = post?.author ?? \"Val Town Team\";\n\n return (\n \n \n \n \n\n \n {title === \"Val Town Blog\" ? title : `${title} | Val Town Blog`}\n \n \n \n {post?.pubDate && (\n \n )}\n\n {/* RSS */}\n \n\n {/* Open Graph / Facebook */}\n \n \n \n \n {post?.pubDate && (\n \n )}\n {post?.author && (\n \n )}\n\n \n \n\n {/* Twitter */}\n \n \n \n \n \n\n {/* AT Protocol (Bluesky) */}\n {/* Publication discovery hint: https://standard.site/docs/verification/#discovery-hint */}\n \n {/* Document verification: https://standard.site/docs/verification/#document-verification */}\n {post?.pubDate && (\n \n )}\n\n {\n /* Sitemap\n \n */\n }\n \n\n> As a startup we value the ability to ship features quickly, and OTs were unnecessarily complex for our problem space. So we built a custom multiplayer system that's simpler and easier to implement.\n>\n> At the time, we weren’t sure building this feature was the right product decision. No one was clamoring for a multiplayer design tool—if anything, people hated the idea.\n>\n> ...Our bet paid off, and these days it’s obvious that multiplayer is the way all productivity tools on the web should work, not just design.\n>\n> – Evan Wallace, Co-founder, Figma, [_How Figma’s multiplayer technology works_](https://www.figma.com/blog/how-figmas-multiplayer-technology-works/)\n\nWe too built something that nobody was clamoring for, but that we hope will seem obvious in the retrospect. Experienced programmers will feel right at home in Val Town Projects – branching, forking, merging, and pull requesting – collaborating directly within the same platform that you're iterating and deploying on.\n\nNew programmers might never need to learn git at all. I can still remember the frustration of learning git for the first time, and I'm excited to think that we might be able to save a new generation of programmers from that pain. Particularly in this new age of LLM programming, we think there's an opportunity to build something simpler.\n\nAnd if you really need git, Projects are git-friendly, so we will soon build a way to sync them to and from a git repo. We're racing to release a Beta API for Projects, which will allow you to have full control over your source code, sync to and from a git repo, and wherever else you want.\n\nLearn more about Val Town Projects [here in our docs](https://docs.val.town/projects).\n\n## What you can build\n\nUp till now, Val Town has excelled at small pieces of code, such as little integrations and automations, such as [social media monitoring, data sync, and single-page sites](https://www.val.town/examples/use-cases). We're excited to see what you can build with a much larger scope in Projects, such as:\n\n- internal tools\n- durable workflows\n- a backend for your app\n- API prototypes – like [Campsite](https://www.campsite.com/blog/how-we-prototype-integrations-with-val-town), [Glif](https://www.val.town/u/jamiedubs)\n- a starter template for your API – like [PostHog](https://posthog.com/tutorials/rss-item-capture), [Framer](https://www.framer.com/developers/fetch/examples), [tldraw](https://x.com/tldraw/status/1869770077828272134), [Browserbase](https://www.val.town/u/browserbase),\n- multi-page, full-stack apps\n\nWe want to highlight the biggest Project built so far. [Nathanael](https://nates.foo/about) is building [JournalGroove](https://www.journalgroove.org/), an app to help people get more insight from their journaling practice. The frontend of the app is simply Notion, and the backend is hosted on Val Town. Nathanael has a series of [cron vals](https://docs.val.town/vals/cron/) that run throughout the day to process your latest journal entries and deliver personalized, AI-generated reports back into your private Notion space.\n\nBefore Projects, Nathanael had dozens of vals scattered across folders, and versioned separately. He was able to migrate them all into a single Val Town Project in about 30 minutes.\n\n

\"\" — \n [JournalGroove](https://www.journalgroove.org/) - a project to help people\n get more insight from their journaling practice by [Nathanael\n Silverman](https://nates.foo/about)\n

\n\n> \"Val Town enabled me to grow this from an idea into a fully-fledged project much faster than I would have expected. I like that Projects are separate from my other random vals. And I look forward to using branches, eventually collaborating with other people.\"\n>\n> \\- [Nathanael Silverman](https://nates.foo/about)\n\n## Start building\n\nYou can start building with Projects right now.\n\n

\n\nWe can't wait to see what you build! If you're new to Val Town, you can [sign up free here](https://www.val.town/auth/signup).\n",[],[191,1025],"a745c90b-da0a-4019-ae2a-bf8a069ba40c","2025-01-22-val-town-newsletter-23.md","86bdbacc-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town Newsletter 23\ndescription: 'Projects beta, Deno 2, Use cases, and more'\npubDate: '2025-01-22T05:00:00.000Z'\nauthor: Steve Krouse\ntags: [newsletter]\n---\n\n🥳 Happy 2025! Last year, our product grew from a JavaScript playground to a stable FaaS (Functions as a Service) platform that thousands of engineers use in personal and professional contexts. Some highlights from 2024:\n\n- Raised [our seed round](/seed)\n- [Max](https://maxmcd.com/), [Jackson](https://jxnblk.com/), [Justin](https://just-be.dev/), [Sophie](https://sophiehouser.com/), and [Charmaine](https://www.charmaineklee.com/) joined our team\n- Our infra became [fast](https://github.com/val-town/val-town-product/discussions/113#discussioncomment-10813787) and stable\n- Launched [Townie](/townie), a shockingly useful code assistant\n- 23k of you made 135k vals that've run 214m times\n\nWe have big plans for 2025. By the end of this year, we aim to be a mature cloud platform that serious engineering organizations use for core infrastructure. Here's some of the highlights:\n\n- Projects – groups of vals, files folders – with forks & 3-ways merges (currently in private beta)\n- Scalable infra & usage-based billing\n- Team accounts for companies to collaborate\n- Improvements to Discovery, Search, Townie, Observability\n- Continuing to [fast-follow with LLMs](/fast-follow/)\n- SOC 2\n\nWe have two new additions to the Val Town team! [Sophie](https://sophiehouser.com/) is a designer, programmer, and artist, who just moved from Berlin to join us here in Brooklyn. She's best-known for [a viral app she made as a teenager](https://sophiehouser.com/tampon-run-), but most recently worked with Nike and Delivery Hero. [Charmaine](https://www.charmaineklee.com/) is a technical product manager, with a deep background in developer tools. She's worked at Snapchat, Unity Labs, and Microsoft. We're thrilled to have them both on board.\n\n## Val Projects Beta\n\nVals are great for small projects, but what if you want to build something bigger?\n\n**Introducing Val Projects**, a new way to organize multiple vals, files, and folders into a single place. Where a Val Town Val is like a GitHub Gist, a Val Town Project is like a GitHub Repo.\n\n $\"Val$ \n\nProjects support:\n\n- Static assets, like images, videos, CSS, HTML, and more\n- Multiple files and folders\n- Scoped environment variables (that code outside the project _can't_ access)\n\nProjects will soon support:\n\n- Branches, forking, pull requests, and three-way merges\n- Townie, with the ability to understand and edit multiple files\n\nProjects are currently in private beta. We're looking for more passionate Val Town users to try it out and give feedback. If you're a Pro user, please reach out to steve@val.town, and we'll add you to the beta.\n\n## Templates\n\n\"What can I do with Val Town?\" We get this question a lot, so we made [a page to help answer it](https://www.val.town/examples/use-cases).\n\n $\"Val$ \n\n[Check it out](https://www.val.town/examples/use-cases) and see if it sparks any ideas for you! Maybe you'll find a val that's perfect for your needs and you can fork it and make it your own. If you don't find what you're looking for, let us know. If you want to submit a val to be included, we'd love that too!\n\n## Upgraded to Deno 2\n\n[Deno 2](https://deno.com/blog/v2.0) is out, and Val Town is now running it. We're currently on Deno v2.1.2, but you can always check which version we're running [here](https://www.val.town/v/std/deno_version).\n\nIn addition to some performance and security upgrades, Deno 2 also includes a couple exciting features:\n\n- `process` and `Buffer` are now globally available, which means you don't have to import them anymore, which increases compatibility with Node.js code ([example](https://www.val.town/v/stevekrouse/processBufferDeno2Example))\n- WebAssembly imports are supported out-of-the-box ([example](https://www.val.town/v/just_be/deno2wasmExample))\n\n## Updated pricing\n\nWe made three changes to our pricing:\n\n1. We now have a yearly plan for $100! This is a 20% discount on our monthly plan 🥳\n2. We've explicitly listed custom domains on the Pro plan, with a limit of 10\n3. We've changed the val runs limits from per minute to per day. Now you get:\n - Free: 100k runs / day\n - Pro: 1m runs / day\n\n $\"Val$ \n\nOne final word to the wise: we plan to increase our pricing soon, but existing customers will keep the same price, so sign up soon!\n\n## Blogs, podcasts, and livestreams\n\nThere's been a lot of great content about Val Town recently, a podcast, a livestream, 5 blogs, and (unfortunately) 2 post-mortems:\n\n- 📺 [Every Podcast: Do a 60-Minute Coding Task in 60-seconds](https://x.com/danshipper/status/1864324178901651828) with Steve Krouse\n- 📺 [Finsweet livestream: Exploring Val Town to supercharge your Webflow projects ](https://www.youtube.com/watch?v=Cm-yqc8Hve8) ft. Steve Krouse\n- 📝 [Turning data into calendar feeds](https://mattrossman.com/2025/01/06/turning-data-into-calendar-feeds) by Matt Rossman about how to get any data onto any calendar app supports iCal (which is all of them) super quickly\n- 📝 [Reimplementing Unsplash Source](https://paul.af/reimplementing-unsplash-source) by Paul Esch-Laurent about reimplementing the now discontinued Unsplash Source API\n- 📝 [Using Val Town to chart dependency bloat](https://blog.val.town/blog/devstats/) by Tom MacWright about [Devstats](https://www.val.town/v/tmcw/devstats), a val lets you post statistics from your GitHub Actions runs to build charts of change over time\n- 📝 [Bandcamp wrapped](https://macwright.com/2024/12/06/bandcamp-wrapped) by Tom MacWright about a val that generates a Spotify Wrapped-style report for your Bandcamp purchases\n- 📝 [What we learned copying all the best code assistants](https://blog.val.town/blog/fast-follow/) by Steve Krouse\n- 📝 [Post-mortem: esm.sh breaking changes](https://blog.val.town/blog/post-mortem-esm-sh/) – esm.sh unexpectedly released breaking changes that impacted 2% of Val Town users\n- 📝 [Post-mortem: Exhausted Host Connection Pool](https://blog.val.town/post-mortem-host-connection-pool) – one of our servers had degraded performance due to a deadlock\n\n## Other updates\n\n- The logged-in Val Town homepage is now the dashboard, which shows your recent vals and projects. You can find the old homepage at [val.town/home](https://www.val.town/home).\n- We updated val Run & Save buttons to be easier to find, and provide more immediate feedback\n\n## 🛣️ Roadmap\n\n- Val Projects\n- Faster & scalable runtime\n- Observability upgrades: searchable & filterable logs and traces\n- Metrics to track your usage and limits\n- New Explore page for discovering vals\n- Improved code search\n- Comments on pull requests\n- Team accounts\n- Ability to follow a Val Town user\n- SOC2 compliance\n- …and much more!\n\nPlease [make feature requests and vote](https://github.com/val-town/val-town-product/discussions?discussions) on what you'd like to see next.\n\n## 🌟 Community vals\n\n- Steve relaunched [Cerebras Coder](https://www.val.town/v/stevekrouse/cerebras_coder), an app that generates websites in under a second. Over 100k projects have been created with it!\n- [Paul](https://www.val.town/u/paulkinlan), who leads Chrome’s Developer Relations team, made an [Email Summary Service for newsletters](https://paul.kinlan.me/projects/email-summary-service/)\n- [Michael](https://www.val.town/u/michaelfromyeg) from Notion made a [Notion web clipper](https://x.com/michaelfromyeg/status/1873087505224089742) to save Notion content to Wallabag\n- [Ellie Huxtable](https://www.val.town/u/ellie/forks) added a [“recent listening” widget](https://x.com/ellie_huxtable/status/1866590562620420550) to their website\n- [Sam](https://www.val.town/u/samkingco) powers the “Latest YouTube video” part of his site with data from a val (post on bluesky)\n- [Bao](https://www.val.town/u/bao), an iOS engineer at Notion, created the [Cipher Challenge](https://www.val.town/v/bao/cipher), a game to decode a letter substitution cipher as quick as you can. It's even multiplayer! ([post on bluesky](https://bsky.app/profile/baolei.org/post/3leqdekcylk2g))\n- [Bryce](https://www.val.town/u/amoeba) made [a daily email digest of a github repo’s activity](https://www.val.town/v/amoeba/githubRepoDailyEmailDigest) ([post on bluesky](https://bsky.app/profile/stevekrouse.com/post/3ldsxj5xgrk2l))\n- [Morry](https://wttdotm.com/) made a website where people can [take pictures of his cats in his window and submit them via QR Code + email](https://x.com/WTTDOTM/status/186654745801826339)\n- [Prasham](https://www.val.town/u/prashamtrivedi) made a [Val Town README Generator](https://www.val.town/v/prashamtrivedi/valreadmegenerator) - type in your username and val’s name and it’ll generate a readme with AI you can copy and paste\n- [Max](https://www.val.town/u/maxm) used vector embeddings to get [\"vibes\" search on emojis](https://www.val.town/v/maxm/emojiInstantSearch)\n- [John](https://www.val.town/u/johnphamous) made cron vals to [snipe hard to get nintendo museum tickets](https://x.com/JohnPhamous/status/1877781209750913223)\n- [Jan](https://www.val.town/u/yawnxyz) made a [fresh - an invoice generator](https://www.val.town/v/yawnxyz/fresh)\n- [Prasham](https://www.val.town/u/prashamtrivedi) made [a calendar app for viewing ICS files](https://www.val.town/v/prashamtrivedi/icsviewer)\n- [Steve](https://www.val.town/u/stevekrouse) made a val that shows [the top 100 Val Town users by number of public vals](https://www.val.town/v/stevekrouse/prolific) - did you make the cut? ([tweet](https://x.com/stevekrouse/status/1879282475823952138))\n- [Steve from tldraw](https://www.val.town/u/steveruizok) made an [example custom endpoint for tldraw computer’s data component](https://x.com/tldraw/status/1870477971519406316). Jordan agrees that [tldraw and Val Town are besties](https://bsky.app/profile/jdan.me/post/3lefgekm73s2u)\n- [Victor](https://www.val.town/u/vawogbemi/) made Let Him Cook, which lets you [query database with natural language](https://www.val.town/v/vawogbemi/lhc) via email\n- [Mixer](https://www.val.town/u/mixer) made a val to show their [Spotify Listening History](https://mixer-listening.web.val.run/)\n- [Trent](https://www.val.town/u/tr3ntg) built [a val that calls him via Twilio whenever it receives a webhook](https://www.val.town/v/tr3ntg/makeCallOnEmail)\n- [Charmaine](https://www.val.town/u/charmaine) [made Townie her personal illustrator](https://x.com/charmaine_klee/status/1880307658013466712)\n- [Steve](https://www.val.town/u/stevekrouse) made [an Anonymous doc publishing tool](https://x.com/stevekrouse/status/1874921337300623825)\n- [Colin](https://www.val.town/u/colin) from Zod built Josh Goldberg [a mood diary](https://x.com/stevekrouse/status/1865049638429946311)\n- [David](https://www.val.town/u/dthyresson) made [AI Rapped 2024](https://dthyresson-ai_rapped_2024.web.val.run/)\n- [Danny](https://www.val.town/u/postpostscript) made Interruptible Chain, a simple interface for [pausing and resuming execution chains](https://www.val.town/v/postpostscript/interruptibleChain)\n- [Vlad](https://www.val.town/u/vlad) made a val that converts [Bluesky Threads to RSS](https://vlad-blueskythreadtorss.web.val.run/)\n- [Nolen](https://www.val.town/u/itseieio) [made the opengraph image for everyuuid.com](https://x.com/itseieio/status/1866167891550179352) with [a Townie app he made](https://www.val.town/v/itseieio/opengraphImageCreator)\n- [Charmaine](https://www.val.town/u/charmaine) added a new [Reddit Keyword Alerts template](https://www.val.town/v/valdottown/templateRedditAlert) with an [SMS version](https://www.val.town/v/charmaine/redditKeywordSMS)\n- [Victor](https://www.val.town/u/vawogbemi/) made [BYOB, Build Your Own Bot](https://www.val.town/v/vawogbemi/BYOB)\n- [Boris](https://www.val.town/u/boristane/) made a [cyberpunk weather display](<(https://x.com/boristane/status/1868010048225776000)>) with Townie\n- [Jackson](https://www.val.town/u/jxnblk) made GitHub Sync to [sync vals from Val Town to GitHub and back](https://www.val.town/v/jxnblk/gitHubSync)\n- [Chet](https://www.val.town/u/chet) made a [Landscape Lighting Calculator](https://x.com/ccorcos/status/1867769136274977119) with Townie in minutes\n- Mariano [showed his users how to extend his app with a val](https://bsky.app/profile/marianoguerra.org/post/3lczyqqg6gs2y)\n- [Danny](https://www.val.town/u/postpostscript) made expiringBlob, [create-and-forget blobs using UUIDv7](https://www.val.town/v/postpostscript/expiringBlob)\n- [Adtiya](https://www.val.town/u/adiadd) from AWS made a site that [gives you ideas from arXiv papers](https://x.com/adiaddxyz/status/1871302483022426476)\n- [Peter](https://www.val.town/u/peterqliu) made a dashboard that shows [the most popular wikipedia articles in a given time range](https://peterqliu-ww_top1k_dashboard.web.val.run/)\n- [Inferable](https://www.val.town/u/inferable) made an [integration with Val Town to create AI agents](https://x.com/inferablehq/status/1879876394001555893)\n- [John](https://x.com/John__Wetzel) embraced Scott Belsky’s DIY software and made a [1000 hours outside challenge tracker](https://x.com/John__Wetzel/status/1876000610413396075)\n- [Jamie](https://www.val.town/u/jamiedubs) from [Glif](https://glif.app/) made [a val that fetches details about media files](https://www.val.town/v/jamiedubs/mediainfo) (images, video, audio, etc) using Mediainfo.js. e.g. codecs used, duration, file sizes, ID3 tags, etc.\n- [mint](https://www.val.town/u/mintexists) made a [Annas Archive/Libgen+ OPDS library catalog](https://www.val.town/v/mintexists/libgenopds) ([tweet](https://x.com/eva_exists/status/1876104402605617277))\n- [Dave](https://www.val.town/u/dave_smith) made [a holiday drink menu](https://www.val.town/v/dave_smith/holidaycocktailmenu) with Townie ([post on bluesky](https://bsky.app/profile/davesmith.dev/post/3ldycgviygs2v))\n- [Dean](https://val.town/u/loading) made [a game out of captchas](https://loading-captcha.web.val.run/)\n- Ben made [a site for friends to cast predictions on the game awards](https://gameawards.mitchinson.dev) ([post on bluesky](https://bsky.app/profile/mitchinson.dev/post/3ld7asiu4zs23))\n- [Arfan](https://www.val.town/u/arfan) made [a kanban board todo list](https://www.val.town/v/arfan/kanbanTodoList) (as of this writing it was broken, but [this fork works](https://www.val.town/v/stevekrouse/kanbanTodoList))\n- [dh1011](https://www.val.town/u/dh1011) made [a tool to extract important quotes from an article](https://www.val.town/v/dh1011/quoteExtractor)",[],[191,1032],"4bab746f-0da7-4e3d-a9f8-5189f68a06c6","2025-01-16-post-mortem-esm-sh.md","86bd9e0c-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: 'Post-mortem: esm.sh Breaking Changes'\ndescription: >-\n Some vals experienced lockfile and React errors due to breaking changes in\n esm.sh\npubDate: '2025-01-16T05:00:00.000Z'\nauthor: Steve Krouse\ntags: [engineering]\n---\n\nYesterday [esm.sh](https://esm.sh) deployed breaking changes that caused two kinds of errors for our users: lockfile errors and client-side React errors. The first error was resolved by esm.sh rolling back the change related to React version pinning. The second error was resolved by us manually regenerating all affected lockfiles. We are working to upgrade our systems with the esm.sh maintainer to ensure this doesn't happen again.\n\n### Timeline\n\n

\n $\"Chart$ \n

\n\n- **12:00am ET** - Vals started experiencing lockfile errors. Some vals using client-side React stopped working.\n- **4:09am ET** - esm.sh maintainer [announced the changes in the Val Town Discord](https://discord.com/channels/1020432421243592714/1020432421243592717/1329014642223218741)\n- **9:36am ET** - esm.sh begins [rolling back the change related to React version pinning](https://github.com/esm-dev/esm.sh/pull/1032)\n- **10:34am ET** - All client-side React vals resume working\n- **10:39am ET** - We stopped sending unnecessary lockfile error emails to users (because the lockfiles would automatically regenerate)\n- **4:00pm ET** - We manually regenerated the lock files for all remaining affected vals. Most lockfile errors were resolved.\n\n### Impact\n\nThese breaking changes impacted approximately 2% of Val Town users.\n\n#### Lockfile errors\n\nThe lockfile errors happened the first time a val using esm.sh was run yesterday, because [esm.sh changed the URL structure of their imports](https://github.com/esm-dev/esm.sh/releases/tag/v136), which results in a lockfile integrity error.\n\nSo on first run, the val would throw an error. Then we'd send an error email to the user. Some users received many such error emails. The lockfiles would automatically regenerate, so the error emails were unnecessary. At 10:39am we decided to silence lockfile error emails.\n\nThis lockfile error would only happen the first time a val was run after the esm.sh change. However, we still wanted to prevent this error, so we manually regenerated the lockfiles for all remaining affected vals. After 5pm ET, almost all lockfile errors were resolved. We are still tracking some remaining few errors per hour that are still happening.\n\n#### Client-side React errors\n\nThe client-side React errors were unrelated to the lockfile errors, but were also bundled into the v136 esm.sh release. In v136, esm.sh stopped pinning `react-dom` to use the same version of `react` automatically, which is behavior that many vals were unintentionally relying on. This issue was more easily resolved, when by simply having [esm.sh roll back that change](https://github.com/esm-dev/esm.sh/pull/1032).\n\n### Next Steps\n\nWe take stability extremely seriously and we're sorry for the downtime this caused. We're taking steps to ensure this doesn't happen again:\n\n1. We're working with esh.sh’s creator to ensure this doesn't happen again.\n2. Longer term, we're planning to replicate all your backend dependencies on our infrastructure, so we can control any infrastructure updates that may affect val reliability. We're also exploring patterns to make frontend dependencies more reliable.\n3. Shorter term, we're going to rewrite lockfile errors to be less confusing, so that if val has integrity errors, it'll be much clearer what happened and what action you need to take, if anything.\n\nIf you have any questions or concerns, please reach out to me directly at steve@val.town.\n",[],[191,1039],"15cbc39d-7182-4240-ae6e-2ac824b14dfd","2025-01-03-fast-follow.md","86bd3534-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: What we learned copying all the best code assistants\ndescription: >-\n From GitHub Copilot to ChatGPT to Claude Artifacts, how Val Town borrowed the\n best of all the code generation tools\npubDate: '2025-01-03T05:00:00.000Z'\nauthor: Steve Krouse\ntags: [engineering]\n---\n\nSince the beginning of Val Town, our users have been clamouring for the state-of-the-art LLM code generation experience. When we launched our code hosting service in 2022, the state-of-the-art was GitHub Copilot. But soon it was ChatGPT, then Claude Artifacts, and now [Bolt](https://bolt.new/), [Cursor](https://www.cursor.com/), and [Windsurf](https://codeium.com/windsurf). We've been trying our best to keep up. Looking back over 2024, our efforts have mostly been a series of _fast-follows_, copying the innovation of others. Some have been successful, and others false-starts. This article is a historical account of our efforts, giving credit where it is due.\n\n### GitHub Copilot Completions\n\nThe story starts, of course, with GitHub Copilot. From day 1, Val Town users asked for a GitHub-Copilot-like completions experience.\n\nWe were wary of building this ourselves, but one day we stumbled upon Asad Memon’s [codemirror-copilot](https://github.com/asadm/codemirror-copilot), and hooked it up. That gave us our first taste of LLM-driven autocomplete, but behind the scenes, it was using ChatGPT. The prompt essentially asked ChatGPT to cosplay as an autocomplete service and fill in the text at the user’s cursor. So it was fairly slow, occasionally the model would forget its role and do something unexpected, and it didn't have the accuracy of a purpose-built autocomplete model.\n\nWe wanted a faster, more accurate autocomplete sytem, one that used a model trained for the task - which is technically called ['Fill in the Middle'](https://arxiv.org/abs/2207.14255). Finding an option that we could use within a product like Val Town was tricky – Copilot and most of its competitors lack documented or open APIs. But [Codeium](https://codeium.com/) did, and they also had very good accuracy and performance. We [launched Codeium completions](https://blog.val.town/blog/val-town-newsletter-16/#-codeium-completions) in April 2024 and open-sourced our [codemirror-codeium](https://github.com/val-town/codemirror-codeium) component. It's been pretty great. It's enabled by default for new users.\n\n![Codeium](https://assets.blog.val.town/legacy/val-town-newsletter-16/codeium.png)\n\n### ChatGPT\n\nThen came ChatGPT. We found our users asking it to write Val Town code, and copying and pasting it back into Val Town. We figured we could automate that process for our users: provide an interface with a pre-filled system prompt and a one-click way to save the generated code as a val. The [first version of Townie](https://blog.val.town/blog/val-town-newsletter-18/#-townie) was born: a simple chat interface, very much inspired by ChatGPT, powered by GPT-3.5.\n\n![Townie](https://assets.blog.val.town/legacy/val-town-newsletter-18/townie.png)\n\nIt was just ok. It didn't get much use, mostly because it was hard to iterate on its results. Getting good results from an LLM usually requires a conversation because programming-via-English is pretty imprecise, and you need follow-up requests to clarify your needs.\n\n### ChatGPT Tool Use\n\nEarlier this year, ChatGPT Function Calling, now called 'tool-use', was seen as the next big thing. The promise was that with a good OpenAPI spec, AI would be able to do just about anything on Val Town. So we dutifully cleaned up our OpenAPI spec, and [rebuilt Townie around it](https://blog.val.town/blog/openapi/#our-ai-townie-can-now-call-our-rest-api).\n\n\n\nIt was, ahem, fine. Function calling was a disappointment. You do all the work to provide the LLM with a strict definition of what functions it can call and with which arguments. But even with all of that, the LLM would hallucinate functions that didn't exist. Function calling has improved since, with the introduction of [Structured Outputs](https://platform.openai.com/docs/guides/function-calling#structured-outputs).\n\nBut for us, the issue was that the interface was too generic. In theory, it was capable of doing anything (editing your blobs or sqlite data), but it wasn't very useful at any specific thing. Most notably, it wasn't a good interface for iterating on code. It could write a first version of code, but it wasn't optimized to let you run that code, see the output, debug it, let you ask the AI for more help. In other words, the feedback loop was bad.\n\n### Claude Artifacts\n\nWe had begun to see the potential of Claude for code generation with the amazing results produced by [Websim](https://websim.ai/). But it was the [launch of Claude 3.5 Sonnet and Claude Artifacts](https://www.anthropic.com/news/claude-3-5-sonnet) that really got our attention. Claude 3.5\nSonnet was dramatically better at generating code than anything we'd seen before. It blew all of our minds. And Claude Artifacts solved the tight feedback loop problem that we saw with our ChatGPT tool-use version. And thus after about a month of prototyping and building, the [current version of Townie](https://blog.val.town/blog/codegen/) was born in August 2024.\n\nFor a couple weeks there, it felt like we had one of the best tools in the space. Townie can generate a fullstack app, with a frontend, backend, and database, in minutes, and fully deployed. The space has since gotten crowded. Live by the fast follow; die by the fast follow.\n\n### Our Contributions\n\nWhile we were out in front, we invested in trying to stay there, and we made some contributions of our own that have since found there way into other tools in the space.\n\n#### Speed\n\nThe biggest problem with all current codegen systems is the speed of generation. It takes minutes to generate just a couple hundred lines of code. If you regenerate the whole file every time – which is how most systems work – that means minutes between every feedback loop. (Not to mention the cost of regenerating the whole file every time, even when you are making a small change.)\n\nWe worked hard to get the LLM producing diffs, based on [work we saw in Aider](https://aider.chat/docs/unified-diffs.html). We were able to get it working most of the time, but not reliably enough. It's now off by default, but you can ask Townie to \"reply in diff\" if you'd like to try your luck with it.\n\nOur system prompt has always been open (you can view it in your Townie settings), so you can see how we're doing that. Here's the relevant section:\n\n````md\nFollow the requirements above and respond by generating code in a format based on whether or not the user explicitly requests diff format in their most recent prompt:\n\n- If the user does not explicitly request diff format in their prompt, generate the entire val:\n Use as the basis for generating code if it is provided.\n Write code that is complete and directly runnable.\n DO NOT omit code or use comments such as \"more content here\" or \"code remains unchanged.\"\n Write the code in `val code fences.\nInclude the val type as metadata on the code fence, e.g.: `val type=script\n If this is a new val, decide what val type is appropriate based on the user's prompt. Default to choosing http type vals unless the user has requested specific functionality that requires a different type.\n\n- If the user requests diff format in their prompt, follow these steps:\n Write a valid unified diff with change hunk headers. The file headers can be omitted.\n Base the diff off of the tags below.\n Use the ```diff language code fence.\n````\n\nWe've gotten scared off of investing more time in diffs right now, but I expect it may have been solved by others in the space already, or will be shortly. Anthropic's long-rumored \"fast-edit mode\" solves this problem in one fell swoop. OpenAI launched their own [Predicted Outputs](https://platform.openai.com/docs/guides/predicted-outputs), which is also compelling, but then we'd have to switch to OpenAI. Or maybe the solution is simply faster models, smaller, mini-models, or faster chips, like Groq or Cerebras. A couple weeks ago I built [Cerebras Coder](https://cerebrascoder.com) to demonstrate how powerful an instant feedback loop is for code generation. [Try it out yourself](https://cerebrascoder.com) or [fork it here](https://www.val.town/v/stevekrouse/cerebras_coder).\n\n

\n\nDeepSeek [recently open-sourced an almost-Sonnet-3.5-level model that's twice as fast and trained for only $6m](https://x.com/deepseek_ai/status/1872242657348710721). A boy can dream of a world where Sonnet-3.5-level codegen (or even smarter!) is available on a chip like Cerebras at a fraction of Anthropic's cost. I think that would unleash a whole new class of innovation here.\n\n#### Autodetecting errors\n\nWe did contribute one possibly-novel UI interaction, where the LLM automatically detects errors and asks you if you'd like it to try to solve them. We detect server-side errors by polling our backend for 500 errors in your logs. We detect client-side errors in the iframe by prompting Townie to import [this client-side library](https://www.val.town/v/std/catch), which pushes errors up to the parent window.\n\n

\n\nIt's not _particularly_ novel (in that others would have thought of this if we didn't), but maybe the folks at [Anthropic](https://support.anthropic.com/en/articles/9949260-try-fixing-with-claude-for-artifact-errors) or Bolt saw our implementation and it inspired their own. I'd like to think we're not _only_ free-riding in this space.\n\n### Hosted runtime and included APIs\n\nMaybe some of our UI ideas made it into GitHub Spark too, including deployment-free hosting, persistent data storage, and the ability to use LLMs in your apps without your own API key – their versions of [@std/sqlite](https://docs.val.town/std/sqlite/) and [@std/openai](https://docs.val.town/std/openai/), respectively. In other words, you can say, \"make me a ChatGPT clone with persistent thread history\", and in about 30 seconds, you'll have a deployed app that does exactly that.\n\n

\n\nBut we're not the first hosting company to provide an LLM tool; that\nhonor likely goes to Vercel's [v0](https://v0.dev/).\n\n### Cursor\n\nThe next big thing was [Cursor](https://www.cursor.com/). I must admit that I never personally fell in love with it, but given how many people I respect love it, I think that's a me-problem. I think Cursor is best for development in larger codebases, but recently my work has been on making vals in Val Town which are usually under 1,000 lines of code. (Our upcoming launch of multi-file Projects, now in private beta, will change this.) However Cursor is a real pioneer in the space, and has some UI interactions there that we have an eye to copy.\n\n### Windsurf\n\nOver the holiday, I fell in love with [Windsurf](https://codeium.com/windsurf) by the folks at Codeium. Its Cascade feature is a chat interface, which has tool use and multi-turn agentic capabilities, to search through your codebase and edit multiple files. It feels a bit like we're coming full-circle back to when we did our tool-use version of Townie. However, I think we now all understand that you can't simply give your OpenAPI spec to an LLM and expect good results. The magic of Windsurf is that they carefully crafted what actions their agent can take, and that it can take multiple actions in a row without your input.\n\nI am salivating at the idea of giving Townie some of these capabilities. Imagine if Townie could search through all public vals, and maybe even npm, or the public internet, to find code, docs, and other resources to help you.\n\n### Devin\n\nWatching Windsurf take multiple actions on my behalf without my input is very inspirational. I'm dreaming of a world where Townie not only detects errors, but also automatically tries to fix them, possibly multiple times, possibly in parallel across different branches, without any human interaction. Here, of course, we'd be getting into territory mostly explored by the folks at [Devin](https://devin.ai/).\n\nFor starters, we could feed back screenshots of the generated website back to the LLM. But soon you'd want to give the LLM access to a full web browser so it can itself poke around the app, like a human would, to see what features work and which ones don't. Maybe then it'd even write some tests, also like a human would, to make sure things don't break as it continues to iterate.\n\nI have a vague sense by the end of this year that you'll be able to tell Townie to \"make a fully realistic Hacker News Clone, with user accounts, nested comments, upvotes, downvotes\" and it could iterate for potentially hours on your behalf. You could even go to bed and wake up with it done.\n\n### Collaboration and competition\n\nIs this fast-following competitive or is it collaborative? So far it's been feeling mostly collaborative. The pie is so freaking large -- there are millions and maybe billions who are jumping at the chance to code -- that we're all happy to help each other scramble to keep up with the demand. I love that, and hope it remains this way. We at Val Town certainly don't keep (m)any secrets. Our system prompt is open, and we blog about all our interesting technical choices. This very post is a case in point.\n\n### Should we bow out?\n\nAll this copying, and how fast everything is moving begs the question: Should we get out of this race entirely? How can we hope to compete against better funded competitors? Should we instead focus on improving our core differentiator, and do a better job integrating with AI editors like VSCode, Cursor, Windsurf, and Bolt? Maybe! We're planning to dip our toes into integrating: we plan to improve our local development experience, which would allow editors like VSCode, Cursor, and Windsurf to directly edit files in Val Town. We also plan to improve our API, so tools like Bolt could \"deploy to Val Town\", like they currently deploy to Netlify.\n\nHowever, it still feels like there's a lot to be gained with a fully-integrated web AI code editor experience in Val Town – even if we can only get 80% of the features that the big dogs have, and a couple months later. It doesn't take _that_ much work to copy the best features we see in other tools. The benefits to a fully integrated experience seems well worth that cost. In short, we've had a lot of success fast-following so far, and think it's worth continuing to do so.\n\n### Townie\n\nIf you've made it this far in the article, you should really [try out Townie](https://www.val.town/townie). It's still is one of the best tools to create fullstack web apps. Make yourself a ['what did I work on today' app that pulls from Linear and GitHub](https://www.val.town/v/danphilibin/what_did_i_work_on_today) or [a tool to extract dominant colors from an image](https://x.com/destroytoday/status/1856709997737984089) or [an AI clone for your personality](https://deeperfates.com/). Your imagination is the limit. And if you do, please let me know (steve@val.town) what features from other tools you'd like to see in Townie. We're eager to learn from you.\n\n_Thanks [Tom MacWright](https://macwright.com/), [JP Posma](https://janpaulposma.nl/), and [Simon Willison](https://simonwillison.net/) for feedback on drafts of this article._",[],[191,1046],"0209afcb-292d-4b8f-9704-51680c822330","2024-12-31-post-mortem-host-connection-pool.md","a13fc228-7bd9-11f1-8271-1607ee4eb77e",["D",1784053703462],"---\ntitle: 'Post-mortem: Exhausted Host Connection Pool'\ndescription: A single degraded host impacted certain Vals for some time.\npubDate: '2024-12-31T05:00:00.000Z'\nauthor: Max McDonnell\ntags: [engineering]\n---\n\n_An\n[earlier version of this post](https://github.com/val-town/val-town-blog/commit/1059e2aff2973aa841b06c46c1ce6c3e680d2d45)\nposited that the cause of this downtime was a degraded host network. We now\nthink that initial hypothesis was incorrect, so we have updated this post with\nour current understanding of the cause._\n\n### Timeline\n\nAt 12:15am ET a flurry of requests made many parallel edits to a Val. This\npattern of requests caused a few database queries to deadlock. Our database\nconnection pool lacked query timeouts and these locked connections permanently\noccupied slots in our connection pool.\n\nOne of our servers entirely exhausted its database connection pool and started\nto experience long delays on both inbound and outbound requests. Vals on that\nserver experienced network timeout errors and other issues. Some existing\nconnections continued to serve requests and some inbound and outbound requests\nwere successful, but most requests stalled indefinitely in a pending state. Our\nhealth checks for this server did not check database state so the server\nremained in degraded state for some time.\n\nThis persisted until 7:04am ET when we manually terminated the server after a\nuser report.\n\n### Impact\n\nWe make our best effort to route Val requests to the same hosts so that they can\nbe picked up by a warm worker. As a result, certain Vals were repeatedly routed\nto this host and experienced extended degradation. Those Vals failed to start\nentirely, or experienced issues when trying to make outbound requests.\n\n### Next Steps\n\nWe care deeply about reliability and are working to prevent this type of failure\nin the future.\n\n- We've updated our health checks to incorporate the health of the database\n connection pool, and we've added timeouts to pool queries.\n- We've identified the cause of the deadlock, added tests, and changed the logic\n to be sure it can't deadlock again.\n- Our external val execution health checks were\n being routed to other hosts. We've updated these checks to make sure that\n every host is hit with each health check request.\n\nMore generally, the lack of timeouts caused the server to reach capacity without\nexhibiting backpressure. We'll be doing a more extensive review to be sure that\nservers reaching capacity start to reject requests instead of queuing\nindefinitely.\n\nWe are also in the middle of an ongoing effort to remove database queries from\nthe path of Val requests. In the future a failure like this would only lead to\nAPI downtime and not Val execution downtime.\n",[],[191,1053],"2991a525-5b34-4273-91a3-08f10cb57e3e","2024-12-13-devstats.md","86bcdd14-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Using Val Town to chart dependency bloat\ndescription: Building a simple aggregate view of our codebase\npubDate: '2024-12-13T05:00:00.000Z'\nauthor: Tom MacWright\nthumbnail: 'https://assets.blog.val.town/legacy/devstats/screenshot.png'\ntags: [engineering]\n---\n\n[Devstats](https://www.val.town/v/tmcw/devstats) is a Val that you can use to easily track send point-in-time statistics from your GitHub Actions (or other CI) runs and turn them into pretty charts. Here's ours!\n\n![Devstats](https://assets.blog.val.town/legacy/devstats/screenshot.png)\n\nSo far, we're using it to track dependency bloat. Much has been written about dependency bloat. A lot of web projects end up with a multi-gigabyte `node_modules` directory filled with agglomerated complexity.\n\nMaking a pretty chart of our `node_modules` size over time doesn't force us to cut weight, but it does help motivate such efforts: who doesn't like to see a big drop in a chart after making some strategic cut?\n\nThere are many other analytics stacks that we could've used for this, but I wanted to make something dead-simple:\n\n- schemaless, so you can send any statistic without having to declare it beforehand\n- using a barebones HTTP request to send statistics, so all you need is cURL\n\nSo, devstats was born! I see it as a very basic riff on the legacy of [StatsD](https://www.etsy.com/codeascraft/measure-anything-measure-everything/) and [RRDtool](https://oss.oetiker.ch/rrdtool/) but over HTTP and with a less [fancy wire protocol](https://github.com/b/statsd_spec).\n\nHere's what our GitHub Actions configuration has at the very end\nto send statistics to devstats:\n\n```yaml\n- name: devstats\n run: |\n curl -X \"POST\" \"https://tmcw-devstats.web.val.run/\" \\\n -H 'Authorization: Bearer ${{ secrets.DEVSTATS_TOKEN }}' \\\n -H 'Content-Type: application/json; charset=utf-8' \\\n -d $\"{ \\\"name\\\": \\\"node_modules_kb\\\", \\\"value\\\": $(du -sk node_modules | awk '{print $1}') }\"\n curl -X \"POST\" \"https://tmcw-devstats.web.val.run/\" \\\n -H 'Authorization: Bearer ${{ secrets.DEVSTATS_TOKEN }}' \\\n -H 'Content-Type: application/json; charset=utf-8' \\\n -d $\"{ \\\"name\\\": \\\"package_lock_lines\\\", \\\"value\\\": $(wc -l package-lock.json | awk '{print $1}') }\"\n curl -X \"POST\" \"https://tmcw-devstats.web.val.run/\" \\\n -H 'Authorization: Bearer ${{ secrets.DEVSTATS_TOKEN }}' \\\n -H 'Content-Type: application/json; charset=utf-8' \\\n -d $\"{ \\\"name\\\": \\\"package_lock\\\", \\\"value\\\": $(wc -l package.json | awk '{print $1}') }\"\n```\n\nNot bad! `curl` is already available, so this is all you need\nto start tracking your `node_modules` size, `package.json` lines,\nand `package-lock.json` lines over time. This is all using Linux\nutilities like `du`, `awk`, and `wc`, but you could do the same\nwith JavaScript or any other language to calculate more complex\nmetrics.\n\n## Using it\n\nTo use devstats yourself, just\n\n1. [Fork the val. You can rename it, and so on!](https://www.val.town/v/tmcw/devstats)\n2. Come up with a random authentication token, and [add it to your repository secrets](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions) and to your [Val Town environment variables](https://docs.val.town/reference/environment-variables/)\n3. Add a step like shown above to your GitHub Actions configuration.\n\n_Not sure how to generate a random authentication token?_ One way is to run OpenSSL:\n\n```\nopenssl rand -hex 20\n```\n\nThat's it! The 'wire format' for devstats is simply sending an object\nlike this:\n\n```json\n{\n \"name\": \"coffees\",\n \"value\": 10\n}\n```\n\nYou can send any `name` you want and it'll be added to the dashboard.\n\n## How it works\n\nThis val uses our [SQLite](https://docs.val.town/std/sqlite/) support as its database, [Observable Plot](https://observablehq.com/plot/) for charts, and [Hono](https://hono.dev/) to do some routing and rendering.\n\n## Notes\n\nYou might be wondering: Val Town's `node_modules` directory is around 900 megabytes: what's in there! I'll give you a peek into our top-10 module (namespaces):\n\n```\nval.town/node_modules\n$ du -sk * | sort -n -r | head -n 10\n47592\t@opentelemetry\n40132\t@sentry\n33012\tlucide-react\n24700\t@clerk\n24136\t@ai-sdk\n23568\t@biomejs\n23168\ttiktoken\n22200\ttypescript\n21044\t@heroicons\n20616\thappy-dom\n```\n\nWe spend a lot of megabytes on our monitoring stack, using [Open Telemetry](https://github.com/open-telemetry) and [Sentry](https://sentry.io/), and another chunk on the icon sets [Lucide](https://lucide.dev/) and [heroicons](https://heroicons.com/). We use [tiktoken](https://github.com/dqbd/tiktoken) to truncate text before sending it off to be encoded for [semantic search](https://blog.val.town/blog/val-vibes/).\n\n### One issue: triple distributions\n\nJavaScript has been in a painful transition phase between module systems, moving from CommonJS, which was Node.js's module system for many years, to ESM, which is standardized and designed to also work in browsers. This is one of the culprits behind our `node_modules` size.\n\nOpen Telemetry is a pretty heavyweight dependency for us. One issue is that they ship each line of code [three times](https://unpkg.com/browse/@opentelemetry/otlp-transformer@0.56.0/build/): not just dual modules between [CommonJS & ESM](https://nodejs.org/api/esm.html), but also an esnext build for bleeding-edge JavaScript features, and they include source maps for all three versions.\n\nLucide does the same thing with a triple build, but for ESM, CommonJS, and [UMD](https://github.com/umdjs/umd) module types. Heroicons does a double build, for CommonJS and ESM.\n\n### Another: big binaries\n\nAnother big issue we have – that probably many other projects have as well – is the big binaries embedded in some NPM modules.\n\nSentry includes a binary distribution of their [CLI, a native Rust tool](https://github.com/getsentry/sentry-cli), which weighs 16MB. [Biome](https://biomejs.dev/), which we use to lint and format our code, is another 23MB.\n\n### Lots of copies of esbuild\n\nThe biggest challenge that isn't on this list is [esbuild](https://esbuild.github.io/), which is used internally by lots of modules that we depend on.\n\n![Grand Perspective](https://assets.blog.val.town/legacy/devstats/grandperspective.png)\n\nThat's [Grand Perspective](https://en.wikipedia.org/wiki/GrandPerspective) showing all of the files in `node_modules` as a treemap, with the seven copies of `esbuild` glowing.\n\nAll in all, we spend about 94MB of space storing 7 versions of esbuild and another 3 versions in sub-packages, a fact that I figured out\nusing this epic (for me) chain of Linux commands:\n\n```\nfind . -type f -name 'esbuild' |\n xargs du -sk * |\n sort -n |\n grep \"esbuild$\" |\n awk '{print $1}' |\n awk '{s+=$1} END {print s}'\n```\n\nThe problem with esbuild for us is that some transitive dependency will depend on an old or specific version of esbuild, and NPM will dutifully download and store multiple copies of esbuild to satisfy all consumers. Absent some coordination from all the community, we might have to use an [override to pin esbuild to a specific version](https://docs.npmjs.com/cli/v9/configuring-npm/package-json#overrides), and hope that our dependencies don't really need such a specific version range.\n\n### Endnotes\n\nLike most decisions, there were good reasons behind all of these. Hybrid CommonJS + ESM modules make sense for a [still-divided world](https://github.com/wooorm/npm-esm-vs-cjs) of people using NPM modules. Using a conservative version range for esbuild makes sense to limit the chances that your module will unexpectedly break on a new esbuild release. NPM's ability to download multiple versions of a transitive dependency is [actually great and solves a problem inherent in other systems like pip and Cabal](https://lexi-lambda.github.io/blog/2016/08/24/understanding-the-npm-dependency-model/).\n\nSo, there aren't many opportunities to point fingers. So far I haven't found any NPM modules that accidentally included a 40MB MP4 video of _Buffy The Vampire Slayer_ and that's why they were bloated. We've used [knip](https://knip.dev/) to identify a handful of dependencies that we forgot to remove once we stopped using them. Most of the big stuff in `node_modules` is there for a reason.\n\n---\n\nSo, it's definitely a goal to cut big dependencies that we aren't using and slim down the ones that we need to use. Devstats is helping with that, and we're working on more developer tooling for building Val Town _with_ Val Town!\n",[],[191,1060],"6a23cd23-3556-4a86-8831-7c738394fe44","2024-12-02-val-town-newsletter-22.md","86bc052e-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town Newsletter 22\ndescription: 'Townie upgrades, Scoped API permissions, Fal partnership'\npubDate: '2024-12-02T05:00:00.000Z'\nauthor: Steve Krouse\ntags: [newsletter]\n---\n\n\n\nVal Town is growing up! We're working on features and infra that will fundamentally unlock the scope and scale of what you can build in Val Town. We have two major efforts running in parallel: projects and scalability.\n\n_Projects_ are containers for multiple vals, folders, and files. The enable you to create infinitely complex real-world apps, instead of being limited to a single long val. Internal to a project, vals will use relative imports (ie `./foo/bar`), so they can be forked as a unit, and merged back in as a unit. Projects will also improve Townie's ability to help write your vals in more modular files, so that it doesn't have to regenerate all the code on every change.\n\n_Scalability_ will enable you to run production workloads in Val Town, with usage-based pricing, custom alerts, and caps.\n\nWe hope to have both of these efforts live in the coming months. In the meantime, we've shipped many other features and improvements, including a dozen Townie upgrades, scoped API permissions, a partnership with Fal, and more.\n\n[Justin Bennett](https://just-be.dev) has joined our team! He is well-known for the [DevtoolsFM podcast](https://www.devtools.fm/), and was most recently at [The Recurse Center](https://www.recurse.com/), [Membrane](https://www.membrane.io/), and [Oxide](https://oxide.computer/). We've known Justin for years, and are thrilled he's on the team.\n\nAs per usual, the [highlight of our newsletter is at the bottom: a list of the most interesting community vals we've seen this month](), including a bunch working with the Bluesky API, a Val Town clone built on Val Town, Stripe examples, virtual money, data journalism, lightning-fast Cerebras 2k-tokens-per-second demos, and dozens more!\n\n## 🤖 Townie upgrades\n\nTownie has been massively upgraded this month. We shipped some of the most common feature request and fixed the worst bugs.\n\n

\n\n_This video shows me adding analytics to [Cerebras Coder](https://www.val.town/v/stevekrouse/cerebras_coder), a minimal Townie clone, powered by [Cerebras](https://cerebras.ai/)'s lightning-fast inference. The video shows switching from Haiku to Sonnet; Townie detecting a server-side error; fixing it upon request; fixing another bug; viewing older versions of the val; and a notice to start a new chat because longer chats make Townie dumber._\n\n- By far the most common feature request for Townie has been the ability to preview the older versions of HTTP vals. This is now live! There's a dropdown in the bottom of the Preview tab in Townie to select an older version to preview, with the ability to restore that version. Thanks [Charlie](https://www.val.town/u/dcm31) for the tip to add the Restore button, and [Matt](https://www.val.town/u/neverstew) for the feedback on the wording of the button 🙏\n- In order to make Townie sustainable, we rolled out usage limits. Now free-tier users can try out Townie for about a dozen messages on a Haiku, a cheaper (albeit still fantastic) model. We hope this will help you get a sense of what Townie can do, and encourage you to upgrade to Pro if you want to use it regularly. If you find that you're hitting your limits as a Pro user, please reach out to me at steve@val.town with more context on your use case. We're eagerly looking for feedback on a higher-level Pro tier or usage-based pricing.\n- We deployed Anthropic prompt caching ([try it on Val Town](https://www.val.town/v/stevekrouse/anthropicCaching)), which saves 40% on cost, and improves latency by up to 85% for long prompts.\n- We rolled out Sonnet 3.5 New, which is supposed to be even better at coding, but unfortunately had to rollback to Sonnet 3.5 Old, because the New version was resistant to writing very large blocks of code in their entirety. It added a lot of `// code remains unchanged here`. We fixed that issue with Anthropic's help, and are working through a couple other system prompt issues with Sonnet 3.5 New, and hope to get it rolled out in the coming weeks.\n- Townie now detects many client-side and server-side errors automatically, and has a clean UI to view the errors to help you decide if you want to debug them manually, dismiss them, or ask Townie to solve them for you.\n- At long-last we fixed bugs with the Townie code editor getting out of sync or showing \"unsaved changes\" when there weren't any.\n\nIf you'd like to read more about behind-the-scenes of Townie, Tom gave a talk, which he turned into a blog post: [Building a Code Writing Robot](https://blog.val.town/blog/building-a-code-writing-robot/).\n\n## 🔐 Scoped Permissions\n\nWe added API scopes to Val Town API tokens to give you more granular control over our REST API.\n\nWe have read/write scopes for:\n\n- `val` – [vals](https://docs.val.town/openapi#tag/vals)\n- `user` – [user account details](https://docs.val.town/openapi#tag/me)\n- `blob` – [blob storage](https://docs.val.town/openapi#tag/blobs)\n- `sqlite` – [sqlite database](https://docs.val.town/openapi#tag/sqlite)\n- `email` – [ability to send emails](https://docs.val.town/openapi#tag/emails)\n\nThe [API tokens page](https://www.val.town/settings/api)\nnow lets you view and configure the scope of each token.\nYou can also configure the scope of for each individual val on its settings page.\n[Learn more ➡](/api-token-scopes/)\n\n## 🌄 Fal Partnership\n\nWe announced a partnership with [Fal](https://fal.ai/), to bring their\nlightning-fast image generation API to Val Town users.\n[Learn more ➡](/fal/)\n\n

\n \n $\"Simplest$ \n

\n Simplest possible Fal app\n

\n \n

\n\n

\n \n $\"Simplified$ \n

\n \"Faltown\" – variants, API client, blob storage\n

\n \n

\n\n## Updated Pricing Page\n\nWe've made some long-overdue updates to our [pricing page](https://www.val.town/pricing).\n\n $\"Val$ \n\nHere's what changed:\n\n- Expanded the Pro tier from 100 to 500 HTTP val req/min – thanks [Izu](https://www.val.town/u/izuchukwu) for the request 🙏\n- Listed custom domains on the Pro tier\n- Lowered the number of private or unlisted vals on the Free tier from 100 to 10. (We have yet to enforce this. When we enforce it, we will grandfather in any vals created before.)\n- Removed some stale limitations we never enforced\n\nWe don't enforce all of these limits today. For example, Free tier users can make over 10 runs per minute. We're figuring out the right balance between signalling what we can support at scale, without adding unnecessary friction for users.\n\n## Blob Storage admin\n\nWant to view and edit the blobs stored in your [Val Town Blob Storage](https://docs.val.town/std/blob/)? Now you have two great options.\n\n

\n \n $\"Blob$ \n

\n Blob storage in Val Town Settings\n

\n \n

\n\n

\n \n $\"Blob$ \n

\n \"Blob Admin\" - built in userspace, forkable, customizable, more features, more bugs\n

\n \n

\n\n## 📺 Videos\n\nVideos are a great way to learn the pro tips & tricks for Val Town. This month I live coded with three wonderful guests:\n\n- **[Creating customer-enrichment workflows](https://www.youtube.com/live/wOyfS9-Q01I) with [Justin Duke](https://jmduke.com/)**, founder of [Buttondown](https://buttondown.email/)\n- **[Live coding viral AI apps with](https://www.youtube.com/live/plubVSqiQAw) with [Hassan](https://x.com/nutlope)**, of [TogetherAI](https://www.together.ai/)\n- **[Prototyping Markdown Extensions](https://www.youtube.com/live/oBuJoVV_yWc) with [Boris Berenberg](https://www.nothingeasyaboutthis.com/)**\n\nWe were also featured in [three](https://www.youtube.com/watch?v=j5SfOA7gGfw) [YouTube](https://www.youtube.com/watch?v=l9NyuajxjrI) [videos](https://www.youtube.com/watch?v=yMHDBQp5hbE), which led to thousands of new subscribers. If that includes you – welcome! Thanks AICodeKing, Julian Goldie, and Emre Kabli for the videos 🙏\n\n## 🎉 Other updates\n\n- You can now view all your vals on your own profile page – even private ones – configurable via a filter – thanks [Fabian](https://fabian.ai/) for the request 🙏\n- Simplified and unified buttons to edit a val's name, type, etc – thanks [Fabian](https://fabian.ai/) for the request 🙏\n- Allow cron vals to be paused but remain cron-type - thanks [Jan](https://www.val.town/u/yawnxyz) for the request 🙏\n- The 'Recent vals' section on your private dashboard now shows private and unlisted vals (before it only showed your public vals) – thanks [Arti](https://www.val.town/u/artivilla) for the bug report 🙏\n- [Others' val code is now focusable and searchable](https://x.com/stevekrouse/status/1851670974019559621)\n- HTTP errors now always have stacktraces, which link directly to val source lines\n- Improved val generated names and validation\n- Fix our CORS headers – thanks [Andreas](https://www.val.town/u/rozek) 🙏\n- Improved our Custom Domains UI and docs\n- Added buttons to the val versions page to preview old versions, view their code, and delete them\n- Fixed the logs, removing old traces that stalled\n- Fixed cron UI bug\n- Discontinued Eval API and upgraded all HTTP vals to the new concurrent runtime\n- [Fixed beta & structured outputs for std/openai](https://www.val.town/v/stevekrouse/openai_structured_output_demo) – thanks [Alfonso](https://www.val.town/u/alfongj) for the bug report 🙏\n- Added a link to edit your profile photo on the profile settings page – thanks [Ivan](https://www.val.town/u/spiralganglion) for the bug report 🙏\n- Removed all browser `alert` and `confirm` in favor of proper modals\n- Improve import autocomplete for vals in context\n- Fix log association for replayed val traces\n- Button to start a Townie chat from the blue `New` button in the top-right, in the navbar\n- HTTP Preview Share button & modal (to make it clearer that HTTP vals are deployed and sharable) – thanks [Rudd](https://www.val.town/u/rudd) for the suggestion 🙏\n- Made the [Dashboard](https://www.val.town/dashboard) load up to 5x faster\n- Made HTTP Preview iframes lazy load, so your focus won't be stolen by a val that's not in view – thanks [Troy](https://val.town/u/trob) for the bug report 🙏\n- Add Cmd+K actions for opening the focused val in the full-screen editor or in its detail page – thanks [Arti](https://www.val.town/u/artivilla) for the suggestion 🙏\n\n## 🛣️ Roadmap\n\n### Projects\n\nOur biggest effort coming up is _Projects_ - groups of vals, files, folders that will be forkable as a unit and mergeable as a unit, supporting relative imports, and more. Where a val is like a Github gist, a Project is like a Github repo. Projects will enable you to build infinitely complex real-world apps in Val Town. We're excited to share something hopefully in Dec 2024, and if not in Jan 2025.\n\n $\"Screenshot$ \n\nWe shared the above screenshot of an example Projects a couple weeks back in our Discord. One of our users, [Victor], was so inspired that he's started building larger projects in Val Town already in advance of Projects, living in the future by pretending that they already exist. His most recent large val project was [NotUber](https://vawogbemi-notuber.web.val.run/), a surprisingly full-featured Uber/Lyft clone. The best part is that he built [another val that acts as the 'project view' for the 8 vals that NotUber depends on](https://vawogbemi-notuberfolder.web.val.run/). We love Victor's vision, and we can't wait to see what everyone else builds once Projects ship for real!\n\n### Scale\n\nWe're also working on _Scale_ – a new infra architecture that will enable you to run production workloads in Val Town, with scale to infinity, usage-based pricing, with usage alerts, and caps. Websocket-type vals will also launch with this effort.\n\n### Other features coming soon\n\n- Val- & Project- scoped environment variables\n- Searchable & filterable logs and traces\n- /explore - a new page to find interesting vals\n- Allow hyphens in val names\n- Improved code search\n- Comments on pull requests\n- Ability to follow a user on Val Town\n- Team accounts\n- SOC2 compliance (I told you we're growing up!)\n- ...and much more!\n\nPlease [make feature requests and vote](https://github.com/val-town/val-town-product/discussions?discussions) on what you'd like to see next.\n\n## 🌟 Community vals\n\n- [Dan](https://www.val.town/u/danphilibin), an engineer at [Campsite](https://campsite.com), wrote [How we prototype API integrations with Val Town](https://www.campsite.com/blog/how-we-prototype-integrations-with-val-town) – it's an excellent read about how they \"speedrun prototypes of bots, features, and 3rd party integrations\" ([tweet](https://x.com/DanPhilibin/status/1852052119521542552))\n- [Max](https://val.town/u/maxm) built [Val Town Town](https://blog.val.town/blog/val-town-town/) – a clone of Val Town, built in Val Town! If you want to run your users' or AI-generated server-side JavaScript, now you can do that in Val Town by forking and customizing Val Town Town\n- [Victor](https://www.val.town/u/vawogbemi) added [Stripe to our docs](https://docs.val.town/integrations/stripe/) showing how you can accept payments in Val Town in minutes\n- Daniel created [an app that helps industrial robot programmers visualize their program call graph](https://bsky.app/profile/stevekrouse.com/post/3lbu7qbq6mc2b)\n- [osspledge.com](https://osspledge.com/)'s redirect to [opensourcepledge.com](https://opensourcepledge.com/) is [hosted on Val Town](https://www.val.town/v/vladh/opensourcepledgeRedirect?v=5) (_[Join the pledge here](https://opensourcepledge.com/join/)!_)\n- In honor of Bluesky, here are all the awesome Bluesky vals:\n - [Dean](https://www.val.town/u/loading) built [WinSky](https://loading-xp.web.val.run/), a Windows XP-styled BlueSky client that you can OAuth into with your Bluesky account, make posts, etc!\n - [Stef](https://www.val.town/u/stef) built a [Bluesky feed generator](https://www.val.town/v/stef/feeds) that subscribes to a feed of posts without the letter 'E', forked from [@nws/feeds](https://www.val.town/v/nws/feeds), forked from [@jordan/feeds](https://www.val.town/v/jordan/feeds)\n - [Jonas](https://www.val.town/u/jonasmerlin) made a [Bluesky bot template](https://www.val.town/v/jonasmerlin/bluesky_bot_template) ([original post](https://bsky.app/profile/jonasmerlin.bsky.social/post/3lcdnfizwrs2a))\n - [Wilhelm](https://www.val.town/u/wilhelm) is [mirroring his Bluesky posts](https://www.val.town/v/wilhelm/bsky). It's easy to fork it and mirror your own posts! I did [here](https://www.val.town/v/stevekrouse/bsky).\n - [@karkowg](https://www.val.town/u/karkowg) built a beautiful [Matrix-style Bluesky firehose visualization](https://www.val.town/v/karkowg/theOneSky)\n - [Wilhelm](https://www.val.town/u/wilhelm) figured out how to [make a val http endpoint your Bluesky handle](https://www.val.town/v/wilhelm/did)!\n - [Steve](https://www.val.town/u/stevekrouse) created a [Bluesky Firehose](https://val.town/v/stevekrouse/firehose) visualizer that displays real-time updates from the Bluesky network, with word cloud\n- [Robin Sloan is tracking solar storms with a cron val!](https://bsky.app/profile/stevekrouse.com/post/3lbuwlperjc2b)\n- [Jan](https://www.val.town/u/yawnxyz) built [Scribe](https://yawnxyz-scribe.web.val.run/), a SuperWhisper / Granola alternative that transcribes your voice in realtime, powered by Huggingface, VAD, and Groq\n- [Arti](https://www.val.town/u/artivilla) built a [Farcaster Keyword Alerter](https://www.val.town/v/artivilla/farcasterKeyHook), which sends Slack notifications when a keyword is mentioned on Farcaster, powered by [Neynar](https://neynar.com/) ([original post](https://warpcast.com/artivilla.eth/0x1bc32ac7))\n- [Steve](https://www.val.town/u/stevekrouse) built a website showcasing [how Val Town uses Val Town to build Val Town](https://www.val.town/v/stevekrouse/dogfooding)\n- [Nulo](https://www.val.town/u/nulo) is [tracking and graphing tweets and likes for President Milei of Argentina to expose bots and archive deleted tweets](https://x.com/stevekrouse/status/1854255504681210055)\n- [Charlie](https://www.val.town/u/dcm31) made [a hilarious short-video about a tour-guide app with various intelligence levels](https://www.tiktok.com/@charliemocomedy/video/7441621954571832619). The app in the video was made with Townie!\n- [Peter](https://www.val.town/u/peterqliu) built a [mayor_setup](https://www.val.town/v/peterqliu/mayor_setup), a val that gives you a bird's eye view across all your vals, lets you organize vals into composable collections, batch actions on multiple vals, and uninstall itself with no trace left behind\n- [Jon](https://www.val.town/u/jonbo) built [a custom Twitter Alert bot that detects when specific accounts tweet relevant phrases, and uses an LLM to decide which Discord channel to send the alert to](https://www.val.town/v/jonbo/cabinAdjacentTweets)\n- [Max](https://www.val.town/u/maxm) built [WIDE](https://maxm-wide.web.val.run/), a service to write and query unstructured data, powered by Clickhouse, powered by his new [ValSession](https://www.val.town/v/maxm/valSession) authentication method\n- [@trob](https://www.val.town/u/trob) built an [interactive prompt-to-code app](https://www.val.town/v/trob/prompt_to_code_auto_refresh_codebed) that generates JavaScript code from prompts and allows previewing the output with user input. [James](https://www.val.town/u/JamesAndrew) forked it, and made a [Three.js Scene Generator](https://www.val.town/v/JamesAndrew/tenaciousPlumWoodpecker) to create 3D scenes by via natural language prompts. We love when people build on each other's work!\n- [Dean](https://www.val.town/u/loading) built a [peer-to-peer file transfer app](https://www.val.town/v/loading/file) using WebRTC\n- [Wolf](https://www.val.town/u/wolf) built [a magic importer](https://www.val.town/v/wolf/UseDynamic) that lets you dynamically generate functions at runtime in the import URL, powered by an HTTP val! ([Discord discussion](https://discord.com/channels/1020432421243592714/1279542466357366804/1295911703040622662))\n- \"Who needs blockchain when you have Val Town SQLite?\" – [Jamie](https://www.val.town/u/jamiedubs) built [Glifbux](https://www.val.town/v/jamiedubs/glifbux), virtual money on Val Town\n- Steve built [three vals to demo the insanely-fast Cerebras inference](https://x.com/stevekrouse/status/1851994923957985567) – [Cerebras Searcher](https://www.val.town/v/stevekrouse/cerebras_searcher) to search and summarize the web, [Cerebras Coder](https://www.val.town/v/stevekrouse/cerebras_coder) to generate full websites in miliseconds, and [Cerebras Debater](https://www.val.town/v/stevekrouse/cerebras_debater) to get to the bottom of age old questions\n- SimpleScraper wrote [\"Building a free beach-chair detector using Simplescraper, LLMs and public webcams\"](https://simplescraper.io/blog/building-beach-chair-detector-ai-webcam) powered by a cron val and @std/openai\n- [Dan](https://www.val.town/u/danphilibin) built a [What did I work on today?](https://www.val.town/v/danphilibin/what_did_i_work_on_today) in 6 prompts with Townie, which pulls from Github and Linear to generate a daily summary of what you worked on ([tweet](https://x.com/DanPhilibin/status/1849823537034305704))\n- [Juan](https://www.val.town/u/juansebsol) built a [D3.JS Defi pool explorer](https://juansebsol-blockchainecosystembubblemap.web.val.run/) to visualize Blockchain Pool APYs\n- [Cam](https://www.val.town/u/cameronpak) made a [Tool Drawer](https://www.val.town/v/cameronpak/tooldrawer) to showcase the apps on the [Techless Wisephone II](https://www.wisephone.com/)\n- Jonnie Hallman [built a tool with Townie to extract the dominant colors from an image](https://x.com/destroytoday/status/1856709997737984089)\n- [Juan](https://www.val.town/u/juansebsol) built a [EchoSearch](https://juansebsol-echosearch.web.val.run/), a site that lets you search across Reddit, HN, Github Discussions, StackOverflow all at the same time\n- @deepfates built [deeperfates.com](https://deeperfates.com/), their AI clone\n- [Victor](https://www.val.town/u/vawogbemi) built [TaylorSwiftMoments](https://www.val.town/v/vawogbemi/taylorswiftmoments), an image generator that allows you to create visuals inspired by different Taylor Swift eras with enhanced prompts\n- [Steve](https://www.val.town/u/stevekrouse) built a [Proxied Web Browser](https://www.val.town/v/stevekrouse/dns_record_debugger) for debugging DNS records, so you can view web pages without DNS caching issues\n- [Jan](https://www.val.town/u/yawnxyz) built a [Open Graph Image Generator](https://www.val.town/v/yawnxyz/og) using satori and a Deno-friendly SVG WASM, cached with Val Town Blob Storage\n- [Steve](https://www.val.town/u/stevekrouse) showed [how to use OpenAI Structured Output with zod](https://www.val.town/v/stevekrouse/openai_structured_output_demo) to ensure responses are in the right JSON format\n- [vprtwn](https://www.val.town/u/vprtwn) created a [typing practice app with word generation and stat tracking](https://vprtwn-svaltown.web.val.run/) for the [Svalboard](https://svalboard.com/)\n- [Andre](https://www.val.town/u/andreterron) built a [Bomb Cyclone Power Outage Tracker](https://www.val.town/v/andreterron/bombCycloneOutageTracker) to help everyone estimate when power will be restored\n- [@g](https://www.val.town/u/g) built a [GitHub Line Counter](https://g-countgithublocui.web.val.run/) that efficiently counts the lines of code in any GitHub repository from your browser, powered by a [val](https://www.val.town/v/g/countGithubLOCUI)\n- [@bladesquad](https://www.val.town/u/bladesquad) built a [React app for reporting stray cats and finding nearby emergency vet contacts in Algeria](https://www.val.town/v/bladesquad/thrillingAmaranthKiwi)\n- [Max](https://www.val.town/u/maxharper26) built an [API to set up data to a Vercel database](https://www.val.town/v/maxharper26/api_v1)\n- [@trollishka](https://www.val.town/u/trollishka) built a [AI powered gift suggestion app](https://trollishka-giftsuggestionapp.web.val.run/) – with RAG powered by the Serp API\n- [Steve](https://www.val.town/u/stevekrouse) built a [tool to compare the speed and output of different Anthropic models](https://www.val.town/v/stevekrouse/claude_speed_test)\n- [Michael](https://www.val.town/u/michaelwschultz) built a [val that generates an image displaying current weather information](https://www.val.town/v/michaelwschultz/generateframeImage) for his e-ink display\n- [Nico](https://www.val.town/u/nbbaier) built [simpletext](https://www.val.town/v/nbbaier/simpletext), an implementation of the [simpletext](https://github.com/jonfraser/simpletext) by jonfraser\n- [Jan](https://www.val.town/u/yawnxyz) built a [Collaborative JSON Viewer](https://www.val.town/v/yawnxyz/jsonpub), powered by [InstantDB](https://www.instantdb.com/) ([tweet](https://x.com/yawnxyz/status/1860133805442703553))\n- [@trob](https://www.val.town/u/trob) built an [interactive prompt-to-code app](https://www.val.town/v/trob/prompt_to_code_auto_refresh_codebed) that generates JavaScript code from prompts and allows previewing the output with user input.\n- [Jan](https://www.val.town/u/yawnxyz) built a [GitHub Repository Parser](https://www.val.town/v/yawnxyz/githubParser) to get the number of bytes and tokens in a repo (to estimate cost for sending a whole repo to an LLM)\n- [Tanner](https://www.val.town/u/spinningideas) built a [Web Page Summarizer](https://www.val.town/v/spinningideas/webpage_summarizer) that concisely summarizes the content of any website using OpenAI's language model\n- [Regina](https://www.val.town/u/reginafoley) created a [Flashcard App](https://www.val.town/v/reginafoley/createflashcardsval) that allows users to study with flashcards from a CSV file\n- [Mike](https://www.val.town/u/mikehiggins) created [a 90s-style text analyzer](https://www.val.town/v/mikehiggins/sanguinecyanmastodon) that calculates word frequencies, TF-IDF scores, sentiment analysis, and generates fun facts\n- [Max](https://www.val.town/u/maxm) built a [Simple Wikipedia Instant Search](https://www.val.town/v/maxm/simpleWikipediaInstantSearch) to search Wikipedia articles with fuzzy matching and instant results.\n- [@janu](https://www.val.town/u/janu) built a [passworded Bitcoin price dashboard](https://www.val.town/v/janu/dataDash) that displays the current price, 24-hour change, and a chart of the last 30 days.\n- [Prasham ](https://www.val.town/u/prashamtrivedi) built [summarizeHn](https://www.val.town/v/prashamtrivedi/summarizehn) which summarizes Hacker News discussions using Gemini Models\n- [Daniel](https://www.val.town/u/danieladmsf) created a [Seller Dashboard](https://www.val.town/v/danieladmsf/gregariousIvoryOstrich) that displays real-time updates of quotes, savings, and a list of sellers with filtering and action buttons.\n- [Dhanush](https://www.val.town/u/dhanush_suvarna) built [Flighty](https://www.val.town/v/dhanush_suvarna/flighty), an API endpoint to get recent flight data\n- [Muhammad](https://www.val.town/u/muhammad_owais_warsi) built a [realtime image generator](https://www.val.town/v/muhammad_owais_warsi/imageGen) ([tweet](https://x.com/MO_warsi786/status/1856995667790397885))\n- [@vip](https://www.val.town/u/vip) built [RSS Feed AI Analysis](https://www.val.town/v/vip/rss_feed_ai_analysis)\n- [Jared](https://www.val.town/u/jaredsilver) created [Startup Clicker Game](https://www.val.town/v/jaredsilver/startup_clicker) which simulates running a tech startup\n- [willthereader](https://www.val.town/u/willthereader) built a [reminder app](https://val.town/v/willthereader/SafeReminder) that lets you quickly set alarms and timers, and sends web notifications\n- [Prasham](https://www.val.town/u/prashamtrivedi) built a [Prompt Token Counter](https://www.val.town/v/prashamtrivedi/tokencounter) for Anthropic and Google Gemini models\n- [Steve](https://www.val.town/u/stevekrouse) built [The Password Game](https://www.val.town/v/stevekrouse/passwordgame), a web app that generates increasingly silly password requirements using AI with Cerebras\n- [@all](https://www.val.town/u/all) built [jamz](https://www.val.town/v/all/jamz), a music player app that lets you add songs, edit their details, and sort them by date or title\n- [@janu](https://www.val.town/u/janu) built [airSource](https://www.val.town/v/janu/airsource), which generates an SVG displaying real-time air quality data for major Australian cities\n- [@torlanco](https://www.val.town/u/torlanco) built [Shirtgen](https://torlanco-shirtgen.web.val.run/), an app that lets you generate AI-powered t-shirt designs\n- [Jared](https://www.val.town/u/jaredsilver) built a [Leadership Styles Inventory](https://www.val.town/v/jaredsilver/leadership_styles_inventory) quiz to assess your leadership tendencies\n- [@d17e](https://www.val.town/u/d17e) created a [QR code generator](https://www.val.town/v/d17e/qrCodeWebhook) that returns an SVG of a plottable QR code from a given value and configuration\n- [@spinningideas](https://www.val.town/u/spinningideas) built an [AI Interview Assistant](https://www.val.town/v/spinningideas/interview_practice) that lets you practice job interviews with an AI interviewer\n- [Grant](https://www.val.town/u/gwoods22) built a script to [toggle AWS EventBridge rules](https://www.val.town/v/gwoods22/aws_eventbridge_toggler)\n- [@gigmx](https://www.val.town/u/gigmx) built a [link-in-bio](https://www.val.town/v/gigmx/linkinbiotemplate) featuring a Cheshire Cat AI chat widget and social links\n- [Alfonso](https://www.val.town/u/alfongj) built an [Ethereum Address Generator](https://x.com/alfongj/status/1811672444124197568) to generate test Ethereum addresses and private keys client-side, [embedded in the Crossmint docs](https://x.com/stevekrouse/status/1852024470958989517)\n- [@all](https://www.val.town/u/all) built an [interactive code editor](https://wittyturquoisedove.web.val.run/) powered by Cerebras that generates HTML, CSS, and JavaScript based on natural language prompts.\n- [@all](https://www.val.town/u/all) built [savvyPurpleSpoonbill](https://www.val.town/v/all/savvyPurpleSpoonbill), an interactive code editor powered by Cerebras AI that generates HTML, CSS, and JavaScript code based on user prompts\n- [@willow](https://www.val.town/u/willow) built an [HTML to Markdown Converter](https://willow-htmltomarkdownconverter.web.val.run/) that supports tables and links\n- [@jidun](https://www.val.town/u/jidun) created [Ms_Spangler](https://www.val.town/v/jidun/Ms_Spangler), an AI tutor specializing in the Texas junior high curriculum that provides grade-appropriate academic support while fostering critical thinking and real-world applications\n- [@zlpkhr](https://www.val.town/u/zlpkhr) built a [neighborhood order app](https://zlpkhr-neighborhoodorderapp.web.val.run/) where users can propose group orders and others can sign up.\n- [Nick](https://www.val.town/u/nemonicui) built a [guided breathing instructions app](https://www.val.town/v/nemonicui/breathinginstructional) with techniques like 4-7-8 Breathing, Box Breathing, and Alternate Nostril Breathing to help manage stress and promote relaxation\n- [@tk120404](https://www.val.town/u/tk120404) built a [3D visualization of our solar system](https://tk120404-solarsystem.web.val.run/), complete with rotating planets, moons, meteor trails, and a starry background\n- [Steve](https://www.val.town/u/stevekrouse) built a middleware [usageAlert](https://www.val.town/v/stevekrouse/usageAlert) that sends an email alert if your HTTP val exceeds 1k requests per minute\n- [@mashimi](https://www.val.town/u/Mashimi) built an [Insect Identifier](https://www.val.town/v/Mashimi/insectIdentifier) that recognizes insects from uploaded images using the Grok.AI API\n- [Will](https://www.val.town/u/willthereader) made an [AI-powered Safari extension](https://x.com/Willthereader/status/1859023415421845824), powered by a val for the backend\n- [Natalie](https://www.val.town/u/natsa) built a [todo list app for learning Node.js](https://www.val.town/v/natsa/genuinebrownermine) with expandable sub-topics and progress tracking\n- [@minhee](https://www.val.town/u/minhee) built a [single file implementation of ActivityPub using Fedify](https://www.val.town/v/minhee/FedifyOnValTown), allowing your fediverse account to follow and be followed back by a demo account\n- [@meta](https://www.val.town/u/metart43) built [a page displaying color schemes used by major news media outlets for US election map visualizations](https://www.val.town/v/metart43/us_election_color_scheme)\n- [Jan](https://www.val.town/u/yawnxyz) built a [Counter](https://val.town/v/yawnxyz/counter) - an all-in-one word, byte, and token counter that saves results to browser history\n- [Victor](https://www.val.town/u/vawogbemi) built a [Solana Pay QR Code Generator](https://www.val.town/v/vawogbemi/solanaPayDemo) that allows merchants to generate QR codes for accepting Solana payments\n- [Jesi](https://www.val.town/u/jesi_rgb) built a [Secret Santa group creator app](https://www.val.town/v/jesi_rgb/secretsanta) that allows participants to join and be randomly matched while excluding certain pairings ([tweet](https://x.com/jesi_rgb/status/1856654868611530782))\n- Ben Wheeler [built a social media share link generator in 5 min with Townie](https://x.com/benjiwheeler/status/1859703701046427760)\n- [Prasham](https://www.val.town/u/prashamtrivedi) built a [Markdown to Rich Text Converter](https://prashamtrivedi-mdtort.web.val.run/) to paste reports from AI into Confluence or other systems which use rich text\n- [Ash](https://www.val.town/u/ashryanio) built put his [record collection online](https://ashryanio-randodisc.web.val.run/)\n- [Will](https://www.val.town/u/willthereader) built a [Thank You Note Generator](https://val.town/v/willthereader/ThankYouNoteGenerator) that generates personalized thank you notes for any occasion\n- [Max](https://www.val.town/u/maxm) built a beautiful [Draggable Emoji Garden](https://maxm-draggableemojigarden.web.val.run/) that lets you drag and drop emojis to create a garden\n- [Gueejla](https://www.val.town/u/gueejla) built a [Square Root Calculator](https://gueejla-squareroot.web.val.run/?inputNum=16) that visualizes the square root of a number",[],[191,1067],"8d7d3e02-b3c6-4ab9-957e-31d24e09a23a","2024-11-08-building-a-code-writing-robot.md","86bbb736-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Building a code-writing robot and keeping it happy\ndescription: Our experiences running LLMs in production\npubDate: '2024-11-08T05:00:00.000Z'\nauthor: Tom MacWright\nthumbnail: 'https://assets.blog.val.town/legacy/code-writing-robot/slide-1.png'\ntags: [engineering]\n---\n\nI recently gave this talk at a lovely event put on by our friends at\n[Jamsocket](https://jamsocket.com/), where we discussed different experiences\nrunning LLMs in production. With [Townie](https://blog.val.town/blog/townie/),\nwe've been dealing with the magic and eccentricities of this new kind\nof engineering.\n\n![Title slide with illustration of robot](https://assets.blog.val.town/legacy/code-writing-robot/slide-1.png)\n\nFor those who couldn't make it or are interested in following along, here\nare some slides and speaker notes.\n\n
\n\n![Introduction to Val Town](https://assets.blog.val.town/legacy/code-writing-robot/slide-2.png)\n\nVal Town is mostly a platform for running little bits of JavaScript that we call \"vals\", and it makes programming accessible to a lot of new people because vals can be super small, simple, and they don't require any configuration. But we've been using LLMs to make it accessible to even more people who want to create things with natural language.\n\n
\n\n

\n\nHere's the feature I'm talking about today - Townie. It's the Val Town bot. If you want to be old-fashioned, you can write the code yourself with your hands and fingers, but Townie will let you write it indirectly with English. It's similar in broad strokes to [Anthropic Artifacts](https://www.anthropic.com/news/artifacts) or [Vercel v0](https://v0.dev/), but one of the biggest differences is that the artifacts have full-stack backends, can be shared and forked, and so on.\n\n
\n\n![Flowchart of Townie's LLM usage, including using Sonnet and OpenAI](https://assets.blog.val.town/legacy/code-writing-robot/slide-4.png)\n\nWe're running a pretty plain-vanilla LLM setup! The meat and potatoes of Val Town are the challenges of running a lot of user code, at scale, with good security guarantees, and building community and collaboration tools. We aren't training our own models or running our own GPU clusters.\n\nSo for LLMs, we're riding on the coattails of [Anthropic](https://www.anthropic.com/) and OpenAI and running none of our own LLM infrastructure.\n\n
\n\n![LLMs are chaotic](https://assets.blog.val.town/legacy/code-writing-robot/slide-5.png)\n\nBut despite the simplicity, LLMs are pretty chaotic, and this has taken longer to build than we expected.\n\n
\n\n![LLMs are great but have drawbacks like being expensive, unreliable, unpredictable, and slow](https://assets.blog.val.town/legacy/code-writing-robot/slide-6.png)\n\nLLMs are unlike everything else on the backend: they can easily blow up your server costs, they have plenty of downtime, they're really hard to tune, and they can be super, super slow. Here's a quick run-through of how we've been dealing with these issues.\n\n
\n\n![Good LLM models are expensive and eat money](https://assets.blog.val.town/legacy/code-writing-robot/slide-7.png)\n\nThe good models are expensive. LLMs eat money. We're practicing financial discipline in a non-zero interest-rate world. How do we do it?\n\n
\n\n![We fix this by using different LLMs](https://assets.blog.val.town/legacy/code-writing-robot/slide-8.png)\n\nThe easiest cost and performance alpha is just using different providers for different tasks. For example, when someone creates a val, we want to generate the TypeScript code for that val, which is pretty nitpicky work: it needs to work with [Deno](https://deno.com/)'s TypeScript environment and it should reliably know whether it's running on the frontend or the backend, and so on. For that we use Sonnet, which is pretty expensive.\n\nBut then once we have the generated code, we want to generate a name for it, given both the prompt and response to that prompt: for that we use [OpenAI GPT-4o](https://openai.com/index/hello-gpt-4o/), which is cheap, fast, and good enough.\n\n
\n\n![Use am abstraction library because you will switch models](https://assets.blog.val.town/legacy/code-writing-robot/slide-9.png)\n\nAnd to use different models, it's really nice to use an abstraction library like [@vercel/ai](https://sdk.vercel.ai/docs/introduction), which is what we've been using. I know that there are several hundred options, so that isn't a fully-researched approved recommendation, but it's worked fine for us.\n\n
\n\n![Large context windows get expensive and chat interfaces lead to them](https://assets.blog.val.town/legacy/code-writing-robot/slide-11.png)\n\nSo the other problem is that this is a chat interface, and none of the LLM providers provide persistence: you have to put the whole chat history into context for every single new message. And the responses that we get from Claude can be pretty long - these are complete programs that we're getting back, lots of tokens. So we've basically dispatched three different approaches to this problem.\n\n
\n\n![We remind users on a regular basis that long conversations are worse and more expensive](https://assets.blog.val.town/legacy/code-writing-robot/slide-12.png)\n\nOur first line of defense against this is just reminding users that Townie gets dumber when you give it more input. This is pretty similar to the UIs in other tools that say the same thing. Basically once you get to the tenth reply or so, Townie will start losing the thread of what you wanted in the first place.\n\n
\n\n![caching old messages saves us a lot of money](https://assets.blog.val.town/legacy/code-writing-robot/slide-14.png)\n\nCaching old messages somehow lets Anthropic amortize the cost of parsing messages and turning them into embeddings or whatever kind of fancy internal representation they have. It costs more money to create the cached stuff initially, but less to use it later.\n\n
\n\n![We can visualize our cache usage with Observable](https://assets.blog.val.town/legacy/code-writing-robot/slide-15.png)\n\nStoring all of this in the database was also a pretty big win - so I was able to just query this from our messages table right into an [Observable](https://observablehq.com/) notebook.\n\nThis works! You can see some of the effects here, where these blue lines that are spiking up are people having super long conversations with Townie, but instead of hitting the tokens statistic, they're being read from the cache.\n\n
\n\n![Looking at a specific chart of cache usage](https://assets.blog.val.town/legacy/code-writing-robot/slide-16.png)\n\nHere's the cache read tokens, zoom and enhance style, colored by which thread they're part of: you can see some spots where it's just the same exact thread that's benefiting from the caching.\n\n
\n\n![Truncating old messages doesn't work](https://assets.blog.val.town/legacy/code-writing-robot/slide-17.png)\n\nA funny thing that doesn't work is trying to truncate old messages. Like, you're sending code back and forth, right - so if the LLM writes code and fixes some bug you have two copies of the code in the context window. So we tried replacing old code replies with comments, or something like \"truncated for brevity,\" but the LLM took this very literally. We found that if _we_ truncated code in the conversation history, the LLM decides to truncate code in its responses, too.\n\n
\n\n![Using diffs is too unreliable](https://assets.blog.val.town/legacy/code-writing-robot/slide-18.png)\n\nAnother potential solution that we've struggled to implement successfully is _diffs_. Claude can convincingly provide a good [unified diff](https://en.wikipedia.org/wiki/Diff#Unified_format) some percentage of the time, but it is very sloppy most of the time. Diffs will have the wrong line numbers or missing context, and they won't work with any off-the-shelf merge implementations. We've implemented our own fuzzy merge solution, but even then, diffs haven't been reliable enough for regular usage.\n\n
\n\n![Anthropic uptime](https://assets.blog.val.town/legacy/code-writing-robot/slide-20.png)\n![OpenAI uptime](https://assets.blog.val.town/legacy/code-writing-robot/slide-21.png)\n\nUnfortunately, we've also hit reliability issues with both Anthropic and OpenAI – both center around two nines of uptime recently. They're doing magical things, it's understandable that there are some speed bumps along the way! But there's not too much we can do about this without hosting the models ourselves, so we have just decided to really double down on implementing good error handling in our application and have recently adopted [neverthrow](https://github.com/supermacro/neverthrow) to improve how we use errors in TypeScript.\n\n
\n\n![Tuning our custom prompt with Braintrust](https://assets.blog.val.town/legacy/code-writing-robot/slide-26.png)\n\nWe've also spent a lot of time tuning our custom prompt, which felt like shooting in the dark until we started using a system for evals. [Braintrust](https://braintrust.dev/) has been great for this.\n\n
\n\n![Progress with braintrust](https://assets.blog.val.town/legacy/code-writing-robot/slide-27.png)\n\nHere's our progress with Braintrust over time. We highlight different bad behaviors that we want to forbid from our outputs: things like using `require()` instead of `import`, using the `alert()` method, or omitting code from results. Some evals rely on simple string matching, while others use a secondary LLM to judge results from the \"smart\" LLM.\n\n
\n\n![Notifications on pull requests](https://assets.blog.val.town/legacy/code-writing-robot/slide-28.png)\n\nBraintrust gives us notifications on each Pull Request, letting us know whether a given system prompt change is a regression or improvement in quality.\n\n
\n\n![Implementing our UI with streaming](https://assets.blog.val.town/legacy/code-writing-robot/slide-34.png)\n\nThe final challenge of Townie is simply implementing the UI, which is anything but simple. We've found a nice middle ground between an overly traditional request/response flow and a fully real-time, streaming, WebSocket-based infrastructure. We're using tRPC’s [Batch Stream Link](https://trpc.io/docs/client/links/httpBatchStreamLink), which pairs traditional POST requests with streaming responses. It gives us the nice type-safety of tRPC and lets us structure our responses using [Async Generators](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncGenerator) on the server-side.\n\n---\n\nThat's it! Thanks for following along.",[],[191,1074],"804ef780-9cfe-4412-8ece-5cf854a3bb2a","2024-11-01-val-town-town.md","86bb6ee8-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town Town\ndescription: Can we implement Val Town on Val Town?\npubDate: '2024-11-01T04:00:00.000Z'\nauthor: Max McDonnell\ntags: [engineering]\n---\n\n[![](https://assets.blog.val.town/legacy/val-town-town/screenshot.png)](https://maxm-valtowntown.web.val.run)\n\n_[Val Town Town](https://www.val.town/v/maxm/valtowntown) is Val Town\nimplemented on Val Town. Fork it, extend it, build stuff on it, or read on if\nyou'd like to learn more._\n\nVal Town lets you build all sorts of tools with vals: HTTP handlers, crons,\nemail endpoints, frontend applications, the list goes on. But _can you build Val\nTown with Val Town_? With a few caveats, the answer is yes: let's dive in.\n\n### Handling a Request with import\n\nSo we want to accept user code, parse and execute it, and use it to handle a web\nrequest.\n\nThe most naïve and solution is to use JavaScript's\n[dynamic import](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import)\nfunctionality to load their code.\n\nSupposing the user-provided HTTP handler looks like this:\n\n```ts\nexport default (req: Request) => Response.json(\"I work!\");\n```\n\nWe wrap it in our runtime by creating a\n[data URL](https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data)\nincluding the code, calling import, and running the default export\n([source](https://www.val.town/v/maxm/VTTnosecurity#L1)):\n\n```tsx val\nexport default async function (req: Request): Promise {\n const mod = await import(\n `data:text/tsx,${\n encodeURIComponent(\n `export default (req: Request) => Response.json(\"I work!\")`,\n )\n }`\n );\n return mod.default(req);\n}\n```\n\nThis [works](https://www.val.town/v/maxm/VTTnosecurity), but is very dangerous:\nthere's almost no separation between the system code and the user code. The user\ncode has access to the same memory and permissions that we do. It could read our\nsecrets in `Deno.env` or mess with objects on `globalThis`. This is barely\nbetter than the much-feared\n[eval()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval)\nmethod. _(Fun side note: Val Town was originally called \"Eval Town\" but was\nshortened when we started calling the blocks of code \"vals\".)_\n\nHow can we lock things down?\n\n### Using Web Workers\n\nI recently [published a Val](https://www.val.town/v/maxm/eval) that demonstrates\nhow to execute untrusted code safely, using Deno's\n[Web Worker](https://docs.deno.com/runtime/reference/web_platform_apis/#web-workers)\nand\n[granular permissions](https://docs.deno.com/runtime/reference/web_platform_apis/#specifying-worker-permissions).\nWe can build all sorts of things with this, like secure playgrounds for writing\ncode, or scripting interfaces for existing applications.\n\nWith a little cleverness, we can use Web Workers as a sandbox for user-defined\nHTTP handlers!\n\nBy wrapping things in a Deno Web Worker, we can isolate user code from the\nparent process. This'll give us much better security guarantees than the\n`import()` method did.\n\nHere's a very minimal example:\n\n```ts\nconst workerEval = (code: string, args: any[]): any => {\n return new Promise((resolve) => {\n const worker = new Worker(\n `data:text/tsx,${\n encodeURIComponent(`\nself.onmessage = async (e) => {\n const mod = await import(\\`data:text/tsx,\\${encodeURIComponent(e.data.code)}\\`);\n self.postMessage(mod.default(...e.data.args))\n}`)\n }`,\n { type: \"module\", deno: {permissions: {net: false}},\n );\n worker.postMessage({ code, args });\n worker.onmessage = (e) => {\n resolve(e.data);\n worker.terminate();\n };\n });\n};\n```\n\nNotice that we've added the permission `{net: false}` to lock things down even\nfurther. Now the running code also doesn't get to make HTTP requests.\n\nWeb Workers make us _work_ a little harder to call that user-provided function:\nwe're passing messages here instead of calling functions directly. That's the\ncost of better security.\n\nWe're passing messages back and forth using\n[postMessage](https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage),\nwhich has some limitations on the kinds of objects it can transmit – passing\nnumbers back and forth like this is fine:\n\n```ts\nawait workerEval(`export default (a, b) => a+b`, [1, 2]);\n// => 3\n```\n\nBut we can't just pass a complex object like a `Request`.\n\n```ts\nawait workerEval(`export default (req) => req`, [\n new Request(\"https:/val.town\"),\n]); // => {}\n```\n\nOnly\n[structured-cloneable](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm#supported_types)\ntypes are supported.\n\nThat's a bummer - how are we going to implement an HTTP handler if we can't send\nrequests and responses between the user code and the system?\n\n### Put a Server in the Worker\n\nIf we can't send `Request` and `Response` using `postMessage`, then let's set up\na web server in the Worker and communicate directly with that, instead of using\n`postMessage`. Here's how it works with\n[reqEvaltown](https://www.val.town/v/maxm/reqEvaltown) the library I've created\nto handle HTTP requests within a Worker.\n\n```ts\nexport async function serveRequest(\n req: Request,\n importUrl: string,\n): Promise {\n let port = await getFreePort();\n const worker = new Worker(\n `https://esm.town/v/maxm/evaltownWorker?cachebust=${crypto.randomUUID()}`,\n {\n type: \"module\",\n deno: {\n permissions: {\n net: [`0.0.0.0:${port}`, `esm.sh:443`, `esm.town:443`],\n read: false,\n write: false,\n },\n },\n },\n );\n worker.postMessage({\n port,\n importUrl,\n });\n worker.onmessage = (event) => console.log(event.data);\n setTimeout(worker.terminate, 10_000); // request timeout\n await isPortListening(port);\n const { pathname, search } = new URL(req.url);\n const url = new URL(\".\" + pathname, \"http://localhost:\" + port);\n url.search = search;\n const resp = await fetch(url, {\n method: req.method,\n headers: req.headers,\n body: req.body,\n redirect: \"manual\",\n });\n return resp;\n}\n```\n\nWithin this function, we:\n\n1. Find an available port.\n2. Spawn a Worker using our\n [script](https://www.val.town/v/maxm/evaltownWorker). We're running the\n Worker with limited permissions to allow the server to run and allow some\n https imports to work, but otherwise block network activity.\n3. Send a message to the Worker with the port and importUrl.\n4. Wait for the post to be available.\n5. Proxy our request to the Worker and return the response.\n\nPhew, certainly a little more complicated than before. Over in our\n[Worker code](https://www.val.town/v/maxm/evaltownWorker), we handle the rest of\nthe exchange.\n\n```ts\nconst start = (port: number, importUrl: string) => {\n import(importUrl)\n .then((m) => {\n mod = m;\n })\n .catch((err) => {\n modError = err;\n console.error(err);\n })\n .finally(() => {\n doneLoading = true;\n for (const { req, resolve } of pendingRequests) {\n handleRequest(req).then(resolve);\n }\n });\n Deno.serve(\n {\n port,\n onListen: () => {},\n onError: (err) => {\n console.error(err);\n return errorResponse(err);\n },\n },\n (req) => {\n if (!doneLoading) {\n return new Promise((resolve) => {\n pendingRequests.push({ req, resolve });\n });\n }\n return handleRequest(req);\n },\n );\n};\n```\n\nWe start a server, queue up requests that arrive while we're loading, and then\nserve any pending requests with the loaded module.\n\nThere we have it, now we're ready to handle requests with a user-provided\nhandler.\n\n### UI\n\nNow for the fun part: we head over to [Townie](https://www.val.town/townie) to\ncreate the UI. I asked for a simple site with minimal styling that accepted user\ncode in a text box and stored it in a SQLite database. From there I made a few\nmanual tweaks, hooked up the code in the database to the request handler\nendpoint, added some stylistic touches, and tweaked a few things with Townie's\nhelp.\n\nTry it out 👉 https://maxm-valtowntown.web.val.run\n\nThe functionality is quite limited compared to Val Town, but you can still do\nlots of things. You can\n[host TLDraw](https://maxm-valtowntown.web.val.run/handler/29), build\n[a React Playground](https://maxm-valtowntown.web.val.run/handler/31), and even\n[stream Server Sent Events](https://maxm-valtowntown.web.val.run/handler/30).\n\n
\n

\n
\n\nAnd there you have it: you can implement a subset of Val Town's functionality on\nVal Town itself!\n\nWorkers provide a viable security sandbox, but it's important to note that it is\nnot as isolated or safe as\n[the runtime strategy that we use today](https://blog.val.town/blog/first-four-val-town-runtimes/),\nwhich uses process isolation as well. Be careful when implementing this and any\nother security-sensitive code on Val Town in your accounts!\n\nThere are many more features that would be possible to implement:\n\n1. We currently spawn one Worker per request, would it be possible to share a\n Worker across many requests?\n2. We use `data:text/tsx` for import urls, could you implement your own module\n host so that you can use https imports?\n3. How could you capture request logs and request metadata for users to view?\n4. Could you provide user accounts? Environment variables? Database access?\n5. Val Town Town\n [cannot run itself](https://maxm-valtowntown.web.val.run/handler/37), can you\n fix that so we can make Val Town Town Town?!\n\nFork the code and see what you can make!\n",[],[191,1081],"c4158536-21db-4950-88e0-54e76386055b","2024-11-01-api-token-scopes.md","86bb573c-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: API Tokens Scopes\ndescription: Improving security with granular control over permissions\npubDate: '2024-11-01T04:00:00.000Z'\nauthor: Max McDonnell\ntags: [product, engineering]\n---\n\nWe've added API scopes to Val Town API tokens\nto improve security and give you more granular\ncontrol over your use of our [REST API](https://docs.val.town/openapi).\n\nWe have read/write scopes for:\n\n- `val` – [vals](https://docs.val.town/openapi#tag/vals)\n- `user` – [user account details](https://docs.val.town/openapi#tag/me)\n- `blob` – [blob storage](https://docs.val.town/openapi#tag/blobs)\n- `sqlite` – [sqlite database](https://docs.val.town/openapi#tag/sqlite)\n- `email` – [ability to send emails](https://docs.val.town/openapi#tag/emails)\n\nThe [API tokens page](https://www.val.town/settings/api)\nnow lets you view and configure the scope of each token.\n\n## Scoping vals\n\nThis change also lets you configure the scope of vals.\nWhen a val runs, we inject an API token into a `valtown` environment variable,\nwhich that val uses to access Val Town Standard Library functions, such as\nblob storage, sqlite, and email sending.\n\nWe are now allowing you to configure the scope of that injected `valtown`\nAPI token environment variable, effectively scoping the permissions of the val,\nensuring that it only has access to the\nresources that it needs.\n\n![](https://assets.blog.val.town/legacy/api-token-scopes/val-names-permissions.png)\n\nAdditionally, vals now receive short-lived API tokens when they are running, limiting the\npotential damage from a compromised val.\n\n## Safer Defaults\n\nStarting today, the default scope for vals will include every scope\nexcept `val:write` and `user:write` to limit potential damage from\nmisconfiguration, vulnerable libraries, or account compromises.\nWithout the ability to mutate val resources, unauthorized actors now cannot execute\nnew code or modify existing code in your account.\n\nFurther down the road, we're considering even fewer default scopes,\nand having a slick UI to let you opt-into extra scopes at runtime\nwhen your code accesses them. Think of the way iOS apps ask for\npermissions when they need them.\n\n## Github Secret Scanning\n\nWe're joining the [Github Secret Scanning program](https://docs.github.com/en/code-security/secret-scanning/introduction/about-secret-scanning), so if your API token is leaked\nin a public Github repository, we'll be notified and can revoke the token.\n\n## Migrating current users\n\nWe've updated 99% of existing API tokens to use these safer\ndefaults.\n\nWe identified which users are using the `val:write` scope, and have held off migrating their tokens, and work with them to upgrade. If you're one of these users,\nexpect an email shortly.\n\nWe're always looking for ways to make our platform more secure. If you have\nsecurity-related feedback or new feature suggestions please make yourself heard\nin our [Discussions](https://github.com/val-town/val-town-product/discussions)\nor reach out to security@val.town.\n",[],[191,1088],"0b32365f-3168-4440-8a19-2b98b605557d","2024-10-31-fal.md","86bb2ca8-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town + Fal Partnership\ndescription: Bringing lightning fast AI image generation to Val Town\npubDate: '2024-10-31T04:00:00.000Z'\nauthor: Steve Krouse\ntags: [engineering]\n---\n\n\n\n[Fal](https://fal.ai/) is an API for generating AI images lightning fast.\n\nToday we're announcing a partnership with Fal to bring their API to Val Town users. You can now use Fal in your vals right away – no Fal API key necessary.\n\n## Quickstart vals\n\nThe quickest way to get started is to fork one of these starter apps:\n\n

\n \n $\"Simplest$ \n

\n Simplest possible Fal app\n

\n \n

\n\n

\n \n $\"Simplified$ \n

\n \"Faltown\" – variants, API client, blob storage\n

\n \n

\n\nClick \"Fork\" on either of these apps, and you'll instantly get that app deployed in your Val Town account. No further setup necessary. You can then ask [Townie](https://val.town/townie) to customize it to your needs. Want a DnD character generator? [No problem!](https://stevekrouse-genuinevioletswordtail.web.val.run/) Want a tool to help you design your next tattoo? [Done!](https://stevekrouse-tattoodesignapp.web.val.run/)\n\n## Instructions for Townie\n\nAnother quick way to get started is to copy & paste instructions into Townie. The following instructions will give Townie everything it needs to use [Flux Schnell](https://fal.ai/models/fal-ai/flux/schnell),\na state-of-the-art 12b parameter image model by [Black Forest Labs](https://blackforestlabs.ai/#get-flux).\nFal supports [the whole Flux suite](https://fal.ai/flux), and you can use any of them by changing the `FAL_ENDPOINT` variable.\n\n````markdown\nThe following are instructions on how to use Fal,\nan image generation API, in your Val Town app.\n\nImport the necessary modules:\n\n```ts\nimport { falProxyRequest } from \"https://esm.town/v/stevekrouse/falProxyRequest\";\nimport { createFalClient } from \"https://esm.sh/@fal-ai/client\";\n```\n\nSet up the Fal client on the client side:\n\n```ts\nconst FAL_ENDPOINT = \"fal-ai/flux/schnell\";\nconst fal = createFalClient({\n proxyUrl: \"/api/fal/proxy\",\n});\n```\n\nUse the Fal client to handle image generation requests on the client:\n\n```ts\nconst result = await fal.subscribe(\"fal-ai/flux/schnell\", {\n input: {\n prompt,\n image_size: \"landscape_4_3\",\n num_inference_steps: 4, // must be 12 or less\n num_images: 1,\n enable_safety_checker: true,\n sync_mode: true,\n },\n});\nconst image_url = result.data.images[0].url;\n```\n\nSet up the Fal proxy request on the server side:\n\n```ts\nexport default async function server(req: Request): Promise {\n const url = new URL(req.url);\n if (url.pathname === \"/\") {\n /* Return HTML */\n }\n\n if (url.pathname === \"/api/fal/proxy\") {\n return falProxyRequest(req);\n }\n}\n```\n````\n\n## Limitations\n\nThis partnership is intended for prototyping and small-scale projects, and is rate-limited as follows:\n\n- Val Town Free Users: 10 requests per hour\n- Val Town Pro Users: 50 requests per hour\n\nWhen your project needs scale, you can sign up for a [Fal API key here](https://fal.ai/).\n\n## How it works\n\nNormally to use the Fal API, you'd need to sign up for an API key on their website, but with this partnership, you can start prototyping with Fal in your vals without any setup. This is possible because we're proxying requests to the Fal API through [this HTTP val](https://www.val.town/v/fal/faltownproxy), which uses your Val Town API token to authenticate and rate-limit you. In this way you're using your Val Town API key in place of a Fal API key.\n\nThis is how our [OpenAI](https://docs.val.town/std/openai/) integration works as well, through [this proxy](https://www.val.town/v/std/openaiproxy). I recently built [a similar proxy](https://www.val.town/v/stevekrouse/socialDataProxy) for [SocialData](https://socialdata.tools/), a Twitter data API provider, which Val Town Pro users can use to run this [twitterAlert](https://www.val.town/v/stevekrouse/twitterAlert) notifying.\n\nWe think of these 'userspace proxies' as the beginnings of what we hope will become 'Val Town Marketplace'. If you want to bring your API to Val Town users, email me at steve@val.town.",[],[191,1095],"be50aca3-adc7-440d-88b8-5140add315c5","2024-10-11-val-town-newsletter-21.md","86bad8d4-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town Newsletter 21\ndescription: Townie upgrades & faster HTTP vals\npubDate: '2024-10-11T04:00:00.000Z'\nauthor: Steve Krouse\ntags: [newsletter]\n---\n\n[Townie](https://val.town/townie), our AI code assistant, got huge upgrades this month. We fixed a dozen bugs and shipped many of your feature requests. We also made big strides in HTTP val performance, are deprecating some old APIs, and made a number of other upgrades and fixes, detailed below.\n\nYou all created hundreds of awesome vals last month! Our favorite 50 are featured at the bottom of this newsletter.\n\n## 🤖 Townie upgrades\n\nTownie can blow-your-mind and it can make you tear your hair out. We're working hard to make it more of the former and less of the latter. Here's some of the improvements we've made:\n\n- Townie can create & edit any kind of val (not just HTTP vals)\n- You can upload images to Townie to help it understand your request\n- You can view requests and logs from within Townie\n- We made it easier to revert to past versions (more improvements incoming)\n- Townie can detect and fix syntax errors and client-side errors\n- Tom fundamentally re-architected to be less buggy\n- Jackson setup a suite of evals to improve Townie's accuracy\n- Townie diffs are copyable without the `+` and `-` characters\n- Diffs are off by default. You can request them explicitly\n- Townie doesn't overwrite your hand-written code as often\n- Townie has a [docs page](https://docs.val.town/reference/townie/) with tips & tricks\n\nPlease continue to reach out when Townie does something amazing or awful. Your feedback is invaluable. And as some of you know, I (Steve) am always happy to jump on a call or try to fix bugs for you. I'm steve@val.town.\n\n## ⚡️ Fast Vals\n\nVals continue to get faster and more scalable. Now [a 'hello world' val](https://www.val.town/v/stevekrouse/hello_world) responds in about 50ms, which is comparable to other cloud providers. Our servers are in Ohio and we're testing here in New York, so your mileage may vary if you live further from Ohio.\n\n## Deprecating old APIs\n\nWe are deprecating a number of old APIs that are no longer in use:\n\n- Express vals – discontinued on 10/7/2024\n- Old HTTP runtime – migrating on 10/20/2024 ([how to upgrade](https://docs.val.town/troubleshooting/migrating-deprecated-http-vals/))\n- Eval API – discontinuing on 11/1/2024\n\nThese changes effect less than 1% of our users. If you're effected, you will receive an email with more information on how to upgrade.\n\nThe exciting part about deprecations is that it means we'll be able to ship faster because we won't be maintaining old code that almost nobody is using. Max rebuilding our code-running infrastructure to even faster and more scalable, and not having to support these old APIs will make his job a lot easier.\n\n## Prettier errors\n\nHTTP vals now have prettier errors. They sometimes have stack traces and link to the relevant source. (We're working improving the stack traces & links.)\n\n $\"Pretty$ \n\n## Videos\n\nVideos are a great way to learn the pro tips & tricks for Val Town.\n\nI did two livestreams with wonderful guests:\n\n- [**Creating design tools**](https://www.youtube.com/watch?v=OtqmldWYJoo) with [Alexander Vilinskyy](https://x.com/vilinskyy) – remaking his [viral SVG generator](https://x.com/vilinskyy/status/1834982920685269308) with Townie\n- [**Creating a Dog-E-Dex**](https://www.youtube.com/watch?v=vsvBroU9I68) with [Cynthia](https://x.com/yescynfria), a designer at Notion, who has a dream to meet every breed of dog\n\nAnd a handful of other videos:\n\n- **How to add Google Login to your val in 30 seconds** - [quick demo](https://x.com/stevekrouse/status/1844426213353455701), [longer explanation](https://x.com/stevekrouse/status/1842328023049117787)\n- [**Watch AI make me a SQLite Dashboard from 7 in minutes**](https://youtu.be/pJJfCMsklqg)\n- [**Watch me make & deploy a blog in 24 minutes with AI**](https://x.com/stevekrouse/status/1836457427677458826)\n- [**Lucia Auth is DEAD! Long live implementing sessions by hand?**](https://youtube.com/live/lmEgcuVm0jI)\n- [**Building a Bookshelf Scanner with AI**](https://youtube.com/live/xPvLUHvTgwM)\n\n## 🎉 Other updates\n\n- The Val Town team now gets woken up for downtime (PagerDuty) 🚨\n- AI bots are blocked from HTTP vals\n- Updated Deno types\n- Fixed a renaming bug – thanks Robin Sloan for the report 🙏\n- Fixed logs and errors with new HTTP runtime\n- Removed pinned vals UI cruft\n- Changed the order of logs (newest at the bottom)\n- Disabled autofill on env variables – thanks Matthew Bunday for the suggestion 🙏\n- Improved our list of adjectives for new val names\n- Made POST v1/vals/id a no-op if no changed – thanks Andres Rodriguez for [the request](https://github.com/val-town/val-town-product/issues/185) 🙏\n\n## 🛣️ Roadmap\n\nThe main projects over the next couple months are:\n\n- Discontinuing the old HTTP runtime and Eval API\n- Continued Townie AI improvements\n- Fine-grained API Token Scopes\n- Val Projects (to manage complex multi-val collections)\n- Townie writing READMEs\n- New val runtime architecture for faster & more scalable vals\n\nPlease [make feature requests and vote](https://github.com/val-town/val-town-product/discussions?discussions) on what you'd like to see next.\n\n## 🌟 Community vals\n\n- Dan made [a bunch of vals to work with the Campsite API](https://www.val.town/u/campsite) and [a demo video](https://x.com/DanPhilibin/status/1833619842303463748)\n- Sarah made a [Slack notifier](https://x.com/SarahChieng/status/1833602967410528594) whenever topics of interest are mentioned on Twitter, Hacker News, and Reddit, powered by Browserbase\n- Jonnie made a [Slack bot](https://x.com/destroytoday/status/1834408549549494302) that reminds [designtown.nyc](https://designtown.nyc) folks of the keypad code\n- Guy [generates X slams Y memes](https://x.com/gvy_dvpont/status/1841818282719924430), autotweeted to Mastadon\n- Thej wrote [an excellent blog on AQI alerts in India](https://thejeshgn.com/2024/09/11/simple-imd-alerts/)\n- Steve upgraded his [twitter alert](https://www.val.town/v/stevekrouse/twitterAlert) to use SocialData's API, so any Val Town Pro user can seamlessly fork it\n- Guy built a site to convert [Landscape Video -> Blurred Portrait](https://x.com/gvy_dvpont/status/1840894297698128246)\n- Steve [converts error emails from Render](https://www.val.town/v/stevekrouse/renderEmails) into alerts on Discord and Github Issues, assigned to the right person, in the current milestone\n- The Crossmint team made a [bunch of awesome vals](https://x.com/stevekrouse/status/1835778062237008084)\n- Nathan made a [color app](https://x.com/stevekrouse/status/1836031718249083128) for his daughter\n- Steve launched [stringify.org](https://stringify.org) a mini-site that converts any text into a valid string you can paste into a JS file\n- Ash made a [chore app](https://x.com/stevekrouse/status/1836136881702031425) for his daughter\n- Eric's eight-year-old made a [word picture match game](https://efoley-wordpicturematch.web.val.run/) [for his two-year-old using Townie](https://x.com/ericdfoley/status/1840478366408798277)\n- Aman built a [flux playground](https://x.com/_amankishore/status/1840522105403961439) with Modal\n- Eli made [an info site for California's 2024 ballot propositions](https://madprops2024.org/)\n- Seamus made a [perlin noise animation generator](https://x.com/stevekrouse/status/1838666792484638992)\n- Wolf made a [chrome extension](https://x.com/stevekrouse/status/1838592604734202102) to run github gists in Val Town\n- Steve made a [domain name finder](https://x.com/stevekrouse/status/1834709297869070764)\n- Seamus made a [drum machine](https://x.com/stevekrouse/status/1838590564993765706)\n- Steve made a [github graph drum machine](https://x.com/stevekrouse/status/1839421869466018061) and Tom made it [sound nicer](https://x.com/stevekrouse/status/1839700676760186896)\n- Harry made a snazzy [LangChain Ambassador card](https://x.com/zhanghaili0610/status/1840602999443640343)\n- Steve made a [synth playground](https://x.com/stevekrouse/status/1838248962865647838) where you can create melodies with a sequencer and play them\n- Carl built a [DnD character creator](https://x.com/stevekrouse/status/1832391129586638943)\n- Jan made a very good prompt for [news & research with Groq](https://x.com/yawnxyz/status/1841613186531000522)\n- Sharan made a [personal writing assistant](https://x.com/sharanbabu2001/status/1832549769111015900) with Cerebras\n- Max sent some Val Town users [personalized deprecation emails](https://x.com/stevekrouse/status/1843310733511324131)\n- Steve created [helper functions](https://www.val.town/v/stevekrouse/fileToDataURL) for base64 encoding images for sending to ChatGPT, Anthropic, and Gemini\n- Stefan made an [allergy food tracker](https://x.com/stefanboesen/status/1833364392232296897)\n- Moe made a [hex maze visualization](https://x.com/0xMoe_/status/1840079699344588805)\n- Ankit made a [hold the clock button game](https://x.com/ankitkr0/status/1833410323543404961)\n- Dustin made a [slack bot](https://x.com/dctalbot/status/1840478366408798277) that translates messages to pirate speak\n- Steve made a [realtime chess game](https://x.com/stevekrouse/status/1838654708342481267) where you play as the king and move with the arrow keys\n- Paras made a [book recommendation mind map](https://x.com/paraschopra/status/1835311841024614581)\n- Steve made a [bookshelf scanner](https://x.com/stevekrouse/status/1835740596884951317) with Gemini 1.5 flash\n- Ash made a [val to show how to get and render images from Val Town Blob Storage](https://www.val.town/v/ashryanio/getBlobAndRenderAsImage)\n- Steve made a demo showing how [Cerebras is an LLM fast enough for sliders](https://x.com/stevekrouse/status/1842308619897069918)\n- Konrad made a [retro car inventory app](https://val.town/v/konradmoneta/carInventory)\n- @macro21kgb made a [site to convert WebP images to PNG](https://www.val.town/v/macro21kgb/imageConverterSite)\n- DT made [a twist on the classic Game of Life](https://www.val.town/v/dthyresson/gameOfLifeSite)\n- Steve made a mini-site for calculating [idealized weekly growth](https://stevekrouse-growth.web.val.run/)\n- Nico made a [JSON visualizer](https://x.com/nbbaier/status/1840308862856896721)\n- Steve cloned Dan Shipper's [book photo analyzer](https://x.com/stevekrouse/status/1836781662161588486)",[],[191,1102],"21eba855-2a6e-4872-a226-a9975ee81f8e","2024-09-26-townie.md","86baafb2-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Introducing Townie AI\ndescription: 'Meet Townie, our AI assistant that helps you ship apps fast'\npubDate: '2024-09-26T04:00:00.000Z'\nauthor: Steve Krouse\ntags: [product]\n---\n\nLast month, we launched a major overhaul of Townie, our AI assistant that helps you build apps. Val Town was already the best website to turn an idea into a deployed app. Townie can help you ship much faster.\n\n

\n\n\nIn the following video, I use Townie to make an RSVP page – with a backend and database – in just a few prompts and a couple minutes.\n\n

\n\nBut the best way to see what Townie is like is to just go play with it.\n\n\n Start creating with Townie\n\n\n## Community creations\n\nYour creativity never ceases to amaze us. Here are some of the cool things built with Townie so far.\n\n#### Fal Image Generator\n\nIn under a minute and a single prompt, Batuhan built a tool to generate images with Fal AI.\n\n\n \n $\"\"$ \n \n\n\n#### Twitter Tracker\n\nCharmaine built a site that tracks mentions of her company's product on Twitter.\n\n\n \n $\"\"$ \n \n\n\n#### Blog summaries\n\nWith a handful of prompts, Peter made a neat tool to get Claude to summarize a blog post just by pasting a link.\n\n\n \n $\"\"$ \n \n\n\n#### Mood tracker\n\nMarcus was able to build a mood-tracking app in minutes.\n\n\n \n $\"\"$ \n \n\n\n#### Slack Bot\n\nJohnnie built a Slack bot for his team.\n\n\n \n $\"\"$ \n \n\n\n#### The sky's the limit\n\nTownie can make almost anything you can imagine. Here's more inspiration from the community to get you started:\n\n- [Carl Lange](https://www.val.town/u/csl_) built a [DNDoodle](https://dndoodle.redfloatplane.lol/), a scheduling app for D&D groups with improved features like setting poll end dates and identifying the DM's availability separately\n- [David Thyresson](https://www.val.town/u/dthyresson) built a [Bedtime Story Maker](https://dthyresson-bedtimestorymaker.web.val.run/) that generates personalized children's stories with illustrations using OpenAI and Fal AI\n- [@yawnxyz](https://www.val.town/u/yawnxyz) built [Newsly](https://www.val.town/v/yawnxyz/newsly), a val that stores emails in a SQLite database, allowing you to access your newsletters\n- [@jdan](https://www.val.town/u/jdan) built an [interactive Wikipedia Family Tree Crawler](https://www.val.town/v/jdan/infoboxCrawler) that visualizes parent-child relationships by crawling Wikipedia infoboxes\n- [@moe](https://www.val.town/u/moe) created an awesome [Number Chain Puzzle game](https://www.val.town/v/moe/numbergame) where players drag between adjacent squares with the same number to score points\n- [@roramigator](https://www.val.town/u/roramigator) built a [Hacker News digest](https://www.val.town/v/roramigator/hackernewsdigest) that emails you the top stories from Hacker News every 3 days\n- [Giles](https://www.val.town/u/gilesb) built [an email expenser](https://www.val.town/v/gilesb/emailExpenser) that processes email content using GPT4, extracts key information like income and expenses, and logs it to a Google Sheet\n- [@roramigator](https://www.val.town/u/roramigator) created a [idle clicker game about spreading a religion](https://val.town/v/roramigator/religionClickerGame)\n\n## Resources\n\nWe have some helpful [prompting tips in our docs](https://docs.val.town/reference/townie/).\n\nYou can also read more about [how we built Townie](https://assets.blog.val.town/legacy/how-we-built-townie) and [Simon Willison's blog post about Townie's prompt](https://simonwillison.net/2024/Aug/28/system-prompt-for-townie/).\n\nIf you get stuck, please reach out in [our Discord](https://discord.gg/dHv45uN5RY), or email me directly (steve@val.town), and include a link to your val. We're here to help!\n\n## Start building today\n\nHave an idea for a site, game, or app for yourself, friends, family or work?\n\nTownie is free and available now in public beta. We can't wait to see what you build.\n\n\n Start creating with Townie\n",[],[191,1109],"db010bd6-eae5-43f0-86fa-28de5f55a139","2024-09-07-val-town-newsletter-20.md","86ba15e8-7bcf-11f1-b068-1607ee4eb77e",812,["D",1787078486614],"---\ntitle: Val Town Newsletter 20\ndescription: \"Townie AI, Faster vals, and Lockfiles\"\npubDate: \"2024-09-07T04:00:00.000Z\"\nauthor: Steve Krouse\ntags: [newsletter]\n---\n\nThe newest version of Townie is _incredible_ at writing vals. It creates\ndeployed apps with a backend and database in seconds. Honestly, if you have five\nminutes, [you gotta try it out](https://val.town/townie)!\n\nWe shipped a number of other improvements, including a huge speedup to vals,\nlockfiles to secure your code, the ability to receive email attachments, a new\nversion history UI, and opened up support for hundreds more NPM packages.\n\nQuick hiring update: we're still looking for a couple more engineers to join our\nfounding team in Brooklyn. If you have any brilliant friends in NYC, please send\nthem our way!\n\nThere was such an overwhelming amount of amazing vals created this month that I\nhad to [build a custom tool](https://www.youtube.com/watch?v=-PI5CIkcETc) to\nhelp me write the \"community vals\" section of this newsletter. You guys rock!\n\n## 🤖 Townie AI\n\nThis month we released our third wholesale rewrite of Townie, and it's\n_incredible_. It's now able to generate full-stack applications with a backend\nand database in seconds. It's like Claude Artifacts, with a backend.\n\n

\n\n[If](https://x.com/stevekrouse/status/1826728366549975506)\n[you](https://x.com/stevekrouse/status/1824823444485415000)\n[want](https://x.com/stevekrouse/status/1823825618246779241)\n[more](https://x.com/stevekrouse/status/1826336463543566826)\n[Townie](https://x.com/stevekrouse/status/1828831012966777263)\n[demos](https://x.com/stevekrouse/status/1825978445697819067),\n[I'm](https://x.com/stevekrouse/status/1828864172567712103)\n[tweeting](https://x.com/stevekrouse/status/1827046592806891580)\n[up](https://x.com/stevekrouse/status/1827036933823848553)\n[a](https://x.com/stevekrouse/status/182636536262528211)\n[storm](https://x.com/stevekrouse/status/1826356154525778404)\n[of](https://x.com/stevekrouse/status/1826317269657395227)\n[them](https://x.com/stevekrouse/status/1831046366904353066). If you want to\nknow the backstory behind Townie, JP released\n[an excellent blog post about how we prototyped Townie](/codegen).\n\n## ⚡️ Faster Vals\n\nVals now run way faster. You'll see 2-10x speedups, depending on the context.\nWe're finally competitive with premier hosting providers. For example if you hit\nan HTTP val's endpoint in NYC, you'll often get a response in under 100ms. (Our\nservers are in Ohio. Multi-region is still a while off. Sorry Japan!)\n\nThis is thanks to our new load balancer, which:\n\n1. Routes around a slow part of our stack (more on this later)\n2. Directs traffic to where your val is already running, so you skip cold starts\n\nExpect more improvements and a fuller blog on this shortly!\n\n## 🔒 Lockfiles\n\nWe've introduced lockfiles in Val Town to enhance stability, security, and\nperformance. When you save a val, a deno.lock file is generated, ensuring the\nsame versions of dependencies are used in the future. This prevents issues\ncaused by changing libraries and malicious dependencies, and improves\nperformance by caching the dependency tree. Learn more in\n[our announcement post](/lockfiles).\n\n![Charts of requests to esm.town and worker initialization time decreasing](https://assets.blog.val.town/legacy/lockfiles/metrics.png)\n\n## 💸 We joined the OSS Pledge\n\nWe're excited to announce that we've joined the\n[OSS Pledge](https://osspledge.com/). We commit to donating at least $2k per\nyear per full-time developer at Val Town to open-source. You can learn more in\n[our announcement post](/oss). We donated have donated $9,600 so far in 2024:\n\n| Project | What it does | Amount |\n| ------------------------------------------------ | ---------------------------------------- | -----: |\n| [esm.sh](https://esm.sh/) | CDN for ESM modules | $1000 |\n| [Sindre Sorhus](https://github.com/sindresorhus) | Prolific open source maintainer | $1000 |\n| [tsx](https://tsx.is/) | Easy way to run TypeScript directly | $1000 |\n| [Fastify](https://fastify.dev/) | Web framework | $1000 |\n| [Zod](https://zod.dev/) | Data validation library | $1000 |\n| [Unified](https://unifiedjs.com/) | Markdown transformation module ecosystem | $1000 |\n| [Drizzle](https://orm.drizzle.team/) | Database query builder | $800 |\n| [CodeMirror](https://codemirror.net/) | Code editor framework | $800 |\n| [Mock Service Worker](https://mswjs.io/) | Testing helper modules | $500 |\n| [rclone](https://rclone.org/) | Upload and download files to S3 and more | $500 |\n| [Hono](https://hono.dev/) | Web standards-based web framework | $500 |\n| [husky](https://typicode.github.io/husky/) | Nice way to set up git hooks | $500 |\n\n## 🎉 Other updates\n\n- Email handler vals can now receive attachments – thanks\n [danielrw7](https://github.com/danielrw7) for the\n [feature request](https://github.com/val-town/val-town-product/discussions/75)\n 🙏\n- [New version history UI for vals](https://x.com/stevekrouse/status/1831027036220989664)\n- [Support for hundreds more NPM packages](https://x.com/mxmcd/status/1828881056969150616)\n- [Ability to trigger LLM completions manually](https://x.com/stevekrouse/status/1821229088209862848)\n (alt-\\\\)\n- [Go to a val's full-screen editor by pressing .](https://x.com/stevekrouse/status/1822023518793404828)\n (like on GitHub) - thanks [David Siegel](https://val.town/u/dvdsgl) for the\n suggestion 🙏\n- [Redesigned Profile menu](https://x.com/stevekrouse/status/1819005362877407357)\n (in the top-right corner)\n- [Ability to set HTTP headers in `@std/email`](https://github.com/val-town/val-town-product/discussions/142) -\n thanks [wilhelm](https://github.com/wilhelmklopp) for the suggestion 🙏\n- Val READMEs are now CodeMirror, with LLM completions (triggered manually,\n alt-\\\\)\n- A new val settings tab\n- Protections for certain kinds of denial of service attacks\n- \"Forked from\" information on val cards\n- Updated Deno to 1.46.3\n\n## 🛣️ Roadmap\n\nHere's some of the top features we're planning to work in the coming months.\nPlease\n[make feature requests and vote](https://github.com/val-town/val-town-product/discussions?discussions)\non what you'd like to see next.\n\n- Continued Townie AI improvements\n- Rolling out `HTTP (Preview)` as the default `HTTP` val type\n- Continued performance improvements\n- [LLM-generated READMEs](https://github.com/val-town/val-town-product/discussions/26)\n\n## 🌟 Community vals\n\n- Guy Dupont made an incredible\n [digital \"bumper sticker\" that shows what he's listening to right now](https://x.com/gvy_dvpont/status/1831412749647409286) +\n [an amazing video explaining it](https://www.youtube.com/watch?v=mWRPRW6pHIY)!\n- Ben Bleikamp wrote a couple of\n [crons to automate some posts on campsite.co and send posts to Slack](https://x.com/bleikamp/status/1832073946151207029)\n- [Schpet](https://www.val.town/u/schpet) built a val that sends a daily post to\n Slack listing all the Linear issues assigned to you that changed status,\n [powered by Val Town](https://www.val.town/v/schpet/linearStandup)\n- Guy Dupont\n [injected a val into his hand](https://x.com/gvy_dvpont/status/1823342935208755558)!!!\n- [csl\\_](https://www.val.town/u/csl_) built a\n [DNDoodle](https://dndoodle.redfloatplane.lol/), a scheduling app for D&D\n groups with improved features like setting poll end dates and identifying the\n DM's availability separately\n- [charmaine](https://x.com/charmaine_klee) made a val to help her at work\n (Snapchat)\n [track all mentions of SnapAR or Lens Studio on Twitter](https://x.com/charmaine_klee/status/1831525120294236222)\n- [@isidentical](https://www.val.town/u/isidentical) built a\n [Flux Image Generator](https://www.val.town/v/isidentical/fluxImageGenerator)\n that generates images from user prompts using the fal.ai API in\n [under 1 minute from scratch with Townie](https://x.com/isidentical/status/1828614519259832725)!\n- Larry Hudson made\n [a netifly plugin that allows you to redirect with authentication](https://netlify-plugin-redirect-with-auth.netlify.app/)\n so you can submit to private vals\n- [Sam Rose](https://x.com/samwhoo) built a\n [collaborative 6-word story app](https://x.com/samwhoo/status/1816582937707528586)\n- Simon Willison wrote an excellent\n [blog post](https://simonwillison.net/2024/Aug/28/system-prompt-for-townie/)\n on Townie's prompt\n- [David Siegel](https://www.val.town/u/dvdsgl) used Glide's bulk API\n [to sync all of Glide’s issues and PRs into an app](https://x.com/dvdsgl/status/1817397810880479315)\n- [Moe](https://www.val.town/u/moe) built an awesome\n [SpaceX Launch tracker](https://moe-spacex.web.val.run/) that displays\n upcoming and past SpaceX launches with data visualizations\n- [@wilg](https://val.town/u/wilg) made [tax.moi](https://tax.moi/) to explain\n how graduated federal income tax works\n- [Perbhat Kumar](https://www.val.town/u/PerbhatK) built\n [an app that weighs his coffee using the mac trackpad](https://x.com/PerbhatK/status/1831860667604934904)\n- [@jxnblk](https://val.town/u/jxnblk)\n [rebuilt](https://x.com/jxnblk/status/1826252329940832560)\n [jxnblk.com](https://jxnblk.com/) on val town – it's awesome!\n- [@moe](https://val.town/u/moe) created a\n [P5.js helper for Val Town](https://val.town/v/moe/p5) and used it to make\n [two](https://val.town/v/moe/exampleP5sketch) awesome\n [examples](https://val.town/v/moe/lostKeys)\n- The [Cannoli](https://cannoli.website/) docs\n [teach their users how to use Val Town](https://docs.cannoli.website/Blog/Release+Notes+8-11-2024#Val+Town)!\n- [David Thyresson](https://www.val.town/u/dthyresson) built a\n [Bedtime Story Maker](https://dthyresson-bedtimestorymaker.web.val.run/) that\n generates personalized children's stories with illustrations using OpenAI and\n Fal AI\n- [Jackson](https://www.val.town/u/jxnblk) built\n [test](https://www.val.town/v/jxnblk/test), a utility and badge to show test\n suite status in your val readme.\n- [Justin Bennett](https://www.val.town/u/just_be) is creating\n [content-addressable JavaScript](https://just-be.dev/rc/impossible-day)\n- [Peter Liu](https://www.val.town/u/peterqliu) revamped their\n [VT App v2](https://www.val.town/v/peterqliu/vt_app_v2) with JWT\n authentication, user accounts, and a way to change emails or delete accounts.\n- [Carl Lange](https://www.val.town/u/csl_) made\n [a site that can measure your tennis speed based on the sound of the ball hitting the racket](https://tennisspeed.redfloatplane.lol/)\n- [yawnxyz](https://www.val.town/u/yawnxyz) built\n [Newsly](https://www.val.town/v/yawnxyz/newsly), a val that stores emails in a\n SQLite database, allowing you to access your newsletters\n- [@jdan](https://www.val.town/u/jdan) built an\n [interactive Wikipedia Family Tree Crawler](https://www.val.town/v/jdan/infoboxCrawler)\n that visualizes parent-child relationships by crawling Wikipedia infoboxes\n- [@moe](https://www.val.town/u/moe) created an awesome\n [Number Chain Puzzle game](https://www.val.town/v/moe/numbergame) where\n players drag between adjacent squares with the same number to score points\n- [@roramigator](https://www.val.town/u/roramigator) built a\n [Hacker News digest](https://www.val.town/v/roramigator/hackernewsdigest) that\n emails you the top stories from Hacker News every 3 days\n- [Giles](https://www.val.town/u/gilesb) built an email expenser that processes\n email content using GPT-4, extracts key information like income and expenses,\n and logs it to a Google Sheet,\n [powered by Val Town](https://www.val.town/v/gilesb/emailExpenser).\n- [roramigator](https://www.val.town/u/roramigator) created a\n [idle clicker game about spreading a religion](https://val.town/v/roramigator/religionClickerGame)\n- [@stevekrouse](https://www.val.town/u/stevekrouse) built a\n [typing challenge game](https://www.val.town/v/stevekrouse/typetest) where you\n can test your typing speed and submit your score to a leaderboard\n- [David Thyresson](https://www.val.town/u/dthyresson)\n [generating movie mashups](https://www.val.town/v/dthyresson/movieMashup) with\n posters, titles, and treatments.\n- [@ankit](https://www.val.town/u/ankitkr0) built a\n [website that tracks clicks](https://clicktracker.ankitkr0.web.val.run/) and\n adds confetti with each click, backed by a SQLite database.\n- [@all](https://www.val.town/u/all) built an\n [Icon Explorer](https://www.val.town/v/all/iconExplorer) that lets you search,\n filter, and copy React icon components from popular libraries like Feather,\n Font Awesome, and Ant Design.\n- [David Thyresson](https://www.val.town/u/dthyresson) built\n [Play Me A Song](https://www.val.town/v/dthyresson/playMeASong), a web app\n that lets you search for and open songs in Spotify.\n- [Adam Kazwell](https://www.val.town/u/kaz) built a\n [site that helps you know understand your personal healthy boundaries](https://x.com/kaz/status/1829022200147706175)\n- [@szymonator](https://val.town/u/szymonator) made a\n [an API demo to showcase how the Teamsurfboard API can be used](https://val.town/v/szymonator/surfboardDemoSchedule2)\n- [@all](https://www.val.town/u/all) built a\n [Val Town Prompt Generator](https://promptgen.all.esm.town/) that creates\n optimized prompts for Val Town\n- [@thatsmeadarsh](https://www.val.town/u/thatsmeadarsh) built a script that\n allows OpenAI assistants to\n [star a GitHub repository via natural language commands](https://www.val.town/v/thatsmeadarsh/star_a_github_repository_with_natural_language),\n powered by Composio\n- [Alex Phan](https://www.val.town/u/alexdphan) and\n [Sarah Chieng](https://www.val.town/u/sarahxc) built\n [Slack Scout](https://www.val.town/v/alexdphan/slackScout), which sends Slack\n notifications whenever your keywords are mentioned on Twitter, Hacker News, or\n Reddit, powered by Browserbase\n- [Max Krass](https://www.val.town/u/maxkrass) built a gateway to access\n Microsoft Graph data for event catalogs, powered by\n [a val](https://www.val.town/v/maxkrass/bondingkataloggateway).\n- [Brent Jackson](https://www.val.town/u/jxnblk) built a\n [service to generate OG images](https://www.val.town/v/jxnblk/blogOGImage) for\n his blog, powered by Val Town.\n- [@weaverwhale](https://www.val.town/u/weaverwhale) created\n [FindTrendsUsingGPT](https://www.val.town/v/weaverwhale/FindTrendsUsingGPT),\n which analyzes merchant transaction data and generates a markdown report with\n charts and tables identifying trends, fraud regions, refund rates, and\n merchant profit/loss analyses.\n- [@movienerd](https://www.val.town/u/movienerd) created a\n [pixel-art idle clicker game](https://www.val.town/v/movienerd/clicker) where\n you can earn money and upgrade various factories and workers, all written in\n React and powered by Val Town.\n- [Dan Thyresson](https://www.val.town/u/dthyresson) built a\n [GraphQL API to query Spotify tracks](https://www.val.town/v/dthyresson/tracksapi)\n he has listened to\n- [Jackson](https://x.com/jxnblk) built\n [a minimal web-based markdown editor with vim keybindings](https://x.com/jxnblk/status/1823725027109290039)\n- [Kaz](https://www.val.town/u/kaz) built a\n [\"Would You Rather\" game](https://www.val.town/v/kaz/reluctantcoffeegayal)\n that ranks answers with an Elo-based system and tracks progress.\n- [Sharan Babu](https://www.val.town/u/sharanbabu) built a\n [Text Summarization Comparison Tool](https://www.val.town/v/sharanbabu/textSummarizationComparisonTool)\n that uses the Cerebras LLM API to generate summaries through direct and\n extractive + cohesive methods\n- [@pinjasaur](https://www.val.town/u/pinjasaur) built a\n [proxy for Kagi's Smallweb RSS feed](https://pinjasaur-smallweb.web.val.run/)\n to limit the size and make it more RSS reader-friendly\n- [Curt](https://www.val.town/u/curtcox) built a\n [QR Code Scanner](https://x.com/curtcox/qrcodescanner) that uses your device's\n camera to decode QR codes and display their content and additional information\n- [Jackson](https://www.val.town/u/jxnblk) built\n [Indirection](https://www.val.town/v/jxnblk/indirection), a game of riddles to\n play with friends by guessing words from clues\n- [jdan](https://www.val.town/u/jdan) created\n [dialog](https://www.val.town/v/jdan/dialog), a val that renders Windows\n 98-style dialog boxes as SVGs\n- [EJ Fox](https://www.val.town/u/mrejfox) built a\n [daily contribution checkerboard](https://x.com/mrejfox/status/1830343140168839336)\n for Val Town\n- [@tempdev](https://www.val.town/u/tempdev) built a\n [RabbitStream](https://www.val.town/v/tempdev/rabbitstream) val that provides\n an easy-to-deploy TypeScript implementation for quickly accessing RabbitStream\n video sources\n- [Sharan Babu](https://www.val.town/u/sharanbabu) built a\n [search engine prototype with autocomplete functionality](https://www.val.town/v/sharanbabu/legitimateTanTiger)\n powered by the Cerebras LLM API and cached in the browser\n- [Mauricio](https://www.val.town/u/mauriciobc) built\n [a cat feeding app](https://www.val.town/v/mauriciobc/miau-food) that allows\n you to log when you fed your cats and which cats were fed\n- [@tempdev](https://www.val.town/u/tempdev) created a\n [Flix](https://www.val.town/v/tempdev/flix), a movie and TV show streaming\n platform with search, watch, and info pages\n- [@gio](https://www.val.town/u/gio) built\n [Razza](https://www.val.town/v/gio/razza), a web app that generates QR codes\n for initiating SMS conversations, categorized by topics like Film, Technology,\n and Sports\n- [@yawnxyz](https://www.val.town/u/yawnxyz) built\n [Diesel](https://www.val.town/v/yawnxyz/diesel), a lightweight data\n manipulation library for querying and transforming JSON, YAML, CSV, and TOML\n data\n- [@gio](https://www.val.town/u/gio) built\n [The Here Times](https://www.val.town/v/gio/theheretimes), a map-based news\n aggregator using the Google Maps API, Geonames, and NewsAPI\n- [@cephalization](https://www.val.town/u/cephalization) built a\n [SMS Journaler Text Relay](https://cephalization-smsjournalertextrelay.web.val.run)\n that lets you have conversational SMS exchanges with an AI assistant to\n organize your daily work activities\n- [@stevekrouse](https://www.val.town/u/stevekrouse) built\n [lucia_middleware](https://www.val.town/v/stevekrouse/lucia_middleware), a\n middleware for handling user authentication and sessions, backed by Val Town\n SQLite\n- [@vawogbemi](https://www.val.town/u/vawogbemi) built a\n [URL shortener](https://vawogbemi-short.web.val.run/)\n- [David Thyresson](https://www.val.town/u/dthyresson) created\n [a Hono demo app with Unkey rate-limiting middleware](https://www.val.town/v/dthyresson/honoWithUnkeyRateLimiting)\n that limits requests to 2 per minute\n- [Matthew Weaver](https://www.val.town/u/mjweaver01) built\n [FindFraudTrendsUsingGPT](https://val.town/v/mjweaver01/FindFraudTrendsUsingGPT),\n which generates real-time fraud trend reports using GPT-4o, complete with data\n tables and Mermaid charts.\n- [Carl Lange](https://www.val.town/u/csl_) created his\n [redfloatplane.lol](https://redfloatplane.lol/), hosted on Val Town\n- [Alex](https://www.val.town/u/alexpetros) built a\n [val that monitors Hacker News and Lobsters for posts from certain sites](https://www.val.town/v/alexpetros/aggregatoralert)\n and sends an email notification with the matched stories\n- [Jackson](https://www.val.town/u/jxnblk) built an\n [example of using his test val to test his tuna val](https://www.val.town/v/jxnblk/tunaTestSuite),\n showcasing tuna's CSS-in-JS features.\n- [Sam Rose](https://www.val.town/u/samwho) built a\n [simple feature flag implementation](https://www.val.town/v/samwho/featureflags)\n that uses hashing and rollout percentages to determine flag behavior for\n users.\n- [Matt](https://www.val.town/u/neverstew) created a val that\n [supercharges query params](https://www.val.town/v/neverstew/superchargedQueryParams)\n to allow for operators like `>`, `<`, and `!=` in addition to `=`.\n- [Kyle](https://www.val.town/u/kylem) built a\n [Dependency License Checker](https://www.val.town/v/kylem/dependencyLicenses)\n that generates a table of licenses for all dependencies in a package.json file\n- [Kian](https://www.val.town/u/kian) built a\n [webcam-based image processing app](https://www.val.town/v/kian/webcameffects)\n that applies fun effects like Pink Zone, Goth Girl, and Weed Mode to your\n webcam feed.\n- [@yawnxyz](https://www.val.town/u/yawnxyz) created a\n [Perlin Noise Fabric Color Generator](https://www.val.town/v/yawnxyz/perlinNoiseFabricColor)\n that renders a hypnotic, organic pattern using Processing.js and the Perlin\n noise function\n- [@roramigator](https://www.val.town/u/roramigator) created a\n [Loot Generator API](https://www.val.town/v/roramigator/lootGenAPI) that\n generates random legendary loot items with adjectives, materials, and item\n types, based on a rarity algorithm.\n- [Steve Krouse](https://www.val.town/u/stevekrouse) built a\n [Virtual Confessional](https://www.val.town/v/stevekrouse/confessionalApp)\n where users can confess sins and receive penance in the form of Catholic\n prayers from ChatGPT\n- [@yawnxyz](https://www.val.town/u/yawnxyz) reorganized\n [Val Town's SQLite adapter for Lucia](https://www.val.town/v/yawnxyz/luciaValtownSqlite),\n making it easier to work with SQLite databases for authentication.\n- [Sam Willis](https://www.val.town/u/samwillis) got\n [WASM Posgres running in Val Town](https://x.com/samwillis/status/1829142581831450859)\n- [Raul Galera](https://www.val.town/u/rgd) made\n [a surf forecast for his hometown with Townie](https://x.com/raulgalerad/status/1827049028862828825)\n- [Midas](https://x.com/TheMoonMidas) built\n [a quote image generator in just a couple requests in Townie](https://x.com/TheMoonMidas/status/1826973638697861283)\n- [Will Krouse](https://x.com/Willthereader) made\n [an infosite for Jumpchain](https://x.com/Willthereader/status/1825565234263978267)\n- [Peter Hartree](https://x.com/peterhartree) made\n [a 5 min Townie demo video of an app that summarizes blogs](https://x.com/peterhartree/status/1825254990169927976)\n- [Austin Crim](https://x.com/crim_codes) built\n [a notes app that works with iPhone shortcuts](https://x.com/crim_codes/status/1818112710149578764)\n- [Marcus Sandstrom](https://x.com/MarcusSandstrom) made\n [mood tracker app](https://x.com/MarcusSandstrom/status/1830544711850823697)\n- [Will Krouse](https://x.com/Willthereader) made\n [a Safari Extension to stream in ChatGPT to define words inline](https://x.com/Willthereader/status/1825969919147597933)\n- [Steve Krouse](https://x.com/stevekrouse) made\n [demo that shows Cerebras 8b generating HTML pages in under a second](https://x.com/stevekrouse/status/1831076208681111782)",[],[191,1116],"a63eeb51-f80e-45ff-9611-b6955065d591","2024-08-30-oss.md","86ba02b0-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Joining the Open Source Pledge\ndescription: 'We donated $9,600 to open-source!'\npubDate: '2024-08-30T04:00:00.000Z'\nauthor: Tom MacWright\nthumbnail: >-\n https://assets.blog.val.town/legacy/oss/opensourcepledge-logo-horiz-color-opengraph.png\ntags: [company]\n---\n\n\n\n\nVal Town couldn't exist without open source software. Like most startups, we build applications on top of open source frameworks, modules, languages, and more. It's crucial to us that these projects thrive and their maintainers have the time, energy, and justification to keep working on them. We contribute patches upstream and are gradually [releasing open source modules](https://github.com/val-town) of our own, but today we're supporting these projects more directly: with funding.\n\nWe're joining the [Open Source Pledge](https://osspledge.com/), and have donated $9,600 so far in 2024. The goal is to donate at least $2,000 per full-time employee per year, which we've already exceeded, but we plan to donate more throughout the year.\n\nSo far we've donated to:\n\n| Project | What it does |Amount |\n| ------ | ----- | -----: |\n| [esm.sh](https://esm.sh/) | CDN for ESM modules |$1000 |\n| [Sindre Sorhus](https://github.com/sindresorhus) | Prolific open source maintainer | $1000 |\n| [tsx](https://tsx.is/) | Easy way to run TypeScript directly | $1000 |\n| [Fastify](https://fastify.dev/) | Web framework | $1000 |\n| [Zod](https://zod.dev/) | Data validation library | $1000 |\n| [Unified](https://unifiedjs.com/) | Markdown transformation module ecosystem | $1000 |\n| [Drizzle](https://orm.drizzle.team/) | Database query builder | $800 |\n| [CodeMirror](https://codemirror.net/) | Code editor framework | $800 |\n| [Mock Service Worker](https://mswjs.io/) | Testing helper modules | $500 |\n| [rclone](https://rclone.org/) | Upload and download files to S3 and more | $500 |\n| [Hono](https://hono.dev/) | Web standards-based web framework | $500 |\n| [husky](https://typicode.github.io/husky/) | Nice way to set up git hooks | $500 |\n\nThanks so much to all the maintainers of these projects! We are so grateful for your work. We hope these donations go a small way towards showing our appreciation and making open-source more sustainable.\n\nDo you rely on open source software? You probably do: ask your company to [join the pledge](https://osspledge.com/join/) too!\n",[],[191,1124],"81ae05e3-af89-4b6f-bdc0-146be0d7fef8","2024-08-23-lockfiles.md","86b9e2c6-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: How we lock your dependencies\ndescription: 'Adding lockfiles to Val Town for stability, security, and performance.'\npubDate: '2024-08-23T04:00:00.000Z'\nauthor: Max McDonnell\nthumbnail: 'https://assets.blog.val.town/legacy/lockfiles/metrics.png'\ntags: [engineering]\n---\n\nOver the past few months, we rolled out dependency lockfiles in Val\nTown. When you save a val, we generate a\n[deno.lock](https://docs.deno.com/runtime/manual/basics/modules/integrity_checking/)\nfile. When your val is run any time in the future, it'll use those exact same versions\nof your dependencies – effectively pinning your unpinned dependencies. When you save a new\nversion of that val, we regenerate the lockfile from scratch, re-pinning your unpinned\ndependencies to whatever their current versions are at that new time.\n\n## Imports in Val Town\n\nVal Town doesn't have a package manager like npm, and it doesn't use package.json files to declare dependencies. Imports\nare listed at the top of the val and are resolved by our runtime, Deno. We support imports from the [NPM](https://www.npmjs.com/) and [JSR](https://jsr.io/) registries using `npm:` and `jsr:` protocols, as well as imports\nfrom any URL using `https://`. You can optionally specify\na version in the import statement itself. For example:\n\n```ts\nimport lodash from \"npm:lodash\"; // unpinned, gets the latest\nimport lodash from \"npm:lodash@4.17.21\"; // pinned to 4.17.21\n\nimport { sqlite } from \"https://www.val.town/v/std/sqlite\"; // unpinned, gets the latest\nimport { sqlite } from \"https://www.val.town/v/std/sqlite?v=6\"; // pinned to version 6\n```\n\n## Everything's better with lockfiles\n\nYou'll see improvements on three fronts from lockfiles: stability,\nsecurity and performance.\n\n**Stability**: Libraries would change over time and vals would break. Now, if you\nbuild something and it's working, it should generally keep working.\n\n**Security**: If there's a\nmalicious dependency, it's easy for the vulnerability to proliferate rapidly\nwithout dependency pinning. Lockfiles will slow things down, and prevent certain\ntypes of attacks from being effective. Additionally we do integrity\nchecking on the contents of dependencies. If the contents of a module have\nchanged unexpectedly, we won't run your code.\n\n**Performance**: Lockfiles cache the whole dependency tree,\nso we make fewer requests, and we can make requests in parallel.\nModule fetches to Val Town\nfell by 80%. Worker initialization latency (P50) dropped 50%.\n\n![Charts of requests to esm.town and worker initialization time decreasing](https://assets.blog.val.town/legacy/lockfiles/metrics.png)\n\n## Getting clever with updates\n\nIf you're a passionate Val Town user, you may notice an issue with this new\nlockfile scheme as described so far. What happens when you are working on a\ncollection of vals that all depend on each other?\n\nMaybe you have your CSS styles in one val that's imported into an HTTP val.\nIf you update your styles, you'd expect the HTTP val to pick up the changes\nautomatically.\n\nNaively, lockfiles would break this pattern. You would update your CSS and you'd\nneed to re-save all vals that import that one to pick up the changes.\n\nWe don't want to make you do that. So when you save a val, we find all the vals\nin your account that depend on that val and mark their lockfiles for regeneration.\nThat way you can keep rapidly iterating and we'll make sure to update dependency\nrelationships in your vals.\n\nTo be clear this pattern only works for you and your\ndependencies. If you and a friend are iterating closely on a collection of vals,\nyou will need to re-save your vals that import your friend's vals to pick up\ntheir newest versions. We think this is a good compromise\nbetween security and convenience for now. Longer term, we plan to create [shared\ncollections of vals](https://github.com/val-town/val-town-product/discussions/139)\nthat teams can fluidly iterate on together.\n\n## Going forward\n\nWe've seen a variety of situations where an https import has a subtle change\nand a val is halted from running because of an integrity error against what should\nbe the same exact version. If you run into this issue and are confident it's\nnot an exploit, you can fix it by re-saving your val, which will regenerate\nthe lockfile. We're working on the best way to handle these issues without\nremoving integrity checks entirely.\n\nThis is just the beginning of surfacing more granular control over dependencies\nin Val Town. If you have a specific feature you'd like to see, or want to edit\nyour dependencies in specific ways please make a request in our [product\ndiscussions](https://github.com/val-town/val-town-product/discussions).\n",[],[191,1131],"5d689c74-3a10-4c57-8b7a-dcb5482bf412","2024-08-22-codegen.md","86b968fa-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: How we built Townie – an app that generates fullstack apps\ndescription: 'Like Claude Artifacts, but with a backend and database'\npubDate: '2024-08-22T04:00:00.000Z'\nauthor: JP Posma\ntags: [engineering]\n---\n\n[Townie](https://www.val.town/townie) has been completely redesigned in the past couple weeks. It's seriously good at writing fullstack apps. This is the post about how I prototyped this new version of Townie a couple weeks ago.\n\n

\n The redesigned Townie, currently in beta at \n val.town/townie.\n

\n\n\n## Codegen\n\nRecent advances in code generation (\"codegen\"), particularly in Claude 3.5 Sonnet, has enabled a completely new style of building software through conversation with an LLM.\n\nSome of the most successful products are:\n\n- [Claude Artifacts](https://www.anthropic.com/news/claude-3-5-sonnet): build interactive websites and other code through conversation. Websites can be viewed directly.\n- [Cursor](https://www.cursor.com/): a fork of VSCode built for AI. It can generate changes across an codebase.\n- [Vercel v0](https://v0.dev/): build website UIs from prompts.\n- [Websim](https://websim.ai/): similar to Claude Artifacts, but built around \"fake websites\". So going to catgifs.com will generate a website with cat gifs on the fly.\n- [VSCode Copilot](https://code.visualstudio.com/docs/copilot/overview): intelligent autocompletions of code within larger projects, for lots of programming languages.\n\n\n

\n $\"Claude$ \n

Claude Artifacts: making a game.

\n $\"Websim:$ \n

Websim: catgifs.com.

\n\n\nThe beauty of products like Vercel v0, Claude Artifacts, and Websim is that you immediately get a working program back even if you don't know anything about programming! For example, in Websim you don't even see the code, you just see the generated website.\n\nSo far these LLM-generated programs have been limited to the frontend, or copy-pasted into other places. Tools like VSCode Copilot and Cursor let you build larger software with a frontend and a backend, but now you need a place to deploy it, which is a significant barrier to non-programmers, and slows the iteration cycle.\n\nWe see many Val Town users generate their code in an LLM and then copy it into Val Town, since our \"vals\" (tiny full-stack webapps) get instantly deployed. We figured we're in a good position to tighten the feedback loop of codegen to full-stack deployment, and to finally approach [end-user programming](/end-programmer-programming):\n\n> The dream is that the full power of computers be **accessible to all**, that our\n> digital worlds will be as **moldable as clay**.\n\nThis vision is taking shape with frontend-only apps, but if we're really talking about \"the full power of computers\", then this should include entire applications, with persistence, authentication, external API calls — really everything that professional programmers build.\n\nI spent most of July 2024 prototyping codegen ideas in Val Town, with great results. In this blog post I'll show you what I built and what I learned. And since I built it all within Val Town itself, it's all open source, and you can fork it!\n\n

\n My prototype codegen with an instantly deployed backend and database.\n

\n\n\n# Generate a full-stack app with backend and database\n\nFirst I built a basic version of code generation. We can do this in only 80 lines (and only a few dependencies). You can fork it to play with it.\n\n\n\nI used [Vercel's AI SDK](https://sdk.vercel.ai/) to easily switch between models. To teach the LLM how to write code on Val Town, I went for a maximalist approach: I downloaded as many public vals as I could fit in a context window and put them in [`valleGetValsContextWindow`](https://www.val.town/v/janpaul123/valleGetValsContextWindow). I also found adding [our docs](https://docs.val.town) to the context window helpful (as raw markdown pages).\n\nI tell it to start outputting a Typescript fence immediately (```ts), so it starts writing code right away. I then strip out the code fence later.\n\nThis is enough to generate a basic Hacker News clone with a backend and database, which is instantly deployed on its own subdomain:\n\n

\n \n \n \n

\n Hacker News clone with backend and database, generated with gpt-4o\n (generation is sped up).\n

\n\n\n# Advanced prototype\n\nOver the course of several weeks, I ended up adding lots of features to this prototype:\n\n- Side panel with code and preview\n- CodeMirror for syntax highlighting\n- Follow-up prompts to iterate\n- Generate a new val for every iteration, so you can easily go back\n- Generate multiple vals (by opening multiple tabs when submitting the form)\n- Loading an existing val\n- Editing code manually\n- Switching between models (Claude, OpenAI) and context window size\n\n

\n\n\nI called this prototype VALL-E, and you can [fork it yourself](https://www.val.town/v/janpaul123/valle). Let's look at the details of what we ran into.\n\n### Database persistence\n\nEach Val Town user has a [SQLite](https://docs.val.town/std/sqlite/) database powered by [Turso](https://turso.tech/). We ran into a problems teaching the LLM to adhere to the idiosyncrasies of Val Town SQLite. So I punted on SQLite and just got the LLM to save everything in Val Town Blob Storage, which is basically S3 lite, so it had no trouble understanding it.\n\nLater Steve was able to fix the issues with SQLite by instructing the LLM to use his [LLM-safe shim](https://www.val.town/v/stevekrouse/sqlite). Instead of trying to fix this through prompting, it works better to write a wrapper around the API that transforms the data into something that LLMs expects. Adapt your code to the LLM, not the LLM to your code.\n\n### Make real\n\nYou might have seen the [\"Make Real\"](https://github.com/tldraw/make-real) demos of [tldraw](https://tldraw.dev/), where you can draw shapes on a canvas, and turn it into HTML. I made a prototype \"Make Real with Backend\". For this, I put tldraw in a [val](https://www.val.town/v/janpaul123/valledraw) (which you can fork), and put my VALL-E prototype in an iframe:\n\n

\"Make Real with Backend\"

\n\n\nIt currently only works with text prompts, but it should be easy to pass the SVGs of arbitrary drawings to the LLM, as with the original Make Draw demos.\n\nI modified the VALL-E prototype to pass the name of generated val up to tldraw, using the [`postMessage` API](https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage). That way you can use a previously generated val as the basis for a new one, such as when I write \"add more sample stories\" in the video above.\n\n### Model choice\n\nClaude Sonnet 3.5 is clearly the best model for writing code right now. However we found that it can be very deterministic, so it can help to crank up its `temperature`.\n\nWe also played with `gpt-4o` vs `gpt-4o-mini`. The `mini` version is much cheaper but much worse, though it's pretty good at making websites, especially when given some examples. However, we don't want to optimize too much for cheap models, because the premier models of today will be the cheap models of tomorrow.\n\n### Putting it all together\n\nOnce I had built all this, I gave a lightning talk at an event hosted by South Park Commons. It's 5 minutes of building lots of little apps.\n\n\n\n# Reducing costs\n\nI started with a maximalist approach of filling the context window with as many example vals and docs as possible. Claude 3.5 Sonnet has a 200k token context window and I put hundreds of vals in there. However, a single query with a full context window costs at least $0.60 (just for the input tokens), which is a lot.\n\nA full context window is also slower, and I hit rate limits much more quickly. I managed to get custom rate limits from Anthropic, but it's not ideal. Let's look at how I optimized our costs and speed.\n\n### Evals\n\nBefore cutting down our context window, I wanted to measure how good our models are, so I could see how much performance would be affected by a smaller context window.\n\nIn the world of LLMs these measurements are called \"evaluations\" or \"evals\" (presumably because people forgot that the word \"benchmark\" already existed). For us the general idea is this: generate a bunch of websites, then make sure they work. For this I built [\"E-VALL-UATOR\"](https://www.val.town/v/janpaul123/eVALLuator):\n\n

\n\nThis runs some basic prompts like \"Create a simple todo list app\" or \"Create a quiz game with a scoring system\", and then checks if it runs without errors, in which case we award 3 points. If there are errors, we'll retry but subtract a point. The maximum score for 10 prompts is thus 30 points.\n\nHow do we evaluate if the generated code runs without errors? Let's look at the different types of mistakes we can capture:\n\n1. Syntax errors.\n2. Typescript errors.\n3. Backend errors on `GET /`.\n4. Frontend errors on `GET /`.\n5. Frontend errors when interacting (clicking on stuff).\n6. Backend errors when interacting (persistence issues, issues on other pages).\n7. Visual issues (site doesn't look good).\n8. Site doesn't do what we expect.\n\nFor this prototype I implemented 1,3,4. I made a [val](https://www.val.town/v/janpaul123/httpValErrorCatcher) (actually VALL-E made it for me) that wraps any arbitrary HTTP val, and injects a `\n\n### Nice new REST API documentation\n\n![Val Town REST API Documentation](https://assets.blog.val.town/legacy/openapi/docs.png)\n\nFirst thing, we get **way better REST API documentation**. It has example\nrequests, with everything from cURL to Python clients. It describes the precise\nshapes of the objects you'll get back. You can even test APIs right in\nthe API docs. Hats off to [Scalar](https://scalar.com), whose\n[open source package](https://github.com/scalar/scalar) is powering this.\n\nThe docs are based on our OpenAPI spec, which is emitted from Fastify and\npulled from our server's routes. From there on, setting up nice documentation\nis basically as simple as this:\n\n```html\n\n\n```\n\n### A TypeScript SDK\n\nMeet our [new TypeScript SDK](https://github.com/val-town/sdk),\n`@valtown/sdk`, which we autogenerate from our OpenAPI spec using\n[Stainless](https://www.stainlessapi.com/):\n\n```ts title=\"example.ts\"\n// Import the SDK\nimport ValTown from \"npm:@valtown/sdk\";\n// Initialize it\nconst valTown = new ValTown();\n// Use it\nconsole.log(await valTown.me.profile.retrieve());\n```\n\nThe SDK covers all of our major API methods, and has robust TypeScript\ntypes for everything. Methods in our API that are paginated, like listing\ncomments, vals, or search results, are implemented as async iterators\nin the SDK so you don't even have to implement any pagination logic.\n\nFor Node.js, you can install it as an NPM package:\n\n```bash\nnpm install @valtown/sdk\n```\n\nThe SDK works in Deno, of course, but also in Node.js, Bun, and other major\nruntimes. We're generating it with tools from our friends at [Stainless](https://www.stainlessapi.com/), and their system keeps it updated\nso that every time we update the REST API, we can quickly roll a new release\nof the SDK.\n\n### Our AI, Townie, can now call our REST API\n\n\n\n[Townie](https://www.val.town/townie) is Val Town's bot - it started as an\nexperiment to see if we could coax an LLM into writing code that fit\nVal Town's runtime better. It has a long way to go before it reaches\nsentience, fortunately/unfortunately, but it can already write some pretty\ndecent code.\n\nBut it couldn't really interact with Val Town until now. Townie just got\nthe ability to call our API directly, which means that it can do things like\ncreating vals, searching for existing code, updating readmes – anything\nthat the API can do, Townie can do too.\nTo be safe, Townie will run `GET` requests against the API freely, but asks\nfor permission before running `POST`, `PUT`, or `DELETE` requests.\n\nTownie knows the whole layout of the API because OpenAPI 3.1 is\n[compatible with JSON Schema][oa31], and\nthe [OpenAI Function calling][calling]\nsystem is based on JSON Schema-defined parameters. So we can smoothly\ntranslate from our OpenAPI specification for our REST API, into a definition\nof functions that OpenAI can call – along with LLM-friendly descriptions\nfor every function, parameter, and return value – and voila, the robot\ncan use the machine.\nAvoiding OpenAPI-OpenAI mixup typos was one of the more\nmentally challenging parts of implementing this.\n\nUnfortunately ChatGPT does hallucinate incorrect parameters\noccasionally, despite having a very precise definition of what is correct.\nIt does this so often that we added a prompt reminding it that it likes\nto make a particular mistake, but it shouldn't make that mistake:\n\n> When calling the valsCreate function, make sure to follow the schema,\n> which requires that 'code' and 'name' properties be nested in a 'body' object.\n\nThis new era of software is funny and awe-inspiring!\n\n[last_post]: https://blog.val.town/blog/fastify/\n[openapi]: https://en.wikipedia.org/wiki/OpenAPI_Specification\n[oa31]: https://www.openapis.org/blog/2021/02/18/openapi-specification-3-1-released\n[calling]: https://platform.openai.com/docs/guides/function-calling",[],[191,1152],"4c59ac36-f347-437c-a456-bf4d1801742f","2024-07-23-val-town-newsletter-19.md","86b8dc3c-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town Newsletter 19\ndescription: >-\n Full-screen Editor, 5x faster HTTP vals, new Dashboard, improved Search,\n Townie redesign, Trending redesign, Profile filters, and more!\npubDate: '2024-07-23T04:00:00.000Z'\nauthor: Steve Krouse\ntags: [newsletter]\n---\n\nIn the last month, we shipped a full-screen val Editor, a new 5x faster HTTP val runtime, a Dashboard for your account, a new Search experience, a redesign for our AI bot Townie and the ability to use tools, a new Trending leaderboard, User Profile filters and sorts, and a bunch of other improvements and bug fixes.\n\nWe've also been on a blogging spree: [Moving from Express to Fastify](/fastify/), [Here's why we're also using TypeBox](/typebox), and [Why spawning a new process in Node is so slow?](/node-spawn-performance).\n\nOn the livestream, I coded with [Mary Rose Cook on concert ticket alerts](https://www.youtube.com/watch?v=RJ2uqaqD10E), [Sahil Lavingia on a stock buyback form](https://www.youtube.com/watch?v=3-TXoB0BmSA), and [Peter van Hardenberg on a Automerge change notifier](https://www.youtube.com/watch?v=CIVgi4mRXxk).\n\n## ↔️ Full-screen editor\n\nAt long last, you can edit your vals in full-screen. This new editor also features a classic left-sidebars with all your vals and folders. Thanks [bcjordan](https://www.val.town/u/bcjordan) and [hunter](https://www.val.town/u/hunter) for the [feature](https://github.com/val-town/val-town-product/discussions/58) [requests](https://github.com/val-town/val-town-product/discussions/137) 🙏\n\nTry it out: [val.town/editor](https://val.town/editor)\n\n $\"Demo$ \n\nOne of my favorite features is the ability to multi-select vals (shift-click) and drag them into folders to clean up your workspace.\n\n## ⚡️ 5x Faster HTTP vals at scale\n\nWe built a new runtime for HTTP vals that is much faster at scale by re-using existing Deno processes. We've seen up to a 5x speedup for some vals. Try it out by selecting `HTTP (Preview)` for your val's type. Learn more in [our announcement post](/http-preview/).\n\n

\n\n## 📊 Dashboard\n\nWe built a Dashboard to help you more quickly navigate your vals and see what's happening in Val Town. Thanks [nbbaier](https://www.val.town/u/nbbaier) for the [feature request](https://github.com/val-town/val-town-product/discussions/8) 🙏\n\nYou can access your dashboard today at [val.town/dashboard](https://val.town/dashboard).\n\n $\"Demo$ \n\n## 🔎 Val Search\n\nSearch results are now powered by exact matching, exact phrase matching, and \"vibes-based\" vector search. We sort results in part by indicators of that val's quality, ie how many times it's been forked or liked. You can filter results by val type and by whether the val was created by you or not.\n\nTry some searches: [anthropic](https://val.town/search?q=anthropic) | [openai](https://val.town/search?q=openai) | [substrate](https://val.town/search?q=substrate) | [embeddings](https://val.town/search?q=embeddings) | [react client side](https://val.town/search?q=react%20client%20side)\n\n $\"Search$ \n\n## 🤖 Townie redesign & tool-use\n\nTownie has a new look! It can also use the entire Val Town API for you. This means you can ask Townie to list, create, update, and delete not only vals but also blobs and sqlite data – it can run anything in our [new API](https://docs.val.town/openapi). Townie automatically performs GET requests and asks you to confirm before running any mutative action.\n\nIn the following demo video, you'll notice a couple times Townie didn't do exactly what I wanted, but we were able to work around it by doing it ourselves or asking it again. Townie is still a work in progress, but it's a lot of fun to play with.\n\n

\n\n## 🦾 VALL-E Demo\n\nLLM code generation is so exciting right now that we weren't able to limit ourselves to just Townie. We built a prototype inspired by Anthropic Projects called [VALL-E](https://val.town/v/janpaul123/VALLE) that's great at writing vals for you. We built another prototype, [VALL-E Draw](https://www.val.town/v/janpaul123/VALLEdraw), which embeds it in TLDraw! Both were built entirely in vals – so you can fork and tweak em!\n\nThe video below demos how to create multiple apps simultaneously with VALL-E Draw: a Hacker News clone with a real backend and a simple hello world example app with craazzzy styles. Here's [two](https://x.com/stevekrouse/status/1813699804729663495) [demos](https://x.com/stevekrouse/status/1812980867381489962) of the original VALL-E.\n\n

\n\n## 🔥 Trending\n\nCollect likes, forks, and references to gain fame and fortune – well at least fame – on the new and improved Val Town trending leaderboard. The new trending has a fresh look and much more useful rankings.\n\nCheck it out: [val.town/trending](https://val.town/trending)\n\n $\"Trending$ \n\n## ❤️ Profile filters and sorts\n\nProfile pages now have filters and sorts to help you find the vals you're looking for. You can filter by val type and sort by date created, date updated, likes, and alphabetically.\n\nTry it out on some power users' profiles: [pomdtr](https://val.town/u/pomdtr) | [iamseeley](https://val.town/u/iamseeley) | [nbbaier](https://www.val.town/u/nbbaier) | [substrate](https://val.town/u/substrate) | [yawnxyx](https://www.val.town/u/yawnxyz)\n\n

\n\n### 🎉 Other updates\n\n- [Cancel a running val](https://github.com/val-town/val-town-product/discussions/117). Thanks Dan Shipper, Matt McClure, and Jan Zheng for requesting this feature 🙏\n- Added a refresh button on the http preview iframe\n- Fixed links to errors in stacktraces\n- Upgraded the README editor to CodeMirror\n- Upgraded OpenAI in the standard library to `gpt-4o-mini` for free users (up from `gpt-3.5`)\n- Support updating val types through the API\n- Cleaned up Val Town SDK\n- Rewrote our [Saving data from a web page](https://docs.val.town/guides/saving-data-from-a-web-page/) guide. Thanks [mharris717](https://www.val.town/u/mharris717) [for the request](https://x.com/GFunk911/status/1808239507444990185)\n- Automatically generate the new Val Town SDK when our API changes\n- Val Town took stewardship of [val-town-playground](https://github.com/val-town/val-town-playground)\n and [val-town-vscode](https://github.com/val-town/val-town-vscode) from the amazing [pomdtr](https://val.town/u/pomdtr)\n\n## 🛣️ Roadmap\n\n### 💪 What we're working on\n\nNext month we focusing on onboarding, polish, performance, reliability, and hiring – [join us!](https://val.town/careers)\n\n- Rolling out `HTTP (Preview)` as the new default `HTTP` val type\n- [Val runtime performance](https://github.com/val-town/val-town-product/discussions/113)\n- [LLM-generated vals](https://github.com/val-town/val-town-product/discussions/47), ie Townie and VALL-E improvements\n\n### 🔜 Coming soon\n\nHere's some of the top features we're planning to work in the coming months. Please [make feature requests and vote](https://github.com/val-town/val-town-product/discussions?discussions) on what you'd like to see next.\n\n- [Receive attachments from email handlers](https://github.com/val-town/val-town-product/discussions/75)\n- [Usage tracking and pricing](https://github.com/val-town/val-town-product/discussions/19)\n- [LLM-generated READMEs](https://github.com/val-town/val-town-product/discussions/26)\n\n## 🌟 Community Vals\n\n- The [Substrate](https://substrate.run) is on 🔥 with a dozen vals that show how much faster they run multi-step LLM flows: [sub.audio](https://sub.audio/), [Illustrated Primer](https://www.val.town/v/substrate/illustratedPrimer), [Hacker News RAG](https://www.val.town/v/substrate/hackerNewsRAG), [Similar Sites](https://www.val.town/v/substrate/similarSites), [Spanish Lesson](https://www.val.town/v/substrate/spanishLesson), [WebSim](https://www.val.town/v/substrate/websim)\n- [Kayla](https://www.val.town/u/kayla_lin) built a [yenny.app](https://www.yenny.app/), a yen conversion app, [powered by Val Town](https://x.com/kayladotdev/status/1808293579468550211)\n- [Nico](https://www.val.town/u/nicosuave) gave [ChatGPT the ability to install python packages](https://x.com/nicoritschel/status/1814716092801294786), powered by a val\n- [Sam Rose](https://www.val.town/v/samwho) built [Ping](https://samwho.dev/ping/) so you can send him anonymous messages, [powered by a val](https://x.com/samwhoo/status/1811304859334750493)\n- [Achille](https://www.val.town/u/pomdtr) found a [way to embed a full stack vite + honojs app in a single JSR import, ie excalidraw](https://x.com/pomdtrr/status/1805928035670994998)!\n- [Jan](https://val.town/u/yawnxyz) built [a Japanese Translator in Google Sheets](https://x.com/yawnxyz/status/1807120870248505434), [a clone of a viral drag effect](https://x.com/yawnxyz/status/1813713602819580376), and a function to [get citations](https://www.val.town/v/yawnxyz/citation)\n- [pvh](https://www.val.town/u/pvh) built a [cron that notifies of changes to Automerge documents](https://www.val.town/v/pvh/watcher)\n- [Mux](https://www.val.town/u/mux) built an [API to dub videos using Sieve](https://www.val.town/v/mux/seiveDubbing)\n- [Lu \"todepond\" Wilson](https://www.val.town/u/todepond) built a [magic iframe](https://x.com/TodePond/status/1814947568389931057) powered by Val Town\n- I built a bunch of fun utilities: [\"the create react app\"](https://www.val.town/v/stevekrouse/reactClientDemo) of Val Town and [a video explaining React, SSR, and Val Town](https://www.youtube.com/watch?v=HyhsMd0CBnk), [website load performance tester](https://www.val.town/v/stevekrouse/perf), [DNS lookup site](https://www.val.town/v/stevekrouse/dns), [lookup your internet twin](https://stevekrouse-twin.web.val.run/) with ExaAI, [played around with Browserbase](https://x.com/stevekrouse/status/1807812485711622617), [explained when esm.town transpiles](https://www.val.town/v/stevekrouse/esmTownTranspileDemo)\n- [Jackson](https://www.val.town/u/jackson) built a [VIM-mode markdown editor](https://www.val.town/v/jxnblk/vimarkdown)\n- [Max](https://www.val.town/u/maxm) built a [Infinite Graph of SVGs](https://maxm-infinitesvggraph.web.val.run/)\n- [Pete Millspaugh](https://www.val.town/u/petermillspaugh) wrote and sent the [June 2024 Edition](https://www.val.town/v/petermillspaugh/june2024) of his wonderful newsletter\n- [dglazkov](https://www.val.town/u/dglazkov) built a [better Discord helper function](https://www.val.town/v/dglazkov/discordBot), a [sample implementation of the Breadboard Servive Endpoint](https://www.val.town/v/dglazkov/googlenews), a [BaaS](https://www.val.town/v/dglazkov/bbaas) (Breadboard As a Service), and a [CORS-safe proxy for qdrant](https://www.val.town/v/dglazkov/qdrant)\n- [mharris717](https://mharris717.github.io/val-tools-doc/) wrote a [guide to use vals in tools for LLM calls](https://mharris717.github.io/val-tools-doc/)\n- Peter Zakin built [a similar websites explorer](https://x.com/pzakin/status/1814404672301814133)\n- [Will](https://www.val.town/u/willthereader) built a [jumpchain.net](https://jumpchain.net/) ([source](https://www.val.town/v/willthereader/Jumpchain_Website)), an info site and resource guide for the Jumpchain community\n- [Sam](https://www.val.town/u/samwillis) ran [Postgres Lite](https://www.val.town/v/samwillis/pglite_http_preview_runtime) in a val, which [runs much faster on our new HTTP runtime](https://x.com/samwillis/status/1814016232133173383)\n- [Triozer](https://www.val.town/u/triozer) built a [API to query if someone is live on YouTube or Twitch](https://www.val.town/v/triozer/livestatus)\n- [Kyle](https://www.val.town/u/kylem) built a [git release notes generator](https://www.val.town/v/kylem/gitReleaseNotes)\n- [Brian](https://www.val.town/u/brianleroux) built a [cap friendly scraper](https://x.com/brianleroux/status/1808228023029977482)\n- [Steve](https://www.val.town/u/saolsen) put the [Ace editor in a val](https://www.val.town/v/saolsen/ace)\n- [Matthias](https://www.val.town/u/matthiasraimann) built a [SAGA scraper and notifier](https://www.val.town/v/matthiasraimann/SagaImmoScraper)\n- [Park](https://www.val.town/u/park) built a [clean simple BTC price site](https://www.val.town/v/park/btc)\n- [bohaska](https://www.val.town/u/bohaska) built a [cron that automatically answers issues from NationStates](https://www.val.town/v/bohaska/nationstatesSentimentAI) using sentiment analysis\n- [vipshek](https://www.val.town/u/vipshek) built a [cron that detects new website contents](https://val.town/v/vipshek/detectNewWebsiteContents)\n- [jonbo](https://www.val.town/u/jonbo) built an [API that generates a fake server-side delay](https://www.val.town/v/jonbo/dynamicSleep)\n- [Seeley](https://www.val.town/u/iamseeley) built a [enterprise-grade FizzBuzz](https://www.val.town/v/iamseeley/fizzFuss)\n- [edgeeffect](https://www.val.town/u/edgeeffect) built a [smart garbage day notifier](https://www.val.town/v/edgeeffect/whatDoYouThinkOfItSoFar)\n- [Will](https://www.val.town/u/willthereader) built a [work time calculator](https://www.val.town/v/willthereader/Work_Time_Calculator_2)",[],[191,1159],"1653ad82-7498-4366-83c5-9f29968ea70d","2024-07-19-node-spawn-performance.md","86b87e9a-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Why is spawning a new process in Node so slow?\ndescription: At Val Town we spawn a lot of processes. We're working on making it faster\npubDate: '2024-07-19T04:00:00.000Z'\nauthor: Max McDonnell\ntags: [engineering]\n---\n\nAt Val Town we run your code in Deno processes. We recently noticed, that under\nload, a single Val Town's Node server cannot exceed 40 spawns/s. It spends 30%\nof its time with the main thread blocked on calls to `spawn`. Why is it so slow?\nCan we make it any faster?\n\nTo simulate this pattern we'll write an HTTP server that spawns a new process\nfor each request. Like this:\n\n```js\nimport { spawn } from \"node:child_process\";\nimport http from \"node:http\";\nhttp\n .createServer((req, res) => spawn(\"echo\", [\"hi\"]).stdout.pipe(res))\n .listen(8001);\n```\n\nWe'll write similar implementations in Go\n([here](https://github.com/maxmcd/process-per-request/blob/fb2f5f9518d62f058f7e587580c302b56f7a5781/go/main.go))\nand Rust\n([here](https://github.com/maxmcd/process-per-request/blob/0a6442f656fe7bc8f6c61ef2c5fdef65c6afa0f1/rust/src/main.rs))\nand run this example with Node, Deno and Bun.\n\nI am running all of these on a Hetzner CCX33 with 8 vCPUs and 32 GB of ram. I am\nbenchmarking with [bombardier](https://github.com/codesenberg/bombardier)\nrunning on the same machine. The command I'll run to benchmark each server is\n`bombardier -c 30 -n 10000 http://localhost:8001`. 10,000 total requests over 30\nconnections. I prewarm each server before running the benchmark. I'm using Go\nv1.22.2, Rust v1.77.2, Node v22.3.0, Bun 1.1.20, and Deno 1.44.2.\n\nHere are the results:\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | ---------------------------------- |\n| Node | 651 | `node baseline.js` |\n| Deno | 2,290 | `deno run --allow-all baseline.js` |\n| Bun | 2,208 | `bun run baseline.js` |\n| Go | 5,227 | `go run go/main.go` |\n| Rust (tokio) | 5,466 | `cd rust && cargo run --release` |\n\nOk, so Node is slow. Deno and Bun have figured out how to make this faster, and\nthe compiled, thread-pool languages are much faster again.\n\nNode's `spawn` performance does seem to be notably bad. [This\nthread](https://github.com/nodejs/node/issues/14917) was an interesting read,\nand while in my testing things have improved since the time of that post, Node\nstill spends an awful lot of time blocking the main thread for each Spawn call.\n\nSwitching to Bun or Deno would improve this a lot. That is great to know, but\nlet's try and improve things with Node.\n\n## Node `cluster` Module\n\nThe simplest thing we can do spawn more processes and run an http server\nper-process using Node's `cluster` module. Like so:\n\n```js\nimport { spawn } from \"node:child_process\";\nimport http from \"node:http\";\nimport cluster from \"node:cluster\";\nimport { availableParallelism } from \"node:os\";\n\nif (cluster.isPrimary) {\n for (let i = 0; i < availableParallelism(); i++) cluster.fork();\n} else {\n http\n .createServer((req, res) => spawn(\"echo\", [\"hi\"]).stdout.pipe(res))\n .listen(8001);\n}\n```\n\nNode shares the network socket between processes here, so all of our processes\ncan listen on `:8001` and they'll be routed requests round-robin.\n\nThe main issue with this approach for me is that each HTTP server is isolated in\nits own process. This can complicate things if you manage any kind of in-memory\ncaching or global state that needs to be shared between these processes. I'd\nideally find a way to keep the single thread execution model of javascript and\nstill make spawns fast.\n\nHere are the results:\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | --------------------------------------------- |\n| Node | 1,766 | `node cluster.js` |\n| Deno | 2,133 | `deno run --allow-all cluster.js` |\n| Bun | n/a | \"node\\:cluster is not yet implemented in Bun\" |\n\nSuper weird. Deno is slower, Bun doesn't work just yet, and Node has improved\na lot, but I would have expected it to be even faster.\n\nNice to know there is some speedup here. We'll move on from it for now.\n\n## Move The Spawn Calls To Worker Threads\n\nIf the `spawn` calls are blocking the main thread, let's move them to worker\nthreads.\n\nHere's our `worker-threads/worker.js` code. We listen for messages with a\ncommand and an id. We run it and post the result back. We're using `execFile`\nhere for convenience, but it is just an abstraction on top of `spawn`.\n\n```js\nimport { execFile } from \"node:child_process\";\nimport { parentPort } from \"node:worker_threads\";\n\nparentPort.on(\"message\", (message) => {\n const [id, cmd, ...args] = message;\n\n execFile(cmd, args, (_error, stdout, _stderr) => {\n parentPort.postMessage([id, stdout]);\n });\n});\n```\n\nAnd here's our `worker-threads/index.js`. We create 8 worker threads. When we\nwant to handle a request we send a message to a thread to make the spawn call\nand send back the output. Once we get the response back, we respond to the http\nrequest.\n\n```js\nimport assert from \"node:assert\";\nimport http from \"node:http\";\nimport { EventEmitter } from \"node:events\";\nimport { Worker } from \"node:worker_threads\";\n\nconst newWorker = () => {\n const worker = new Worker(\"./worker-threads/worker.js\");\n const ee = new EventEmitter();\n // Emit messages from the worker to the EventEmitter by id.\n worker.on(\"message\", ([id, msg]) => ee.emit(id, msg));\n return { worker, ee };\n};\n\n// Spawn 8 worker threads.\nconst workers = Array.from({ length: 8 }, newWorker);\nconst randomWorker = () => workers[Math.floor(Math.random() * workers.length)];\n\nconst spawnInWorker = async () => {\n const worker = randomWorker();\n const id = Math.random();\n // Send and wait for our response.\n worker.worker.postMessage([id, \"echo\", \"hi\"]);\n return new Promise((resolve) => {\n worker.ee.once(id, (msg) => {\n resolve(msg);\n });\n });\n};\n\nhttp\n .createServer(async (_, res) => {\n let resp = await spawnInWorker();\n assert.equal(resp, \"hi\\n\"); // no cheating!\n res.end(resp);\n })\n .listen(8001);\n```\n\nResults!\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | ---------------------------------------------- |\n| Node | 426 | `node worker-threads/index.js` |\n| Deno | 3,601 | `deno run --allow-all worker-threads/index.js` |\n| Bun | 2,898 | `bun run worker-threads/index.js` |\n\nNode is slower! Ok, so presumably we are not bypassing Node's bottleneck by\nusing threads. So we're doing the same work with the added overhead of\ncoordinating with the worker threads. Bummer.\n\nDeno loves this, and Bun likes it a little more. Generally, it's nice to see\nthat Bun and Deno don't see much of an improvement here. They're already doing a\ngood job of keeping the sycall overhead off of the execution thread.\n\nOnward.\n\n## Move Spawn Calls to Child Processes\n\nIf threads are not going to work, let's use child processes to do the work.\nWe're spawning processes to spawn processes, but we'll spawn a small number of\nworker processes from the main thread and distribute work between them. This way\nwe only pay the spawn cost on startup in the main thread.\n\nThis is quite easy. We simply swap out the worker threads for processes spawned\nby `child_process.fork` and change how we send and receive messages.\n\n```diff\n$ git diff --unified=1 --no-index ./worker-threads/ ./child-process/\ndiff --git a/./worker-threads/index.js b/./child-process/index.js\nindex 52a93fe..0ed206e 100644\n--- a/./worker-threads/index.js\n+++ b/./child-process/index.js\n@@ -3,6 +3,6 @@ import http from \"node:http\";\n import { EventEmitter } from \"node:events\";\n-import { Worker } from \"node:worker_threads\";\n+import { fork } from \"node:child_process\";\n\n const newWorker = () => {\n- const worker = new Worker(\"./worker-threads/worker.js\");\n+ const worker = fork(\"./child-process/worker.js\");\n const ee = new EventEmitter();\n@@ -21,3 +21,3 @@ const spawnInWorker = async () => {\n // Send and wait for our response.\n- worker.worker.postMessage([id, \"echo\", \"hi\"]);\n+ worker.worker.send([id, \"echo\", \"hi\"]);\n return new Promise((resolve) => {\ndiff --git a/./worker-threads/worker.js b/./child-process/worker.js\nindex 5f025ca..9b3fcf5 100644\n--- a/./worker-threads/worker.js\n+++ b/./child-process/worker.js\n@@ -1,5 +1,4 @@\n import { execFile } from \"node:child_process\";\n-import { parentPort } from \"node:worker_threads\";\n\n-parentPort.on(\"message\", (message) => {\n+process.on(\"message\", (message) => {\n const [id, cmd, ...args] = message;\n@@ -7,3 +6,3 @@ parentPort.on(\"message\", (message) => {\n execFile(cmd, args, (_error, stdout, _stderr) => {\n- parentPort.postMessage([id, stdout]);\n+ process.send([id, stdout]);\n });\n```\n\nNice. And the results:\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | --------------------------------------------- |\n| Node | 2,209 | `node child-process/index.js` |\n| Deno | 3,800 | `deno run --allow-all child-process/index.js` |\n| Bun | 3,871 | `bun run worker-threads/index.js` |\n\nGood speedups all around. I am very curious what the bottleneck is that is\npreventing Deno and Bun from getting to Rust/Go speeds. Please let me know if\nyou have suggestions for how to dig into that!\n\nOne fun thing here is that we can mix Node and Bun. Bun implements the Node IPC\nprotocol, so we can configure Node to spawn Bun child processes. Let's try that.\n\nUpdate the `fork` arguments to use the `bun` binary instead of Node.\n\n```js\nconst worker = fork(\"./child-process/worker.js\", {\n execPath: \"/home/maxm/.bun/bin/bun\",\n});\n```\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | ----------------------------- |\n| Node + Bun | 3,853 | `node child-process/index.js` |\n\nHah, cool. I get to use Node on the main thread and leverage Bun's performance.\n\n## Stdio\n\nLogs. The previous implementations assume there will be minimal log output, but\nwhat if there's a lot? We could send the logs using `process.send`, but that\nwill be quite expensive if our output bytes are serialized to JSON.\n\nI spent a lot of time in this rabbit hole. Here's a rough summary of the things\nI tried:\n\n1. Passing file descriptors between processes. Like passing the stdout/err back\n up to the parent process. I tried this a few different ways but couldn't get\n it working so that we'd always capture all the bytes written.\n2. Just using `process.send`. This works, but is only performant if you use\n `serialization: \"advanced\"` so that you can send bytes without serialization.\n This doesn't work in Deno and Bun.\n3. I created a pair of [Abstract\n Sockets](https://man7.org/linux/man-pages/man7/unix.7.html#:~:text=be%20inspected.%0A%0A%20%20%20%20%20%20%20abstract-,an%20abstract%20socket,-address%20is%20distinguished) for each spawn\n call and sent the logs over the socket. This spends too much time setting up\n the sockets to be worth it.\n\nAlso Abstract Sockets are crazy. I'm familiar with [Unix Domain\nSockets](https://en.wikipedia.org/wiki/Unix_domain_socket) where you have a file\ncalled (eg) `something.sock` and you can listen on it and connect to it just\nlike a network address. Turns out, that if you use a Unix socket and the\nfilename starts with a null byte, like `\\0foo` the socket will not exist on the\nfilesystem and it'll be automatically removed when no longer used. Weird! Cool!\n\nAfter all this testing I have two approaches that work pretty well.\n\n1. Set up a pool of processes with `.fork()` and also set up a separate abstract\n socket for each one to send logs.\n2. Simply use `process.send` but use `serialization: \"advanced\"`.\n\nLet's see how those work out.\n\nWe'll need something that outputs a lot of logs. So I grabbed the `main.c` file\nfrom Sqlite's source. This is a 163Kb file. We'll run the command `cat main.c`\nto print it out.\n\nHere's our `baseline.js` again with that update:\n\n```ts\nimport { spawn } from \"node:child_process\";\nimport http from \"node:http\";\nhttp\n .createServer((_, res) => spawn(\"cat\", [\"main.c\"]).stdout.pipe(res))\n .listen(8001);\n```\n\nI've updated the Go and Rust code as well. Let's see how they do:\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | ---------------------------------- |\n| Node | 374 | `node baseline.js` |\n| Deno | 667 | `deno run --allow-all baseline.js` |\n| Bun | 1,374 | `bun run baseline.js` |\n| Go | 2,757 | `go run go/main.go` |\n| Rust (tokio) | 3,535 | `cd rust && cargo run --release` |\n\nFascinating. It's cool to see Bun and Rust pull ahead here compared to the\nprevious benchmarks. Node is still slow very slow and Deno is surprisingly\nunhappy with this workload.\n\nNext let's try my abstract socket communication channel implementation. It's\ngetting quite complex so I won't post it here, but you can [take a look\nhere](https://github.com/maxmcd/process-per-request/tree/7528cd8045c998c8b5451961e0818473b4a81860/child-process-comm-channel).\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | ---------------------------------------------------------- |\n| Node | 1,336 | `node child-process-comm-channel/index.js` |\n| Node + Bun | 2,635 | `node child-process-comm-channel/index.js` |\n| Deno | 862 | `deno run --allow-all child-process-comm-channel/index.js` |\n| Bun | 1,833 | `bun child-process-comm-channel/index.js` |\n\nHaha. I had seen some random benchmark results where Node+Bun was faster than\nbun alone, but it never netted out in the final runs.\n\nThe Deno results are quite perplexing. In implementing this example I had a\n\"bug\" where I was buffering the response as a string. Here's the diff of me fixing it:\n\n```diff\n@@ -88,9 +88,8 @@ const spawnInWorker = async (res) => {\n worker.child.send([id, \"spawn\", [\"cat\", [\"main.c\"]]]);\n- let resp = \"\";\n worker.ee.on(id, (msg, data) => {\n if (msg == MessageType.STDOUT) {\n- resp += data.toString();\n+ res.write(data);\n }\n if (msg == MessageType.STDOUT_CLOSE) {\n- res.end(resp);\n+ res.end();\n worker.requests -= 1;\n```\n\nThis 'fix' makes Deno a lot slower, but Node and Bun a lot faster! I wonder if\nthat's because one has a faster `toString()` implementation or higher overhead for\n`res.write`?\n\n| Language/Runtime | Req/s | Command |\n| -------------------- | ----- | ---------------------------------------------------------- |\n| Deno + string buffer | 1,453 | `deno run --allow-all child-process-comm-channel/index.js` |\n\nWeird!\n\nFinally, here is the `process.send` implementation. It is fast and also\nincredibly simple to implement. I am a little unexcited about this solution\nbecause it is slower than I'd like, doesn't support Deno and Bun, and there's\nvery little space to improve things. However, this implementation is deeply\npractical and easy to understand, which is beautiful. Here's the source of\n`worker.js`, the rest [is here](https://github.com/maxmcd/process-per-request/tree/7528cd8045c998c8b5451961e0818473b4a81860/child-process-send-logs).\n\n```ts\nimport { spawn } from \"node:child_process\";\nimport process from \"node:process\";\n\nprocess.on(\"message\", (message) => {\n const [id, cmd, ...args] = message;\n const cp = spawn(cmd, args);\n cp.stdout.on(\"data\", (data) => process.send([id, \"stdout\", data]));\n cp.stderr.on(\"data\", (data) => process.send([id, \"stderr\", data]));\n cp.on(\"close\", (code, signal) => process.send([id, \"exit\", code, signal]));\n});\n```\n\n| Language/Runtime | Req/s | Command |\n| ---------------- | ----- | --------------------------------------- |\n| Node | 1,179 | `node child-process-send-logs/index.js` |\n\nVery nice, probably the practical choice if you are only targeting Node.\n\n## Load Balancing\n\nA quick note on load balancing between processes. Both Go and Rust [have\ncomplicated schedulers](https://rakyll.org/scheduler/) that [distribute work\nefficiently](https://tokio.rs/blog/2019-10-scheduler). So far, when picking a\nworker I've been grabbing a random one:\n\n```ts\nconst workers = await Promise.all(Array.from({ length: 8 }, newWorker));\nconst randomWorker = () => workers[Math.floor(Math.random() * workers.length)];\n```\n\nHowever, we can also implement round-robin, and least-connections style load\nbalancing. [See a wonderful writeup on those\nhere](https://samwho.dev/load-balancing/).\n\n```ts\nconst pickWorkerInOrder = () => workers[(count += 1) % workers.length];\nconst pickWorkerWithLeastRequests = () =>\n workers.reduce((selectedWorker, worker) =>\n worker.requests < selectedWorker.requests ? worker : selectedWorker\n );\n```\n\nSadly I didn't see consistent performance improvements with these approaches.\nThey all perform about the same. Maybe more typical workloads where the spawn\ncalls are not entirely uniform would benefit more from these changes.\n\n## Library?\n\nIt seems possible, given all of these findings, to implement a `child_process`\nlibrary that implements the same API surface as `node:child_process` but farms\nthe spawn calls out to a process pool. Maybe I will write that, or maybe you\nwill. Please [let me know](https://x.com/mxmcd) if there's interest.\n\n## Final Thoughts\n\nWe're sadly at the limits of my knowledge/experimentation, but I wonder what\ncould unlock more performance.\n\nIt was really fun to see improved performance and what didn't, and the random\nmoments where Deno/Bun/Node were affected differently.\n\nUsing Node and Bun together is a fun pattern and it's nice to see it lead to\nsuch a speedup. Please support Node's IPC, Deno!\n\nLet me know if there's anything else I should experiment with here! See you next\ntime :)\n",[],[191,1166],"7f5bec1d-f67a-454e-aa81-6935faac830d","2024-07-17-http-preview.md","86b85c12-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: New HTTP Val Runtime in Preview\ndescription: We built a new runtime for HTTP vals that is up to 5x faster at scale\npubDate: '2024-07-17T04:00:00.000Z'\nauthor: Max McDonnell\ntags: [engineering, product]\n---\n\nWe built a new runtime for HTTP vals that is much faster at scale. We've seen up to a 5x speedup for some vals. This is a preview feature, so expect rough edges. We'd love your feedback on it.\n\n## Usage\n\nWe created a new val type, `HTTP (Preview)`, to let you opt-into the new runtime. Here's how:\n\n1. Create a normal `HTTP` val\n2. Click on the val's type `HTTP` and select `HTTP (Preview)` in the dropdown menu.\n\nThat's it! Run your val 5 times in quick succession to see the difference.\n\n{/* Here's a simple script you can run to test it out yourself: TODO */}\n\n{/* Or if you'd prefer, here's a utility website to do the same thing: TODO */}\n\n## Architecture\n\nUp until now, every time you run a val it starts a new Deno process and installs all your dependencies from scratch. This takes 100 milliseconds or more, depending on how many dependencies you have.\n\nWith the new `HTTP (Preview)` val, we keep your Deno process around and forward multiple requests to it. This means that the first request will still have the 100ms+ cold start time, but every request after that will be _much_ faster. For example, the simplest possible val that just returns \"hello world!\" will take 100ms to boot up the first time, but then only 10ms for subsequent requests. To be clear, that's not including network latency to the val (which from NYC is 90ms), just the time it takes for the val to respond.\n\nA user from NYC hitting a \"hello world!\" val can expect to see a 180ms response time for the first request and 90ms for every request after that using `HTTP (Preview)`. Whereas with the normal `HTTP` val type, they would see 190ms for every request. The difference is even more pronounced for vals with more dependencies and large dependencies like WASM modules. We migrated the Date Me Directory and saw total request time go from 800ms to 200ms!\n\nWe want to be conservative with this rollout, so we're only keeping `HTTP (Preview)` vals alive for 10 seconds after the end of the last request they receive. As we get more comfortable keeping vals alive for longer, we'll increase this timeout window, particularly for Pro users. We're considering ways to keep vals alive indefinitely, but that will require some changes to our pricing to make it sustainable.\n\n## Limitations\n\nThere are currently two limitations to `HTTP (Preview)` compared to the normal `HTTP` type:\n\n1. `console.log` and other logging output aren't captured\n2. You can’t cancel a val that is currently running\n\nWe expect to have fixes for these limitations in the next couple weeks. In the meantime, we recommend using the normal HTTP val type in development and then switching to HTTP (Preview) for production.\n\n## Breaking changes\n\nThere is one breaking change with this new feature: it only re-runs your val handler, not any of the other code in your val. This means that if you have any code outside of your handler that you want to run on every request, you'll need to move it into your handler.\n\nFor example, this val will return a new random number for every request in our existing system, but after you switch to the next version, it'll return the same number - because the call to `Math.random()` is only run when the val is started up, not when a new request is handled.\n\n```ts\nconst number = Math.random();\n\nexport default function(req: Request) {\n return Response.json(number);\n}\n```\n\nThe upgrade path is to move anything you want to happen on every request into the request handler:\n\n```ts\nexport default function(req: Request) {\n const number = Math.random();\n return Response.json(number);\n}\n```\n\nNow each call to this val will return a new random number. This is how most serverless platforms work, and we think it's a good model to follow. You'll only need to migrate your code if you upgrade to the new runtime.\n\n## Rollout\n\nThis new runtime has the potential to be much faster and more scalable than our current approach, so we want it to be the default soon. But it's a big change, so we're being careful. Today we're rolling it out as an opt-in val type, `HTTP (Preview)`.\n\nAfter we've ironed out the kinks and achieved feature parity between the two runtimes, we'll make this the default type for new HTTP vals,\ndrop the (Preview), and encourage everyone to upgrade to the new system for that big performance boost! Depending on how it works out, we might be also be able to auto-upgrade some vals to the new system.\n\n## Feedback\n\nWe'd love to hear your feedback on this new feature. You can send a message in our `#general` or `#bugs` channels in [our Discord](https://discord.gg/dHv45uN5RY) or email us at [feedback@val.town](mailto:feedback@val.town).\n",[],[191,1173],"8480ee95-3450-4cc7-be01-22eb9e93c897","2024-07-15-typebox.md","86b81a72-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Zod is amazing. Here’s why we're also using TypeBox\ndescription: Why we’re using TypeBox instead of Zod for our server validation layer\npubDate: '2024-07-15T04:00:00.000Z'\nauthor: Tom MacWright\nthumbnail: 'https://assets.blog.val.town/legacy/typebox/card.png'\ntags: [engineering]\n---\n\nI've written a bit about how much I like [Zod](https://zod.dev/). I've also [contributed a bunch of improvements to Zod](https://github.com/colinhacks/zod/pulls?q=is%3Apr+author%3Atmcw+is%3Aclosed), especially around performance.\n\nZod has revolutionized how developers think about data validation because, I'd say, it has a wonderful API. Creating a shape for some data in Zod is really developer-friendly:\n\n```ts\nimport { z } from \"zod\";\n\nconst User = z.object({\n username: z.string(),\n});\n\nUser.parse({ username: \"Ludwig\" });\n\n// extract the inferred type\ntype User = z.infer;\n// { username: string }\n```\n\nHowever, for [Val Town’s recent transition from express to Fastify](https://blog.val.town/blog/fastify/), we've been using [TypeBox](https://github.com/sinclairzx81/typebox) in place of Zod for validation. Fastify can support either Zod or TypeBox [via a Type Provider system](https://fastify.dev/docs/latest/Reference/Type-Providers/), but TypeBox was the clear option for us.\n\nWhy? Mostly because TypeBox plays well with JSON Schema, and we need JSON Schema\ndefinitions for our [OpenAPI](https://en.wikipedia.org/wiki/OpenAPI_Specification)\nspecification. Plus, it's pretty fast.\n\n### TypeBox ↔ JSON Schema ↔ OpenAPI\n\nAs we've been migrating to [Fastify](https://fastify.io/) for our API server, one of our major goals has been to strictly define all of the inputs and outputs, and to be able to express them all in an [OpenAPI specification](https://en.wikipedia.org/wiki/OpenAPI_Specification) that will have a lot of trickle-down benefits.\n\nOpenAPI 3.1.0 [is 100% compatible with JSON Schema](https://www.openapis.org/blog/2021/02/18/openapi-specification-3-1-released) – the types of query parameters, request bodies, response bodies, and more are all defined with [JSON Schema](https://json-schema.org/) objects.\n\nHere's an example of one of our simplest routes - the [esm.town](https://docs.val.town/api/esm-town/) endpoint that exposes Val contents as TypeScript or JavaScript modules so that they can be run in Deno or browsers:\n\n```ts\nfastify.route({\n method: \"GET\",\n url: \"/v/:handle/:name\",\n schema: {\n description: \"Get source code for a val\",\n // TypeBox provides this Type variable\n // which we use to build the schema\n querystring: Type.Object({\n v: Type.Optional(Type.Integer({ minimum: 0 })),\n }),\n params: Type.Object({\n handle: Type.String(),\n name: Type.String(),\n }),\n },\n // …\n});\n```\n\nLet's look at the `querystring` type (and [here's a Val that shows doing just this](https://www.val.town/v/tmcw/typeboxExample)):\n\n```ts\nconst querystring = Type.Object({\n v: Type.Optional(Type.Integer({ minimum: 0 })),\n});\n\nconsole.log(JSON.stringify(querystring));\n// {\"type\":\"object\",\"properties\":{\"v\":{\"minimum\":0,\"type\":\"integer\"}}}\n```\n\nTypeBox objects are JSON Schema objects with extra in-memory magic!\n\nA fun way to think about TypeBox is that it does for JSON Schema what [drizzle does for SQL](https://orm.drizzle.team/) - it's a nice, TypeScript-friendly builder syntax for an existing spec that tries to expose all of the spec's power. That's not _all_ TypeBox is -\nit has an extremely fast built-in [type checker](https://github.com/sinclairzx81/typebox?tab=readme-ov-file#typecheck),\nit can [create example values based on schemas](https://github.com/sinclairzx81/typebox?tab=readme-ov-file#values), and a lot more.\n\nThere _are_ ways to use Zod to produce JSON Schema. For example, there's [zod-to-json-schema](https://github.com/StefanTerdell/zod-to-json-schema):\n\n```ts\nimport { z } from \"zod\";\nimport { zodToJsonSchema } from \"zod-to-json-schema\";\n\nconst mySchema = z\n .object({\n myString: z.string().min(5),\n myUnion: z.union([z.number(), z.boolean()]),\n })\n .describe(\"My neat object schema\");\n\nconst jsonSchema = zodToJsonSchema(mySchema, \"mySchema\");\n```\n\nThat said, TypeBox’s supported validators mirror what you can do in JSON Schema precisely, whereas Zod's often stretch into things that JSON Schema can't do, for two reasons:\n\n- Zod is most often used without reference to JSON Schema, so there's no sense in it trying to be a _subset_ of JSON Schema.\n- Zod follows the _parse, don't validate philosophy_.\n\n### Parse, don't validate\n\nAlexis King [coined this snappy slogan and popularized it in this great blog post, \"Parse, don't validate\"](https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/). Here's a vastly oversimplified example of how these two approaches might look in TypeScript:\n\n```ts\n// A validation-based approach\nif (isValid(object)) {\n // This is valid\n console.log(object);\n}\n\n// A parsing-based approach\nconst { error, value } = parse(object);\nif (!error) {\n // This is valid\n console.log(value);\n}\n```\n\nValidating is the default approach of a module like [ajv](https://ajv.js.org/guide/typescript.html#utility-types-for-schemas) and the behavior of how TypeBox’s [Check](https://github.com/sinclairzx81/typebox?tab=readme-ov-file#check) method works: they do some form of [type narrowing](https://www.typescriptlang.org/docs/handbook/2/narrowing.html) to let the type system know that the given data is valid, but they do not copy or mutate the data.\n\nZod, on the other hand, parses, and it creates a [deep clone](https://zod.dev/?id=parse) of its input data:\n\n```ts\nconst stringSchema = z.string();\n\nstringSchema.safeParse(\"billie\"); // => { success: true; data: 'billie' }\n```\n\nThis, in most cases, is _awesome_. The flexibility of Zod lets you do a lot of useful work in the 'parsing' step. You can [transform inputs](https://zod.dev/?id=transform) and [strip extra keys on objects](https://zod.dev/?id=strip). Zod can guarantee that that your data is 'clean' - it has the expected keys, but doesn't have any extra keys. Most of the time, this is a thing about Zod that I greatly appreciate.\n\nHowever, for validating the types of our Fastify schema, this approach has two downsides:\n\n- All of the methods in Zod that rely on the \"parse, don't validate\" architecture won't be convertible to JSON Schema. JSON Schema doesn't let you transform data, or pick object properties: it's a system for validation.\n- Parsing and deep-copying inputs is one of the main reasons why Zod's performance [lags other type validators](https://moltar.github.io/typescript-runtime-type-benchmarks/). It is harder to optimize its approach than it is to optimize an approach that doesn't require copying data.\n\n### Performance\n\nNow that I've mentioned performance, I should reiterate something I've written before a bunch of times: performance wasn't the main reason for using TypeBox instead of Zod for this problem. Performance also wasn't the main reason for [switching to Fastify from express](https://blog.val.town/blog/fastify/).\n\nAnd this applies even though fastify is [benchmarked faster](https://fastify.dev/benchmarks/) than express and TypeBox is benchmarked faster than Zod.\n\nI am a strong believer in Carlos Bueno’s [mature optimization handbook](https://carlos.bueno.org/optimization/), which dictates that:\n\n> The trickiest part of speeding up a program is not doing it, but deciding whether it’s worth doing at all.\n\nAnd, later:\n\n> Before you can optimize anything you need a way to measure what you are optimizing. Otherwise you are just shooting in the dark. Before you can measure you need a clear, explicit statement of the problem you are trying to solve. Otherwise, in a very real sense, you don’t know what you are doing.\n\nSo, I keep in mind that everything in the application that is not the bottleneck, is not the bottleneck. I know in my mind that there's a performance difference between different kinds of loops in JavaScript, but in almost every case that doesn't matter.\nThe overhead of a `.forEach` call versus a `for` loop is going to be\ninsignificant versus a single database call or network request.\n\nAnd in the vast majority of applications, validation and parsing is\nnot the bottleneck. That's the case in Val Town. Our bottlenecks are things like network topology, cold-start times for processes, database access, and disk access. We have metrics in [Honeycomb](https://www.honeycomb.io/) that break down the time cost of running vals, and Fastify overhead isn't a major contributor.\n\nTake it from [the author of Arktype](https://www.reddit.com/r/typescript/comments/17cmt0q/comment/k5rcmvi/), one of the extremely fast Zod alternatives:\n\n> I'll tell you a secret- validator performance is mostly negligible in ~95% of situations and Zod's is totally reasonable.\n\nNow, I do write this as someone who spent a bunch of time writing those [performance-oriented Zod pull requests](https://github.com/colinhacks/zod/pulls?q=is%3Apr+author%3Atmcw+is%3Aclosed), which contributed to a pretty decent [performance improvement on some benchmarks](https://observablehq.com/@tmcw/zod-perf-by-versions). Why?\n\nBecause at the time, I was working on [Placemark](https://play.placemark.io/), a geospatial data-editing tool, and that tool was using Zod to validate geospatial data. Geospatial data is large, often in the hundreds of megabytes to gigabytes and more. And it contains a lot of nested JSON structures, like huge arrays of longitude, latitude positions that describe the precise curves of a line.\n\nSo on that project I genuinely used the [Chrome Profiling tools](https://developer.chrome.com/docs/devtools/performance) and got a result in which Zod was a big part of the total time spent, so it became a high priority to optimize it.\n\n### See also\n\nThere is a [lot of competition](https://npmtrends.com/@badrap/valita-vs-@sinclair/typebox-vs-arktype-vs-valibot-vs-zod) happening in the TypeScript validation space right now, which is super exciting for me, free-riding on all this innovation. So forgive me for not comprehensively covering all of the projects.\n\n[Valibot](https://github.com/fabian-hiller/valibot) is cool because it be a much smaller addition to your JavaScript bundle than Zod.\n\n[Arktype](https://github.com/arktypeio/arktype) is extremely cool because it essentially does for the TypeScript type system what TypeBox does for JSON Schema: the Arktype idea is that you don't need to learn a new type syntax - you can just write TypeScript-style type syntax and it'll get compiled to type checkers that can run at runtime. TypeBox has something similar in its [codegen module](https://github.com/sinclairzx81/typebox-codegen).\n\nNot to mention, [Colin](https://x.com/colinhacks) just got funding to build [the next version of Zod, v4](https://zod.dev/blog/clerk-fellowship), which will undoubtedly be a major step up for performance and power.\nThe future is bright!\n",[],[191,1180],"e7f206ea-17a7-4e00-9fd9-4f4d948d0d22","2024-07-02-fastify.md","86b7c158-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: 'Moving from express to fastify, pt 1'\ndescription: How switching to Fastify let us embrace runtime and compile-time types\npubDate: '2024-07-02T04:00:00.000Z'\nauthor: Tom MacWright\ntags: [engineering]\n---\n\nThe JavaScript ecosystem has changed a lot in the last 15 years.\nWe've gone from CommonJS to ESM, from browserify to webpack to vite,\nfrom mocha to vitest, from the request module to using web-standard\nfetch everywhere. The constant reinvention can\nbe exciting or annoying from one day to the next.\n\n### The success of Express\n\nAmid all this churn, [express][express] has persevered.\nExpress is a web framework for Node.js, introduced\nby [TJ Holowaychuk][tj] in 2009. Like many early JavaScript\nmodules and some of TJ's projects, it took inspiration\nfrom the land of Ruby - in particular, [Sinatra][sinatra], which\nwas established in 2007.\n\n```js title=\"express-example.js\"\n// This is a tiny example of using\n// express. Notice how there's almost no boilerplate\n// involved in creating a tiny web server.\nconst app = require('express')();\napp.get('/', (req, res) => {\n res.send('Hello World!');\n});\napp.listen(3000);\n```\n\nExpress is a beautifully light abstraction: a\n'hello world' example is barely over 10 lines. Compared to the boilerplate\nrequired to do the same in Ruby on Rails or a Java server at the time, it\nwas a breath of fresh air.\n\n $\"Weekly$ \n\nExpress became the default for new JavaScript projects and has stayed\nthere. In a typical week, express is downloaded from NPM 30 million\ntimes. It's a roaring success in every way.\n\n### A need for types\n\nExpress is an _expressive_ framework: like a classic Ruby or\nJavaScript framework, it lets you color outside the lines using\nloosely defined objects and types. Query string parameters in express\nbecome [freeform objects](https://expressjs.com/en/4x/api.html#req.query).\nThe same goes for the bodies of requests, and the kinds of responses\nyou can send: it's all a bit ad-hoc by default.\n\nBut the vibes have shifted: we write [TypeScript][ts] now,\nand everyone's excited about validation modules like [Zod](https://zod.dev/)\nand [TypeBox](https://github.com/sinclairzx81/typebox). Interfaces\nare less ad-hoc and more declared, tested, and strictly typed.\n\nModern tooling is written to take full advantage of types\nand schemas. With [tRPC](https://trpc.io/), we're writing backend\nendpoints and calling them from auto-generated, auto-typed functions\non the frontend. Other backends are embracing [OpenAPI][openapi]\n(née Swagger) to generate clients, test suites, and more.\nEnd-to-end type-safety is in vogue, and it is good.\n\n### The promise of OpenAPI\n\nIn our case, we want our types as an [OpenAPI](https://en.wikipedia.org/wiki/OpenAPI_Specification)\nspecification.\nOpenAPI is a JSON/YAML document that lets you describe your entire\nREST API strictly: what URLs are supported, how parameters, headers,\nand request bodies should be formatted, what kind of responses\nwill be produced. It lets you enrich this strict machine-readable\ninformation with human-readable descriptions and helpful examples.\nIt's a pretty exciting time for the OpenAPI spec.\n\n```mermaid\nflowchart TD\n FR[Fastify routes] --> FS[@fastify/swagger]\n FS --> OA[openapi.json]\n OA --> C[OpenAPI consumers]\n OA --> SSDK[TypeScript SDK]\n OA --> DOCS[API Docs]\n OA --> OAPI[OpenAPI Function calling]\n OA --> V[Linter]\n V -->|lint feedback| FR\n```\n\nA number of companies and open source projects, including\n[Stainless](https://www.stainlessapi.com/), [Speakeasy](https://www.speakeasyapi.dev/),\nand [OpenAPI TypeScript](https://openapi-ts.pages.dev/), are\nwriting tools that can generate functional, clean SDKs straight\nfrom OpenAPI specifications.\n\nYou can also generate great documentation from OpenAPI specifications.\nWe're already [generating api.val.town/documentation][openapi_docs]\nusing a library from [Scalar](https://scalar.com/).\n\nAnd in the near future, we should be able to use our OpenAPI specification\nto give our [AI assistant](https://x.com/tmcw/status/1803132938269175956)\nthe ability to interact with our API via [function calling](https://platform.openai.com/docs/guides/function-calling):\nconveniently, OpenAPI and the OpenAI function calling\nsystem are both based on [JSON Schema](https://json-schema.org/),\nso translating between the two is possible.\n\n### Issues with express\n\nVal Town started off with Express. It has served us pretty well.\nBut the papercuts started to accumulate.\n\nWhen we started creating [a public API with an OpenAPI spec][rfc],\nwe immediately started to get a bad feeling about using express\nas we were doing before. The OpenAPI specification was making\nstrong promises about the kinds of requests we accepted and responses\nwe'd return. But the specification was just a dead document,\na YAML file - it wasn't part our codebase, or our test suite. So it\nwasn't enforced or validated, and sure enough, it was slightly incorrect\nfrom day one.\n\nThe other pain point was async. We write the Val Town backend\nas TypeScript but try to transpile it to modern code, so\nwe keep our [async functions][async_mdn] and ESM syntax when we\nrun our code. Express far predates the existence of the\nasync & await keywords in JavaScript, and it [does not mix well with them][express_async].\n\n### Finding an alternative\n\nBecause express is incredibly popular, there are ways to implement\nOpenAPI with it, and support async handlers - plugins like\n[express-openapi](https://github.com/kogosoftwarellc/open-api)\nand [express-async-errors](https://github.com/davidbanham/express-async-errors). But these are a bit bolted-on:\nthey make it technically possible to achieve the goal, but\nnothing about them is idiomatic.\n\nAnother option would be waiting for the next version of express, v5,\nwhich includes support for async functions. That release is\ntaking [a little while to complete][express_5], so we didn't want\nto bet on it landing soon.\n\nIt is great, though, that express\nis still actively maintained, and I don't envy the task of rolling\na major update to such a heavily-adopted project. There are certainly\ncodebases with tens of thousands of lines of code that rely on\nexpress and can't easily jump to an alternative.\n\nBut Val Town's backend is not an enormous legacy project.\nSwitching web frameworks wasn't going to require\na [deadly from-scratch rewrite][never_rewrite]. We were ready to\nbid adieu to express.\n\nTechnology choice is complicated. I don't want to make a decision\nbased on some simplistic criteria like GitHub Stars, but it's equally\nfraught to rely on vibes and instinct. So I investigated whether\nprojects had good documentation and maintainership, if they had\na decent history of stability, and perhaps most importantly, whether\nthey were trying to solve the problems that we needed to solve.\n\n### Fastify\n\n![Fastify](https://assets.blog.val.town/legacy/fastify/fastify-logo.png)\n\nWe landed on [fastify][fastify] as the technology to build the\nfuture of Val Town's public API. Some of the strongest points for\nfastify were:\n\n- A comprehensive [ecosystem](https://fastify.dev/ecosystem/),\n which includes a lot of high-quality, first-party plugins. We're\n building a system in production, which means things like rate\n limiting are going to be necessary, and it's amazing that\n fastify's [rate limiter](https://github.com/fastify/fastify-rate-limit)\n is well-architected.\n- Great, thorough [documentation](https://fastify.dev/docs/latest/Reference/Routes/),\n and an active community that has been really helpful for support.\n- Solid support for [plugins and encapsulation](https://fastify.dev/docs/latest/Reference/Encapsulation/) that let us structure the application into sub-applications.\n- Support for [validation and serialization](https://fastify.dev/docs/latest/Reference/Validation-and-Serialization/) with multiple validators and validation for all inputs and ouputs.\n- [Good support for async/await](https://fastify.dev/docs/latest/Reference/Routes/#async-await), like you might expect from a modern framework.\n- Off-the-shelf integrations for [opentelemetry](https://www.npmjs.com/package/@opentelemetry/instrumentation-fastify) and [Sentry](https://docs.sentry.io/platforms/javascript/guides/fastify/) mean\n that we could have tracing and telemetry from day one.\n- We were already using the [light-my-request](https://github.com/fastify/light-my-request) module to test\n our express server. light-my-request is written and maintained\n by the fastify folks, and it is built into fastify.\n- Finally, [@fastify/express](https://github.com/fastify/fastify-express)\n let us incrementally migrate from express to fastify.\n\nOf course, fastify is also very fast, but web framework overhead\nwasn't a problem with express and most likely won't be a problem with\nany framework we choose: most of our performance envelope has to\ndo with operating [the Val Town runtime][runtimes] and interacting\nwith the database.\n\nWe considered a few other options, like [Hono](https://hono.dev/) and [Elysia](https://elysiajs.com/).\nBoth are lovely projects - we use and recommend Hono a ton\nin Vals. Elysia has convenient typesafety and a lot of features,\nbut its support for Node.js is very experimental. And Hono has\na [first-party OpenAPI plugin](https://github.com/honojs/middleware/tree/main/packages/zod-openapi), but some of the ecosystem around fastify - for tracing,\nincremental migration from express, and plugins - was more built-out.\n\nFor simpler projects and especially for situations where we want\nmulti-platform compatibility so that code can run on Node.js, Deno,\nor Cloudflare Workers, [we reach for Hono](https://www.val.town/examples/packages/hono).\n\n### Incrementally porting our application, route by route\n\nBy using [@fastify/express](https://github.com/fastify/fastify-express)\nwe were able to do this migration entirely incrementally: the first\nfastify PR just added the fastify dependency and wrapped our express\nserver with fastify. Then each successive PR moved a few routes\nfrom express to fastify, until the application has been\ncompletely ported.\n\n $\"Open$ \n\nAs of today, we've ported all but two routes in our backend server.\nIt's been an incredibly incremental effort, thanks to @fastify/express -\nsuch that we never had to stop work on feature development or freeze\npart of the codebase while switching frameworks.\n\nAnd the best is yet to come – next we'll be talking about our\n[new TypeScript SDK][sdk], which is generated directly from the OpenAPI\ndocuments that fastify produces, as well as [improved documentation](https://api.val.town/documentation)\nand more robust API endpoints. Stay tuned!\n\n[openapi]: https://en.wikipedia.org/wiki/OpenAPI_Specification\n[async_mdn]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function\n[express_async]: https://zellwk.com/blog/async-await-express/\n[tj]: https://github.com/tj\n[express]: https://expressjs.com/\n[sinatra]: https://sinatrarb.com/\n[ts]: http://typescript.com/\n[rfc]: https://blog.val.town/blog/expanding-the-vals-api-rfc/\n[express_5]: https://github.com/expressjs/discussions/issues/233\n[never_rewrite]: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/\n[fastify]: https://fastify.dev/\n[runtimes]: https://blog.val.town/blog/first-four-val-town-runtimes/\n[sdk]: https://github.com/val-town/sdk\n[openapi_docs]: https://api.val.town/documentation\n",[],[191,1187],"dfe81352-2c7f-4666-b662-3016729aa9fc","2024-06-25-val-town-newsletter-18.md","86b71d84-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town Newsletter 18\ndescription: 'HTTP Streaming, Semantic Search, Townie AI Chat, and a TypeScript SDK'\npubDate: '2024-06-25T04:00:00.000Z'\nauthor: Steve Krouse\ntags: [newsletter]\n---\n\nIn the last month, we shipped HTTP streaming, semantic search, and Townie, an AI chatbot that knows how to write vals. We also released a TypeScript SDK on top of our new OpenAPI spec for the Val Town API, redesigned profile pages, public likes, and nameable API tokens.\n\nWe're currently working on making it easier to navigate and organize vals, improvements to search, the ability to stop a running val, and performance improvements.\n\nYou all have been making some amazing vals! We've seeing AI apps, streaming demos, personal websites, emojis, authentication middleware, and a hundred others listed at the bottom of this newsletter.\n\nIf you enjoy reading these newsletters, join our team at Val Town to help write them! We're hiring someone to do [content-focused devrel](https://val-town.notion.site/DevRel-at-Val-Town-117abe97069449e58df53d7a794ebfac?pvs=4) to inspire and support the next 100k programmers to build on Val Town.\n\n### 🌊 HTTP Streaming\n\nYou can stream from vals!!! Our new open-source [deno-http-worker](https://github.com/val-town/deno-http-worker) also enables us to increase request size limits from 2mb to 100mb and make response sizes unlimited. Read more in the [HTTP Streaming announcement](/http-streaming/).\n\n

\n\n## 🔎 Val Semantic Search\n\nAre there any vals related to...? We have a new search mode that uses embeddings to give you a val with the same \"vibes\" as your search query. It's currently its own mode labelled \"semantic search\" – [try it out yourself](https://www.val.town/search?searchType=semantic)! We're working on combining it with the exact search to give you a wholistic search experience. Learn more in the [Semantic Search announcement](/val-vibes/).\n\n $\"Semantic$ \n\n## 🤖 Townie\n\nWe built a Townie, a chatbot that knows common Val Town patterns. It's surprisingly good at generating working code. You can see below me asking it for help on a fairly complex task [from Pat LaVarre](https://x.com/pelavarre/status/1804191746760544676). It doesn't give me what I ask for – it gives me something better – and it immediately ran, zero-shot (on its first try). It works even better if you give it feedback in a couple iterations. [Try it out!](https://www.val.town/townie)\n\n $\"Townie$ \n\n## ✨ Upgraded OpenAPI & TypeScript SDK\n\nAfter a herculean effort to port our Express server to Fastify, we were able to generate a good [OpenAPI schema](https://api.val.town/openapi.json), [interactive API docs](https://api.val.town/documentation), and a [TypeScript SDK](https://github.com/val-town/sdk). [Try it out yourself!](https://www.val.town/v/stevekrouse/demoSDK) Shout-out to our friends at [Stainless](https://www.stainlessapi.com/) for help generating the SDK.\n\n $\"Demo$ \n\n## ❤️ Profile redesign, public likes\n\nYour profile page is now prettier and a bit more functional. You can view your vals in a list or a grid, as well as the vals you liked. We're reimagining users' profile as a jumping off point to explore. Coming soon: sorting and filtering for vals in your profile.\n\n $\"Screenshot$ \n\n## 𝍌 Custom formatter settings\n\nYou can use tabs instead of spaces and banish trailing semicolons. Customize dprint to your heart's content in your [editor settings](https://www.val.town/settings/editor). Shout out to [David Sherret](https://x.com/DavidSherret) for creating the wonderful [dprint](https://dprint.dev/). Thanks Adrian Sieber for the original feature request and Nikita for [the final kick](https://x.com/nikitavoloboev/status/1801038544623882590) to get it done.\n\n

\n\n## 📺 Livestreams\n\nLivestreams are a great way to learn Val Town by watching others. And if you watch live, you can ask questions and we'll answer! Or heckle – that's fun too.\n\nSince the last newsletter, I spoke with:\n\n- [Simon Willison](https://www.youtube.com/watch?v=9pmC3P1fUFo) - prototyped a [search-based RAG](https://simonwillison.net/2024/Jun/21/search-based-rag/) (Retrieval Augmented Generation) with the new Claude 3.5 Sonnet\n- [Linus Lee](https://www.youtube.com/watch?v=5nYPeK9_X6o) - made a [websim clone](https://www.val.town/v/thesephist/webgen) to hallucinate websites\n- [Geoffrey Litt](https://www.youtube.com/watch?v=X9h5wxUuuTI) - created [a twitter email digest](https://x.com/geoffreylitt/status/1800952306998845660) where you control the algorithm\n- [Max Bittker](https://www.youtube.com/watch?v=6QGjdDGEkyI) - built a backend to power a community showcase\n\nTune into the next episode [this Thursday, June 27th at 10am ET with Mary Rose Cook](https://www.youtube.com/watch?v=RJ2uqaqD10E)!\n\nOur friend Beniamin also is making wonderful Val Town livestreams! I loved this one where he [used GPT4 summarization to combine many newsletters into one](https://www.youtube.com/watch?v=u9tU5p3uSKQ).\n\n### 🎉 Other updates\n\n- Fixed an extremely annoying bug where the cursor would reset to the beginning of the first line after formatting. Now it stays where you left it.\n- Added the ability to create a val via the API with a specific type (except cron)\n- Improved log streaming & a button to refresh logs\n- Ability to name Val Town API tokens\n- Added three \"getting started\" examples for new users\n- Reorganized val logs for better readability\n- The URL box in the Val Preview iframe is now fairlyully selectable\n- Long val READMEs get truncated with a “view more” button\n- [val.town/newest](https://www.val.town/newest) now has list-view options\n- There's now a \"copy code\" button in markdown js, ts, jsx, tsx snippets\n- Moved the search icon to a better spot in mobile header\n- Improved observability and usage tracking\n- Upgraded database to high-available cluster\n- Made VIM logo background transparent\n- Refactored the code that runs vals, fixing inconsistencies\n\n## 🛣️ Roadmap\n\n### 💪 What we're working on\n\n- Side panel view for vals and folders\n- [Full-screen val editor](https://github.com/val-town/val-town-product/discussions/14)\n- Filters & sorts in user profiles\n- Val detail page improvements\n- [Usage tracking](https://github.com/val-town/val-town-product/discussions/19)\n- [Stop a running val](https://github.com/val-town/val-town-product/discussions/117)\n- [Val runtime performance](https://github.com/val-town/val-town-product/discussions/113)\n- [Search improvements](https://github.com/val-town/val-town-product/discussions/69) (combining exact & semantic)\n\n### 🔜 Coming soon\n\nHere's some of the top features we're planning to work in the coming months. Please [make feature requests and vote](https://github.com/val-town/val-town-product/discussions?discussions) on what you'd like to see next.\n\n- [Receive attachments from email handlers](https://github.com/val-town/val-town-product/discussions/75)\n- [Dashboard](https://github.com/val-town/val-town-product/discussions/8)\n- Usage-based pricing plan\n- [LLM-generated vals](https://github.com/val-town/val-town-product/discussions/47)\n\n## 🌟 Community Vals\n\n- [Achille](https://val.town/u/pomdtr) stunned us all again with an authentication middleware val so powerful, it just may be the [\"last login\" you'll ever need](https://www.val.town/v/pomdtr/lastlogin). It also comes with a [hono adapter](https://www.val.town/v/pomdtr/lastloginHono).\n- [Max](https://val.town/u/maxm) made a series of demos to show us all how to stream: [hello world for streaming HTTP responses](https://www.val.town/v/maxm/streamingResponseExample) that sends a new message every second, [simple server-sent events demo](https://www.val.town/v/maxm/SSEDemo), a [clone of robpike.io](https://www.val.town/v/maxm/robPikeIO), [streaming OpenAI threads](https://www.val.town/v/maxm/openAIStreaming), a [mini ChatGPT clone](https://www.val.town/v/maxm/valTownChatGPT), a [GIF that streams frames](https://www.val.town/v/maxm/streamingGif), [multiplayer circles](https://www.val.town/v/maxm/multiplayerCircles), [an endless maze](https://www.val.town/v/maxm/endlessMazeStream), [NYC's traffic cameras as streaming ASCII images](https://www.val.town/v/maxm/asciiNycCameras)\n- [Thomas](https://www.val.town/u/iamseeley) made and remade his personal website 8 times (!!!) to show different frontend patterns on Val Town and their [relative performance](https://www.val.town/v/iamseeley/webPerformance): [Client-side React](https://www.val.town/v/iamseeley/reactServer), [Client-side React via CDN](https://www.val.town/v/iamseeley/clientMultirouteReact), [Svelte](https://www.val.town/v/iamseeley/multirouteSvelte), [Preact](https://www.val.town/v/iamseeley/multiroutePreact), [Vue](https://www.val.town/v/iamseeley/multirouteVue), [Hono](https://www.val.town/v/iamseeley/multirouteHono), [React](https://www.val.town/v/iamseeley/multirouteReact), and [HTML as String](https://www.val.town/v/iamseeley/multirouteHTML)\n- [Steve](https://val.town/u/stevekrouse) rebuilt the [Date Me Directory](https://dateme.directory) (currently in Hono HTML) as [client-side React (with React Router)](https://www.val.town/v/stevekrouse/dateme_react_client), as well as a version that also has [server-side rendering](https://www.val.town/v/stevekrouse/dateme_react_hydrated). We'll figure out the frontend patterns if we have to build and rebuild all our vals a million times to do it!\n- [Alvi](https://val.town/u/alvi) is [collecting his Steam playtime history](https://www.val.town/v/alvi/SteamPlaytimeHistory_playHistoryToSqlite) to a SQLite database, so he can analyze his gaming habits\n- [Jordan](https://www.val.town/u/jdan) made a val to [explode and explore emoji](https://www.val.town/v/jdan/emojiExploder) and [piped all emoji into a Notion database](https://x.com/jdan/status/1802386599235518473) [with backlinks](https://x.com/stevekrouse/status/1800876339928687047)\n- [@charlypoly](https://val.town/u/charlypoly) made a bunch of vals with Browserbase: [val that watches a blog for changes](https://www.val.town/v/charlypoly/valBlogWatcherCron), a [val that checks if a website has changed](https://www.val.town/v/charlypoly/websiteHasChanged), a [Puppeteer example](https://www.val.town/v/charlypoly/browserbasePuppeteerExample), and [browserbase utils](https://www.val.town/v/charlypoly/browserbaseUtils)\n- [Steve](https://val.town/u/stevekrouse) made an [uptime status page](https://www.val.town/v/stevekrouse/status) built on top of data collected by an [uptime monitor cron](https://www.val.town/v/healeycodes/isMyWebsiteDown)\n- [Thomas](https://val.town/u/iamseeley) made [a series of vals around JSON Resume](https://iamseeley-resumehandler.web.val.run/)\n- [Thomas](https://val.town/u/iamseeley) is [sending SMS](https://www.val.town/v/iamseeley/sendSMS) via email!\n- [Victor](https://val.town/u/vawogbemi) made a [Date Me Directory clone](https://www.val.town/v/vawogbemi/dateme) but for a man in finance (6'5, blue eyes)\n- [@midnightlightning](https://val.town/u/midnightlightning) made [an endpoint to calculate additional metadata for a Uniswap liquidity position](https://www.val.town/v/midnightlightning/uniswapV3Position)\n- [Jon](https://val.town/u/jonbo) made [a bot that ensures a Supabase websocket listener is alive](https://www.val.town/v/jonbo/MADBOTS)\n- [Achille](https://val.town/u/pomdtr) somehow figured out how to [run Astro in Val Town](https://www.val.town/v/pomdtr/astro)! Here are the [Val Town Docs hosted in Val Town](https://pomdtr-docs.web.val.run/) 🤯\n- [@arrudaricardo](https://val.town/u/arrudaricardo) [changes his GitHub status](https://www.val.town/v/arrudaricardo/random_change_github_status) to a random emoji every day\n- [Jan](https://val.town/u/yawnxyz) made an awesome [new blob admin](https://www.val.town/v/yawnxyz/blobbyFace)\n- [@jcoleman](https://val.town/u/jcoleman) [saves email signups](https://www.val.town/v/jcoleman/saveEmailSignupsDeerSpringBakery) for Deer Spring Bakery\n- [Achille](https://val.town/u/pomdtr) brought a [taste of Observable Framework to Val Town](https://www.val.town/v/pomdtr/freeformServer)\n- [Max](https://val.town/u/maxm) cobbled together [some Deno fresh charts](https://www.val.town/v/maxm/freshChartsExample) as HTML or images\n- [Domingo](https://val.town/u/domingo1987) created a [Python problem exchange backend](https://www.val.town/v/domingo1987/problemPython) for a custom GPT\n- [Jordan](https://www.val.town/u/jordan) made some [vals relating to the National Weather Service](https://www.val.town/u/nws), including [a Bluesky feed generator](https://www.val.town/v/nws/feeds)\n- [@kingish](https://val.town/u/kingishb) gets a [weekly email digest of good times to go on a bike ride](https://www.val.town/v/kingishb/blackLobster)\n- [Achille](https://val.town/u/pomdtr) created [middleware that adds handy debug links](https://www.val.town/v/pomdtr/devtools) to get to a val's code and log\n- [Steve](https://val.town/u/stevekrouse) made [a little clone](https://www.val.town/v/stevekrouse/favFarm) of Wes Bos's wonderful [fav.farm](https://fav.farm/) for when you need a quick little favicon out of an emoji\n- [Achille](https://val.town/u/pomdtr) found a clever workaround to [set temporary environment variables](https://www.val.town/v/pomdtr/setEnv)\n- [Jan](https://val.town/u/yawnxyz) built an [vector search](https://www.val.town/v/yawnxyz/buildclubProjectSearch) on top of Lunr and Airtable\n- [@camajudson](https://val.town/u/camajudson) built a [Discord bot for the Cama community](https://www.val.town/v/camajudson/handleDiscordInteraction)\n- [Steve](https://val.town/u/stevekrouse) made a [simple OpenAI streaming demo](https://www.val.town/v/stevekrouse/openaiStreamingDemo) and [one that streams back _rendered_ HTML](https://www.val.town/v/stevekrouse/openaistreaminghtml). Why make HTML when ChatGPT will write it for you?\n- [Jan](https://val.town/u/yawnxyz) made a [blurb generator](https://www.val.town/v/yawnxyz/blurbs) that inputs a link and outputs a blurb and a bunch of tags\n- [@tfayyaz](https://val.town/u/tfayyaz) demoed [search messages](https://www.val.town/v/tfayyaz/searchmessages) with HTMX\n- [Chet](https://val.town/u/chet) made an [HTML formatter](https://www.val.town/v/chet/HtmlFormatter)\n- [Steve](https://val.town/u/stevekrouse) demoed how to get [geolocation data in a website](https://www.val.town/v/stevekrouse/geolocation_api_demo)\n- [@nmsilva](https://val.town/u/nmsilva) made a [Supabase cache](https://www.val.town/v/nmsilva/supabaseCache)\n- [@nerdymomocat](https://val.town/u/nerdymomocat) syncs to [Habitify from Todoist with AI](https://www.val.town/v/nerdymomocat/add_to_habitify_from_todoist_w_ai)\n- [Max](https://val.town/u/maxm) showed us how to [import a wasm binary manually](https://www.val.town/v/maxm/imageMagickWasmExample)\n- [@l3gacyb3ta](https://val.town/u/l3gacyb3ta) wrote [a script to calculate the heat at a point](https://www.val.town/v/l3gacyb3ta/heat_at_point) on a map\n- [Daniel](https://val.town/u/danielkerrigan) fetched [the most popular Obsidian plugins](https://www.val.town/v/danielkerrigan/obsidianPlugins) from Github\n- [Thomas](https://val.town/u/iamseeley) made a [text to image playground](https://iamseeley-texttoimageplayground.web.val.run/) for Fal.ai\n- [@banebot](https://val.town/u/banebot) made a [greeting card generator](https://www.val.town/v/banebot/GreetingCard)\n- [Jan](https://val.town/u/yawnxyz) made a [val that gets content from a URL](https://www.val.town/v/yawnxyz/getContentFromUrl) for summerizers. It combines https://r.jina.ai/URL and markdown.download's Youtube transcription getter to do its best to retrieve content from URLs.\n- [Achille](https://val.town/u/pomdtr) got [WANIX](https://www.val.town/v/pomdtr/wanix) (a web-native unix) running in a val\n- [Thomas](https://val.town/u/iamseeley) made a [val that backs up his vals to Github](https://www.val.town/v/iamseeley/valBackup)\n- [Nico](https://val.town/u/nicoalbanese) demoed [Vercel's AI SDK Streaming](https://www.val.town/v/nicoalbanese/tomatoMinnow)\n- [Alex](https://val.town/u/rattrayalex) made a site to paste & clean [Slack messages](https://www.val.town/v/rattrayalex/slack_cleaner)\n- [Jason](https://val.town/u/jasonhibbs) made a [price tracker](https://www.val.town/v/jasonhibbs/priceTracker) for the Emma Luxe Cooling Mattress\n- [Sean](https://val.town/u/seanehalpin) is [tracking their figma plugin statistics](https://www.val.town/v/seanehalpin/placemarkGlobeMonitor)\n- [Achille](https://val.town/u/pomdtr) made a [val that serves a Github repo](https://www.val.town/v/pomdtr/serveGithubRepo)\n- [@hunty](https://val.town/u/hunty) made a [Spotify endpoints for play state](https://www.val.town/v/hunty/spotify)\n- [@bdon](https://val.town/u/bdon) made a [USD to TWD converter](https://www.val.town/v/bdon/usd_twd_converter)\n- [Matthias](https://val.town/u/matthiasraimann) made a [web component example](https://www.val.town/v/matthiasraimann/WebComponentExample)\n- [Steve](https://val.town/u/stevekrouse) made [a site to lookup DNS records](https://www.val.town/v/stevekrouse/dns)\n- [Jan](https://val.town/u/yawnxyz) made the beginnings of a [UI for a Simon Willison's RAG search](https://www.val.town/v/yawnxyz/ragwithclaude_ui)\n- [Steve](https://val.town/u/stevekrouse) made a [streaming demo with Anthropic Claude](https://www.val.town/v/stevekrouse/anthropicStreamDemo)\n- [@ashryan](https://val.town/u/ashryanio) made a [QR code generator](https://www.val.town/v/ashryanio/generateQrCode)\n- [@zerovox](https://val.town/u/zerovox) made [an RSS feed from his items from Readwise reader](https://www.val.town/v/zerovox/readwiseReaderReadItemsRss) to be ingested to Workflowy\n- [Jan](https://val.town/u/yawnxyz) demoed [DuckDB](https://www.val.town/v/yawnxyz/duckdbExample)\n- [Thomas](https://val.town/u/iamseeley) made an [HuggingFace API Gateway](https://www.val.town/v/iamseeley/hfApiGateway)\n- [@ashryan](https://val.town/u/ashryanio) made a [random secret generator](https://www.val.town/v/ashryanio/generateRandomSecret)\n- [@gwoods22](https://val.town/u/gwoods22) made a [cron to get updates on the Macdonald Block Reconstruction Project](https://www.val.town/v/gwoods22/macdonald_block_updates)\n- [@saolsen](https://val.town/u/saolsen) made a [Codemirror TypeScript Emacs demo](https://www.val.town/v/saolsen/codemirrorTsBrowserEmacs) on top ofs [Max's Codemirror TypeScript demo](https://www.val.town/v/maxm/codemirrorTsBrowser)\n- [Brian](https://val.town/u/brianleroux) is [scraping CapFriendly](https://www.val.town/v/brianleroux/scrapecapfriendly)\n- [Jan](https://val.town/u/yawnxyz) is archiving [Sidebar.io](https://www.val.town/v/yawnxyz/SidebarToArchive) to Google Sheets\n- [Thomas](https://val.town/u/iamseeley) made a [3D emoji generator](https://www.val.town/v/iamseeley/threedEmoji)\n- [@triptych](https://val.town/u/triptych) made a [hit counter](https://www.val.town/v/triptych/hitCounter) you can attach to any website via fetch\n",[],[191,1194],"c950fb45-b2a5-4736-99e9-5cdd262b0f17","2024-06-18-val-vibes.md","86b6d23e-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: 'Val Vibes: Semantic search in Val Town'\ndescription: >-\n How to build semantic search with embeddings for Val Town within Val Town\n itself\npubDate: '2024-06-18T04:00:00.000Z'\nauthor: JP Posma\ntags: [engineering]\n---\n\nIf you've been following along, you know that [code search is hard](/search-notes/). Matching a search query to a particular val, allowing for misspellings, adding ranking signals to prioritize the best content, is a complicated and error-prone quest.\n\nBut what if it wasn't? Instead of parsing code to extract terms and creating inverted indexes for trigrams, what if we used _vibes_ to search?\n\n### Similar vibes\n\n“Search” is about finding documents based on a query, using some sort of sorting mechanism that ranks the most relevant documents. You might rank a document higher if it contains the search term more often, or prioritize documents with an exact match over those that only roughly match the search term.\n\nWe'll look at a completely different sorting mechanism called “semantic search”, which uses one of those crazy features to come out of the world of machine learning: embeddings. It works like this: you put in a string into an API, and you get an array of numbers, called an “embedding”, with the property that things that have the same “vibe” will get similar numbers.\n\nFor example, “animal that barks” should have similar vibes to “dog”. And “dog bites man” should have different vibes than “man bites dog”. And “discord bot” should have _very_ different vibes still.\n\n

\n\nSo we need to generate an embedding when a document is created or updated. Then, when someone searches for something, we create an embedding for their search query. Then we find the document embeddings that are closest to the query embedding.\n\nThat's the idea, though in practice embeddings can be quite unpredictable and surprising. Our examples above already break down… “animal that barks” vibes more with “woof woof woof” than with “dog”. And “dog bites man” and “man bites dog” have even more similar vibes than that, presumably because there's lots of biting going on in both cases.\n\n

\n\nFor our purposes though, embedding similarity is high between “discord bot” and “github api”, or “chat server integration”. This helps in finding code that is not exactly the same, but still relevant to the search query.\n\n

\"\" — \n Similar vibes, plotted by [this val](https://www.val.town/v/janpaul123/semanticSearchBlogPostPlot).\n

\n\n### Prototyping in Val Town\n\nI started prototyping this on my computer, but then realized: this is Val Town, the ultimate prototyping tool for things like this. I ended up making three different semantic search vals, using different storage engines: [Postgres](https://www.postgresql.org/) (using [Neon](https://neon.tech/)), [SQLite](https://sqlite.org/) (using [Turso](https://turso.tech/)), and in-memory (using [blob storage](https://docs.val.town/std/blob/)). Let's look at Postgres first, and at the end I'll also show how to do this in-memory.\n\nThere are two phases to semantic search:\n\n1. **Indexing**: generating embeddings for all public vals.\n2. **Querying**: generating an embedding for the search query, and comparing to the indexed embeddings.\n\nFor indexing, we first create a Postgres table on Neon (using the [pgvector](https://github.com/pgvector/pgvector) extension):\n\n```sql\nCREATE TABLE vals_embeddings (id TEXT PRIMARY KEY, embedding VECTOR(1536));\n```\n\nThen we create a [cron val](https://docs.val.town/quickstarts/first-cron/) (or as I like to call them: inter-val…), which we set to run every hour.\n\nFirst we query all vals, for which Achille Lacoin had already made an excellent val:\n\n

\"\" — \n Querying public vals, by Achille Lacoin.\n

\n\nWe filter out vals for which we have existing embeddings, put the rest in batches of 100 vals.\n\n

\n\nThen we fetch embeddings from OpenAI for each batch of 100 in parallel, and save them to Postgres.\n\n

\n\nThis works! Here are the logs from a recent incremental run:\n\n

\n\nQuerying is simpler still. We turn the query string into an embedding, and sort by similarity:\n\n

\n\nFor a nicer UI, I forked an HTTP val (also by Achille Lacoin) and hooked it up to my search function:\n\n

\n\nThis works pretty well! Check out all my semantic search prototypes [here](https://www.val.town/v/janpaul123/valtownsemanticsearch).\n\n### Product\n\nOnce the Postgres prototype worked well, I integrated it into Val Town itself. The implementation is roughly the same: I added a pgvector `embedding` column to our Postgres table that stores vals, and generate an embedding whenever a val is created or updated.\n\nYou can use it by going to the [search page](https://www.val.town/search) and selecting “semantic search”.\n\n

\n\nWe're still iterating on this feature. Here are some ideas we have to improve it, but if you have your own, please let us know on [Discord](https://discord.gg/dHv45uN5RY) or [Github](https://github.com/val-town/val-town-product/discussions)!\n\n- Combine exact and semantic search. Label them differently by highlighting exact matches for exact search, and maybe label semantic results different.\n- For searches that return a lot of exact results, improve ranking by using the semantic distance as a ranking feature, in lieu of not having [tf-idf](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) stats with our current search setup.\n- For both exact and semantic results, add popularity stats (references, forks, likes) as ranking signals. Show all of these in the search results for easier debugging of ranking (currently only likes are shown).\n- Do fuzzy search (e.g. [levenshtein](https://en.wikipedia.org/wiki/Levenshtein_distance) or similar) in JavaScript, using the top N results from semantic search, to account for misspellings or similar spellings.\n- Split exact search on spaces, giving ranking preference to exact phrases and number of parts matching.\n\n### Bonus: In-memory\n\nAs promised, here is the implementation using in-memory comparisons and blob storage.\n\nThis doesn't require any sort of database, just a place to store bytes. I used Val Town's standard library, which contains [blob storage functions](https://docs.val.town/std/blob/). You can use it to store binary data, or JSON (which gets serialized/deserialized). So we can simply store a blob for every batch of 100 vals, and keep a separate JSON storage to keep track of which embedding is stored where:\n\n

\n\nThen to query, we load all the blobs in memory, and compare them all against the embedding from the query string. This is of course not as efficient as using an index optimized for this sort of sorting, but not too bad for our (still) small number of vals.\n\n

\n\nThanks for reading!\n\n

",[],[191,1201],"c402cecf-29d6-40ac-a20c-de5df065c042","2024-06-13-http-streaming.md","86b68edc-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: HTTP Streaming in Val Town\ndescription: >-\n Val Town supports HTTP streaming for streaming LLMs, large requests and\n responses, and server-sent events\npubDate: '2024-06-13T04:00:00.000Z'\nauthor: Max McDonnell\ntags: [product, engineering]\n---\n\nVal Town now supports HTTP streaming, [our most-requested feature](https://github.com/val-town/val-town-product/discussions/14).\n\n $\"The$ \n\nYou may recognize me as the guy who originally requested that feature. Now I work here and I shipped it!\n\nStreaming opens up many new use-cases: streaming LLM responses, large requests and responses, and server-sent events.\n\n## Streaming LLM Responses\n\nLLMs are a classic use-case for streaming: they send responses word-by-word as they're generated. Here is a minimal example of a streaming OpenAI response: OpenAPI’s SDK lets us request a streaming response by specifying `stream: true`. They return an [AsyncIterator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncIterator) and we turn that into a [ReadableStream](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/ReadableStream) for our response.\n\n```ts\nimport { OpenAI } from \"https://esm.town/v/std/openai\";\n\nexport default async function (req: Request): Promise {\n const openai = new OpenAI();\n const stream = await openai.chat.completions.create({\n stream: true,\n messages: [\n {\n role: \"user\",\n content: \"Write a poem in the style of beowulf about the DMV\",\n },\n ],\n model: \"gpt-3.5-turbo\",\n max_tokens: 2048,\n });\n return new Response(\n new ReadableStream({\n async start(controller) {\n for await (const chunk of stream) {\n controller.enqueue(\n new TextEncoder().encode(chunk.choices[0]?.delta?.content)\n );\n }\n controller.close();\n },\n }),\n { headers: { \"Content-Type\": \"text/event-stream\" } }\n );\n}\n```\n\nIt's a little bit of a dance to connect the two tools together, but the result is fun:\n\n

\n\n[Fork on Val Town](https://www.val.town/v/maxm/openAIStreamingExample) | [View live output](https://maxm-openaistreamingexample.web.val.run)\n\n### The request limit is now 100mb\n\nWe've increased the limit on request body size from 2mb to\nto 100mb. Here's [an example val](https://www.val.town/v/maxm/reportBodySize) that returns the size of the request body without holding all of it in memory:\n\n```ts\nexport default async function (req: Request): Promise {\n if (req.method !== \"POST\") {\n return new Response(\"Method not allowed\", { status: 405 });\n }\n const reader = req.body.getReader();\n let totalBytes = 0;\n console.log(performance.now());\n while (true) {\n const { done, value } = await reader.read();\n if (done) break;\n console.log(value.byteLength, performance.now());\n totalBytes += value.byteLength;\n }\n return new Response(`${totalBytes}`);\n}\n```\n\nIf we upload a large file to that endpoint it will respond as we'd expect:\n\n```bash\n$ curl --data @image.png https://maxm-reportbodysize.web.val.run\n66799801\n```\n\n## Unlimited response sizes\n\nWe used to buffer and store responses, which imposed a 10mb limit on how much send from an HTTP val. Now we don't, and you can send very big responses:\n\nIn the example below we proxy the latest [130mb\nsupernovae image](https://www.flickr.com/photos/nasawebbtelescope/53782948438/in/album-72177720313923911/) from the Webb Telescope. The client can begin downloading the image before the server has finished downloading it.\n\n```ts\nexport default async function (req: Request) {\n return fetch(\n \"https://live.staticflickr.com/65535/53782948438_9b85e57a6c_o_d.png\"\n );\n}\n```\n\n
\n\n $\"James$ \n\n[Fork on Val Town](https://www.val.town/v/maxm/jamesWebbImageProxy) | [View live output](https://maxm-jameswebbimageproxy.web.val.run)\n\n## Server-Sent Events\n\n[Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events) are kind of like a one-way WebSocket. The ChatGPT example above actually used server-sent events. They're useful when you want to work with a stream of discrete events, rather than a continuous stream of data. Here's an example of constructing a server-sent event stream manually to make a clone of [robpike.io](https://robpike.io/).\n\n```ts\nconst msg = new TextEncoder().encode(\"💩\");\nconst initialDelay = 20;\nexport default async function (req: Request): Promise {\n let timerId: number | undefined;\n const body = new ReadableStream({\n start(controller) {\n let currentDelay = initialDelay;\n function writeToStream() {\n currentDelay *= 1.03;\n controller.enqueue(msg);\n timerId = setTimeout(writeToStream, currentDelay);\n }\n writeToStream();\n },\n cancel() {\n if (typeof timerId === \"number\") {\n clearInterval(timerId);\n }\n },\n });\n return new Response(body, {\n headers: {\n \"Content-Type\": \"text/event-stream\",\n },\n });\n}\n```\n\n
\n\n

\n\n[Fork on Val Town](https://www.val.town/v/maxm/robPikeIO) | [View live output](https://maxm-robpikeio.web.val.run)\n\nFor a more complex application using server-sent events, check out my [little ChatGPT clone](https://www.val.town/v/maxm/valTownChatGPT).\n\n### Under the hood\n\nWe had to entirely reorient the way we run user code to support streaming.\n\nPreviously, both requests and responses were read into buffers before being sent to and from the val. This had both speed and scalability issues, as we wouldn't want to buffer a very large response in memory. Now, we only communicate with user code over HTTP.\n\nWith HTTP vals, we proxy the incoming request straight to the worker. This way we don't mess too much with the request structure and you'll see a request lifecycle that is very similar to running your own HTTP server.\n\nIf you're running a script, email, or cron val we send an HTTP request with the relevant information and wait for a response. Now that we're talking HTTP, this paves the way for future support for [websockets](https://github.com/val-town/val-town-product/discussions/144).\n\nWe built most of this functionality in a library called deno-http-worker, [which we've open sourced on GitHub](https://github.com/val-town/deno-http-worker).\n\nAnd let this be a lesson to you: be careful what features you wish for – you just may end up joining the company and building them.\n\n### Go forth and stream\n\nWe're excited to see what you build! Here're a few more examples to inspire your next val:\n\n- [maxm/multiplayerCircles](https://www.val.town/v/maxm/multiplayerCircles) - a\n collaborative canvas of movable circles that polls the database on the backend\n and streams changes to the frontend.\n- [thesephist/webgen](https://www.val.town/v/thesephist/webgen) - ask chatGPT to generate a website and watch it render bit by bit in front of your eyes. (watch this built live [on Steve's live stream](https://x.com/i/broadcasts/1yNGaZYkdVXJj))\n- [maxm/asciiNycCameras](https://www.val.town/v/maxm/asciiNycCameras) - streaming ASCII videos of NYC traffic cameras.",[],[191,1208],"37d35453-023d-4046-8836-dafd0bdd292f","2024-05-29-val-town-newsletter-17.md","86b5f2e2-7bcf-11f1-b068-1607ee4eb77e",["D",1784053703462],"---\ntitle: Val Town Newsletter 17\ndescription: 'Improved performance, redesigned val page, live coding, and much more!'\npubDate: '2024-05-29T04:00:00.000Z'\nauthor: Steve Krouse\ntags: [newsletter]\n---\n\n\n\nOver the last month, we drastically improved val runtime performance, redesigned the val page, and worked on dozens of other improvements and bug fixes. Jackson and JP joined the team! Steve started a weekly live coding series.\n\nNow we're working on HTTP streaming and improving user onboarding. Over the coming weeks and months, we'll work on better search, improved val organization, a new pricing plan, more performance improvements, and more.\n\nThere were so many amazing community vals this month! Here are a few highlights: [Static Chess](https://www.val.town/v/maxm/staticChess), [Robin Sloan's book preorder email handler](https://www.robinsloan.com/lab/at-home-in-high-dimensional-space/#valtown), and [Cameron's homeless resources Custom GPT](https://www.val.town/v/cameronpak/homeless). See below for _sixty_ (!!) more awesome vals from the last month.\n\n### 🤗 Welcome Jackson & JP!\n\nWe're so excited to welcome Jackson and JP to the team!\n\n[Jackson](https://jxnblk.com/) was most recently an indie game designer, and before that a Staff Engineer at Gatsby. He's known for [styled-system](https://styled-system.com/), [theme-ui](https://theme-ui.com/), and [MDX Deck](https://mdxjs.com/). He's joining full-time as our founding design engineer.\n\n[JP](https://janpaulposma.nl/) was most recently uncovering ancient Roman secrets at [Vesuvius Challenge](https://scrollprize.org/), and before that founded [Zaplib](https://zaplib.com/) with Steve. He is joining part-time for the summer.\n\n### ⚡️ Vals are 2x faster!\n\nMax shipped [a huge performance improvement](https://github.com/val-town/val-town-product/discussions/113#discussioncomment-9203997) to the val runtime by cutting out a network hop on imports. Vals are at least 60% faster, and vals with many imports are many times faster. For example, the [Date Me Directory](https://dateme.directory) (one of the most complex vals) runs 6x faster now – half a second vs 3 seconds!\n\n### 🎨 Val Page Redesign\n\nTom [redesigned the val page](/redesigning-val-pages/), and then Jackson redesigned it again!\n\n

\n $\"Val$ \n

Before

\n $\"Val$ \n

After

\n\n### 📺 Livestreams\n\nSteve started a weekly live coding series. Tune into the next episodes:\n\n- [Max Bittker – May 30, 12pm ET](https://www.youtube.com/watch?v=6QGjdDGEkyI)\n- [Linus Lee – June 6, 12pm ET](https://www.youtube.com/watch?v=5nYPeK9_X6o)\n\n

hello!

Diff · v824