Simon Willison’s Weblog

\n Subscribe\n

\n\n

\n\n\n\n

Load Llama-3.2 WebGPU in your browser from a local folder (via) Inspired by a comment on Hacker News I decided to see if it was possible to modify the transformers.js-examples/tree/main/llama-3.2-webgpu Llama 3.2 chat demo (online here, I wrote about it last November) to add an option to open a local model file directly from a folder on disk, rather than waiting for it to download over the network.

I posed the problem to OpenAI's GPT-5-enabled Codex CLI like this:

git clone https://github.com/huggingface/transformers.js-examples\ncd transformers.js-examples/llama-3.2-webgpu\ncodex\n

Then this prompt:

\n
Modify this application such that it offers the user a file browse button for selecting their own local copy of the model file instead of loading it over the network. Provide a \"download model\" option too.
\n

Codex churned away for several minutes, even running commands like curl -sL https://raw.githubusercontent.com/huggingface/transformers.js/main/src/models.js | sed -n '1,200p' to inspect the source code of the underlying Transformers.js library.

After four prompts total (shown here) it built something which worked!

To try it out you'll need your own local copy of the Llama 3.2 ONNX model. You can get that (a ~1.2GB) download) like so:

git lfs install\ngit clone https://huggingface.co/onnx-community/Llama-3.2-1B-Instruct-q4f16\n

Then visit my llama-3.2-webgpu page in Chrome or Firefox Nightly (since WebGPU is required), click \"Browse folder\", select that folder you just cloned, agree to the \"Upload\" confirmation (confusing since nothing is uploaded from your browser, the model file is opened locally on your machine) and click \"Load local model\".

Here's an animated demo.

$\"GIF.$

I pushed a branch with those changes here. The next step would be to modify this to support other models in addition to the Llama 3.2 demo, but I'm pleased to have got to this proof of concept with so little work beyond throwing some prompts at Codex to see if it could figure it out.

According to the Codex /status command this used 169,818 input tokens, 17,112 output tokens and 1,176,320 cached input tokens. At current GPT-5 token pricing ($1.25/million input, $0.125/million cached input, $10/million output) that would cost 53.942 cents, but Codex CLI hooks into my existing $20/month ChatGPT Plus plan so this was bundled into that.

\n\n

Posted 8th September 2025 at 8:53 pm

\n\n

Recent articles

GPT-5 Thinking in ChatGPT (aka Research Goblin) is shockingly good at search - 6th September 2025
V&A East Storehouse and Operation Mincemeat in London - 27th August 2025
The Summer of Johann: prompt injections as far as the eye can see - 15th August 2025

\n\n\n

\n\n

\n\n\n \n \n javascript\n 720\n \n \n \n ai\n 1551\n \n \n \n generative-ai\n 1360\n \n \n \n llama\n 77\n \n \n \n local-llms\n 139\n \n \n \n llms\n 1331\n \n \n \n ai-assisted-programming\n 230\n \n \n \n transformers-js\n 11\n \n \n \n webgpu\n 9\n \n \n \n llm-pricing\n 53\n \n \n \n vibe-coding\n 39\n \n \n \n gpt-5\n 13\n \n \n \n codex-cli\n 3\n \n \n\n\n

\n Monthly briefing\n

\n Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.\n

\n Pay me to send you less!\n

\n \n Sponsor & subscribe\n \n

\n \n

\n\n

\n\n\n\n\n\n\n\n\n\n

Colophon
©
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025

\n\n\n\n\n\n\n`;\n\n// let td = new turndown({\n// headingStyle: \"atx\",\n// hr: \"---\",\n// codeBlockStyle: \"fenced\",\n// });\n\n// td.addRule(\"strikethrough\", {\n// filter: [\"del\", \"s\", \"strike\"],\n// replacement: function (content) {\n// return \"~\" + content + \"~\";\n// },\n// });\n\n// td.remove(\"script\");\n// td.remove(\"style\");\n\n// let res = td.turndown(htmlExample);\n\nlet dp = new DOMParser();\n\nlet doc = dp.parseFromString(htmlExample, \"text/html\");\n\nlet { content, ...res } = new Defuddle(doc, {\n url: \"https://simonwillison.net/2025/Sep/8/webgpu-local-folder/\",\n markdown: true,\n}).parse();\n\nconsole.log(res);","deleted","label",[98],"permalink","https://www.val.town/x/matthamlin/untitled-9497/code/main.ts","extension","actionData","errors"]