Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in miliseconds.
Part of Val Town Semantic Search.
Generates OpenAI embeddings for all public vals, and stores them in Turso, using the sqlite-vss extension.
- Create the
vals_embeddings
andvss_vals_embeddings
tables in Turso if they don't already exist. - Get all val names from the database of public vals, made by Achille Lacoin.
- Get all val names from the
vals_embeddings
table and compute the difference (which ones are missing). - Iterate through all missing vals, get their code, get embeddings from OpenAI, and store the result in Turso.
- When finished, update the
vss_vals_embeddings
table so we can efficiently query them with the sqlite-vss extension.- This is blocked by a bug in Turso that doesn't allow VSS indexes past a certain size.
- Can now be searched using janpaul123/semanticSearchTurso.
Migrated from folder: semanticSearchPrototype/indexValsTurso