This val hosts an HTTP server that lets you search all vals based on vibes. If you search for "discord bot" it shows all vals that have "discord bot" vibes.
It does this by comparing embeddings from OpenAI generated for the code of all public vals, to an embedding of your search query.
This is an experiment to see if and how we want to incorporate semantic search in the actual Val Town search page.
I implemented three backends, which you can switch between in the search UI. Check out these vals for details on their implementation.
- Neon: storing and searching embeddings using the pg_vector extension in Neon's Postgres database.
- Searching: janpaul123/semanticSearchNeon
- Indexing: janpaul123/indexValsNeon
- Blobs: storing embeddings in Val Town's standard blob storage, and iterating through all of them to compute distance. Slow and terrible, but it works!
- Searching: janpaul123/semanticSearchBlobs
- Indexing: janpaul123/indexValsBlobs
- Turso: storing and searching using the sqlite-vss extension. Abandoned because of a bug in Turso's implementation.
- Searching: janpaul123/semanticSearchTurso
- Indexing: janpaul123/indexValsTurso
All implementations use the database of public vals, made by Achille Lacoin, which is refreshed every hour. The Neon implementation updates every 10 minutes, and the other ones are not updated. I also forked Achille's search UI for this val.
Please share any feedback and suggestions, and feel free to fork our vals to improve them. This is a playground for semantic search before we implement it in the product for real!
Migrated from folder: semanticSearchPrototype/valtownsemanticsearch