A lightweight, optimized vector database built on Val Town's SQLite (powered by Turso/libSQL). Supports any OpenAI-compatible embedding provider.

• ✅ Semantic search with cosine similarity + distance scores
• ✅ Multi-provider support - Nebius, OpenAI, OpenRouter, or any OpenAI-compatible API
• ✅ Smart re-embedding - only re-embeds when text changes (content hash)
• ✅ Optimized storage - float8 compression, tuned DiskANN index
• ✅ Scale testing tools - seed data, calibrate thresholds, detailed stats
• ✅ Metadata filtering - filter by meta fields on search
• ✅ Hybrid search - vector + keyword boosting
• ✅ Batch upserts - submit many records in one call
• ✅ Chunking helper - split long docs into chunks
• ✅ Export/import - migrate data between providers

Provider	Env Vars	Default Model	Dimensions
Nebius (default)	NEBIUS_API_KEY	Qwen3-Embedding-8B	4096
OpenAI	EMBEDDING_PROVIDER=openai + OPENAI_API_KEY	text-embedding-3-small	1536
OpenRouter	EMBEDDING_PROVIDER=openrouter + OPENROUTER_API_KEY	openai/text-embedding-3-small	1536
Custom	See below	(configurable)	(configurable)

EMBEDDING_API_URL=https://your-api.com/v1/embeddings
EMBEDDING_API_KEY=your-key
EMBEDDING_MODEL=your-model-name
EMBEDDING_DIM=1536

Metric	Value
Storage per record	~22 KB
Max records per 1GB	~47,500
Avg embedding time	~460ms
Recommended maxDistance	0.6 - 0.65

Method	Endpoint	Description
POST	/upsert	Insert/update {id, text, meta?}
POST	/search	Search {query, k?, maxDistance?}
POST	/delete	Delete {id}
GET	/get?id=...	Get single record
GET	/list?limit=...&offset=...&prefix=...	List record IDs
POST	/upsert_chunked	Chunk + upsert {id, text, meta?, chunkSize?, overlap?}
GET	/export?limit=...&offset=...	Export records
POST	/import	Import records (batch upsert)

Method	Endpoint	Description
GET	/	API info + current provider
GET	/ping	Health check
GET	/stats	Detailed storage stats
GET	/seed?n=100	Seed N synthetic records
GET	/calibrate?q=...	Suggest distance thresholds
POST	/reindex	Recreate optimized index
POST	/clear?confirm=yes	Delete ALL records

curl -X POST https://YOUR_ENDPOINT/upsert \
  -H "Content-Type: application/json" \
  -d '{"id": "doc-1", "text": "Dogs are loyal pets", "meta": {"category": "animals"}}'

curl -X POST https://YOUR_ENDPOINT/search \
  -H "Content-Type: application/json" \
  -d '{"query": "furry pets", "k": 5, "maxDistance": 0.64}'

curl -X POST https://YOUR_ENDPOINT/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "machine learning",
    "k": 10,
    "filters": { "category": "tech" },
    "hybrid": { "enabled": true, "alpha": 0.25 }
  }'

curl -X POST https://YOUR_ENDPOINT/search \
  -H "Content-Type: application/json" \
  -d '{"query": "notes", "k": 10, "offset": 10}'

curl -X POST https://YOUR_ENDPOINT/upsert \
  -H "Content-Type: application/json" \
  -d '[
    {"id":"doc-1","text":"A short note","meta":{"category":"notes"}},
    {"id":"doc-2","text":"Another note","meta":{"category":"notes"}}
  ]'

curl -X POST https://YOUR_ENDPOINT/upsert_chunked \
  -H "Content-Type: application/json" \
  -d '{"id":"doc-long","text":"...long text...","chunkSize":800,"overlap":100}'

curl "https://YOUR_ENDPOINT/export?limit=200&offset=0"

curl -X POST https://YOUR_ENDPOINT/import \
  -H "Content-Type: application/json" \
  -d '{"records":[{"id":"doc-1","text":"hello"}]}'

curl "https://YOUR_ENDPOINT/calibrate?q=machine+learning"

Distance	Meaning	Recommendation
0.0 - 0.4	Very similar	Always include
0.4 - 0.6	Related	Include (tight mode)
0.6 - 0.7	Somewhat related	Include (balanced mode)
0.7+	Likely unrelated	Filter out

Default: Use maxDistance: 0.64 for balanced results.

import * as db from "https://esm.town/v/kamenxrider/slimarmor/vectordb.ts";

// Check provider configuration
console.log(db.getProviderInfo());
// → { provider: "nebius", model: "Qwen3-Embedding-8B", dimensions: 4096, ... }

// Setup (creates table + index)
await db.setup();

// Upsert
await db.upsert("doc-1", "Your text here", { category: "notes" });

// Search
const results = await db.search("search query", 10, 0.64);
// → [{ id, text, meta, distance }, ...]

// Delete
await db.remove("doc-1");

// Stats
const stats = await db.stats();
// → { count: 105, estimated_storage_mb: 2.26 }

⚠️ Important: Embeddings from different providers/models are incompatible. If you switch:

1. Export any data you need
2. Clear the database: POST /clear?confirm=yes
3. Update environment variables
4. Re-insert your data

Tip: You can set EMBEDDING_DIM=auto to auto-detect dimensions on first setup.

SlimArmor uses optimized DiskANN settings for storage efficiency:

Setting	Value	Effect
metric	cosine	Cosine similarity
max_neighbors	64	66% fewer neighbors vs default
compress_neighbors	float8	75% less index storage

INDEX_METRIC=cosine            # cosine (default) or l2
INDEX_MAX_NEIGHBORS=64         # 8-256
INDEX_COMPRESS_NEIGHBORS=float8 # float8 (default) or none

File	Description
vectordb.ts	Core library - import this in your vals
api.ts	HTTP API endpoints
README.md	This documentation
GUIDE.md	Step-by-step beginner guide
HANDOVER.md	Technical handover notes

• Runtime: Val Town (Deno)
• Database: Val Town SQLite (Turso/libSQL)
• Vector Index: DiskANN with cosine similarity
• Embeddings: Any OpenAI-compatible API

MIT