A lightweight, optimized vector database built on Val Town's SQLite (powered by Turso/libSQL). Supports any OpenAI-compatible embedding provider.

• ✅ Semantic search with cosine similarity + distance scores
• ✅ Multi-provider support - Nebius, OpenAI, OpenRouter, or any OpenAI-compatible API
• ✅ Smart re-embedding - only re-embeds when text changes (content hash)
• ✅ Optimized storage - float8 compression, tuned DiskANN index
• ✅ Scale testing tools - seed data, calibrate thresholds, detailed stats
• ✅ Metadata filtering - filter by meta fields on search
• ✅ Hybrid search - vector + keyword boosting
• ✅ Batch upserts - submit many records in one call
• ✅ Chunking helper - split long docs into chunks
• ✅ Export/import - migrate data between providers

EMBEDDING_API_URL=https://your-api.com/v1/embeddings
EMBEDDING_API_KEY=your-key
EMBEDDING_MODEL=your-model-name
EMBEDDING_DIM=1536

curl -X POST https://YOUR_ENDPOINT/upsert \
  -H "Content-Type: application/json" \
  -d '{"id": "doc-1", "text": "Dogs are loyal pets", "meta": {"category": "animals"}}'

curl -X POST https://YOUR_ENDPOINT/search \
  -H "Content-Type: application/json" \
  -d '{"query": "furry pets", "k": 5, "maxDistance": 0.64}'

curl -X POST https://YOUR_ENDPOINT/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "machine learning",
    "k": 10,
    "filters": { "category": "tech" },
    "hybrid": { "enabled": true, "alpha": 0.25 }
  }'

curl -X POST https://YOUR_ENDPOINT/search \
  -H "Content-Type: application/json" \
  -d '{"query": "notes", "k": 10, "offset": 10}'

curl -X POST https://YOUR_ENDPOINT/upsert \
  -H "Content-Type: application/json" \
  -d '[
    {"id":"doc-1","text":"A short note","meta":{"category":"notes"}},
    {"id":"doc-2","text":"Another note","meta":{"category":"notes"}}
  ]'

curl -X POST https://YOUR_ENDPOINT/upsert_chunked \
  -H "Content-Type: application/json" \
  -d '{"id":"doc-long","text":"...long text...","chunkSize":800,"overlap":100}'

curl "https://YOUR_ENDPOINT/export?limit=200&offset=0"

curl -X POST https://YOUR_ENDPOINT/import \
  -H "Content-Type: application/json" \
  -d '{"records":[{"id":"doc-1","text":"hello"}]}'

curl "https://YOUR_ENDPOINT/calibrate?q=machine+learning"

Default: Use maxDistance: 0.64 for balanced results.

import * as db from "https://esm.town/v/kamenxrider/slimarmor/vectordb.ts";

// Check provider configuration
console.log(db.getProviderInfo());
// → { provider: "nebius", model: "Qwen3-Embedding-8B", dimensions: 4096, ... }

// Setup (creates table + index)
await db.setup();

// Upsert
await db.upsert("doc-1", "Your text here", { category: "notes" });

// Search
const results = await db.search("search query", 10, 0.64);
// → [{ id, text, meta, distance }, ...]

// Delete
await db.remove("doc-1");

// Stats
const stats = await db.stats();
// → { count: 105, estimated_storage_mb: 2.26 }

⚠️ Important: Embeddings from different providers/models are incompatible. If you switch:

1. Export any data you need
2. Clear the database: POST /clear?confirm=yes
3. Update environment variables
4. Re-insert your data

Tip: You can set EMBEDDING_DIM=auto to auto-detect dimensions on first setup.

Use the self-check endpoint to validate core behavior:

curl https://YOUR_ENDPOINT/validate

Optional write tests (require auth, embedding key, and ALLOW_WRITE_TESTS=1):

curl -H "Authorization: Bearer $ADMIN_TOKEN" \
  "https://YOUR_ENDPOINT/validate?write=yes"

If you can’t send headers (embedded preview), temporarily set ALLOW_WRITE_TESTS_NOAUTH=1 and call:

curl "https://YOUR_ENDPOINT/validate?write=yes"

Latest validation run (2026-02-03):

• Status: OK in 3597 ms
• Provider: Nebius Qwen/Qwen3-Embedding-8B (4096 dims)
• Tests: auth, stats, list, search passed; write tests passed (write_upsert, filter_search, hybrid_search, write_delete)

SlimArmor uses optimized DiskANN settings for storage efficiency:

INDEX_METRIC=cosine            # cosine (default) or l2
INDEX_MAX_NEIGHBORS=64         # 8-256
INDEX_COMPRESS_NEIGHBORS=float8 # float8 (default), float16, floatb16, float32, float1bit, or none

# DiskANN knobs (optional; omit to use libSQL defaults)
INDEX_ALPHA=1.2                # >= 1; lower = sparser graph (faster, less accurate)
INDEX_SEARCH_L=200             # query-time effort (higher = more accurate, slower)
INDEX_INSERT_L=70              # insert-time effort (higher = better graph, slower inserts)

After changing index settings, rebuild the index:

curl -X POST -H "Authorization: Bearer $ADMIN_TOKEN" https://YOUR_ENDPOINT/reindex

• Runtime: Val Town (Deno)
• Database: Val Town SQLite (Turso/libSQL)
• Vector Index: DiskANN with cosine similarity
• Embeddings: Any OpenAI-compatible API

MIT