Search

- doing 2sec responses is EASY; sub 1-sec prob requires vps
- network hops + isolate warmups kill latency - cosine dist calc and embeddings calc don't matter as much
- small/cheap embeddings seem to not be a problem (?? quality??)
- generating embeddings locally w/ a small model requires downloading a model (size+computationally expensive) while generating embeddings from an API costs around ~600ms
- loading an 80mb xenova--all-miniLM is really good/fast but you it sucks for serverless and for mobile users to download that
  - possibly best way is to host this somewhere w/ the onnx and everything saved locally and you can just run it
- cloudflare ai embeddings are very fast, but worker needs to be warmed up; if not warm then expect ~800ms-2000ms
  - if you ping the cf ai embeddings with a fake request for warmup then it's fast (e.g. while a user is typing)
- for testing at least, loading in a massive json file into memory takes a very long time 10+ seconds

// Hugging Face Inference Client + Qwen3-Embedding-8B + Cosine Similarity Strategy
// Uses @huggingface/inference SDK for embeddings via API
// Model: Qwen/Qwen3-Embedding-8B (8B params, up to 4096 dimensions, 32k context)
      if (!HF_TOKEN) {
        throw new Error("HF_TOKEN, HF_API_KEY, or HUGGINGFACE_API_KEY not found - embeddings disabled");
      }
// Generate embeddings using Hugging Face Inference Client
export const generateEmbeddings = async (content: string): Promise<number[] | null> => {
  try {
    if (Array.isArray(output)) {
      // Handle nested array format [[...embeddings...]]
      if (Array.isArray(output[0])) {
      }
      // Handle flat array format [...embeddings...]
      return output;
    
    // Handle object format with embeddings property
    if (output && typeof output === 'object') {
      if (output.embeddings && Array.isArray(output.embeddings)) {
        return Array.isArray(output.embeddings[0]) ? output.embeddings[0] : output.embeddings;
      }
    const errorMessage = error instanceof Error ? error.message : String(error);
    console.error("HF Inference Client embeddings failed:", errorMessage);
    return null;
    const queryEmbedStart = performance.now();
    const queryEmbedding = await generateEmbeddings(query);
    if (enableTiming) {
    for (const page of pages) {
      const pageEmbedding = page.embeddings;
      if (!pageEmbedding || pageEmbedding.length !== queryEmbedding.length) {
        continue; // Skip pages without embeddings or wrong dimension
      }

// Cloudflare Workers AI BGE-Large Strategy
// Uses Cloudflare's @cf/baai/bge-large-en-v1.5 model for embeddings with local cosine similarity
export const generateEmbeddings = async (content: string): Promise<number[] | null> => {
  try {
    
    // Cloudflare wraps the response in: { result: { shape: [1, 1024], data: [[...embeddings...]] }, success: true, errors: [], messages: [] }
    if (responseData.result && responseData.result.data && Array.isArray(responseData.result.data) && responseData.result.data.length > 0) {
    const errorMessage = error instanceof Error ? error.message : String(error);
    console.error("Cloudflare embeddings failed:", errorMessage);
    if (error instanceof Error && error.stack) {
    const queryEmbedStart = performance.now();
    const queryEmbedding = await generateEmbeddings(query);
    if (enableTiming) {
    for (const page of pages) {
      const pageEmbedding = page.embeddings;
      if (!pageEmbedding || pageEmbedding.length !== queryEmbedding.length) {

// Uses Mixedbread's Store API for document storage and semantic search
// No local embeddings needed - Mixedbread handles everything
// This function is required for compatibility with the recalculation system
// For Mixedbread managed Store, embeddings are handled by Mixedbread internally
// Return a dummy array to satisfy the recalculation script
// The actual embeddings are generated and stored by Mixedbread when documents are uploaded
export const generateEmbeddings = async (_content: string): Promise<number[] | null> => {
  // Return a dummy embedding array to satisfy recalculation
  // you should use `deno task recalc-mxbai` instead, which uploads docs to Mixedbread
  return [0]; // Dummy value - actual embeddings are handled by Mixedbread Store
};
  name: "mixedbread",
  description: "Managed AI search using Mixedbread Stores (handles storage, embeddings, and search)",
  search: async (query: string, _pages: Page[], options: SearchOptions = {}): Promise<SearchResult[]> => {

```typescript
import { searchStrategy, generateEmbeddings } from "./transformers-local-onnx.ts";
```
| **transformers-cosine** | ~160-180ms | Free | None (auto-download) | Development |
| **mixedbread-embeddings** | ~50-100ms | Free tier | API key | High accuracy |
| **openai-cosine** | ~100-200ms | Paid | API key | Reliability |
- **`transformers-cosine.ts`** - Auto-download ONNX models
- **`mixedbread-embeddings-cosine.ts`** - Mixedbread API + local cosine
- **`openai-cosine.ts`** - OpenAI embeddings + local cosine
- **`hf-inference-qwen3-cosine.ts`** - HuggingFace Qwen3-8B embeddings
- **`cloudflare-bge-cosine.ts`** - Cloudflare Workers AI
- **`mixedbread.ts`** - Mixedbread Stores (managed)
- **`placeholder.ts`** - Fake embeddings for testing
### Generate Embeddings
```typescript
async function generateEmbeddings(
  content: string
  ↓
Compare with Page Embeddings (cosine similarity, <1ms per page)
  ↓
1. **Use local ONNX models** for production (fastest, most reliable)
2. **Pre-calculate embeddings** during recalculation (don't generate at query time)
3. **Cache the pipeline** (automatically done, but worth noting)
```typescript
import { searchStrategy, generateEmbeddings } from "./transformers-local-onnx.ts";
```
This regenerates all embeddings with the new strategy.
   
   export const generateEmbeddings = async (content: string) => {
     // Generate embeddings
   };

You can then use the model to compute embeddings like this:
// Compute sentence embeddings
const sentences = ['This is an example sentence', 'Each sentence is converted'];

  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",

import { searchStrategy, generateEmbeddings } from "../transformers-local-onnx.ts";
  const start = performance.now();
  const embedding = await generateEmbeddings(query);
  const elapsed = performance.now() - start;

import { searchStrategy, generateEmbeddings } from "../transformers-local-onnx.ts";
// Test 1: Generate embeddings for a simple query
console.log("Test 1: Generate embeddings for a query");
console.log("Query: 'What is Groq?'\n");
const start = performance.now();
const embeddings = await generateEmbeddings("What is Groq?");
const elapsed = performance.now() - start;
if (embeddings) {
  console.log(`✅ Generated embeddings successfully!`);
  console.log(`   Dimensions: ${embeddings.length}`);
  console.log(`   First 5 values: [${embeddings.slice(0, 5).map(v => v.toFixed(4)).join(", ")}...]`);
  console.log(`   Time: ${elapsed.toFixed(2)}ms`);
} else {
  console.log(`❌ Failed to generate embeddings`);
}
// Test 2: Generate embeddings for multiple queries (to test caching)
console.log("Test 2: Generate embeddings for multiple queries (testing cache)");
  const queryStart = performance.now();
  const queryEmbedding = await generateEmbeddings(query);
  const queryElapsed = performance.now() - queryStart;
    content: "Groq is a fast AI inference platform that provides APIs for various language models.",
    embeddings: await generateEmbeddings("Groq is a fast AI inference platform that provides APIs for various language models."),
  },
    content: "Learn how to create and manage your Groq API keys for authentication.",
    embeddings: await generateEmbeddings("Learn how to create and manage your Groq API keys for authentication."),
  },
    content: "Groq supports various language models including Llama, Mixtral, and Gemma.",
    embeddings: await generateEmbeddings("Groq supports various language models including Llama, Mixtral, and Gemma."),
  },

// Comment out the current strategy
// import { searchStrategy, generateEmbeddings } from "./transformers-cosine.ts";
// Uncomment the local ONNX strategy
import { searchStrategy, generateEmbeddings } from "./transformers-local-onnx.ts";
```
✅ Generated embeddings successfully!
   Dimensions: 384
Test 2: Generate embeddings for multiple queries (testing cache)
✅ "How to use Groq API?"
```typescript
import { searchStrategy, generateEmbeddings } from "./transformers-local-onnx.ts";
```
### Optional: Recalculate Embeddings
If you were using a different strategy before, regenerate embeddings:
This ensures all page embeddings use the same model.

├─ Need the best accuracy?
│  └─ YES → Use mixedbread-embeddings-cosine.ts or openai-cosine.ts
│
└─ Want managed search (no embeddings management)?
   └─ YES → Use jigsawstack-orama.ts or mixedbread.ts
| **transformers-cosine** | ~3-5s | ~150ms | ~10-30ms | ~160-180ms | ✅ First run only |
| **mixedbread-embeddings** | N/A | N/A | ~50-100ms | ~50-100ms | ✅ Every query |
| **openai-cosine** | N/A | N/A | ~100-200ms | ~100-200ms | ✅ Every query |
| **transformers-cosine** | $0 | ∞ | 100% free, runs locally |
| **mixedbread-embeddings** | $0-$ | Generous | Free tier: 150 req/min, 100M tokens/mo |
| **openai-cosine** | $$ | Limited | $0.0001/1K tokens (text-embedding-3-small) |
| **transformers-cosine** | all-MiniLM-L6-v2 | 384 | ~58 | Same as local |
| **mixedbread-embeddings** | mxbai-embed-large-v1 | 1024 | ~64 | Higher quality |
| **openai-cosine** | text-embedding-3-small | 1536 | ~62 | Reliable, tested |
**Use**: `hf-inference-qwen3-cosine.ts` or `mixedbread-embeddings-cosine.ts`
- No embedding management needed
- Handles storage, search, and embeddings
- Less code to maintain
- When you want a managed solution
- Don't want to store embeddings yourself
- Prefer APIs over local computation
   // Before
   // import { searchStrategy, generateEmbeddings } from "./openai-cosine.ts";
   
   // After
   import { searchStrategy, generateEmbeddings } from "./transformers-local-onnx.ts";
   ```
3. Recalculate embeddings (if dimensions differ):
   ```bash

// Mixedbread Embeddings + Cosine Similarity Strategy
// Uses Mixedbread's Embeddings API for vector generation with local cosine similarity
// Model: mxbai-embed-large-v1 (state-of-the-art English embeddings)
      if (!MIXEDBREAD_API_KEY) {
        throw new Error("MIXEDBREAD_API_KEY or MXBAI_API_KEY not found - embeddings disabled");
      }
// Generate embeddings using Mixedbread Embeddings API
export const generateEmbeddings = async (content: string): Promise<number[] | null> => {
  try {
    
    // Generate embeddings using the SDK's embeddings.create() method
    const response = await client.embeddings.create({
      model: "mixedbread-ai/mxbai-embed-large-v1",
    
    console.warn("Unexpected Mixedbread Embeddings API response format:", JSON.stringify(response, null, 2));
    return null;
    const errorMessage = error instanceof Error ? error.message : String(error);
    console.error("Mixedbread embeddings failed:", errorMessage);
    if (error instanceof Error && error.stack) {
export const searchStrategy: SearchStrategy = {
  name: "mixedbread-embeddings-cosine",
  description: "Semantic search using Mixedbread Embeddings API (mxbai-embed-large-v1) with local cosine similarity",
  search: async (query: string, pages: Page[], options: SearchOptions = {}): Promise<SearchResult[]> => {
    const queryEmbedStart = performance.now();
    const queryEmbedding = await generateEmbeddings(query);
    if (enableTiming) {
    for (const page of pages) {
      const pageEmbedding = page.embeddings;
      if (!pageEmbedding || pageEmbedding.length !== queryEmbedding.length) {
        continue; // Skip pages without embeddings or wrong dimension
      }
      metadata: {
        strategy: "mixedbread-embeddings-cosine",
        similarity,

    title?: string;
    embeddings?: number[] | null;
    metadata?: any;
    
    // Embeddings are arrays - need to handle them specially
    // fast-json emits arrays element by element, so we need to collect them
    const embeddingsBuffer = new Map<number, number[]>();
    
    fastJson.on('results[*].embeddings[*]', (value) => {
      if (!resultsMap.has(currentIndex)) resultsMap.set(currentIndex, {});
      if (!embeddingsBuffer.has(currentIndex)) embeddingsBuffer.set(currentIndex, []);
      embeddingsBuffer.get(currentIndex)!.push(value as unknown as number);
    });
    
    // When embeddings array is done, store it
    fastJson.on('results[*].embeddings', (value) => {
      if (!resultsMap.has(currentIndex)) resultsMap.set(currentIndex, {});
      if (Array.isArray(value)) {
        resultsMap.get(currentIndex)!.embeddings = value;
      } else if (embeddingsBuffer.has(currentIndex)) {
        // Use collected array elements
        resultsMap.get(currentIndex)!.embeddings = embeddingsBuffer.get(currentIndex)!;
        embeddingsBuffer.delete(currentIndex);
      } else {
        resultsMap.get(currentIndex)!.embeddings = value as unknown as number[] | null;
      }

// Fake embeddings function for placeholder (not actually used, but kept for interface consistency)
export const generateEmbeddings = async (content: string): Promise<number[] | null> => {
  // Not used in placeholder strategy, but kept for consistency
  // Returns fake embeddings for caching purposes
  console.debug("Generating fake embeddings for placeholder strategy");
  

  title?: string;
  embeddings?: number[] | null;
  metadata?: any;