chatter

user notes / don't delete!

• figure out how the data model can be API first, e.g. don't rely too much on Dexie, we want this to possibly be an API for other chat clients in the future!

TL;DR: Replace normalized chats + messages tables with atomic sessions documents. Each session is a self-contained JSON object with embedded message tree and branches. Benefits: trivial export/import, perfect portability, simple implementation. Trade-off: whole-document writes (mitigated by debouncing). Perfect for short Q&A chats.

1. Goals & Current State
2. Design Options Reconsidered ← Why atomic sessions win
3. Proposed Schema ← New data structure
4. Migration Plan ← How to upgrade
5. UI Changes ← Phase 1 & 2 rollout
6. Branching Actions ← How to fork/switch branches
7. Backend & Streaming ← Minimal changes needed
8. Export/Import ← The killer feature
9. Performance ← What's fast, what to watch
10. Rollout Plan ← Phased deployment
11. Testing ← Comprehensive test list
12. Quick Reference ← Implementation steps

• Support multiple branching responses per conversation while remaining backward compatible with the current linear model.
• Maximize portability: Each chat session should be a self-contained, inspectable JSON document.
• Minimize disruption to UI logic, Dexie storage, and backend streaming/jobs.
• Keep export/import trivially simple: one session = one JSON file.
• Optimize for the actual use case: short Q&A chats, not 10,000-message conversations.

• No server-side persistence changes right now (branches live in Dexie, as today).
• No immediate large refactors of streaming/job code; aim for additive changes that do not break current flows.
• Not optimizing for extreme scale (1000+ message conversations with deep branching).

Dexie tables (normalized relational model):
- chats:    [{ id: 1, title: "Chat 1", ... }, { id: 2, title: "Chat 2", ... }]
- messages: [{ id: 1, chatId: 1, content: "..." }, { id: 2, chatId: 1, ... }, { id: 3, chatId: 2, ... }]
- folders:  [{ id: 1, name: "Work", ... }]
- systemPrompts: [{ id: 1, name: "Preset 1", ... }]
- notes:    [{ id: 1, title: "Note 1", ... }]

Issues with current approach:

• Messages for ALL chats mixed together in one table
• Requires JOINs (via where('chatId').equals(id)) to load a conversation
• Export requires serializing relationships and IDs
• Import requires remapping IDs and maintaining referential integrity
• Hard to inspect: can't just look at one record to see the full conversation
• Adding branches would require complex parent-pointer traversal across the messages table

• UI maintains a single linear messages array per chat
• Reruns/regenerations mutate the tail of the list (truncate then stream new assistant message)
• Backend is stateless: /api/jobs streams assistant deltas; the client persists to Dexie
• Limitation: No notion of branching. Regenerations overwrite the tail; alternatives cannot coexist.

• Allow multiple alternative assistant replies to a prior point in the conversation (fork at any message).
• Allow users to name/select branches and continue a branch independently.
• Present a linear view of the currently selected path so most UI remains unchanged.
• Backward compatible: existing chats remain usable without any branching changes.
• Keep performance acceptable for short Q&A conversations (typical use case).
• Preserve streaming behavior, message editing, deletion, export/import.
• Make export/import trivial: one session = one self-contained JSON file.

• Messages carry a branchId. Forking creates a new branch ID and subsequent messages adopt it.
• ❌ Cons: Shared pre‑branch messages need duplication OR complex cross-branch handling; non-portable; same JOIN problems as current architecture.

• Each message gets a parentId to form a tree. A branch is a named pointer to a leaf message.
• ✅ Pros: No duplication; natural for forking; easy migrations.
• ❌ Cons for this use case:
  • Still requires complex parent traversal
  • Non-portable (can't just save/load a JSON file)
  • Export requires serializing the graph structure
  • Import requires rebuilding parent chains
  • Hard to inspect (need to traverse pointers to understand conversation)

• Each chat is one self-contained document with embedded message tree and branches.
• Each session document contains: metadata + full message tree + branch pointers + active state.

Why Option C is better for this use case:

✅ Portability:

• Export = JSON.stringify(session) ← literally one line
• Import = validate + insert ← no ID remapping, no relationship rebuilding
• Share a conversation = send one JSON file
• Inspect = open JSON in any editor, see everything

✅ Simplicity:

• One read to load a chat: db.sessions.get(id)
• One write to save: db.sessions.put(session)
• No JOINs, no complex queries
• Delete chat = delete one record (atomically removes everything)

✅ Perfect for your use case:

• Short Q&A chats (50-200 messages typical)
• Not building Slack (10K messages per channel)
• Modern browsers handle 500KB-1MB JSON easily
• Streaming works great (update in-memory, debounce writes)

✅ Better branching:

• Branch tree lives inside the session document
• Easy to see all branches at once
• Export includes full branch structure automatically
• No orphaned messages possible (atomic document)

❌ Theoretical cons that don't apply here:

• "Large rewrites" → Not an issue for short chats
• "Hard to stream" → Update in-memory, debounce to disk works fine
• "Corruption risk" → Use transactions, same risk as any write
• "Multi-user conflicts" → This is single-user, client-side only

Recommendation: Option C (Atomic Session Documents)

Dexie v11 tables:
- sessions:      [{ id: 1, meta: {...}, tree: { messages: [...], branches: [...] } }, ...]
- folders:       [{ id: 1, name: "Work", ... }] (unchanged)
- systemPrompts: [{ id: 1, name: "Preset", ... }] (unchanged)
- notes:         [{ id: 1, title: "Note", ... }] (unchanged)

Store signature:

++id, title, slug, createdAt, folderId, updatedAt

Full document structure:

type Session = {
  // Dexie auto-increment ID
  id: number;
  
  // Metadata (indexed for sidebar/list view)
  title: string;
  slug?: string;
  createdAt: number;
  updatedAt: number;
  folderId?: number | null;
  
  // Preset/settings metadata (for sidebar preview/filtering)
  systemPromptId?: number | null;
  activeSettingsSource?: 'preset' | 'custom' | null;
  
  // The atomic session data (embedded JSON)
  tree: {
    // All messages with parent-pointer structure
    messages: Message[];
    
    // Named branch pointers (if branching enabled)
    branches: Branch[];
    
    // UI state
    activeBranchId?: number | null;
    branchingEnabled?: boolean;
  };
  
  // Settings snapshot (for this session)
  settings?: {
    selectedModel?: string;
    customChatBaseUrl?: string;
    temperature?: number;
    maxTokens?: number;
    reasoningEffort?: 'low' | 'medium' | 'high';
    streamEnabled?: boolean;
    jsonMode?: boolean;
    useBrowserSearch?: boolean;
    useCodeInterpreter?: boolean;
  };
  
  // System prompt snapshot (embedded, not referenced)
  systemPrompt?: string;
};

type Message = {
  id: number;                    // Local ID within this session
  parentId: number | null;        // Points to parent message (null = root)
  createdAt: number;
  role: 'user' | 'assistant' | 'system';
  content: string;
  reasoning?: string;
  model?: string | null;
  promptLabel?: string | null;
  depth?: number;                 // Distance from root (for UI/validation)
};

type Branch = {
  id: number;                     // Local ID within this session
  leafMessageId: number;          // Points to tip of branch
  name: string;                   // e.g. "main", "alternative 1"
  baseMessageId?: number;         // Where branch diverged (optional metadata)
  createdAt: number;
  updatedAt?: number;
};

Key design decisions:

1. Everything embedded: Messages and branches live inside the tree object
2. Simple IDs: Message/branch IDs are simple increments within each session (no global uniqueness needed)
3. Self-contained: Export = grab the session record. Import = validate + insert.
4. Indexed metadata: Only top-level fields (title, createdAt, folderId) are indexed for sidebar queries
5. Snapshot settings: Settings are embedded, not referenced (preserves exact state at time of conversation)

Goal: Transform normalized chats + messages tables into atomic sessions documents.

Strategy:

1. Create new sessions table
2. Migrate each chat + its messages into one session document
3. Keep old tables temporarily for rollback
4. After validation, drop old tables in v12

// In chatCore.js initDb()
state.db.version(11).stores({
  // NEW: Atomic sessions (replaces chats + messages)
  sessions: '++id, title, slug, createdAt, folderId, updatedAt',
  
  // Keep old tables for migration (will remove in v12)
  chats: '++id, title, slug, createdAt, folderId, systemPromptId, systemPromptText, settings, activeSettingsSource, customPromptText',
  messages: '++id, chatId, createdAt, role, content, model',
  
  // Unchanged tables
  folders: '++id, name, createdAt, order, collapsed',
  systemPrompts: '++id, name, content, systemPrompt, settings, createdAt, updatedAt',
  notes: '++id, title, content, createdAt, updatedAt'
}).upgrade(async tx => {
  // Migration logic runs once per user
  console.log('🔄 Migrating v10 → v11: chats+messages → sessions');
  
  const chats = await tx.table('chats').toArray();
  console.log(`Found ${chats.length} chats to migrate`);
  
  for (const chat of chats) {
    try {
      // Load all messages for this chat
      const messages = await tx.table('messages')
        .where('chatId')
        .equals(chat.id)
        .sortBy('createdAt');
      
      // Build linear parent chain (no branching yet)
      const treeMessages = messages.map((msg, idx) => ({
        id: idx + 1,  // Local IDs start at 1
        parentId: idx === 0 ? null : idx,  // Point to previous message
        createdAt: msg.createdAt,
        role: msg.role,
        content: msg.content,
        reasoning: msg.reasoning || undefined,
        model: msg.model || undefined,
        promptLabel: msg.promptLabel || undefined,
        depth: idx
      }));
      
      // Create session document
      const session = {
        // Metadata (copy from chat)
        title: chat.title,
        slug: chat.slug || undefined,
        createdAt: chat.createdAt,
        updatedAt: Date.now(),
        folderId: chat.folderId || undefined,
        systemPromptId: chat.systemPromptId || undefined,
        activeSettingsSource: chat.activeSettingsSource || undefined,
        
        // Embedded tree
        tree: {
          messages: treeMessages,
          branches: [],  // No branches yet (linear)
          activeBranchId: null,
          branchingEnabled: false
        },
        
        // Snapshot settings/prompt
        settings: chat.settings || undefined,
        systemPrompt: chat.systemPromptText || chat.customPromptText || undefined
      };
      
      // Insert session
      await tx.table('sessions').add(session);
      console.log(`✅ Migrated chat ${chat.id}: "${chat.title}" (${messages.length} messages)`);
      
    } catch (err) {
      console.error(`❌ Failed to migrate chat ${chat.id}:`, err);
      // Continue with other chats - don't fail entire migration
    }
  }
  
  console.log('✅ Migration v10 → v11 complete');
   });

New function: loadSession()

export async function loadSession(state, id) {
  if (id === state.ephemeralChatId) {
    // Handle ephemeral new chat
    state.currentChatId = null;
    state.activeSearchId = state.ephemeralChatId;
    state.messages = [{ 
      role: 'assistant', 
      content: 'Hi! How can I help you today?', 
      createdAt: Date.now() 
    }];
    return;
  }
  
  // Load full session document
  const session = await state.db.sessions.get(id);
  if (!session) {
    state.messages = [];
    return;
  }
  
  // Extract current branch path (for now, just linear order)
  state.currentChatId = id;
  state.activeSearchId = id;
  
  // Build linear view from tree
  const tree = session.tree || { messages: [], branches: [] };
  state.messages = tree.messages.map(m => ({
    id: m.id,
    role: m.role,
    content: m.content,
    reasoning: m.reasoning,
    createdAt: m.createdAt,
    model: m.model,
    promptLabel: m.promptLabel
  }));
  
  // Apply session settings
  if (session.settings) {
    if (session.settings.selectedModel) state.selectedModel = session.settings.selectedModel;
    if (session.settings.temperature != null) state.temperature = session.settings.temperature;
    // ... etc
  }
  
  // Load system prompt
  state.chatPromptText = session.systemPrompt || '';
  state.chatPromptId = session.systemPromptId ? String(session.systemPromptId) : null;
}

New function: saveSession()

export async function saveSession(state) {
  if (!state.currentChatId || !state.db) return;
  
  const session = await state.db.sessions.get(state.currentChatId);
  if (!session) return;
  
  // Update tree with current messages
  session.tree.messages = state.messages.map((m, idx) => ({
    id: m.id || idx + 1,
    parentId: idx === 0 ? null : idx,
    createdAt: m.createdAt,
    role: m.role,
    content: m.content,
    reasoning: m.reasoning,
    model: m.model,
    promptLabel: m.promptLabel,
    depth: idx
  }));
  
  session.updatedAt = Date.now();
  
  // Save back to DB
  await state.db.sessions.put(session);
}

export async function loadSessions(state) {
  // Only load metadata for sidebar (not full tree)
  const rows = await state.db.sessions
    .orderBy('createdAt')
    .reverse()
    .toArray();
  
  // Map to lightweight objects for sidebar
  state.chats = rows.map(s => ({
    id: s.id,
    title: s.title,
    slug: s.slug,
    createdAt: s.createdAt,
    folderId: s.folderId
  }));
}

No major changes needed! Streaming works the same:

1. Load session into memory: const session = await db.sessions.get(id)
2. Update state.messages array as deltas arrive (in-memory)
3. Debounce writes to disk:

// After each delta or every 500ms
const debouncedSave = debounce(async () => {
  await saveSession(state);
}, 500);

// In applyJobDelta
if (delta.content) {
  // Update in-memory
  state.messages[idx].content += delta.content;
  
  // Queue save
  debouncedSave();
}

• Count sessions created = chats migrated
• Spot-check a few sessions to ensure messages present
• Export a session and verify it's valid JSON
• Import it back and verify it works

If v11 migration fails:

1. Old chats + messages tables still exist
2. Drop sessions table
3. Revert code to v10
4. Users' data is safe

Goal: Get atomic sessions working with current linear behavior.

Changes needed:

1. Replace loadChats() with loadSessions() (same interface)
2. Replace selectChat() with loadSession() internally
3. Replace incremental message saves with saveSession() (debounced)
4. Keep state.messages as-is (linear array)

What stays the same:

• UI renders state.messages as before
• Streaming updates state.messages in-memory
• No branch UI yet

Goal: Allow users to fork conversations and switch branches.

New UI state:

// In chatterApp.js state
currentBranchId: number | null;  // Which branch is active
sessionTree: {                   // Loaded from session.tree
  messages: Message[];
  branches: Branch[];
  branchingEnabled: boolean;
};

Path building function:

// Build linear view from current branch
function buildMessagePath(session, branchId) {
  const tree = session.tree;
  
  // If no branching, just return messages in order
  if (!tree.branchingEnabled || !tree.branches.length) {
    return tree.messages;
  }
  
  // Find the branch
  const branch = tree.branches.find(b => b.id === branchId);
  if (!branch) return tree.messages;
  
  // Walk from leaf to root
  const path = [];
  let currentId = branch.leafMessageId;
  const visited = new Set();
  
  while (currentId != null && !visited.has(currentId)) {
    visited.add(currentId);
    const msg = tree.messages.find(m => m.id === currentId);
    if (!msg) break;
    
    path.unshift(msg);
    currentId = msg.parentId;
  }
  
  return path;
}

Branch UI Components:

1. Branch selector dropdown (in message header)

   • Shows all branches for current session
   • Switch between branches

2. "Fork" button (on each message)

   • Create new branch from any point
   • Generates alternative responses

3. Branch visualization (optional)

   • Tree view showing all branches
   • Visual indicator of current branch

Enable branching for a session:

async function enableBranching(sessionId) {
  const session = await db.sessions.get(sessionId);
  if (!session) return;
  
  session.tree.branchingEnabled = true;
  
  // Create default "main" branch pointing to last message
  if (session.tree.messages.length > 0) {
    const lastMsg = session.tree.messages[session.tree.messages.length - 1];
    session.tree.branches.push({
      id: 1,
      leafMessageId: lastMsg.id,
      name: 'main',
      baseMessageId: null,
      createdAt: Date.now()
    });
    session.tree.activeBranchId = 1;
  }
  
  await db.sessions.put(session);
}

Fork/create alternative at a message:

async function forkAtMessage(sessionId, messageId) {
  const session = await db.sessions.get(sessionId);
  if (!session || !session.tree.branchingEnabled) return;
  
  // Find the message to fork from
  const baseMsg = session.tree.messages.find(m => m.id === messageId);
  if (!baseMsg) return;
  
  // Generate new message as alternative
  const newMsgId = Math.max(...session.tree.messages.map(m => m.id), 0) + 1;
  const newMessage = {
    id: newMsgId,
    parentId: baseMsg.parentId,  // Same parent (sibling)
    createdAt: Date.now(),
    role: 'assistant',
    content: '',  // Will be filled by streaming
    depth: baseMsg.depth
  };
  
  session.tree.messages.push(newMessage);
  
  // Create new branch pointing to this message
  const newBranchId = Math.max(...session.tree.branches.map(b => b.id), 0) + 1;
  const newBranch = {
    id: newBranchId,
    leafMessageId: newMsgId,
    name: `alternative ${newBranchId}`,
    baseMessageId: baseMsg.parentId,
    createdAt: Date.now()
  };
  
  session.tree.branches.push(newBranch);
  session.tree.activeBranchId = newBranchId;
  
  await db.sessions.put(session);
  return { messageId: newMsgId, branchId: newBranchId };
}

Switch branches:

async function switchBranch(sessionId, branchId) {
  const session = await db.sessions.get(sessionId);
  if (!session) return;
  
  const branch = session.tree.branches.find(b => b.id === branchId);
  if (!branch) return;
  
  // Update active branch
  session.tree.activeBranchId = branchId;
  await db.sessions.put(session);
  
  // Rebuild UI messages array
  state.messages = buildMessagePath(session, branchId);
}

Delete/archive a branch:

async function archiveBranch(sessionId, branchId) {
  const session = await db.sessions.get(sessionId);
  if (!session) return;
  
  // Remove branch from list (messages stay in tree)
  session.tree.branches = session.tree.branches.filter(b => b.id !== branchId);
  
  // If this was active, switch to first available branch
  if (session.tree.activeBranchId === branchId) {
    session.tree.activeBranchId = session.tree.branches[0]?.id || null;
  }
  
  await db.sessions.put(session);
}

Note on orphaned messages: With atomic sessions, orphaned messages aren't a problem—they just live in the tree. You can add a cleanup function to prune unreachable messages if desired, but it's not required.

Good news: Backend stays mostly the same! The atomic session approach doesn't change streaming.

1. Save frequency: Instead of saving each message individually, save the whole session (debounced)
2. Message payload construction: Still builds from state.messages (linear array)
3. Streaming deltas: Still update state.messages[idx].content in-memory

• Backend streams deltas the same way
• applyJobDelta updates messages in-memory
• No backend changes needed for branching

// Existing streaming code mostly unchanged
this.applyJobDelta = async (jobId, parsed) => {
  const ctx = (this._jobCtx || {})[jobId];
  if (!ctx) return;
  
  const choice = Array.isArray(parsed.choices) ? parsed.choices[0] : null;
  const delta = choice?.delta || {};
  
  if (delta.content) {
    // Update in-memory array (same as before)
    if (ctx.chatId === this.currentChatId) {
      const idx = this.messages.findIndex(m => m.id === ctx.assistantMessageId);
      if (idx >= 0) {
        this.messages[idx].content += delta.content;
      }
    }
    
    // NEW: Debounce session save instead of immediate DB write
    this.debouncedSaveSession();
  }
  
  if (delta.reasoning) {
    // Same as before...
    this.messageReasoningMap[ctx.assistantMessageId] += delta.reasoning;
    this.debouncedSaveSession();
  }
};

// Debounced save helper
this.debouncedSaveSession = debounce(async () => {
  await saveSession(this);
}, 500);

Single session:

async function exportSession(sessionId) {
  const session = await db.sessions.get(sessionId);
  if (!session) return;
  
  // That's it. Literally just grab the session.
  const json = JSON.stringify(session, null, 2);
  
  // Download
  const blob = new Blob([json], { type: 'application/json' });
  const url = URL.createObjectURL(blob);
  const a = document.createElement('a');
  a.href = url;
  a.download = `${session.slug || session.id}-session.json`;
  a.click();
  URL.revokeObjectURL(url);
}

All data:

async function exportAllData(state) {
  const sessions = await state.db.sessions.toArray();
  const folders = await state.db.folders.toArray();
  const systemPrompts = await state.db.systemPrompts.toArray();
  const notes = await state.db.notes.toArray();
  
  return {
    meta: {
      type: 'chatter-export-v11',
      exportedAt: Date.now(),
      version: 11
    },
    data: {
      sessions,      // Each session is self-contained!
      folders,
      systemPrompts,
      notes
    }
  };
}

Single session:

async function importSession(jsonFile) {
  const text = await jsonFile.text();
  const session = JSON.parse(text);
  
  // Validate basic structure
  if (!session.tree || !Array.isArray(session.tree.messages)) {
    throw new Error('Invalid session format');
  }
  
  // Remove old ID (will get new auto-increment ID)
  delete session.id;
  
  // Insert (that's it!)
  const newId = await db.sessions.add(session);
  return newId;
}

All data:

async function importDataAppend(state, input) {
  const parsed = JSON.parse(input);
  const data = parsed.data || parsed;
  
  // Import sessions (no ID remapping needed!)
  for (const session of data.sessions || []) {
    delete session.id;  // Get new ID
    await state.db.sessions.add(session);
  }
  
  // Import other tables (same as before)
  for (const folder of data.folders || []) {
    // ... handle folder remapping for folderId references
  }
}

Why this is huge:

• Share a conversation = send one JSON file
• Import into another app = just parse the JSON
• Archive conversations = save JSON files
• Backup strategy = export all sessions weekly
• No complex graph rebuilding on import
• No ID remapping for messages (they're local to session)

function validateNoCycles(session, newMessage) {
  if (newMessage.parentId === null) return true;  // Root is always valid
  
  const visited = new Set();
  let currentId = newMessage.parentId;
  
  while (currentId != null) {
    if (visited.has(currentId)) {
      throw new Error('Cycle detected in message tree');
    }
    visited.add(currentId);
    
    const msg = session.tree.messages.find(m => m.id === currentId);
    if (!msg) break;
    currentId = msg.parentId;
  }
  
  return true;
}

With atomic sessions, multi-tab conflicts are naturally handled:

• Each tab loads session independently
• Last write wins (standard Dexie behavior)
• No partial state corruption (whole document is atomic)
• Consider adding optimistic locking if needed:

  session.version = (session.version || 0) + 1;

Not a problem with atomic sessions! Messages live in the tree even if no branch points to them. You can:

1. Leave them (harmless, take up space but minimal)
2. Add a cleanup tool to prune unreachable messages
3. Show "orphaned messages" view for recovery

• ✅ Loading a session: One read, instant
• ✅ Saving a session: One write, debounced
• ✅ Sidebar list: Only loads metadata, not full trees
• ✅ Linear conversations: No path building needed
• ✅ Branch path building: O(depth) but depth is small (10-50 messages typical)

• ⚠️ Very long conversations (500+ messages): Session size grows, but still manageable
• ⚠️ Many branches (10+ per session): Path building gets slower, but rare
• ⚠️ Rapid saves during streaming: Debounce to 500ms prevents DB thrashing

1. Debounce saves: Only write to disk every 500ms during streaming
2. Lazy load trees: Sidebar only loads metadata, trees loaded on demand
3. Path caching: Cache the current branch path in memory
4. Prune old branches: Archive branches after X days of inactivity

Goal: Replace normalized tables with sessions, keep linear behavior.
Risk: Low (keeps existing behavior, just changes storage)

• Add Dexie v11 schema definition in chatCore.js
• Write migration upgrade function (chats+messages → sessions)
• Add cycle detection validation
• Add migration error handling and logging
• Test migration with sample data locally

• Replace loadChats() → loadSessions() (metadata only)
• Replace selectChat() → loadSession() (full document)
• Add saveSession() function
• Add debounce helper for session saves
• Update createNewChat() to create session
• Update deleteChat() to delete session
• Update duplicateChat() to duplicate session

• Update sendMessage() to use debounced saves
• Update applyJobDelta() to call debouncedSaveSession()
• Update rerunFromUser() to use sessions
• Update rerunAssistant() to use sessions
• Test streaming with rapid deltas

• Update deleteMessage() to work with sessions
• Update deleteMessagesBelow() to work with sessions
• Update message editing to work with sessions
• Ensure reasoning map updates trigger saves

• Simplify exportAllData() to include sessions
• Simplify importDataReplace() for sessions (no ID remapping)
• Simplify importDataAppend() for sessions
• Add single session export function
• Add single session import function
• Test export/import round-trip

• Fresh install works (no migration needed)
• v10 → v11 migration runs successfully
• Verify all chats appear in sidebar
• Verify all messages present in each chat
• Test creating new chat
• Test sending messages
• Test streaming
• Test regenerate/rerun
• Test editing messages
• Test deleting messages
• Test deleting chats
• Test duplicate chat
• Test export/import
• Performance test: load chat with 100+ messages
• Performance test: sidebar with 50+ chats

• Code review
• Deploy to staging/dev
• Monitor browser console for migration errors
• Validate in production-like environment
• Deploy to production
• Monitor for issues
• Update documentation

Goal: Add UI for creating and switching branches.
Risk: Medium (new UI patterns, user education needed)

• Create branchManager.js module
• Implement enableBranching(sessionId) function
• Implement forkAtMessage(sessionId, messageId) function
• Implement switchBranch(sessionId, branchId) function
• Implement archiveBranch(sessionId, branchId) function
• Implement renameBranch(sessionId, branchId, newName) function
• Implement buildMessagePath(session, branchId) function
• Add branch name uniqueness validation

• Create branch selector dropdown component
• Add "Enable branching" button to chat settings
• Add "Fork here" button to message actions menu
• Add branch indicator to chat header
• Add branch count badge
• Style branch UI elements
• Add keyboard shortcuts for branch operations

• Add currentBranchId to app state
• Add sessionTree to app state
• Update loadSession() to handle branches
• Update saveSession() to preserve branches
• Add branch validation on save

• Update sendMessage() to append to current branch
• Update applyJobDelta() to write to current branch
• Add validation: prevent streaming to inactive branch
• Test streaming while switching branches
• Test concurrent branch generation

• Update "Regenerate" to create sibling branch
• Add UI to select which regeneration to keep
• Test multiple alternatives at same point
• Test switching between alternatives

• Verify session export includes full branch tree
• Test import of branched sessions
• Add branch count to export metadata
• Test cross-app import (external tools)

• Enable branching for test session
• Fork at various message points
• Create 5+ branches in one session
• Switch between branches rapidly
• Test streaming to different branches
• Test regenerate creates sibling
• Test delete branch
• Test rename branch
• Test export/import branched session
• Test orphaned message handling
• Performance: switch between branches < 100ms
• Performance: fork operation < 50ms

• Add tooltip explanations for branch features
• Create user guide for branching
• Add visual indicators for current branch
• Add confirmation for destructive branch operations
• Add undo for branch deletion

• Deploy behind feature flag
• Enable for beta users
• Gather feedback
• Fix bugs and iterate
• Gradual rollout to all users
• Monitor usage and performance
• Update help documentation

Goal: Enhance branching with power-user features.
Risk: Low (all optional enhancements)

• Design tree view UI
• Implement visual branch tree renderer
• Add zoom/pan controls
• Add click to switch branches from tree
• Show branch names on tree nodes
• Highlight current branch path

• Implement branch merge/combine
• Add diff view between branches
• Add branch comparison tool
• Add "cherry-pick" messages between branches
• Add branch templates/favorites

• Track which branches are used most
• Track regeneration patterns
• Show "best path" recommendations
• Add branch quality scoring

• Export single branch as separate conversation
• Import branch into existing session
• Share specific branch via URL
• Add branch annotations/comments

• Add "Find orphaned messages" tool
• Add "Prune unused branches" function
• Add "Flatten to linear" option
• Auto-archive old branches

• Implement path caching
• Lazy-load branch trees
• Virtual scrolling for large trees
• Index optimization for branch queries

• Fresh install works (no migration needed)
• v10 → v11 migration runs successfully
• All chats appear in sidebar after migration
• All messages present in migrated chats
• Settings/prompts preserved after migration
• Export v10 → import v11 works
• Export v11 → import v11 works (round-trip)

• Create new chat works
• Send message works
• Streaming displays correctly
• Regenerate works
• Edit message works
• Delete message works
• Delete chat works
• Duplicate chat works
• Folder organization works
• Search/filter works

• Enable branching for session
• Fork at any message creates branch
• Switch between branches works
• Each branch maintains independent content
• Streaming to specific branch works
• Regenerate creates sibling branch
• Delete branch works
• Rename branch works
• Branch count shows correctly
• Cannot create duplicate branch names

• Load chat with 100 messages: < 100ms
• Load chat with 500 messages: < 500ms
• Save session during streaming: < 50ms (debounced)
• Switch between 5 branches: < 100ms
• Sidebar with 100 chats: < 200ms
• Export session with 200 messages: < 500ms
• Import session: < 1s

• Empty session (no messages)
• Session with only assistant greeting
• Very long single message (10K+ chars)
• Rapid branch switching
• Stream interrupted mid-generation
• Browser refresh during stream
• Multi-tab editing (last write wins)
• Orphaned messages (no branch points to them)

✅ Short Q&A chats → Small, fast documents
✅ Portability priority → One file = one conversation
✅ Simple export/import → No complex ID remapping
✅ Easy to inspect → Open JSON, see everything
✅ Client-side only → No concurrency issues

• Replace chats + messages tables → sessions table
• Load session: one read instead of JOIN
• Save session: one write (debounced) instead of incremental
• Export: grab session, done
• Import: validate + insert, done

• UI still uses state.messages linear array
• Streaming still updates in-memory
• Backend unchanged
• Folder organization unchanged
• All existing features work

1. Phase 1 (1-2 weeks): Migrate to sessions, keep linear behavior
2. Phase 2 (2-3 weeks): Add branching UI and features
3. Phase 3 (ongoing): Advanced branch features as needed

• Export a chat → one line of code
• Import a chat → one line of code
• Share with colleague → send JSON file
• Archive conversations → save files locally
• Cross-app compatibility → any tool can read the format
• Future-proof → structure is self-documenting and inspectable

Bottom line: Atomic sessions trade theoretical scalability (which you don't need) for practical simplicity and portability (which you do need).

• Add Dexie v11 schema with sessions table
• Keep old chats and messages tables for migration
• Write .upgrade() function with migration logic
• Add migration logging and error handling

state.db.version(11).stores({
  sessions: '++id, title, slug, createdAt, folderId, updatedAt',
  chats: '++id, title, slug, createdAt, folderId, ...',      // Keep for migration
  messages: '++id, chatId, createdAt, role, content, model', // Keep for migration
  folders: '++id, name, createdAt, order, collapsed',
  systemPrompts: '++id, name, content, systemPrompt, settings, createdAt, updatedAt',
  notes: '++id, title, content, createdAt, updatedAt'
}).upgrade(async tx => {
  // See "Migration Plan" section for full code
});

• Add loadSessions() - loads metadata only (for sidebar)
• Add loadSession(id) - loads full session document
• Add saveSession() - saves full session
• Add debounce helper (500ms delay)
• Update createNewChat() to create session
• Update deleteChat() to delete session
• Update duplicateChat() to duplicate session
• Keep state.messages array (UI compatibility!)

• Add debouncedSaveSession helper
• Update sendMessage() to call debounced save
• Update applyJobDelta() to call debounced save
• Update rerunFromUser() to use sessions
• Test rapid streaming doesn't thrash DB

// Add debounce helper
this.debouncedSaveSession = debounce(async () => {
  await saveSession(this);
}, 500);

// In applyJobDelta:
if (delta.content) {
  this.messages[idx].content += delta.content;
  this.debouncedSaveSession(); // NEW
}

• Update exportAllData() to export sessions
• Simplify importDataReplace() - no ID remapping
• Simplify importDataAppend() - sessions are atomic
• Add single session export function
• Add single session import function

// Single session export
async function exportSession(id) {
  const session = await db.sessions.get(id);
  return JSON.stringify(session, null, 2);
}

// Single session import
async function importSession(json) {
  const session = JSON.parse(json);
  delete session.id;
  return await db.sessions.add(session);
}

• Export all v10 data as backup
• Test migration on copy of real data
• Verify session count = chat count
• Spot-check messages in random sessions
• Test creating new chat
• Test sending messages
• Test streaming
• Test regenerate
• Test delete/edit
• Test export/import round-trip
• Performance test with large chats

• Code review
• Deploy to dev/staging
• Monitor migration logs
• Validate in production-like environment
• Deploy to production
• Monitor for errors in first 24h
• Keep v10 tables for 1-2 weeks (safety)
• Update documentation

Purpose: Database schema and core data operations
Current: v10 schema with normalized chats + messages tables
Changes needed:

• Add Dexie v11 schema with sessions table
• Write migration .upgrade() function
• Replace loadChats() → loadSessions() (metadata only)
• Replace selectChat() → loadSession() (full document)
• Add saveSession() function
• Update createNewChat() to create session
• Update deleteChat() to delete session
• Update duplicateChat() to duplicate session
• Keep message CRUD operations for backward compatibility (optional)

Key functions to modify:

export async function initDb(state)           // Add v11 schema
export async function loadSessions(state)     // NEW - replace loadChats
export async function loadSession(state, id)  // NEW - replace selectChat
export async function saveSession(state)      // NEW - debounced save
export async function createNewChat(state)    // Update to create session
export async function deleteChat(state, id)   // Update to delete session
export async function duplicateChat(state, id) // Update to duplicate session

Purpose: Main application state and streaming logic
Current: Incremental message saves to Dexie
Changes needed:

• Add debouncedSaveSession helper function
• Update sendMessage() to use debounced saves
• Update applyJobDelta() to call debouncedSaveSession()
• Update rerunFromUser() to work with sessions
• Update rerunAssistant() to work with sessions
• Keep state.messages array (UI compatibility)

Key sections to modify:

// Add in init()
this.debouncedSaveSession = debounce(async () => {
  await saveSession(this);
}, 500);

// In applyJobDelta (around line 1591)
if (delta.content) {
  this.messages[idx].content += delta.content;
  this.debouncedSaveSession(); // ADD THIS
}

// In sendMessage (around line 1545)
// Replace individual message saves with debouncedSaveSession()

// In rerunFromUser (around line 1383)
// Replace individual message saves with debouncedSaveSession()

Purpose: Export and import functionality
Current: Complex ID remapping for normalized tables
Changes needed:

• Update exportAllData() to export sessions instead of chats+messages
• Simplify importDataReplace() - no ID remapping needed
• Simplify importDataAppend() - sessions are atomic
• Add exportSession(id) for single session export
• Add importSession(json) for single session import

Key functions to modify:

export async function exportAllData(state)
export async function importDataReplace(state, input)
export async function importDataAppend(state, input)
// Add these new functions:
export async function exportSession(state, id)
export async function importSession(state, json)

Purpose: Alpine.js initialization and presetsPanel
Current: Has its own Dexie schema (up to v7)
Changes needed:

• Update presetsPanel initDb() to include v11 for consistency
• Keep v10 definitions for backward compatibility
• No functional changes (presets are separate table)

Section to update (around line 31):

initDb() {
  // ... existing versions ...
  // Add v11 to match chatCore.js
  this.db.version(11).stores({
    sessions: '++id, title, slug, createdAt, folderId, updatedAt',
    chats: '++id, title, slug, createdAt, folderId, ...',
    messages: '++id, chatId, createdAt, role, content, model',
    folders: '++id, name, createdAt, order, collapsed',
    systemPrompts: '++id, name, content, systemPrompt, settings, createdAt, updatedAt',
    notes: '++id, title, content, createdAt, updatedAt'
  });
}

Purpose: Inline message editing
Current: Saves individual messages to Dexie
Changes needed:

• Replace db.messages.put() with saveSession()
• Ensure edits trigger debounced session save

Functions to update:

export async function commitEditingMessage(state)
// Replace db.messages.put() with saveSession()

✅ backend/ (entire directory) - Backend is stateless, no changes needed
✅ frontend/components/*.html - UI components use state.messages (unchanged)
✅ frontend/apiClient.js - API calls unchanged
✅ frontend/chatterLib.js - Helper functions unchanged
✅ frontend/contentRenderers.js - Rendering unchanged
✅ frontend/dragDrop.js - Drag/drop unchanged
✅ frontend/i18n.js - Internationalization unchanged
✅ frontend/preferences.js - Settings persistence unchanged
✅ frontend/presets.js - Presets system unchanged
✅ frontend/timerUtils.js - Timer utils unchanged
✅ frontend/traceUI.js - Trace UI unchanged
✅ frontend/uiTools.js - UI utilities unchanged

Purpose: Branch operations and management
Functions to implement:

export async function enableBranching(state, sessionId)
export async function forkAtMessage(state, sessionId, messageId)
export async function switchBranch(state, sessionId, branchId)
export async function archiveBranch(state, sessionId, branchId)
export async function renameBranch(state, sessionId, branchId, newName)
export function buildMessagePath(session, branchId)
export function validateNoCycles(session, newMessage)

Purpose: Branch dropdown UI component
Features:

• Show all branches for current session
• Highlight active branch
• Switch between branches
• Create new branch
• Rename/delete branches

Purpose: Visual tree view of branches
Features:

• SVG/Canvas tree rendering
• Click nodes to switch branches
• Zoom/pan controls
• Show branch names and dates

/Users/janzheng/localhost/smallweb/chatter/
├── backend/
│   ├── config.js                    ✅ No changes
│   ├── index.js                     ✅ No changes
│   ├── routes/
│   │   ├── chatStream.js            ✅ No changes
│   │   ├── chatStreamSSE.js         ✅ No changes
│   │   ├── jobs.js                  ✅ No changes
│   │   ├── search.js                ✅ No changes
│   │   ├── static.js                ✅ No changes
│   │   ├── styles.js                ✅ No changes
│   │   └── system.js                ✅ No changes
│   └── utils.js                     ✅ No changes
├── frontend/
│   ├── apiClient.js                 ✅ No changes
│   ├── chatterApp.js                ✏️ MODIFY - streaming & saves
│   ├── chatterLib.js                ✅ No changes
│   ├── components/
│   │   ├── appSettings.html         ✅ No changes
│   │   ├── chatControls.html        ✅ No changes
│   │   ├── chatWindow.html          ✅ No changes
│   │   ├── globalMenu.html          ✅ No changes
│   │   ├── sidebarDesktop.html      ✅ No changes
│   │   └── sidebarMobile.html       ✅ No changes
│   ├── contentRenderers.js          ✅ No changes
│   ├── core/
│   │   ├── chatCore.js              ✏️ MODIFY - schema & core ops
│   │   ├── jobManager.js            ✅ No changes
│   │   └── branchManager.js         📄 NEW (Phase 2)
│   ├── dataIO.js                    ✏️ MODIFY - export/import
│   ├── dragDrop.js                  ✅ No changes
│   ├── experimental/
│   │   └── streaming.js             ✅ No changes
│   ├── i18n.js                      ✅ No changes
│   ├── init.js                      ✏️ MINOR - schema sync
│   ├── messageEditing.js            ✏️ MODIFY - use sessions
│   ├── preferences.js               ✅ No changes
│   ├── presets.js                   ✅ No changes
│   ├── timerUtils.js                ✅ No changes
│   ├── traceUI.js                   ✅ No changes
│   └── uiTools.js                   ✅ No changes
├── BRANCH_DATA_MODEL.md             📖 This document
├── chatCompletion.js                ✅ No changes
├── chatStreaming.js                 ✅ No changes
├── index.html                       ✅ No changes
├── jsonUtils.js                     ✅ No changes
└── main.js                          ✅ No changes

Total estimated changes: ~800-1,000 lines of code across 5 files (Phase 1)
New code: ~300-500 lines (Phase 2)

Common issues:

• Migration stuck? Check browser console for errors
• Messages missing? Check session.tree.messages array
• Streaming not saving? Check debounce is working
• Can't import? Validate JSON structure matches schema

Debug tools:

// In browser console:
const db = new Dexie('chat_db');
await db.open();

// Check sessions
const sessions = await db.sessions.toArray();
console.log('Sessions:', sessions);

// Check a specific session
const session = await db.sessions.get(1);
console.log('Session tree:', session.tree);

// Check migration status
console.log('DB version:', db.verno);