Obsidian Inbox - LLM Context Guide

A Val.town serverless service providing a temporary inbox for content before Obsidian vault processing. This is a buffer layer between input sources (email/HTTP) and processing clients (MCP/scripts).

Key Principle: Simple, stateless inbox. Complex processing happens in clients, not here.

Critical Context:

• Runtime: Deno (not Node.js)
• No filesystem access - use SQLite/blob storage only
• Import npm packages via https://esm.sh/package
• Use Val.town stdlib via https://esm.town/v/std/library
• Serverless execution model - no long-running processes

Standard Library Available:

• sqlite - SQLite database
• blob - Binary blob storage
• email - Send/receive emails
• Vals are exposed as HTTP endpoints, email handlers, or cron jobs

Input Sources → Handlers → Services → Storage → Clients
    ↓              ↓          ↓         ↓         ↓
  Email        emailHandler InboxService SQLite  MCP Server
  HTTP API  →  httpHandler  AttachService Blob   Custom Scripts

Handlers (entry points for Val.town):

• emailHandler.ts - Email trigger val, processes incoming emails
• httpHandler.ts - HTTP val, exports app.fetch for API endpoints
• cleanupCron.ts - Cron val, runs daily to delete old processed records

Services (data layer):

• server/InboxService.ts - CRUD operations on inbox records in SQLite
• server/AttachmentService.ts - Store/retrieve binary files in blob storage
• server/auth.ts - Bearer token auth & email sender allowlist

Core App:

• server/app.ts - Hono HTTP application with all routes
• server/db.ts - Drizzle ORM schema definition

Client Integration:

• mcp/server.ts - MCP server setup (runs locally, not on Val.town)
• mcp/tools/ - MCP tool implementations
• apiClient.ts - HTTP client using Hono's RPC client
• localMcpServer.ts - Entry point for local MCP server

Location: server/InboxService.ts

{
  raw: string,                    // Full text content (email body, etc.)
  summary?: string,               // Optional summary
  from?: string,                  // Source identifier (email, "manual", etc.)
  subject?: string,               // Subject line or title
  date?: Date,                    // When received (coerced to Date)
  attachmentBlobIds: string[]     // Array of blob storage keys
}

Validation: Use inboxRecordSchema (Zod) for validation Middleware: validateInboxRecord for Hono routes

Table: inbox_records (defined in server/db.ts)

CREATE TABLE inbox_records (
  id TEXT PRIMARY KEY,              -- UUID
  record TEXT NOT NULL,             -- JSON-serialized InboxRecord
  created_at TEXT DEFAULT current_timestamp,
  processed_at TEXT                 -- NULL until marked processed
);

ORM: Drizzle ORM with LibSQL driver Access: Via db export from server/db.ts

Keys:

• {uuid} - Binary file data (ArrayBuffer)
• {uuid}:meta - JSON metadata: {filename, contentType, size}

Interface: StoredAttachment in server/AttachmentService.ts

All routes except GET / require Bearer token authentication via bearerAuthMiddleware.

1. Ingestion: Item arrives via email or HTTP POST

   • Email: emailHandler.ts → validates sender → saves
   • HTTP: POST /api/record → validates schema → saves

2. Storage:

   • InboxService.save() creates UUID, saves JSON to SQLite
   • AttachmentService.save() stores binary + metadata to blob
   • processed_at is NULL

3. Retrieval:

   • Client calls GET /api/inbox → returns items where processed_at IS NULL
   • Client gets attachment metadata → GET /api/record/:id/attachments
   • Client downloads files → GET /api/record/:id/attachments/:blobId

4. Processing:

   • Client transforms item (e.g., creates Obsidian notes)
   • Outside this service - client is responsible

5. Completion:

   • Client calls PATCH /api/record/:id/processed
   • Sets processed_at = current_timestamp

6. Cleanup:

   • Cron runs daily via cleanupCron.ts
   • Deletes records where processed_at <= date('now', '-30 days')
   • Deletes associated blobs via AttachmentService.delete()

Required in Val.town secrets:

• INBOX_API_TOKEN - Bearer token for HTTP API authentication
• INCOMING_EMAIL_ALLOWLIST - Comma-separated email addresses

Required for MCP client (local):

• INBOX_API_URL - URL to deployed httpHandler val
• INBOX_API_TOKEN - Same token as above

Location: server/auth.ts

Middleware: bearerAuthMiddleware

• Checks Authorization: Bearer {token} header
• Compares with INBOX_API_TOKEN env var
• Returns 401 if missing/invalid

Location: server/auth.ts

Function: isAllowedSender(email: Email)

• Extracts email address from From: header
• Checks against INCOMING_EMAIL_ALLOWLIST env var
• Logs rejection if not allowed
• Returns boolean

Steps:

1. Update inboxRecordSchema in server/InboxService.ts
2. Change table name in server/db.ts:

   // DON'T use ALTER TABLE - Val.town limitations
   export const inboxRecords = sqliteTable("inbox_records_2", { ... });

3. Update handlers if field usage changes
4. Redeploy vals

Rationale: Changing table name avoids migration complexity in serverless env.

Steps:

1. Add route in server/app.ts:

   .get("/api/new-endpoint", bearerAuthMiddleware, async (c) => {
     // handler logic
   })

2. Add auth middleware if needed (usually yes)
3. Return proper status codes (200/201/400/401/404/500)
4. Update README.md API reference

Steps:

1. Create mcp/tools/newTool.ts:

   import type { McpServer } from "npm:@modelcontextprotocol/sdk/server/mcp.js";
   import type { Client } from "../../apiClient.ts";
   
   export function registerNewTool(server: McpServer, client: Client) {
     server.tool("tool-name", "description", { /* schema */ }, async (params) => {
       // Call client API
       const response = await client.api.endpoint.$get();
       // Return result
     });
   }

2. Register in mcp/tools/index.ts:

   import { registerNewTool } from "./newTool.ts";
   
   export function registerAllTools(server: McpServer, client: Client) {
     // ... existing
     registerNewTool(server, client);
   }

3. Test with Claude Desktop

Steps:

1. Create new handler file (e.g., webhookHandler.ts)
2. Create new val in Val.town with appropriate trigger type
3. Transform input to InboxRecord format
4. Call InboxService.save(record)
5. Handle attachments via AttachmentService.save(file)

Principle: Fail-fast with context

• Don't catch unless you can handle locally - let errors bubble
• Use Zod validation for input schemas → automatic error messages
• Return descriptive errors in HTTP responses
• Log before rejecting (e.g., email sender not allowlisted)

Example:

// DON'T
try {
  const record = await InboxService.save(data);
} catch (err) {
  console.error(err);
  throw err; // Useless catch
}

// DO
const record = await InboxService.save(data); // Let it throw

• TypeScript with explicit types on public interfaces
• Functional style preferred over classes
  • Services are const objects with methods
  • Pure functions where possible
• Small, focused functions - single responsibility
• No try-catch unless local resolution exists
• Zod schemas for all external input

No automated tests - manual testing via Val.town web interface

Email Testing:

1. Send test email from allowlisted address
2. Check Val.town logs for processing
3. Verify via GET /api/inbox

HTTP Testing:

# Create record
curl -X POST https://your-val.web.val.run/api/record \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"raw": "test", "attachmentBlobIds": []}'

# List unprocessed
curl https://your-val.web.val.run/api/inbox \
  -H "Authorization: Bearer TOKEN"

# Mark processed
curl -X PATCH https://your-val.web.val.run/api/record/{id}/processed \
  -H "Authorization: Bearer TOKEN"

MCP Testing:

1. Configure Claude Desktop with local MCP server
2. Ask Claude to interact with inbox
3. Check Claude Desktop logs: ~/Library/Logs/Claude/mcp*.log

Focus Areas:

• Malformed input (invalid JSON, missing fields)
• Auth failures (wrong token, unlisted sender)
• Missing data (nonexistent record IDs, blob keys)
• Edge cases (empty attachments, special characters)

Current Limitations:

• No retry logic for failed processing
• No notification when new items arrive (clients must poll)
• 30-day retention is hardcoded
• No bulk operations (must process one-by-one)

Potential Improvements:

• Push notifications / webhooks when new items arrive
• Configurable retention period
• Bulk mark as processed endpoint
• Attachment size limits
• Rate limiting on HTTP endpoints

Initial Setup:

1. Create Val.town account
2. Set secrets: INBOX_API_TOKEN, INCOMING_EMAIL_ALLOWLIST
3. Deploy emailHandler.ts as email trigger val
4. Deploy httpHandler.ts as HTTP val
5. Deploy cleanupCron.ts as cron val (daily schedule)
6. Note HTTP val URL for client configuration

Local MCP Setup:

1. Install Deno
2. Create .env with INBOX_API_URL and INBOX_API_TOKEN
3. Configure Claude Desktop (see README.md)
4. Restart Claude Desktop
5. Test: Ask Claude "What's in my inbox?"

Verification:

1. Send test email → check inbox appears
2. Call HTTP API → verify auth works
3. Use MCP tools → verify connection
4. Wait 24h → verify cron runs

Redirects:

• Don't use Response.redirect() - doesn't work
• Use: new Response(null, {status: 302, headers: {Location: url}})

Imports:

• Always use https://esm.sh/ for npm packages
• Val.town stdlib: https://esm.town/v/std/library
• Can reference other vals: https://esm.town/v/username/valname

State:

• No persistent filesystem - use SQLite or blob storage
• No global state across invocations - serverless
• Use Val.town's sqlite and blob from stdlib

Debugging:

• Check Val.town logs in web interface
• console.log() appears in val logs
• Errors are captured and displayed

MCP Server runs locally, calls remote API:

• Uses apiClient.ts with Hono RPC client
• Type-safe calls via hc<AppType>(url)
• Tools map 1:1 to API endpoints

Available Tools:

• inbox_read → GET /api/inbox
• inbox_add_record → POST /api/record
• inbox_mark_processed → PATCH /api/record/:id/processed
• inbox_get_attachments → GET /api/record/:id/attachments
• inbox_download_attachment → GET /api/record/:id/attachments/:blobId

Direct HTTP API:

• All endpoints require Bearer token
• Use any HTTP client
• Parse JSON responses
• Handle 401/404/500 errors

Workflow:

1. Poll /api/inbox for new items
2. Process items (your logic)
3. Mark each as processed
4. Handle idempotency (items may reappear if processing fails)

Why separate MCP server from Val.town?

• Val.town vals are HTTP/email/cron only
• MCP requires stdio transport (local process)
• Separation keeps concerns clear: remote storage, local processing

Why not store processed items separately?

• Simple lifecycle: unprocessed → processed → deleted
• SQLite index on processed_at is efficient
• No need for complex archival

Why 30-day retention?

• Balance between "oops, need to reprocess" and storage costs
• Processed items should be in Obsidian by then
• Can be changed if needed (see InboxService.hardDeleteProcessedRecords)

Why JSON-serialized records in SQLite?

• Schema flexibility without migrations
• Val.town SQLite is lightweight - not for complex queries
• Simple key-value pattern matches use case

Why Hono instead of native Deno HTTP?

• Type-safe RPC client via hc<AppType>
• Middleware support (auth)
• Familiar Express-like API
• Small bundle size for serverless

1. server/InboxService.ts:

   export const inboxRecordSchema = z.object({
     // ... existing fields
     newField: z.string().optional(),
   });

2. Update handlers if needed (emailHandler.ts, etc.)

3. Redeploy vals

server/InboxService.ts in hardDeleteProcessedRecords:

sql`${inboxRecords.processed_at} <= date('now', '-60 days')` // Change 30 to 60

See "Adding MCP Tools" section above.

# Check Claude Desktop logs
tail -f ~/Library/Logs/Claude/mcp*.log

# Test MCP server directly
deno run --allow-net --allow-env localMcpServer.ts
# Then send MCP protocol messages via stdin

Call from Val.town REPL or add endpoint:

await InboxService.hardDeleteProcessedRecords();

When working on this codebase:

1. Consult this file first for architecture
2. Consult CLAUDE.md for AI assistant specific guidelines
3. Consult README.md for user-facing documentation
4. Remember: This is a simple buffer, not a full application
5. Val.town constraints: Deno runtime, no filesystem, serverless
6. Keep services pure - handlers contain business logic
7. Let errors bubble - don't catch unless you can handle
8. Test manually - no test suite, use Val.town interface

Most common tasks:

• Schema changes → update Zod schema, change table name
• New endpoints → add to server/app.ts with auth
• New MCP tools → create in mcp/tools/, register in index.ts
• Debug → check Val.town logs, Claude Desktop logs