hello-mcp

Hello MCP is a remix of OpenAI's hello-realtime demo that adds Model Context Protocol (MCP) support for server-side tool execution.

This demo showcases:

• WebRTC-based voice conversations with OpenAI's Realtime API
• MCP (Model Context Protocol) for secure server-side tool execution
• A demo getFavoriteFood tool to demonstrate MCP functionality
• Toggle between standard and MCP-enabled modes

If you remix the app, you'll just need to pop in your own OPENAI_API_KEY (from platform.openai.com).

MCP enables server-side tool execution, keeping your business logic and API keys secure on the server rather than exposing them to the client.

Try asking questions like:

• "What's Thomas's favorite food?"
• "What does Brian like to eat?"
• "Tell me about Yana's favorite food"

The tool knows about: Thomas, Brian, Emil, Yana, Alex, Max, Justin, and Sam.

1. Standard WebRTC Flow:

   • Browser connects to frontend
   • Creates WebRTC offer
   • /rtc endpoint handles SDP negotiation with OpenAI
   • Observer established to monitor session

2. MCP-Enhanced Flow (when enabled):

   • Same WebRTC setup as above
   • OpenAI discovers tools via MCP protocol (/mcp/agent/:agentId)
   • Tools execute server-side when invoked
   • Results returned through MCP protocol
   • Client triggers response generation after tool completion

• GET / - serves the WebRTC-based app with MCP toggle
• POST /rtc - creates a new WebRTC-based Realtime API session (with optional ?mcp=true)
• POST /sip - handles an incoming Realtime API SIP call
• POST /observer - internal endpoint to establish a Realtime API websocket for session control
• POST /mcp/agent/:agentId - MCP protocol endpoint for tool discovery and execution
• GET /mcp/agent/:agentId/tools - Debug endpoint to list available tools

├── frontend/     
│   └── index.html          # HTML/JS application for WebRTC with MCP support
├── main.tsx                # Main entrypoint for the val.town app
└── routes/
    |── mcp.ts              # MCP protocol handler with getFavoriteFood tool
    |── observer.ts         # Server-side session control for SIP & WebRTC
    |── rtc.ts              # WebRTC session setup with MCP configuration
    |── sip.ts              # SIP webhook handling and session setup
    └── utils.ts            # Realtime API session configuration with MCP support

1. Start the app - Click "Start" and allow microphone access
2. Check MCP is enabled - Ensure "Use MCP Tools" checkbox is checked
3. Ask about favorite foods - Try "What's Thomas's favorite food?"
4. Watch the logs - You'll see MCP events including tool discovery and execution
5. Try without MCP - Uncheck the box and restart to see the difference

The MCP implementation follows the JSON-RPC 2.0 specification:

{
  "method": "initialize",
  "params": { "protocolVersion": "2025-06-18" }
}

{
  "method": "tools/list",
  "result": { "tools": [...] }
}

{
  "method": "tools/call",
  "params": {
    "name": "getFavoriteFood",
    "arguments": { "name": "Thomas" }
  }
}