Hello Realtime is a OpenAI Realtime app that supports both WebRTC and SIP (telephone) users. You can access the app via WebRTC at hello-realtime.val.run, or via SIP by calling 425-800-0042.

This demo shows off the new SIP API, the new all-in-one WebRTC API, and the new server-side websocket interface.

If you remix the app, you'll just need to pop in your own OPENAI_API_KEY (from platform.openai.com), and if you want SIP, the OPENAI_SIGNING_SECRET.

1. WebRTC Flow:
   • Browser connects to frontend
   • creates WebRTC offer
   • /rtc endpoint handles SDP negotiation with OpenAI
   • observer established to monitor session
2. SIP Flow:
   • Phone calls triggers webhook
   • /sip endpoint verifies webhook and accepts call
   • observer established to monitor session

• GET / - serves the WebRTC-based app
• POST /rtc - creates a new WebRTC-based Realtime API session
  • Supports optional model URL parameter to override the default model
  • Example: POST /rtc?model=gpt-4o-realtime-preview
• POST /sip - handles an incoming Realtime API SIP call
• POST /observer - internal endpoint to establish a Realtime API websocket for session control

├── frontend/     
│   └── index.html          # HTML/JS application for WebRTC
├── main.tsx                # Main entrypoint for the val.town app
└── routes/
    |
    ├── agent.ts            # Realtime API session configuration and tools
    ├── observer.ts         # Server-side session control for SIP & WebRTC
    ├── rtc.ts              # WebRTC session setup
    ├── sip.ts              # SIP webhook handling and session setup
    └── utils.ts            # Various HTTP helpers

• frontend/index.html
• main.tsx
• routes/agent.ts
• routes/observer.ts
• routes/rtc.ts
• routes/sip.ts
• routes/utils.ts