Log inSign up
decepulis
decepulisMuxAITranscript
Public
Like
MuxAITranscript
Code
2
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in miliseconds.
Sign up now
Code
/
Search
Open in new tab
README.md

Generate CuePoints and transcripts for your Mux video

Adapted from the blog post, Build an AI-powered interactive video transcript with Mux Player CuePoints

This Val exposes an HTTP endpoint that takes a Mux Asset ID and a list of speakers and

  1. Uses Mux's auto-generated captions to generate a CuePoints object for Mux Player
  2. Uses AssemblyAI for speaker labeling (diarization)
  3. Uses GPT-4o to format text

Fork it and use it as a foundation for your own interactive video transcript project.

Usage:

Required environment variables:

  • Mux Access token details (MUX_TOKEN_ID, MUX_TOKEN_SECRET) This endpoint requires an existing Mux asset that's ready with an audio-only static rendition associated with it. You can run this val to create a new one for testing.
  • AssemblyAI API key (ASSEMBLYAI_API_KEY). Get it from their dashboard here
  • OpenAI API key (OPENAI_API_KEY). Get it from their dashboard here

Make a POST request to the Val's endpoint with the following body, replacing the values with your own asset ID and the list of speakers. Speakers are listed in order of appearance.

{
  "asset_id": "00OZ8VnQ01wDNQDdI8Qw3kf01FkGTtkMq2CW901ltq64Jyc",
  "speakers": ["Matt", "Nick"]
}

Limitations

This is just a demo, so it's obviously not battle hardened. The biggest issue is that it does this whole process synchronously, so if the any step takes longer than the Val's timeout, you're hosed.

Code