Skip to content

Tayari AI Course Media

I built a lesson media pipeline that generates narration through ElevenLabs or Google Text-to-Speech and creates clips from Veo scene prompts. FFmpeg concatenates the clips and merges the narration into the video.

How it’s built View architecture

How it’s built

Tayari AI Course Media · AI lesson media pipeline generating narration and multi-clip video for language courses

Tayari AI Course Media generates curriculum, narration and video for tutors building language courses on the Tayari marketplace. Inside the team's Express backend I wrote provider classes for ElevenLabs and Google text-to-speech and for Veo 2 video. The Veo provider renders up to 8 clips from distinct, culturally specific scene prompts, joins them with FFmpeg and merges the narration track. Seven instructor-only routes expose outline, lesson, exercise, audio, video, metadata and lecture generation.

instructor-only AI generation routes
7
maximum Veo clips per video
8
TTS providers (ElevenLabs eleven_multilingual_v2, Google TTS)
2
LLM providers behind one interface (Gemini, OpenAI, Claude)
3
my commits to tayari_marketplace_v2 (96 files, +5,095 lines)
11
lines of contentGenerationService.js authored by me
525 of 825

Technology and what it does here

Veo 2 (veo-2.0-generate-001)
Generates each lesson clip through the Gemini predictLongRunning API
FFmpeg
Concatenates clips and loops video under the narration with libx264
ElevenLabs
Multilingual v2 narration for lesson audio
Google Text-to-Speech
Alternative narration provider via google-tts-api
Gemini / OpenAI / Claude
Write outlines, lesson plans, exercises, scene prompts and refined video prompts
Cloudinary
Hosts generated video and audio buffers
Express + Mongoose
Instructor routes, course lookup for prompt context and AI cost logging
React
ContentGenerator dashboard and subsection modal that trigger generation
Streamlit + Hugging Face (course-content-gen)
Earlier prototype with flan-t5-base, NLLB-200 translation and Coqui YourTTS

Implementation details

  • For multi-clip videos an LLM first writes exactly N distinct scene descriptions from the lecture audio so clips do not repeat, then each scene is rendered separately.
  • Clips are joined with the FFmpeg concat demuxer and the result is merged with narration using -stream_loop -1 so video loops when the audio is longer.
  • TTS providers share a BaseTTSProvider contract so ElevenLabs and Google can be swapped without changing callers.

System architecture

The course media workflow takes lesson content into speech synthesis and assembles generated clips with the resulting narration.

Drag to rotate

Lesson narration

  1. Passes to the next component: Lesson script

Video assembly

  1. Passes to the next component: Generated clips

System connections

Component details

Technologies

  • Node.js
  • ElevenLabs
  • Google Text-to-Speech
  • Veo
  • FFmpeg

Next project · Quant Trader

Sigmant

Portfolio assistant

Gabriel’s projects, experience, and research