Project
Tayari AI Course Media
I built a lesson media pipeline that generates narration through ElevenLabs or Google Text-to-Speech and creates clips from Veo scene prompts. FFmpeg concatenates the clips and merges the narration into the video.
How it’s built View architectureHow it’s built
Tayari AI Course Media · AI lesson media pipeline generating narration and multi-clip video for language courses
Tayari AI Course Media generates curriculum, narration and video for tutors building language courses on the Tayari marketplace. Inside the team's Express backend I wrote provider classes for ElevenLabs and Google text-to-speech and for Veo 2 video. The Veo provider renders up to 8 clips from distinct, culturally specific scene prompts, joins them with FFmpeg and merges the narration track. Seven instructor-only routes expose outline, lesson, exercise, audio, video, metadata and lecture generation.
- instructor-only AI generation routes
- 7
- maximum Veo clips per video
- 8
- TTS providers (ElevenLabs eleven_multilingual_v2, Google TTS)
- 2
- LLM providers behind one interface (Gemini, OpenAI, Claude)
- 3
- my commits to tayari_marketplace_v2 (96 files, +5,095 lines)
- 11
- lines of contentGenerationService.js authored by me
- 525 of 825
Technology and what it does here
- Veo 2 (veo-2.0-generate-001)
- Generates each lesson clip through the Gemini predictLongRunning API
- FFmpeg
- Concatenates clips and loops video under the narration with libx264
- ElevenLabs
- Multilingual v2 narration for lesson audio
- Google Text-to-Speech
- Alternative narration provider via google-tts-api
- Gemini / OpenAI / Claude
- Write outlines, lesson plans, exercises, scene prompts and refined video prompts
- Cloudinary
- Hosts generated video and audio buffers
- Express + Mongoose
- Instructor routes, course lookup for prompt context and AI cost logging
- React
- ContentGenerator dashboard and subsection modal that trigger generation
- Streamlit + Hugging Face (course-content-gen)
- Earlier prototype with flan-t5-base, NLLB-200 translation and Coqui YourTTS
Implementation details
- For multi-clip videos an LLM first writes exactly N distinct scene descriptions from the lecture audio so clips do not repeat, then each scene is rendered separately.
- Clips are joined with the FFmpeg concat demuxer and the result is merged with narration using -stream_loop -1 so video loops when the audio is longer.
- TTS providers share a BaseTTSProvider contract so ElevenLabs and Google can be swapped without changing callers.
System architecture
The course media workflow takes lesson content into speech synthesis and assembles generated clips with the resulting narration.
Technologies
- Node.js
- ElevenLabs
- Google Text-to-Speech
- Veo
- FFmpeg
