Case study
An AI video platform, from an empty AWS account to production
Semantic search across thousands of TV-length videos: upload, transcode, enrich with AI models and index, built and run end to end as the founding engineer.
Simplified and generalized from my own work: no confidential details, internal names or non-public metrics. Every number matches my resume.
Results
- production videos searchable by meaning
- 3,500+
- cheaper reprocessing (~$3.00 → under $0.10 per video)
- 30x
- lower transcoding cost with optimized H.264 QVBR profiles
- 93%
- automated tests, 80%+ coverage
- 6,600+
The problem
- Customers upload very large files (up to 100 GB) that must never pass through our servers.
- Every video has to become searchable by meaning, which means several AI models per segment, each slow and quota-limited.
- Models and prompts change often, so reprocessing the library has to be cheap.
The architecture
Follow one video from the moment it is uploaded until it is searchable. Press play, or step through the stages at your own pace.
Decisions and trade-offs
Step Functions for work that mostly waits
The pipeline spends minutes waiting on models. Step Functions gives per-stage retries, a visible state for every video, and waiting costs nothing, unlike a worker billed while it sleeps.
Idempotent by design
Every search document is keyed by video and segment, so a re-run overwrites instead of duplicating. A video becomes searchable only after indexing succeeds.
Send each stage only the data it needs
Transcription first received the whole video and timed out on multi-GB files. Streaming the file through FFmpeg to extract only the audio turned a 50 GB video into about 300 MB.
Stay under the model quota
A retry storm against the model quota taught us to add a kill switch and a dispatch gate that caps concurrent pipelines at about 80% of the quota, with the queue holding the backlog.
Cache every stage
Stage-level caching means a model change only recomputes the stages it affects, which is what made reprocessing 30x cheaper.