Data teams designing their first generative media pipeline usually copy the architecture they know from LLM applications: synchronous request, spinner, response. Then they add a video model, the p95 hits seven minutes, and the architecture collapses. The runtime data explains why, and it is worth looking at before you write a line of pipeline code.
The dataset
At each::labs we operate a production catalog of 475 generative media model endpoints spanning video (230 endpoints), image (108), audio (17), and utility models, with 456 reporting live median runtimes from real traffic. Because every job runs through the same infrastructure, the medians are comparable across models in a way vendor-published numbers are not. The numbers below are observed medians from production workloads, not lab measurements.
Video is not “slow images.” It is a different regime.
The median image endpoint completes in 20 seconds. The median video endpoint completes in 120 seconds. That six-fold gap understates the real difference, because the distributions have different shapes.
For image endpoints, the interquartile range runs from 10 to 46 seconds: tight enough that a single timeout policy and a simple retry rule cover most of the catalog. For video, the interquartile range runs from 64 to 205 seconds, with the slowest endpoints reaching 500 seconds for a single generation. Only about 25 percent of video endpoints have a median under one minute, and a full third sit above three minutes.
Audio, on the smaller sample we have (17 endpoints), medians around 12 seconds and behaves architecturally like image.
The engineering consequence is blunt: any user-facing flow built on synchronous video generation is designing for the fastest quartile and will fail on the rest of the catalog. Video workloads belong on asynchronous infrastructure by default: job queues, webhooks over polling where available, and user experiences designed around “we will notify you” rather than a progress bar that lies.
Runtime spread is the hidden cost lever
The gap between the fastest and slowest video endpoint in the catalog spans two orders of magnitude. Nothing else in your pipeline, not batching, not caching, not infrastructure tuning, moves latency and throughput as much as which model a job lands on.
This reframes model selection as an economics problem. If a 70-second endpoint produces output your use case accepts at a similar rate to a 205-second endpoint, routing to it nearly triples your throughput per worker and cuts the wall-clock cost of every retry. Multiply by the retry behavior of generative work, where an accepted output routinely takes more than one attempt, and routing policy starts to dominate the cost line.
A practical routing rule we see hold up in production: classify jobs by acceptance sensitivity. Draft-quality and high-volume jobs go to fast endpoints; hero assets that justify long tails go to slow, high-fidelity endpoints. Pipelines that route every job to the “best” model pay the 205-second tax on work that never needed it.
Designing timeout and retry budgets from percentiles
Three data-driven rules translate the distribution into configuration:
- Set per-endpoint timeouts from that endpoint’s tail, not a global constant. A 90-second global timeout kills a third of legitimate video jobs in this catalog while being ten times too generous for the median image job.
- Budget retries in wall-clock, not attempt counts. Three retries on a 205-second endpoint is a ten-minute worst case per asset. Whether that is acceptable is a product decision, and it should be an explicit one.
- Fan out where acceptance is probabilistic. For workloads where one in several outputs is accepted, generating two or three candidates in parallel on a fast endpoint often beats sequential retries on a slow one, on both latency and cost per accepted output.
Where to explore the data
Per-endpoint median runtimes are published on every model page in the each::labs catalog, browsable through the AI models API, which also exposes the same metadata programmatically for teams that want to build routing logic on top of it. The broader lesson holds regardless of platform: in generative media, latency is not an infrastructure property. It is a model property, and the catalog is the map.
FAQ
Why are median runtimes better than vendor benchmark times?
Vendor numbers describe ideal conditions. Production medians include queueing, load variance, and provider degradation, which is what your users actually experience.
Should video generation ever be synchronous?
Only when the specific endpoint’s tail, not its median, fits inside your request budget. In this catalog that describes roughly the fastest quartile of video endpoints, and it should be a per-endpoint decision.
How does model routing reduce generative media costs?
Runtime spreads across equivalent-task models span orders of magnitude. Sending each job to the cheapest endpoint that meets its acceptance bar reduces both compute time and the wall-clock cost of retries, which is where most generative budgets actually leak.
Bartu Varol works at each::labs, the production platform for AI media running 475 image, video, and audio model endpoints with continuously measured runtime data.