The State of Real-Time Translation in 2026

We pushed our production system until something gave, and wrote down every number. This is what live translation can hold today, how it grows when the room fills up, and what we still have to prove.

  • 300live sessions held at once by a single small server
  • 18,086translated captions in a 26-minute production ramp, zero failed starts
  • 3 to 8servers the cluster added for itself during that ramp, zero restarts
  • 200/secaudience questions accepted and translated, zero errors
Four numbers from our 2026 load tests.

For most of the last decade, live translation was a demo. It worked on stage, with one speaker, a good microphone and a patient audience. Then someone said a product name, or four hundred people joined at once, and the demo was over.

That has changed. In 2026 a person can speak in one language while a room reads along in another, about as quickly as a human interpreter could say it. The interesting question is no longer whether it works. It is how much of it you can run at once, and what happens at the edge.

So we went looking for the edge. In late summer we ran a series of load tests against our own system, adding traffic step by step until sessions started to fail. This post is the result: the numbers, the charts, and the two things that broke.

How a sentence becomes a caption

Every caption makes the same trip. Audio streams in from a meeting, a webinar or a phone. Speech recognition turns it into text while the person is still talking. An assembler decides where one sentence ends and the next begins, which matters more than it sounds: translate too early and you get half a thought, too late and the reader falls behind. The sentence is translated, and the result is pushed to every reader's screen at once.

  1. Audio inMeeting, webinar or phone microphone
  2. Speech to textWords appear as they are spoken
  3. Sentence assemblyFinds where one thought ends
  4. Translation raceThree attempts at once, first good answer wins
  5. Captions outPushed to every reader's screen
speechtarget: under 2 seconds
The five stages every caption passes through. Our design target for the whole trip is two seconds; we have not yet measured it end to end.

The translation step is the one we watch most closely. Across 90 days of production traffic, a single translation call took a median of 1,005 milliseconds. That is comfortable. The trouble is the tail: one call in ten took longer than 5.5 seconds. A reader does not experience a median. They experience the one sentence that hung.

So we do not make one call. For every finished sentence we start three at once: the main model, a second attempt at the same model, and a model from a different provider. The first good answer goes on screen and the other two are cancelled.

One sentence, three attempts
Main modelcancelled
Same model, againshown
Different providercancelled
Illustration, not to scale. Whichever attempt returns a usable translation first is the one readers see.

This costs us 2.96 times as much per sentence. We think a caption that arrives on time is worth it.

While the speaker is still mid-sentence, readers see interim captions: a fast draft that grows as the words arrive and is replaced by the final version when the sentence ends. The model behind those drafts answers in a median of 983 milliseconds.

How much one server holds

We started small on purpose. One server with a single CPU core and one gigabyte of memory, and sessions added ten at a time, each level held for five minutes. We kept going until the same level failed twice.

Live sessions held by one server1 CPU, 1 GB memory, five-minute hold at each level. September 2026.
SetupSessions held
1 language, interim captions on300
3 languages, interim captions on190
6 languages, interim captions on80
6 languages, final captions only290
Each bar is the last level that passed. The next level up (310, 200, 90 and 300 respectively) is where it failed.

Two things stand out. A single small server carried 300 simultaneous live sessions before the 310th tipped it over. And the cost of a session is mostly about how much text you ask for: every extra language, and every interim update, is another translation. Six languages with interim captions is the heaviest thing a host can request, and one server still held 80 of those at once, delivering 40,927 finished captions without a single language stream going missing.

We do not run servers at the number where they passed. Production admits sessions at 20 percent below the last passing level, so a measured 300 becomes a working limit of 240. When a server is full, the next session goes to another one.

Then we did it in production

A lab number tells you about a server. It says nothing about the system around it: the database, the cache, the service that pushes messages to screens, the speech provider. So on August 30 we ran the whole path in production. Forty live sessions to start, two more every 45 seconds, no upper limit, stop when something fails.

Live sessions during the production rampAug 30, 2026 · 26 minutes · full production path
  • Every session started and stayed healthy
  • 104: one session received no captions, ramp stopped
StepLive sessions
140
248
356
464
572
680
788
896
9104
Every fourth step shown. The test added two sessions every 45 seconds with no target ceiling.

Twenty-six minutes later we were at 104 sessions. Along the way the system delivered 18,086 translated captions. Not one session failed to start, and not one was marked unhealthy, at any step.

  • Translation servers3 to 8, all ready, zero restarts
  • Database35,632 requests, slowest 16 ms, none throttled
  • CachePeak load 70%, nothing evicted
  • Message delivery74,655 messages, zero errors under load
How the rest of the system held up during the production ramp.

What stopped the ramp was not our code. At 100 sessions we reached the concurrency quota on our speech recognition account: exactly 100 simultaneous streams. At 104, one session connected and heard nothing back. A quota is a line in a contract, and raising it is a conversation, not a rewrite. But finding it in a test is far better than finding it during your all-hands.

What happens when the room fills up

Nobody sizes a system for its busiest hour and leaves it running that way. Ours grows and shrinks by itself, and the two directions follow very different rules.

Servers during the rampServers in the cluster during the ramp: 3, 5, 7, 8.
  • Growing is fastTwo more servers every minute, after a 30-second check that the load is real.
  • Shrinking is slowOne server at a time, at most every two minutes, only after five quiet minutes.
  • Leaving is politeA departing server takes no new sessions and waits up to two hours for its live meetings to end.
The cluster went from three servers to eight during the August 30 ramp without anyone touching it.

That last rule is the one we care about most. Saving the cost of a server is never a reason to cut someone off mid-sentence, so a server that is asked to leave finishes the meetings it is already hosting first.

The pieces around the translation servers scale the same way. The tier that handles sign-in, sessions and questions runs two to three instances, and the service that pushes captions to screens runs between two and ten units depending on load.

Live Q&A: when everyone asks at once

Translation load arrives steadily. Questions do not. A speaker says "any questions?" and hundreds of people type at once, in a dozen languages, and every question has to be stored, translated and put in front of the moderator.

We tested exactly that burst. Across 2,000 separate sessions we sent 45,600 questions in total, in thirty-second bursts of increasing size.

Time to accept a question, by burst size95th percentile, in milliseconds. 2,000 sessions, 30-second bursts. Aug 30, 2026.
Questions per second95th percentile, ms
10234
20479
40348
80673
160796
2001,350
2201,318
2406,679
The last bar is cut short: at 240 questions per second the wait jumped to 6.7 seconds, more than three times our bar.

At 200 questions per second, all 6,000 questions in the burst were accepted and all 6,000 were translated, with zero errors, and 95 percent of them were accepted within 1.35 seconds. At 220 per second the system still cleared our bar: fewer than 1 percent errors, and 95 percent of questions accepted within two seconds. At 230 it did not, so that is where we stopped calling it good.

These are August 2026 measurements. Since then we have tightened per-client rate limits to protect events from abuse. We will publish fresh numbers once we have re-measured under the new rules.

What broke

Load tests exist to find problems. These found two worth telling you about.

A shared key

In an early Q&A run, 8 of 384 questions were accepted but never translated. The cause was small and a little embarrassing: several translators shared one connection and, under load, briefly overwrote each other's credentials. We moved the credential onto each individual request, added a test that hammers the shared connection from many directions at once, and ran it again. 384 of 384.

A stampede at the exit

When the production ramp ended, the test closed all 104 sessions in the same instant. The burst of goodbye messages overloaded the delivery service, and in the confusion some speech streams were not released promptly. The service stayed healthy throughout. But real events end this way too: a webinar finishes and everyone leaves at once. It went to the top of our list.

What we are not claiming

There is no single "maximum users" number in this post, and that is on purpose. Capacity depends on how many languages you ask for and whether interim captions are on, as the single-server chart shows. Our long-duration soak tests are not finished. And we have measured each stage of the two-second trip more carefully than the trip as a whole; that end-to-end measurement is next.

When those are done we will publish them here, including the ones that come out worse than we hoped.

What this means for you

If you run a weekly team meeting in two languages, none of this will ever matter to you, which is the point. The system has room to spare and you will never see it working.

If you are planning something larger, the lesson from our own tests applies to you as well: find the edge before the day, not on it. Tell us what you are planning and we will tell you plainly what we have measured.