“Real-time” needed a definition
LiveCap turns spoken Vietnamese and English into bilingual captions. The first useful demo was simple: speak, watch text appear, then export the transcript. Once that worked, I needed a better answer to a harder question: how long does someone actually wait?
I could point at the interface and call it fast. That would say very little about which part of the system was fast, what I measured, or whether another person could reproduce the result.
The useful change was treating latency as several questions instead of one number.
The first words and the final caption are different events
Partial text gives someone early feedback that the application is listening. A finalized caption is the text the system has committed to. Translation adds another operation, with its own timing boundary.
For the recorded-audio benchmark, I kept three measurements separate:
- First partial: from the first PCM sample sent by the client to the first partial result received. This includes leading silence in the recording.
- Finalized-caption lag: from the client frame-send time corresponding to a provider segment's audio end to receipt of the finalized caption.
- Translation operation: the backend operation around translation, including SDK, network, and executor work. This is not a measurement of AWS processing time alone.
These are different clocks around different events. I cannot add their medians together and call the result total latency. I also cannot take a backend send completing as proof that text has already been painted on the screen.
That distinction matters because a customer sees an interaction, while the backend sees operations. A benchmark needs to say which one it observed.
A small experiment with a repeatable input
The benchmark ran on October 3 in an isolated Singapore environment with one ECS Fargate task: 0.5 vCPU and 1 GiB of memory. Sessions used Cognito authentication, dual English/Vietnamese Transcribe streams, and Vietnamese translation.
The client replayed the same 33.623-second Harvard speech sample from the Open Speech Repository, resampled from 8 kHz to 16 kHz. Each session sent the full recording. There were three measured runs at each concurrency level: one, two, and four sessions.
That produced nine runs and 21 attempted sessions. There were no generated-speech sessions, warmup sessions, or session retries.
Using a recording gave me a repeatable input. It also limited the claim: this was one short speech fixture, not a sample of real meetings with different speakers, microphones, and background noise.
What the results actually showed
| Concurrent sessions | Completed | First partial | Final caption lag | Translation |
|---|---|---|---|---|
| 1 | 3/3 | 1.676 s | 2.043 s | 35.51 ms |
| 2 | 6/6 | 1.686 s | 2.064 s | 95.50 ms |
| 4 | 12/12 | 1.686 s | 2.125 s | 93.95 ms |
All 21 sessions completed, with no unexpected disconnects observed in this sample. At four concurrent sessions, first-partial p50 was about 1.69 seconds and finalized-caption lag p50 was about 2.13 seconds.
The sample counts are easy to misread. The first-partial medians use 3, 6, and 12 session observations. Finalized-caption lag and translation use 30, 60, and 120 segment or call observations. Several segments come from the same session, so they are correlated. They do not become 120 independent users just because they produce 120 rows.
The experiment also does not establish a stable p95 or p99. Reporting a precise tail percentile from this small sample would make the result look stronger than the evidence.
The run did not follow every original condition
The original plan called for one continuously warm task across all nine runs. That did not happen. The coordinator was interrupted after run two. The task restarted before run three, then again before run seven after collector credentials expired.
Every measured session began after readiness, but the campaign covered three warm task epochs. I kept that deviation in the report because it changes how the result should be interpreted. It is evidence from warm sessions, not a cold-start measurement or one uninterrupted stability run.
The different concurrency stages are not a before-and-after optimization study either. A change between those rows does not establish that an implementation change improved performance.
Checking the current interface is a separate job
I opened the public application in Google Chrome again on October 25 and checked the authenticated workspace. The current interface exposes session status, audio-source controls, caption output, and transcript export. It also has an audience-room flow for sharing finalized captions.
The supplied screenshot shows Vietnamese text and an English translation from the browser test. It supports a narrower claim than the benchmark: bilingual output was visible in the interface. It does not establish timing, recognition accuracy across other speech, or successful transcript export.
Screenshots help explain what someone uses. They cannot establish transcription accuracy or caption latency. Keeping those kinds of evidence separate makes both more useful.
What I would measure next
This experiment does not measure word error rate, cold starts, maximum sustained capacity, production reliability, or incremental billed cost. A successful short recording does not answer those questions.
The next useful work is to vary speech fixtures and session duration, measure cold starts separately, and collect enough independent sessions to examine tail behavior. Accuracy needs its own reference transcripts and scoring, rather than being inferred from a session completing successfully.
LiveCap taught me that a performance number becomes useful when I can explain its start event, end event, sample, and limits. That explanation is part of the engineering work. Without it, “real-time” is just a label on the landing page.
The benchmark report in the LiveCap repository records the method and measurement boundaries. The speech fixture comes from the Open Speech Repository's American English recordings.