Tencent Hunyuan Open-Sources AI Simultaneous Interpretation Model with 3-Second Latency and Real-Time Voice Cloning
Tencent Hunyuan has open-sourced Hy-SimulMT, a Chinese-English bidirectional real-time simultaneous interpretation model with about 3-second latency and voice cloning support. In evaluations, its latency was the lowest among the compared systems; its translation quality surpassed the compared open-source models and 4 of 5 commercial APIs, and was comparable to professional CATTI interpreters. Deployment instructions and demo videos are included.
On October 26, Tencent Hunyuan open-sourced Hy-SimulMT, a real-time Chinese-English bidirectional simultaneous interpretation model. There are three core points: about 3-second latency, voice cloning while translating, and open weight downloads.
What does a 3-second latency mean?
The hard part of simultaneous interpretation is “waiting or rushing.” Wait a little longer, and the translation is more complete, but listeners have to wait uncomfortably; rush to speak, and it is faster, but more error-prone. Hy-SimulMT's strategy is to listen and translate at the same time, continuously producing translations as the information stream arrives.
In the evaluation, the number 3 seconds is defined as StreamLAAL, which measures the lag time of streaming translation: after a speaker finishes a segment, how long does the model need to speak the corresponding translation? The lower the value, the less waiting listeners feel. It is not equivalent to “the time to hear the first word.”
In the open-source RealSI long-speech evaluation, the official results are: Hy-SimulMT's StreamLAAL is about 3 seconds, the lowest among the compared systems; its GEMBA translation quality score exceeds the compared open-source models and 4 of 5 commercial APIs.
For short-speech benchmarks, FLEURS is 90.3, ACL6060 is 81.2, and RealSI-Sentence is 84.7, the highest score among all systems in the table. CoVoST2 is 75.5, lower than Qwen3.8-LiveTranslate's 82.1. It does not dominate across the board, but it does lead in key dimensions.
Compared with professional interpreters
A test closer to real-world scenarios is comparison with professional interpreters. Tencent collected 18 online audio clips that newly appeared after September 2026, about 6 minutes each, covering 10 fields including software and AI, medicine, manufacturing and energy, and law; it invited 6 CATTI-certified interpreters, 3 Level 1 and 3 Level 2, and took the highest score at each level for comparison. The result: Hy-SimulMT's translation quality is comparable to these professional interpreters, and its response speed is often faster. Note that this is a small-sample evaluation, so do not rush to conclude that “AI has comprehensively surpassed human simultaneous interpreters.”
The repository README also provides 4 comparison videos of humans and the model, demonstrating differences in information delivery, response timing, and translation wording.
Voice cloning
Voice cloning is an easily overlooked highlight. Cross-language communication is not just information transfer; who is speaking and what their tone is like are also part of the context. Hy-SimulMT preserves the original speaker's timbre when outputting translated speech, so in multi-speaker scenarios, listeners do not have to guess from subtitles “who said this sentence.”
The following two videos are English-to-Chinese and Chinese-to-English simultaneous interpretation plus voice cloning demos. Note that these are offline playback; the playback pace cannot be used as a measurement of real-time latency:
English → Chinese simultaneous interpretation and voice cloning demo (1 min 39 sec)
Chinese → English simultaneous interpretation and voice cloning demo (35 sec)
Deployment and trying it out
The barrier to hands-on deployment is not too high. The model uses a MoE architecture, with 30B total parameters and 3B activated, and is called Hy-SimulMT-30B-A3B. FP8 quantized and GGUF quantized versions are also provided, as well as an Interleave version that can output both the source text and the translation. The repository provides the SimulInfer inference service, and one command can bring up the vLLM backend and Web interface:
git clone https://github.com/Tencent-Hunyuan/Hy-SimulMT.git
cd Hy-SimulMT/SimulInfer
bash deploy.sh --install-server --model tencent/Hy-SimulMT-30B-A3B-Interleave
Speech output and voice cloning require separately configuring a TTS service. The GGUF format can be run with llama.cpp. The official note says: support for Hy-SimulMT has not yet been merged into the llama.cpp main branch, so it is recommended to wait for a new version that supports this model. The repository also includes the SimulTrain training tool, which can continue training based on the official checkpoint.
The model can be found on Hugging Face, ModelScope, and GitHub, with links at the end of the article. Tencent also provides the Hy Translate App, which you can experience after downloading.
What are people saying?
Some commenters are already asking: can the 3-second latency handle two people speaking at the same time (crosstalk)? The official team has not directly responded. Some commenters also clarified in the comments: 3 seconds is only the translation lag; speech output still requires a separate TTS service. Another commenter said that when listening to English lectures, the 3-second latency still causes audio and slides to be out of sync. The model currently only supports Chinese-English bidirectional translation; those who want smaller languages will have to wait.
A few judgments
Wang Weiwei of Guangdong University of Foreign Studies put it plainly: “To control latency, the system starts speaking first and then self-corrects; this kind of correction sounds close to an interpreter's spoken self-correction.” This observation is quite interesting; the correction strategy itself is also a direction worth researching in AI simultaneous interpretation.
Finally, a few judgments. This open-source release has lowered the barrier to AI simultaneous interpretation to a level ordinary developers can reach. Conference calls, cross-border interviews, and live-streamed speeches may all be changed by this type of model. But saying that simultaneous interpreters will immediately lose their jobs is an exaggeration. In scenarios involving liability determination and high-risk decisions, professional interpreters remain irreplaceable. Tencent's own positioning is the same: first use it in low-risk scenarios, then continuously improve through human-machine collaboration.
Resource links:
发布时间: 2026-10-27 02:01