IT之家 On October 26, Tencent's Hunyuan simultaneous interpretation large model Hy-SimulMT was released and open-sourced today, featuring low latency and high accuracy, and supporting real-time speech-to-speech translation between Chinese and English — the speaker's audio can be input into the Hy-SimulMT model in real time, and the model translates while listening, providing the translated text and speech in real time. It also supports real-time voice cloning, which can highly restore the speaker's voice and tone. The model debuted on Tencent's Hy Translation App,offering free service experience。
From the official introduction, IT之家 learned that low latency is a major highlight of the model: the response time of the Hunyuan simultaneous interpretation large model Hy-SimulMTwas measured at around 3 seconds, while the average level of professional human interpreters is 4-5 seconds. Evaluation results show that the model's translation performance surpasses mainstream open-source simultaneous interpretation models on the market, and even exceeds most commercial simultaneous interpretation APIs.
According to the official statement, on RealSI, an open-source evaluation set for long audio commonly used in the industry, Hy-SimulMT demonstrated good simultaneous interpretation performance, as shown in the figure below:
StreamLAAL on the horizontal axis represents the latency metric of simultaneous interpretation, reflecting the number of seconds the model's output translation lags behind the input audio; the lower the better. The vertical axis represents translation quality, scored using GPT6.1-Sol; the higher the better. Hy-SimulMT achieves the best performance among open-source models at the lowest latency, even surpassing most commercial APIs.
On the short audio test set, Hy-SimulMT also performed well, as shown in the figure below:
To more accurately evaluate Hy-SimulMT's true capability, the team, within the scope permitted by compliance, extensively collected newly emerging Chinese and English audio from the internet, covering ten fields in total: software AI, medicine, manufacturing and energy, natural science, agriculture and food, business management, law, education, culture and tourism, and sports and life,involving multiple scenarios such as speeches, interviews, and conferences.。
On this data, the team not only tested Hy-SimulMT's performance, but also invited multiple human simultaneous interpretation experts to interpret these audios.
There were 6 human simultaneous interpretation experts in total, of whom 3 hold CATTI Level 1 interpretation certificates and 3 hold CATTI Level 2 interpretation certificates, and all experts have many years of practical simultaneous interpretation experience. The evaluation compared the translations of Hy-SimulMT and the human simultaneous interpretation experts in terms of latency and translation quality, with the human experts' score taken as the highest among the 3 people. In this capability positioning evaluation,Hy-SimulMT reached the level of human simultaneous interpretation experts in translation quality (with a higher score), while being faster (with lower latency)。
References: