By Wenle, from Qfeisi Temple
QbitAI | Official account QbitAI
Not to be blunt, but AI today really does write code fast and churn out impressive research;
yet when it comes to voice chat, I have to ask:
what exactly is it rushing for?!!
Before you finish speaking, it jumps in; when your emotions run high, it still reads out answers stiffly; you want to interrupt and change topics, but it insists on finishing its last sentence……
That's not a conversation — that's pure arm-wrestling.
△ Image AI-generated
To put it plainly, AI passed the threshold of 'being able to speak' long ago, but 'being able to chat' still has a long way to go.
To chat well, it has to understand emotions and subtext, keep up with your speaking rhythm, and promptly yield the floor when you interrupt.
These small experiential awkwardnesses may look trivial, but behind them lies an entire pipeline: speech recognition, model understanding, real-time transmission, and terminal response.
If any link slows down, the conversational experience easily falls apart.
And precisely because of this,real-time interaction has become a shared research focus for model, application, and hardware teams。
The just-concludediRTE2026 Real-time Intelligence Conferencebrought all these teams together:
those researching next-generation interaction models, building customer-service Agents, and making smart hardware were all discussing the same thing —
how to make AI keep up with real conversations。
A bellwether conference for conversational AI
From the perspective of an industry observer, it is clear that the core evolutionary line of conversational AI today hasshifted from simply 'generating a one-shot answer' to deeply 'continuously engaging in real conversations.'
OpenAI's GPT-Realtime brings real-time reasoning and tool calling into speech models, letting AI decide how to respond and which tools to call after understanding the request.
Google's Gemini Live lets dialogue and deep thinking run in parallel, so users can keep talking while complex tasks are processed in the background; GPT-Live, together with Decisions, lets Agents trigger actions during the user's speech based on incremental transcription.
From full-duplex listening-while-speaking, to full-modal judgment combining voice and visuals, AI is gradually stepping out of the turn-based question-and-answer format.
Leading vendors keep pushing real-time interaction forward, and iRTE has become a window for observing how these technologies reach industry.
As early as 2020, this industry conference was upgraded from RTC to RTE, expanding its topics from real-time audio-video communication to richer real-time interactive experiences;
this year, an 'i' (standing for Intelligent, real-time intelligence) was added to RTE, formally upgrading it to iRTE, further focusing on the fusion of real-time interaction and generative AI.
The name gained just one letter, but the work it has to take on has grown a great deal more.
The topics of the special sessions make it easy to see the core direction of this year's iRTE conference: more than ten forums and over 100 in-depth talks together probe the three practical questions that must be crossed for conversational AI to land in the real world.
The first question is the alignment between model capability and business outcomes.
Meituan, Alibaba, MiniMax, and Agora appeared together in the enterprise-grade customer service agent track to discuss this: no matter how impressive a large model's conversational performance is, if it cannot fit real business processes, it still cannot generate commercial value.
The industry urgently needs to bridge the gap between model approaches, engineering implementation, and business goals.
The second proposition: behind fluent conversation lies an entire foundational technology base that cannot be absent.
Achieving natural real-time conversation cannot rely on the large model alone. Technical experts from enterprises and universities/institutions including StepFun, The Chinese University of Hong Kong, Shenzhen, the Tsinghua University Speech and Audio Technology Lab, Qwen Audio, Ant Group's Bailian, Xiaomi, and Soul held discussions at the Conversational AI and Audio AI Technology session, covering directions such as interaction models, target voice extraction, multi-speaker recognition, and real-time speech generation.
The industry has reached a consensus: any shortcoming in any of these foundational technologies will directly damage the conversation experience.
The third proposition is the real-world test that comes after technology moves from the lab into terminal hardware.
When AI capabilities are embedded into hardware devices, technical solutions must confront complex, real physical environments.
At the Smart Hardware and Embodied AI session, Rockchip, Uniview, and PETKIT pointed out that merely connecting a large model to hardware is far from enough; on-device compute constraints and real-world environment perception must ultimately be weighed against product value itself.
Once equipped with AI, devices must develop differentiated capabilities that genuinely make users willing to keep using them.
The main forum roundtable pushed the discussion further toward implementation.
Chen Wei, founder of Xieyue Intelligence, believes that the key to embodied AI is truly converting human intent into robot action. Robots must not only understand voice commands, but also integrate multimodal information such as vision, speech, and their own state to complete perception, planning, and decision-making about the physical environment, and continuously adjust task execution based on environmental changes after acting.
Cai Guoxiang, co-founder of Rokid, proposed that the naturalness of conversation is the core factor determining the user experience of AI wearable devices;
Geng Xianglei, founder of Guanke Technology, noted that latency in voice seats and the timing control of interruption interactions directly affect business outcomes. Whether in smart hardware or enterprise customer service scenarios, all tests ultimately converge into the brief interaction window of just a few seconds after the user starts speaking.
While models handle understanding and reasoning, real-time networks, audio/video processing, and interaction orchestration must also keep up in coordination.
NVIDIA Principal Architect and winner of the 2025 Technology & Engineering Emmy AwardAndrey Norkinpresented a talk on AV2, the new generation of video coding, filling in the key link of media delivery and exploring paths to guarantee high-quality video interaction experiences on top of higher compression efficiency.
From model capability to business closed loops, from terminal hardware to media transmission, the industry trends presented at iRTE are gradually becoming clear:
the competition in conversational AI has already extended to the full chain of complete, real interaction.。
And the foundational base that supports this chain, determines the lower bound of interaction quality, and places the highest demands on real-time performance is precisely the field thatAgorahas cultivated deeply.
What Agora aims to do is build the real-time infrastructure for the generative AI era.
Viewed along this chain, the work Agora must undertake is also changing.
In the past, video conferencing, online classrooms, and voice chat mainly required the smooth transmission of sound and images between people.
Now that Agents have joined in, speech recognition, model inference, tool calls, and terminal feedback all need to coordinate in a timely manner during conversations.
The "five levels of conversational capability" framework proposed by Agora founder and CEO Zhao Bin at the main forum further elaborates this change:
from L1's one-question-one-answer, to L2's mastery of interaction rules and coordination of speaking and listening timing, then to L3's understanding of individual intent and context, L4's understanding of multi-party relationships and communication boundaries, and ultimately toward L5's deep communication based on long-term context.
Viewed through this framework, fast responses and natural-sounding voices are only part of the capability. AI also needs to understand who it is talking to, why the other party says what they say, and what response is appropriate.
The most immediate test at present is handling pauses, interruptions, and task execution.
The collaboration of the "fast brain and slow brain"addresses precisely this kind of problem.
This framework lets the interaction model maintain real-time conversation while the backend model handles deep reasoning, search, and tool calls. Users can continue to add conditions or change requirements, and the system adjusts execution accordingly.
But once running in parallel, the several models must also stay consistent: if a new request has already been accepted up front, the backend cannot continue operating according to the old task.
The one displayed by AgoraAgora Conversational Model, uses a hybrid dialogue system with multi-model collaboration, organizing sound perception, full-duplex response, reasoning, tools, memory, and voice expression, coordinating continuous listening, intent judgment, and action.
The role of real-time infrastructure has also extended from connection and transmission to model orchestration during conversations.
what Zhao Bin introducedConversational Agent System, combining the above capabilities with a global real-time network, audio processing, and inference infrastructure. Sound and visuals must be continuously transmitted, while turns, context, and business state must also be synchronized in a timely manner.
What he further proposedDistributed Real-Time Inference Clouddirection, but also bringing models closer to users to run, and dynamically selecting models, tools, and paths based on latency, load, and scenario.
As these capabilities enter businesses and end devices, there are corresponding products to support them.
Conversational AI engineConnects models, Agent runtime frameworks and users, compatible with both cascaded and end-to-end architectures, providing capabilities such as turn management, interruption handling, and attention locking;
Development kitHelps hardware extend from voice conversations to visual and motion interaction;
Call AgentAimed at telephone business, offering tools for line configuration, workflow orchestration, testing and deployment, and monitoring.
Developers, meanwhile, can use the SDK, Skills, CLI and Recipes to get access support, AI coding assistance, and scenario examples, combining models, tools and memory with open-source projects such as the TEN Framework to reduce repeated building of fundamental components.
From the network and interaction processing to products and tools, Agora is organizing the capabilities needed for humans and Agents to communicate into a set of reusable infrastructure.
This also echoes the conference's shift from RTE to iRTE.
As real-time interaction begins to carry intelligence, the underlying system needs to let a model's understanding, reasoning and action continuously work together in real conversations.
As for ordinary users, they may not care how many models collaborate behind the scenes or how many network nodes are involved. Our requirements have always been simple:
Listen until I finish speaking; keep up when I change my mind; and get the job done well once I've assigned it.——
AI is already so smart—when it comes to chatting, it should finally give people some peace of mind.