IT Home October 23 news: The Artificial Intelligence Research Institute of the China Academy of Information and Communications Technology (abbreviated as "CAICT"), the Key Laboratory of AI Large Models and Software-Hardware Evaluation of the Ministry of Industry and Information Technology, and the China Artificial Intelligence Industry Development Alliance jointly released the "Large Model Token Service Performance Monitoring Report (2026)" today (October 23).
The report states that with the rapid iteration of large model technology, token (Token) services have become the key point from technology to application, and China's daily average token call volume keeps climbing.In June 2026, the domestic daily average token consumption was about 500 trillion, nearly 4 times that of December 2025 (2025-12)。
In the industry, more and more MaaS (Model as a Service) platform vendors provide large model token services through public cloud APIs. However, users lack an intuitive basis for judging token service performance,and face various problems such as difficulty in selection。
Based on monitoring data from January to August 2026 (1-8/2026), the report analyzes dimensions including each platform's token service call success rate, throughput speed, time to first token, stability, price, and context window, compares the official platforms of first-party large models as well as the performance differences of the four open-source models DeepSeek, Qwen, GLM, and Kimi on third-party platforms, further analyzes the key factors affecting token service performance, summarizes core industry observations, and clarifies next-step work plans, aiming to provide a reference for users' product selection and to offer directional guidance for the industry to optimize service capabilities.
1. Domestic token service per-second throughput doubled compared to 2025, still lower than abroad
The report states that in August, the average TPS (tokens output per second) of domestic models reached 78.23 (tokens/second),doubling compared to the 2025 average of 38.25 (tokens/second). However, in August, the average TPS of typical foreign models was 112.82 (tokens/second), compared with 51.35 (tokens/second) in December 2025, an increase of more than double, overall better than domestic.
2. Token service latency has increased, with consistent trends at home and abroad
The report states that since this year, the average TTFT (time to first token) of domestic models has shown a continuous upward trend, reaching an average of 1.26 seconds in August,a marked increase compared to the 2025 average of 0.78 seconds. But at the same time, the average TTFT of typical foreign models in August reached 2.08 seconds, also a marked increase compared to 1.04 seconds in April.
It is worth noting that the rise in TTFT is not simply performance degradation, but the result of multiple factors occurring simultaneously, such as increased model parameter counts, increased context length, and increased call volume.
3. Token service prices rebounded notably in 2026
The report states that after the price wars of the past two years, model service prices both at home and abroad rose notably in 2026.The domestic normalized average price in the second half of 2025 was 6.8 yuan, while the average price this year is 13.46 yuan, an increase of nearly 1 time. However, foreign prices remain higher than domestic ones; the average normalized price of typical models released since 2026 is about 117.8 yuan.
4. Model context length keeps expanding toward the million-token level
The report states that among the monitored models, the proportion with context length reaching 1M (1 million tokens) is 47.8%, and those with 200K and above total 95.7%, an increase of more than 76 percentage points compared with the proportion at the end of 2025, while the proportion of models below 200K keeps decreasing.
5. Open-source model token serving performance improved significantly, with large performance fluctuations for the same model across different platforms
The report states that in May, new versions of 4 open-source models were updated and put under monitoring, and the results show TPS continued to rise while TTFT latency was steadily compressed. For example, in August, DeepSeek-V4-Pro's TPS had already reached 2 times that of DeepSeek-V3.2.
In terms of latency, after the main models completed their generational upgrades in May, TTFT generally saw a stage-wise rise; as inference optimization measures were implemented, latency metrics gradually fell back to pre-upgrade levels.
At the same time,the 4 models showed varying TPS, TTFT, and invocation success rates across different platforms. Some third-party platforms clearly outperformed the model services provided by official platforms, and the TPS gap for the same model across different platforms could approach a factor of two at most. The differing inference acceleration, optimization strategies, and cost control approaches each platform applies to the same model are the main reasons for this performance divergence.