Key Points
- The open-source OpenMobile-2 project trains AI smartphone agents in an environment with more than 110 simulated apps.
- App diversity matters more than data volume. With the same amount of training data, increasing the number of apps pushed the success rate on unseen applications from 60.7 to 67.5 percent.
- On real Android devices, the larger model scored 59.6 points on SPA-Bench, the highest result among open models.
More data alone doesn't make smartphone agents better. With the same amount of data, what matters most is how many different apps an agent trains on.
AI agents that operate apps on their own learn mainly through practice, but the data they need is mostly locked up at big companies. Open models lag behind as a result, write researchers from Nanjing University, AI company SenseTime, and several other universities.
OpenMobile-2 is their attempt to close that gap. They're releasing two agent models built on Qwen3.5 with 9 billion and Qwen3.6 with 27 billion parameters, along with the full training environment, data, and instructions to reproduce everything. The authors say their model ranks near the top among current smartphone agents.
Why rebuilding 110 apps beats training on real ones
Training on real apps is painful because logins, real purchases, and no way to reset state make the process expensive and hard to repeat. So the researchers rebuilt 35 apps popular mainly in China, including Taobao, JD, DiDi, and Meituan, plus globally used services like Slack and Google Drive. Combined with existing simulations and apps running in an Android emulator, the practice ground covers more than 110 applications.Ad
Human reviewers compared the rebuilt apps to the originals and kept refining until workflows and interfaces matched. Orders, budgets, and travel plans persist across sessions, and data can pass between apps via the clipboard. Every simulation resets to its starting state automatically, while the agent only sees screenshots and has no access to internal app data, mimicking how a person would actually use them.
From this setup, the team generated about 11,600 example workflows and more than 2,000 practice tasks with automated success checks. Google's Gemini 3.1 Pro served as the teacher model that produced the examples, which means OpenMobile-2 is effectively learning from a closed system.
Real-phone performance jumps the most
Both models show clear gains on established benchmarks. On AndroidWorld, the larger model jumps from 67.1 to 79.9 points, while the smaller one goes from 54.3 to 74. The predecessor OpenMobile-1 scored 64.7 on the same test. On the harder MobileWorld benchmark, which covers longer tasks spanning multiple apps, the larger model improves from 35.9 to 50.4 points.
The biggest jump shows up on real Android devices, where the larger model climbs from 31.9 to 59.6 points on SPA-Bench. The researchers say that's the best score among open models.Ad
Training environment diversity matters more than sheer data volume here. The researchers held the amount of data constant and only increased the number of apps the agent practiced on, which pushed the success rate on unseen apps from 60.7 to 67.5 percent. With the same data budget, app variety beats extra examples from a handful of apps.
Reinforcement learning on top brought only limited gains, mainly in environments that resembled the training setup. On AndroidWorld, the larger model actually scored slightly worse afterward, landing at 79.9 instead of 80.8 points.
Direct commands let agents skip the menu maze
An experimental part of the study goes beyond tapping and swiping. More than 50 of the simulated apps offer direct commands like "add to cart" so the agent can skip navigating through menus, and the agent freely switches between both approaches depending on what's available.
On its own, though, this doesn't help much because the models tend to ignore the shortcuts without targeted training. Once they've been trained on them, the agents use them well, solving up to 10.3 percentage points more tasks while needing about five fewer steps. The researchers say this ability also transfers to tools the models never encountered during training.
During the later reinforcement learning phase, tool use got a bonus factor whenever a run included at least one direct command, though the reward still depended on whether the task was actually completed. Penalties kicked in when the agent reported a success that wasn't real. These interfaces only exist in the simulated apps for now, and the researchers call the approach experimental themselves.
Some results come from a custom benchmark the team built with 215 tasks, nearly half of which span multiple apps. Task completion isn't judged by a language model but by a program that compares the app's stored data before and after each run. The authors also checked for overlap between training and test tasks and found only minor similarities.
Microsoft Research had already classified API agents as more reliable and GUI agents as more flexible in a 2025 study, pointing to mobile apps as a typical case where external interfaces don't exist. Oppo takes a different route with its open agent X-OmniClaw, which runs on real devices and replays previously shown paths in apps like Taobao or Meituan via deeplinks instead of clicking through menus again. Among closed models, Google has built smartphone control directly into Gemini 3.5 Flash.
Source: Arxiv