Two new developments in artificial intelligence highlight the stark gap between ambitious AI assistants and actual performance. A compact 1-billion-parameter model, MiniCPM5-1B, can now run local agents on smartphones, handling tool calling and multi-step tasks offline. Meanwhile, a rigorous new benchmark called Claw-Anything reveals that even leading models struggle to manage long-horizon personal assistant duties, scoring far below their reported capabilities on simpler tests.
MiniCPM5-1B, released by OpenBMB, is designed for resource-constrained hardware. It supports the Model Context Protocol (MCP) and native tool calling out of the box, enabling agentic workflows without cloud connectivity. The model scores an average of 42.57 across agentic and reasoning benchmarks, surpassing the next-best 1B-class competitor's 35.61. It fits a 128K token context window—roughly 96,000 words—allowing persistent memory across long sessions, roleplay, or document analysis. Built using InfLLM v2, a trainable attention mechanism that processes only 5% of surrounding tokens during inference, the model achieves competitive results with just 8 trillion training tokens, significantly less than rivals like Qwen 3 (36 trillion). In practical tests, the model successfully called MCP servers to fetch real-time info such as the Bitcoin price and gave coherent stock recommendations (Amazon, Microsoft, Nvidia). However, it also hallucinated on a classic logic trap, demonstrating the limits of small-scale reasoning.
On the other end of the spectrum, researchers from Huawei Technologies, Beijing Institute of Technology, Peking University, and the Chinese Academy of Sciences introduced Claw-Anything, a benchmark that evaluates AI agents as true personal assistants over simulated months of user activity. Tasks involve cross-referencing data across email, calendar, notes, and multiple devices (CLI and GUI Android), with an average context window of 191,700 words. The benchmark scores pass@1—the probability of completing a task correctly on the first attempt. OpenAI's GPT-5.5, built for long-horizon agentic work, managed only 34.5%. Proactive assistance—where the agent spots needs without being asked—was even worse, with a mere 6.7% success rate. The study underscores that current benchmarks treat agents as isolated task-solvers, not as assistants embedded in messy, real-world data streams.
These contrasting findings highlight both progress and persistent challenges in AI agent design. While on-device models can now handle local agent functions securely and efficiently, even the most advanced cloud-based systems remain unreliable for complex, multi-service coordination. The researchers behind Claw-Anything have open-sourced their data pipeline and training environments, hoping to spur improvements in cross-service reasoning.
