Artificial intelligence has achieved a milestone breakthrough in the field of on-premise deployment. The open-source project Edge0 recently officially released and opened source the Edge0-145B and Edge0-120B models, along with a unified inference framework and deployment tools. As the world’s first open-source model with 145 billion parameters that claims to run smoothly locally on personal computers, Edge0-145B successfully brings the programming, inference, and agent capabilities of cutting-edge cloud-based large models directly to users’ local laptops.
Breaking through hardware limits: How does it run a 145B model on a laptop?
For a long time, edge AI has been struggling to balance model capabilities with hardware operationality: large models have many parameters and require extremely high memory and bandwidth; small models, while taking up little space, are difficult to handle complex professional tasks. By handing over the heavy work to the cloud, it is also inevitable to face continuous API costs, data privacy restrictions, and a strong dependence on the network.
To address this industry pain point, the Edge0 team has carried out comprehensive innovation across four aspects: underlying architecture, training, loading, and scheduling.
Sparse Expert Architecture (MoE): The model cleverly shares large matrices among a vast number of sparse experts, significantly reducing the weights that need to be stored and transferred while keeping the total number of experts unchanged. Out of the 145B total parameters, only approximately 10B are involved in each inference calculation.
Endside collaborative training from zero: The model integrates strict endside deployment constraints into its design from the early stages of training, avoiding performance losses caused by post-training compression or pruning in traditional cloud-based models.
Three-layer storage intelligent pipeline: Through clever design, the access to GPU memory, system memory, and solid-state drive (SSD) forms an efficient pipeline. High-frequency “thermal experts” reside in GPU memory, with a database of approximately 40GB stored in system memory for on-demand loading, while approximately 48GB of word table data is kept on SSD for real-time reading, avoiding serial waiting.
CPU and GPU efficient collaboration: The GPU is responsible for computing the cached experts, while the CPU handles the rest of the work in parallel. Using speculative decoding techniques, the cost of weight reading is distributed across more outputs.
Through these innovations, Edge0-145B achieves remarkable real-world performance on compatible PCs—such as Apple M-series chip-based MacBook Pros or Windows platforms equipped with RTX4090/5090—with a peak memory usage of only 29GB, a prompt preprocessing speed of over 1,100 tokens/s, a decoding generation speed of over 80 tokens/s, and typical power consumption of around 120W. It supports ultra-long contexts of 128K to 256K.
Programming and reasoning capabilities are aligned with the cloud, allowing you to effortlessly manage complex local workflows.
In terms of core capabilities, Edge0-145B demonstrated strong competitiveness in multiple international authoritative benchmark tests. Its six benchmark scores averaged 72.8. Compared to mainstream cloud or large-scale baseline models that require 70GB to 87GB of memory under a standard 4-bit deployment, the actual memory usage is reduced by approximately 40% to 60%, yet its capabilities remain in the same tier:
Programming and Agent: Score 64.6 on DeepSWE1.1, score 86.52 on Terminal-Bench2.1, and score 80.6 on Instruction Following IFBench.
Reasoning and Knowledge: The score on GPQA-Diamond is as high as 90.4, demonstrating solid logic and knowledge foundation.
With its excellent code generation, tool invocation, self-diagnosing, and visual inspection capabilities, Edge0-145B can independently support complex local workflows. In the actual application scenarios shown by the official team, it can not only be used for daily chatting, but also complete the entire production process of a Tesla Roadster explanation video within one hour locally, build a Manhattan 3D city scene from scratch in Blender, develop a playable medieval RPG game, write the flight code to control a real drone through obstacles, and even render from the PCB circuit board layout to the Unreal Engine assembly animation in KiCad. The entire process runs completely locally offline, which not only saves significant cloud API costs, but also ensures that core sensitive data remains firmly in the user’s hands.
Build a side-mounted open-source suite to drive AI toward “side-mounted priority”.
In addition to the powerful 145B model, Edge0 has released four major series, totaling 200 models of different sizes over the past year, forming a complete matrix covering voice, text, vision, and memory.
Edge0 series: covers 145B, 120B (for laptops), and 35B, 8B (for smartphones), fully empowering vibe coding and AI tasks.
Audio8 series: Focused listening and speaking capabilities, including multilingual speech recognition, synthesis, and end-to-end interaction models ranging from 0.1B to 3B.
Decision0 series: The 0.6B and 3B multimodal rapid decision model delays decisions to within 40 milliseconds.
Embedding0 series: 0.6B multimodal embedding model, fully supporting local long-term memory and RAG retrieval for text, voice, images, and videos.
Currently, the complete model weights for Edge0-145B and Edge0-120B have been officially opened source on Hugging Face. The unified inference framework and one-click deployment guide have also been uploaded to GitHub. Additionally, the official Edge0App is available without environment configuration. As the efficiency of large models on edge hardware continues to improve, the application paradigm of AI is steadily moving from “cloud优先” in the past to a new era of “edge优先, cloud assistance”.