ITHome October 15 news: StepFun today officially released the Step5Preview weights. This release includes: BF16 and FP8 model weights, a technical blog, and accompanying open-source development tools.
According to the official introduction, Step 5 Preview is designed for long-horizon Agent tasks, adopts a sparse MoE architecture with a total of 600B parameters and only 27B activated parameters per token, supports a 1 million token context window, natively supports text and visual input, and performs outstandingly in scenarios such as AI programming, software engineering, professional knowledge work, and finance. Recently, Step 5 Preview ranked second globally in both OpenRouter's weekly and daily call volume rankings.
Currently, the model weights are available on Hugging Face and ModelScope, with adaptation support for multiple chip platforms and inference frameworks. Developers can deploy API services compatible with the OpenAI interface via vLLM or SGLang, or use Transformers for offline inference, connecting the model to their own data, tools, and systems to build agents, applications, workflows, and services.
ITHome noted that the technical blog released alongside explains how Step 5 Preview supports longer and more complex Agent tasks through the joint optimization of architecture and training. Key designs include:
A 92-layer narrow-and-deep architecture: provides deeper computation paths for implicit multi-hop reasoning while controlling parameter scale and computational cost.
Sparse GQA: reduces the computational overhead of the million-token context through block-level indexing and sparse computation merging. Among these, the 8-token block-level index can reduce indexing and top-k selection overhead to 1/8 of the original.
More stable training: retains FP32 precision on key states and parameters, combined with fine-grained gradient clipping, to mitigate abnormal activation growth and gradient spikes in deep networks.
Million-scale verifiable environments: covering software engineering, scientific reasoning, machine learning R&D, and professional knowledge work, with continuous calibration of task difficulty, diversity, and evaluation reliability.
Reinforcement learning for long-horizon tasks: incorporates context compression into end-to-end policy training, letting the Agent learn to organize task states and continue execution; combined with finer-grained reward attribution, it improves learning on long trajectories.