Pre-release / continuous build: this is an experimental build, not a stable release.
feat: add GLM5Next MTP, optimize (#29928) * llama : add GLM5-Next NextN (MTP) graph Build the GLM5-Next multi-token-prediction head as graph_mtp: the NextN block embeds enorm(tok)+hnorm(h) through eh_proj, runs one plain DSA layer and the shared lm_head, reusing the trunk's builders through the no_build tag ctor. llama_memory_recurrent also tolerates a partial seq_rm when the context holds no recurrent layers, which is what the MTP draft context needs. Assisted-by: Claude * llama : glm5-next: skip dead compute in headless NextN forwards A NextN forward with no output rows (the MTP catch-up and the draft-context prefill) persists only through its cache writes, so the headless graph keeps the MLA latent, indexer key|gate and pooled-key writes and drops the query path, the indexer selection, the attention body, the FFN and the LM head. The 4-token catch-up falls from 6.9 ms to 0.33 ms of kernels; the greedy output hashes and the draft acceptance are unchanged. Assisted-by: Claude * llama : glm5-next: fix NextN extraction contracts and shared-tail rollback Three fixes from the architectural review. The headless graph prune now also requires that no unmasked nextn extraction is live, because that mode reads n_tokens hidden rows regardless of the logits flags. Masked extraction publishes the hidden rows gathered by the output ids, so a batch whose output flags are not a prefix exports the right rows. A partial recurrent rollback whose tail cell is shared with another sequence is now rejected instead of silently moving that sequence's tail. Assisted-by: Claude * llama : glm5-next: tidy comments in the MTP changes Assisted-by: Claude * llama : glm5-next: crop the MTP graph to the output rows instead of pruning it Replace the headless NextN prune with the crop pattern the other MTP graphs use: gather the attention output and the block input at the output ids before the position-wise FFN and the shared head. A NextN forward with no output rows (the MTP catch-up and the draft-context prefill) then runs the FFN and the head over zero rows. The 4-token catch-up falls from 6.9 ms to 2.9 ms of kernels; greedy output hashes are unchanged. Assisted-by: Claude * glm5-next: use the nextn crop helpers in the MTP graph Replace the local crop condition and the masked select of t_h_nextn with crop_before_nextn and crop_after_nextn, so the MTP graph narrows its rows the same way as the main graph and the other models. Describe the shared cell and empty filter branches of the recurrent partial rollback. * glm5-next: load MTP-only and trunk-only GGUF files Make the trunk tensors optional when the file only holds the NextN layer, and the NextN tensors optional when the file only holds the trunk, so the split MTP GGUF loads as a draft model. Co-authored-by: Georgi Gerganov --------- Co-authored-by: Pascal Co-authored-by: Georgi GerganovWebsite: - https://llama.app
Attestations: - https://github.com/ggml-org/llama.cpp/attestations/53581440
macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework
Linux: - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries - Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries - Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries - Ubuntu x64 (ROCm 10.0) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) - Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide
Android: - Android arm64 (CPU) - Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide
Windows: - Windows x64 (CPU) - Windows arm64 (CPU) - Windows arm64 (OpenCL Adreno) - Windows x64 (CUDA 12) - CUDA 12.4 DLLs - Windows x64 (CUDA 13) - CUDA 13.4 DLLs - Windows arm64 (CUDA 13) - CUDA 13.4 DLLs - Windows x64 (Vulkan) - Windows arm64 (Vulkan) - Windows x64 (OpenVINO) - Windows x64 (SYCL) - Windows x64 (ROCm 10.0)
openEuler: - DISABLED - openEuler x86 (310p) - openEuler x86 (910b, ACL Graph) - openEuler aarch64 (310p) - openEuler aarch64 (910b, ACL Graph)
UI: - UI