LLM news without the noise.

Introducing GPT-6.1 Terra

Bringing fast and capable computer use to everyone

OpenAI models are coming to Google Cloud

OpenAI and Google Cloud are bringing frontier models and Codex workflows to Gemini’s Model Garden, available through Google Cloud Marketplace.

Patch the Planet: global response coordination as agents uncover critical vulnerabilities

Patch the Planet connects security researchers, maintainers, and incident responders to help address critical vulnerabilities across open source.

b12005

opencl: add initial q20 support (#31340) opencl: add q20 AOS mv Co-authored-by: Shaofei Qi Assisted-by: Codex opencl: add q20 SOA mv and tiled mm Co-authored-by: Shaofei Qi Assisted-by: Codex opencl: add q20 noshuffle gemm/gemv for adreno Assisted-by: Codex opencl: add q20 dp4a for adreno opencl: optimize q20 adreno decode and narrow batches Co-authored-by:

b12004

common : end reasoning budget forcing on an end tag from the prompt (#31374) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu 26.04 s390x (CPU) Ubuntu 26.04 s390x (zDNN) Ubuntu x6

b12003

sycl: FLASHATTNEXT failures (#30761) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu 26.04 s390x (CPU) Ubuntu 26.04 s390x (zDNN) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu

b12002

ggml : add batched tensor copy API (#30881) ggml : add generic batched tensor copy API Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) llama : use batched tensor copy API in context io Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) ggml : add batched tensor

b12001

ROCm/HIP: add RDNA4M family for gfx1170/gfx1171 (#31373) gfx117X is more like RDNA4, with 128b WMMA. But they're not identical. Adjust the defines accordingly to have a new RDNA4M and use it where it makes sense. Assisted-by: Codex Assisted-by: GitHub Copilot Co-authored-by: Mario Limonciello Website: link Attestations: link macOS/iOS: macOS Apple Silicon (a

Backdoors in official Claude Code extensions

b12000

Quant: OCP FP8 E4M3 support (#27512) WIP OCP FP8 E4M3 support Fix python type-checker Materialize inputscales also for FP8 when converting Fix conversion of link Per row FP8 wrongly fell to NVFP4 conversion, as NVFP4 checked on dims alone. Also check on dtype for FP8 Need to reshape FP8 QKV projections scales in the same way that weights are reshaped Allow r

Using thousands of AI agents to uncover new leads in biological data

See how OpenAI, PerturbAI, and Scripps Research used distributed AI agents to uncover promising research leads in data from millions of brain cells.

Patch release v5.22.1

Patch release v5.22.1 We just had to fix the documentation! [Fix] Remove WIP doc for Ml4 (link) from @remi-or

b11998

model: mistral le chonk (#31376) llama: extend DSA and sdpark to new model fix ci Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu 26.04 s390x (CPU) Ubuntu 26.04 s390x (zDNN) Ubun

b11997

llama : reuse the tensor staging buffer and skip the direct-io bounce buffer (#31321) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu 26.04 s390x (CPU) Ubuntu 26.04 s390x (zDNN)

b11988

model : Support Limite 1B - Violetto (#29433) gguf-py : add Limite arch Assisted-by: Claude Opus 5.5 Signed-off-by: Giuseppe Scrivano convert : add Limite support Assisted-by: Claude Opus 5.5 Signed-off-by: Giuseppe Scrivano llama : add Limite support Assisted-by: Claude Opus 5.5 Signed-off-by: Giuseppe Scrivano Signed-off-by: Giuseppe Scrivano Website: link

b11986

opencl: initial support for iq2 (#31267) opencl: add iq2s AOS mv opencl: add iq2s SOA mv and tiled mm opencl: add iq2s noshuffle gemm/gemv for adreno opencl: add iq2s dp4a for adreno opencl: optimize iq2s adreno decode and narrow batches Co-authored-by: Hongqiang Wang opencl: add iq2xs AOS mv opencl: add iq2xs SOA mv and tiled mm opencl: add iq2xs noshuffle

b11985

metal : faster MXFP4 mat-vec, full threadgroups for unary/binary/GLU (#31075) metal : faster MXFP4 mat-vec kernel Decode the fp4 values with the fp16 bit trick instead of a threadgroup lookup table, and load the packed nibbles as aligned 32-bit words. The kernel no longer uses threadgroup memory. metal : launch contiguous unary and binary ops with full threa

Anthropic hacked to deliver remote access trojan

An updated Model Spec for more capable and beneficial AI

OpenAI’s updated Model Spec outlines five core values to guide more capable AI toward sound judgment, human agency, and faithfulness to human intent.

LLM Voice

We're releasing the weights of Mistral Large 4

We ran AI agent swarms in Minecraft and Mindustry

v5.22.0: Mistral Large 4 (Le Chonk)

New Models Mistral Large 4 (Le Chonk) Mistral Large 4 is Mistral's largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters i

b11982

hexagon: support for UPSCALE op (#31331) hex-upscale: basic upscale ops (not fully vectorized) hex-upscale: optimize nearest 2x, 4x and bilinear 4x hex-upscale: add support for non-contig rows and dims collapsing hex-upscale: use vgather instead of scalar staging hex-upscale: move all kernel select logic to the host, review comments and cleanups tests: add p

b11981

hexagon: improve DDR bw utilization in transposed CONCAT (#31277) hex-concat: double row block size for transposed CONCAT hex-concat: double buffer scratchpads hex-concat: chunked src1 rows in transposed concat hex-concat: pad src1 VTCM stride to spread gather lanes hex-vgather: apply similar vgather opts to SSMCONV and GDN Co-authored-by: Max Krasnyansky We

b11980

[SYCL] add DPAS flash attention kernel for quantized KV cache (#31125) sycl : add DPAS flash attention kernel for quantized KV cache Decode and small batches (1 to 31 query tokens) now run flash attention straight on q40 and q80 KV blocks, instead of scalar dequant or converting the whole cache to f16 per call. Q.K uses s8 x s4 and s8 x s8 DPAS on XMX, V.P s

b11979

sycl : vectorize reordered-layout dequantization stores (Q40, K-quants, IQ3, MXFP4) (#30976) sycl : widen reordered q40 dequantization stores The reordered-layout Q40 conversion kernel processed one qs byte per work-item with two scalar stores, strided 16 elements apart. Give it the same treatment the Q80 kernel got in #30775: a flat sycl::range<1> launch wi

b11978

tests : fix comppadding CUDA graph reservation (#31344) Assisted-by: Codex Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu 26.04 s390x (CPU) Ubuntu 26.04 s390x (zDNN) Ubuntu x64

b11977

ggml-openvino: slice view inputs of SCALE (#31334) SCALE now accepts a src0 view with arbitrary row strides. translatescale used the raw input, and on the dynamic path a VIEW translates to its whole source tensor, so a strided view input scaled the entire source. The model output then had the source's size, and copying it back overran the result buffer, whic

b11976

Hexagon: IM2COL improvement - block based HVX parallelization (#31050) hexagon: IM2COL over multiple threads with output-col blocks Assisted-by: Claude Opus 5.5 hex-im2col: fix padding handling hex-im2col: loop unroll common cases Co-authored-by: Max Krasnyansky Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm6

b11972

common, mtmd: bound control vector layer indices and M-RoPE position loops (#31336) common: bound control vector layer indices by the model layer count A direction tensor layer index is parsed from the tensor name with no upper bound, and the buffer size and write offset are computed in int. A large index overflows both, so the accumulation writes before the

b11971

opencl: add tiled q4k GEMM for adreno (#30412) opencl: run the tiled q4K lmhead through a tiled GEMM at batch > 1 opencl: split tiled dp4a q4k gemm into its own file and test for dp4a Assisted-by: Codex Co-authored-by: Li He Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS In

b11970

hexagon: add conv2d support (#30995) hexagon: add conv2d op hex-conv2d: add double-buffered staging hex-conv2d: full 64-wide activation block transpose straight from staged buffer hex-conv2d: implicit gemm hex-conv2d: remove scalar path and kernel hex-conv2d: unify conv2d kernel params for all paths hex-conv2d: scale HMX result before f16 storage and then re

v0.40.13

LFM3 on Linux and Windows Liquid AI's LFM3 models, including text-to-speech in lfm3-audio and lfm3-omni, now run on Linux and Windows as well as Apple silicon. Full Changelog: link

b11969

Speed up IQ4 quant creation with pre-computed halfway points (#31201) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu 26.04 s390x (CPU) Ubuntu 26.04 s390x (zDNN) Ubuntu x64 (Vulk

b11968

ggml-cpu: SVE 256 bit implementation of gemmq6K8x8q8K kernel (#27472) q6k GEMM working cleanup review comments review comment addressed - removed pg168 incorporated the review comments - replaced switch with if and other points marking q8kblocklen UNUSEDto disable warning for non-i8mm cases Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm

b11966

ci(s390x): switch to Ubuntu 26.04 runners (#31317) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu 26.04 s390x (CPU) Ubuntu 26.04 s390x (zDNN) Ubuntu x64 (Vulkan) Ubuntu arm64 (V

b11963

sycl: work-group argmax for short TOPK rows (#31110) sycl: work-group argmax for short TOPK rows Used angelX to optimize SYCL/drama 3 coordination, results are as follows Rows with ncols <= max work-group size: one work-item per column, k rounds of work-group argmax. Ties take the lower column, NaN is never picked. Longer rows keep the existing paths. Arc Pr

b11962

cuda: fuse MoE (MULMATID) gate/up and GLU in MMQ (#31188) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libr

Mistral 4 had a release, makes it comparable to Chinese open-weight models

Granite Switch 4.1.1 — Update models like you update software

Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5

How we detect frontier model misalignment

OpenAI uses layered monitoring to detect frontier model misalignment and unauthorized agent behavior so responders can intervene before incidents escalate.

How Ropes & Gray is developing AI for transactional diligence

Ropes & Gray is developing AI for transactional diligence with OpenAI. Lawyers review grouped documents, summaries, and reports before advising clients.

Claude Code regularly sends each background agent’s transcript to its model

Gemini 4 Carbon

Gemini 4 Carbon: Our next frontier model

Securing our frontier model training environment

OpenAI explains how we strengthened containment for frontier model training and evaluation, including offline environments and high-compute red-teaming.

v0.8.0

Overview This release adds the LiquidAI LFM3 family (text, vision, audio and omni) together with a new experimental streaming-audio API in mtmd, and wires realtime audio in and out of the server through the new /v1/realtime and /tts endpoints. A batch of new architectures landed: DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite

b11960

llama.cpp : bump version to 0.8.0 (#31309) llama.cpp : bump version to 0.8.0 authors : update Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) U

b11954

tests : adjust constants to reduce irrelevant warnings (#31254) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.

b11953

cuda : add top-k MoE fusion for 384 experts (#31288) Assisted-by: Codex Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) -

b11951

chat : stop qwen3-coder reasoning and content before template separators (#30557) chat : strip trailing whitespace from parsed reasoning and content Assisted-by: Claude Fable 5.1 chat : stop qwen3-coder reasoning and content before template separators Assisted-by: Claude tests : remove the Qwen parser case from the chat session tests Assisted-by: Claude comm

b11946

server : add realtime conversation and transcription on /v1/realtime (#31269) server : add realtime conversation and transcription on /v1/realtime Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 examples : default the realtime client to the server port 9931 Assisted-by: Claude Opus 5.5 server : sample realtime responses right after their own dec

b11945

model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275) model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B Assisted-by: Claude Opus 5.5 convert : keep the speech task templates out of the d1 GGUFs Assisted-by: Claude Opus 5.5 small refactoring: nobatching Co-authored-by: Xuan Son Nguyen Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm

b11944

server: expose the reasoning effort levels of the chat template (#31271) server: expose the reasoning effort levels of the chat template The capability probe renders a user turn for each candidate level (minimal, low, medium, high, xhigh, max) and for an unknown level. A level is honored when it renders without raising and differently from the unknown level,

b11941

model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257) model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B Co-authored-by: Xuan Son Nguyen Assisted-by: Claude Opus 5.5 convert : read the LFM3-Audio config.json as released A transformers that knows lfm3audio rewrites the config into another layout (no lfm/encoder keys). Assisted-by: Cl

b11940

common : support deferred tools and toolreference content parts (#31256) common : support deferred tools and toolreference content parts Assisted-by: Claude Opus 5.5 common : credit the deferred tools co-author Co-authored-by: Aldehir Rojas <765227+aldehir@users.noreply.github.com> Assisted-by: Claude Opus 5.5 tests : fix the type check of the deferred tools

b11938

model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252) model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B Co-authored-by: Sigbjørn Skjæret Co-authored-by: Xuan-Son Nguyen Assisted-by: Claude Opus 5.5 tests : set the SWA rope base of the LFM3 test model Assisted-by: Claude Opus 5.5 llama : enable recurrent state rollback for LFM3 Assisted-by: Cla

b11936

llama : build the k-pool sparse selection independently of the kqmask shape (#31243) llama : build the k-pool sparse selection independently of the kqmask shape qwen4exp (QSA) and glm5-next pick cache cells by index and no longer read the kqmask values, but they still shaped the selection like the dense kqmask. Build the selection as [nkv, ntokens] over the

b11935

rpc : catch std::lengtherror in recvmsg (#31204) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubu

b11934

tests : speed up tensor data generation and model saving (#31145) tests : speed up tensor data generation and model saving Refactor savemodels to create only the model, not a llamacontext, which is not needed for saving. Replace the slow std::normaldistribution draw with a splitmix64 + box-muller normal generator. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek

b11933

vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MULMAT (#31239) vulkan: allow dim 3 broadcast in MULMAT supportsop rejected every MULMAT whose src0 and src1 differ in ne3, although mulmm, mulmmq and mulmatvec already broadcast src0 over dim 3 through broadcast3. Such a matmul then ran on the CPU with a copy of the weight, and the split layout changed

b11932

mtmd-cli: Reduced scope of batch pointer (#31155) (#31165) mtmd-cli: Reduced scope of batch pointer (#31155) mtmd-cli: Remove CLI regression test Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s39

b11930

server: add router device memory margin parameter for dynamic unloading (#21231) server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold estimate with to-be-loaded model size included use noalloc to get memory requirements for model load only set model memorymb if not previously calculated use

b11929

common : --model or -hf is required (#31242) Assisted-by: Codex Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.

b11928

model: Add MiniCPM-OCR-Jev (#30980) model : add MiniCPM-OCR-Jev support The model is a Qwen3.5-VL trunk with a value head read at the last token: it scores an OCR text against an image rendered from it, it does not generate. Add the openbmb-minicpm-ocr-jev decision type, served by POST /v1/systemone. The class scores are the pooled output of the sequence, re

b11927

metal : add GGMLOPTOPKMASK (#31230) metal : add GGMLOPTOPKMASK Port the fused top-k mask from the CUDA backend: a grid-stride kernel fills the output with -inf, then one threadgroup per row copies the mask values of the kept cells over it. Function constants drop the score, blocks and extra paths that a given graph does not use. Assisted-by: dsh:llama.cpp/gg

b11926

glm5-next : QSA without kqmask (+ DSA) (#31009) glm5-next : QSA without kqmask DSA: dsa top-k mask change tested with layers 0,3 of DeepSeek-V3.2 Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s39

b11925

ci : use ccache depend mode for ROCm release builds (#31231) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 l

b11923

ggml : relax scale to non-contiguous views (#31146) ggml : relax scale to non-contiguous views SCALE requires src0 to be 1D-contiguous. Relax this to dim-0 contiguity with arbitrary row strides (ggmliscontiguousrows), same as the unary ops in the Metal backend. ggml: relax the assert in ggmlscaleimpl cpu: address src0 rows through their byte strides (f32, bf

b11922

tests : fix multi-seq split replay and partial-state cases for trained models (#31216) tests : fix multi-seq split replay against trained models The rollback snapshot is captured by the graph that decoded the rolled-back tail, while the reference context decodes the same prefix with different batch shapes. The two states differ in the last bits and trained m

b11921

llama : remove obsolete npad from memory instances (#31200) Every caller passes npad = 1 and the only two users clamp it to 256. Drop the parameter, the members and the trivially true kvsize assert. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macO

b11919

opencl: refine mul mat id -1 index (#31170) Assisted-by: Codex Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8

b11918

opencl: initial support for q3k (#31172) opencl: add q3k AOS mv opencl: add q3k SOA mv and tiled mm opencl: add q3k noshuffle gemm/gemv for adreno opencl: add q3k dp4a gemm for adreno Co-authored-by: Hongqiang Wang opencl: tune q3k kernels for adreno Co-authored-by: Hongqiang Wang Co-authored-by: Hongqiang Wang Website: link Attestations: link macOS/iOS: mac

b11913

hexagon: add GGMLHEXAGONVTCMSIZE (#31221) hexagon: add GGMLHEXAGONVTCMMB and GGMLHEXAGONNTHREADS caps Change-Id: Idffe1613c8db6f96aa842d43d1b2efaed45462a8 hexagon: rename VTCM cap to GGMLHEXAGONVTCMSIZE, drop NTHREADS cap Change-Id: If4c5b9ce9fd6b725c3561421c2295d40599d012d hexagon: make GGMLHEXAGONVTCMSIZE an override, accept K/M/G suffixes Change-Id: Ib8ee

v0.40.12

Liquid AI LFM3 LFM3 is Liquid AI's new family of small models built for local agents, tool calling, and structured output, on Liquid's new tri-hybrid architecture. The family includes: lfm3: a 1.7B text model with a 256K-token context window lfm3-vl: adds image understanding lfm3-audio: speech recognition, translation, and text-to-speech in 18 languages lfm3

b11912

llama : enable moe amax rescaling for laguna (#31147) llama : enable moe amax rescaling for laguna Assisted-by: Devin llama : declare laguna sm tensor support Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CP

b11910

opencl: write KV-cache views in settensor so a state restore is not dropped (#30471) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64

b11909

hexagon: MULMATID row skipping optimizations and improvements (#31169) hex-mmid: vectorize skipped row zeroing and added zeroing to fused MMIDNX hex-mmid: remove scalar div Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ub

b11908

llama : move noalloc out of llamahparams (#31141) llama : move noalloc out of llamahparams Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2)) cont : clean-up cont : clean-up Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2)) tests : add a noalloc case to test-llama-context Assisted-by: dsh:llama

b11907

models : replace ggmltensor out-params with output structs (#31142) The hyper-connection and DSA graph builders wrote their extra outputs (post / comb gates, carried top-k selections, inject gates) through ggmltensor out-params. Return them in a small struct next to the primary output instead. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacB

b11906

ggml-cpu : rewrite concat as one row kernel for all types (#31179) Every dst row comes from src0 or src1, or for dim 0 from the row of src0 followed by the row of src1, so a single kernel covers every type: a dense source row is one memcpy, quantized types included, and a strided source is copied element by element with the element size known at compile time

b11905

vulkan: rms norm optimization using subgroup reductions (#29882) RMS Norm Subgroup reductions optimizaton Fix build error Improved optimization Make usage of BLOCKSIZE consistent Add fast path to skip 2nd subgroupAdd Add AMD Vendor to subgroup reduction opt Add comment clarifying rmsdict and mergemaps behavior Add unroll Rebase Fixed syntax issue Add 128/256

b11904

sycl: template the plain MMVQ launch triples (#31013) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 librarie

b11901

sycl : bound reorder temp memory to 32 MiB (#31062) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries

b11900

common: pin model resolution on real HF repo layouts (#30548) tests: pin model resolution on real repo listings Extend test-model-resolution with 63 repo listings captured verbatim from the HF API, chosen to cover every file naming pattern found across 1237 GGUF repos of the major publishers and pruned to whole quant groups. Each repo pins its resolved plans

b11899

mtmd: add mtmdhelpergetcapabilities (#31183) mtmd: add mtmdhelpergetcapabilities cap audiostream add back old api for compat audiogenspkref, textgen Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu

b11898

ggml-cpu : fix unused variables in arm arch (#31180) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries

b11897

tests : refactor test-llama-archs to not link internal libllama symbols (#30880) test-llama-archs included src/llama-arch.h and src/llama-model-saver.h, which prevented building on Windows with shared libraries since the used symbols are not exported from the DLL. declare llamaarch wrappers around the internal llm arch query functions in the llama-ext.h stag

b11896

ci : fix race in webgpu memset (#31162) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (

b11895

metal : stop overflow from happening in large matrix output offsets (#30072) Metal : stop overflow from happening in large matrix output offsets Assisted-by: Codex metal : widen batch offsets and MULMATID token indices Assisted-by: Codex Co-authored-by: feni6 <270037094+feni6@users.noreply.github.com> Website: link Attestations: link macOS/iOS: macOS Apple S

b11890

deepseekv4.1: follow-up review fixes (#31137) kv-cache : use ggmlbackendtensorcopy to refill the DSV4 K mirrors llama : move the Engram pad id to llamahparams and the token map to llamamodel The saver no longer needs the deepseek41 model class. deepseek41 : drop the F32 src1 request on the HC mixes matmul ggml : add softplus TODOs in the backends that still

b11889

cuda : use std::string instead of VLA for archName (#30463) cuda : use std::string instead of VLA for archName Modernize the code (old C style) and fix the following warnings: ggml/src/ggml-cuda/ggml-cuda.cu:180:19: warning: variable length arrays in C++ are a Clang extension [-Wvla-cxx-extension] 180 | char archName[archLen + 1]; | ^ ggml/src/ggml-cuda/ggml

b11888

common, grammar: add recursive depth limit, make it a requirement (#31153) common, grammar: add recursive depth limit, make it a requirement catch more recursive functions common : tidy up peg parser changes grammar : reject candidates in a loop, without a depth limit The depth limit in llamagrammarrejectcandidatesforstack counted the code points of a vocab

b11887

tests : adjust default threads for test-llama-context (#31157) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8

b11886

qwen: merge tensors on convert (#31028) qwen: merge tensors on convert convert: use generator functions qwen4exp: remove conts and pre-scale Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (C

b11884

llama: fix exhausting CPU and MEM caused by dry sequence breaker (#31035) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12)

b11883

llama-quant : fix 4D tensor quantization using ggmlnrows() and name-based expert detection (#30820) llama-quant : fix quantizing 4D tensors nrowstotal was ne[1]ne[2], so for a 4D tensor only the first ne[2] slices were quantized and the rest of the rows were dropped. Use ggmlnrows(tensor) to count all rows, including ne[3]. Fixes #29916 llama-quant : detect

b11882

grammar: limit recursion depth to prevent stack overflow (#29495) (#29755) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12)

b11881

llama : respect noalloc in all memory caches (#31087) llama : respect noalloc in memory caches Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) llama : TODO: move noalloc out of llamahparams Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) llama : TODO: move

b11880

tools : reject truncated GGUF tensor data in merge (#31044) Co-authored-by: Shreya Agrawal Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubun

b11879

CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows (#29948) cuda: fuse FFN gate/up and GLU in MMQ with interleaved gate/up rows For prompt processing, run the gate and up matmuls of a dense FFN as one mulmatq launch and apply the GLU in the write-back. The loaders take the gate weights as a second pointer and fill the weight tile wit

b11878

ggml-cuda: fuse qwen MRoPE + set-rows (#31025) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubunt

b11877

vulkan: Add a larger tile size for conv3d (coopmat2 only) (#29920) vulkan: Add a larger tile size for conv3d (coopmat2 only) This adds a larger tile size, in practice it is only for coopmat2 where we don't need shared memory to store the output (would use more than 64KB). Add some additional representative backend tests. At least, this is what codex came up

b11876

vulkan : one warp tile per subgroup in the small matmul tiles (#30106) WM of the small tiles was 32, or 8 for 8-wide subgroups, whatever the tile's BLOCKSIZE and WARP. With 16-wide subgroups (Mali) the second subgroup of a workgroup wrote into the next tile's columns. With 8-wide subgroups some id tiles left rows uncomputed. Derive WM so there is one warp ti

b11875

hexagon: generic dims collapse/reshape for unary, binary and matmul ops (#31071) hex-tools: support for --compare option in the profile.py hex-unary: improve collapse logic to also do splits if nrows is small hex-colapse: handle all colapsing at the host and make it generic hex-colapse: remove leftover unary helper Website: link Attestations: link macOS/iOS:

b11874

opencl: add topk and lightning indexer (#31026) opencl: add topk opencl: optimize topk opencl: add fp lightning indexer opencl: optimize lightning indexer opencl: refactor opencl: refactor tiled topk kernel opencl: adjust format Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macO

b11873

opencl: add initial q2k support (#30999) opencl: add mulmvq2kf32 opencl: add q2k SOA mv and tiled mm opencl: add q2k noshuffle gemm/gemv for adreno opencl: add q2k dp4a gemm for adreno Co-authored-by: Hongqiang Wang opencl: tune q2k kernels for adreno Co-authored-by: Hongqiang Wang Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS

b11871

chat : reject JSON nested more than 128 levels before walking it (#30242) chat : reject JSON nested more than 128 levels before walking it The schema builder, the jinja engine and the decision code walk request JSON with recursion. A request nested deep enough overflows the stack of the HTTP thread and llama-server dies (SIGBUS on macOS, SIGSEGV on Linux): f

b11870

model: DSV4.1 flash (#31086) model : add DeepSeek-V4.1-Flash (deepseek41) deepseek41 : remove MXFP8, convert the Engram table to BF16 deepseek41 : compute the compressed-KV read layers once in the cache deepseek41 : use getcells() for the shared-history check, drop llamakvcache::seqpostokle deepseek41 : gather the Engram rows in-graph like qwen4exp, drop the

b11869

jinja : fix select and selectattr argument handling (#31059) jinja : fix select and selectattr argument handling jinja : address review feedback jinja : Simplified test attr Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) U

b11868

hexagon: KV-head team split for multicore HMX flash attention (#31055) hexagon: KV-head team split for HMX flash attention in multicore hexagon: merge FAHEADSPLIT and FASPLIT into single GGMLHEXAGONFASPLIT flag 0 = off (token-split only); 1 = head-split, plus team-split when nkvheads < Ncores so stranded cores still participate (e.g. gemma-4 with 2 KV heads

b11867

server: refactor server entrypoint (#31113) serverrole cont Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 li

b11866

metal : remove mulmmid amax rescaling (#30924) The src1 power-of-two rescale in kernelmulmmid is superseded by the per expert-token amax scaling done in the compute graph (ref: link), which keeps the MoE down-projection activations within f16 range before the op runs. Drop the two-stage reduction kernels, their scratch buffer and the GGMLPRECF32 dispatch add

b11865

rpc : add bounds check in rdmarecv (#30399) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x

b11861

minimax-m3 : forward quant scales to the graph (fixes #31063) (#31097) The generic scale pass in llama-model.cpp loads the per-expert NVFP4 scale2 tensors (ffn{up,gate,down}expss) and the per-tensor MXFP8 scales for the dense FFN and the shared expert, but minimax-m3.cpp never forwarded any of them to the graph. buildmoeffn() defaults these to nullptr, so bu

b11860

tests : fix test-llama-context for large nbatch (#31083) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libra

b11858

llama : deny -sm tensor for arches with moe amax rescaling (#31081) The amax POOL1D reduction spans the whole matmul row, so a device cannot compute it from an axis-0 split. The Meta split-state case reported MIRRORED, which gives each device a partial max of its own slice. Drop that case and deny -sm tensor for the arches which build the amax graph, see [TA

b11857

ggml : allow MULMATID -1 index to skip the computation (#26631) ggml : allow MULMATID -1 index to skip the computation nits update vulkan code ggml-webgpu: ensure all threads reach workgroup barriers in mulmatidvec (cherry picked from commit d8a4b9efb6e2b9abf3a866e847756b7e6bbc6b69) fix cuda openvino exclude CUDA from this PR add TODO tag Co-authored-by: yom

b11856

tests : add speculative rejection draft sampling test (#30213) tests : add speculative rejection draft sampling test This commit adds a test for speculative rejection sampling test that excercises the speculative rejection sampling mechanism in common/speculative.cpp. The motivation for this is have a test that verifies that the speculative draft models emit

b11854

vulkan: ggml src1 precision support (#30656) vulkan: update accumulator precision selection handle fp32 overflows through scaling handle overflows in copy-to-fp16 activations add missing cases simplify code add tests add ggml prec get helpers, use in Vulkan backend clean up comments revert comment edit trim tests also handle fp32 src0, fix test scaling src0

b11853

HIP: add byte permute fallback when builtinamdgcnperm is unavailable (#30915) HIP: add byte permute fallback when builtinamdgcnperm is unavailable amdgcnspirv rejects target-specific builtins unless they are guarded by builtinamdgcnisinvocable, so 66 of 145 HIP files failed to build Wrap the perm calls in a helper that keeps vpermb32 when the target has it a

b11852

mtmd: move buildinptemporalmerge to base class (#31065) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 librar

b11851

mtmd : lfm2 resize matches torchvision (#30943) mtmd : lfm2 bilinear resize matches torchvision Lfm2VlImageProcessor and d1-omni vision.py resize uint8 images with torchvision (antialias=True). That is Pillow's algorithm, except that the weights are int16 with as many fractional bits as the largest weight allows, where Pillow uses 22 bits. The Pillow port wa

b11850

mtmd : add Soprano TTS support (#28607) Add Soprano-80M and Soprano-1.1-80M (link) as an audio generation model: a Qwen3 backbone plus a Vocos decoder that runs as a gen-only mmproj. convert: detect Soprano checkpoints and export the backbone as qwen3 and the decoder.pth Vocos weights as the mmproj vocab: add the soprano pre-tokenizer (lowercase, whitespace

b11849

hexagon: add SGN op (#30346) hexagon : add SGN support Assisted-by: OpenCode hexagon : optimize SGN with sign bit Assisted-by: OpenCode hex-sgn: add f16 hex-unary: update SGN to use generic unarycolapse and relax colapse checks to apply to more conditions hex-unary: improve dims colapsing and run it in the host-side solver Co-authored-by: Max Krasnyansky Web

b11848

common : allow concurrent downloads across processes (#29312) Signed-off-by: Adrien Gallouët Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ub

b11847

hip: fix bf16 WMMA operand vector type for gfx11/gfx12 (#30885) The builtinamdgcnwmmaf3216x16x16bf16w32 (and gfx12) builtins take their A/B operands as vectors of short (ExtVector<16, short> / ExtVector<8, short>), not bf16. Declaring the fragments as extvectortype(N) bf16 fails to compile: error: cannot initialize a parameter of type 'short attribute((extve

v0.40.11

Speech Ollama can now generate speech on macOS. ollama run plays text or a chat model's reply aloud as the audio is generated, and apps can stream audio through the OpenAI-compatible /v1/audio/speech endpoint. ollama run phonon "Hello from Ollama." d3 decision models d3 brings vLLM Semantic Router's decision models to Ollama. Give a model text, structured st

b11846

ggm-cuda: Enforce DP4A MMQ tile height multiple of warp size (#30837) Added static assertion to enforce DP4A MMQ tile height requirement. Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)

b11840

hexagon: add CEIL op (#31006) hex-ceil: add CEIL(f32) op hex-ceil: add f16 version hex-ops: update supported ops hex-ops: update docs with supported ops Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubu

v0.40.10

NVIDIA Nemotron 3.5 Bolt E4B Nemotron 3.5 Bolt E4B is NVIDIA’s smaller, Gemma 4-based vision-language model for fast local conversations, image understanding, and tool use. It supports a 128K-token context window. The model includes a DFlash draft for speculative decoding. ollama run nemotron-3.5-bolt:e4b Full Changelog: link

b11838

ggml : fix time overflow in ggmltimeus (#30969) fix time overflow use double Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 1

v0.12.5

Added 🧭 Analytics session explorer. Administrators can now open a Sessions tab in Admin Analytics that lists each chat with its tokens, responses, errors and time span, filters them by search, model, token range or sub-agents, and shows a chat's saved replies and tool calls with their timing and outcome, as long as "ENABLEADMINCHATACCESS" is on. Commit 📑 Pag

v1.4.0rc2

Python package version The Python wheels for this release are published as 1.4.0rc2.post1. Container images and documentation remain at 1.4.0rc2. Use 1.4.0rc2.post1 when installing the Python package. The wheel-only patch removes a direct Git dependency that prevented PyPI publication (#21128) and updates the Boost download mirror used during builds (#21133)

b11836

CANN: missing const qualifiers added for consistency (#30933) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8

b11835

sycl : pass device into mulmatusesonednn (#31014) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ub

b11829

server: Substitute rerank placeholder from the original template only. (#30864) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUD

b11828

server: realtime API: send complete after committed audio drains (#30994) server: realtime API: send complete after committed audio drains server: realtime API: complete a commit at its end row Completion is triggered when the stream runs dry, which is a timing proxy: audio appended while the tail drains leaks into the committed item, a final commit before t

b11827

opencl: add depthwise conv2d (#30900) opencl: add Adreno depthwise conv2d Assisted-by: OpenCode opencl: use local program for depthwise conv2d Assisted-by: OpenCode opencl: use singed type for args Co-authored-by: Li He Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (

b11826

mtmd : add MiniCPM-OCR 1.0 support (#30979) Signed-off-by: tc-mb Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12

b11823

glm5 : optimize inputs (#30932) glm5 : avoid selmask2d input cont : avoid newpoolflat input Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubu

b11822

model : fix qwen4exp layer inputs for DFlash drafters (#30927) qwen4exp published the hc-wide residual [nembd, hc, ntokens] as tlayerinp, but the drafter reads layer inputs as [nembd, ntokens] rows: llamacontext sizes each embdlayerinp buffer as nembd nbatch and the DFlash encoder input is nextract nembd wide. Evaluating a DFlash draft over a qwen4exp target

b11821

Multi-column on Intel XMX matrix engines refactored (#29864) Multi-column on Intel XMX matrix engines refactored Multi-column (2 to 80 activation columns) mulmatvecq for reordered K-quant and Q80 weights on the Intel XMX matrix engines (int8 DPAS, ESIMD), for the SYCL backend. It targets speculative decoding verification (2 to 8 tokens per step) and short pr

b11819

nemotron : support non-standard GQA, BPE tokenizer and domyn-small pre-type (#23343) nemotron : support non-standard GQA, BPE tokenizer and domyn-small pre-type gguf-py : write Metaspace prependscheme as GGUF metadata vocab : parse metaspace prependscheme into enum, rename key vocab : use metaspace prependscheme in detokenize, add TODOs for other tokenizers

b11818

server : support typed content input for RERANK endpoint (#29625) server : support typed content input for rerank endpoint Assisted-by: Opencode/Qwen3.8 27B server : address rerank multimodal review findings reject multimodal rerank input when the model has no rerank chat template (images were silently dropped) translate handlemedia download/base64 failures

b11815

sycl: fix CONV2D wrong results for non-contiguous input (#30363) sycl: fix CONV2D wrong results for non-contiguous input The kernel indexes the input as a contiguous [W, H, C, N] tensor, but supportsop claimed the op for any layout. A non-contiguous (e.g. CWHN permuted) input was read with the wrong element offsets, silently producing wrong results. Gate the

b11814

vocab : add cleanspaces GGUF key (#30940) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64

b11813

vocab : use Tekken pre-tokenizer for pixtral rather than the Llama 3 one (#30960) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (C

b11812

models : Record layer inputs for all granite models (#30957) Branch: GraniteSpeculatorSupport AI-usage: full (Claude + Sonnet 5.5) Signed-off-by: Gabe Goodhart Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (C

b11811

kleidiai : add Q80 SVE dotprod GEMV and SVE i8mm GEMM kernels (#30808) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - C

b11810

sycl: merge the fp16 and fp32 dequant switches (#30740) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 librar

b11809

webgpu : add Q20 support (#30914) webgpu : add Q20 row dequantization webgpu : add Q20 matrix multiplication Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu a

b11808

dflash : load optional FC input scale (#30945) Assisted-by: Codex Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 1

b11806

lora : support prism.hadamard folded weights (#30910) lora : support prism.hadamard folded weights tests : remove LoRA test from test-llama-archs Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s39

v0.40.9

Flint on Linux and Windows Flint now runs on Linux and Windows as well as macOS. Other improvements Streaming OpenAI Responses and Anthropic Messages requests now report generation failures instead of ending silently after partial output. ollama launch now passes cloud models' context lengths and capabilities to coding tools, fixing incorrect context and com

b11804

opencl: add F32-to-I32 ARGMAX (#30897) opencl: add F32-to-I32 ARGMAX Assisted-by: OpenCode opencl: init to 0 to be consistent with cpu Co-authored-by: Li He Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)

b11803

models : add Flint model (FlintForCausalLM) (#30928) Add Flint model (FlintForCausalLM). Assisted by Codex cleanup one more Co-authored-by: Sigbjørn Skjæret Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)

b11802

jinja : fix namespace attributes shadowed by dict methods (#30941) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA

b11801

model: Add support for SenseNova U1 model (#28919) model : add SenseNova U1 understanding support Add support for the dense SenseNova U1 understanding branch, including the U1.5-8B-MoT model. Add HF-to-GGUF conversion for NEOChatModel, excluding MoT generation tensors and concatenating temporal and spatial Q/K normalization weights. Register the sensenovau1

b11800

llama: create new fused op for QSA mask (#30640) llama: create new fused op for QSA scoring enroll glm5.3-next too tests : initialize the remaining tensors in testkpoolmask The harness sentinels were left uninitialized, so leftover memory could read as NaN and fail the NaN check. ggml: generalize the k-pool mask op into ggmltopkmask The k-pool mask op only c

b11799

sycl: q8 fast dequant (#30775) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13)

b11798

jinja: support hexadecimal string escapes (#30929) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries U

v0.40.8

Recursive Flint Flint brings Recursive's coding model to Ollama. It is designed for the work coding agents do across a repository: understanding unfamiliar code, tracing bugs, implementing features, and adding tests. Reasoning, multi-turn tool use, and a 262K-token context window support longer tasks across files and tool interactions. Flint is an Apache 2.0

b11797

ggml-opencl : add IM2COL3D support (#30895) ggml-opencl : add IM2COL3D support Assisted-by: OpenCode opencl: use same type for kernel and host opencl: fix for older adreno Co-authored-by: Li He Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux

b11796

mtmd: add state object for mtmdgen (#30902) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x

b11795

server : forward llama.cpp sampling params on /v1/messages (#30856) server : forward llama.cpp sampling params on /v1/messages server : pass the request through on /v1/messages instead of copying listed keys NIT: Change comment from multiline to single line server : accept llama.cpp params under extraparams on /v1/messages server : remove test case for trivi

b11793

llama : optimize vocabulary loading (#30679) llama : optimize vocabulary loading Update src/unicode.h Co-authored-by: Georgi Gerganov CR : added reserve in ggufkvtostr CR : trailing space fix Co-authored-by: Georgi Gerganov Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Int

b11792

llama : place d2t by its setrows use so it can stay on the GPU (#30912) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) -

b11791

metal : fix memory leaks and cleanup logic in GPU capture path (#30890) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) -

b11790

hexagon: neon-vectorized and threaded repack (tiling) (#30901) add vectorized q40 repack apply to more types hex-repack: neon-vectorize the rest of the types q1/q2/f16/... hex-repack: add simple chunking / threading Co-authored-by: hubezhou Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) D

b11789

opencl: add dp4a GEMM variants to vector load activation (#30753) opencl: add dp4a GEMM variants with vector activation loads opencl: rename kernels and env vars Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64

b11788

sycl : fix fa crash for Query head dimension mismatch (neq0 != HSK / D=2048/4096) (#30286) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubun

b11787

sycl: collapse duplicate needcheck launch bodies in mmq (#30739) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12

b11786

ggml-cpu : compute f32 geluquick and gegluquick without the fp16 table (#30737) ggml-cpu : compute f32 geluquick and gegluquick without the fp16 table ggml-cpu : keep the GGMLGELUQUICKFP16 table path ggml-cpu : add the VXE geluquick path tests : add test-gelu-quick for the f32 CPU kernels tests : drop inf/nan inputs from test-gelu-quick Website: link Attesta

b11785

llama : scale moe down projection per expert token (#30861) llama : scale moe down projection per expert token Replace the global amax rescaling for the MoE down projection with a per-expert-token factor computed by a POOL1D max reduction along the intermediate dimension. Undo the scale on the output with the same exact power-of-two factor. Handle POOL1D in

b11784

Sycl iq3 crash fix (#30604) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13) - C

b11782

server : fix usage.completiontokens undercount with n > 1 (#29486) Assisted-by: Copilot Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu

b11781

server : add POST /tts endpoint for TTS models (#26603) mtmd: add audio out stream api add /tts endpoint add docs wire up mtmdhelpermodelcanchat upload speakerref via form-data expose seed to api fix header for pcm wip fix penaltylastn fix serveroutputlimits fix pocket-tts stream wip add serverschema for tts add server routing server: coerce seed and stream

b11780

server: add realtime transcription API (#30636) server: add realtime transcription API drop transcriptionsessions proxy ws add docs server: let the API key middleware guard the websocket httplib runs the pre-routing handler before the websocket upgrade, so the API key middleware answers 401 at handshake like any other route. This covers the router proxy, who

b11778

model : support Nemotron-3(.5)-Embed (#30877) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu

b11777

qwen: reshape output to 2d instead of 3d (#30823) qwen: reshape output to 2d instead of 3d remove final reshape, fix comments Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64

b11776

server: Support logprobs with speculative decoding (#27196) return logprobs in spec path server: add regression test for logprobs with speculative decoding Asserts accepted draft tokens report the target model's logprobs, comparing a draft-enabled run against a no-draft baseline (#24271). Fails on master (toplogprobs empty, logprob 0.0), passes with the fix

b11774

mtmd : lfm2 tiles and layout match transformers (#30261) mtmd : lfm2 tiles stretch the image to the grid Lfm2VlImageProcessor.cropimagetopatches resizes the image straight to (512 rows, 512 cols) before it cuts the tiles. llama.cpp kept the aspect ratio and added black bars for LFM2 (only d1-omni stretched), so the tiles of d1-3B and every LFM2-VL / LFM2.5-V

b11773

model : add H2O-Lightning-4B decision model (#30361) model : add H2O-Lightning-4B decision model Adds the h2o-lightning decision type for h2oai/h2o-lightning-4b (Qwen3.5-4B with a merged decision LoRA). None of the existing types match this model. It answers with " A" rather than "A", uses up to 255 labels, and was trained on Python json.dumps text, so float

b11772

mtmd: fix gemma 4 ffn act (#30882) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA

b11770

ggml-cuda : support ssmconv kernel size 7 (#30873) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries U

b11769

server: Router wasn't propagating the client disconnects fully to the child, the response wait could cause a delayed cancellation (#29707) Router wasn't propagating the client disconnects fully to the child, the response wait could cause a delayed cancellation tests: satisfy router disconnect type check server : remove timing-sensitive router cancellation te

b11768

server: fix missing outputindex in Responses API SSE events for compatibility (#30863) server: include outputindex in Responses API SSE events server: reuse Anthropic streaming code for fix outputindex server: remove unnecessary generatedtoolcallids and oairesptoolcallsstarted Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Appl

b11767

mtmd : read LFM2 tiling params from GGUF metadata (#30840) mtmd : read LFM2 tiling params from GGUF metadata Write mintiles, maxtiles, tilesize and maxpixelstolerance from processorconfig.json when converting LFM2-VL mmproj, read them in clip.cpp with the previous constants as defaults. doimagesplitting=false maps to mintiles=maxtiles=1 (no tiling). Rebased

b11765

model : drop the redundant precision overrides in the Kolibri-1 router (#30865) model : drop the redundant precision overrides in the Kolibri-1 router Follow-up to the review in #30673: the router matmul works with the default precision, so both ggmlprecset calls can go away. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

b11763

opencl: add several gemm variants for small batches and some fixes (#30850) opencl: fix OOB read in mulmvmxfp4f32flat opencl: remove large-N guard that forces CPU weight placement this condition claims that large batch produces incorrect results on adreno and cannot be reproduced model loader uses N512 to probe op support, resulting in cpu fallback thus this

b11761

ggml-cpu : select the NEON fp32->fp16 conversion (#30505) The ARMNEON branch defines both GGMLCPUCOMPUTE macros but selects only FP16TOFP32, so FP32TOFP16 falls through to the portable software conversion while neoncomputefp32tofp16, a single fcvt, stays defined and referenced by nothing. POWER9VECTOR and riscvzfhmin select both. Add an #error pair that refu

b11760

HIP/CUDA: Fix reduction in mmf getting tile widths not devisiable by 64 (#30836) Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CU

b11759

ggml-zdnn: introduce flash attention op (#30512) ggml-zdnn: add scratch buffer pool Signed-off-by: Aaron Teo ggml-zdnn: implement fattn, ditch scratch slots in favour of scratch buffer pool Signed-off-by: Aaron Teo docs: add note about fattn Signed-off-by: Aaron Teo Signed-off-by: Aaron Teo Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm

b11757

CUDA: Clear CUDA graphs after legacy memory pool flush (#26771) Assisted-by: Claude Opus 5.5 Website: link Attestations: link macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ub

b11756

server : pass idslot through the Anthropic /v1/messages conversion (#28554) server : pass idslot through the Anthropic /v1/messages conversion serverchatconvertanthropictooai copies a small whitelist of common params from the Anthropic request into the internal OAI-style body and left idslot out. Every other completion endpoint honours idslot; a proxy that p

Dayjob: Tax – evaluating AI agents on 50 professional tax assignments

Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II)

Learn how OpenAI scaled Habitat with reliable multi-tenancy, global caching, and Azure Cosmos DB to deliver 12× faster reads for ChatGPT.