This website requires JavaScript.
396159c9d5
ggml-rknpu2: graceful abort with W8A8 hint on NPU IOVA exhaustion
2026-07-20 04:21:49 +02:00
e76bfa0237
docs: scrub internal hostname from campaign results
2026-07-19 20:04:06 +02:00
9d64138d24
arg: don't assert n_gpu_layers < 0 for accel-type backends
2026-07-19 17:56:40 +02:00
3ee6ce283e
docs: add Rockchip RK3588 NPU section to top-level README
2026-07-19 10:50:37 +02:00
f574ed927c
ggml-cpu/repack: ROCKET_NOREPACK_TENSORS env to force tensors off CPU_REPACK
2026-07-18 19:16:40 +02:00
67316774a0
ggml-rocket: make ROCKET_MIN_BATCH runtime-tunable via GGML_ROCKET_MIN_BATCH
claude-noether
2026-07-18 15:21:55 +02:00
6d5a075a6c
ggml-rocket: K-segmentation for K>8192 (int8-partial float-accumulate)
2026-07-14 22:52:59 +02:00
288d70413a
ggml-rocket: multi-fd (3-core) column-split + task_num param
2026-07-14 13:25:51 +02:00
412cba2601
ggml-rocket: CBUF weight-reuse via multi-task job chaining
2026-07-14 13:13:33 +02:00
367143edf3
ggml-rocket: implement offload_op; add GGML_NO_REPACK measurement guard
2026-07-14 12:36:39 +02:00
3d395c1612
ggml-rocket: per-call BO pool + column-major weight caching (prefill 39->49 t/s)
2026-07-14 12:15:19 +02:00
698c6b731c
ggml-rocket: min_batch floor — keep M<32 (decode) on CPU
2026-07-14 11:45:05 +02:00
5550257f4c
ggml-rocket: fix Fable review blockers (mmap leak, scale aliasing, K backstop)
2026-07-14 11:31:47 +02:00
8e49f43486
ggml: add rocket NPU backend (RK3588 mainline accel driver)
2026-07-14 10:52:06 +02:00
0ebb855635
docs: int4 closed — no weight-only int4 dtype on RK3588 (hardware limit)
2026-07-10 16:58:33 +02:00
2635371317
docs: E8 prefill pipeline result — flat, DDR-bandwidth-bound
2026-07-10 16:24:40 +02:00
7fbbd7acc3
Merge rknpu-int8-clamp: saturate int8 activation quantization
2026-07-10 12:17:32 +02:00
f7449d2698
docs: RKNPU2 perf campaign results + matmul micro-benchmarks
2026-07-10 12:16:24 +02:00
447672d647
rknpu2: saturate int8 activation quantization
2026-07-10 09:01:57 +02:00
3cd6e67b55
docs: remove local infrastructure references from gemma4-optimized.md
2026-07-10 07:26:03 +02:00
3258224a57
docs: re-measure CPU/NPU baselines on an idle host
2026-07-10 07:13:39 +02:00
5c66767159
docs: gemma4-optimized.md — NPU serving config, models, measurements
2026-07-10 06:53:21 +02:00
02a883fea0
Merge tag 'refs/tags/b9549' into mtp-b9549
2026-07-09 23:24:17 +02:00
137b4ceebe
rknpu2: add Q4_K dequant + W4A4_HADAMARD default pattern
2026-07-09 23:24:08 +02:00
05cf2f9dd5
E5 NEON-vectorize C-collect (fp32/int32/int16)
2026-07-09 01:52:20 +02:00
3705e84765
rknpu2: hoist per-row A-prep scratch to per-thread pools
2026-07-09 01:37:12 +02:00
8adda8448c
rknpu2: guard OMP parallel regions with if(M>=8) for decode
2026-07-09 01:25:39 +02:00
c5a47492ff
rknpu2: skip dst memset + RMW when single K-segment
2026-07-09 00:59:39 +02:00
04d2c6e548
rknpu2: M-bucketing — ceil-to-128 for M>128 instead of next_power_of_two
2026-07-09 00:38:02 +02:00
39622ccb32
rknpu2: add 2d (a)sync backend iface NULL slots for newer ggml
2026-07-08 23:57:15 +02:00
aecc8fbc6e
Merge tag 'b9101' into dspark
2026-07-08 21:00:48 +00:00
04eb4c446d
llama : add Gemma4 MTP (#23398 )
2026-06-07 20:50:54 +08:00
8a091c47ab
spec : fix vocab compatibility check (#24256 )
2026-06-07 13:43:52 +02:00
465b1f0e75
arg: Skip mmproj download when user supplied mmproj (#24239 )
2026-06-07 11:18:44 +02:00
f71af352a5
convert : fix Gemma4 with no audio encoder (#24242 )
2026-06-07 08:43:05 +02:00
3f7c79d7b5
docker : bump cuda13 to 13.3.0 (#24228 )
2026-06-07 08:31:58 +02:00
98d5e8ba8a
common/chat : fix LFM2/LFM2.5 reasoning round-trip and <think> leak (#24234 )
2026-06-06 22:39:21 +02:00
31e82494c0
mtmd: support "frame merge" for qwen-vl-based models (#21858 )
2026-06-06 21:17:25 +02:00
6b80c74f28
completion : remove useless statics (#24226 )
2026-06-06 12:16:16 +02:00
588f0dc2ce
completion : fix format specifier in LOG_INF (#24213 )
2026-06-06 11:24:27 +02:00
f5c6ae1827
mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913 )
2026-06-06 11:06:51 +02:00
5a69c97439
vulkan: check coopmat2 features before reporting support (#24186 )
2026-06-06 09:11:35 +02:00
5343f4502a
model : rename local n_layer_all variable (#24209 )
2026-06-06 06:07:20 +02:00
603300b008
context : fix off-by-one comparisons to n_gpu_layers (#24208 )
2026-06-06 06:06:47 +02:00
308f61c31f
opencl: improve get_rows, cpy, concat and q6_k flat gemv (#24160 )
2026-06-05 13:45:25 -07:00
da87e9b612
common/chat : unify and fix LFM2/LFM2.5 tool parser (#24178 )
2026-06-05 21:31:56 +02:00
e82beaa60d
vulkan: add fwht support for Intel with shmem reduction (#23964 )
2026-06-05 19:44:40 +02:00
c4a278d68e
model: fix build failed (#24193 )
2026-06-05 18:12:27 +02:00
64086f2b2f
model, mtmd: Granite4 Vision (#23545 )
2026-06-05 09:44:59 -06:00
6effcecd0b
TP: round up granularity to 128 (#24180 )
2026-06-05 17:35:13 +02:00
86591c7536
cli: fix model params not propagated (#23893 )
2026-06-05 08:29:41 -07:00
96fbe00393
model : fix llama_model::n_gpu_layers() (#24188 )
2026-06-05 17:11:42 +03:00
2016bf2b3b
ui: run npm install when package-lock.json is newer than node_modules (#24171 )
2026-06-05 14:57:32 +02:00
9c955c48b0
Fix link to available UI settings (#24169 )
2026-06-05 13:39:32 +01:00
cc7bef34e2
ui: add ignore-scripts=true to npmrc (#24149 )
2026-06-05 14:31:03 +02:00
ad1b88ca0d
docs: Update quantization readme (#24133 )
2026-06-05 12:21:26 +02:00
59917d3922
minor : fix lint issues (#24165 )
2026-06-05 11:17:54 +03:00
7acb4e8cd2
hparams : refactor hparams.n_layer (#24060 )
2026-06-05 11:09:36 +03:00
3ecfb150a4
kleidiai : dynamic chunck-based scheduling for hybrid execution (#23819 )
2026-06-05 09:11:47 +02:00
2154a0fdcf
CUDA: enroll mul_mat_vec_q_moe into pdl (#24087 )
2026-06-05 08:37:34 +02:00
46fa662b1f
ci : build-msys job slimming [no ci] (#24157 )
2026-06-05 07:57:36 +02:00
7fe2ae45ab
sycl : port multi-column MMVQ from CUDA backend (#21845 )
2026-06-05 01:10:31 -04:00
7c158fbb4a
server : disable on-device spec checkpoints (#24108 )
2026-06-04 19:30:59 +03:00
260862b8ca
arg: fix double mtp downloads (#24128 )
2026-06-04 18:23:48 +02:00
42b2d60e57
webui: [a11y] fix keyboard navigation issues in chat interface and sidebar (#23132 )
2026-06-04 08:59:00 -07:00
e7bcf1c3a8
Move duplicated imatrix code into single common imatrix-loader.cpp (#22445 )
2026-06-04 11:45:40 -04:00
21444c822e
ui: Fixed packages (#24119 )
2026-06-04 16:23:08 +02:00
526977068f
ui: added single line reasoning preview (#23601 )
2026-06-04 21:09:43 +07:00
0dbfa66a1f
return filter to save memory (#24125 )
2026-06-04 21:56:33 +08:00
e8023568d0
convert: Fix Gemma 4 Unified conversion (#24118 )
2026-06-04 15:21:38 +02:00
4c51309617
ggml: vectorize ggml_vec_dot_q4_1_q8_1 with WASM SIMD128 (#22209 )
2026-06-04 18:42:38 +05:30
6f3a9f3dee
server: avoid unnecessary checkpoint restore when new tokens are present (#24110 )
2026-06-04 21:09:01 +08:00
a121232fdc
agents: refactor, include more guidelines (#24111 )
2026-06-04 13:40:23 +02:00
4586479852
webui: fix tool selector toggle/counter, key tools by stable identity (#24065 )
2026-06-04 13:09:49 +02:00
4d742877b2
build : use umbrella Headers directory for XCFramework module map (#23974 )
2026-06-04 03:58:25 -07:00
0066404085
server : add header to tools/server/server-http.h (#24089 )
2026-06-04 05:14:46 -05:00
7ac5a4225e
cmake: skip cvector-generator and export-lora when CPU backend is disabled (#24053 )
2026-06-04 04:13:19 -06:00
e3ba22d6cc
fix(mtmd): handle Gemma 4 audio projector embedding size (#24091 )
2026-06-04 02:51:23 -07:00
6ddc9430b1
readme : add status badges (#24104 )
2026-06-04 10:58:13 +03:00
65ef50a0a4
tests : refactor test-save-load-state to accept token input (#24073 )
2026-06-04 08:06:36 +03:00
3d1998634e
metal : reduce rset heartbeat from 500ms -> 5ms (#24074 )
2026-06-04 08:05:32 +03:00
e8c54893f2
ggml-webgpu: FlashAttention refactor + standardize quantization support (#23834 )
2026-06-03 22:05:04 -07:00
3c7450cee1
ggml-cpu: extend RVV quantization vec dot to higher VLENs (#22754 )
2026-06-04 10:03:40 +05:00
f478f1b6d7
sycl : Improve SYCL doc (#23025 )
2026-06-03 22:02:54 -07:00
94a220cd67
mtmd: fix Gemma 4 unified FPE (#24088 )
2026-06-03 12:51:18 -07:00
166fe29492
qwen35: use post-norm hidden state for MTP (#24025 )
2026-06-04 01:29:09 +08:00
c8d6a00636
mtmd: enable non-causal vision for gemma 4 unified (#24082 )
2026-06-03 19:05:17 +02:00
a731805ced
mtmd, model: allow skip build_vit() (#24077 )
2026-06-03 17:10:35 +02:00
ee4cf705bb
ui: Mermaid Diagrams in chat + interactive preview (#24032 )
2026-06-03 16:55:36 +02:00
9e58d4d692
Avoid PDL race conditions by disabling __restrict__ when PDL is used (#24030 )
2026-06-03 13:56:42 +02:00
3571fa5435
ggml-cpu: use runtime SVE width in FWHT (#24059 )
2026-06-03 12:45:10 +02:00
f8f0a47a55
cuda: reserve space for quantize kv-cache at startup (#23907 )
2026-06-03 18:39:59 +08:00
06938ac129
tests : add support for qwen3 SSM archs (#24031 )
2026-06-03 10:15:27 +03:00
d545a2a993
update BoringSSL to 0.20260526.0 (#23794 )
2026-06-03 02:42:58 -03:00
4da6370d43
ci : disable ccache for msvc windows release jobs (#23911 )
2026-06-03 08:05:21 +03:00
e3666269f9
arg : removed unecesary mmproj download when users pass --no-mmproj (#23425 )
2026-06-02 22:04:46 -07:00
63e66fdd23
opencl: use flat variants of q4_K and q6_K gemv for very large M (#24006 )
2026-06-02 14:16:17 -07:00
5c394fdc8b
hexagon: profiler output fix and script updates (#24042 )
2026-06-02 14:08:29 -07:00
4fb16eccce
model: add Mellum architecture (#23966 )
2026-06-02 21:11:12 +02:00
bfb4308b05
model : support granite multilingual embeddings R2 (ibm-granite/granite-embedding-{97,311}m-multilingual-r2) (#22716 )
2026-06-02 11:55:11 -04:00
mfritsche
claude-noether