This website requires JavaScript.
ee7c30578a
Update WebGPU support and add link to blog/demo (#23483 )
2026-05-21 11:00:27 -07:00
47c0eda9d4
vulkan: fuse snake activation (mul, sin, sqr, mul, add) (#22855 )
2026-05-21 19:39:42 +02:00
5306f4b3b5
fix(flash-attn): replace f32 with kv_type and q_type (#23372 )
2026-05-21 10:58:49 -04:00
40d5358d3c
tests : move save-load-state from examples to tests (#23336 )
2026-05-21 14:41:50 +03:00
b65bb4baae
server: expose prompt token counts in /slots endpoint (#23454 )
2026-05-21 13:29:13 +02:00
a1a69f777a
metal : optimize concat kernel and fix set kernel threads (#23411 )
2026-05-21 13:34:08 +03:00
52fb93a2bd
server : free draft/MTP resources on sleep to fix VRAM leak (#23461 )
2026-05-21 16:11:11 +08:00
c9021714e8
server: re-inject subcommand when router spawns children under unified binary (#23442 )
2026-05-21 10:09:19 +02:00
1d7ab2b947
app : add batched-bench, fit-params, quantize & perplexity (#23459 )
2026-05-21 09:29:44 +02:00
12e5d99078
mtp: use inp_out_ids for skipping logit computation (#23433 )
2026-05-21 15:23:14 +08:00
7ea23ddf7b
vocab : add Carbon-3B (HybridDNATokenizer) support (#23410 )
2026-05-21 08:34:32 +02:00
2fc8d1851e
doc: fix spec mtp typo (#23435 )
2026-05-21 08:30:55 +02:00
5e932a1c8d
ui: Improve Git Hooks for UI development (#23403 )
2026-05-21 08:27:50 +02:00
2754ce1b3e
ggml : Check the right iface method before using the fallback 2d get (#23306 )
2026-05-21 06:24:40 +00:00
eeeaf6180b
llama-graph: fix null-buffer crash in llm_graph_input_attn_kv_iswa for SWA-only models (#23131 )
2026-05-20 23:20:51 -07:00
0be84685bd
hexagon: ssm-conv fix for large prompts (#23307 )
2026-05-20 22:14:13 -07:00
ce02093fdd
app : show version (#23426 )
2026-05-21 06:21:13 +02:00
6a257d4463
mtmd, model : merge HunyuanOCR into HunyuanVL and fix OCR vision precision (#23329 )
2026-05-21 06:35:37 +08:00
3a479c9132
ui: Add max image size option (#22849 )
2026-05-21 00:00:09 +02:00
ad27757261
Move to backend sampling for MTP draft path (#23287 )
2026-05-20 22:34:45 +05:30
3a6db741a8
opencl: refactor backend initilization (#23318 )
2026-05-20 09:57:36 -07:00
510b5c2a35
common/speculative : fix nullptr crash in get_devices_str (#23386 )
2026-05-20 19:44:30 +03:00
a8681a0ed2
mtmd : DeepSeek-OCR image processing fixes, img_tool::resize padding refactor (#23345 )
2026-05-20 17:37:10 +02:00
acd604fb27
vulkan: optimize operations in the IM2COL shader (#22685 )
2026-05-20 17:15:13 +02:00
6ce96713de
feat: Add WAV MIME type variants and improve audio format detection (#23396 )
2026-05-20 16:55:24 +02:00
c9872a2575
hexagon: HMX quantized matmul rework (#23368 )
2026-05-20 07:39:01 -07:00
e947228222
Programmatic Dependent Launch (PDL) for more performance on newer NVIDIA GPUs (Hopper+) (#22522 )
2026-05-20 13:59:02 +02:00
29f1482221
app : introduce the llama unified executable (#23296 )
2026-05-20 13:22:22 +02:00
e6b4acfe86
refactor: Move text attachments up before the message content in chat completions payload (#23406 )
2026-05-20 13:04:01 +02:00
e2b129e1bf
mtmd: fit_params now take into account mmproj (#21489 )
2026-05-20 11:27:44 +02:00
7e50ef7d79
docker : copy conversion files (#23370 )
2026-05-20 11:03:18 +02:00
5028447384
ui: Refactor isMobile as reactive value in viewport store (#23330 )
2026-05-20 10:52:00 +02:00
585080d310
fix: Div wrapper no pointer events on hidden (#23390 )
2026-05-20 09:46:31 +02:00
57ebaf4edd
metal : optimize pad + cpy (#23354 )
2026-05-20 09:42:00 +03:00
871b0b70f8
snapdragon: update toolchain to v0.6 (#23369 )
2026-05-19 22:04:04 -07:00
b39a7bf1b0
ggml-cuda: tune RDNA3 Q6_K MMVQ nwarps (#23349 )
2026-05-20 03:52:21 +02:00
b28a2f372a
opencl: add MoE support for q4_k, q5_k, q6_k on Adreno (#23303 )
2026-05-19 14:29:00 -07:00
17d22a35b2
hexagon: add MROPE and IMROPE support in HTP rope op (#23317 )
2026-05-20 02:40:13 +05:30
67ace021da
refactor: Chat Screen UI rendering (#23333 )
2026-05-19 22:38:42 +02:00
a8078675a6
github: mention --log-file in issue templates (#23277 )
2026-05-19 21:35:10 +02:00
57cb35c886
common: fix --help for --verbosity (#23278 )
2026-05-19 21:34:04 +02:00
7256fce047
common: fix --fit verbosity with --verbosity 4 (#23282 )
2026-05-19 21:33:23 +02:00
b7393a4d19
convert : update mtp related help (#23334 )
2026-05-19 21:16:58 +02:00
81eff6a45c
Add includes for algorithm (#16 )
2026-05-19 19:19:18 +02:00
ac76808e4d
hexagon: enable support for NORM op (#23319 )
2026-05-19 22:18:21 +05:30
baf3cc6e1d
model : clarify MTP layer comment in qwen35.cpp [no ci] (#23338 )
2026-05-19 18:41:44 +02:00
d14ce3dab4
llama : MTP clean-up (#23269 )
2026-05-19 15:32:58 +03:00
6db130445d
ui: Bump packages + address build warnings (#23300 )
2026-05-19 10:16:04 +02:00
4b262ab662
ci : install libssl-dev (#23325 )
2026-05-19 10:11:04 +02:00
00c461ce1a
ci : install server kleidiai runner dependencies (#23259 )
2026-05-19 09:06:56 +02:00
ccee426426
server-context: guarantee there is at least 1 token to decode (#23280 )
2026-05-19 08:49:01 +02:00
3c81c8deea
server : print graphs reused in slot timings (#23279 )
2026-05-19 09:46:58 +03:00
cd963fee6a
save-load-state : refactor tests and improve readability (#23196 )
2026-05-19 09:46:34 +03:00
d2e179a477
llama-eval : add per-task summary stats (#23151 )
2026-05-19 09:46:05 +03:00
c85a242ed0
ggml-webgpu : extend GDN for K>1 (#23299 )
2026-05-18 23:45:41 -07:00
aabee047d8
[SCYL] add chapter for performance reference in SYCL.md (#23315 )
2026-05-19 14:44:51 +08:00
f1c1c5c057
convert : filter lora tensor names (#23077 )
2026-05-19 08:44:25 +02:00
439f1b193d
sycl: add GGML_SYCL_USE_ASYNC_MEM_OP env toggle (#22153 )
2026-05-18 23:44:02 -07:00
c3e9ade6dd
rpc : keep last_graph_uid in the device context (#23273 )
2026-05-19 09:42:36 +03:00
9a532ae4ba
hexagon: add support for TRI op (#22822 )
2026-05-18 14:04:57 -07:00
b7340443d4
ggml-hexagon: add PAD op HVX kernel (#23078 )
2026-05-18 13:39:36 -07:00
5cbaa5e69e
docker : add OCI image labels for version and build date (#21653 )
2026-05-18 15:14:45 -05:00
45b455e66f
common : remove hf cache migration (#23266 )
2026-05-18 17:11:47 +02:00
3a9c1b854d
ui: Update KaTeX package and clean up logs from sass warnings (#23275 )
2026-05-18 16:26:01 +02:00
b9a2170fce
feat: add scroll-to-bottom button to chat + prevent forced scroll down (#23270 )
2026-05-18 16:17:21 +02:00
1ff0fc1384
ui: Refactor models store, MCP service, and gate logs behind VITE_DEBUG (#23236 )
2026-05-18 16:09:40 +02:00
a135ec0baa
ui: Centralize monospace font styles in app.css (#23272 )
2026-05-18 15:10:14 +02:00
232f466583
webui: fix Tailwind v4 utility classes missing when built via cmake (#23253 )
2026-05-18 14:08:02 +02:00
49c21f97cd
llama: initialize pre-norm embedding mask flag (#23256 )
2026-05-18 04:20:49 -07:00
77e38d68f2
add myself to conversion (#23261 )
2026-05-18 12:42:56 +02:00
053e01dff6
ci : added kleidiai-server to server-self-hosted workflow (#22435 )
2026-05-18 10:14:57 +01:00
c3f95c1f06
scripts : allow wc2wt with an existing branch (#23189 )
2026-05-18 08:57:28 +03:00
0caf2a1d48
sycl: scalar SWAR byte-subtract in Q6_K MMVQ dot product (#22156 )
2026-05-17 22:12:21 -07:00
5511965b19
sycl: route small f32 matmuls to oneMKL, bypass oneDNN (#22150 )
2026-05-17 22:11:51 -07:00
e98bcfec28
sycl : fix error when use -mg 1 error (#23140 )
2026-05-18 13:11:19 +08:00
1867a0c692
update bid to match each layers MTP source (#23237 )
2026-05-17 21:37:12 -07:00
dd7cad7197
cmake : do not check for bin install dir (#23234 )
2026-05-18 02:33:14 +02:00
726704a160
feat: Support d_conv=15 for ssm-conv.cu (#23017 )
2026-05-17 15:05:11 -06:00
87589042ca
cmake : fix LLAMA_BUILD_UI logic (#23190 )
2026-05-17 14:42:26 -04:00
e0de4c2419
cmake : do not install conversion script (#23204 )
2026-05-17 18:07:21 +02:00
84c678242a
CUDA: Continue directly including cuda/iterator (#23102 )
2026-05-17 18:00:10 +02:00
3e12fbdea5
llama: avoid copying logits during prompt decode in MTP (#23198 )
2026-05-17 23:30:25 +08:00
39cf5d6191
common : delegate assistant continuation to underlying template handlers (#23089 )
2026-05-17 07:36:05 -04:00
a6d6183dbc
ggml-vulkan/CMakeLists: add a check for SPIRV-Headers (#22009 )
2026-05-17 14:12:11 +03:00
fcae601e44
vulkan: add cpy bf16 -> f32 pipelines (#22677 )
2026-05-17 11:31:20 +02:00
7ba22c6a09
vulkan: Support unaligned tensors for ROPE (#22637 )
2026-05-17 04:30:16 -05:00
f4cc787b9f
common : enable streaming JSON argument values (#23173 )
2026-05-17 04:44:34 -04:00
3fbadb06dc
vulkan: fuse SSM_CONV + BIAS + SILU (#22653 )
2026-05-17 03:25:50 -05:00
1a68ec9378
server : honor --embd-normalize CLI arg (#23125 )
2026-05-16 23:39:04 -07:00
a16cce81d3
ngram : reduce noisy logs (#23185 )
2026-05-17 01:38:17 -05:00
4f13cb7424
webui: support video files as input (#22830 )
2026-05-17 08:13:44 +08:00
b64739ea39
server: (router) alloc tmp buffer on heap (#23159 )
2026-05-16 23:42:16 +02:00
64b38b561b
server: skip device enumeration in router mode to avoid creating CUDA primary context (#23137 )
2026-05-16 21:21:06 +02:00
6049906133
vulkan: removed duplicate #include <memory> in headers (#23144 )
2026-05-17 01:57:35 +08:00
0253fb21f5
ui: Add request timeout for MCP tool calls (#23138 )
2026-05-16 15:20:27 +02:00
3a92bc99db
sync : ggml
2026-05-16 15:59:45 +03:00
e6c37a1adc
ggml : bump version to 0.12.0 (ggml/1494)
2026-05-16 15:59:09 +03:00
560445bf34
metal : tighten input-position loop in kernel_conv_transpose_1d (ggml/1477)
2026-05-10 16:45:00 +02:00
2eb3e6b242
ggml: install ggml.pc in <libdir>/pkgconfig (ggml/1480)
2026-05-10 16:35:38 +02:00
25b1bc9c2f
ui: Correct links in tools/ui/README.md [no ci] (#23139 )
2026-05-16 14:42:38 +02:00
Reese Levine