Sent my model to quantize and eval this morning but seems the first two quants are still in running since this morning. Are they really running? 8 hours seems a bit long.
ManniX PRO
ManniX-ITA
AI & ML interests
None yet
Recent Activity
liked a Space about 22 hours ago
Intel/low_bit_open_llm_leaderboardOrganizations
None yet
reacted to wenhuach's post with ā¤ļø about 22 hours ago
Post
1221
š We provide **free** hardware to quantize models at the [Intel Low Bit Open LLM Leaderboard]( Intel/low_bit_open_llm_leaderboard), currently supporting
ā If you find it useful, please consider starring the AutoRound project on [GitHub](https://github.com/intel/auto-round)!
Pure RTN mode powered by AutoRoundā If you find it useful, please consider starring the AutoRound project on [GitHub](https://github.com/intel/auto-round)!
- 2 replies
liked a
Space about 22 hours ago
updated a
model about 22 hours ago
Are there any plans for the qwen3.6-35B model?
2
#1 opened 1 day ago
by 
jian2023
posted an update 1 day ago
Post
90
š Gemma-4-A4B 98e v6-coder (C6v3lcb) ā LCB-targeted code prune of Gemma 4 26B-A4B, 20.8B MoE (4B-active). Same C6 recipe as v5-coder, re-steered specifically at LiveCodeBench-medium ā the one code bench pruning hurt most.
Not only keeps the lead on Python and closes the gap to 1-2pp in the other coding languages.
It's actually reasoning better, fixing the under-thinking and over-thinking failures of the full experts router.
All this comes with a cost with only 20b, on top of being very specific to coding; about 3x the thinking tokens in LiveCodeBench but it's good thinking that brings home not only more correct answers but in general a more precise and concise output.
š SCORES (Q6_K, llama.cpp, greedy, EVAL_PROTOCOL v3)
HumanEval 98.78 ā HumanEval+ 93.29 ā LCB-medium-55 v4 96.36
LCB-medium-100 96.00 ā MultiPL-E macro 88.00 (Rust/Java/JS)
MATH-500 91.00 ā GPQA-D 67.17 ā AIME 63.33 ā IFEval 92.00
vs v5-coder: +10.91 LCB-medium / +7.0 MultiPL-E / +10 AIME, HE+ tie
LCB targeting closed the ā9.10pp hole and pushed +1.81pp past the unpruned 128e. Top of the 14ā22B coder band: +9.2pp HE over Qwen2.5-Coder-14B-Instruct (89.6 ā 98.78).
š¦ GGUF SWEEP (all imatrix; Q4_K_M plain ā imatrix hurt it)
Q6_K ā 17.81 GB ā 93.29% (cohort top)
Q3_K_M ā 10.51 GB ā 92.68% ā value leader (imatrix lifted the 3-bit tiers hard)
IQ4_XS ā 11.01 GB ā 92.07% ā safe 4-bit
IQ3_XS ā 9.22 GB ā 92.07% ā smallest on the plateau
IQ2_S ā 7.83 GB ā 89.02% ā sub-8 GB code-grade
āļø SAME-RIG vs Qwen2.5-Coder-14B (RTX 3090, greedy)
Iso-disk 10.5 GB: Q3_K_M 92.68 vs Qwen Q5_K_M 83.54 ā +9.14pp at the same file size
LCB-medium-55 v4, identical split: 96.36 vs 18.18
bf16:
ManniX-ITA/gemma-4-A4B-98e-v6-coder-it
GGUF:
ManniX-ITA/gemma-4-A4B-98e-v6-coder-it-GGUF
Ollama:
https://ollama.com/mannix/gemma4-98e-v6-coder
Not only keeps the lead on Python and closes the gap to 1-2pp in the other coding languages.
It's actually reasoning better, fixing the under-thinking and over-thinking failures of the full experts router.
All this comes with a cost with only 20b, on top of being very specific to coding; about 3x the thinking tokens in LiveCodeBench but it's good thinking that brings home not only more correct answers but in general a more precise and concise output.
š SCORES (Q6_K, llama.cpp, greedy, EVAL_PROTOCOL v3)
HumanEval 98.78 ā HumanEval+ 93.29 ā LCB-medium-55 v4 96.36
LCB-medium-100 96.00 ā MultiPL-E macro 88.00 (Rust/Java/JS)
MATH-500 91.00 ā GPQA-D 67.17 ā AIME 63.33 ā IFEval 92.00
vs v5-coder: +10.91 LCB-medium / +7.0 MultiPL-E / +10 AIME, HE+ tie
LCB targeting closed the ā9.10pp hole and pushed +1.81pp past the unpruned 128e. Top of the 14ā22B coder band: +9.2pp HE over Qwen2.5-Coder-14B-Instruct (89.6 ā 98.78).
š¦ GGUF SWEEP (all imatrix; Q4_K_M plain ā imatrix hurt it)
Q6_K ā 17.81 GB ā 93.29% (cohort top)
Q3_K_M ā 10.51 GB ā 92.68% ā value leader (imatrix lifted the 3-bit tiers hard)
IQ4_XS ā 11.01 GB ā 92.07% ā safe 4-bit
IQ3_XS ā 9.22 GB ā 92.07% ā smallest on the plateau
IQ2_S ā 7.83 GB ā 89.02% ā sub-8 GB code-grade
āļø SAME-RIG vs Qwen2.5-Coder-14B (RTX 3090, greedy)
Iso-disk 10.5 GB: Q3_K_M 92.68 vs Qwen Q5_K_M 83.54 ā +9.14pp at the same file size
LCB-medium-55 v4, identical split: 96.36 vs 18.18
bf16:
ManniX-ITA/gemma-4-A4B-98e-v6-coder-it
GGUF:
ManniX-ITA/gemma-4-A4B-98e-v6-coder-it-GGUF
Ollama:
https://ollama.com/mannix/gemma4-98e-v6-coder
updated 3
models 2 days ago
published a
model 2 days ago
published a
model 3 days ago
updated a
model 3 days ago
Can I use MTP?
1
#1 opened 4 days ago
by jian2023
updated 2
models 5 days ago
posted an update 5 days ago
Post
170
š Gemma-4-A4B 98e v5-coder ā code-leaning 20.8B MoE (4B-active), C6 layer-relevance-weighted prune of Gemma 4 26B-A4B. Best 20B-class coder I've shipped.
š SCORES (NVFP4A16, vLLM 0.20.2, greedy, EVAL_PROTOCOL v3)
HumanEval 98.17 ā HumanEval+ 92.68 ā LCB-medium-55 v4 85.45
MATH-500 92.00 ā GPQA-D 68.69 ā IFEval 94.00
vs v4: +1.22 HE / +1.22 HE+ / +7.27 LCB-medium
Top of the 14ā22B coder band: +8.6pp HE over Qwen2.5-Coder-14B-Instruct (89.6 ā 98.17). HE+ sanity-audited ā no memorization, no silent-empty.
š¦ EXTENSIVE GGUF SWEEP (16 plain + IQ tiers + 5 CD recipes, all imatrix-calibrated)
Q8_0 ā 21.16 GB ā 93.90% (cohort top)
Q4_K_S ā 12.21 GB ā 93.29% ā plain sweet spot
IQ4_XS ā 11.01 GB ā 93.29% ā sub-12 GB top
ā TWO EXCELLENT SUB-10 GB CONTRIBDYNAMIC CD PICKS (per-layer + IQ-codebook overrides)
CD-IQ4_K_M (Canary W) ā 10.29 GB ā 92.07% ā recommended sub-11 GB
CD-IQ3_XS_L ā 9.27 GB ā 90.24% ā smallest viable code-grade
āļø SAME-RIG vs Qwen2.5-Coder-14B-Instruct (RTX 3090, greedy HE+)
11 GB band: v5-coder IQ4_XS wins +9.75pp at -1.49 bpw
12 GB band: Q4_K_S wins +8.53pp
8 GB band: IQ2_S wins +0.61pp at lower bpw
bf16:
ManniX-ITA/gemma-4-A4B-98e-v5-coder-it
GGUF:
ManniX-ITA/gemma-4-A4B-98e-v5-coder-it-GGUF
NVFP4A16:
ManniX-ITA/gemma-4-A4B-98e-v5-coder-NVFP4A16
Ollama:
https://ollama.com/mannix/gemma4-98e-v5-coder
āāā
š BONUS ā Qwen3.6-27B-Omnimerge-v4-MTP-GGUF
Same v4 weights with the native MTP head retained for llama.cpp speculative decoding (PR #22673, --spec-type draft-mtp). 7 imatrix tiers Q8_0 ā IQ2_M.
HumanEval: 2.0x decode tok/s
MBPP: 2.33x decode tok/s
Both at +1-2pp pass@1 vs the non-MTP build. GPQA Diamond comparison in flight.
MTP-GGUF:
ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF
š SCORES (NVFP4A16, vLLM 0.20.2, greedy, EVAL_PROTOCOL v3)
HumanEval 98.17 ā HumanEval+ 92.68 ā LCB-medium-55 v4 85.45
MATH-500 92.00 ā GPQA-D 68.69 ā IFEval 94.00
vs v4: +1.22 HE / +1.22 HE+ / +7.27 LCB-medium
Top of the 14ā22B coder band: +8.6pp HE over Qwen2.5-Coder-14B-Instruct (89.6 ā 98.17). HE+ sanity-audited ā no memorization, no silent-empty.
š¦ EXTENSIVE GGUF SWEEP (16 plain + IQ tiers + 5 CD recipes, all imatrix-calibrated)
Q8_0 ā 21.16 GB ā 93.90% (cohort top)
Q4_K_S ā 12.21 GB ā 93.29% ā plain sweet spot
IQ4_XS ā 11.01 GB ā 93.29% ā sub-12 GB top
ā TWO EXCELLENT SUB-10 GB CONTRIBDYNAMIC CD PICKS (per-layer + IQ-codebook overrides)
CD-IQ4_K_M (Canary W) ā 10.29 GB ā 92.07% ā recommended sub-11 GB
CD-IQ3_XS_L ā 9.27 GB ā 90.24% ā smallest viable code-grade
āļø SAME-RIG vs Qwen2.5-Coder-14B-Instruct (RTX 3090, greedy HE+)
11 GB band: v5-coder IQ4_XS wins +9.75pp at -1.49 bpw
12 GB band: Q4_K_S wins +8.53pp
8 GB band: IQ2_S wins +0.61pp at lower bpw
bf16:
ManniX-ITA/gemma-4-A4B-98e-v5-coder-it
GGUF:
ManniX-ITA/gemma-4-A4B-98e-v5-coder-it-GGUF
NVFP4A16:
ManniX-ITA/gemma-4-A4B-98e-v5-coder-NVFP4A16
Ollama:
https://ollama.com/mannix/gemma4-98e-v5-coder
āāā
š BONUS ā Qwen3.6-27B-Omnimerge-v4-MTP-GGUF
Same v4 weights with the native MTP head retained for llama.cpp speculative decoding (PR #22673, --spec-type draft-mtp). 7 imatrix tiers Q8_0 ā IQ2_M.
HumanEval: 2.0x decode tok/s
MBPP: 2.33x decode tok/s
Both at +1-2pp pass@1 vs the non-MTP build. GPQA Diamond comparison in flight.
MTP-GGUF:
ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF
MTP version
3
#1 opened 6 days ago
by 
DzmitryTheOtherOne
updated a
model 5 days ago
published a
model 5 days ago
updated a
model 6 days ago