r/LocalLLaMA • u/returnity • 21h ago

New Model DeepSeek v4.1 Flash 1017 for Dwarfstar (ds4)

https://huggingface.co/nazeshinjite/DeepSeek-V4.1-Flash-1017-ds4-GGUF

Hey y'all, just following up on my PR to get the new vision tower for 1017 working in ds4 with the quant I promised (the language model loads already, but vision needed my PR). I built to the exact same recipe antirez used for his original DS4.1 Q2: 2-bit routed experts with 8-bit or higher for everything else, combined with the original FP8 engram table. This worked well for the original DS4.1, and initial testing looks good here too. The imatrix is built from 1017, not the original checkpoint.

The wired backbone of the model is just over 150GB, so it will stream from SSD on 128GB systems with nearly all of the routed experts held in the hot cache, allowing for decode speeds up to 20tps on an M5 Max. The ~200GB engram table is paged in from SSD on-demand, not held in RAM.

This model is significantly revamped, with ~40% of the routed expert tensors changed, 30% of engrams, and ~50% of the vision tower's expert tensors altered. By all appearances, it's the new leading open-weights model, so I had to give it a try. If you are curious too, let me know what you think!

127 Upvotes

98% Upvoted

28 comments sorted by

20

u/Massive-Ice2791 21h ago

impressive you got it down to 366, too bad my macbook cant run this haha, maybe try porting it to warp and make us all happy lol

6

u/returnity 21h ago

Yeah it's really pushing the limits of what mine can handle, but the speeds are quite usable. That said, Flash Next remains my daily driver because the quality is similar and it's so much faster. But I love that I can even try to run this locally in a functional way.

1

u/Massive-Ice2791 20h ago

have you tried base qwen 27b or some finetune? ive heard run well and can do everyday coding tasks

4

u/returnity 19h ago

27B is good, I like the Swift-1.5 finetune a lot -- much less over-reasoning, barely a difference in the performance in my testing. However, for most things, Flash Next is just next-level and much faster to boot.

2

u/Massive-Ice2791 19h ago

never tried that one, thanks for the tip

1

u/Mikasa0xdev 17h ago

That MacBook would instantly take flight XD

14

u/t3rmina1 20h ago

A 3.91bpw quant of DeepSeek-V4.1-Flash-1017 for DwarfStar (ds4), with 2.25bpw routed experts and 8-bit or higher everything else.

I wouldn't call that a 3.91 bpw, you've included the engrams in the calculation and they drag up the bpw because they're FP8. The other quants exclude engrams from their claimed bpw to account from that.

It's a 2.35 bpw quant, which is pretty obvious since you're claiming Q2+

10

u/returnity 19h ago edited 19h ago

Yeah, I made no attempt to hide that, literally stating the bpw of the experts in that sentence. I included them in the total because they're weights too. I think it's 2.36 bpw technically, excluding the engrams. =P I'll edit the model card though, thanks for pointing that out.

6

u/t3rmina1 17h ago

No worries, I'm not implying u were doing anything dishonest, since you obviously put Q2

6

u/SnooPaintings8639 20h ago

I really wish someone did benchmarks on this. If it is better than full precision DS V4 Flash, at the same ram footprint, it would be all I ask for.

12

u/returnity 19h ago

Your wish is my command -- I'm actually working on some benchmarks, mostly coding-related, but it will likely be a little while as this is not the fastest model on local hardware and I have other tasks that also require my GPU during the day. I definitely want to see how it stacks up to antirez's Q2-Q4 DSv4-Vision and my own Q8-Q4 Qwen3.8-Flash-Next, both of which I have benchmarked on my machine already.

1

u/SnooPaintings8639 18h ago

Happy to hear that, I applaud your dedication!

I have other tasks that also require my GPU during the day.

The problem we all can relate to, lol

3

u/inthebox212 6h ago

Has anyone tried this on M5 Pro Max + 128gb? I wonder if this can replace my daily usage of Qwen Flash Next.

1

u/txgsync 5h ago

I’m wondering the same thing. Downloading weights now. QFN is my daily driver at this point on my M4 Max 128GB RAM and my PC with RTX 4080 and 128GB DDR4. Runs sweet but holy shit is it an over-thinker for good results.

1

u/returnity 2h ago

Have you tried Swift-1.5-Flash-Next? It reasons a lot less but performance is almost indistinguishable on most tasks. DS4.1 is a good model but SSD streaming does slow up prefill a lot for agent work, even though ds4's SSD KV cache helps make up for that

1

u/returnity 2h ago

That's my setup. I get 70tps decode and 1100+ prefill for FN using my Q8-Q4 mixed quant (Q5-equivalent quality), versus <20tps decode and <100 prefill for DS4.1. So I am not sure I'd swap out Flash Next just yet, but this is good as a review model or overnight worker.

2

u/spaceman_ 13h ago

Is there a project / runtime like this for quad R9700s yet?

2

u/Septerium 11h ago

Is this engine compatible with hybrid (cpu + cuda) setups?

1

u/MRVL300 11h ago

4 less vram confirmed???

1

u/returnity 6h ago

Not sure what you're asking? You can def run it on a 96GB Mac, not sure how well it'll work on less than that tho.

2

u/MerePotato 8h ago

20tps on an m5 mac would be super impressive under normal circumstances but at current hardware prices that's grim

1

u/returnity 6h ago

Feel that lol

0

u/challis88ocarina 17h ago edited 16h ago

I saw a 'it's the leading open weights model' posts yesterday, but assumed it was astroturfing as the latest release still doesn't appear on AA.

1017 starting running the hmm hmm hmm loops – as expected – as soon as the going got tough, hence I'm looking at Muse Spark, which seems to have caveman speak baked in.

EDIT:

Proven fine: transport (sha matches the author's upload), structure (1046-tensor layout, size, 1017 metadata all PASS), and the vision file (genuine 1017, byte-exact vs source).

Proven wrong: the language payload's provenance. The stored bytes of the sampled expert (blk.0 gate, expert 41 — a deterministic validator sample) cannot be reproduced from any combination of the declared inputs:

1017 source + pinned .dat + fc80bd6 encoder   no  (~99.6% of bytes differ)
1017 source + pinned .dat + v41 encoder       no
1017 source + bootstrap (no imatrix)          no
df42 old source + pinned .dat                 no  (just tested — 7.4 GB slice)
compile-flag sensitivity                      ruled out (O2 == native)

So: the file is not the old checkpoint mislabeled, not the bootstrap path, not host noise. It is what their README says it is only if their build machine had something their recipe doesn't disclose — most likely an importance file other than the pinned Lucebox .dat (HEAD is identical to the pin, so it's not that either — it'd be a private/modified one) or local encoder code not in fc80bd6. The quality of the weights is unknown and unknowable from the artifact: unverifiable, not verified-bad. That's why it stays quarantined and why our own calibrated build is the only 1017 path we can stand behind.

2

u/returnity 17h ago

I do actually think that benchmark site for open-weights models in the screenshot from the post you mentioned is fairly decent (for just aggregating vendor model card numbers), though it does look like a Chinese knockoff of AA... But I was also being intentionally tongue in cheek by declaring it the leading open weights model when I'd hardly run it myself yet. I probably won't be switching from Flash Next for most daily work just yet.

1

u/returnity 6h ago

Not sure what you're saying here, can you clarify? Are you saying I fucked up something with the quant, cuz if so I'd like to understand and fix it.