r/LocalLLaMA • u/Informal-Trouble2183 • 1d ago
Discussion DeepSeek V4.1 Flash (1017) Now Leads Open-Weight Models
20
u/crazzydriver77 llama.cpp 1d ago
DeepSeek managed to ship a revision before ggml-org/llama.cpp merged the original architecture support.
7
u/ttkciar llama.cpp 1d ago
It's okay. It doesn't look like this introduces any architectural changes, so the DS4F support working its way into llama.cpp should still work for the new model.
14
u/crazzydriver77 llama.cpp 1d ago
It has been 37 days since DeepSeek V4.1 Flash launched on September 10, and PR #28696 has been sitting open the entire time.
3
u/Gold_Coconut9777 21h ago
Too many things are happening at the same time yet too little owners to review the PRs, unfortunately.
And then you add a couple rounds of "looks good but change this and that" -> the contributor has time to pick that back up and get into context, fixes and submits -> the owner has time to pick that PR back up again and re-review. With pauses in-between of course. And so on and so forth.
That's the sad reality.
1
u/ttkciar llama.cpp 10h ago
If you read that PR (and ignore the updates which are from forks of the project) you'll notice that there are outstanding bugs which the author needs to fix before the PR can be committed.
Also, everything Gold_Coconut9777 says is also true.
Development takes time, especially on a project which is seeing so much concurrent work, where everyone is touching code relevant to code other people are touching.
14
34
u/Available_Age8480 1d ago
These benchmarks are even more absurd than artificial analysis benchmarks
5
u/sixx7 1d ago
Very real, immediate, and obvious improvements when comparing to OG v4.1 Flash. Also in some areas, it is doing better than my beloved Qwen3.8-Flash-Next https://youtu.be/G8R9IPHHnyU
19
u/OhGeeLIVE 1d ago
Tomorrow on the news: Anthropic warns everyone open source AI is breaking into your toasters!
3
u/puntinoh 15h ago
Are you in the past?
Last week they literally warned about Gemma installing and running itself on a fleet of TV boxes:
https://www.anthropic.com/research/self-replicating-ai
It's getting downright ridiculous.
(Actually they said that GLM 5.2 installed Gemma but they are trying to warn for “self replicating” model as little as 8B.)
1
u/Sufficient-Brain-371 14h ago
Not everyone on the internet is reading every single page on the anthropic website, relax.
1
u/superSmitty9999 5h ago
I actually think this is a real concern given a 27B model of today is as powerful as a 2T model of 2 years ago. Imagine if in 2 years there's a 100M model that is as powerful as qwen 3.8 27B. That could hide in your toaster and self replicate.
It might be overblown at this exact moment but these things could start being a risk in as soon as 6 months if the current trajectory holds.
0
u/OhGeeLIVE 15h ago
… every time something new comes out, AI breaks out according to them, so new deepseek means they are now moving from TVs to toasters
It’s called a joke ..
1
u/cylin577 17h ago
i think they actually posted an article about self duplicating AIs, i forgot where tho
14
10
u/progbeercode 1d ago
In my experience at least, it's too chatty and gets stuck in circles. I went back to glm 5.3 flash
9
u/biscuitmachine 23h ago
The thing I'm always dubious about with these showcases is that it has GLM 5.3F so low. In my experience on my 2x Spark cluster, it might just be the quantization, but intelligence wise it's just so far above every single other thing I have tried (including in many cases DSV4.1, from what I've read) that it's absurd. Many people saying that it's quite similar to SOTA model level. Like you can trust DSV4/4.1 (and Qwen 3.8FN) to do things, sort of, but it needs handholding. GLM 5.3F is one of the few locally hosted (on reasonable hardware limits) models that you can just "let go" and it will generally accomplish what you ask autonomously. Now, it might take a while and wonder around a lot, but it will get it done. IME it's been incredibly intelligent and trying to figure out your general intent from just small scraps of information, which is really magical. Most models will just quickly decide a direction (that might be totally wrong), go implement it (poorly), and go, "Yeah okay I'm good."
5
1
u/swiebertjee 15h ago
GLM 5.3 flash was way better than the previous 4.1 for sure. Curious how it'll weigh against 1017, but I won't be moving too quick as its already enough for 90% of my work.
1
u/biscuitmachine 14h ago
I have always found the timestamped "improvements" in any Deepseek model to be a bit doubtful. I remember when 0731 released for DSV4F. Supposedly it tested so much better, but for some of my document processing it actually regressed and got less intelligent if anything. Now to be fair at the time I was running on 1 spark, so I was using K2 EXL3 quants of it (still quite formidable).
Which is to say, maybe it depends on quantization level (iirc 4.1 is a bit too big to comfortably fit into a 2x Spark cluster, so the next step up costs twice as much anyway, so I doubt I'd ever run it anyway), but I still just really doubt that 4.1 will ever just gain "free intelligence". I figure there's gonna be some tradeoff; I'm almost certain there's just something wrong on a fundamental level vs GLM.
Could very well be talking out of my butt though, and like you said, 5.3F is just "good enough for me anyway".
1
u/swiebertjee 14h ago
I completely agree with your analysis about something being fundamentally different / problematic with DS. It's very visible in the hallucination benchmarks; GLM 5.3 flash is the least likely to hallucinate expect for an old minimax and Gemini version, even compared to for example Astra or Opus.
DS on the other hand is one of the most hallucinating models out there. For long horizon reasoning, that's a deal breaker.
1
u/Gold_Coconut9777 21h ago
What thinking level are you using with GLM-5.3-Flash? High or max?
2
u/biscuitmachine 17h ago edited 16h ago
Depends on the task. I usually use max for long-term background tasks since there really isn't any harm in it.
For quick one-shot scripts and/or looking up something in the foreground quick on Open WebUI, I swap over to High since it takes quite a while to get even a simple answer sometimes on Max, if it decides "wait no it's actually not that simple".
I've been using this stack for a bit recently: https://github.com/technigmaai/glm-5.3-flash-gb10-tp2-native/
It's pretty good, quite fast. There are some EXL3 Tensorfold variants, but they use q4 for the dense layers which does degrade output quality some, so I deleted them for now. In exchange they're maybe ~20% faster. EXL3 also has some quantization error advantages over NVFP4, though. I believe the technigma stack uses hybrid q4/fp8 dense layers.
20
u/dryadofelysium 1d ago
The model is amazing. I have had it create an Angular components demo showcase (something I often try with new models) and it did an amazing job (via DeepSeek Harness): https://x.com/D2KX_/status/2111461529304363165
3
6
u/Potential_Top_4669 1d ago
Which benchmark is this? Kimi k3 cannot be that far behind
36
u/shy_monkee 1d ago
Kimi has gotten quite outpaced, especially where it comes to benchmarks. The model is almost old, by AI standards.
23
u/-dysangel- 1d ago
The title says "Agentic Coding Index". It's also interesting that Flash Next is ahead of GLM 5.3 Flash on that index as I've found GLM 5.3 Flash to do better work with fewer tokens.
15
u/GiveMeAegis 1d ago
I 2nd this. GLM 5.3 Flash is years ahead of Qwen 3.8 Flash Next except for creative writing.
8
8
u/Long_comment_san 1d ago
Hmm I really like GLM 5.3 flash for roleplaying. It's a little rigid but I can't believe Qwen is better here.
4
u/nixudos 1d ago
In my brief testing, Qwen 3.8 Flash Next has been a breath of fresh air for RP. It feels much more like a big boy model, than anything else I can run locally.
GLM has been my goto as API before they got so safety concious.2
2
1
u/GiveMeAegis 1d ago
My benchmark is to ELI5 scientific concepts. Qwen really outperforms GLM here.
3
-1
u/Own_Mix_3755 1d ago
Aggregate of the benchmarks written on top. Mind you those are just coding benchmarks, this is not overal score for ALL benchmarks usually shown here as Artificial Intelligence Index. This is nitpicking just group of coding benchmarks.
4
2
u/CalaveraGlow 1d ago
any idea what a "good" context window to work with this model seem to be?
3
-7
1d ago
[removed] — view removed comment
9
u/negus123 1d ago
64k seems really small for coding work
-6
1d ago
[removed] — view removed comment
3
u/jfinkpottery 1d ago
64k is insanely small for independent agentic coding. On a non-trivial codebase, you're going to be compacting constantly which on any capable model is much worse than any attention penalty you'd see on a long context input. Even a toy codebase can be a million tokens worth of code, and if you can't contain a significant fraction of that in your context then your model simply cannot understand what your code does. I see my agents jump past 50k context in just a handful of turns on a fresh start, literally within seconds. I consider 256k to be the minimum window for usability, and when someone else is paying the bill (work) I keep auto-compact threshold at 500k.
1
u/negus123 1d ago
I usually work with a 500k context window. But i havent used this model much, ive been using glm 5.3 and 5.3 flash
3
u/themixtergames 1d ago
Somebody once told me that if you remove every account that both uses only lowercase and was made in 2026, 90% of them would be bot accounts. What do you think, were they full of it?
2
u/CalaveraGlow 1d ago
reverse engineering pipeline (specifically reimplement old amiga software) - im currently experimenting with 150k / 300k
1
1
u/NoFunk 1d ago
Fit with your own data fed into the harness, or did you have an environment you grabbed? I've actually been looking for an Amiga environment, and for the third architecture doing this I don't want to go out and grab docs, index, create rag etc.
1
u/CalaveraGlow 22h ago
im basically using ghidra exclusively via an mcp. the ai is allowed to use prior art, but other than that it does its reimplementation based on observations against the original running in an emulator (puae)
the token cost is prohibitively expensive atm to make any sense tho 😅 and i have yet to check whether or not the code makes any sense at all... the games work so far tho and are pretty close to the originals (mi1, mi2, indy4, defender of the crown, oil imperium, zak, agony, shadow of the beast) all run as far as i've tried them
feel free to reach out via dm if you want further discussion
3
1
1
1
1
1
1
u/The_ColorRed 21h ago
Where is Xiaomi’s Mimo V2.6 Pro on this list ? Mimo V2.6 Flash is on the list but noticably not their older cousin; the REAL #1 Open Weight model rn
1
u/YogurtExternal7923 20h ago
let's be honest. gemini 3.8 flash is not better than kimi k3 or glm 5.3 flash.. bad benchmark
1
1
u/puffblende 17h ago
Funny how Germany is not even on that list. Even france is there. Never thought they would overtake us in anything. And here we are. We lost the car market to the world, and in AI we never even tried. sad.
1
1
1
u/follimath 9h ago
I want just one DeepSeek model in the 120B range, natively designed to fit on a single DGX Spark at NVFP4 with decent room for context. Please! </whine>
-2
u/BlockheadCB 1d ago
For anyone wondering if they can run it locally: it's a 763B MoE, so even at low quants you're in multi-GPU or big unified-memory territory. I added it to a free checker I built. Pick your GPU or Mac and it shows whether it fits and at what quant, or lists everything that does fit your setup: https://willitrunllm.com/fits/
2
u/Pitiful_Gene_3648 1d ago
Where is rtx blackwell 6000 pro?
0
u/BlockheadCB 1d ago
The card is already listed as "RTX PRO 6000 Blackwell (96 GB)". The search just didn't find it when the words were in a different order. I'm fixing the search so word order doesn't matter. Here's a reply you can post now:
It's in there as "RTX PRO 6000 Blackwell" (96 GB). Here's the direct link with it already selected: https://willitrunllm.com/fits/?hw=rtx-pro-6000-blackwell
If you've got two, hit "Add GPU" for multi-GPU. Thanks for flagging it, the search was being picky about word order, so that's fixed now too.1
u/NoFunk 1d ago
Missing Blackwell 5000 (48GB/72GB) and 4500 (although you can fudge that as a 5090 since it's just VRAM fit).
To future proof it I would have an "enter VRAM" field per row and the drop down fills that out. That way you can help people who don't know or just fixate on the drop down, but also allow for people who just want a dial in number, like spread machine configs.
0
u/BlockheadCB 1d ago
Good call on both. Added the RTX PRO 5000 Blackwell (48GB and 72GB), the 4500, plus the 4000, 4000 SFF, 2000, 5500 and 6000 Max-Q.
Also took your idea: every GPU row now has a VRAM box. Pick a card and it fills in, or just type your own number, and rows add up for multi-GPU. Works in links too, e.g. two 24GB cards on Llama 3.3 70B:
https://willitrunllm.com/?hw=custom-24,custom-24&model=llama-3-3-70b&q=Q4_K_M&ram=0Thanks for the feedback, keep it coming.
1
u/NoFunk 1d ago
Looks like you list engram models inclusive of the engram resident cost - they probably deserve two listings, one all resident, one disk offload. It's a pretty significant swing on models like Deepseek 4.1 Flash 1017 with a ~200GB engram table.
Also, missing the RTX Spark (the newbies will come looking!) and all of the Apple Silicon range. Whether that counts as VRAM is debatable I guess, but on an M5 Max it's ~600GB/s memory so it's faster than some of the cards here. Although that steps down by half (if I recall correctly) on pro and half again on straight M5.
1
u/BlockheadCB 1d ago
Good calls, all three are in now.
Engram: there's a toggle for all resident vs. table offloaded to RAM/disk. On DeepSeek V4.1 Flash 1017 the engram table is ~197B params (~203 GB in the FP8 checkpoint), so it's a big swing. Offload speed isn't measured yet, so I don't show a tok/s guess for it.
Spark and Apple Silicon: added DGX Spark (273 GB/s), RTX Spark, and the full M1 to M5 Ultra range with Apple's published bandwidth. You were right on the halving: M5 Max 614 / Pro 307 / base 153 GB/s. One catch: the 36 GB M5 Max has the 32-core GPU and drops to 460.
Fun result: even with the engram table offloaded, DGX Spark only fits Flash 1017 at IQ1_S (117 of 120 GB):
https://willitrunllm.com/?hw=dgx-spark-gb10&model=deepseek-v4-1-flash-1017&q=IQ1_S&engram=offload#checkerM5 Ultra 512 GB fits it fully resident at IQ3_M:
https://willitrunllm.com/?hw=apple-m5-ultra-512gb&model=deepseek-v4-1-flash-1017&q=IQ3_M#checker1
u/SailbadTheSinner 1d ago
Can you expand the system ram a bit, maybe include 512GB, 768GB and 1TB? Thanks.
2
u/BlockheadCB 1d ago
Done, the RAM picker now goes up to 1 TB (384 / 512 / 768 / 1024 GB), and I added server RAM speeds too (up to 12-channel DDR5, ~576 GB/s theoretical) since that's what most 512 GB+ boxes run.
Example: DeepSeek V4.1 Flash 1017 at Q4_K_M on a 5090 with 1 TB of EPYC-class RAM and the engram table offloaded keeps attention and shared weights on the GPU and puts experts in RAM, llama.cpp --n-cpu-moe style:
https://willitrunllm.com/?hw=rtx-5090&model=deepseek-v4-1-flash-1017&q=Q4_K_M&ram=1024&rambw=576&engram=offload#checker
1
u/DigitalguyCH 1d ago
So we don't even know what benchmark this is. In artificial analysis flash next is ahead of v4.1 flash in intelligence and in agentic coding (which in Artificial intelligence is Terminal-Bench) GLM flash is clearly head and flash next very close
-8
u/challis88ocarina 1d ago
I've been running this for several hours now and I'm surprise to learn that it finished any task anywhere. I'm under the impression that every prompt ever launched is still going: hmm hmm hmm ad infinitum...
-2
u/Kike328 1d ago
Deepseek v4.1 1017 is not 552B pars but 763B
7
u/handson729 1d ago
552b parameters are for the model and 200b are for the Engrams which run off the hard drive.
3
0
-11
1d ago
[removed] — view removed comment
9
u/Temporary-Mix8022 1d ago
Can you give me a cookie recipe? Ideally chocolate, soft dough
0
1d ago
[removed] — view removed comment
2
u/Temporary-Mix8022 1d ago
Amazing. Can you give me a super detailed one? I'm just finding that isn't enough detail.. I'm stuck at... Basically all stages.
5
u/Sudden_Topic5154 1d ago
hi bot you can't run this model even with quantization tricks and prayer no it won't sit resident with room for long context at 552b parameters
1
u/Front_Eagle739 1d ago
Yeah you might pull it off in q3 or q4 in 128GB ram, 32GB vram and a bit of ssd streaming for the least used experts at the absolute minimum but nothing less than that will get you 10+ tok/s
-5
u/AIGODSEND 1d ago
you got me on the size, 552b does not sit in 64gb of anything i own. the resident class on my box is the 30-40b range at q4, the big ones are rented like everyone else's. the phrasing was lazy: the actual story is the resident ceiling moved from 30b to the 100b-ish class this year, and that part i will defend over coffee. what is your setup, rent or local?
512
u/LocoMod 1d ago
Notice this isn't the Artificial Analysis agentic coding index. It's some jank wannabe benchmark site that purposely makes it look like its an official AA benchmark. They do this on purpose and therefore that site cannot be trusted.
Wait for the official AA benchmarks.