Testing it now. At the very least, competitive with DS4-Flash indeed. On my small (and per Sol's words, _very_ semantically dense) C test codebase, it found things that only gpt-5.2 managed to find back in the day, but also made a stupidly incorrect initial observation that a memfd_create()/mmap was used for IPC (funnily enough - sol missed that as well in its review, until I pointed it out). Re: the claims vs deepseek v4 - both flash and pro are expected to get a "general availability" release very soon (i.e. well-"post-trained"), so things can change in a... well, flash, as per usual in the current environment.
> edit: on bigger tests, got it to loop pretty easily unfortunately, probably local settings.
Been playing around for a few hours with the poolside/Laguna-S-2.1-NVFP4 + poolside/Laguna-S-2.1-DFlash-NVFP4 + vLLM, been seeing the same behaviour. Usually new model releases are plagued with issues at release though, best to wait 1-2 weeks then retry, or better yet, investigate yourself :) Personally I haven't found any obvious issues.
Update: Seems quite literally they have bugs on the hardware I'm trying to run this with:
From Poolside CEO Eiso Kant on Twitter:
> Learning we have some bugs on the RTX6000. We’re on it. Team has worked non stop last days and it’s getting late for a lot of the inference folks, so might be until tomorrow till we have a solution. - https://x.com/eisokant/status/2079693050796785720
Update2: I'm now running poolside/Laguna-S-2.1-NVFP4 with vLLM 0.23.1rc1.dev1378+gd6dbdb9b0 (FlashInfer 0.6.14) and seeing slightly better results in regards to the looping. I can't see any specific changes that would affect this though, strangely enough.
> The BF16 checkpoint doesn't exhibit any of the complaints [...] with our current quants, the models tend to choose the wrong logits sometimes [...] why we'll need a requant [...] We're not aware of any bugs in any runtimes themselves [...] We have two remaining things that we're trying to tackle: some people are reporting thinking being too hard to trigger (^^) , and others are saying it thinks too much. We've seen much more of the latter internally
Deleted earlier, didn't see you post, pasting here:
/* Just started testing with the gguf (with gpu offload, m5 max 128gb), q4_k_m, running seemingly well. Speed is initially slightly faster than antirez/ds4 - decode tok/s in the 30s, prefill ~400-ish. Expected, given the slightly smaller size. Looks to be working fine, but too early to tell. Definitely likes to "think". */
Anyways, guessing that discord might be focusing on the nvfp4 stuff. I've noticed spelling mistakes in the thinking traces, tool calls have been fine so far.
Well, just ran said gguf on the GeneralsX codebase with a pretty open-ended "Explain this codebase to me, and the general game loop." prompt, and...
Let me also look at the GameLogic::update() to see the rest of the update flow, especially the object update loop.
Actually, I think I have enough information now. Let me also check the GameLogic::UPDATE to understand the full update flow.
...repeating forever.
Since they mentioned that they're working on new quants, guess I'll wait. From earlier tests on work stuff, it's definitely capable.
More updates from the Discord (also mostly about the NVFP4):
> we've got an NVFP4 checkpoint that has a KD of 0.135 relative to the BF16 checkpoint (for W4A4, on some GSM8K prompts). [...] We're still seeing some looping on W4A4 (although much less than before), whereas for W4A16 we've not been able to make it loop. Given that W4A4 is still a little broken, we won't make this an official release (we have one more idea planned to fix that). [...] The RC1 for NVFP4 is reachable as poolside/Laguna-S-2.1-NVFP4 @ RC1
...they'd messed something up in that quant, apparently fixed promptly. ds4 now supports it as well and it's... well, context-limited (<=250k on a 128gb machine) but it positively flies on an m5 max - the 60tok/s decode / ~500tok/s prefill is real.
Running deepseek flash on something locally now, this will have to wait a bit. I still stand by my initial quick assessment - looks capable. Some people on r/localllama also reported loops. We'll see in ~10 hours. Hopefully I haven't terribly mislead people.
Might wanna try it now, seems to have been largely fixed. Check huggingface threads and reddit. Works for me, very memory hungry and PP speed drops off a cliff around 200k context (~30tok/s decode and ~40tok/s pp - like... hope it isn't debugging with big logs), but it brings a fresh perspective alongside ds4-flash. The more, the merrier - I prefer keen eyeballs more than speed anyways.
Looks impressive, and this size fits achievable home hardware.
That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.)
Thanks for flagging. From the few benchmarks I can find, it looks there or thereabouts with Qwen 3.6-35B-A3B, or maybe a touch below. I'm interested to compare a model that is a big jump larger with pretty impressive benchmarks, but more heavily quantized to fit.
The XS model is genuinely confounding. So far I do not know what to make of it.
Maybe the chat template is broken, maybe not. But I can't get it to think/reason, or at least sometimes it seems to decide not to even when asked. And it is wildly inconsistent when it does it, sometimes thinking out loud.
However, without thinking enabled it is shockingly good at researching using web tool calls. It's also very fast.
I am assuming something is not yet right with the chat template, even using a recent build of llama.cpp, and I guess I will revisit this model again at some point in the future.
I am going to see what I can get out of this streaming from SSD vs. Qwen 27B. Laguna is a seriously impressive model. I already have a lot of success with 27B A3B.
Replying to myself, seems this PR was merged into main and it the model does work with a Vulkan backend on my Framework desktop, I’m getting about 220 tok/s prompt processing and 21 tok/s output on the 4-bit quant. This is really a sweet spot imo on this machine between maximizing ram use and still having decent speed due to the expert size. This looks really promising.
It’s the Strix halo (AMD) with 128 GB shared memory. The 4bit quant is ~75GB.
Unfortunately I don’t know about the best way of running on an Nvidia gpu, you could try llama.cpp and offloading as many layers as possible into the gpu and using RAM for the rest, not sure if that would slow it down too much though.
I got usable token rates (10-20tps from memory, so marginal) with the Qwen A10B a while back, well before all the new speculative speedups landed in llama.cpp. There's an unmerged branch which allegedly supports this, but I don't know how well yet. Looks worth investigating but I might give it a few days to see what bugs get shaken loose.
Whoa whoa whoa, 118b params, 8b active MOE, long context reasoning, open weights - music to my ears. Hadn't heard of this lab before but I am very excited, will definitely try this out tomorrow - this is a real sweet spot I think in terms of model size and performance.
This is exactly the kind of model that's been needed in the middle. Realistically self-hosted, Good Enough intelligence, MoE so it's fast on limited bandwidth systems like Strix Halo and DGX Spark.
For a while there's been nothing to run on my Strix Halo that's notably better than what I can run on my dual 32GB GPU desktop (Gemma 4 or Qwen 3.6 dense models), but this seems likely to be the step up in size that actually works better than those.
I've tried it, but, it's not reasonable speed, at all. It's 9-13 tokens per second, which is not usable interactively and not worth using for long-running API stuff when DeepSeek V4 Pro is so cheap via their API.
Laguna S 2.1 runs at 15-28 tokens per second, depending on context and...something about how long it's been running, which is very comfortable for chatting, but still not usable for interactive agentic coding. Their `pool` agent just times out when I try to use it with the Strix Halo-hosted instance of the model.
A lot of people are testing it, and reporting disappointed results / benchmaxxxing claim. But do not realize that thinking has a issue with the default configuration.
Important - make sure that THINKING is enabled. By default it wasn't although I was passing the flag --default-chat-template-kwargs '{"enable_thinking": true}' in vllm recipe. The generation_config.json file that is included has by default max_new_tokens as 32k which seems to be cutting off thinking altogether so increase it.
At first I was very disappointed with the output I was seeing, but once thinking is enabled, the code quality seems to be MUCH better. More real world testing to be done.
Really impressive signal that this 128B model can beat DeepSeek V4 (1.6T) on most coding benchmarks!
Also, I really like Poolside's habit to compare not only to other top models in its weight class (others don't do it, looking at you Mistral), but also to the very top open-weight models, even much bigger ones like the 2.5T Kimi-K3!
I am SO impressed by this model! It's cheap as deepseek v4 flash, better than minimax m3 I would say, and almost a glm 5.2. 120b params only for such a good result, with 258k context. This is amazing.
Yesterday I was making experiments running laguna xs 2.1 in my local machine (a smaller version), with a ryzen 5700x, 64 RAM and a 4060 ti 8gb nvidia
It was able to run locally with a context of 128k using llama-cpp and I tried using it for local development in a project that used uv, python, tool calls, file system explorations, websearches - all worked as a charm... in my own hardware
I was SO impressed. Then I saw open router is letting people try laguna S 2.1 for free - it's a model with 120b params and 258k context - if you top up at least 10 usd, it gives you 1000 requests per day for free so you can try it out. but the paid version is super cheap as well, specially the cache.
Yep! I loved Qwen 3.5 and 3.6, jumped to Minimax 2.5 and 2.7, but Laguna S2.1 feels like I have SOTA at home.
We’re going to get to the point where the enormous size of models will not be able to keep being updated with the huge amount of content being generated online, and so I think a strong SOTA-esque thinking model with a good harness, fast web crawler, and large MCP capabilities will be the future tool of choice, rather than larger and larger static models (unless someone creates upgradable but compact embeddings)
This seems like a great option for local usage. With 16 GiB of VRAM on the official (https://huggingface.co/poolside/Laguna-S-2.1-GGUF) Q4_K_M quant (which seems to be recommended) I can get about 50 t/s in and 10 t/s out. Definitely not winning any speed benchmarks but totally usable for background tasks.
The intelligent and decision making looks good overall. It seems to roughly compare to unsloth/Qwen3.6-27B-GGUF:UD-Q8_K_XL while being way faster on my hardware. It does often get stuck considering the same decision over and over again but maybe this is how it makes the better decisions.
However it does seem to have a bad habit of corrupting things (most often my name). It will rewrite kevincox to kevinc or kevcox or kevancox. Then fail to read directories or rewrite code with the wrong string and mess it up. Then it can't identify it's mistake.
Maybe I'll have try try out the Q8 model, but the description seems to push away from this one.
Even more important, subjectively, is that this model will run very well on Strix Halo (e.g. Framework Desktop), DGX Spark kinds of devices.
Looking forward to Unsloth dynamic mtp quants.
P.S. Looking at the HF release they already offer Q4_K_M and DFlash drafter for speculative decoding!
I love this. Is it possible to give a feel of how this stacks up to the good old Opus 4.5 in coding quality? For me that was the turning point where agentic coding in Claude Code etc became usable. Have we hit that threshold?
I am about 1 hour into using it with pi.dev. Do you have thinking on high? It is doing good but at one point i had to stop it and say 'you're overthinking this' haha
Yes full send mode on thinking. I have moved on from watching my agents and I don't really care how it thinks. I look at the end result and so far this thing has been blowing me away. No way this is as good as it is this small and fast. Outside Fable, this might be the best thing I've ever used.
"What we've done in this model is not necessarily add more intelligence, but improve the behaviors that lead to a more capable model: more verification, less taking things for granted, not declaring victory early, and being more persistent.”
This is fantastic work, really impressive is an understatement. I really hope this sets a new DeepSeek-esque standard and starts another the death knell for companies continuing to cosplay as frontier labs (like Cohere).
Immediate reaction is that it seems to be a bit behind Meta Muse Spark 1.1 performance at approximately the Deepseek v4 Flash price point. That's quite good given Muse Spark benchmarks a lot better than Deepseek v4 Flash (assuming benchmarks mean anything, which they don't).
For me, poolside.ai “came out of nowhere” a week or so ago when I discovered their local coding harness ‘pool’ and their smaller 33G MOE model that runs fast and is effective on my old 32G mac mini. Really good work!!
initial impressions, great model for coding, probably swapping it out for qwen 27b for a while to long-term test, more sycophantic than any I've run locally myself
It's an open-weight model so it literally doesn't matter whether it refuses by default or not, because it's pretty trivial to uncensor[1] any open-weight model and make it not refuse.
have only tested a few prompts, but it failed my favorite non-coding question that dsv4 flash aces. Benchmarks look excellent though (don't they always!)
I've been following work on the second-order effects that ripple through the system for a while. This is the first treatment I've seen that the framing reveals an assumption that isn't explicitly defended.
Anyways, keep 'em coming.
reply