I've been considering something like this for a while.
It only makes sense as a free, open-source, decentralized sharing protocol (where anyone can host theorems and no one can limit sharing them).
If a company were to manage to commercialize this, it would end public open research.
Im taking a lot of inspiration from MathOverflow, where I’ve been a user for several years. I think there is a protocol approach to this project that can be built, but it’s not what I’m building. I’m optimizing more for building in community features, since that is the genre of site I have enjoyed using in the past.
The mystery of why .NET got so many things right is simply that C# was built several years later by the exact same Microsoft engineers who had previously worked on extending Java, giving them a perfect blank slate to fix the architectural flaws they had already encountered
virtual thread instead of async/await is a counter example.
Java is more used than C#, they can wait before delivering a new feature (given their leader position) but cannot deliver a flawed implementation that would stay in the language forever. Glad to have virtual threads and the backward compatibility that comes with it instead a Async version of sync methods + async and await keywords all over the code and Task as a return type in my interfaces methods to allow implementations to do non blocking I/O calls if they need.
.NET had green threads (fibers) in its first iteration. They were abandoned because practical use of .NET was also FFI-heavy - WinForms etc - and native code generally doesn't play well with non-native threads. The benefit of async is that it desugars into callbacks with state, which is something that can be easily expressed in terms of the C ABI (which is the de facto interop standard on all mainstream platforms). Which is why you can have async C# code calling into async C++ or Rust code, or for that matter async Python calling into async C#. Given that .NET was historically supposed to be a multi-language runtime, before they went all in on C#, async made a lot of sense.
C# did not ship with async/await, and Java didn't have virtual threads back then. I am specifically referring to the initial choices made in C#'s foundation.
That's true in a general sense, not true in every specific sense.
Before the mass murdering on Oct 2023, Israel didn't fight in the sense of an open conflict/war for a time. It tried to normalize relationships with Arabic countries, but didn't really work on solving the conflict with Gaza. It became more and more obvious they are trying to replace the population in the West Bank, further devalidating political/peaceful approaches by Fatah, fueling Hamas' approach of terror and murder.
Fighting for survival foremost mean to architect a sustainable solution for Gaza and westbank.
Let’s look at the actual historical record, because the pattern is undeniable: the root cause of this conflict is the absolute refusal of the regional and local leadership to ever accept a sovereign Jewish state within any borders.
In 1937 and 1947, the Jewish community accepted partition, but the local leaders rejected it and chose war.
Following the Six-Day War in 1967, the Arab League shut down any path to compromise by issuing its 'Three No's': no peace, no recognition, and no negotiations.
Decades later, Israel engaged in direct talks at Madrid in 1991 and made massive concessions under the 1993 Oslo Accords to establish the Palestinian Authority, only for the process to be dismantled by waves of suicide bombings.
At Camp David in 2000 and the Taba Summit in 2001, Israel offered an independent state encompassing Gaza and over 90% of the West Bank, but their leadership walked away to launch the Second Intifada.
When the Arab League proposed a plan in 2002, they conditioned it on non-negotiable terms regarding refugees and borders that compromised Israeli security, while Palestinian factions rejected the deal entirely regardless.
In 2003, Israel accepted the international Roadmap for Peace, but the process stalled permanently because the local government failed to fulfill its basic obligation to dismantle terrorist infrastructure.
In 2005, in a one-sided prayer for peace, Israel completely dismantled every single settlement in Gaza, dragged out its own citizens by force, withdrew every soldier, and handed the entire territory over to Palestinian custody.
In 2008, Israel proposed its most sweeping offer yet, including near-total territorial withdrawal and international administration of holy sites, which Mahmoud Abbas flatly refused to sign without any counteroffer.
You cannot blame Israel for a lack of effort when it has repeatedly offered statehood and land for peace. While the West labels Hamas as an extremist group, the reality on the ground is that they are viewed by a vast number of Palestinians as a mainstream governance structure and a legitimate movement of armed resistance against occupation. The true obstacle to peace remains this foundational objective, which Hamas champions and the local political culture reinforces, to establish an Islamic state over the entirety of the land and eliminate Israel entirely. If at any point there is a Palestinian majority and a leadership that genuinely seeks peace, it will happen in a second, despite whatever imperfections Israel may have.
Yeah, but the azure supply chain attack explains why all of a sudden they can make this change.
It seems that if you want to get something important changed in npm, you simply need exploit some of its short comings against Microsoft instead of discussing why it’s necessary.
> And to be fair 2: The other package repos also suck.
If you mean other languages, then yeah a lot of similar issues and weirdness there as well. Maven dependencies in any complex project are a "fun" challenge as well.
Though the sort of recurring supply chain attacks you see within the npm ecosystem is something I haven't seen elsewhere to this degree.
Maybe I have nostalgia blinders on but I do NOT remember putting up with this much bullshit in the Ruby ecosystem and I didn't even like ruby. Gemfiles were pretty okay, and gemfiles are what everyone assumed npm would be a copy of.
FYI, towardsdatascience has been banned on HN since about 2020 due to its general spamminess: https://news.ycombinator.com/from?site=towardsdatascience.co... (turn on "showdead" in your settings if you don't know what I mean). I would recommend using a more reputable platform for any future blog posts. Also, I would recommend making such posts about something other than the TurboQuant debacle. People who deeply care about quantization algorithms have probably mostly heard about it by now, and everyone else was only interested in the first place because of the misplaced hype. Putting additional effort into setting the record straight is unlikely to achieve much more than you already did.
Hi, thanks! I appreciate your input and generally agree. The TDS article wasn't really aimed at the HN crowd, but it did help a bit with the more general audience.
I do plan to also develop an interactive guide that breaks down post-rotation quantization fundamentals in a more educational, hands-on way.
TurboQuant is a recent paper from Google and NYU that has gained massive traction in mainstream media and the AI community. As implementations of TurboQuant are integrated into various popular projects, it is important to note its relation to EDEN quantization.
TurboQuant is essentially a partial implementation of EDEN quantization (first work published in NeurIPS 2021, extention published on ICML 2022). The few differences that do exist make EDEN significantly better.
TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555.
We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federated learning, vector retrieval, databases, inference engines, and KV-cache.
It would be appropriate to receive credit for this. Furthermore, it is baffling to see the name "TurboQuant" repeated in this context, considering the many works published from 2021 onwards.
The blog post mentioned above essentially guides you through EDEN quantization but ultimately settles on a sub-optimal MSE-minimizing version and an unbiasing trick. This trick often costs a full bit more than DRIVE/EDEN requires to achieve the same results using the unbiasing scale shown in the original 2021 paper.
For example, TurboQuant makes use of QJL (quantized Johnson Lindenstrauss transformations). One of the first papers to characterize the QJL and in fact the rate distortion tradeoff for quantized matrix multiplication in general is "Optimal Quantization for Matrix Multiplication" (https://arxiv.org/abs/2410.13780) by Ordentlich and Polyanskiy.
There is also a more accessible survey paper around quantized matrix multiplication called "High-Rate Quantized Matrix Multiplication: Theory and Practice" (https://arxiv.org/abs/2601.17187), by the same authors.
TurboQuant is starting to look like a case study in how to turn a fragile paper into a breakthrough story.
The attribution is thin, the “6x compression” headline is not clearly separated from prior KV-cache quantization baselines like KIVI, and the RaBitQ comparison is hard to take seriously: single-core CPU for the baseline, A100 GPU for TurboQuant. It is comparing apples-to-datacenter. Worse, there are also public OpenReview comments saying that even the reported accuracy results are not reproducible.
Hard to believe this is the standard for something being promoted as a breakthrough. If this came from a random startup blog, people would be much harsher about it.
But how can these poor googlers be expected to sift through the thousands of research papers published on these topics to find relevant citations? They don’t have time for such trivialities. They have far more important work to be doing not being evil. /s
Gemini helped them build it but didn’t / couldn’t attribute it from its corpus. I think we will see a surge of “rediscovery” that’s unattributed training surfacing of prior work that wasn’t widely recognized at the time.
Gemini is perfectly capable of searching the web. Pretty good at it really. As are most agents. If such a surge happens, it’s purely because of laziness.
I believe our claim at this point is more fundamental than just lack of citation.
The quantizer in TurboQuant is EDEN quantization (2021) applied to the KV-cache. It is neither a novel quantizer nor an improvement in quantization techniques.
In DRIVE/EDEN, we already introduced the version used in "TurboQuant"'s paper and suggested an optimal scale configurations which are better in both mse-minimizing and unbiased scenarios.
Wow, yes - you are completely correct (read through the note in detail now).
Though, as your paper also notes, the quantizer values themselves aren't fundamentally novel to either paper.
Lloyd Max scalar quantizers have been studied for a very, very long time.
And the specific Lloyd Max values for the Gaussian input distribution have been obtained in many papers across signal processing and information theory.
It is worth noting that taking advantage of the post-rotation distribution was not actually done until DRIVE (2021), which was made possible via our proper scaling. Furthermore, applying a Lloyd-Max codebook post-rotation was introduced EDEN.
We consider these to be the foundational works in this regard.
> Thanks for that! It is worth noting that taking advantage of the post-rotation distribution
I again feel this claim is too strong. Rotations have been used in information theory/wireless communications for decades at this point, with appropriate scaling done at channel inputs/outputs to hit channel capacity. The signals then pass through the appropriate codebooks that take advantage of the post-rotated+whitened signal.
Our cellphones today are powered by such technology.
I agree with your claim when restricted to deep learning. But I do not agree with the broad characterization that taking advantage of post-rotation distributions was only first done in your work.
Thanks for the pushback, and I appreciate the reference to classical information theory.
While I probably overstated things by using the very general phrase "taking advantage," I want to be very precise about the claim, as I believe these works are foundational to quantization, beyond the scope of deep learning. The mechanism of applying a deterministic biased quantizer, such as Lloyd-Max, to the induced post-rotation distribution, alongside mathematically correcting its inherent bias, is a distinct contribution (which asymptotically improves the worst-case error).
If there is a classical paper that utilizes such a combination, I would genuinely be very eager to review it. But to my knowledge, this was not introduced prior to DRIVE and EDEN.
"This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN extended to any bits per coordinate; we refer to them collectively as EDEN.
First, TurboQuant is a special case of EDEN obtained by fixing EDEN's scalar scale parameter to . EDEN supports both biased and unbiased quantization, each optimized by a different (chosen via methods described in the EDEN works). The fixed choice used by TurboQuant is generally suboptimal, although the optimal for biased EDEN converges to as the dimension grows; accordingly TurboQuant approaches EDEN's behavior for large .
Second, TurboQuant combines a biased -bit EDEN step with an unbiased 1-bit QJL quantization of the residual. It is suboptimal in three ways: (1) its -bit step uses the suboptimal ; (2) its 1-bit unbiased residual quantization has worse MSE than (unbiased) 1-bit EDEN; (3) chaining a biased -bit step with a 1-bit unbiased residual step is inferior to unbiasedly quantizing the input directly with -bit EDEN.
Third, some of the analysis in the TurboQuant work mirrors that of the EDEN works: both exploit the connection between random rotations and the shifted Beta distribution, use the Lloyd-Max algorithm, and note that Randomized Hadamard Transforms can replace uniform random rotations.
Experiments support these claims: biased EDEN (with optimized ) is more accurate than TurboQuant, and unbiased EDEN is markedly more accurate than TurboQuant, often by more than a bit (e.g., 2-bit EDEN beats 3-bit TurboQuant). We also repeat all accuracy experiments from the TurboQuant paper, showing that EDEN outperforms it in every setup we have tried."
I wonder how often this happens in practice - by "this", I mean industry/LLM world not noticing* some research until a bigger player repeats it with louder PR.
If we go only by the cases that have been publicly known it already happens all the time. Lots of patents are a race to register by multiple parties too and it's rarely done fairly.
> The technique implemented here consists of the scalar case of the HIGGS quantization method (Malinovskii et al., "Pushing the Limits of Large Language Model Quantization via the Linearity Theorem", NAACL 2025; preprint arXiv:2411.17525): rotation + optimized grid + optional re-normalization, applied to KV cache compression. A first application of this approach to KV-cache compression is in "Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models" (Shutova et al., ICML 2025; preprint arXiv:2501.19392). Both these references pre-date the TurboQuant paper (Zandieh et al., ICLR 2026).
EDEN is clearly relevant prior work for HIGGS. But reducing HIGGS to “an extension of EDEN” seems unfair to the authors of HIGGS. Similar primitive, different problem setting, different constraints, different contribution.
Curious: where do you draw the line between “related prior work” and “an extension of EDEN”?
In the vLLM documentation quoted above, TurboQuant (which is a restricted version of EDEN) is referred to as a specific case of HIGGS. Note the symmetry: EDEN acts as a special case of HIGGS; hence, HIGGS functions as a generalization of EDEN.
In any case, the quantizer is indeed an extension, regardless of whether it was explicitly framed that way in the paper. I say this not to diminish their contribution at all, but just to clarify the relationship, as it was also stated in the vLLM doc.
Thanks for that!
Note that the residual chain is empirically and theoretically inferior to our unbiased scale; furthermore, it requires an additional bit in certain cases.
Additionally, TurboQuant was not the first to apply EDEN to KV-cache (see for example https://arxiv.org/abs/2411.17525 from 2024).
The note includes extensive experiments and reproduces many of the figures from the TurboQuant paper in our Section 5. Honestly, I think our case is pretty clear-cut as is. I am not sure what the overhead for those specific benchmarks would be, but we will look into it.
(In any case, I want to emphasize that TurboQuant quantizer is a private case of EDEN)
with the amount of traction this has gotten... coming with a clear set of experiments even on arxiv paper would be of great help to showcase your improvements. And if they're easily reproducible, they could get integrated in the mainstream inference engines as well, as the main point here is compression with little degradation.
When you use TurboQuant, you are essentially using the EDEN quantizer under a different name applied to KV-cache.
Both EDEN and its 1-bit variant have been implemented in PyTorch, JAX, and TensorFlow across numerous open-source libraries and are used in various applications. I am currently writing a blog post that will document these in detail.
EDEN defines a scale parameter, S, for which we suggest specific optimal values for both biased and unbiased versions. As shown in the note I shared, these values lead to clear empirical improvements. Consequently, users who rely on the less optimal S value and the unbiasing method popularized by TurboQuant will generally see inferior results compared to those using EDEN with the optimal scale values suggested in our original papers.
This is a great development for KV cache compression. I did notice a missing citation in the related works regarding the core mathematical mechanism, though. The foundational technique of applying a geometric rotation prior to extreme quantization, specifically for managing the high-dimensional geometry and enabling proper bias correction, was introduced in our NeurIPS 2021 paper, "DRIVE" (https://proceedings.neurips.cc/paper/2021/hash/0397758f8990c...). We used this exact rotational approach and a similar bias correction mechanism to achieve optimal distributed mean estimation. I also presented this work and subsequent papers in a private invited talk at Google shortly after publication. Given the strong theoretical overlap with the mechanisms in TurboQuant and PolarQuant, I hope to see this prior art acknowledged in the upcoming camera-ready versions.
LOL. This is a classical technique, Johnson-Linderstrauss etc. In this context, rediscovered every few years (recently months), e.g. here's 2017: https://proceedings.mlr.press/v70/suresh17a
We do mention and the paper you shared. Please read our paper to see how the rotation-aware bias correction we introduced efficiently fixes the bias and provides a better worst-case error.
I just today learned about Multi-Head Latent Attention, which is also sort of a way of compressing the KV cache. Can someone explain how this new development relates to MHLA?
Multi-Head Latent attention is a redesigned attention mechanism that produces lower-dimensional KV-cache entries. Vector quantization can store KV-cache entries using a small number of bits per dimension while ensuring that the resulting attention scores don't change too much. So MLA needs to be part of the model from the beginning of training, whereas VQ can be retrofitted afterwards, and you could also combine the two.
MLA makes it so the keys and values used are a function of a smaller latent vector you cache instead of a key and a value for each token. KV cache quantization reduces the size of the values in the cache by using less bits to store each value. These two approaches operate on different parts of the process so they can be used in combination. For example, you can quantize the latents that are stored for MLA.
In this context, the rotation is for spreading energy and ensuring predictable coordinate distributions rather than diagonalization; it makes coordinate-wise quantization much more computationally efficient, though it throws away learnable structure.
ah ok, so intuitively it's like minimizing the error when replacing the values with a well-known distribution. So all you need to carry along is the rotation and the assumption that there is some amount of loss.
There are papers that try to quantize angles associated with weights because angles have a more uniform distribution. I haven't read this specific paper, but it looks like it uses a similar trick at a glance.
But if they read your paper enough that they invited you to a talk, that probably means they were far enough along to independently inventing it they were going to do so anyway, and wanted to chat with someone who was also doing the thing they were already doing. Good ideas tend to reveal themselves to anyone who is aware of the problem.
To be clear, I am not claiming they stole an idea. They have made significant independent research. However, a specific part regarding the treatment of rotation with bias correction relates to prior work, and it would be appropriate to have that recognized.
What they're saying is that the error for a vector increases with r, which is true.
Trivially, with r=0, the error is 0, regardless of how heavily the direction is quantized. Larger r means larger absolute error in the reconstructed vector.
Yes, the important part is that the normalized error does not increase with the dimension of the vector (which does happen when using biased quantizers)
It is expected that bigger vectors have proportionally bigger error, nothing can be done by the quantizer about that.
If a company were to manage to commercialize this, it would end public open research.