No way to avoid the memory cartel, even if CXMT catches up.

Each minute that a GPU isn't running is money evaporating

Wow, if only the Chinese knew that! So easy!

Matrix multiplication is one of the few operations that isn't regularly limited by memory bandwidth. BLAS implementations come with several heavily optimized, architecture-specific versions of sgemm.

Well, it does rely on decomposing the problem to optimize cache efficiency.

I don't read Portuguese, but the tables of results seem to imply they are tuning block sizes that leverage the L3 cache. They also talk about prefetch, which tends to matter more as you are approaching a streaming pattern.

So, a single core result may not scale linearly for multicore, given that there will be some cache contention, right? It's a very different tuning problem to optimize each of N cores to use its 1/N fraction of cache while sharing the available bandwidth for cache misses.

Isn't the delta fed to an arithmetic coder?

Not as many as the ones that the US snatched in operation paperclip

Exactly, the dog was cloned in the US

I'd bet on the inverse: China scaling DRAM production until the price crashes, and the whole US stock market that is propped on top of that scarcity going down with it.

Tarrifs.

Plenty of demand outside the US. Why would the hyperscalers not buy the chinese RAM for all of their datacenters across the world besides the US one?

Rising supply from China will impact prices even in countries where there are tariffs.

The best Chinese RAM on the market is 50% larger and requires more power and thus emits more heat, as it is a 16nm feature size. If they can get to competitive sizes, then of course data centers will purchase it.

EPYC 9965: 614GBps of 12-channel DDR5-6400

A100: 1935GBps of HBM2e

Most of those FLOPS are constrained by memory bandwidth.

> Most of those FLOPS are constrained by memory bandwidth

I believe inference with large enough batch size is almost always compute bound, simply due to algorithmic complexity.

Each step of tiled matric multiplication with square tiles of size N^2 takes O(N^2) memory loads and O(N^3) compute operations. With N = 32 or 64, you will likely saturate compute even on iGPUs with DDR4 or DDR5 memory pretending to be VRAM.

Prefill (GEMM) is compute bound, decode (GEMV) is memory bound.

> decode (GEMV) is memory bound

Decode with batch size 1 is GEMV. Batching makes the decode GEMM too.

A social safety net

And definitely not more evil than the workers at current Meta.