New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
EDN

Last-level cache has become a critical SoC design element

LLC, positioned between external memory and internal subsystems, stores frequently accessed data close to compute resources.

New memory architecture targets AI inference bottlenecks

Lightbits Labs Ltd. today is introducing a new architecture aimed at addressing one of the most stubborn bottlenecks in large-scale artificial intelligence inference: the growing mismatch between the ...
Electronic Design

Hardware Compression Works at the Memory Cache Level

How lossless data compression can reduce memory and power requirements. How ZeroPoint’s compression technology differs from the competition. One can never have enough memory, and one way to get more ...
Forbes

Memory And Design Advances From The AI Infra Summit

Forbes contributors publish independent expert analyses and insights. This article discusses memory and chip and system design talks at the 2025 AI Infra Summit in Santa Clara, CA by Kove, Pliops and ...
Neowin

AMD's new patent suggests Ryzen 3D V-cache CPUs may get lot more powerful and faster

AMD recently published a new patent that reveals that the company is working on making its 3D V-cache tech even better. Back in early 2021, we started hearing the first whispers and murmurs of a new ...
Electronic Design

Server Processors Stack Up to 1.1 GB of 3D Cache

AMD is leveraging one of its latest families of EPYC server CPUs, code-named Genoa X, in-house to run the electronic design automation (EDA) tools it uses for product development. Based on TSMC's 5-nm ...
Embedded

Understanding cache placement

In a computer, the entire memory can be separated into different levels based on access time and capacity. Figure 1 shows different levels in the memory hierarchy. Smaller and faster memories are kept ...