Awesome KV Cache Optimization

Paper Repository

We build an Awesome-style survey repository, collecting and categorizing recent research papers on KV cache optimization for large language model (LLM) serving. Useful for both researchers and system practitioners working on efficient LLM inference!

The repository is for our survey paper:

Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
Jiantong Jiang1, Peiyu Yang1, Rui Zhang2, Feng Liu1
1The University of Melbourne, 2Huazhong University of Science and Technology

We deeply investigate recent papers on system-aware, serving-time, KV-centric optimization methods that improve system metrics without retraining or architecture modification (which we call this scope sKis). We systematize recent advances through a distinct system behavior-oriented taxonomy, organizing existing efforts into three behavioral dimensions:
🔷 Temporal — when is KV cache accessed or computed?
🔷 Spatial — where is KV cache placed and migrated?
🔷 Structural — how is KV cache represented and managed?

Figure 1

Contributing

Both the survey and the repository are still under active development and will be updated regularly.

🙋 If you would like to include your paper in this survey and repository, please feel free to submit a pull request or open an issue on GitHub with the paper's title and a brief summary highlighting its key techniques. You can also contact us via email. Please let us know if you find out a mistake or have any suggestions! We greatly appreciate your feedback regarding the repository or survey!

🌟 If you find this resource helpful for your work, please consider giving us a star on GitHub or citing our research.

Figure 2

BibTeX

@article{jiang2025towards,
  title = {Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization},
  author = {Jiang, Jiantong and Yang, Peiyu and Zhang, Rui and Liu, Feng},
  journal = {Authorea Preprints},
  year = {2025},
  publisher = {Authorea},
  url = {http://dx.doi.org/10.36227/techrxiv.176046306.66521015/v1},
  doi = {10.36227/techrxiv.176046306.66521015/v1},
}