Preprint Finds LFU Hard to Beat in Large-Language-Model Caches
A preprint study finds LFU a strong protocol-specific baseline for LLM caches, while quality-adjusted hit rates were far below raw rates on LMSYS and QQP.
Developing Light
A preprint study finds LFU a strong protocol-specific baseline for LLM caches, while quality-adjusted hit rates were far below raw rates on LMSYS and QQP.
Developing Light