Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets
llm-d 1.0 is an open-source Kubernetes stack for distributed LLM inference, lifting Gemini prefix cache hits to 70%.
MarkTechPost · Asif Razzaq · https://www.facebook.com/MarkTechPost/
Entities: GoogleGoogle CloudGemini