GitHub - kekzl/imp: LLM server for NVIDIA Blackwell only (GeForce RTX 50, RTX PRO). On one RTX 5090, Qwen3.8-27B one user: 96.3 tok/s, 1.38x vLLM without speculative decoding, 1.23x llama.cpp on the same GGUF. Serves the 123.6 GiB Qwen3.8-Flash-Next from a 32 GB card at 92.1 tok/s. OpenAI and Anthropic API, tool calls, one docker run.
LLM server for NVIDIA Blackwell only (GeForce RTX 50, RTX PRO). On one RTX 5090, Qwen3.8-27B one user: 96.3 tok/s, 1.38x vLLM without speculative decoding, 1.23x llama.cpp on the same GGUF. Serves the 123.6 GiB Qwen3.8-Flash-Next from a 32 GB card at 92.1 tok/s. OpenAI and Anthropic API, tool calls, one docker run. - kekzl/imp
GitHub
Entities: OpenAINVIDIAAnthropicAnthropic API