Artificial intelligence / news brief
SGLang adds a unified GPU-memory arena for hybrid models
- Event
- Published

LMSYS introduced an experimental SGLang allocator that lets full-attention KV, sliding-window KV and recurrent state share one GPU-memory arena instead of fixed pools. In author-run H200 tests, it fit about 1,000 concurrent GPT-OSS-20B requests versus 618 with default separate pools, but the feature still has model and backend restrictions.
Why it matters
Dynamic allocation could reduce stranded GPU memory for hybrid-model serving, though the reported throughput and capacity gains are not independently reproduced.
Sources
- Unified Memory in SGLang: One Arena for Every Token Lifetime (external site)LMSYS · primary source ·
- SGLang Unifies Memory for Model State (external site)AIB · coverage ·