vllm

Deepseek has open-sourced DeepInfer, the internal inference framework that serves its V4.1-Flash model, under the Apache License 2.0. The company says it will also open-source code for future models and has released deepseek-recipe to reproduce its API's token processing in self-hosted setups.

Mistral AI's engineering team documented their investigation of a memory leak in vLLM that caused 400MB/minute memory growth during disaggregated serving with Mistral Medium 3.1. The leak, which only appeared with specific conditions including graph compilation and NIXL-based KV cache transfer, was eventually traced to mmap allocations outside the traditional heap that standard profiling tools couldn't detect.