Novita AI, a global leader in artificial intelligence cloud platforms, has announced a strategic partnership with SGLang, a fast-serving inference engine for large language and vision-language models. This collaboration will see Novita AI delivering high-performance GPU resources to power SGLang’s research, benchmarking, and optimization initiatives.
Highlights
1. Strategic Infrastructure Partnership
- GPU Powerhouse Support: Novita AI will provide high-performance GPU cloud resources to support SGLang’s advanced AI development, enabling efficient training and serving of large-scale models.
- Joint Development: The collaboration extends to the co-development of key projects like SGLang’s multi-turn reinforcement learning framework and the Prism multi-LLM serving system.
2. About SGLang: Next-Gen Inference Engine
- Optimized Runtime: SGLang delivers exceptional inference efficiency through innovations like RadixAttention cache reuse and zero-overhead batch scheduling.
- Software-Hardware Co-Design: By tightly integrating structured language primitives with backend performance tuning, SGLang supports scalable generation workflows and multi-modal applications.
3. Backed by Industry and Academia
- Broad Adoption: SGLang is supported by major tech leaders including NVIDIA, AMD, xAI, Oracle Cloud, and Google Cloud.
- Academic Backing: Leading research groups from Stanford, UC Berkeley, and UCLA contribute to SGLang’s development, reflecting strong community and academic engagement.
4. Collaborative Innovation
- RL and Multi-LLM Milestones: Novita AI’s infrastructure has enabled the development of key innovations like SGLang’s reinforcement learning framework and the Prism system.
- Expert Parallelism Project: Novita AI is also a key partner in SGLang’s large-scale expert parallelism initiative, an open-source effort targeting DeepSeek-level throughput performance.
5. Commitment to Open Ecosystems
- Shared Mission: Both companies are committed to open AI development, with Novita AI supporting diverse research efforts through shared infrastructure.
- Democratizing AI: This partnership underscores Novita AI’s goal of making advanced inference technology accessible to developers across the globe.
The Novita AI and SGLang partnership marks a significant step forward in AI infrastructure and inference performance. By combining GPU cloud power with software innovations, the collaboration sets the stage for breakthrough advancements in large-scale language and vision-language model deployment. Together, these two innovators are pushing the boundaries of what’s possible in open, scalable AI.
