ROLL

ROLL is an efficient, user-friendly library designed for scaling reinforcement learning workflows for large language models across large-scale GPU clusters. It supports diverse training paradigms including RLVR, agentic interaction, and distillation, while integrating advanced backends like Megatron-Core, vLLM, and SGLang. The framework provides robust observability and flexible resource management to enhance performance in complex reasoning and human preference alignment tasks.

ROLL is a high-performance, distributed reinforcement learning library specifically engineered for Large Language Models, utilizing a multi-role architecture powered by Ray to manage complex, large-scale GPU resources. It addresses the challenges of human preference alignment and agentic interaction by integrating advanced inference and training backends like vLLM, SGLang, and Megatron-Core. The framework prioritizes flexibility and scalability, allowing developers to navigate the trade-offs between synchronous and asynchronous training paradigms while supporting diverse hardware environments including NVIDIA GPUs and Ascend NPUs.