Roblox engineered a single, unified transformer model handling 256 language pairs for real-time chat translation, replacing 256 separate models. The core innovation is a Mixture of Experts (MoE) architecture that routes requests to specialized subnetworks instead of processing every parameter. This approach, combined with knowledge distillation reducing the model size from 1B to under 650M parameters, enables the system to process over 5,000 chats per second with a 100-millisecond latency ceiling.

The solution tackles the quadratic scaling problem of pairwise models by training all languages together, improving quality through shared structure and auto-detection capabilities. However, serving this large model efficiently required significant infrastructure optimizations, including dynamic batching and embedding caching between encoder and decoder stages, alongside quantization. The system now handles the platform's global user base spanning 180 countries and 15 million active experiences seamlessly.