Google DeepMind has introduced T5Gemma, a new collection of encoder-decoder language models that adapt pretrained decoder-only models into the encoder-decoder architecture. T5Gemma combines the Gemma 2 framework with newly trained T5-sized models, offering researchers and developers a flexible approach to building more efficient models for tasks like summarization and translation.

The models demonstrate competitive performance against decoder-only counterparts, nearly dominating the quality-inference efficiency pareto frontier across benchmarks like SuperGLUE. In practical testing, T5Gemma 9B-2B delivers significantly higher accuracy than the 2B-2B model while maintaining similar latency to the much smaller Gemma 2 2B model.

T5Gemma shows promising capabilities both before and after instruction tuning, with math reasoning scores improving by over 9 points compared to original Gemma 2 models. Google has released these models on Hugging Face and Kaggle to accelerate research and development in encoder-decoder architectures.