A researcher has discovered that relayering transformer layers works across modern models, not just a single fluke. The method, called RYS (Repeat Your Self), originally boosted Qwen2-72B to the top of the HuggingFace Open LLM Leaderboard by duplicating middle layers without any training. Now, after scanning 3,024 beam search candidates and evaluating 2 million configurations, the technique proves effective on newer models like Qwen3.5-27B.

This discovery reveals a three-phase architecture in transformers: encoding, reasoning, and decoding. The middle reasoning phase operates in a format-agnostic space where content matters more than language. Direct experiments comparing English and Chinese sentences showed cross-language pairs with identical content remained more similar than same-language pairs with different content through most of the network. The research team released scanning code and new RYS models for the community.

The findings suggest transformers have a universal internal language where certain layers can be duplicated without catastrophic distribution mismatch. The encoding and decoding boundaries remain 'blue walls' where relayering fails, but the reasoning core can be profitably expanded. This architectural insight could enable more efficient model optimization and better understanding of how large language models actually process information.