A developer successfully trained a 4-billion parameter model to play Wordle using only reinforcement learning on a MacBook Pro M4. Bypassing supervised fine-tuning, the project used Apple's MLX framework to exploit unified memory architecture, testing if consumer hardware can replicate advanced reasoning capabilities found in massive cluster training.

The 'straight-to-RL' approach faced immediate hurdles, including reward hacking where the model preferred stalling over guessing. To stabilize training, the author implemented aggressive gradient clipping and reduced LoRA rank. A curriculum strategy—starting with partial game history—was essential to prevent policy collapse and help the model learn constraint satisfaction.

Results showed a 3.4x win-rate improvement over the base model in zero-shot scenarios, jumping to 31% with context. Deterministic settings (low temperature) proved superior for logic tasks. This work demonstrates that local devices can handle complex RL research by substituting expert data with heuristic rewards like entropy, avoiding the need for massive cloud clusters.