HeadlinesBriefing favicon HeadlinesBriefing.com

LLMs+ Are Coming: Bigger Context, Better Efficiency

MIT Technology Review AI •
×

When ChatGPT launched in late 2022, it sparked an AI gold rush that transformed the tech industry. Now, after the initial frenzy, the question isn't whether AI will advance, but how. The answer appears to be LLMs+ - more capable versions of today's language models designed to tackle complex, multi-step problems that would take humans days or weeks to solve.

Making LLMs+ a reality requires breakthroughs in efficiency and capability. Mixture-of-experts architectures split models into specialized components, activating only what's needed for each task. Some researchers are exploring alternatives to transformers, like diffusion models traditionally used for image generation. Chinese AI firm DeepSeek demonstrated encoding text in images to reduce computation costs, while context windows have expanded from thousands to a million tokens - equivalent to entire books.

Perhaps the most promising advancement comes from MIT CSAIL researchers who developed recursive LLMs. Instead of processing vast amounts of information at once, these models break tasks into chunks and delegate to copies of themselves, creating a tree-like processing structure. This approach proves far more reliable for long, complex tasks where traditional LLMs often lose track. The result isn't just incremental improvement - it's a fundamental reimagining of how these systems operate.