In 2023, following the release of ChatGPT, the author recognized that research needed more organization than typical within Microsoft Research. Several colleagues agreed, leading to the creation of the AI Frontiers sub-organization within MSR, comprising approximately 50 people. The group focused on organizing coherent research efforts in the "messy middle," tackling compute bottlenecks and developing fundamental capabilities like the Next Latent objective and the Full Bandwidth Transformer.
AI Frontiers also worked to share resources, developing a monorepo shared by default with all of Microsoft Research, including edge scale pretraining code. In early September, the organization was reorged into Microsoft AI. However, the ex-Inflection leadership of MAI brought a philosophy focused on scaling and away from research, which the author considers incompatible and misdirected.
The author argues that returns to research are real, citing Bytedance experiments suggesting the Full Bandwidth Transformer could be comparable to a model twice its size, offering significant cost savings. The reorganization highlights a tension between scaling and innovation within the combined entity.
Source: Hacker News · Summarized by HeadlinesBriefing