Both Qualcomm and AMD are planning to integrate SOCAMM2 memory into their AI product lines. This move aims to boost memory capacity and speed, following NVIDIA's lead with its "Vera" CPU. "Vera" uses LPDDR5X memory on the SOCAMM factor, offering impressive memory bandwidth. The goal is to create high-speed, high-capacity systems for demanding AI tasks.

This trend is driven by the need for fast memory close to AI accelerators. SOCAMM complements the high-bandwidth memory (HBM) already on accelerators. It acts as a fast memory pool, capable of storing entire models in memory. This reduces data transfer to slower SSD flash storage, improving overall performance for AI workloads, which require rapid data access.

Nvidia's approach involves surrounding the CPU with multiple SOCAMM modules. AMD's Instinct MI accelerators, paired with EPYC CPUs, could be next. Qualcomm's existing AI solutions, like the AI200 and AI250, might soon be upgraded with SOCAMM. This transition would bring standardized memory expansion, avoiding the need for individual memory module soldering.

This shift allows manufacturers to easily offer different system versions by adding or removing SOCAMM modules. The standardization of SOCAMM could streamline AI hardware design. It also enables quicker adjustments to meet changing AI model sizes and performance requirements. Expect more announcements from both companies as they push forward on this front in the coming months.