South Korean edge AI chip startup Deep X has taped out its second-generation AI accelerator, the DX-M2, featuring custom processor-in-memory (PIM) technology from Samsung. Designed to bring generative AI to robots, drones, and other edge devices, the DX-M2 enables inference of models with up to 100 billion parameters at less than 5 watts. The chip achieves this within the same power budget as the M1, offering up to 80 TOPS and tripling memory bandwidth to 150 GB/s.
Deep X founder Lokwon Kim highlighted that the most important change between the M1 and M2 is the memory bandwidth. The DX-M2 can run models with 20 billion to 100 billion parameters at 10 to 30 tokens per second, depending on the model. It supports new techniques including speculative decoding and Turbo Quant.
Deep X is collaborating with Taiwanese OEMs to target AI PCs, including laptops and workstations. Robotics companies prototyping on Nvidia GPUs seek lower power and more compute on-device, while automotive companies are also key targets. Deep X simulations show offloading GEMV operations to LPDDR5x-PIM results in 6.7x the tokens per second for Llama3.1-8B.
The company's next generation DX-M3 will support official LPDDR6 PIM technology, promising 5x to 8x higher performance. The only drawback is that extra compute requires 2.7x the power consumption, though Deep X aims for a 2x performance uplift within the same power envelope.
Source: Hacker News · Summarized by HeadlinesBriefing