HeadlinesBriefing favicon HeadlinesBriefing.com

AMD Ryzen AI Halo: Local AI Performance

Hacker News •
×

AMD's new Ryzen AI Halo desktop system puts significant AI processing power directly on a user's desk, aiming to convert ongoing cloud AI expenses into a one-time hardware investment. This compact device, measuring just 150x150x45.4mm and weighing 1.2kg, houses the AMD Ryzen AI Max+ 395 processor. This chip integrates 16 Zen 5 CPU cores, 40 RDNA 3.5 compute units, and an XDNA 2 neural processing unit delivering up to 50 TOPS of AI compute.

The system's standout feature is its 128GB of unified LPDDR5x memory, offering 256GB/s bandwidth. This shared memory pool can be configured with up to 96GB dedicated to VRAM, allowing for the direct loading of massive models like GPT-OSS-120b. The Halo also includes 10Gbps Ethernet and multiple USB-C 4.0 ports for high-speed data transfer.

Testing with LM Studio, the Ryzen AI Halo processed the GPT-OSS-120b model at approximately 45 tokens per second. This performance rivals many cloud-based AI tools, demonstrating the viability of local LLM execution for developers. The system aims to simplify AI development and deployment by offering a dedicated, powerful hardware solution.

This developer platform, available exclusively through Micro Center in both Linux and Windows versions, positions itself as a practical alternative to cloud-based AI services for tasks like running large language models and stable diffusion pipelines locally.