On 25 August, Open AI fully unveiled Jalapeño, the company’s debut AI accelerator chip. Jalapeño delivers up to 13.4 petaflops of 4-bit compute and accesses 232 gigabytes of advanced memory, linking to it at 15.4 terabytes per second. Benchmarks show Jalapeño can reduce end-to-end latency by up to 3.6 times compared to Nvidia’s GB300, while consuming less power.

The chip moved from first architecture concept to first silicon in under 20 months, with only nine months separating first RTL from tape-out. The design team averaged fewer than 100 people, with Broadcom handling physical design from gates onward while Open AI managed end-to-end system design including the inference accelerator, memory hierarchy, and networking.

Richard Ho, vice president of hardware at Open AI, says LLMs give engineers superpowers, enabling faster exploration of design paths. Experts like David Chin of Verkor.io and Andrew Kahng of UC San Diego acknowledge the speed as impressive, though Broadcom’s involvement was deemed essential. Ankur Srivastava of University of Maryland notes LLMs differ from prior automation tools due to their ability to understand language and code, making them well-suited for chip design tasks still in the linguistic domain.