Google has open-sourced TorchTPU, a PyTorch backend that enables users to run PyTorch models and operations on Google's Tensor Processing Units (TPUs). The project includes custom ATen kernels for a range of PyTorch operations, a compilation cache for optimizing execution of XLA computations, and examples for distributed training and model inference.
Key features include support for distributed training through Data Parallelism and Tensor Parallelism, integration with torch.compile() for graph mode execution, and utilities for debugging and benchmarking. Example models cover Llama, Qwen, ResNet, DLRM, minGPT, and UNet.
A user guide and interactive Marimo notebooks are available, which are similar to Jupyter. The notebooks must be run from a TPU VM, with a v6e GCE instance recommended as the easiest option. Python 3.11 and later are currently supported, matching PyTorch's release compatibility matrix.
The latest public release, version 0.2.0, can be installed from PyPI with pip, using the CPU build of PyTorch from an extra index. Developers can also build wheels from source with Bazel, with optimized builds enabled by default and an option to disable Remote Build Execution for those without credentials.
Source: Hacker News · Summarized by HeadlinesBriefing