A technical guide explains how to run the MLPerf Training v5.1 Llama 2 70B benchmark on a multi-node AMD Instinct MI325X cluster without a SLURM scheduler. The setup uses a 4-node cluster with 32 total GPUs and a high-speed network interconnect for RCCL communication. The approach leverages PyTorch's rendezvous mechanism instead of traditional cluster management software.

AMD provides an official Docker container for this benchmark, but it's designed for SLURM. For simpler SSH-based environments, the guide details using PyTorch's `torchrun` with a `c10d` backend to coordinate rank assignment across nodes. This method requires passwordless SSH, a shared filesystem, and careful configuration of distributed environment variables like `MASTER_ADDR`.

Performance results show near-linear scaling: throughput jumped from 2.79 to 11.15 samples/sec when scaling from 8 to 32 GPUs. The single-node convergence time of 20.57 minutes matched AMD's official submission. This demonstrates that properly configured clusters can avoid network bottlenecks, making manual multi-node orchestration a viable alternative to SLURM for AMD hardware.