OpenArch is a PyTorch repository offering hand-written implementations of modern open-source LLM architectures, based on Sebastian Raschka's LLM Architecture Gallery. Written from scratch for readability, each model lives in a single file with explicit structural choices like attention type, normalization, and positional encoding. The goal is clarity and learning, not competition with production libraries.

Models implemented include Llama, Qwen, DeepSeek, Gemma, GPT-OSS, Kimi, and more. Each entry documents its normalization method (e.g., RMSNorm, QK-Norm), positional encoding (Ro PE), attention variant (e.g., multihead, grouped query, sliding window), and whether it supports mixture of experts. Some models are marked usable for forward passes; others are under construction.

The repository is organized into text and multimodal folders, with each model containing a model.py and README.md. Contributors are welcome to add new models, write documentation, or fix bugs. OpenArch prioritizes readability over performance and serves as a learning resource for those interested in how modern LLMs are built.