To understand backpropagation? If you’re trying to understand how modern AI systems like large language models (LLMs) are trained, backpropagation is one of the most important concepts to understand. But if you ask me how I felt when I encountered it, I was completely lost by looking at the math equations. It felt like a mental block for me. I realized and wanted to start from scratch and build my understanding one step at a time. That journey began with my previous article, where we built a neural network from scratch using a simple dataset and understood how it makes predictions.

We will break down backpropagation step‑by‑step, keeping it as simple and intuitive as before. Before we begin, we recall the neural network we built: one hidden layer with two hidden neurons, ReLU activation, and the forward‑pass values we computed earlier. Mean Squared Error (MSE) is our loss function, now depending on the seven parameters w1,w2,w3,w4,b1,b2,b3. To improve the network we must find the parameter values that minimize this loss, which requires computing partial derivatives such as ∂L/∂w1, ∂L/∂w2, …, ∂L/∂b3.

These partial derivatives tell us how sensitive the loss is to each parameter and guide whether to increase or decrease it. Before computing them we need the chain rule, because each parameter influences the loss through a chain of functions. For example, if y = x² and z = y³, then dz/dx = (dz/dy)·(dy/dx). Understanding this rule is the foundation for backpropagation, and we will apply it in the next steps to update the network’s weights.