Function approximation replaces tables for large state spaces, enabling RL to handle problems like 10²⁰ Connect Four states or camera image inputs. Unlike tabular methods, which treat each state separately, approximations generalize across similar states. This shift is critical for real-world applications where explicit state enumeration is impossible.

The core challenge: defining an objective function to minimize prediction error between estimated and true state values. Stochastic Gradient Descent (SGD) updates function weights using experience batches, adapting to non-stationary targets. While convergence guarantees exist for unbiased approximations, practical methods like semi-gradient TD relax this constraint, trading theoretical safety for empirical success in dynamic RL environments.

Linear function approximation uses state features as input vectors, combining weights via inner products. Feature engineering becomes crucial—e.g., in cart pole balancing, angular velocity's impact depends on pole position. While elegant, linear methods struggle with complex relationships, motivating neural networks for non-linear approximations.

n-step semi-gradient TD extends tabular approaches to function approximation, balancing bias-variance tradeoffs. Though lacking strict convergence proofs, these methods power modern RL systems. The technical significance lies in enabling scalable, generalizable solutions—transforming RL from theoretical curiosity to practical deployment in robotics, gaming, and autonomous systems.