Heavy computation in ML, especially with generative AI, necessitates data compression. Autoencoders offer a solution by compressing input data into a lower-dimensional representation while preserving context. These unsupervised learning networks have three main components: an encoder that reduces dimensionality to a bottleneck, the bottleneck itself containing the latent representation, and a decoder that expands the data back to its original size.

During training, the autoencoder's goal is to reconstruct the input data from its compressed form. This process implicitly forces the encoder to capture essential features in the bottleneck, creating a rich latent space. The difference between the original and reconstructed image is measured by reconstruction loss, typically Mean Squared Error (MSE). A key hyperparameter is the latent space dimension, which needs to balance compression with feature preservation. For instance, Stable Diffusion uses an autoencoder to compress a 512 x 512 x 3 image into a 64 x 64 x 4 image, a 48x compression ratio.

Autoencoders have various applications beyond compression, including denoising images by reconstructing clean images from noisy inputs, image inpainting to fill missing parts, and object removal. However, MSE loss can lead to blurry outputs. Advanced variations address this with adjusted loss functions. Overall, autoencoders provide a simple yet powerful method for data compression and feature extraction.