Apple researchers unveiled LiTo, a 3D reconstruction AI that generates objects with lifelike lighting effects using just one image. The model leverages latent space to encode geometry and surface light fields simultaneously, enabling accurate reflections and highlights across viewpoints. Unlike prior methods requiring multi-angle data, LiTo compresses critical visual data into compact vectors, then reconstructs full 3D scenes. Apple claims this breakthrough could transform AR/VR, 3D modeling, and generative design by reducing computational costs.

The system first encodes an object’s surface light field—a combination of geometry and material properties—into a latent representation. A decoder then reconstructs the object from any angle, preserving view-dependent effects like specular highlights and Fresnel reflections. Training involved 150+ viewing angles and three lighting conditions per object, with random data subsets used to teach the model generalization. Interactive comparisons on Apple’s project page show LiTo outperforming TRELLIS in maintaining consistent lighting across perspectives.

LiTo’s innovation lies in unifying geometry and appearance in a single latent space, eliminating the need for separate models. This approach mirrors how transformers process text but applies it to 3D data, enabling efficient generation of complex visual effects. While technical details are sparse, the study emphasizes practical applications: faster 3D asset creation for games, virtual try-ons for retail, and enhanced digital twins for manufacturing.

The development aligns with Apple’s push into generative AI, though no release timeline was provided. Critics may question scalability, but the ability to generate high-fidelity 3D models from single images marks a significant leap for on-device AI. As noted in the study, LiTo’s compact latent representations could enable real-time rendering on consumer hardware, a key hurdle for widespread AR adoption.