OpenAI has released two new implementations in its Baselines library: ACKTR and A2C. A2C (Advantage Actor Critic) is presented as a synchronous, deterministic variant of the popular A3C algorithm. OpenAI notes that A2C delivers performance equal to A3C but offers a more straightforward, deterministic execution model, which can be beneficial for debugging and reproducibility in reinforcement learning research.

This matters because reproducibility is a significant challenge in AI development; a deterministic version simplifies the validation of results. ACKTR (Actor Critic using Kronecker-Factored Trust Region) is a more sample-efficient algorithm compared to both TRPO and A2C. It achieves this efficiency while requiring only slightly more computation per update than A2C.

This efficiency is crucial for training complex agents where data collection is expensive or time-consuming. By open-sourcing these algorithms, OpenAI provides the AI community with robust, high-quality tools to build upon, potentially accelerating advancements in how agents learn complex tasks and reducing the computational resources required for cutting-edge research.