OpenAI's products depend on Habitat, its online storage platform, which must respond in milliseconds to over 1 billion users across 40 geographic regions. Supporting this growth while improving reliability, performance, and developer velocity required rethinking how the platform was designed and operated. OpenAI's approach followed a clear sequence: first become reliable enough for mission-critical traffic, then fast enough for global users, and finally operate at massive scale.
This two-part series traces Habitat's evolution from a simple client library and database into a globally distributed storage platform serving over 500 petabytes of data. Part I covered the move from a Python client to a service, and the techniques that pushed a Python-based service far beyond what many would consider practical.
Part II focuses on scaling the underlying storage component, Azure Cosmos DB. Working closely with the Azure team, OpenAI arrived at a viable multi-tenancy approach using co-managed containers, delivering excellent read performance at massive, global scale.
Multi-tenancy brings trade-offs. Single-tenant systems reduce blast radius and simplify ownership, but become expensive and often lead to significant over-provisioning. Multi-tenant systems centralize provisioning expertise but introduce noisy-neighbor problems, where one tenant consumes an outsized share of resources. OpenAI moved back and forth between these models before settling on a multi-tenant container by default, and says it is still early in its multi-tenancy journey.
Source: OpenAI Blog · Summarized by HeadlinesBriefing