Most executives picture AI costs as GPUs and accelerator racks, overlooking data storage. Yet what happens to data after a model responds is a significant, often unplanned, infrastructure expense. Billions of daily prompts, images and videos generate back-end data that must be stored, and storage is one of the harder costs to plan for.
Training is finite: a run starts, consumes compute, and ends. Inference runs continuously, and every request leaves something behind. WD measured an 8-second AI-generated video that was returned to users as a 3.97MB 720p file but left more than 500MB of associated back-end data, including logs, cached context, renders, backups and safety records. That is a 130x increase over the output the user saw, and longer, higher-resolution clips could push the footprint into hundreds of gigabytes.
The growth is significant. McKinsey projects AI inference demand will grow at a 35% CAGR through 2030, versus 22% for training, making up more than 40% of total data center demand by the end of the decade. A WD-sponsored IDC survey of 763 IT and business decision makers found nearly 95% saw data volume rise due to generative AI over the past year, with roughly 61% reporting growth of 25% or more. As one hyperscale cloud product manager put it, storage capacity helps keep GPUs busy, since idle GPUs waiting for data become expensive quickly.
Unlike compute, retained data does not shrink when demand slows. Logs, synthetic outputs and inference results keep accumulating, and increasingly they are reused as inputs for future AI workloads, including retrieval-augmented generation. The same survey found 96% of respondents expect to need faster archive retrieval. For CFOs and CIOs, this means AI infrastructure budgets scoped only around compute miss a major cost driver, and the storage bottleneck stays hidden until it shows up in operating costs.
Source: MIT Technology Review AI · Summarized by HeadlinesBriefing