HeadlinesBriefing favicon HeadlinesBriefing.com

PostgreSQL Memory Trap Revealed

Hacker News •
×

A production PostgreSQL cluster recently crashed after consuming 2 TB of RAM due to a single query with work_mem set to 2 MB. The memory management behavior surprised even experienced database administrators. The investigation revealed a lesser-known function that could have prevented this disaster: pg_log_backend_memory_contexts, introduced in Postgres 14, which dumps the full memory context tree of a backend into logs.

The query created 524,059 chunks inside a single ExecutorState context. Each chunk used up to work_mem worth of memory, but none were released until the entire operation completed. The problematic query structure used a function as if it were a table in a join, creating what Postgres saw as one giant operation. Memory contexts are designed to free entire contexts in one shot rather than tracking individual allocations, which explains why the accumulated memory wasn't released mid-query.

Prevention strategies include fixing statistics with ANALYZE and CREATE STATISTICS, rewriting problematic queries, setting statement timeouts, and monitoring with the memory context function. The root cause was a badly written query consuming far more memory than intended. While Postgres's memory behavior is by design, not a bug, understanding this helps explain why "simple selects" can bring down production during peak operations.