HeadlinesBriefing favicon HeadlinesBriefing.com

Polars 2.0 Release Candidate: Streaming Engine Default

Hacker News •
×

Polars has released the first release candidate for version 2.0, focusing on removing past design decisions and updating defaults rather than adding major features. The most significant change makes the streaming engine the default for all Lazy Frame queries via collect(), which the team expects to be roughly 5x faster with massive memory improvements for casual users. This required a major version bump because the streaming engine doesn't guarantee row-order by default for operations like join, group_by, and unpivot. Users needing observable row-order can opt in with maintain_order=True, or revert to the in-memory engine via pl.Config.set_engine_affinity("in-memory") or per-query engine="in-memory".

Polars 2.0 also enforces stricter behavior to fail fast and catch errors up-front, increasingly valuable for AI-driven development where agents validate query structure early via collect_schema(). Examples include is_in lossless type-coercion: previously, Int64 values compared against Float64 lists would silently coerce, risking false positives above 2^53 precision limits. Now this raises an Invalid Operation Error requiring explicit casting. Horizontal concatenation also becomes stricter.