DuckDB 2.0 alpha is now available and delivers significant performance improvements, particularly for S3 queries. Testing on an M5 laptop showed 7.7 seconds for a query versus 18.8 seconds in version 1.5.5—a more than 2x speedup. The key advancement is async I/O, which separates network downloads from CPU decoding work.
Instead of workers sequentially downloading, waiting, and decoding, version 2.0 uses a dedicated download pool that keeps dozens of row groups in flight while workers focus solely on decoding. This parallel approach keeps both network and CPU busy simultaneously. The test used a 2.2 GB Parquet file from S3 containing 228 million rows.
Users building pipelines rather than database engines will notice the difference most when working with cloud storage. Full testing requires understanding data shape and modeling approaches. Run your own benchmarks before quoting results.
Source: Hacker News · Summarized by HeadlinesBriefing