HeadlinesBriefing HeadlinesBriefing.com

Fast BCP 1.6 exports SQL Server backup to Parquet instantly

Hacker News •
×

Fast BCP 1.6 exports a table from a SQL Server full backup to Parquet with no restore and no SQL Server instance. On a 320 GB production backup, the first open took 2 min 29 s and later ones 0.3 s. With 24 threads, a 175-million-row table went to Parquet in 29.6 seconds.

Fast Transfer can load the files into SQL Server on any version, or into PostgreSQL. The tool reads a table straight out of a .bak file, naming the table yields Parquet files. There is no restore, no SQL Server instance, and no credentials to manage.

Parquet is the only output; Fast Transfer loads those files into the SQL Server instance of choice, even an older version. A first export uses the connection type backup_mssql. Point the tool at a directory holding backups and a pattern matching the backup file.

With backup as the source, parallelmethod Physical cuts the table into slices equal to the paralleldegree, producing one file per chunk. To export a subset of columns, use --query with a column list: SELECT o_orderkey, o_orderdate FROM dbo.orders. The driver Arpe MSBackup reads the backup file independently.

Supported SQL Server versions: 2017, 2019, 2022, 2025. Supported backup kinds: Full, Copy-only full. Restrictions: Full backups only; differential and log backups are not applied.

Encrypted backups and TDE databases are refused. Parquet output only on Windows x64 and Linux x64. For an exact image, back up while the table is quiet.

Source: Hacker News · Summarized by HeadlinesBriefing