;sql-splitter

SQL dump toolkit at ~1 GB/s

Split, merge, convert, validate, sample, shard, diff, redact, query, and generate synthetic data from SQL dump files. MySQL, PostgreSQL, SQLite, and MSSQL. Written in Rust.

Why sql-splitter?

Working with large SQL dumps is painful. Importing a 10GB file takes forever, and you often just need a few tables. sql-splitter lets you split the dump into individual files so you can import only what you need.

13 Powerful Commands

Quick Start

Get up and running in seconds.

curl -fsSL https://sql-splitter.dev/install.sh | sh

brew install helgesverre/tap/sql-splitter

cargo install sql-splitter

sql-splitter split database.sql --output=tables

# Each table gets its own .sql file
ls tables/
# users.sql orders.sql products.sql ...

Common Workflows

# Convert MySQL to PostgreSQL
sql-splitter convert mysql.sql --to postgres -o postgres.sql

# Stream directly to database
sql-splitter convert mysql.sql.gz --to postgres -o - | psql "$PG_CONN"

# Sample 10% with FK integrity preserved
sql-splitter sample prod.sql -o dev.sql --percent 10 --preserve-relations

# Anonymize sensitive data
sql-splitter redact dev.sql -o safe.sql --hash "*.email" --fake "*.name"

# Validate all dumps with JSON output
sql-splitter validate "dumps/*.sql" --strict --json --fail-fast

Ready to get started?

GitHub|Releases|MIT License|llms.txt

Built by Helge Sverre