Skip to main content
Version: 0.4 (Latest)

Quick start guide

This guide takes you from zero to a first export: every table of a PostgreSQL schema written as Parquet files to a local directory. Once it works, the next steps show how to target cloud storage or publish to a data platform.

1. Install LakeXpress​

Download the LakeXpress archive for your platform and unzip it. The trial archive bundles the FastBCP engine; with the full edition, FastBCP is installed separately. See the Installation guide for details.

2. Prepare the databases​

LakeXpress connects to two databases: the source database you export from, and the LakeXpress DB, where it stores its configuration and sync history.

Source database: a read-only user​

Create a user with SELECT privileges on the schema you want to export:

CREATE USER lakexpress_reader WITH PASSWORD 'your_password';
GRANT USAGE ON SCHEMA public TO lakexpress_reader;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO lakexpress_reader;
ALTER DEFAULT PRIVILEGES IN SCHEMA public
  GRANT SELECT ON TABLES TO lakexpress_reader;

For other source databases, see Database Connections.

LakeXpress DB: an empty database​

Create an empty database and a user that owns it. LakeXpress creates its own tables inside it.

CREATE USER lakexpress_admin WITH PASSWORD 'your_password';
CREATE DATABASE lakexpress_log OWNER lakexpress_admin;
Quick testing

SQLite or DuckDB can also be used as the LakeXpress DB. They need no server: point to a file path in the credentials file. For these and the other backends (SQL Server, MySQL), see Set up the LakeXpress DB.

3. Create the credentials file​

All connections are described in one JSON file. Save this as credentials.json:

{
  "lxdb_postgres": {
    "ds_type": "postgres",
    "auth_mode": "classic",
    "info": {
      "username": "$env{LX_LXDB_USER}",
      "password": "$env{LX_LXDB_PASSWORD}",
      "server": "localhost",
      "port": 5432,
      "database": "lakexpress_log"
    }
  },
  "source_postgres": {
    "ds_type": "postgres",
    "auth_mode": "classic",
    "info": {
      "username": "$env{LX_PG_USER}",
      "password": "$env{LX_PG_PASSWORD}",
      "server": "localhost",
      "port": 5432,
      "database": "production_db"
    }
  }
}

The top-level keys (lxdb_postgres, source_postgres) are the IDs you pass to the commands below.

$env{VAR_NAME} reads a value from an environment variable (an error is raised if it is not set). Plain-text values also work. Set the variables before running LakeXpress:

$env:LX_LXDB_USER = "lakexpress_admin"
$env:LX_LXDB_PASSWORD = "your_password"
$env:LX_PG_USER = "lakexpress_reader"
$env:LX_PG_PASSWORD = "your_password"

4. Initialize the LakeXpress DB​

Create the LakeXpress tables and check that the connection works:

.\LakeXpress.exe lxdb init `
  -a credentials.json `
  --lxdb_auth_id lxdb_postgres

This step is optional -- the first sync creates the tables if needed -- but it catches connection problems early.

5. Create a sync configuration​

A sync configuration says what to export and where. It is stored in the LakeXpress DB and reused by every sync.

.\LakeXpress.exe config create `
  -a credentials.json `
  --lxdb_auth_id lxdb_postgres `
  --source_db_auth_id source_postgres `
  --source_schema_name public `
  --output_dir .\exports `
  --n_jobs 4 `
  --fastbcp_p 2

This exports every table of the public schema of production_db (the database named in source_postgres), 4 tables in parallel, each split into 2 partitions. The relative path ./exports is resolved against the current directory and saved as an absolute path, so later syncs write to the same place wherever they are run from (LakeXpress 0.4.9 or later; with earlier versions, pass an absolute path). The exports directory is created if it does not exist, and the FastBCP engine bundled in the trial archive (engine/) is found automatically. With the full edition, or if FastBCP is installed elsewhere, add --fastbcp_dir_path <folder containing FastBCP>.

The command ends by printing a sync_id and the exact sync command to run next.

6. Run the sync​

.\LakeXpress.exe sync `
  -a credentials.json `
  --lxdb_auth_id lxdb_postgres `
  --sync_id <SYNC_ID>

Replace <SYNC_ID> with the sync_id printed by config create. LakeXpress shows progress as tables are exported.

7. Check the result​

Each table is written as Parquet files under exports/public/<table_name>/, in the directory where you ran config create. To see the status of the run:

.\LakeXpress.exe status `
  -a credentials.json `
  --lxdb_auth_id lxdb_postgres `
  --sync_id <SYNC_ID>

To export again later, run the same sync command: the configuration is reused.

Next steps​

Copyright © 2026 Architecture & Performance.