Databricks Reference

Reference material for Databricks Publishing -- data type mappings, CLI arguments, and querying.

Data Type Mapping

Source types are mapped to Databricks types automatically.

PostgreSQL to Databricks

PostgreSQL Type	Databricks Type
INTEGER, INT4	INT
BIGINT, INT8	BIGINT
SMALLINT, INT2	SMALLINT
NUMERIC(p,s)	DECIMAL(p,s)
REAL, FLOAT4	FLOAT
DOUBLE PRECISION	DOUBLE
VARCHAR(n), TEXT	STRING
DATE	DATE
TIMESTAMP	TIMESTAMP_NTZ
TIMESTAMPTZ	TIMESTAMP
BOOLEAN	BOOLEAN
BYTEA	BINARY

SQL Server to Databricks

SQL Server Type	Databricks Type
INT	INT
BIGINT	BIGINT
SMALLINT	SMALLINT
TINYINT	TINYINT
DECIMAL(p,s)	DECIMAL(p,s)
FLOAT	DOUBLE
REAL	FLOAT
VARCHAR(n), NVARCHAR(n)	STRING
DATE	DATE
DATETIME, DATETIME2	TIMESTAMP_NTZ
DATETIMEOFFSET	TIMESTAMP
BIT	BOOLEAN
VARBINARY	BINARY

CLI Reference

Databricks Publishing Arguments

Option	Type	Description
--publish_target ID	String	Credential ID for Databricks publishing (required)
--publish_schema_pattern PATTERN	String	Schema naming pattern (default: {schema})
--publish_table_pattern PATTERN	String	Table naming pattern (default: {table})
--databricks_table_type TYPE	Enum	external or managed (default: external)
--databricks_catalog CATALOG	String	Override catalog from credentials
--n_jobs N	Integer	Parallel workers for table creation (default: 1)

Querying Tables

Databricks SQL:

SELECT * FROM lakexpress_catalog.tpch_1.customer LIMIT 10;

Databricks Notebooks (Python):

df = spark.table("lakexpress_catalog.tpch_1.customer")
df.display()

Databricks Notebooks (SQL):

%sql
SELECT * FROM lakexpress_catalog.tpch_1.customer LIMIT 10

Delta Table Features (Managed Tables Only)

Time Travel:

SELECT * FROM lakexpress_catalog.tpch_1.customer VERSION AS OF 1;

Table History:

DESCRIBE HISTORY lakexpress_catalog.tpch_1.customer;

Optimize:

OPTIMIZE lakexpress_catalog.tpch_1.customer;

See Also

• Databricks Publishing - Setup and usage guide
• CLI Reference - All command-line options