Databricks guide¶

The Databricks ADBC driver is distributed via the ADBC Driver Foundry, not PyPI. Follow the Foundry installation guide to install it before using DatabricksConfig.

adbc-poolhouse does not need a separate extra for Databricks:

pip install adbc-poolhouse

Connection¶

DatabricksConfig connects to a Databricks SQL warehouse or all-purpose cluster using a personal access token (PAT). You must specify the connection in one of two ways: a full URI or individual fields (host, http_path, and token together).

Construction raises ConfigurationError if neither mode is fully specified.

URI mode¶

from adbc_poolhouse import DatabricksConfig, create_pool

config = DatabricksConfig(
    uri="databricks://token:dapi...@adb-xxx.azuredatabricks.net:443/sql/1.0/warehouses/abc123",  # pragma: allowlist secret
)
pool = create_pool(config)

Individual fields¶

Set host, http_path, and token together. The driver constructs the URI internally, percent-encoding the token so that special characters (+, =, /) do not corrupt the connection string.

from pydantic import SecretStr
from adbc_poolhouse import DatabricksConfig, create_pool

config = DatabricksConfig(
    host="adb-xxx.azuredatabricks.net",
    http_path="/sql/1.0/warehouses/abc123",
    token=SecretStr("dapi..."),  # pragma: allowlist secret
)
pool = create_pool(config)

Default catalog and schema¶

Set catalog and schema to pin a default namespace. The driver appends them to the connection string, so you can query unqualified table and view names instead of writing catalog.schema.table every time.

from pydantic import SecretStr
from adbc_poolhouse import DatabricksConfig, create_pool

config = DatabricksConfig(
    host="adb-xxx.azuredatabricks.net",
    http_path="/sql/1.0/warehouses/abc123",
    token=SecretStr("dapi..."),  # pragma: allowlist secret
    catalog="main",
    schema="sales",
)
pool = create_pool(config)

With this config, SELECT * FROM orders resolves to main.sales.orders. Both fields are optional: set one, the other, or neither. In URI mode, put the namespace in the DSN yourself, and adbc-poolhouse returns the URI untouched.

Loading from environment variables¶

DatabricksConfig reads all fields from environment variables with the DATABRICKS_ prefix. For individual field mode, all three variables must be set at the same time. Setting only DATABRICKS_HOST or DATABRICKS_TOKEN alone causes ConfigurationError at construction.

export DATABRICKS_HOST=adb-xxx.azuredatabricks.net
export DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/abc123
export DATABRICKS_TOKEN=dapi...  # pragma: allowlist secret

Set DATABRICKS_CATALOG and DATABRICKS_SCHEMA to pin the default namespace from the environment, the same way the catalog and schema fields do:

export DATABRICKS_CATALOG=main
export DATABRICKS_SCHEMA=sales

config = DatabricksConfig()  # reads host, http_path, and token from env
pool = create_pool(config)

For URI mode, set DATABRICKS_URI instead of the three individual variables.