Skip to content

Databricks guide

Installation

The Databricks ADBC driver is distributed via the ADBC Driver Foundry, not PyPI. Follow the Foundry installation guide to install it before using DatabricksConfig.

adbc-poolhouse does not need a separate extra for Databricks:

pip install adbc-poolhouse

Connection

DatabricksConfig connects to a Databricks SQL warehouse or all-purpose cluster using a personal access token (PAT). You must specify the connection in one of two ways: a full URI or individual fields (host, http_path, and token together).

Construction raises ConfigurationError (wrapped as a Pydantic ValidationError) if neither mode is fully specified.

URI mode

from adbc_poolhouse import DatabricksConfig, create_pool

config = DatabricksConfig(
    uri="databricks://token:dapi...@adb-xxx.azuredatabricks.net:443/sql/1.0/warehouses/abc123",  # pragma: allowlist secret
)
pool = create_pool(config)

Individual fields

Set host, http_path, and token together. The driver constructs the URI internally, percent-encoding the token so that special characters (+, =, /) do not corrupt the connection string.

from pydantic import SecretStr
from adbc_poolhouse import DatabricksConfig, create_pool

config = DatabricksConfig(
    host="adb-xxx.azuredatabricks.net",
    http_path="/sql/1.0/warehouses/abc123",
    token=SecretStr("dapi..."),  # pragma: allowlist secret
)
pool = create_pool(config)

Default catalog and schema

Set catalog and schema to pin a default namespace. The driver appends them to the connection string, so you can query unqualified table and view names instead of writing catalog.schema.table every time.

from pydantic import SecretStr
from adbc_poolhouse import DatabricksConfig, create_pool

config = DatabricksConfig(
    host="adb-xxx.azuredatabricks.net",
    http_path="/sql/1.0/warehouses/abc123",
    token=SecretStr("dapi..."),  # pragma: allowlist secret
    catalog="main",
    schema="sales",
)
pool = create_pool(config)

With this config, SELECT * FROM orders resolves to main.sales.orders. Both fields are optional: set one, the other, or neither. In URI mode, put the namespace in the DSN yourself, and adbc-poolhouse returns the URI untouched.

Loading from environment variables

DatabricksConfig reads all fields from environment variables with the DATABRICKS_ prefix. For individual field mode, all three variables must be set at the same time. Setting only DATABRICKS_HOST or DATABRICKS_TOKEN alone causes ConfigurationError at construction, wrapped as a Pydantic ValidationError.

export DATABRICKS_HOST=adb-xxx.azuredatabricks.net
export DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/abc123
export DATABRICKS_TOKEN=dapi...  # pragma: allowlist secret

Set DATABRICKS_CATALOG and DATABRICKS_SCHEMA to pin the default namespace from the environment, the same way the catalog and schema fields do:

export DATABRICKS_CATALOG=main
export DATABRICKS_SCHEMA=sales

config = DatabricksConfig()  # reads host, http_path, and token from env
pool = create_pool(config)

For URI mode, set DATABRICKS_URI instead of the three individual variables.

Lakehouse//RT and the Python connector

This ADBC backend uses the Thrift protocol, which Databricks Lakehouse//RT does not accept. RT requires the Statement Execution ("kernel") path, and Databricks lists ADBC as unsupported for it. To reach a Lakehouse//RT warehouse, use DatabricksPythonConfig instead, which runs on the databricks-sql-connector package. See the Databricks Python connector guide.

See also