Databricks integration
Databricks data, in your warehouse.
A built-in ingestr connector: add credentials, pick tables, schedule it.
name: raw.bronze_raw_data
type: ingestr
parameters:
source_connection: databricks
source_table: 'bronze.raw_data'
destination: snowflake
incremental_strategy: merge$ bruin run assets/raw/databricks.asset.yml
- extract · Databricks bronze.raw_dataincremental
- load · snowflake raw.bronze_raw_datamerged
- checks · not_null, unique
Loaded and checked. Downstream models can run.
How it connects
Connected in three steps.
Databricks is a unified data analytics platform for data engineering, data science, and analytics.
- 01
Add a Databricks connection with its credentials.
- 02
Pick the tables to load and how: replace, append or merge.
- 03
Bruin runs it on your schedule and checks every load.
Connection parameters
- token
- Personal access token (use as username)
- host
- Workspace URL
- port
- Port number (usually 443)
- http_path
- SQL endpoint HTTP path
Tables
3 tables, ready to load.
bronze.raw_datasilver.cleaned_datagold.aggregates
Step-by-step
Pick a destination.
The platform
Part of the Bruin platform.
Data in, ready for everything downstream: the models, the checks, the lineage and the AI layer.
Sources
DatabasesWarehousesApps & APIsFiles & storageStreams & webhooksWeb scrapingMove
Data IngestionFrequently asked
Questions about Databricks.
Does Bruin have a built-in Databricks integration?
Yes. Built-in ingestr source. Built-in ingestr destination. Runs SQL and Python assets.
Which Databricks tables can Bruin load?
bronze.raw_data, silver.cleaned_data, gold.aggregates.
Where can Databricks data go?
Snowflake, BigQuery, Databricks, Redshift, ClickHouse, Postgres, DuckDB, MotherDuck, Microsoft Fabric and more, plus files on S3 and GCS.
How fresh is the data?
As fresh as your schedule. Incremental loads append, merge or replace a time window, every few minutes if you like.
Do we need Bruin Cloud?
No. The Bruin CLI and ingestr run locally, in CI or in your own orchestrator. Bruin Cloud adds scheduling, lineage, alerts and the AI data analyst on top.
Ready to connect Databricks?
$100 in credits and 50 AI tasks. No credit card.
A demo walks through your own data.