Deploy Bruin with Apache Airflow
Orchestrate Bruin pipelines from Apache Airflow using BashOperator or KubernetesPodOperator, with secure credentials and scheduled DAGs.
What
Goal - Run Bruin pipelines from an existing Apache Airflow deployment.
Best for - Teams that already operate Airflow and want Bruin CLI inside their existing DAG scheduling and monitoring layer.
Prerequisites
- Apache Airflow installed and running
- Access to deploy DAGs and configure workers or Kubernetes pods
- A Bruin project ready to deploy
- Production data platform credentials
- Worker or pod network access to your data platforms
Bruin CLI runs cleanly as a task inside Airflow. If you do not want to operate Airflow, Bruin Cloud provides managed scheduling, monitoring, lineage, runs, backfills, notifications, and secure connection management.
More guides
Capstone: add returns to the pipeline
Extend the project, prove the result, and assess the finished pipeline.
Describe the model in its asset definition
Keep the metric definition, owners, limits, and checked examples next to the SQL.
Load and profile the source data
Load local commerce data and check its shape before modeling it.