Guide
Build a Self-Healing Pipeline Agent
Set up an agent that can diagnose data problems, test fixes outside production, and verify results across the pipelines it supports.
What
A pipeline fails overnight. Someone opens the run, finds the asset that broke, reads the log, queries the table, works out whether the numbers downstream are wrong, and decides what to do. Most of that is evidence gathering, and evidence gathering is something an agent is genuinely good at.
What self-healing means here
One sequence:
- Read the failure.
- Find the likely cause.
- Make the smallest fix.
- Test it.
- Apply it.
- Check the data actually came out right.
How much of that the agent does on its own is a setting, not a law. Some teams want a diagnosis and nothing else. Some want the whole loop to run unattended overnight. Step 3 is where you decide, and it covers the controls that make the decision stick.
What you will build
Plan for about 70 minutes. Most of it goes into building context across the pipelines the agent supports.
- Bruin can import existing database tables as assets and use AI to enrich many asset definitions at once.
- Your team then reviews the generated details and adds the business rules that only people know.
finance.order_marginis one small example used to show what a complete asset definition looks like.
The guide uses one small example so the commands and incident are easy to follow. In a real project, the same import, enrichment, review, and testing process applies to the assets across all pipelines the agent supports.
Who this is for
Analytics engineers, data engineers, and software engineers who have already built a Bruin pipeline.
How
- Look at what your project already documents, and find the gaps.
- Build context across your pipelines: import existing database tables as assets, enrich asset definitions in batches, and add the business rules that need human review.
- Give the agent a database it can break, and set the scope you want it working within.
- Run a failure end to end, then apply the same loop to your other pipelines.
Choosing a path
- Local - when you want to work with a local agent in your IDE, terminal, or coding agent app, such as Cursor.
- Cloud - when the team needs a shared agent in Bruin Cloud.
Both paths use the same project notes and testing process.
Before you start
- A Bruin project with one or more pipelines. If you do not have one, follow Create your first pipeline.
- The pipelines and assets the agent should support. This guide uses
finance.order_marginas a small follow-along example; apply the same setup to the relevant assets across those pipelines. - A database you can write to that is not production, plus read-only access to your production sources
- A coding tool that can run Bruin MCP, if you want the local path
- Optional: a Bruin Cloud project for the Cloud path
Guide overview
- 1Context reviewYour project already holds most of the context an agent needs. Find out what is there and what is missing.
- 2Build your context layerWhat a complete asset file gives an agent, which parts you generate, and the habits that keep it true.
- 3Configure your agentsCreate a test environment, fill context in batches, and test what the agent can reach.
- 4Self healing in actionRun the demo pipeline, watch an agent troubleshoot it, then walk the failure types you will actually hit.
Get help & contribute