Technical
9 min read

How to Build a Data Ingestion Pipeline Without Managing Servers (2026)

A 2026 guide to serverless data ingestion: run a pipeline without standing up or babysitting a server. Managed SaaS vs a serverless CLI on Cloud Run, Lambda, or GitHub Actions, with a concrete ingestr example and the tradeoffs.

Kateryna Kozachenko

Marketing & Growth

TL;DR: To build a data ingestion pipeline without managing servers, you have two clean routes. The managed-SaaS route (Fivetran, and similar) means someone else runs everything and you configure connectors. The serverless-code route means running a lightweight ingestion tool like the ingestr CLI on serverless compute (Cloud Run jobs, AWS Lambda/Fargate, or even GitHub Actions on a schedule), so there is no always-on server to patch or babysit. Avoid the server-based tools (a self-hosted Airbyte instance) if "no servers" is a hard requirement, since they are the thing you are trying not to run.

"Without managing servers" usually means one of three things: no always-on VM to patch, no Kubernetes cluster to operate, or no infrastructure at all. All three are achievable in 2026. The trick is choosing a tool whose form factor matches "serverless," because some popular ingestion tools are fundamentally a server you have to run. This guide shows the two routes that actually give you a serverless pipeline.

What "serverless" rules in and out

ToolServerless-friendly?Why
FivetranYes (fully managed)Someone else runs all of it
ingestrYesA CLI/binary; runs on ephemeral serverless compute
dltYesA library; runs inside a serverless function or job
GitHub Actions + CLIYesScheduled ephemeral runners, no infra you own
Airbyte (self-hosted)NoIt is a long-running server you operate
Debezium / KafkaNoA streaming platform you run and maintain

If "no servers" is non-negotiable, that table is the whole decision. Managed SaaS or a serverless CLI/library. Not a self-hosted server platform.

Route 1: managed SaaS

The zero-infrastructure option is a fully managed service like Fivetran: you configure a source and destination in a UI and it runs everything. You manage nothing. The tradeoffs are cost (usage-based, can climb) and control (closed, cloud-only, less code-first). If your priority is "I never want to think about running anything," this is the route.

Route 2: serverless CLI on ephemeral compute

If you want code-first, open-source, and cheap, but still no server to babysit, run a lightweight ingestion tool on serverless compute that spins up for the sync and disappears. ingestr is a good fit because it is a single binary/CLI with no server component: it starts, moves the data, and exits.

The pattern is the same everywhere: package the command, schedule it on ephemeral compute.

Example: a scheduled Cloud Run job

FROM python:3.12-slim
RUN pip install ingestr
ENTRYPOINT ["ingestr", "ingest", \
  "--source-uri", "postgresql://user:pass@host:5432/appdb", \
  "--source-table", "public.orders", \
  "--dest-uri", "bigquery://my-project", \
  "--dest-table", "raw.orders", \
  "--incremental-strategy", "merge", \
  "--incremental-key", "updated_at", \
  "--primary-key", "id"]

Deploy it as a Cloud Run job and attach a Cloud Scheduler trigger. The container runs on schedule, does the incremental sync, and shuts down. No VM, no cluster, nothing running between syncs. The identical idea works as an AWS Lambda container or Fargate task, or as an Azure Container App job.

Example: GitHub Actions (no cloud infra at all)

name: ingest-orders
on:
  schedule:
    - cron: "*/30 * * * *"
jobs:
  ingest:
    runs-on: ubuntu-latest
    steps:
      - run: pip install ingestr
      - run: |
          ingestr ingest \
            --source-uri "${{ secrets.SOURCE_URI }}" \
            --source-table 'public.orders' \
            --dest-uri "${{ secrets.DEST_URI }}" \
            --dest-table 'raw.orders' \
            --incremental-strategy merge \
            --incremental-key updated_at \
            --primary-key id

For low-frequency syncs, a scheduled Actions runner is genuinely enough and you own zero infrastructure. Keep secrets in the secret store, not in the workflow.

Gotchas

  • Serverless timeouts. Functions have max execution times (Lambda 15 minutes, etc.). A huge initial load can exceed them. Do the first full load somewhere with a longer limit (a Cloud Run/Fargate job), then let short incremental runs fit the function window.
  • Cold starts and state. Serverless compute is stateless. Store incremental watermarks in the destination (which ingestr and dlt do) so a fresh container resumes correctly.
  • Networking to private sources. Reaching a database in a private VPC from serverless compute needs connectors/VPC access configured. Plan for it.
  • Secrets. Use the platform's secret manager, never inline credentials in a Dockerfile or workflow.
  • Concurrency. Make sure two scheduled runs cannot overlap on the same table, or add idempotency (merge on primary key, which ingestr does) so an overlap is harmless.

Serverless ingestion is one step

A serverless sync lands raw data; you still need to transform, quality-check, and orchestrate. ingestr is the ingestion layer of Bruin, an open-source platform that runs those steps too, and Bruin itself runs as a CLI you can execute on the same serverless compute, keeping the whole pipeline serverless rather than just the ingestion step.

Related: the best data ingestion tools in 2026, the fastest open-source ingestion tool, and the hidden costs of DIY pipelines.

I work at Bruin, which makes ingestr and Bruin. Corrections welcome at support@getbruin.com.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.

The signup form is hosted by Brevo. Accept cookies to load it.