Register pipelines with freshness checks on mart and dashboard tables. DataXPipe records check results per run and surfaces failures in the app before BI tools show wrong numbers.

Outcomes

✓Alert on stale marts, not just failed tasks
✓Tie check failures to run history
✓Explain blast radius with lineage

Quality checksRun historyLineageAI failure explanation

Analytics engineer, BI lead

Freshness SLAs on executive dashboards

Define max delay on mart tables that feed KPI dashboards; fail or warn when data is older than the business expects.

Read guide →

Problem

Executives trust dashboards that can be 24–48 hours stale because nobody monitors table freshness separately from job status.

How DataXPipe helps

Add freshness checks in the spec (max_delay_minutes or threshold_hours). Checks run after transforms and post pass/fail to the catalog linked to the run.

Outcomes

✓SLA aligned to business hours, not cron success
✓Warn vs error severity for degraded data
✓Historical check results for postmortems

Freshness checksSeverity levelsRun-linked results

On-call data engineer

Faster incident response with lineage + AI explain

When a check fails, see likely upstream causes, recommended actions, and dashboards at risk — not a raw SQL error.

Read guide →

Problem

Check failures trigger pages with no context. Engineers grep logs and Slack to learn which upstream job or filter caused stale data.

How DataXPipe helps

Use failure explanation with lineage context: summary, blast radius, likely causes, and recommended verification steps. Guided demo shows the flow without warehouse setup.

Outcomes

✓Shorter mean time to diagnosis
✓Structured postmortem inputs
✓Stakeholders see which dashboards are affected

AI failure explanationLineageGet started demo

Data engineer, SRE

Run history and check observability

Every pipeline run records status, timing, and linked check results — query from the app or Catalog API.

Read guide →

Problem

Orchestrator UI shows task logs but not structured check outcomes or cross-pipeline run search.

How DataXPipe helps

Trigger runs from the Pipelines page or post run events from Airflow. View recent runs globally, drill into check pass/fail payloads, and integrate Prometheus metrics from the API.

Outcomes

✓Unified run + check timeline
✓API-accessible run history
✓Foundation for alerting integrations

Runs pageCheck results APIPrometheus metrics

Analytics engineering manager

Catch wrong-but-green KPI drift

Pipeline ran, schema fine, rows present — but the metric moved for the wrong reason.

Problem

Freshness passes while join fan-out or null spikes corrupt executive KPIs.

How DataXPipe helps

Add kpi_baseline, row_count_drift, and distribution_drift checks. Failures revoke passports and explain vs the last trusted warrant.

Outcomes

✓Semantic drift, not just staleness
✓Baselines from prior passing runs
✓Tied to passport revoke + alerts

Semantic / KPI driftQuality checksPipeline Passports

Catalog & lineage

Know what you have and what depends on it.

Platform team, head of data

Central pipeline catalog

One searchable registry for pipeline specs, owners, schedules, connections, and environments — not scattered YAML in repos.

Read guide →

Problem

Pipeline metadata lives in Git, Confluence, and engineers' heads. Nobody can answer how many production pipelines exist or who owns orders_sync.

How DataXPipe helps

Register YAML specs via the Catalog API or app UI. Every pipeline stores sources, targets, checks, lineage, and ownership in one org-scoped catalog.

Outcomes

✓Single source of truth for pipeline metadata
✓Query specs from CI, Airflow, or the app
✓Environment and owner tags on every entry

Pipeline catalogREST APIUpload YAML

dbt developer, analytics engineer

Import dbt projects into the catalog

Paste manifest.json to generate lineage edges, default freshness checks, and a catalog entry without rewriting models.

Read guide →

Problem

dbt knows model dependencies but your orchestration and observability stack does not. Standing up lineage elsewhere means duplicate work.

How DataXPipe helps

Use the guided setup or import API to convert dbt manifest nodes into sources, targets, lineage edges, and starter checks — then register or iterate in YAML.

Outcomes

✓Bootstrap catalog from existing dbt work
✓Lineage graph in minutes, not weeks
✓Bridge dbt transforms with orchestration metadata

dbt manifest importLineageGet started wizard

Airflow operator, data platform engineer

Import Airflow DAG structure

Extract task graphs from existing DAG source to seed catalog lineage before a full spec migration.

Problem

Dozens of DAGs run in production but no catalog documents task dependencies or downstream datasets.

How DataXPipe helps

Import DAG Python source to infer task IDs and edges, then enrich with dataset-level lineage and checks in a declarative spec.

Outcomes

✓Document legacy Airflow without big-bang rewrite
✓Task-level graph as starting point for dataset lineage
✓Gradual migration to spec-driven generation

Airflow importSpec-driven generationLineage

Platform admin

Register pipelines from YAML specs

Connections, sources, targets, checks, and lineage in one file — validate in the app, register to your org catalog.

Read guide →

Problem

Warehouse credentials, transform logic, and quality rules drift across repos, env vars, and wiki pages.

How DataXPipe helps

Author a pipeline YAML spec with a connections block and pipeline metadata. Validate semantics in the UI, then register — connections appear on the Connections page automatically.

Outcomes

✓One contract for credentials and pipeline shape
✓Validate before deploy
✓Free tier includes 2 pipelines to learn the flow

YAML uploadJSON Schema validationConnections registry

Data engineer, architect

Impact analysis before schema changes

Query upstream and downstream dependencies before altering a column, model, or source extract.

Read guide →

Problem

A small change to raw.orders breaks three marts and two Looker explores — discovered only after deploy.

How DataXPipe helps

Lineage edges register with the pipeline spec. Query by dataset ID to list downstream targets and plan migrations with actual dependency data.

Outcomes

✓Blast radius visible before merge
✓Fewer surprise breakages in production
✓Onboarding engineers trace data flow in the app

Lineage graphCatalog APIDataset registry

Data engineer

Postgres, Neon, Snowflake, and BigQuery in one spec

Declare warehouse connections once in YAML; sources and targets reference them by connection_ref.

Read guide →

Problem

Connection strings duplicate across Airflow connections, dbt profiles, and check scripts.

How DataXPipe helps

Define connections in the pipeline spec. Registration stores them in the org connection registry for reuse and documents which pipelines touch which warehouses.

Outcomes

✓Connection registry per organization
✓Clear mapping pipelines → warehouses
✓Smoke-test templates for Neon/Postgres

Connections pageYAML connections blockConnector configs

Platform team building golden paths

Generate Airflow DAGs and check scripts from specs

Validated YAML produces orchestration artifacts, SQL transforms, runnable checks, and metadata bundles.

Read guide →

Problem

Every new pipeline copies boilerplate for Catalog callbacks, check posting, and lineage registration.

How DataXPipe helps

Run the generator on a validated spec to emit Airflow DAGs, SQL, tests, and metadata JSON ready to deploy — aligned with what the catalog expects.

Outcomes

✓Golden path scaffolding for all pipelines
✓Metadata stays aligned with generated code
✓Less copy-paste across repos

Spec-driven generationAirflow integrationCheck harnesses

Analytics lead, compliance

Share Pipeline Passports with execs

Prove a KPI is still warranted — fresh, checked, within SLA — without Slack archaeology.

Problem

Stakeholders ask 'can I trust this number?' Teams screenshot dashboards and hope.

How DataXPipe helps

DataXPipe issues a Pipeline Passport when checks pass and revokes it on silent failure or drift. Share the public token link with finance or auditors.

Outcomes

✓Portable trust artifact per dataset
✓Auto-revoke on check failure
✓SLA-bound expiry built into the warrant

Pipeline PassportsFreshness checksAlerts

Onboarding & teams

Get from zero to governed pipelines fast.

Platform admin, engineering manager

Onboard data teams with roles

Platform admins register pipelines and invite developers with read-only catalog access and check execution.

Read guide →

Problem

Everyone shares one API key or gets over-permissioned access to production registration endpoints.

How DataXPipe helps

Invite teammates by email. Developers receive personal API keys scoped to the org with developer role; platform admins retain manage:pipelines. UI disables actions the role cannot perform.

Outcomes

✓Least-privilege by default
✓Personal keys per teammate
✓Clear role labels in Settings and sidebar

Team invitesRBACPer-user API keys

Individual contributor

Self-service API key recovery

Forgot your key? Email a recovery link, rotate credentials, and copy the new key before continuing — no support ticket.

Problem

API keys are shown once at signup. Lost keys mean Slack messages to admins or shared credentials.

How DataXPipe helps

Request recovery from the sign-in page. Email link issues a new personal key; the app requires copying the key before redirecting. Admins can rotate keys from Settings.

Outcomes

✓Reduce support load for lost keys
✓Secure rotation invalidates old keys
✓Optional team email on admin-initiated rotate

Key recoveryKey rotationSettings

Platform engineer, DevOps

Validate specs in CI before register

Call the validate endpoint in GitHub Actions so broken YAML never reaches production registration.

Read guide →

Problem

Invalid specs merge to main and fail only at deploy time.

How DataXPipe helps

POST spec YAML to /api/v1/specs/validate in CI (no auth required). Block merge on schema or semantic errors; register only from protected branches with platform API keys.

Outcomes

✓Shift-left spec validation
✓Consistent errors in editor and CI
✓Safer self-serve pipeline registration

Spec validation APIJSON SchemaUpload YAML

Evaluator, startup data lead

Self-serve signup and guided evaluation

Create an organization, copy your API key, run the stale-table demo, and register a pipeline — without a sales call.

Read guide →

Problem

Metadata tools require lengthy procurement before you can test on your own data.

How DataXPipe helps

Sign up on app.dataxpipe.com, use Get started wizard (demo, dbt import, or YAML upload), and stay on free tier while evaluating. Upgrade when catalog size grows.

Outcomes

✓Minutes to first catalog entry
✓Demo path with no warehouse required
✓Free tier: 2 pipelines, 2 connections

Self-serve signupGet started wizardFree tier

Data engineer, platform

Preview blast radius on every PR

See downstream datasets, consumer contracts, and passports at risk before merging a pipeline change.

Problem

Schema and model PRs merge with tribal-knowledge impact analysis. Incidents appear Monday morning.

How DataXPipe helps

POST /api/v1/specs/impact diffs the proposed spec against the catalog and comments a blast-radius report on the pull request.

Outcomes

✓Know who you wake up before merge
✓Consumer contracts surface in CI
✓Passports at risk listed explicitly

PR blast-radius previewCI gatesConsumer contracts

Operations & compliance

Runs, incidents, audit trails, and scale.

SaaS operator, security reviewer

Multi-tenant pipeline catalogs

Each organization gets isolated pipelines, connections, API keys, and plan limits on shared infrastructure.

Read guide →

Problem

Building an internal catalog that later needs org isolation requires a painful retrofit.

How DataXPipe helps

DataXPipe SaaS enforces organization_id on catalog rows, resolves API keys to org + role, and applies plan limits (e.g. 2 pipelines on free tier) at registration time.

Outcomes

✓Tenant isolation by API key
✓Plan-based limits enforced in API
✓Stripe billing for Team and Business tiers

Multi-tenant APIPlan limitsBilling

Compliance officer, data governance lead

Compliance and audit documentation

Demonstrate data provenance and pipeline ownership for SOC 2, GDPR, or internal audit requests.

Read guide →

Problem

Auditors ask where metric X comes from; teams assemble screenshots and spreadsheets under deadline pressure.

How DataXPipe helps

Catalog stores owners, schedules, lineage edges, and check history per org. Export pipeline metadata and lineage queries as evidence of documented data flows.

Outcomes

✓Repeatable provenance answers
✓Ownership and environment tags on specs
✓Check history for control evidence

LineageCatalog APIOwnership metadata

Engineering manager, FinOps

Scale catalog size with plans

Start free; upgrade to Team or Business when pipeline count, connections, or retention requirements grow.

Read guide →

Problem

Tooling costs are unpredictable or require enterprise sales for modest team growth.

How DataXPipe helps

Free tier covers evaluation (2 pipelines). Team and Business plans raise limits via Stripe checkout in the app Billing page — limits enforced at API registration time.

Outcomes

✓Predictable upgrade path
✓Usage visible in app and billing UI
✓No surprise blocks — UI shows pipeline quota

BillingPlan limitsStripe integration

BI owner, analytics engineer

Consumer contracts for dashboard owners

Producers write the pipe. Consumers write the promise — max staleness and row floors that CI can enforce.

Problem

Dashboard owners discover breakage late; producer SLAs are informal.

How DataXPipe helps

Register consumer contracts that declare dataset inputs and staleness. Compatibility checks compare contracts to producer freshness and active passports.

Outcomes

✓Explicit producer/consumer handshake
✓At-risk status when SLAs diverge
✓Contracts appear in PR impact reports

Consumer contractsPipeline PassportsCI gates

Pick a starting point

Most teams begin with the guided demo or a YAML smoke test, then import dbt and invite the team.

Run guided setup Explore features Read the use cases guide