Define, manage, and monitor multi-step SQL data pipelines.
## Overview The Pipelines page is the primary interface for defining, managing, and monitoring multi-step SQL data pipelines that transform and load data across the catalog. Data engineers use this page to register pipeline SQL files, manage pipeline lifecycle states, review run history, and associate pipelines with cron schedules for automated execution. Each pipeline belongs to a workspace and is identified by its file path. Pipelines progress through lifecycle states (draft, active, paused, archived) and support approval workflows that gate promotion from draft to active. The run history panel provides a direct link to the Executions page for detailed step-level diagnostics. ## Key Features - **Pipeline creation from SQL files.** Register a pipeline by selecting a SQL file that contains the ordered steps. The pipeline SQL must be idempotent so that re-execution produces consistent results without manual intervention. - **Lifecycle management (activate, pause, resume, archive).** Transition pipelines through well-defined states. Activate a draft pipeline to make it eligible for scheduled runs. Pause a running pipeline to temporarily suspend execution. Archive pipelines that are no longer needed without deleting their history. - **Run history.** View a chronological list of all runs for a specific pipeline, including status, duration, trigger type, and outcome. Click through to the Executions page for step-level logs and error details. - **Approval workflows.** Require one or more approvals before a pipeline transitions from draft to active. Approvers review the pipeline SQL, dependencies, and target tables before granting activation. - **Schedule association.** Link a pipeline to one or more named schedules defined on the Schedules page. Schedules are independent, reusable objects; a single schedule can drive multiple pipelines, and a pipeline can be associated with multiple schedules. ## Workflow 1. Navigate to the Pipelines page from the Workflow sidebar. 2. Click the add pipeline button and select the SQL file that defines the pipeline steps. 3. Review the pipeline summary, including detected dependencies and target tables. 4. Submit the pipeline for approval if approval workflows are enabled. 5. After approval, activate the pipeline and associate it with a schedule. 6. Monitor subsequent runs from the run history panel or the Executions page. 7. Pause or archive the pipeline when it is no longer required for active data processing.