SCHEDULE

Declares a named schedule that decides WHEN a set of pipelines runs: on a cron expression, on a fixed interval, or behind other schedules once their runs finish. Also carries the operational policy for those runs (retries, timeout, concurrency, notifications).

Syntax

SCHEDULE <name>
  { CRON '<expression>' [TIMEZONE '<tz>']
  | INTERVAL_SECONDS <n>
  | AFTER '<parent>' [, '<parent>' ...] [PARENT_FRESHNESS <hours>] }
  [DESCRIPTION '<desc>']
  [RETRIES <n>] [RETRY_DELAY <seconds>]
  [TIMEOUT <seconds>]
  [MAX_CONCURRENT <n>] [MAX_WAVE_CONCURRENT <n>] [PRIORITY <n>]
  [CATCHUP true|false]
  [TARGET_NODES ALL | (<node_id1>, ...)]
  [NOTIFY '<email1>', ...] [WEBHOOK '<url1>', ...]
  [ACTIVE | INACTIVE]

Description

## Overview Declares a named schedule object that controls when and how pipelines are triggered. Schedules encapsulate all operational concerns: cron timing, retry behavior, execution timeouts, concurrency limits, compute node targeting, and notification routing. Pipelines reference schedules by name in their SCHEDULE clause. A single schedule can be shared across multiple pipelines, centralizing operational policy. ## Behavior - The CRON clause is the primary trigger mechanism. The expression is evaluated against the specified TIMEZONE (or UTC if no timezone is specified). - Clauses can appear in any order after the schedule name. The parser processes them until it encounters a semicolon. - RETRIES and RETRY_DELAY work together. When a triggered pipeline run fails, it is retried up to RETRIES times with RETRY_DELAY seconds between attempts. - TIMEOUT applies to each individual run attempt, including retries. A timed-out run counts as a failure for retry purposes. - MAX_CONCURRENT prevents overlapping runs. If a previous run triggered by this schedule is still executing and the concurrency limit is reached, the new trigger is queued. - PRIORITY determines execution order when multiple schedules fire simultaneously and the system is under load. - CATCHUP backfills missed triggers. When a schedule transitions from INACTIVE to ACTIVE, if CATCHUP is true, all cron firings that were missed during the disabled period are executed sequentially. This is useful for date-partitioned ETL where every time window must be processed. - TARGET_NODES ALL broadcasts the pipeline to every healthy compute node, enabling parallel execution across the cluster. Specifying individual node IDs restricts execution to those nodes only. - NOTIFY and WEBHOOK are additive. Both email and webhook notifications can be configured simultaneously. - ACTIVE and INACTIVE set the initial state of the schedule. An INACTIVE schedule is stored in the catalog but does not trigger pipeline runs until re-enabled. ## Access Control | Privilege | Object | Notes | |-----------|--------|-------| | Pipeline management | Workspace | Required to create and manage schedules within a workspace. | ## Compatibility SCHEDULE is a DeltaForge extension with no equivalent in standard SQL. The command is parsed as a first-class statement, not as a DDL variant.

Parameters

Name	Type	Description
name		Specifies the schedule name. This identifier is referenced by PIPELINE declarations via the SCHEDULE clause.
cron_expression		Specifies the cron expression that determines when the schedule fires. Uses the standard five-field format: minute (0-59), hour (0-23), day-of-month (1-31), month (1-12), day-of-week (0-6, where 0 is Sunday). Example: '0 6 * * *' fires daily at 06:00.
timezone		Specifies the IANA timezone for cron evaluation. Default: UTC. Examples: 'America/New_York', 'Europe/Berlin', 'Asia/Tokyo'.
retries		Specifies the maximum number of retry attempts for a failed pipeline run triggered by this schedule.
timeout_seconds		Specifies the maximum allowed execution time in seconds for a pipeline run triggered by this schedule. When exceeded, the run is terminated.
max_concurrent		How many runs of ANY ONE of this schedule's pipelines may be in flight at once (0 = unlimited). Default 1, which is the overlap guard: a pipeline still running is not started again by the next occurrence. To bound how many DIFFERENT pipelines of one fire run together, use MAX_WAVE_CONCURRENT.
interval_seconds		Fires the schedule every N seconds, the alternative to CRON. Must be positive. TIMEZONE does not apply and is ignored: an interval has no wall-clock field to interpret, so it is unaffected by daylight saving. Mutually exclusive with CRON and AFTER.
after		Runs this schedule BEHIND the named schedules instead of on a clock of its own. With one name it fires each time that parent's fire completes. With several it is a fan-in: it fires once ALL of them have completed a fire it has not already reacted to, which is what a step reading several independent sources needs. Completion means every run the parent queued reached a terminal state, whatever that state was; success is deliberately not required, so one failed link cannot park every downstream schedule indefinitely. Mutually exclusive with CRON and INTERVAL_SECONDS.
parent_freshness_hours		How recently every AFTER parent must have fired for this schedule's fire to count as running on current upstream data, in hours. 0 turns the check off. A parent outside the window does NOT hold the fire back: it is named on the schedule row and in the scheduler log, and the fire proceeds. Blocking would let one quiet upstream silently stop a downstream table updating with nothing failing anywhere to show it, while a fan-in step is almost always a rebuild that self-corrects on its next run.
max_wave_concurrent		How many DIFFERENT pipelines of one fire execute at the same time. 0 is unbounded. Because a fire's waves are gated (wave N+1 starts only once every member of wave N is terminal), this is the width of the running wave. Distinct from MAX_CONCURRENT, which bounds overlapping runs of any ONE pipeline. The bound belongs to the schedule because a wave's members usually share one upstream: a dozen pipelines reading one modest source can exhaust its connections while the estate still has worker capacity.

Examples

-- Basic daily schedule
SCHEDULE daily_etl
  CRON '0 6 * * *'
  TIMEZONE 'America/New_York';

-- Schedule with retry policy and timeout
SCHEDULE daily_etl
  CRON '0 6 * * *'
  TIMEZONE 'America/New_York'
  RETRIES 3
  RETRY_DELAY 300
  TIMEOUT 3600;

-- Schedule with notifications and concurrency control
SCHEDULE hourly_sync
  CRON '0 * * * *'
  DESCRIPTION 'Hourly data synchronization'
  MAX_CONCURRENT 1
  PRIORITY 10
  NOTIFY 'ops-team@example.com', 'data-eng@example.com'
  WEBHOOK 'https://hooks.slack.example.com/services/T00/B00/xxx';

-- Schedule targeting all compute nodes
SCHEDULE distributed_refresh
  CRON '0 3 * * *'
  TARGET_NODES ALL
  TIMEOUT 7200
  RETRIES 1;

-- Disabled schedule with catchup enabled
SCHEDULE weekly_archive
  CRON '0 1 * * 0'
  TIMEZONE 'Europe/London'
  CATCHUP true
  INACTIVE;

-- A fan-in step: rebuild the cross-source fact once all three loads finish.
-- One name would be a plain chain; three make it wait for all of them.
SCHEDULE gold_cross_source
  AFTER 'apc_daily', 'norled_daily', 'mpc_daily'
  PARENT_FRESHNESS 26
  DESCRIPTION 'Runs behind the three source loads';

Pitfalls

• Enabling CATCHUP on a schedule that was disabled for a long period can trigger a large backlog of runs. Evaluate the backfill volume before enabling this option.
• TARGET_NODES ALL executes the pipeline on every healthy node in parallel. For pipelines that write to shared tables, this can cause write conflicts unless the pipeline logic is designed for distributed execution.
• TIMEOUT applies per run attempt, not to the total elapsed time across all retries. A schedule with RETRIES 3 and TIMEOUT 3600 could run for up to 4 hours (including the initial attempt).
• A schedule is driven by exactly one of CRON, INTERVAL_SECONDS, and AFTER. Declaring two of them leaves the schedule UNARMED and it never fires, rather than one driver silently winning; GET /schedules/{id}/plan reports the contradiction.
• An AFTER name that no schedule defines holds the chain permanently: the child never becomes ready, and nothing fails to show it. Check the plan endpoint after renaming a parent.
• AFTER waits for its parents to FINISH, not to SUCCEED. A chained step will run after a failed parent. Work that must not run on bad upstream data belongs in the same schedule as its parent, where the batch's fail-fast cancels it.
• MAX_CONCURRENT and MAX_WAVE_CONCURRENT are different limits. MAX_CONCURRENT 1 (the default) stops one pipeline overlapping itself; it does not serialise the schedule. Setting MAX_WAVE_CONCURRENT 1 is what makes a fire run its pipelines one at a time.

See Also

• PIPELINE