On completion, demo-test emits a structured JSON report on stdout with demo identity, per-phase status, per-statement details, timings, and errors. Progress messages go to stderr, so `>report.json` captures only the report.
delta-forge-cli demo-test ... >report.json## Overview `demo-test` emits a single JSON document on stdout when the run finishes, regardless of phase selection or success. All other output (authentication, node resolution, per-statement progress, result preview tables, the end-of-run summary banner) goes to stderr, so `command >report.json` captures only the report and `2>&1` is rarely what you want. ## Behavior - The report is printed via `serde_json::to_string_pretty`, so it is human-readable with two-space indentation. - Top-level fields: `demo_id`, `demo_name`, `zone_name`, `timestamp`, `total_queries`, `passed`, `failed`, `skipped`, `errors`, `auth_success`, `download_success`, `setup_success`, `setup_time_ms`, `setup_steps`, `total_execution_time_ms`, `queries`, `cleanup_success`, `cleanup_steps`. - `failed` and `skipped` are currently always 0. Query failures from the SQL engine appear in `errors` (with `status: "ERROR"` on the matching entry in `queries[]`). `failed` is reserved for future assertion-style checks. - `sample_rows` is omitted on a query entry when the result had no rows. `error_message` is omitted when the query succeeded. `message` and `error` on setup and cleanup steps follow the same optional pattern. - `timestamp` is generated at serialisation time, not at run start. For the full wall time use `total_execution_time_ms`. - The report is always printed before the process exits, including on exit code 1. The only case with no report on stdout is exit code 2, which is reserved for client-initialisation failures before the run begins. ## Compatibility - The schema is stable; regression tooling (including the `demo-validate` and `demo-run` skills) depends on the field names listed above. - Adding new fields is considered non-breaking. Consumers should ignore unknown fields. - To diff two runs, compare `queries | map({query_title, status, row_count})` after sorting; this is less sensitive to timing noise than comparing the whole report.
| Name | Type | Description |
|---|---|---|
demo_id | Demo identifier returned by the provision API. | |
demo_name | Human-readable demo name from the provision response. | |
zone_name | Target zone passed as the ZONE positional argument. | |
timestamp | RFC 3339 UTC timestamp captured when the report is serialised (end of run). | |
total_queries | Number of query blocks parsed out of `queries.sql` for this run. 0 when the queries phase did not run or no queries were parsed. | |
passed | Count of query results with `status: "PASS"`. | |
failed | Reserved for assertion-style failures. Currently always 0; query failures are reported under `errors`. | |
skipped | Reserved. Currently always 0. | |
errors | Count of query results with `status: "ERROR"`. | |
auth_success | True if initial authentication succeeded. When false the run aborts before reaching the report, so in practice this is always true in a printed report. | |
download_success | True if provision + download succeeded. When false the run aborts, so in practice this is always true in a printed report. | |
setup_success | True if the setup phase ran with zero ERROR statements. False if any setup statement failed. True when the setup phase did not run. | |
setup_time_ms | Wall time in milliseconds spent in the setup phase. 0 when setup did not run. | |
setup_steps | Per-statement setup results. Each element: `index` (1-based), `sql`, `status` (`OK` or `ERROR`), `message` (optional backend message on success), `error` (optional error string on failure). | |
total_execution_time_ms | Wall time in milliseconds for the entire command, measured from before authentication to just before the report is printed. | |
queries | Per-query results. Each element: `query_index`, `query_title`, `sql`, `status` (`PASS` or `ERROR`), `row_count`, `execution_time_ms`, `columns[]`, `sample_rows[][]` (up to 5 rows, omitted when empty), `error_message` (omitted on success). | |
cleanup_success | True if the cleanup phase ran with zero ERROR statements. True when cleanup did not run. | |
cleanup_steps | Per-statement cleanup results. Each element: `index` (1-based), `sql`, `status` (`OK` or `ERROR`), `error` (optional error string on failure). |
# Capture the report, inspect query errors
delta-forge-cli demo-test prod-node landing demos/csv/sales-quickstart >report.json
jq '.queries[] | select(.status == "ERROR") | {query_title, error_message}' report.json# Total wall time across every phase
jq '.total_execution_time_ms' report.json# Did any setup statement fail?
jq '.setup_success' report.json# Regression diff between two runs
diff <(jq -S '.queries | map({query_title, status, row_count})' a.json) \
<(jq -S '.queries | map({query_title, status, row_count})' b.json)