Semicolon-terminated statement splitter that respects single-quoted string literals and `--` line comments. Implemented in `demo::split_sql_statements` and shared by `run` and stdin execution.
## Overview DeltaForge's CLI splits a script into individual statements with a purpose-built tokenizer in `demo::split_sql_statements`. The splitter is shared by `run`, piped-stdin execution, and the internal demo-test runner, so all three paths see the same statement boundaries. ## Behavior - Input is iterated character by character with two pieces of state: `in_single_quote` and `in_line_comment`. The splitter is otherwise stateless. - A `;` at top level (not inside a single-quoted string and not inside a line comment) terminates the current statement. Any text before the `;` is trimmed and, if it has at least one non-empty non-comment line, pushed onto the statement list. - Single quotes toggle `in_single_quote`. Two consecutive `'` characters therefore toggle the flag off and back on, which matches the SQL-standard doubled-quote escape for literal apostrophes. - `--` starts a line comment that runs to the next `\n`. Semicolons inside a line comment are preserved as text (they do not split the statement). - Characters inside a line comment or a single-quoted string are appended to the current statement unchanged. The comment or quote is kept verbatim in the SQL sent to the engine. - The final buffer (after the last `;` or EOF with no trailing semicolon) is pushed as a statement if it has SQL content. - Statements that contain only whitespace or only `--`-commented lines are dropped, so trailing `;` and stray comment blocks do not produce empty requests. ## Compatibility - Same function powers `run <FILE>`, `delta-forge-cli < file.sql` piped stdin, and the demo-test setup, queries, and cleanup phases. - The splitter is a pragmatic recognizer, not a full SQL lexer. It tracks only single-quoted strings and `--` line comments; it does not recognise double-quoted identifiers, backtick quoting, dollar-quoted strings, or `/* ... */` block comments. - Multi-statement requests are never batched: each split produces one HTTP request to the compute engine.
-- Semicolons inside single-quoted strings are preserved
INSERT INTO t VALUES ('a;b');
SELECT * FROM t;-- Line comments suppress splitting until end of line
-- DROP TABLE x; this semicolon is ignored
SELECT 1;-- Empty and comment-only blocks are skipped; only statements
-- containing at least one non-empty, non-comment line are sent
-- HEADER ONLY
;
-- another
;
SELECT 1;-- Escaped single quotes (SQL-standard doubled quote) work: the
-- splitter toggles the quote flag on each ' character, so '' leaves
-- the flag in its original state
INSERT INTO t VALUES ('it''s fine; really');
SELECT * FROM t;