Demos

Reference index of 85 demos in DeltaForge: complete catalogue grouped by category, each linking to detailed syntax and examples.

avro

• Avro E-Commerce Orders: Logical Types & Nullable UnionsRead 80 e-commerce orders across two Avro files with mixed codecs, exercising date/timestamp-millis logical...
• Avro Insurance Claims: Schema Evolution Across File VersionsRead 90 insurance claims across three Avro files where v2 adds two new fields, validating automatic...
• Avro IoT Sensors: Smart Building Telemetry at ScaleRead 2,500 sensor readings from 5 building-floor Avro files with schema evolution (v1→v2), mixed codecs...

csv

• CSV Advanced Options TestbenchEight CSV external tables, each exercising one parsing option (delimiter, null_value, comment_char...
• Northwind Trading CompanyClassic Northwind sample database as 11 semicolon-delimited CSV external tables (830 orders, 2,155 line...
• Sales Schema EvolutionSingle external table reads five quarterly CSVs (2024-Q1 to 2025-Q1) whose columns evolve over time; missing...
• Veterinary Clinic Patient RecordsThree-branch veterinary clinic with 75 visits across branch-north/south/east subdirectories; demonstrates...

delta

• CHECK Constraints and Table Properties LifecycleDeclare CHECK constraints inside TBLPROPERTIES and prove they survive every UPDATE, DELETE, and re-INSERT --...
• Change Data Feed -- Row-Level Change TrackingEnable Change Data Feed to emit per-row insert / update_preimage / update_postimage / delete records so...
• Delta Basics -- Create, Insert, Update, DeleteWalk through every CRUD operation on a Delta table and watch the transaction log create new versions you can...
• MERGE Patterns -- Slowly Changing Dimension Type 2 (SCD2)Preserve the full audit trail of dimension changes by expiring current rows with MERGE and inserting new...
• OPTIMIZE then VACUUM -- The Delta Maintenance PlaybookFix the small-files problem caused by daily micro-batch ingestion by compacting with OPTIMIZE and then...
• Partitioning and Dynamic Partition PruningPartition a Delta fact table by region and watch the engine skip entire partition directories when a JOIN or...
• Schema Evolution -- Add Columns and Backfill NULLsAdd new columns to a live Delta table with zero data rewrite, rely on implicit NULL fill for historical...
• Time Travel and RESTORE -- Inspect Before You RecoverRecover from an accidental bulk UPDATE by time-travelling through versions, picking the last good snapshot...

edi

• Advanced JSON Extraction: On-Demand Segment Access via df_transaction_jsonExtracts deeply-nested X12 segments ad hoc using SQL JSON functions, json_typeof, json_array_length...
• EDIFACT Basics: UNB/UNH Envelopes Across DirectoriesParses 22 UN/EDIFACT and EANCOM messages (ORDERS, ORDRSP, INVOIC, CUSCAR, BAPLIE, PAXLST, PNRGOV, DESADV...
• HIPAA Claims Financial, 837 Charges JOIN 835 RemittanceCharge-to-payment reconciliation across three 837 healthcare claims (professional, dental, institutional)...
• Order Lifecycle: Cross-Document 850/855/856/810/861 TraceabilityTraces a purchase order through the full order-to-cash cycle: PO creation (850) to acknowledgment (855) to...
• Repeating Segments & Loops: Indexed / Concatenate / ToJson ModesThree external tables over the same 14 X12 files show how DeltaForge materializes multi-occurrence segments...
• X12 Supply Chain Basics: ISA/GS/ST Envelope + Materialized FieldsParses 14 X12 EDI transactions (850/810/855/856/857/861/997/824) into two external tables, a compact view...

excel

• Excel Multi-Sheet ReportingExpose three sheets (Sales, Returns, Staff) from two regional workbooks as separate external tables and JOIN...
• Excel Options TestbenchValidate each Excel connector option (sheet_name, has_header, skip_rows, max_rows, file_filter) with...
• Excel Sales Analytics: Superstore OrdersUnify four annual Superstore XLSX files (9,994 orders) into a single external table, then isolate one year...

fhir

• FHIR Clinical Observations: Vital Signs & Lab ResultsFlatten HL7 FHIR R5 Observation resources, heart rate NDJSON bulk exports, vital-sign panels, lab results...
• FHIR Clinical Records: Conditions, Procedures & AllergiesIngest three related HL7 FHIR R5 clinical resource types (Condition, Procedure, AllergyIntolerance) from a...
• FHIR Medications: Prescriptions & CoverageIngest HL7 FHIR R5 MedicationRequest and Coverage resources, including contained Medication definitions...
• FHIR Patient Demographics: BasicsIngest HL7 FHIR R5 Patient resources from bulk NDJSON and individual JSON files, flattening nested name...
• FHIR XML Clinical ResourcesIngest native HL7 FHIR R4 XML resources (Patient, Observation), handling the default FHIR namespace...

graph

• GPU Banking Fraud Analytics, 10M Accounts, 48M TransactionsGPU-accelerated fraud and influence analytics on a 10M-account banking network with 48M transactions...
• Global Shipping Routes: Weighted Shortest Path and MSTRoute optimization on 25 world ports and 55 weighted shipping lanes: Dijkstra shortest path, BFS depth...
• Hospital Referral Network: Graph DML (INSERT, UPDATE, DELETE)Mutate a live graph via standard DML on its backing Delta tables, verify changes with Cypher, and re-run...
• Karate Club: Graph Basics, Ingestion, and Verified AlgorithmsLoad Zachary's Karate Club into Delta tables, build an undirected graph, and run Cypher patterns plus 15...
• Research Collaboration Network: Advanced Cypher PatternsUse advanced Cypher on a university collaboration graph (40 researchers, 170 directed edges, 4 relationship...
• Sales Territory Optimization: SQL + Cypher InteropMix Cypher and SQL in a single query: run graph algorithms via the cypher() table function, persist results...

hl7

• HL7 Clinical Workflows - Documents, Scheduling & Edge CasesParse MDM clinical documents, SIU appointment bookings, and a parser-torture ADT with all six HL7 escape...
• HL7 Lab Orders & Results - ORM/ORU WorkflowModel the end-to-end lab workflow with HL7 v2 ORM orders and ORU results, including abnormal flags, value...
• HL7 Patient Administration - ADT LifecycleUnify HL7 v2 ADT admit/update/discharge messages from six EHRs across v2.3 through v2.6 into queryable...

iceberg

• Iceberg V1: Warehouse Inventory SnapshotsRegister a native Apache Iceberg v1 table (489 retail SKUs across 3 warehouses) and run inventory analytics...
• Iceberg V2: Fleet Telemetry AnalyticsRead a native Iceberg v2 table (450 fleet GPS pings) and validate enhanced per-column manifest stats with...
• Iceberg V3: Deletion Vectors (Puffin)Read a native Iceberg v3 table whose 36 faulty-scanner rows are retracted via a Puffin-encoded deletion...

json

• JSON Country Factbook — Deep Nesting & Schema EvolutionExtract deeply nested (4+ level) JSON fields across 10 heterogeneous country files, with exclude_paths and...
• JSON Customers — BasicsRead a flat JSON array file into an external table with include_paths, type inference, and file_metadata.
• JSON Music Catalog — Nested Arrays & ExplodeExplode nested track arrays into per-track rows while also producing a per-album summary that keeps the...
• JSON Subtree Capture — json_pathsCapture complex nested subtrees as serialized JSON string columns with json_paths, and contrast against full...

orc

• ORC Clinical Trials: NULL Handling & CASE LogicExercise ORC's NULL bitmap on a heavily-nullable patient dataset using COALESCE, NULLIF, CASE, and string...
• ORC Energy Meters: Advanced Aggregation at ScaleAggregate 1,500 smart-meter readings across 3 monthly ORC files using HAVING, COUNT DISTINCT, FILTER...
• ORC Insurance Claims: Cross-Table JOINs & SubqueriesJoin two ORC-backed external tables (policies, claims) and prove INNER/LEFT/anti-joins, EXISTS, and...
• ORC Server Logs: Schema Evolution & Glob FilteringRead 5 HTTP access-log ORC files with two schema versions and prove schema evolution, LOCATION glob...
• ORC Warehouse Inventory: Window Functions on Mixed NumericsApply ROW_NUMBER, RANK, LAG/LEAD, NTILE, and running totals to 100 rows of warehouse inventory stored across...

parquet

• Parquet Flight Delays: Airline Delay AnalysisRead three quarterly Parquet files with schema evolution (NULL-filled columns), plus file_filter and...
• Parquet Supply Chain: Order AnalyticsRecursively scan 14 quarterly Parquet files across year-based subdirectories with file_filter, max_rows...

protobuf

• Protobuf Address Book - Contact DirectoryRead proto3 AddressBook binaries, flatten nested PhoneNumber messages, decode PhoneType enums, and handle...
• Protobuf Freight Shipping - Multi-Carrier ManifestRead multi-carrier proto3 ShippingManifest binaries to exercise bool/int64/float scalars, 3-level nesting...
• Protobuf IoT Sensor Network - Manufacturing Floor SensorsRead proto3 SensorNetwork binaries across two factory floors and a warehouse (20 sensors, 82 readings) to...

pseudonymisation

• Pseudonymisation Apply: Clinical Trial De-identificationPharma de-identification workflow: register query-time rules on a clinical-trial table, verify transforms...
• Pseudonymisation Healthcare: HL7, FHIR, and EDI De-identificationExercise all seven transform types and all three linkability scopes across three healthcare formats: HL7 v2...
• Pseudonymisation Lifecycle: Insurance ClaimsInsurance-claims walkthrough of the full pseudonymisation rule lifecycle: CREATE five rules with PRIORITY...
• Pseudonymisation Quickstart: Banking KYCRegister four query-time pseudonymisation rules on a retail-bank KYC table, redact SSN, mask phone...

spatial

• GIS Emergency Response Network: Multi-Step Spatial AnalyticsNYC dispatch scenario combining st_distance nearest-neighbor ranking, multi-algorithm distance comparison...
• GIS Maritime Shipping: PostGIS-Compatible Geospatial FunctionsExercises all 18 PostGIS-compatible st_* functions against 5 cargo vessels navigating 10 world ports with 40...
• H3 + GIS Delivery Optimization: Cross-Function Spatial AnalyticsCombines H3 hexagonal indexing with GIS distance/bearing functions across 3 warehouses and 15 stores to rank...
• H3 GPS Fleet Tracker, 10K Pings, 21 H3 Functions10,000 deterministic GPS pings across 5 world cities, indexed with H3 resolution-9 cells and joined against...
• H3 Point-in-Polygon, 1M Row Geofencing PerformanceRide-share geofencing across 12 pricing zones and 1,000,000 driver GPS positions in 8 world cities, converts...

subsurface

• 2D Survey CDP Navigation IndexA processing centre delivers a reprocessed 2D seismic survey one line at a time, and each line is indexed...
• Aerial Survey Raster CatalogueAn environmental baseline survey delivers orthophoto and elevation tiles for a licence area, and before...
• Depth Surface Handover and Gross Rock VolumeA geophysicist depth-converts a seismic interpretation and hands the result to the mapping team as ZMAP+...
• Drilling Survey Anti-CollisionA directional driller hands over the definitive survey at the end of each bit run, as WITSML, and the well...
• Induced Seismicity MonitoringProduced water from oil and gas production is disposed of by injecting it back underground, and in Oklahoma...
• Licence Divestment Data RoomA buyer's subsurface team has three weeks with the data room for Dutch block P/12 before the bid is due, and...
• Monthly Production AllocationAn operator files monthly produced volumes per facility with the regulator as PRODML, one document per...
• North Sea Demo Field, Five Formats JoinedFive upstream energy formats over one field, read in place with no export step, and then joined: a SEG-Y...
• Offshore Lease FootprintEvery active oil and gas lease on the United States Outer Continental Shelf, as the Bureau of Ocean Energy...
• Reservoir Model Handover AuditA partner delivers a static reservoir model for a unitisation study as a RESQML package, and before the...
• Reservoir Simulation History Match ReviewA reservoir engineer history matches a field before a development plan goes in. Each iteration is a full...
• Seismic Crew Field Record QCA land seismic crew drops one folder of SEG-D field records per acquisition day, and a scheduled loader...
• Static Model Deck IngestionA static-model archive receives ECLIPSE GRDECL decks from two places and the simulation team loads both to...
• Survey Navigation Shot Point DatabaseA survey contractor delivers 2D navigation as UKOOA P1/90, one file per line, and the client builds a...
• Volve Composite Log IngestionReal Volve DLIS composite logs, ingested incrementally into an analysis-ready Delta table. Two deliveries...
• Well Log Library ConsolidationAn operator takes a position in Dutch acreage and receives the regulator's public log package with it: seven...
• Well Log Tape Archive RecoveryAn operator's data-management team brings a LIS-79 well-log tape archive into the lake without a vendor...

xml

• XML Books: Schema EvolutionUnion 5 yearly XML catalog exports (2000-2004) into one external table, with added/dropped elements becoming...
• XML E-Commerce: Order Line ExplosionExplode 3+ level order XML into per-line-item rows with CDATA descriptions, exclude internal audit blocks...
• XML NYT News: RSS Feed AnalysisRead 7 regional NYT RSS XML feeds (231 items, 4 namespaces) into a joined-categories article table and an...
• XML Subtree Capture, xml_paths & nested_output_formatCapture complex XML subtrees (specifications, supplier) as single string columns in JSON or raw XML form...