Pre-builds and persists Vulkan-derived GPU buffers (w_out, in_edge_weights) for a named graph as a compact .dgpu sidecar co-located with the .dcsr topology file, enabling subsequent GPU graph algorithm dispatches to skip the per-query buffer-derivation pass.
CREATE GRAPHGPU <qualified_graph_name>## Overview CREATE GRAPHGPU pre-computes and persists the Vulkan-specific GPU derived buffers for an existing graph definition and writes them as a compact binary .dgpu file. The sidecar is co-located with the corresponding .dcsr CSR topology file (same `_deltaforge/graph/` directory). Subsequent GPU graph algorithm dispatches that consume these buffers (PageRank, ArticleRank, Eigenvector centrality, and any algorithm that needs per-vertex outgoing weight sums or per-reverse-edge weight values) can load them directly from disk instead of running the GPU derivation kernels on every query. The two buffers persisted in v1 of the format are: - `w_out` (length: node_count, f32): per-vertex sum of outgoing edge weights, used by PageRank's pre-divide pass and by Eigenvector centrality's L2 normalization step. - `in_edge_weights` (length: edge_count, f32): per-incoming-edge weight, gathered through the reverse CSR. Used by every iterative centrality kernel that walks predecessors with weights. ## Prerequisites - The graph must already be defined via CREATE GRAPH. - The `.dcsr` topology sidecar must already exist on disk; run CREATE GRAPHCSR first. The .dgpu reader validates that the .dcsr's `(edge_version, vertex_version)` match the .dgpu header, so the two sidecars must be built against the same Delta table commit. - A Vulkan compute-capable adapter (NVIDIA, AMD, or Intel with a current vendor driver) must be present on the host. If `VulkanContext::get_or_init()` returns None, CREATE GRAPHGPU fails with a clear error message; the graph is still queryable through the CPU path. ## Execution Steps 1. The engine resolves the graph by its qualified name and locates the `.dcsr` sidecar in `{edge_table_path}/_deltaforge/graph/{graph_name}.dcsr`. If the .dcsr is missing, the command fails with a message instructing the operator to run CREATE GRAPHCSR first. 2. The CSR is loaded into memory (using the existing disk read path) and weights are repopulated from the source Delta parquet. 3. A Vulkan context is acquired via the shared `VulkanContext::get_or_init()` singleton. 4. The CSR row_ptr, edges, edge_weights, in_row_ptr, and in_edges arrays are uploaded to device-local GPU buffers via `VkGraphBuffers::upload`. 5. Two compute pipelines are compiled and cached: `pr_build_w_out` (per-vertex outgoing weight sum) and `pr_build_in_edge_weights` (edge-parallel reverse-CSR weight gather, with binary search on the reverse row pointer to find the destination vertex). 6. Both kernels are dispatched in a single command buffer, then a transfer barrier is issued and the results are copied back to host-visible staging buffers. 7. The `.dgpu` file is written atomically (temp file plus rename) to `{edge_table_path}/_deltaforge/graph/{graph_name}.dgpu`. The header records the `(edge_version, vertex_version)` fingerprints inherited from the .dcsr so the reader can detect mismatches. 8. All transient Vulkan resources (descriptor pool, pipelines, pipeline layouts, descriptor set layouts, shader modules, staging buffers) are destroyed before the function returns. ## Disk Format The .dgpu file uses a fixed 40-byte header plus per-section length-prefixed payloads (format v1): ``` +-----------------------------------------------+ | Header (40 bytes, uncompressed) | | magic [u8; 4] = b"DGPU" | | format_version u32 = 1 | | base_edge_version i64 (from .dcsr) | | base_vertex_version i64 (from .dcsr) | | node_count u32 | | edge_count u32 | | _reserved [u8; 4] | +-----------------------------------------------+ | Section: W_OUT | | section_marker u32 = 1 | | length_bytes u64 | | data (node_count * 4 bytes, f32 LE) | +-----------------------------------------------+ | Section: IN_EDGE_WEIGHTS | | section_marker u32 = 2 | | length_bytes u64 | | data (edge_count * 4 bytes, f32 LE) | +-----------------------------------------------+ ``` Values are little-endian. The reader rejects any file whose magic, format_version, or version pair do not match the corresponding .dcsr; on mismatch the caller is expected to either rebuild via CREATE GRAPHGPU or fall back to the per-query GPU derivation path. ## Performance Characteristics The derivation kernels are inexpensive in isolation (sub-millisecond on a mid-range discrete GPU for a 10M-edge graph), so the value of CREATE GRAPHGPU is not about saving compute. It is about avoiding the GPU upload + derivation latency on the first query after process start, and amortizing buffer-derivation cost across many short PageRank invocations on the same graph. The sidecar is small relative to the CSR (one f32 per vertex plus one f32 per edge, so roughly `4 * (node_count + edge_count)` bytes uncompressed). For a 50M-edge graph this is around 200 MB. ## Cache Invalidation The .dgpu sidecar is keyed to the `(edge_version, vertex_version)` of its source .dcsr. When the CSR is rebuilt at a new Delta version (via CREATE GRAPHCSR or REBUILD GRAPHCSR), the existing .dgpu is left on disk but is implicitly stale; the next reader will see a version mismatch and reject the file. Operators should re-run CREATE GRAPHGPU after any CREATE GRAPHCSR or REBUILD GRAPHCSR to keep the GPU sidecar in lockstep. ## Access Control | Privilege | Object | Notes | |-----------|--------|-------| | READ or higher | Vertex table | Required to load the CSR for buffer derivation. | | READ or higher | Edge table | Required to load weights into the CSR. | | WRITE | Edge table storage path | Required to write the .dgpu sidecar into the `_deltaforge/graph/` directory. | ## Compatibility CREATE GRAPHGPU is a DeltaForge extension and has no equivalent in standard SQL or openCypher. It is the GPU counterpart of CREATE GRAPHCSR and follows the same lifecycle conventions (explicit user-driven build, atomic temp-and-rename write, version fingerprint in the header).
| Name | Type | Description |
|---|---|---|
name | Fully qualified graph name in the form zone.schema.graph_name. Short names are rejected because two graphs in different zones can share a short name and the sidecar would land in the wrong location. |
-- Build the GPU sidecar after the CSR is in place
CREATE GRAPHCSR demo_zone.gpu_finance_network.gpu_finance_network;
CREATE GRAPHGPU demo_zone.gpu_finance_network.gpu_finance_network;-- Refresh the GPU sidecar after rebuilding the CSR
CREATE GRAPHCSR analytics.social.friendship_graph;
CREATE GRAPHGPU analytics.social.friendship_graph;-- Typical workflow: define graph, build CSR, build GPU sidecar, run PageRank
CREATE GRAPH IF NOT EXISTS warehouse.shipping.routes
VERTEX TABLE warehouse.shipping.locations ID COLUMN loc_id
EDGE TABLE warehouse.shipping.shipments SOURCE COLUMN src TARGET COLUMN dst
WEIGHT COLUMN tonnage
DIRECTED;
CREATE GRAPHCSR warehouse.shipping.routes;
CREATE GRAPHGPU warehouse.shipping.routes;
USE warehouse.shipping.routes;
CALL algo.pageRank() YIELD node, score RETURN node, score ORDER BY score DESC LIMIT 10;