MaxStreamBufferMB

Per-connection upper bound on in-flight result-set buffers held by the driver while streaming. Off by default; set when you need to bound driver memory under a slow consumer.

Category: connection-string

Syntax

MaxStreamBufferMB=<integer>

Description

## What it controls The driver streams query results from the compute node as a sequence of Arrow batches. Each batch is decoded on a worker thread and queued for the consumer thread (the application calling `SQLFetch` / `SQLGetData`). `MaxStreamBufferMB`, when set, is the byte-size cap on that queue: when the queue holds this many bytes of in-flight batches, the worker thread blocks until the consumer drains. The cap exists so a fast engine plus a slow consumer cannot balloon driver memory. Without it, a `SELECT *` on a multi-million-row table can push gigabytes of decoded batches into the queue before the application finishes processing the first one. ## Default: off The byte cap is disabled by default. A previous build enabled a 2 GB default and showed a measurable regression on Windows / .NET PowerShell scans of wide mixed-schema tables, so the default was reverted to off. The connection-string knob, env var, and queue counter are still wired up; they just have no value to compare against unless the user opts in. ## Recommended values when you do opt in | Value | Use case | |---|---| | 512 MB | Containers, VMs, or pods with hard memory limits well below typical workstation RAM. | | 1 GB | Constrained host: a laptop with limited free memory or one running multiple BI tools concurrently. | | 2 GB | Developer laptop or desktop with 16+ GB RAM, comfortably below typical free memory even when other tools are running. | | 8 GB | ETL server or Power BI gateway running scheduled refreshes on dedicated hardware. | | 16 GB | Dedicated BI cluster head consolidating multiple analytical workloads, or a host with abundant RAM and large-result-set workloads. | ## Resolution order If `MaxStreamBufferMB=` is absent from the connection string, the driver falls back in this order: 1. The DSN's `MaxStreamBufferMB=` entry in `odbc.ini` (Linux / macOS) or the registry (Windows). The DSN editor's Compute tab exposes this as the **Streaming Memory** dropdown. 2. The `DF_MAX_INFLIGHT_BYTES` environment variable. Useful for tooling that cannot edit DSNs. 3. Disabled (no cap). The legacy alias `MaxStreamingMemoryMB=` is also accepted for backwards compatibility with earlier connection strings; both names route to the same internal cap. ## Multi-connection notes The cap is **per connection**, not per process. A Power BI Desktop session typically opens between one and four ODBC connections during a refresh; each gets its own cap allotment if one is set. A 2 GB cap x 4 connections is up to 8 GB across the process worst case. On a 16 GB laptop that is still fine; on a smaller host it may warrant lowering the cap. ## What it does not do - It does not bound engine-side execution memory (sort spills, hash-build state, etc.). Those are governed by the engine's own memory pool configuration. - It does not bound consumer-side allocation. Once the consumer reads a batch and copies values into its own buffers, that memory is the application's to manage. - A single oversize cell (a multi-gigabyte JSON or BINARY value) can ride as a one-row batch via the overflow rule and exceed the cap for that one batch. Workloads with such cells should provision the cap accordingly.

Parameters

NameTypeDescription
MaxStreamBufferMBMaximum bytes the driver will hold in its in-flight batch queue, in megabytes. `0` disables the cap (the default). Driver clamps non-zero values to `[4, 65536]`; values outside the range silently fall back to disabled.

Examples

MaxStreamBufferMB=512
MaxStreamBufferMB=2048
MaxStreamBufferMB=8192
Open in interactive docs →   DeltaForge home →