Limits

Limits that apply to Infino Cloud connections. The open-source engine running locally has no request cap, because there is no request: writes go straight to storage.

Request body: 128 MiB

A single data-plane request body is capped at 128 MiB. In practice this applies to append and update, whose bodies carry your rows as Arrow IPC; a request over the cap is rejected with HTTP 413 and nothing is written. A 413 is permanent. Retrying the same batch will fail the same way, so split it.

Target 100 MiB or less per batch

The SDK re-encodes rows before sending, so the bytes on the wire are not the bytes you measured client-side, and the difference can run to tens of percent. A batch sized right at the cap can pass or fail on that difference. Keep a margin: aim for at most 100 MiB of Arrow data per append, and treat the rest as headroom rather than budget. Rough row budgets, dominated by the vector column (4 bytes per dimension):

row shape	approx. bytes per row	rows per 100 MiB batch
`vector(384)` + ~200 B of text	~1.7 KiB	~60,000
`vector(768)` + ~200 B of text	~3.2 KiB	~32,000
`vector(1536)` + ~200 B of text	~6.2 KiB	~16,500

Concurrency: HTTP 503 under a large-ingest burst

Separately from the per-request cap, the service bounds the total in-flight request-body bytes across all concurrent requests. Each body is buffered in memory before it is decoded, so a burst of large concurrent appends is bounded to protect the shared process. When admitting a request would exceed that budget, the request is shed with HTTP 503 and a Retry-After: 1 header. It is not queued: the aim is to keep normal-load throughput untouched and reject only at the memory edge. Unlike a 413, a 503 here is transient. The pressure is in-flight bytes draining as other requests complete, which clears in well under a second, so honour Retry-After and retry the same batch unchanged. Because the budget is shared, a large burst from one workload can shed another’s requests. Practical consequence: raising concurrency past a handful of in-flight 100 MiB appends buys throughput you cannot use. Prefer fewer, larger batches over many large ones in parallel.

Batching and throughput

One append is one atomic commit, so batch size is the main ingest-throughput knob: prefer batches near the 100 MiB target over many small appends, and split large loads into a loop.

BATCH = 30_000  # ~50 MiB for 384-dim vectors with short text
for i in range(0, len(rows), BATCH):
    table.append(rows[i : i + BATCH])

const BATCH = 30_000; // ~50 MiB for 384-dim vectors with short text
for (let i = 0; i < rows.length; i += BATCH) {
  table.append(rows.slice(i, i + BATCH));
}

If you load with the CLI, bulk ingest is already streamed and committed in windows: size them with --batch-size-mb and keep the value under the cap.

Get started

Guides

Infino Cloud

Reference

Request body: 128 MiB

Target 100 MiB or less per batch

Concurrency: HTTP 503 under a large-ingest burst

Batching and throughput

See also

​Request body: 128 MiB

​Target 100 MiB or less per batch

​Concurrency: HTTP 503 under a large-ingest burst

​Batching and throughput

​See also

Request body: 128 MiB

Target 100 MiB or less per batch

Concurrency: HTTP 503 under a large-ingest burst

Batching and throughput

See also