Transactions and concurrency

Wiki / Operations

Transactions and correctness

Each updating statement is atomic: it succeeds as a whole or rolls back. Use an explicit driver transaction when several statements must commit together.

import os
from galactus import Driver

with Driver("bolt://127.0.0.1:7687", "gdb", os.environ["GDB_PASSWORD"]) as driver:
    driver.begin()
    driver.execute_query("CREATE (:Account {id: $id})", {"id": 1})
    driver.execute_query("CREATE (:Account {id: $id})", {"id": 2})
    driver.commit()

Install the Python driver and set GDB_PASSWORD in the application environment to the configured server password. An omitted database uses the server's configured default. Call driver.rollback() to abandon an open transaction deliberately. The context manager closes the connection if the block exits early; a database or transport failure discards the connection. Do not reuse that connection or call rollback after it has been discarded.

The 0.1 drivers do not replay failed writes. If the connection is lost during COMMIT, the outcome can be unknown; check application state before retrying.

An explicit rollback or a dropped connection rolls back an open transaction. A transaction sees its own writes. Existence and node-key constraints are validated at the commit boundary. The durability setting determines when a successful commit is acknowledged; see durability.

Read and write concurrency

Read statements run concurrently, and from version 1.6.1 they do not wait for writers (see snapshot reads). A write statement or an open write transaction has exclusive access for writing, so other writers wait for it to finish, in the order they arrived. Keep write transactions short.

Large scans and projections can use additional read workers controlled by GDB_QUERY_THREADS.

Group durability lets another writer execute while an earlier commit waits for disk sync. With GDB_READ_AFTER_DURABLE=true, a reader that ran while a sync was in flight waits for that sync before replying, so no client receives data that is not yet on disk. With snapshot reads it does not wait at all: it reads the last commit that is already on disk. A reader can still receive its response before the writing client receives its acknowledgement.

Different databases have separate locks and storage streams, so work on one database need not block another. Only one server process may use a durable directory at a time; give each instance its own volume.

Snapshot reads

From version 1.6.1 a read never waits for another connection's write transaction. It reads the last committed state, and never a transaction's uncommitted writes. Nothing is copied: when a commit changes something a reader may still need, the old value is kept, and it is dropped as soon as nobody needs it. With group durability and GDB_READ_AFTER_DURABLE=true, readers see the last commit that is already on disk.

A read transaction (a driver's execute_read, or BEGIN in read mode) reads one snapshot from BEGIN to COMMIT: every statement sees the database as it was when the transaction began (repeatable read).

Old values are capped at GDB_SNAPSHOT_MAX_BYTES per database (64 MiB by default). If a long read transaction holds more than that while writes pour in, the oldest such transactions fail with the transient error Galactus.TransientError.Transaction.SnapshotTooOld; drivers retry transient errors in managed transactions. A read transaction also gets this error if the database was restored, or a replica applied changes, since it began.

While readers are busy, writers give them a turn between statements, so write throughput falls: by 19-36% in our tests with as many readers as writers, while reads went from a few a second to tens of thousands. With no readers, writes are as fast as before. Set GDB_SNAPSHOT_READS=false to make reads wait for open write transactions again.

Storage errors

An indeterminate append or sync failure, or detected storage corruption, makes the affected database refuse further work until it is reopened and recovery checks complete. Inspect the logs and storage before restarting; do not treat this as an ordinary query error or retry writes blindly.

Reading large results

From version 1.2.0, a read outside a transaction that ends in a plain RETURN of variables and their properties, and produces at least 4,096 rows (GDB_STREAM_MIN_ROWS), is not held in server memory as a finished result. The statement finds its rows when it runs; each page a driver fetches is then read from the database as it is asked for. A driver fetching 1,000 rows at a time makes the server hold about that many, not the whole result.

You still see the database as it was when the statement ran, and writes do not wait for you. The first write to that database reads the rest of any such open result into memory before it changes anything, so a slow reader costs a later writer that time and memory rather than blocking it.

Results with ORDER BY, DISTINCT, SKIP, LIMIT, aggregates or computed columns, and results inside an explicit transaction, are held whole as before. Set GDB_STREAM_MIN_ROWS=0 to hold every result whole. With snapshot reads on (the default from 1.6.1) every read result is held whole; this section applies with GDB_SNAPSHOT_READS=false.

Time limits

A statement, or a transaction measured from its BEGIN, can be given a time limit. The server default is GDB_QUERY_TIMEOUT_MS (unset: no limit); a driver may ask for a shorter one with the Bolt tx_timeout on RUN or BEGIN. The shorter of the two applies, and time spent waiting behind another connection's write transaction (by a write, or by a read with snapshot reads off) counts.

When the limit passes, the statement stops at its next row, fails with Galactus.ClientError.Transaction.TimedOut and its writes are rolled back. In an explicit transaction the whole transaction is rolled back, and COMMIT after the limit is refused. A write transaction left idle past its limit is rolled back and its connection closed, so it cannot hold the database's write gate indefinitely.

Listing and stopping running statements

CALL dbms.listQueries();                 // or: SHOW TRANSACTIONS
// queryId, username, database, query, elapsedTimeMillis, connectionId
CALL dbms.killQuery('query-42');         // or: TERMINATE TRANSACTIONS 'query-42'

The stopped statement fails with Galactus.ClientError.Transaction.Terminated and is rolled back. A user sees and may stop their own statements. Seeing or stopping another user's needs the admin procedure privilege, which the built-in admin role has. Run these from a second connection: a connection that is executing a statement reads nothing else until it finishes.

Configuration · Durability · Backups

Planning a deployment? Review compatibility and licence setup for your instance.