Commit durability and recovery

Wiki / Operations

On this page

One authoritative write per commit

Every committed updating unit in either durable engine is encoded as one checksummed frame. In hybrid mode the same bytes are both the redo record and the backing storage for property values:

  1. Cypher statements run inside an undo-backed unit.
  2. Repeated writes to the same property within that unit are coalesced to the final value.
  3. Commit appends one frame. Sync mode then performs one data sync inside the append. Group mode does not: the committer releases the database and waits on the log's sync coordinator until a sync covering its frame has run — the first waiter runs it for everyone queued, so concurrent commits share one fsync and a lone committer syncs at once. Buffered mode (the server default) acknowledges straight after the append and leaves the sync to the periodic flusher.
  4. Hybrid mode installs the returned record locations in its in-memory property key directory once the append has succeeded. Resident mode already has the values in RAM and needs no second storage write.

Each committed update uses one authoritative storage stream. A rollback appends no data.

ModeBehaviourTrade-off
group (recommended for multi-client servers)One data sync per batch of concurrent commits; each commit is acknowledged only after the sync that covers itSame durability guarantee as Sync; concurrent commits share a disk sync. A read that saw a commit still being synced waits for that sync before replying, after releasing its locks (GDB_READ_AFTER_DURABLE).
syncOne data sync inside every appendAn acknowledged commit survives power loss, subject to the platform/filesystem durability contract; every commit pays a full fsync.
buffered (server default)Write to the OS page cache; a periodic flusher syncs every GDB_SYNC_INTERVAL_MSHighest throughput; a power or kernel failure can lose the acknowledged units since the last flush (a process restart cannot — the page cache outlives it). Checksums still prevent accepting torn bytes as a commit.

All three modes write byte-identical logs; a directory carries no memory of the mode that wrote it and opens under any of them.

Choosing a mode

Use group for a server with several clients whose acknowledged writes must survive a power cut. Concurrent commits share each disk sync, and since 1.6.0 readers no longer hold up writers: a reader waits for a sync in flight only after releasing its locks. With 32 writers and 32 readers, writes went from about 600 a second in 1.5.5 to about 8,600, the same pace as with no readers at all (Lightning strikes thrice).

group cannot make a single client that writes one commit after another faster than one disk sync per commit, a few hundred commits a second on fast NVMe. Batch that client's writes (UNWIND, larger transactions) or use buffered.

buffered, the server default, stays the fastest choice for bulk loading and for data you can rebuild, accepting that a power or kernel failure can lose the writes acknowledged since the last background sync. sync gives the same guarantee as group without sharing; prefer group unless you need each commit to sync on its own.

Segment and record integrity

Each segment has a magic/version/id header. A transaction frame has its own independently checksummed length and payload checksum, and every mutation record inside it has another length and checksum. Authenticating the declared frame length before any tail decision prevents a length-bit flip from masquerading as an incomplete append. A property key-directory entry is therefore a stable (segment, offset, length) address, packed into 12 bytes. The directory keeps no copy of the checksum: a read verifies the payload against the checksum in the record's own header. A record too large for the packed form is kept in a small side table.

Recovery is intentionally conservative:

  • a physically incomplete final header or payload in the active wal.log is an uncommitted crash tail and is truncated to the last valid frame boundary;
  • in the active wal.log under Group or Buffered durability, where several not-yet-synced frames can be in flight at once, the first invalid frame (bad checksum, malformed record, out-of-order LSN, or a zero-filled region such as a power loss leaves on NTFS) starts the discarded tail: it and everything after it are truncated. Group-mode acknowledgement waits for a covering sync. Buffered-mode acknowledgement does not: recovery can discard acknowledged buffered changes in this tail. The decoder cannot distinguish every case of post-sync media corruption from a damaged tail, so this is a recovery policy, not a guarantee that arbitrary corruption preserves acknowledged data;
  • under Sync durability, a complete invalid frame with non-zero bytes following it is treated as corruption and opening fails closed. A zero-filled remainder is a torn write and truncates in every mode;
  • sealed-segment corruption is always fatal;
  • a property read validates the record checksum and that the decoded mutation names the expected node/relationship and property key. I/O or corruption is an error, never Cypher null.

One frame is the atomic replay boundary, so recovery never exposes half a transaction.

Snapshot publication, segment rotation, compaction and reset flush file contents and directory changes before removing the previous authoritative log. Power-loss guarantees also depend on the storage platform and filesystem.

Snapshot v3 authenticates its version, replay watermark, payload length and payload together. Valid v2 snapshots remain readable and are rewritten once to v3; their historical payload-only checksum could not retrospectively protect a watermark bit flip that happened before the upgrade.

On first open, a valid legacy GDBWAL v2 file is upgraded once: its state is folded into an atomic compatibility snapshot, the old LSN becomes the watermark, and hybrid properties are seeded into the segmented authoritative log. The rename crash windows are reopenable, recovery retains the only authoritative legacy copy, and the reserved backup is removed only after the new log is validated. The old format did not authenticate its length field independently; before accepting an apparently torn final payload, migration now checks whether the available bytes contain a checksum-valid complete payload and fails closed if so. New writes use the segmented format's direct length authentication.

Server configuration · Backup, restore and transfer

Planning a deployment? Review compatibility and licence setup for your instance.