# Nobody waits: Galactus DB 1.6.1

Yesterday, in [1.6.0](/blog/galactus-db-1-6-0-lightning-strikes-thrice), we
stopped readers from holding up writers. Today we returned the favour.

Galactus DB 1.6.1 stops writers holding up readers. Until now, while a write
transaction was open, every reader on that database queued until it
finished. With writers busy, readers barely got a turn. In 1.6.1 they don't
queue at all: with eight clients running write transactions and eight
reading, reads went from **9 a second to 30,635**, and the slowest reads
from about **10 seconds to under half a millisecond**.

![Reads per second while 8 clients write, 1.6.0 against 1.6.1](/blog/charts/release-161-reads.svg)

That is not a typo, and it is not a new cluster. It is the same database on
the same machine, with readers that no longer wait in line.

## Yesterday's paper

Think of a newsroom. While today's edition is being typeset, nobody hands you
half a page with the ink still wet. But nobody makes you stand at the
printer's door until it is finished, either. They hand you yesterday's paper:
complete, consistent, and yours straight away.

That is what a snapshot read does. When a writer changes something, Galactus
DB keeps the old value for as long as a reader might still need it. A reader
that arrives mid-write is shown the last finished version of the graph: it
never sees half-done changes, and it never waits. When nobody needs the old
copies any more, they are gone.

1.6.0 built this machinery into the engine. 1.6.1 puts it to work in the
server, and turns it on by default.

## What you get

- **Reads that never queue behind writes.** An autocommit read sees
  everything committed so far, and nothing from a transaction still in
  progress. With group durability it reads the last commit that is already
  on disk, so it never waits for a sync either.
- **Repeatable read transactions.** A read transaction (your driver's
  `execute_read`) now reads one snapshot from `BEGIN` to `COMMIT`. Ask the
  same question twice and you get the same answer, whatever commits in
  between.
- **Fair writers.** Writers now take turns in the order they arrived. Before,
  a writer that had just finished could jump straight back in ahead of the
  others, and one unlucky writer could wait seconds for its turn.
- **A memory cap.** Old values are kept only while someone needs them, and
  never more than 64 MiB per database by default. A read transaction that
  stays open through a flood of writes gets a polite, retryable
  `SnapshotTooOld` and runs again; the official drivers retry it for you in
  managed transactions.

It works in every durability mode, and the slowest reads improved most where
the wait was longest:

| Durability | Reads/s, 1.6.0 | Reads/s, 1.6.1 | Slowest 1% of reads, 1.6.0 | 1.6.1 |
|---|---:|---:|---:|---:|
| Buffered | 9 | **30,635** | 10.0 s | **0.4 ms** |
| Group | 26 | **29,115** | 5.0 s | **0.5 ms** |
| Sync | 1 | **18,888** | 10.0 s | **2.2 ms** |

## One footnote

Readers who no longer queue still need a turn at the graph. Writers now hand
them one between statements, so while readers are busy, writers go a little
slower: a third fewer write transactions in `buffered` and `group`, a fifth
fewer in `sync`. With nobody reading, writers go exactly as fast as before.

![Write transactions per second with and without readers](/blog/charts/release-161-writes.svg)

We think that is the right trade. A third off the writers' pace in exchange
for thousands of times more reads, and reads that answer in a fraction of a
millisecond instead of seconds. If your workload says otherwise, it is one
setting to turn off (see below).

## Settings

Snapshot reads are on by default. Two settings, both new:

- **`GDB_SNAPSHOT_READS`** (default `true`): reads and read transactions read
  the last committed state instead of waiting for writers. `false` brings back
  1.6.0's behaviour.
- **`GDB_SNAPSHOT_MAX_BYTES`** (default `67108864`, 64 MiB): memory old values
  may use per database for open read transactions. `0` removes the cap.

The [transactions guide](/docs/operations/transactions#snapshot-reads) has
the details.

### How we measured

A release build of the server ran natively on an Intel Core i9-13900K with
64 GB of RAM and a Samsung 990 PRO NVMe drive, on Windows 11. Clients used the
official Neo4j Python driver 5.28, one process per client, against a graph of
20,000 people with an index on `name` and 40,000 `KNOWS` relationships.
Writers ran `execute_write` transactions of ten property updates and two new
relationships, each statement answered before the next was sent. Readers
alternated an index lookup with a two-hop count. Each run lasted ten seconds;
figures are medians of three rounds that alternated between snapshot reads
off and on. "1.6.0" in the charts is 1.6.1 with `GDB_SNAPSHOT_READS=false`,
which reads exactly as 1.6.0 did.

With snapshot reads off, workloads that never take a snapshot are level with
1.6.0, within 2% in alternating runs.

## Upgrading

Pull and restart:

```text
docker pull ianknowles/galactus-db:1.6.1
```

The data, commit log and snapshot formats are unchanged, so 1.6.0 and 1.6.1
open each other's databases. Nothing to configure: snapshot reads are on as
soon as it starts.

## What's next

Readers no longer wait for writers, and writers no longer wait for readers.
Next, writers that work side by side rather than taking turns.

Two releases in two days. Nobody waits for anybody, including you.