# Storage engines

[Wiki](../README.md) / [Operations](README.md)

## Paged, resident and hybrid databases

Persistence and memory usage are separate choices.

| Engine | Kept in memory | Persistent storage | Typical use |
|---|---|---|---|
| Paged (Orion), from 2.0 | The pages in use, within one memory budget | Checksummed pages: topology, properties and every index | Every database; the default |
| Volatile | The whole graph and indexes | None | Disposable tests |
| Durable resident | The whole graph and indexes | Commit log and full checkpoints | Small, latency-sensitive catalogues |
| Durable hybrid | Topology, indexes and a bounded property cache | Commit log with properties read on demand | Databases made by 1.x that have not moved yet |

The database image writes to `/data/gdb` by default. Mount a volume at `/data`
to retain that data when the container is recreated. The `system` database
is resident; new user databases are Paged. `GDB_STORAGE_ENGINE=hybrid` (or
`resident`) creates them on a 1.x engine instead.

Every engine provides the same Cypher behaviour. A database remembers its
engine, and one made by 1.x keeps its engine until you move it, or until
`GDB_AUTO_UPGRADE` moves it as it opens (see
[upgrading to 2.0](upgrade-2.0.md)).

## The Paged engine (Orion)

A Paged database keeps the graph on disk in checksummed 8 KiB pages: node and
relationship records, adjacency, properties, label lookups and every index
(range, hash, composite, unique constraints, full-text, vector, point and
spatial). The pages a database uses stay in a page cache shared by every
database on the server, and its most used stores (topology, label lookups,
the upper levels of each index) are pinned there whole. From version 2.4.3 a
database's property stores are pinned as well when they fit in what the others
leave of `GDB_PIN_BYTES`, so a scan reads property values straight from
memory. A property store that does not fit is read through the cache.

- **Opening is immediate.** A database opens by reading its latest manifest;
  no index is rebuilt and nothing is replayed beyond the last few writes. A
  2-million-node database opens in about 0.2 seconds.
- **One memory budget.** `GDB_MEMORY_LIMIT` sets what the server may use
  (default: half the container's or machine's memory); the page cache takes
  half of it (`GDB_PAGE_CACHE_BYTES` sets it directly), and pinned pages are
  part of the cache, never on top of it.
- **Larger than memory.** When the graph is bigger than the cache, the pages
  in use are kept and the rest are read as queries need them.
- **Writes flush in the background.** Changes collect in memory and are
  written to new pages on a separate thread while readers and writers carry
  on. The commit log keeps only what the pages do not yet hold.
- **Crash safe.** A database is published one complete generation at a time,
  and every page is checked against its checksum when read.

`CALL gdb.storage()` shows a database's engine, its size on disk and how the
page cache is serving it.

### Changing a live database's engine

Run one administration statement at a time in Explorer or a driver's auto-commit
session:

```cypher
ALTER DATABASE analytics SET ENGINE PAGED;
ALTER DATABASE analytics SET ENGINE HYBRID;
ALTER DATABASE analytics SET ENGINE RESIDENT;
SHOW DATABASES;
```

A switch waits for existing work on that database and blocks new work there
until it finishes. Other databases can continue running. The statement cannot
run inside an explicit transaction.

Switching to resident mode loads all properties into memory, so allow enough RAM
for the complete property set. Conversion work grows with the number of properties.
Volatile databases always remain resident.

## Hybrid: RAM and disk behaviour

Hybrid storage keeps graph topology, adjacency, property locations and indexes
in memory. Only base property values move behind a bounded cache; it is not a
fully disk-paged database.

From version 1.8.0 the topology is bit-packed. Each node's relationship ids
take only the bits the relationship table needs, and each relationship row
only the bits its type and endpoints need. A relationship's structure takes
about 11 bytes, and a node with two relationships about 85.

Size memory for the graph structure, indexes, query results and GDS projections
as well as the property cache. Indexed values may remain in memory even when
their base property value is not cached.

Hybrid checkpoints seal the active log and incrementally reclaim stale property
values. Resident checkpoints write a full snapshot. See
[checkpointing](checkpoints.md) when planning maintenance on a large graph.

## Configuration

Set these values in your Compose service's environment:

```yaml
environment:
  GDB_DATA: /data/gdb
  GDB_DURABILITY: group
  GDB_PROPERTY_CACHE_BYTES: 33554432
  GDB_CHECKPOINT_THRESHOLD: 67108864
```

This selects group commit, a 32 MiB property cache per hybrid database and a
64 MiB active-log checkpoint trigger. The cache budget counts encoded record
bytes. Each cached value also takes about 110 to 160 bytes of heap beyond the
decoded value (from version 1.8.0; about 240 before). Values read or written
recently stay in the cache when it evicts.

`GDB_PROPERTY_SHAPES` (on by default) stores each distinct set of property keys
once and shares it between the elements that have it, which saves memory on
regular data. Set it to `false` for the flat layout. It affects memory only, so
you can change it at any restart. Data with more than 65,536 distinct key sets
falls back to a key list per element.

`GDB_PROPERTY_CACHE_BYTES=0` disables the property cache.
`GDB_CHECKPOINT_THRESHOLD=0` disables automatic checkpointing.
Configure the remaining service settings using the [Docker example](docker.md).

## Related articles

[Configuration](configuration.md) · [Durability](durability.md) · [Backups](backups.md)
