Upgrading from 1.x to 2.0

Wiki / Operations

Galactus DB 2.0 adds the Paged storage engine (Orion): topology, properties and every index live in checksummed pages on disk, read through one shared page cache, so a database no longer has to fit in RAM and opens in a fraction of a second whatever its size. The 1.x engines (Resident and Hybrid) are still in 2.0, so upgrading the server changes nothing on its own: each database moves to Paged when you choose.

On this page

• Before you start
• Path 1: upgrade in place
• Path 2: dump from 1.x, restore into 2.0
• Checking where a database stands
• Going back to 1.x
• What changes for applications
• Measured on a 2M-node database

Before you start

Take a dump of every database while it is still on 1.x, and keep it until you have finished:

CALL gdb.dump('/data/backup/neo4j-before-2.0.gdb');

A dump is a logical image that both 1.x and 2.0 restore, so it is the safe way back from any step below. Over Bolt, gdb.dumpToBackup() with gdb.dumpChunk moves it without holding it in memory (see backups).

Path 1: upgrade in place

1. Upgrade the server image to 2.0 and start it on the same data directory. Every database opens on the engine it had (Hybrid or Resident), with no conversion and no change on disk.

2. Convert each database when it suits you, as an auto-commit statement:

   ALTER DATABASE galactus SET ENGINE PAGED;
   

   The conversion streams the database into a new paged store beside the old one, in bounded memory, then swaps it in and rebuilds the indexes as paged trees. Other databases keep running; the one converting waits for it. A crash at any point leaves either the old store or the new one, never neither.

3. New databases are Paged. Set GDB_STORAGE_ENGINE=hybrid (or resident) to create them on a 1.x engine instead. A directory from a 1.x release so old that it has no storage.meta opens on Hybrid, as it was written, and is never converted unasked.

Or let the server upgrade on start

Set GDB_AUTO_UPGRADE=true and every user database still on a 1.x engine moves to Paged as the server opens it. Each is first dumped to <database>/backup/pre-upgrade-<engine>-<time>.gdb (turn that off with GDB_AUTO_UPGRADE_BACKUP=false), so allow free disk for a dump and a second copy of each database while it converts. A database that cannot move yet (a replica, or one with a change stream on) stays on its engine, keeps serving, and the reason is logged. Databases already on Paged are left alone, so the setting can stay on.

environment:
  GDB_AUTO_UPGRADE: "true"

Path 2: dump from 1.x, restore into 2.0

Use this to move to a new server or volume, or to start from a clean directory.

1. On 1.x: CALL gdb.dump('/data/backup/neo4j.gdb');, or download the dump from Explorer.

2. On 2.0, create the database (Paged unless GDB_STORAGE_ENGINE names another engine) and restore into it, as an auto-commit statement:

   CALL gdb.restore('/data/backup/neo4j.gdb');
   

   Explorer's upload sends the dump as bytes, which works the same way.

A restore into a Paged database reads the dump once, as a stream, and builds the new store beside the live one. Nothing is replaced until the whole dump has been read and its checksum has matched, so a damaged or truncated dump leaves the database exactly as it was.

Checking where a database stands

SHOW DATABASES;
CALL gdb.storage();

SHOW DATABASES ends with engine, storageFormat and writtenBy.

gdb.storage() also reports the page and log bytes on disk and the page cache's budget, use and hit ratio.

Going back to 1.x

A dump taken on 2.0 restores on 1.8 as long as it is under 4 GiB: 2.0 writes larger dumps in a format only 2.0 reads. A 1.x server pointed at a paged directory refuses to open it rather than misreading it.

What changes for applications

Nothing: the same Bolt protocol, the same Cypher, and the same node and relationship ids (id() and element ids are kept by every path above). Memory is governed by one budget, GDB_PAGE_CACHE_BYTES, or half of GDB_MEMORY_LIMIT, or half of the container's or machine's memory; see storage.

Measured on a 2M-node database

2,000,000 nodes and 4,000,000 relationships with a hash and a range index (Galactus DB 2.0.0, Intel Core i9-13900K, 64 GB RAM):

Related articles

Backup, restore and transfer · Resident and hybrid storage · Database administration