Property shards

Wiki / Nebula Cluster

On this page

A database's property values are most of its bytes. Property shards spread them over groups of servers, so that a database can be larger than one server's disk while its structure and every index stay together.

Property shards are a preview in Nebula 3.0.0-alpha.1, turned on with GDB_CLUSTER_PROPERTY_SHARDS_PREVIEW=true on every server.

What is spread

Kept together, on the database's graph shardSpread over its property shards
Nodes and relationships, and how they connectThe property values of nodes and relationships
Labels and relationship types
Every index and constraint
Tables

Because the structure and the indexes stay in one place, nothing changes for an application. A traversal, an index seek, a range, a full-text or vector search and a uniqueness check run as they do in any database, and a commit is acknowledged by the graph shard's servers as in any database of the cluster.

Creating a sharded database

CREATE DATABASE orders OPTIONS {replicas: 3, propertyShards: 4};
SHOW PLACEMENT FOR DATABASE orders;

Each property shard is a group of servers with its own leader, placed, repaired and moved by the cluster like a database. replicas applies to the graph shard and to each property shard.

How properties move

  • A write lands on the graph shard's servers first. Each of them writes its pages every few seconds. Once all have, the properties are handed to the property shards and the graph shard's servers let go of them.
  • A property held by a shard is read from it and kept in a cache, 64 MiB for each database on each server. A statement that reads a property of many rows asks for them a few hundred rows at a time.
  • A statement answered from structure and indexes asks no shard at all.
  • A change to a node whose properties are on a shard brings them back to the graph shard with the change, and they are handed on again afterwards.

More shards

ALTER DATABASE orders SET OPTIONS {propertyShards: 6};

The new shards are placed like any. The database's properties are then evened out over all of them, a part at a time, while it is read and written. SHOW PLACEMENT shows the new shards at once.

Backups

gdb.backup.create backs up the whole database into one directory: its graph shard and each property shard.

CALL gdb.backup.create('orders', {target: '/backups'});
CALL gdb.backup.verify('/backups/orders-20261120T101500Z-48211');
CALL gdb.backup.restore('/backups/orders-20261120T101500Z-48211', {database: 'orders_copy'});
PartFor a sharded database
Kindfull
While it is takenThe database is read and written as usual. Handing properties on waits until the backup is done
WhereGive a target every server can reach
verifyReads the graph shard's pages and every property shard's
restoreMakes a new database with as many property shards. It is built on the server that is asked and copied to the others from there

While a server is away

WhileWhat happens
A server of the graph shard is downThe database serves. What is written stays on the graph shard's other servers until the server returns or its copy is made again elsewhere
A property shard has no leaderStatements that need a property it holds wait up to 10 seconds, then fail with Galactus.TransientError.Cluster.ShardUnavailable and can be run again. Statements that need only structure, indexes or properties still on the graph shard go on
A copy of the graph shard is made on another serverIt is given the graph shard's pages. Properties stay where they are: the new server reads them from the shards

Databases and copies · Back up and restore · Connect applications

Planning a deployment? Review compatibility and licence setup for your instance.