Property shards
On this page
- What is spread
- Creating a sharded database
- How properties move
- More shards
- Backups
- While a server is away
A database's property values are most of its bytes. Property shards spread them over groups of servers, so that a database can be larger than one server's disk while its structure and every index stay together.
Property shards are a preview in Nebula 3.0.0-alpha.1, turned on with
GDB_CLUSTER_PROPERTY_SHARDS_PREVIEW=true on every server.
What is spread
| Kept together, on the database's graph shard | Spread over its property shards |
|---|---|
| Nodes and relationships, and how they connect | The property values of nodes and relationships |
| Labels and relationship types | |
| Every index and constraint | |
| Tables |
Because the structure and the indexes stay in one place, nothing changes for an application. A traversal, an index seek, a range, a full-text or vector search and a uniqueness check run as they do in any database, and a commit is acknowledged by the graph shard's servers as in any database of the cluster.
Creating a sharded database
CREATE DATABASE orders OPTIONS {replicas: 3, propertyShards: 4};
SHOW PLACEMENT FOR DATABASE orders;
Each property shard is a group of servers with its own leader, placed, repaired and moved
by the cluster like a database. replicas applies to the graph shard and to each property
shard.
How properties move
- A write lands on the graph shard's servers first. Each of them writes its pages every few seconds. Once all have, the properties are handed to the property shards and the graph shard's servers let go of them.
- A property held by a shard is read from it and kept in a cache, 64 MiB for each database on each server. A statement that reads a property of many rows asks for them a few hundred rows at a time.
- A statement answered from structure and indexes asks no shard at all.
- A change to a node whose properties are on a shard brings them back to the graph shard with the change, and they are handed on again afterwards.
More shards
ALTER DATABASE orders SET OPTIONS {propertyShards: 6};
The new shards are placed like any. The database's properties are then evened out over all
of them, a part at a time, while it is read and written. SHOW PLACEMENT shows the new
shards at once.
Backups
gdb.backup.create backs up the whole database into one directory: its graph shard and
each property shard.
CALL gdb.backup.create('orders', {target: '/backups'});
CALL gdb.backup.verify('/backups/orders-20261120T101500Z-48211');
CALL gdb.backup.restore('/backups/orders-20261120T101500Z-48211', {database: 'orders_copy'});
| Part | For a sharded database |
|---|---|
| Kind | full |
| While it is taken | The database is read and written as usual. Handing properties on waits until the backup is done |
| Where | Give a target every server can reach |
verify | Reads the graph shard's pages and every property shard's |
restore | Makes a new database with as many property shards. It is built on the server that is asked and copied to the others from there |
While a server is away
| While | What happens |
|---|---|
| A server of the graph shard is down | The database serves. What is written stays on the graph shard's other servers until the server returns or its copy is made again elsewhere |
| A property shard has no leader | Statements that need a property it holds wait up to 10 seconds, then fail with Galactus.TransientError.Cluster.ShardUnavailable and can be run again. Statements that need only structure, indexes or properties still on the graph shard go on |
| A copy of the graph shard is made on another server | It is given the graph shard's pages. Properties stay where they are: the new server reads them from the shards |
Related articles
Databases and copies · Back up and restore · Connect applications