# Property shards

[Wiki](../README.md) / [Nebula Cluster](README.md)

**On this page**

- [What is spread](#what-is-spread)
- [Creating a sharded database](#creating-a-sharded-database)
- [How properties move](#how-properties-move)
- [More shards](#more-shards)
- [Backups](#backups)
- [While a server is away](#while-a-server-is-away)

A database's property values are most of its bytes. Property shards spread them over
groups of servers, so that a database can be larger than one server's disk while its
structure and every index stay together.

Property shards are a **preview** in Nebula 3.0.0-alpha.1, turned on with
`GDB_CLUSTER_PROPERTY_SHARDS_PREVIEW=true` on every server.

## What is spread

| Kept together, on the database's **graph shard** | Spread over its **property shards** |
|---|---|
| Nodes and relationships, and how they connect | The property values of nodes and relationships |
| Labels and relationship types | |
| Every index and constraint | |
| Tables | |

Because the structure and the indexes stay in one place, nothing changes for an
application. A traversal, an index seek, a range, a full-text or vector search and a
uniqueness check run as they do in any database, and a commit is acknowledged by the graph
shard's servers as in any database of the cluster.

## Creating a sharded database

```cypher
CREATE DATABASE orders OPTIONS {replicas: 3, propertyShards: 4};
SHOW PLACEMENT FOR DATABASE orders;
```

Each property shard is a group of servers with its own leader, placed, repaired and moved
by the cluster like a database. `replicas` applies to the graph shard and to each property
shard.

## How properties move

- **A write lands on the graph shard's servers first.** Each of them writes its pages
  every few seconds. Once all have, the properties are handed to the property shards and
  the graph shard's servers let go of them.
- **A property held by a shard is read from it** and kept in a cache, 64 MiB for each
  database on each server. A statement that reads a property of many rows asks for them a
  few hundred rows at a time.
- **A statement answered from structure and indexes asks no shard at all.**
- **A change to a node whose properties are on a shard** brings them back to the graph
  shard with the change, and they are handed on again afterwards.

## More shards

```cypher
ALTER DATABASE orders SET OPTIONS {propertyShards: 6};
```

The new shards are placed like any. The database's properties are then evened out over all
of them, a part at a time, while it is read and written. `SHOW PLACEMENT` shows the new
shards at once.

## Backups

`gdb.backup.create` backs up the whole database into one directory: its graph shard and
each property shard.

```cypher
CALL gdb.backup.create('orders', {target: '/backups'});
CALL gdb.backup.verify('/backups/orders-20261120T101500Z-48211');
CALL gdb.backup.restore('/backups/orders-20261120T101500Z-48211', {database: 'orders_copy'});
```

| Part | For a sharded database |
|---|---|
| Kind | `full` |
| While it is taken | The database is read and written as usual. Handing properties on waits until the backup is done |
| Where | Give a `target` every server can reach |
| `verify` | Reads the graph shard's pages and every property shard's |
| `restore` | Makes a new database with as many property shards. It is built on the server that is asked and copied to the others from there |

## While a server is away

| While | What happens |
|---|---|
| A server of the graph shard is down | The database serves. What is written stays on the graph shard's other servers until the server returns or its copy is made again elsewhere |
| A property shard has no leader | Statements that need a property it holds wait up to 10 seconds, then fail with `Galactus.TransientError.Cluster.ShardUnavailable` and can be run again. Statements that need only structure, indexes or properties still on the graph shard go on |
| A copy of the graph shard is made on another server | It is given the graph shard's pages. Properties stay where they are: the new server reads them from the shards |

## Related articles

[Databases and copies](databases.md) · [Back up and restore](backups.md) · [Connect applications](connect.md)
