Connect applications

Wiki / Nebula Cluster

On this page

An application connects to a cluster as it does to one server: with the Bolt driver it already uses, at an address you choose. No routing driver is needed.

Addresses

Any of these works, and they can be mixed.

Address in the applicationWhat it gets
bolt:// to one serverEverything, through that server
neo4j:// to one serverThe driver asks the server for the cluster's servers and asks again every 30 seconds. It sends writes to the leader and reads to servers that hold the database, uses a server that is added, and moves off one that is lost or drained
bolt:// or neo4j:// to the routerOne address that never changes. The router passes each connection to a server that is up

Each server tells the cluster where applications reach it: its GDB_ADVERTISED_ADDRESS, or its Bolt bind address when that is not set.

The router

The router is a small program of ours, gdb-balancer, that gives the cluster one address. It passes each connection to the server with the fewest connections among those that are in use and ready. It keeps no state, so run two or more behind one DNS name.

SettingDefaultMeaning
GDB_BALANCER_SERVERSrequiredThe metrics endpoint (GDB_METRICS_BIND) of one or more servers, as host:port, separated by commas. It learns the other servers from whichever answers
GDB_BALANCER_BIND0.0.0.0:7687Where applications connect
GDB_BALANCER_CHECK_MS1000How often it asks for the cluster's servers and asks each whether it is ready
GDB_BALANCER_CONNECT_TIMEOUT_MS2000How long a server is given to accept a connection or answer a check
GDB_BALANCER_OPS_BINDoffWhere it answers /health, /ready and /metrics about itself
GDB_BALANCER_SERVERS=a.example:9187,b.example:9187
GDB_BALANCER_BIND=0.0.0.0:7687
  • Set GDB_METRICS_BIND on every server: that is where the router asks a server whether it is ready.
  • A server that is added and put to use starts receiving connections within a check. One that is draining keeps the connections it has and is given no more. One that stops answering is given none until it answers again.
  • A connection stays on the server it was given. When that server is lost the connection ends; the application connects again to the same address and is given another server.
  • Encrypted connections pass through untouched, so each server's certificate must be valid for the name applications use.

What a session sees

  • Any server takes any request. A write sent to a server that does not lead is carried to the leader, and its results come back, on the same connection. That holds for a server with no copy of the database at all, so a server just added serves every database from the start.
  • A read sees every write acknowledged before it, on whichever server it runs. The server first checks how far the cluster has committed and waits until it has caught up.
  • A write sent to another server costs about the same as one sent to the leader: the servers keep connections open between them and reuse them.
  • Users and roles are the cluster's. CREATE USER, GRANT and DROP USER, sent to any server, take effect on all of them. A user made on one server can log in on another straight away, and a user who is dropped is refused everywhere, in open sessions too.
  • A transaction that may write runs on the leader from BEGIN to its end. Open a transaction in read mode (execute_read in a driver) to have it served by the server you are connected to.
  • SHOW DATABASES lists every database of the cluster on every server.

Codes to act on

CodeMeaningWhat to do
Galactus.TransientError.Cluster.LeaderChangedNo leader could take the request, or it was lost before the transaction committed. Nothing was doneRun it again. A driver's managed transaction does
Galactus.TransientError.Cluster.NoQuorumThe server could not confirm what it may read, because it cannot reach more than half the serversRun it again
Galactus.TransientError.Cluster.ShardUnavailableA property shard the statement needs has no leader at the momentRun it again
Galactus.DatabaseError.Cluster.CommitOutcomeUnknownThe commit was not confirmed by more than half the servers in time, or the leader was lost during it. It may or may not have been keptCheck before repeating it
Galactus.DatabaseError.Cluster.MemberFailedThis server's copy of the database has stopped; the message says whyConnect to another server and tell us
Galactus.ClientError.Cluster.ChangeRefusedA change to the cluster was refused; the message says whyRead the message

Driver setup · Start a cluster · Monitoring

Planning a deployment? Review compatibility and licence setup for your instance.