# The frontier database

A database spends most of its life waiting. The morning rush arrives, the
month-end report runs, and in between it sits there holding memory in case
somebody needs it. For years we rented that waiting by the gigabyte: memory
reserved at start-up, on servers sized for a peak that came on the first of
the month.

So we built Galactus DB, and then spent every release taking work out of it.
Somewhere along the way "graph database" stopped covering what it does. We
call it **the frontier database**, and this post is what we mean by that.

## What frontier means

The frontier is the point where a query takes as long as the work takes:
follow this pointer, read that page, send the answer. Past it, the limits are
the machine's and no longer the software's.

We can say where that point is, because we measure it on every release. On
Galactus DB 2.4.3:

- **A graph kept on disk scans 100,000 people in 8.3 ms**, exactly as long as
  the engine that holds the whole graph in memory.
- **Finding a node by an indexed property takes 4.0 microseconds** inside the
  engine, and 0.48 ms by the time the answer has crossed the network to your
  application.
- **Following a relationship costs 66 nanoseconds.**
- **The server idles at 11 MB**, answers its first query 0.8 seconds after
  `docker run`, and ships as a 13 MB image.

The full story of how the disk caught up with memory is in
[Nearing the theoretical limit](/blog/galactus-db-2-4-3-nearing-the-theoretical-limit).

## Graphs, tables and vectors

Galactus DB began as a graph database and still is one: Cypher over Bolt,
with the drivers you already use. But a great deal of what hangs off a graph
is rows. A customer has orders. An account has entries. A sensor has
readings. The usual answer is a second database for the rows and a nightly
job to keep the two in step.

In Galactus DB the rows live on the node they belong to:

```cypher
MATCH (c:Customer {id: 1})
SET c.orders = TABLE (
  id        INTEGER PRIMARY KEY,
  placed_at LOCAL DATETIME NOT NULL,
  sku       STRING NOT NULL,
  amount    DECIMAL(12, 2),
  INDEX by_sku (sku, placed_at)
);

// What did this customer's friends order most recently?
MATCH (:Customer {id: 1})-[:KNOWS]->(f:Customer)
CALL (f) SQL {
  SELECT sku, placed_at, amount FROM f.orders
  ORDER BY placed_at DESC LIMIT 3
}
RETURN f.name, sku, placed_at, amount;
```

That is SQL with joins, window functions, recursive CTEs and `MERGE`, in the
same transaction as the graph around it. It gives the answers SQLite gives,
and we know because we run thousands of generated queries through both and
compare them.

It is light on its feet as well. An order of five columns takes 63 bytes on
disk as a table row, against 85 as a node of its own. On one node's
200,000-row table, `count(*)` takes 0.02 ms and a one-row `UPDATE` by key
0.05 ms.

Add the vector, full-text and spatial indexes, and one query can find a
starting point by meaning, follow its relationships and read the rows at the
far end. Graphs, SQL tables and vector search share one server, change in one
transaction and go into one backup.

## What legacy costs

We use "legacy" for a pattern, and you will know it from the invoice:

- **Memory reserved before a row is stored.** The heap is sized for the
  busiest day and held all day. You pay for it at three in the morning too.
- **A server for each kind of data.** A graph store, a relational store and a
  vector index come to three bills, three backups and three things to patch.
- **Plugins to install and keep in step.** Algorithms in one, utilities in
  another, each with its own version to match.

Nobody chose that. It accumulated, one sensible decision at a time, and now
it is simply what a database costs. Galactus DB takes memory as the data
needs it, inside one budget you set, and everything on that list is already
in the 13 MB image.

## Meeting a new database

The safe choice has always been the database everyone else already runs. So
here is what we do to make a new one safe to choose:

- **We kill it mid-write.** Before each release the server is stopped at
  random points during writes, and every acknowledged transaction has to be
  there when it comes back.
- **We check the SQL against SQLite**, on thousands of generated queries.
- **The benchmark is public.** The
  [suite behind our figures](https://github.com/galactusdb/galactus-db-benchmark)
  runs on your hardware as readily as on ours.
- **We run on it.** Our licensing, billing and Galactus Cloud control plane
  keep their records in Galactus DB.
- **Leaving is easy.** It speaks standard Cypher, Bolt and SQL, licence keys
  are checked locally, and your data stays readable if a licence lapses.

Trying it takes less time than this post did:

```text
docker run -d -p 7687:7687 -e GDB_INITIAL_PASSWORD=<12 or more characters> ianknowles/galactus-db:2.4.3
```

Community is free for any use, production included, with no account and no
licence key. If it takes you longer than a cup of tea to have it answering
queries, tell us. We will want to know what kept you.

## How we measured

- **Over Bolt** (8.3 ms, 0.48 ms): the open
  [graph database benchmark](https://github.com/galactusdb/galactus-db-benchmark)
  against the published Galactus DB 2.4.3 image on its shipped settings, in a
  container limited to 8 CPUs and 4 GB. Each figure is the median of three
  passes over a graph of 100,000 people who each know two others, from a
  Python client.
- **Inside the engine** (4.0 microseconds, 66 nanoseconds): a release build
  of 2.4.3 run natively on the Paged engine with one query worker, on a graph
  of 100,000 nodes and 1,200,000 relationships. Each figure is the fastest of
  five rounds.
- **Start-up, idle memory and image size:** Galactus DB 2.4.3 for AMD64, with
  the first query timed from `docker run` and the image already pulled.
- **Tables:** row sizes are for 100,000 orders across 1,000 customers on
  Galactus DB 2.1.0 with the Paged engine; the 200,000-row figures are from
  Galactus DB 2.3.0.
- **Host:** Intel Core i9-13900K with 64 GB of RAM and a Samsung 990 PRO NVMe
  drive, running Windows 11 Pro and Docker Desktop 28.0.1 on WSL 2.
