Batched subqueries and logical interchange

Wiki / Imports and exports

Batched subqueries and logical interchange

LOAD CSV WITH HEADERS FROM $file AS row
CALL {
  WITH row
  CALL apoc.merge.node(['Person'], {id: toInteger(row.id)}, {name: row.name})
} IN TRANSACTIONS OF 1000 ROWS;

CALL { ... } IN TRANSACTIONS [OF size ROWS] runs on the implicit/autocommit path, defaults to 1000 rows and accepts a parameterised size. Each batch commits independently. A failure rolls back its current batch; earlier batches remain committed. Explicit transactions, nested batching, multiple batched subqueries, UNION and writes outside the batch are refused. This implementation uses the CALL { WITH ... } import syntax, not the newer CALL (variables) { ... } form. ON ERROR chooses what a failed batch does: FAIL (the default) fails the statement, CONTINUE rolls the batch back and carries on with the next, and BREAK rolls it back and runs no further batch. With CONTINUE or BREAK, rows of a batch that did not commit keep their outer values and carry null for the subquery's columns. REPORT STATUS AS s adds a map per row: {started, committed, transactionId, errorMessage}.

LOAD CSV WITH HEADERS FROM 'file:///people.csv' AS row
CALL { WITH row CREATE (:Person {id: toInteger(row.id)}) }
IN TRANSACTIONS OF 1000 ROWS ON ERROR CONTINUE REPORT STATUS AS s
WITH s WHERE NOT s.committed
RETURN DISTINCT s.transactionId, s.errorMessage;

A statement time limit, a termination and a storage fault end the statement whatever ON ERROR says. IN n CONCURRENT TRANSACTIONS is accepted, but batches run one after another because write execution is exclusive.

A direct LOAD CSV ... CALL {...} IN TRANSACTIONS streams records with only a batch of bindings retained. UNWIND retains its input list but does not build all row bindings at once. More complex prefixes are materialised. A trailing RETURN/WITH may materialise output rows. The server's exclusive write gate is held for the statement; batching improves memory/durability boundaries, not concurrent write execution.

  • apoc.load.json(source[, path, config]) loads a JSON object, scalar or array (one result row per array item). Only empty path/config are accepted.
  • apoc.import.json(source[, config]) imports APOC node/relationship JSON or JSONL, remapping exported string IDs. It is one atomic statement, not a streaming batched import. Only empty configuration is supported.
  • apoc.export.json.all(file, {stream: true}) exports APOC-style JSONL and yields data; JSON temporal/point conversion follows the existing JSON encoder.
  • apoc.export.cypher.all(file, {stream: true}) yields cypherStatements, a logical data export preserving labels, relationships, scalar/list/map, temporal and point properties. Bytes and non-finite numbers are refused. Schema/index export and APOC export-format/batching options are not included.

Use a null filename and {stream:true} to return export text without creating a file. File exports create a new file and never overwrite one; filesystem effects are not rolled back with graph transactions. Imports/loaders accept the same local/file/plain-HTTP sources as LOAD CSV; HTTPS and JSONPath remain unsupported. Both export formats currently materialise their text in memory. The existing gdb.cypher.import(path) loads the generated Cypher script.

The deprecated apoc.periodic.iterate interface is intentionally not added; native batching provides the requested import capability.

Backup, restore and transfer · Transactions and concurrency · Server configuration

Planning a deployment? Review compatibility and licence setup for your instance.