NotInTransactionException in Neo4j 4.0.3 embedded

[siavash_sheikhi]

Breaking transactions in chunks leads to this Exception in Neo4j 4.0.3 embedded in java application:
Exception in thread "main" org.neo4j.graphdb.NotInTransactionException: The transaction has been closed.

Example, exception occurs at line #7:
1 try (Transaction tx = graphDatabase.beginTx()) {
2 node = tx.createNode(Label.label("mynome"));
3 // TODO
4 tx.commit();
5 }

6 try (Transaction tx = graphDatabase.beginTx()) {
7 node.setProperty("Id", "xxxx");
8 // TODO
9 tx.commit();
10 }

I'd appreciate any help.

[tony_chiboucas]

Hello Siavash,

Short Anwser: Don't do that

The node object you are retrieving only exists while the transaction exists. You have two options:

• Do node.setProperty in the same transaction.
• Retrieve the node, and then mutate the data, in the second transaction.

Set a breakpoint and inspect node during the first transaction to see why.

try (Transaction tx = graphDatabase.beginTx()) {
  node = tx.createNode(Label.label("mynome")); // BREAKPOINT
  // TODO
  tx.commit();
}

try (Transaction tx = graphDatabase.beginTx()) {
  node.setProperty("Id", "xxxx"); // broken reference
  // TODO
  tx.commit();
}

Long Answer: Don't do that

The Neo4j internal API for things within a transaction, such as Node, Label, Index, etc... are temporary in-memory references to the raw data. When the transaction commits, the data is mutated according to what is in memory for that transaction, and no further commits are allowed. When the transaction is closed, the in-memory references are no longer stable, and likely no longer reference the same data. Allowing these temporary representations of the data to persist, and "re-apply themselves," would make the entire dataset unstable and unreliable, which is why transactions are defined this way to begin with.

Without a single property on the node you created, there would be no way to retrieve it. More, outside of the transaction, or with an async call, there would be no way to to be certain the node from the first transaction is the one you want to set an ID for.

Instead: transaction, mutate, commit.

Label mynomeLabel = Label.label("mynome"); 
// Node node; // BAD! Don't do!

try (Transaction tx = graphDatabase.beginTx()) {
  Node node = tx.createNode(mynomeLabel);
  node.setProperty("Id", "xxxx");
  tx.commit();
}

try (Transaction tx = graphDatabase.beginTx()) {
  Node node = graphDatabase.findNode(mynomeLabel, "Id", "xxxx" );
  // TODO
  tx.commit();
}

[siavash_sheikhi]

Hello Tony, thank you for your comments. Does it mean that my Java application based on neo4j-community-3.5.5, can not rely on node IDs anymore since they are dynamically allocated in this version?
and once I break down very large write transactions into chunks (to reduce memory consumption), I need to read all the graph entities I need again and again? Doesn't this impose some overload and reduce the efficiency as reading from database is always expensive?

[tony_chiboucas · Accepted solution]

siavash.sheikhizadeh:

Does it mean that my Java application based on neo4j-community-3.5.5, can not rely on node IDs

Your example didn't reference any particular id field. There is an internal-id which is consistent, and you may add your own "id" properties, which would also be consistent.

siavash.sheikhizadeh:

I break down very large write transactions into chunks (to reduce memory consumption), I need to read all the graph entities I need again and again?

It really depends on what, exactly, you're trying to do. What you can't do, is make mutations on a reference to an instance of a Node across multiple transactions. What you can do, is retrieve the same node in later transactions, and then add more data.

Also keep in mind that there are many ways to solve your problem:

• put whole maps onto a node, rather than doing it one property at a time.
• Store large documents as JSON or CSV in the import directory, then use a Cypher command instead.

String command = "CALL apoc.load.json(..."
try (
  Transaction tx = db.beginTx();
  Result result = db.execute(command)
) {
  while ( result.hasNext()) {
    Map<String,Object> row = result.next();
    // do something with the row
  }
  tx.success();
} catch (Throwable e) {
  throw new CypherException("Cypher query failed: "+command, e);
}

siavash.sheikhizadeh:

Doesn't this impose some overload and reduce the efficiency as reading from database is always expensive?

Depending on what you're doing, probably, but as far as I know Neo has always worked like this.

Are you trying to put the complete works of Shakespeare on a single node?

[siavash_sheikhi]

Thank you very much for your replies.