Apoc.periodic.iterate can't find node just created

[demiandenker]

I'm trying to execute this query:

MERGE (user:User {userId: "123"})
WITH timestamp() AS ts, user 
CALL apoc.periodic.iterate(
	"WITH $user as user UNWIND $friendsToAdd AS userId return userId, user",
	"MERGE (friend:User {userId: userId}) MERGE (user)-[:IS_FRIEND_OF]-(friend)", {batchSize:100, iterateList:true, parallel:true, params: {user:user, friendsToAdd:$friendsToAdd}}
) YIELD committedOperations, errorMessages
RETURN committedOperations, errorMessages

With these params

:params {"friendsToAdd":["462580974000242713","462580834790996016"]}

I'm always getting this error:

{
  "Unable to load NODE with id 453108.": 2
}

It seems that the queries inside apoc.periodic.iterate doesn't have access to the node created at the beginning. The query works fine when I run it again with the exact same parameters.

Running Neo4j v3.5.19 on a Causal cluster.

Any ideas?

[stefan_armbrust]

The inner statement runs in a separate transaction than the MERGE (user:User {userId:'123'}).

Assuming an empty database that MERGE bascially creates user 123. This transaction is not yet committed when you hand it over to the inner statement hence the inner can not see the pending transaction state (remember, we're using "read committed" isolation level).

When you invoke the statement again user 123 is already there (as a result of the previous attempt). Therefore the merge simply does a match and hands results over to the inner statement.

You need to make sure that the MERGE is run in a separate and finished transaction before you enter the inner statement. Either by two different statements from client side or by wrapping the merge into a

with "123" as myUserId
call apoc.cypher.runMany("merge (user:User{userId:$myUserId}) return user", { myUserId:myUserId}) yield result
with result.user as user
CALL apoc.periodic.iterate(
	"WITH $user as user UNWIND $friendsToAdd AS userId return userId, user",
	"MERGE (friend:User {userId: userId}) MERGE (user)-[:IS_FRIEND_OF]-(friend)", {batchSize:100, iterateList:true, parallel:true, params: {user:user, friendsToAdd:$friendsToAdd}}
) YIELD committedOperations, errorMessages
RETURN committedOperations, errorMessages

(note: I didn't actually test my statement, but the idea should be clarified)

[demiandenker · Accepted solution]

Thanks for your answer!

It makes sense. I had already split the query to create the user separately, however both queries were part of a bigger transaction created by java @Transactional annotation. I removed it to test and started working correctly.

Your suggested approach solves the issue and allows creating the user, if needed, in the same query.
I had to do some adjustments, but the essence is still the same:

WITH "123" as myUserId
CALL apoc.periodic.iterate(
	"RETURN 1",
	"MERGE (user:User {userId: myUserId})", {batchSize:1, iterateList:true, parallel:true, params: {userId:userId, appId:$appId}}) yield committedOperations
MATCH (user:User {userId: myUserId})
WITH user
CALL apoc.periodic.iterate(
	"WITH $user as user UNWIND $friendsToAdd AS userId return userId, user",
	"MERGE (friend:User {userId: userId}) MERGE (user)-[:IS_FRIEND_OF]-(friend)", {batchSize:100, iterateList:true, parallel:true, params: {user:user, friendsToAdd:$friendsToAdd}}
) YIELD committedOperations, errorMessages
RETURN committedOperations, errorMessages

I had to use apoc.periodic.iterate to create the user because neither apoc.cypher.run nor apoc.cypher.runMany can't be used for write statements.

apoc.cypher.doIt does actually allow write statements, but doesn't run them in a separate transaction afaik.

Thanks again for the help.