cancel
Showing results for 
Search instead for 
Did you mean: 

Head's Up! These forums are read-only. All users and content have migrated. Please join us at community.neo4j.com.

apoc.periodic.iterate fail in large data (640 million nodes)

Peter_Lian
Node Clone
The query is the following (there're 640 million process nodes)
 
 
CALL apoc.periodic.iterate("MATCH(e:Process) MATCH (f:Process{parentID:e.pid})    RETURN  e,f",
"CREATE(e)-[r:create_process_to]->(f)",
{batchSize:5000, parallel: true}) YIELD batch

 

It return the error message

 

The allocation of an extra 2.0 MiB would use more than the limit 21.0 GiB. Currently using 21.0 GiB. dbms.memory.transaction.total.max threshold reached

 

After setting the related-configuration, although the error message do not show up but it take lots of time and no result should yet, even try difference batch size (100,500,1000).

 

This may due to there're 640 million node, since in the case of 300 million node it takes 1 hour to show the result and no error message.

 

Problem : How can I get over in this situation except create index for help? Thanks.

0 REPLIES 0
Nodes 2022
Nodes
NODES 2022, Neo4j Online Education Summit

All the sessions of the conference are now available online