cancel
Showing results for 
Search instead for 
Did you mean: 

Head's Up! These forums are read-only. All users and content have migrated. Please join us at community.neo4j.com.

Load CSV memory pool error: "The memory pool limit was exceeded"

martin2
Node Link

I try to load a large CSV to Neo4j 5.2 with cypher-shell and run into an error I have not seen before:

 

Unable to complete transaction.: The memory pool limit was exceeded. The corresponding setting can be found in the error message

 

The Neo4j logs are empty, the heap size is 5 times larger than the total file size. The transaction should even actually fit into memory but I use CALL ... IN TRANSACTIONS.

The query should not run into an Eager operator:

 

LOAD CSV WITH HEADERS FROM 'file:///omop/CONCEPT_RELATIONSHIP_clean.csv' AS line FIELDTERMINATOR ','

CALL {
  WITH line
  MATCH (source:Concept { concept_id: line.concept_id_1 })
  MATCH (target:Concept { concept_id: line.concept_id_2 })
  CREATE (source)-[r:VOCAB_REL]->(target)
  SET r.type = line.relationship_id, r.valid_start_date = line.valid_start_date, r.valid_end_date = line.valid_end_date, r.invalid_reason = line.invalid_reason

} IN TRANSACTIONS;

 

 Any ideas what might cause the error?

2 REPLIES 2

@martin2 

but yo have not specified a transaction batch size.

Change

} IN TRANSACTIONS;

to

} IN TRANSACTIONS OF 1000 ROWS

https://neo4j.com/docs/cypher-manual/current/clauses/call-subquery/#_batching

Thanks! There is a default for the transaction size: "If omitted, the default batch size is 1000 rows"