Load large xml (3G) with apoc.load.xml

[redha_benhissen]

i'm tryin to load large xml (3G) with apoc.load.xml but I have this error

Neo.ClientError.Procedure.ProcedureCallFailed
Failed to invoke procedure apoc.load.xml: Caused by: java.lang.OutOfMemoryError: Java heap space

i made my memory config like this

dbms.memory.heap.initial_size=4G
dbms.memory.heap.max_size=4G
dbms.memory.pagecache.size=4G

But the error is still here (by the way, i have 8G memory in my lapotop)

Anybody have an idea to resolve it

[intouch_vivek]

Hi @redha.benhissen

Welcome to the community!!

Merely setting memory configuration will not help until unless you have sufficient memory in your laptop. There are several other processes running at the machine apart form Neo4j.

Suggested memory for Neo4j is 16 GB

Although try
USING PERIODIC COMMIT 500. It might help you

[redha_benhissen]

Hi @intouch.vivek

Thanks for your answer.

I tried to use PERIODIC COMMIT but it's no working with load xml. I think it working only with load csv.

adding to the first message. I have the following messgage on debug log:

2020-03-17 14:43:21.096+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=2836, gcTime=2927, gcCount=2}
2020-03-17 14:47:23.015+0000 WARN [io.netty.channel.nio.NioEventLoop] Unexpected exception in the selector loop. Java heap space
java.lang.OutOfMemoryError: Java heap space

[intouch_vivek]

Hi @redha.benhissen,

How much memory your have and what is your dataset size.
I know that you might have done good research to overcome from this issue. but you can try below two links
https://neo4j.com/docs/labs/apoc/current/import/xml/

Neo4j Graph Database Platform – 3 Aug 16

Effective Bulk Data Import into Neo4j

Learn how to quickly and easily conduct bulk data imports and ETL for Neo4j from a variety of data sources and formats, including XML, JSON and CSV.

[gordon_degrandi]

Hello, I am new to this forum. I was reading this topic and I have a similar issue when trying to import a large XML file using APOC.load.xml. The file is a 9128KB XML file. I prepared my cypher script using the APOC documentation. When we tried to import the XML file it threw an error Error org.neo4j.driver.v1.exceptions.ClientException: Failed to invoke procedure apoc.load.xml: Caused by: java.lang.OutOfMemoryError:

Java heap space is 25G, how much does APOC.load.XML need?

Was this issue solved? How was it solved?