Neo4J Crashes due to Memory out of exception

[rasheed]

Hello

We have built a Proof of Concept (PoC) application which uses Neo4J v3.0 community edition as Graph Database. The server is Windows 2012 R2 – Data Center edition with 16 GB RAM. When the application is in use, it consistently crashes due to out of memory exception. The only way to recover from this by restarting the neo4J services. This exception consistency happens when the Neo4J memory usage is reaching around 4 GB.

We have tried increasing the heap size to 8GB as per the documentation https://neo4j.com/docs/operations-manual/current/performance/memory-configuration/ but it does not help. The application still crashes when the heap size reaches 4 GB. What are we missing here? Because of this issue, our customers are losing hopes on Neo4J as the documented configuration does not seem to work. Any help to resolve this issue would be greatly appreciated.

Best, Rasheed

[andrew_bowman]

For one, 3.0 is a rather old version, and (assuming you're on 3.0.0) as it was the first major release for 3.0, it's bound to have more bugs than most other releases. We do not recommend using this version.

At the very least, you should use the latest patch version for the minor version you decide to use. 3.4.7 is the latest version currently, so it's 4 minor versions ahead of 3.0. There have been significant improvements, features, and bug fixes since 3.0. Please upgrade.

With the newer versions is a tool you can use to get a memory recommendation for what values to use for heap and pagecache. Please use this to set your value, though you may want to review the memory configuration page in the docs.

[rasheed]

Hello Andrew,

Thanks for your feedback. We have upgraded neo4J and the heap memory issue is resolved. Now we are running a test by simulating 1000 query requests. The memory is stable unlike earlier, however, the tomcat server crashes with the following error:

2018-10-03 11:21:22.468+0000 ERROR [o.n.b.t.p.HouseKeeper] Fatal error occurred when handling a client connection: [id: 0x7ceb59a2, L:/127.0.0.1:7687 - R:/127.0.0.1:53876] An existing connection was forcibly closed by the remote host

java.io.IOException: An existing connection was forcibly closed by the remote host

at sun.nio.ch.SocketDispatcher.read0(Native Method)

at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)

at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)

at sun.nio.ch.IOUtil.read(IOUtil.java:192)

at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375)

at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288)

at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1108)

at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:345)

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)

at java.lang.Thread.run(Thread.java:744)

Please let us know if you any inputs to resolve this issue. Appreciate your assistance.

[andrew_bowman]

There's a github thread that might be related here.

You may also want to double check that you are handling transactions and sessions properly in your code, using try-with-resources blocks to ensure proper cleanup of both.

[harvey_nguyen]

@rasheed Have you resolved this issue ?
@andrew.bowman I have read the github thread, but there is no solution in that, only work around. I have used using blocks in my code (C#) and this issue still happens.
Could you help me ?
Thanks

[andrew_bowman]

Once you're on a newer version, and have your memory configured correctly (Neo4j does like memory, so if you can spare more, we'd recommend it), if you're still having memory issues then you may want to profile your queries and get advice for streamlining them, as inefficient queries may use up much more memory than is needed.

[rasheed]

Thank you very much for your valuable inputs. We will try upgrading and try this out.