Using neo4j-import tool resulted in "java.io.IOException: Exception in the page eviction thread"

[toni]

Hi,

I was using neo4j-import tool in order to create a big database, and after some hours working without any problem, until it stopped with the "java.io.IOException: Exception in the page eviction thread" error.

What should I look in order to see what happened? Below there is the execution logs and I've checked the line at which it stopped and it looks pretty normal

(1/4) Node import 2019-04-05 09:35:13.031+0000
Estimated number of nodes: 908.68 M
Estimated disk space usage: 209.37 GB
Estimated required memory usage: 12.17 GB
.......... .......... .......... .......... .......... 5% ∆4m 45s 884ms
.......... .......... .......... .......... .......... 10% ∆4m 46s 228ms
.......... .......... .......... .......... .......... 15% ∆4m 40s 372ms
.......... .......... .......... .......... .......... 20% ∆11m 44s 147ms
.......... .......... .......... .......... .......... 25% ∆17m 29s 999ms
.......... .......... .......... .......... .......... 30% ∆17m 56s 545ms
.......... .......... .......... .......... .......... 35% ∆17m 17s 782ms
.......... .......... .......... .......... .......... 40% ∆17m 19s 990ms
.......... .......... .......... .......... .......... 45% ∆17m 17s 178ms
.......... .......... .......... .......... .......... 50% ∆17m 21s 299ms
.......... .......... .......... .......... .......... 55% ∆17m 14s 424ms
.......... .......... .......... .......... .......... 60% ∆17m 29s 184ms
.......... .......... .......... .......... .......... 65% ∆17m 14s 392ms
.......... .......... .......... .......... .......... 70% ∆19s 492ms
.......... .......... .......... .......... .......... 75% ∆1ms
.......... .......... .......... .......... .......... 80% ∆1ms
.......... .......... .......... .......... .......... 85% ∆0ms
.......... .......... .......... .......... .......... 90% ∆1ms
.......... .......... .......... .......... .......... 95% ∆0ms
.......... .......... .......... .......... .......... 100% ∆0ms

IMPORT FAILED in 3h 5m 37s 826ms.
Data statistics is not available.
Peak memory usage: 0.00 B
Error in input data
Caused by:ERROR in input
data source: BufferedCharSeeker[source:/var/lib/neo4j/import/outputs/outputs-000000000227.csv.gz, position:2711786, line:12085]
in field: spending_tx_hash:string:9
for header: [:ID(Output-ID), tx_hash:string, output_index:int, required_signatures:string, type:string, value:long, is_spent:boolean, input_index:int, spending_tx_hash:string]
raw field value: d9be3d33b842ae49815356b7583ee87f7c47c6bef13d9d33c4d7b8bc60191f52
original error: java.io.IOException: Exception in the page eviction thread

WARNING Import failed. The store files in /var/lib/neo4j/data/databases/bitcoin.db are left as they are, although they are likely in an unusable state. Starting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually
unexpected error: ERROR in input
data source: BufferedCharSeeker[source:/var/lib/neo4j/import/outputs/outputs-000000000227.csv.gz, position:2711786, line:12085]
in field: spending_tx_hash:string:9
for header: [:ID(Output-ID), tx_hash:string, output_index:int, required_signatures:string, type:string, value:long, is_spent:boolean, input_index:int, spending_tx_hash:string]
raw field value: d9be3d33b842ae49815356b7583ee87f7c47c6bef13d9d33c4d7b8bc60191f52
original error: java.io.IOException: Exception in the page eviction thread

[toni]

For the record this happened because I run out of space in the machine. I didn't realized that before because the size estimations of the final DB weren't accurate compared to the final DB size.

[michael_hunger]

Could you please report this as an issue in

GitHub

neo4j/neo4j

Graphs for Everyone. Contribute to neo4j/neo4j development by creating an account on GitHub.

Thanks a lot.
I think the exception should be more informative.

The estimation is hard to make without doing an actual import (or at least dry run).

[toni]

Here you have it https://github.com/neo4j/neo4j/issues/12194