I want to merge two Neo4j Databases. For that, I am trying to use admin import tool.
As the performance is of highest concern for me, spinning up of a new db is perfect.
Hence, I am trying to get the database details as apoc.export.csv from both the databases and then use admin import function to import the csv files.
bin/neo4j-admin import --database=neo4j \
--nodes=/data/$commonfileNamePrefix"_src_nodeheader.csv",/data/$commonfileNamePrefix"_src_nodes.csv" \
--nodes=/data/$commonfileNamePrefix"_dest_nodeheader.csv",/data/$commonfileNamePrefix"_dest_nodes.csv" \
--relationships=/data/$commonfileNamePrefix"_src_relationheader.csv",/data/$commonfileNamePrefix"_src_relationships.csv" \
--relationships=/data/$commonfileNamePrefix"_dest_relationheader.csv",/data/$commonfileNamePrefix"_dest_relationships.csv" \
--ignore-empty-strings=true --skip-duplicate-nodes=true
However, this solution works fine, when either of the db is empty or both are empty. Its failing when both have data.
Please provide any suggestions or inputs. Any flag which can be used with import command also would help.
Error:
aused by:ERROR in input
data source: BufferedCharSeeker[source:/data/2022.05.02-12.16.37_dest_relationships.csv, position:328792, line:8761]
in field: _end:END_ID:10
for header: [:IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, _start:START_ID, _end:END_ID, _type:TYPE, bit_length:string, start_bit_offset:string]
raw field value: 8799
original error: Requested index -1, but length is 1000000
There were bad entries which were skipped and logged into /var/lib/neo4j/import.report
WARNING Import failed. The store files in /data/databases/neo4j are left as they are, although they are likely in an unusable state. Starting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually
org.neo4j.internal.batchimport.input.InputException: ERROR in input
data source: BufferedCharSeeker[source:/data/2022.05.02-12.16.37_dest_relationships.csv, position:328792, line:8761]
in field: _end:END_ID:10
for header: [:IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, :IGNORE, _start:START_ID, _end:END_ID, _type:TYPE, bit_length:string, start_bit_offset:string]
raw field value: 8799
original error: Requested index -1, but length is 1000000
at org.neo4j.internal.batchimport.input.csv.CsvInputParser.next(CsvInputParser.java:234)
at org.neo4j.internal.batchimport.input.csv.LazyCsvInputChunk.next(LazyCsvInputChunk.java:98)
at org.neo4j.internal.batchimport.input.csv.CsvInputChunkProxy.next(CsvInputChunkProxy.java:75)
at org.neo4j.internal.batchimport.ExhaustingEntityImporterRunnable.run(ExhaustingEntityImporterRunnable.java:57)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
at org.neo4j.internal.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:110)
Suppressed: org.neo4j.internal.batchimport.input.InputException: ERROR in input
data source: BufferedCharSeeker[source:/data/2022.05.02-12.16.37_src_relationships.csv, position:337554, line:8761]
