Neo4j-admin import very slow withou --multiline-fields

[laoikeda]

When I'm importing data through neo4j-admin, it takes less than 20s when using --multine-fields flag.

But when I take that flag out, it get stucked, and after several minutes has not done anything. When I checked the status, it seems to be stucked on the sorting/indexing.

From my understading, it should be faster without the --multiline-fields flag right? Why is it slower?

There is no duplicate or missing nodes, nor multiline fields on the csvs.

When it finished importing with the flag:

IMPORT DONE in 17s 463ms.
Imported:
4284042 nodes
3378408 relationships
9819199 properties
Peak memory usage: 1.07 GB

[michael_hunger]

Perhaps you have a lot of duplicates in your nodes? The deduplication is not most efficient right now.

[laoikeda]

There isn't duplicates, all data has been processed so there is no nodes duplication, so I don't use the --ignore-duplicate-nodes.

The strange thing is the with the multiline-fields it finished very quickly, but if i take that flag out (--multiline-fields=false) it get stuck and don't finish, with the same csv's.

[michael_hunger]

If you can share the CSVs e.g. on an accessible URL I can share them with the team.

[laoikeda · Accepted solution]

I fixed the issue. The issue went away when I created the :ID column in one of the csvs. That csv has only nodes that do not have relationships, so I did not put any :ID since it would not be used, but it made the graph creation be way to slow.
Without the :ID it did not complete in more than a hour, with the :ID it finished in less than 20 seconds.