A fast way to load 2000 objects

[paolodipietro58]

• neo4j 3.5.8, desktop 1.2.2-dev.5, browser 3.2.20

question 1:

I have more than 2000 statements in the form:

MERGE ( n :label1:label2 { name:'xxx' , ns:'yy' , uid:'407be989017e' }) RETURN n;
MERGE ( n :label1:label3 { name:'zzz' , ns:'yy' , uid:'407be989017e' }) RETURN n;

loading them in a single batch into an empty DB using the browser runs for more than 8 hours.

Is there another faster way to load it?

Question 2:

For the data above, I need to create indexes on the uid and name properties.

I was looking on a lighter solution trying the following:

with [label1(uid), label1(name), label2(uid), label2(name), label3(uid), label3(name), label4(uid), label4(name)] as items
unwind items as item
Create Index on item;

but it returns an errors.

I'm unable to build a WITH (:label(uid), :label2(uid) .....) as items
and I don't know how to CREATE INDEX ON item after UNWIND

Any help will be appreciated

[ameyasoft]

Please email me those 2000 records to ameyasoft@gmail.com. I will try to import and let you know.
Thanks

[michael_hunger · Accepted solution]

do this first:

create constraint on (n:label1) assert n.uid is unique;

remove the returns

MERGE ( n :label1 {uid:'407be989017e' }) ON CREATE SET n.name = 'xxx' , n.ns = 'yy', n:label2;

I doubt that the browser's multi-statement editor was meant for that.

I also recommend to rather use cypher-shell for that.

otherwise use

:param items => [{ name:'xxx' , ns:'yy' , uid:'407be989017e' }, { name:'zzz' , ns:'yy' , uid:'407be989017e' }]

and then use

UNWIND $items as item
MERGE ( n :label1 {uid:item.uid }) ON CREATE SET n += item;

If you install the apoc plugin you can also set labels dynamically by adding at the end.

CALL apoc.refactor.addLabels(n, item.labels) yield node return count(*);

[paolodipietro58]

Hi michael, your is THE Solution!

I just used a collection storing the label as attributes and then I used APOC:

UNWIND $items as item
MERGE ( n :Model {uid:item.uid }) ON CREATE SET n += item;

MATCH (n:Model)
UNWIND n.lbl as albl
WITH n, albl
CALL apoc.create.addLabels( id(n), [ albl ] ) YIELD node
REMOVE node.lbl
RETURN node;

and it runs fine.

Now I have a lot of relationships using the following to patterns: can you help with these too?

MATCH (from {uid:'43f59d2203e5'}),(to {uid:'401d182f035b'}) CREATE (from)-[r:_IS_A]->(to)

MATCH ( from {uid:'439f021501f4'})   MERGE (from)-[r:_HAS ]->(n:_relationship {   name : 'TitolarenonAutenticato' , cardinality : '1' , uid : '43bbe9d60388' , complex :'true' });

[michael_hunger]

basically the same, don't forget to sue a label :Model for the matching.

you UNWIND an array of maps/dictionaries with fromUuid, toUuuid and rel-uuid and properties (that you set on ON CREATE SET)

it is all based on my blog post here:

Medium – 11 Aug 17

5 Tips & Tricks for Fast Batched Updates of Graph Structures with Neo4j...

This post was originally published on my blog, but I think that a wider audience can benefit from it.

Reading time: 6 min read

[paolodipietro58]

Thanks Michael,
just a couple of days ago I commented your blog post because the first example doesn't works.
Maybe the new versions of Neo4j, cypher and APOC requires a re-tuning of the examples. Or maybe I'm wrong?

[paolodipietro58]

An error in 3.5.9!!! I tried with 3.5.8 and it run!

michael.hunger:

:param items => [{ name:'xxx' , ns:'yy' , uid:'407be989017e' }, { name:'zzz' , ns:'yy' , uid:'407be989017e' }]

Then you suggest me to use cypher-shell.
I tried but this is the result: really cannot understand ....

 
neo4j> :param items => [{ name:'xxx' , ns:'yy' , uid:'407be989017e' }, { name:'zzz' , ns:'yy' , uid:'407be989017e' }]
Invalid input '=': expected whitespace, comment, GRAPH, DISTINCT, '*' or an expression (line 1, column 8 (offset: 7))
"RETURN => [{ name:'xxx' , ns:'yy' , uid:'407be989017e' }, { name:'zzz' , ns:'yy' , uid:'407be989017e' }] as items"
        ^
neo4j> 

Can you explain?

[michael_hunger]

Oh sorry try to remove the =>
then

[paolodipietro58]

michael.hunger:

:param items => [{ name:'xxx' , ns:'yy' , uid:'407be989017e' }, { name:'zzz' , ns:'yy' , uid:'407be989017e' }]

In a 3.5.7 I tried with these 2 lines of code but recevied a strange error:

neo4j> :param connections  [{conn: '3f212e590331', from: '3f212e3c02b6', to: '3f212da40067', connType: '_HAS'},
Invalid input 'R': expected whitespace, comment, '{', node labels, MapLiteral, a parameter, a relationship pattern, '(', '.', '[', "=~", IN, STARTS, ENDS, CONTAINS, IS, '^', '*', '/', '%', '+', '-', '=', '~', "<>", "!=", '<', '>', "<=", ">=", AND, XOR, OR, ',' or ']' (line 1, column 96 (offset: 95))

neo4j>  {conn: '3f212e600042', from: '3f212da40067', to: '3f212d8003a5', connType: '_HAS'}];
Invalid input '{': expected <init> (line 2, column 1 (offset: 2))
"{conn: '3f212e600042', from: '3f212da40067', to: '3f212d8003a5', connType: '_HAS'}];"
   ^
neo4j>