CPU usage goes more than 100% when trying to add multiple records in one go

[mohit_geeky]

Hi
I am trying to insert multiple records in neo4j

I have two types of nodes
both can have relation with itself and other

but when i try to insert multiple records in one type its taking too long and cpu consumption also goes upto 150%

can you please suggest me all ways that I can implement to make my queries faster

[glilienfield]

Can you post the query and your indexes? 

[mohit_geeky]

Hi Team
this is my query

`WITH $batch as batch\n`+
`UNWIND batch as row\n`+
`MATCH (asset:ASSET {uniqueID: row.asset}), (event:EVENT {uniqueID: row.event, eventId: row.eventId })\n`+
`CREATE (event)<-[:HAS_EVENT]-(asset)`

i am create around 50 60 relations like this 
but this is taking 1 to 2 minutes
and cpu consumption also goes 100 plus
i did indexing as well
like this
CREATE TEXT INDEX FOR (n:ASSET) ON (n.uniqueID)
i created index for all labels like this
on dashboard this is giving result in 1 millsecond 
but from code it is taking lot of time

can you please suggest some better way to optimize this
or suggest some better indexing
I'm using version 4.4
of both neo4j and driver

[glilienfield]

Your second match has two properties. do you really need both properties to uniquely identify the node? UniqueId is not enough?  If you need both, try adding a composite index on both properties to see if the search is better. 

Can you post the 'Explain' plan for the query? Maybe the index is not being used? 

[mohit_geeky]

Hi now i have 1M records and to insert, match and create relation between 100 nodes it taking 10 minutes at least
these are the indexes i created

and i creating data with these queries
query1

WITH $batch as batch\n`+

`UNWIND batch as row\n`+

`CREATE (n:ASSET {uniqueID:row.uniqueID, data:row.data, refUniqueID:row.refUniqueID, eventId:row.eventId})`

query 2

`WITH $batch as batch\n`+

`UNWIND batch as row\n`+

`CREATE (n:EVENT {uniqueID:row.uniqueID, data:row.data, refUniqueID:row.refUniqueID, eventId:row.eventId})`

query 3

`WITH $batch as batch\n`+

`UNWIND batch as row\n`+

`MATCH (asset:ASSET {uniqueID: row.asset}), (event:EVENT { eventId: row.eventId })\n`+

`CREATE (event)<-[:HAS_EVENT]-(asset)`

query4 

`WITH $batch as batch\n`+

`UNWIND batch as row\n`+

`MATCH (a:EVENT {uniqueID: row.event}), (b:ASSET {uniqueID: row.asset})\n`+

`WHERE NOT (a)<-[:HAS_EVENT]-(b)\n`+

`CREATE (a)<-[:HAS_EVENT]-(b)`



all these query taking 10 minutes to execute can you please provide an optimized way to do
and cpu also goes 150 to 200 %

[mohit_geeky]

query 5

`WITH $batch as batch\n`+

`UNWIND batch as row\n`+

`MATCH (a:ASSET), (b:ASSET {uniqueID: row.asset})\n`+

`WHERE a.refUniqueID CONTAINS '"row.asset"' AND NOT (a)<-[:TRANSFORMS_INTO]-(b)\n`+

`CREATE (a)<-[:TRANSFORMS_INTO]-(b)`

[glilienfield]

How large are the batches, i.e., how may rows?  Is it one million rows?  If so, you should use ‘call subquery in transactions’ or apoc.periodic.iterate to group the transactions in to batches of 50,000 or so. 

[mohit_geeky]

No batch size is only 50 to 100

and only query 5 is taking too much time
can you please help me that