Strange delete and insert rels operation problem, need advices

[isayeter]

Hello everyone,

My graph db structure:

Only 1 Node type: "User" with Constraint( id=2, name='constraint_b4f8f106', type='UNIQUENESS', schema=(:User {userID}), ownedIndex=1 )

Only 1 Relation type: "follows".

Suppose that I have a User{userID:"1"} which have 600 follows relations with other users.

Now, suppose that my clients feeding my system with their following data's latest copy every day with network requests to my backend server. So that I'm refreshing the user's follows data everyday in my DB.

When a client feeding the main system, it passes the new following data partially by 200,200.

Scenario:

UserID:1 has already 600 follows in the DB,

UserID:1 started feeding the system,

#network_request1 body:

{ 
    is_first_request:true, 
    post_count:200, 
    total_row: 600, 
    left_row: 400, 
    last_follow_data:...
}

#network_request2 body:

{ 
    is_first_request:false, 
    post_count:200, 
    total_row: 600, 
    left_row: 200, 
    last_follow_data:...
}

#network_request3 body:

{ 
    is_first_request:false, 
    post_count:200, 
    total_row: 600, 
    left_row: 0, 
    last_follow_data:...
}

In my backend, whenever I receive the first request (with "is_first_request:true"), I execute 2 queries,

First one to delete all follows relations of the specified user. like:

MATCH (u:User {userID: "1"})-[r:follows]->(follows:User) delete r

and after that query complete, I am inserting new follows data for the user like:

UNWIND $batch AS row
MERGE (u1:User{userID: row.u1})
MERGE (u2:User{userID: row.u2})
MERGE (u1)-[r:follows]->(u2);

with the parameter array like:

array(
	"batch" => array(
		array("u1" => "1", "u2" => "2"),
		array("u1" => "1", "u2" => "3"),
		array("u1" => "1", "u2" => "4"),
		array("u1" => "1", "u2" => "5")
	)
);

And in other consecutive client requests (with "is_first_request:false"), I just execute inserting incoming new follows data, so that I am appending new incoming follows data by the clients.

So my questions:

1. I have realized now that, it is unnecessary to send 2 seperate queries in first feeding request for deleting and inserting the new data. What I'm going to do now will be like:

MERGE (u1:User {userID: "1"})
WITH u1

OPTIONAL MATCH (u1)-[r:follows]->(User)
DELETE r

WITH DISTINCT u1

UNWIND \$batch AS row
MERGE (u2:User{userID: row.u2})
MERGE (u1)-[r:follows]->(u2)

with the same type of parameters array above. Is there any other recommendation about this query? Am I in right path?

2. What is the best way of doing this operations. My database consists of 2.5 billion follows data right now and feeding requests by the clients are very very often. So, a lot of deleting and inserting queries happens simultaneously right now by separate connections. At the same time, there are another operations by the users which creates shortest paths between desired user nodes and returns the result to the client, these shortest path requests are very often too. So basicly, a lot of deleting inserting and reading operations happens simultaneously in my DB. What should I need to pay attention to?

Sometimes I'm receiving below errors but I don't now why they occurs and what should I do, maybe I should lock user nodes when I'm deleting and inserting new datas? I don't know..

Neo4j errors detected. First one with code "" and message "Relationship 75813490 not found (Neo.DatabaseError.Statement.ExecutionFailed)"

...AS row
MERGE (u1:User{userID: row.u1})
MERGE (u2:User{userID: row.u2})
MERGE (u1)-[r:follows]->(u2); - error: Neo4j errors detected. First one with code "" and message "ForsetiClient[0] can't acquire...Client[1] waits for [0]] (Neo.TransientError.Transaction.DeadlockDetected)"

Neo4j errors detected. First one with code "" and message "ForsetiClient[0] can't acquire ExclusiveLock{owner=ForsetiClient[1]} on NODE(2431018), because holders of that lock are waiting for ForsetiClient[0].
Wait list:ExclusiveLock[
Client[1] waits for [0]] (Neo.TransientError.Transaction.DeadlockDetected)"

[isayeter]

I have edited my post (solved one problem)