I will try to explain my use case clearly. I am trying to generate graphs based on edge lists I have in my python code (they are just simple lists of tuples). For example, Edge Lists can look like: [(0,1), (1,2).., etc].
From the edge list, I have code in this format:
#Adding graph to Neo4j
def add_graph(tx, items):
tx.run(add_graph_query, items=items)
def add_graph_2(tx, items):
tx.run(add_graph_query_2, items=items)
add_graph_query = '''
UNWIND $items AS item
MERGE (f:Paper {id: item.id})
MERGE (e:Paper {id: item.edge_id})
MERGE (f)-[:CITED_BY]-(e)
'''
add_graph_query_2 = '''
UNWIND $items AS item
MERGE (f:Paper {id: item.id})
MERGE (e:Paper {id: item.edge_id})
MERGE (f)-[:CITED_BY]-(e)
'''
params = {"id": int(id), "edge_id": int(edge_id)}
items = []
items_2 = []
with driver.session() as session:
for id, edge_id in edge_lst:
items.append({"id": int(id), "edge_id": int(edge_id)})
if len(items) % 1000 == 0:
session.write_transaction(add_graph, items)
items=[]
session.write_transaction(add_graph, items)
for id, edge_id in edge_lst_2:
items_2.append({"id": int(id), "edge_id": int(edge_id)})
if len(items_2) % 1000 == 0:
session.write_transaction(add_graph_2, items_2)
items_2=[]
session.write_transaction(add_graph_2, items_2)
However, when I log the time, this snippet (from start to end) takes around 40-50 seconds if my edge list has 100 nodes (per edge list, I have 2 of them in total). I wanted to ask if there is any obvious way to speed up the process of inserting nodes into graphs.
I appreciate any help and I thank this community for being so helpful!
