I am trying to count the number of relationships (c:Cell)-[:EXPRSSED_IN]->(g:Gene) by using the query:
CYPHER runtime=pipelined
MATCH (c:Cell)-[r:EXPRESSED_IN]->(g:Gene)
WHERE c.id >= 0 AND c.id < 6000
RETURN count(*) AS count;
The query executed successfully. However, it was slower than what I expected. It took around 6 seconds to count a list of around 45 million rows. I'm not sure if it's the "common case". Despite that, I want to speed it up furthermore by using parallel processing. Below is the query that I'm trying to execute:
MATCH (c:Cell)
WHERE c.id >= 0 AND c.id < 6000
WITH COLLECT(c) AS cells
CALL apoc.cypher.mapParallel2("
MATCH (g:Gene)-[:EXPRESSED_IN]->(_)
RETURN _.id AS cell_id, g.id AS gene_id",
{parallel:True, batchSize:1000, concurrency:20},
cells,
20) yield value
return COUNT(*) as count;
It gives me the error Failed to invoke procedure "apoc.cypher.mapParallel2": Caused by: java.lang.RuntimeException: Error polling, timeout of 10 seconds reached. My guess is that it has something to do with the sockets and ports as the server will probably open some sockets to communicate with its workers.
Could anyone provide me a solution? Thanks in advance.
UPDATE 1: I use neo4j enterprise version 4.1.3
UPDATE 2: My query is actually incorrect. I have fixed it and it executed successfully. However, now it's taking even more time than without doing any parallel processing
