Linking nodes recursively with arbitrarily deep paths

[marc_roth]

Hi community,

we need to link a bunch of nodes connected in one way with arbitrarily deep paths to a :Module node using :includes relations. Therefore we're using patterns like:

match path=(:System:Workflow:Application)-[:raises]->(:System:Workflow:Event)-[:loads]->(:System:Workflow:Element)-[:raises]->(:System:Workflow:Event) ...

where an :Application node is always the start node of the path.

We've already tried the variable-length pattern matching using:

match path=(:System:Workflow:Application)-[:raises|loads*]->(:System:Workflow)

and the spanningTree and expandConfig functions from apoc:

call apoc.path.spanningTree(
  application,
  {
    labelFilter: 'Workflow',
    relationshipFilter: "raises>|loads>",
    bfs: false, // also tried true
    filterStartNode: false,
    limit: -1,
    optional: true  // also tried false
  }
) yield path

to get all necessary nodes. But both approaches failed as some nodes weren't linked to the :Module or due to lack of performance while running the queries.

Next we tried to setup a trigger that fires as soon as a new :includes relation between a :Module and a :Workflow node is created.

call apoc.trigger.add(
  'modules',
  '
    unwind [relation in $createdRelationships where type(relation) = "includes"] as relation
    
    with
      startNode(relation) as version,
      endNode(relation) as workflow
    where (version:System:Version)
      and (workflow:System:Workflow)
    
    optional match (workflow)-[:raises|loads]->(node:System:Workflow)
    
    merge (version)-[:includes]->(node)
   ',
   {
     phase: 'before'
   }
)

From our opinion it should recognize the newly created :includes relation and fire again (and again and again and again ) to get all the other :Workflow nodes linked to the :Module. So ones could say creating the first :includes relation inside a :Workflow should start the trigger recursively.

But testing it with:

match (application:System:Workflow:Application{name: 'app-designer'})
create (version:System:Version{name:'test'})-[:includes]->(application)

only the very first :includes relation is created automatically.

What do we need to change to get all nodes linked properly. Or is there even a completely other approach?

P.S. By the way we're talking round about 8.000 :Workflow nodes that need to be linked to 14 :Modules.

[glilienfield]

sorry for my confusion. Are you looking for a method to create a random hierarchical tree of nodes with some random depth and breadth, starting from an Application node and terminating on Module node? Or, do you just want to generate one path of random length between these two node type?

[marc_roth]

The paths are already there and we want to link each node of those paths to one :Module node.
The main problem is, that we can't match all of the relevant nodes as described above. Hope this additional information helps.

[glilienfield]

I am sorry, maybe it's me, but I am still confused. You mention Module nodes, but I don't see that label used in any of your examples. Also, you state the paths are already there, but you can't match all of them. If they are linked along a path, can't you identify the path, and then iterate through the list of path nodes and link them to your Module node?

Sorry I am not helping.

[marc_roth]

To make it a bit clearer I've taken a screenshot from the first few levels of one of our applications.
As you can see the :Application node (top left) already has it's own :Event (purple-colored) and :Element(dark-grey colored) nodes.
The main problem is that when we iterate through all the paths of an application, no matter how deep they are, using the above mentioned query patterns, apoc functions or triggers, we don't get all of the relevant nodes linked to the :Module (dark-red node at the top) or the transactions run endlessly.
F.e. the :Event node right below the module is for some reason not linked to it...

Hope this one helps to understand our problem.

[michael_hunger · Accepted solution]

For some reason, the unlimited path expansion doesn't do a PRUNING var length expand. If you add some upper limit, it kicks in.

This seems to work fine for me:

profile match (a:System:Workflow:Application)

match path=(a)-[:raises|loads*..20]->(s:System:Workflow)

return count(distinct s)

same for apoc, I just added a limit, otherwise it probably just goes in cycles
and added uniqueness NODE_PATH

[marc_roth]

Thanks Michael,
we finally came up using a bunch of queries describing all possible paths.