How to model entities with self-relationships?

[deepak]

Hi!
I am very new to Neo4J and graph databases. I am learning by writing simple CRUD operations using SDN-RX.
I want to create a social graph with a structure that reflects this data class:

class Person(
        @Id val name: String,
        val age: Int,
        @Relationship(type = "FRIENDS_WITH") var friendsWith: Set<Person>? = null,    //bidirectional
        @Relationship(type = "FOLLOWS") var follows: Set<Person>? = null   //unidirectional
)

Trying to populate the database with the following code doesn't work:

private fun populate() {
        val a = Person("A", 11)
        val b = Person("B", 22)
        val c = Person("C", 33)
        a.friendsWith = setOf(b, c)
        b.friendsWith = setOf(a)
        c.friendsWith = setOf(a)
        a.follows = setOf(b)
        c.follows = setOf(b)
        peopleRepository.save(p)
    }

Any help or documentation would be appreciated. I cannot find any examples that include relations with same entity type.

[gerrit_meier]

There is ongoing work on this topic at the moment: https://github.com/neo4j/sdn-rx/issues/254

It does not work right now because the default relationship direction is outgoing. During the mapping phase the framework follow those and end up in the endless (mapped) loop.

[deepak]

Thank you Gerrit. Is there a temporary workaround to my problem? I thought Neo4J is made for such scenarios and hence was using this example for introducing Neo4J to my team. I am fairly new to SDN-RX, would switching to Neo4J-OGM help?

[gerrit_meier]

Yes, we love relationships 😉
The follows relationship looks good because one person follows another one (direction).
The bidirectional / undirected friends_with relationship is something that is modelled "against" the directed relationship nature of Neo4j. All relationships needs to have a direction in the database.
On an application layer (and in this very basic example) it makes sense to have some concept of undirected relationships that lose direction information and just care about the existence.

Neo4j-OGM would work but the successor of Neo4j-OGM and the current Spring Data Neo4j will be SDN-RX.

[deepak]

@gerrit.meier I still can't find examples using SDN-RX that persist bi-directional data. Here's a slightly different example that triggers an infinite loop using SDN-RX 1.1.0:

    data class Employee(@Id @Property("eid") val id: Long,
                    val name: String,
                    val knows: MutableSet<Employee> = mutableSetOf())


    @Transactional
    fun addEmployees(): Flux<Employee> {
        val e2 = Employee(12, "AA")
        val e3 = Employee(13, "BB")
        e2.knows.add(e3)
        e3.knows.add(e2)
        return employeeRepository.saveAll(Flux.just(e2, e3))
    }

There's a lot of documentation for OGM around such relations (using depth parameters). SDN-RX doesn't seem to have any guidelines for beginners like me. Am I missing some essential piece of documentation? I didn't find any note on depth resolution in the manual.

[gerrit_meier]

I would love to differentiate between two topics that somehow get mixed right now (also from my side):
Undirected relationships and bidirectional ones.
What you are showing in your code ist a bidirectional one: An outgoing relationship to a node that refers back to the very same node it started from. This is clearly a bug and we have a fix in the pipeline for this https://github.com/neo4j/sdn-rx/pull/263
The other thing that we will introduce are undirected relationships: They store just one (directed) relationship in the database but the queries do not care about the direction but just dismiss this information to have them reachable from each side.

There is no depth limit in SDN/RX in general when it comes to persisting and querying nodes. There are some corner cases like self-references where we draw a line in the query depth to avoid endless loops. https://neo4j.github.io/sdn-rx/current/#relationshipquerylimit

[deepak]

Thank you for explaining the difference! It makes sense now.

[upesh_j]

@gerrit.meier I've a similar situation where I need to create a hierarchy of components to represent a larger system (code below). I'm using SDN 6.0.2

@NodeEntity
class Component {

        // Some attributes here

        @Relationship(type = "PART_OF", direction = Relationship.Direction.OUTGOING)
        private Component partOf;
}

Intention is to create a hierarchy where some components are part of other components. So now let's say I've two nodes like N1-[:PART_OF]->N2. When I run repository.getById(N2.getId()) it goes into infinite loop and never returns. I tried the same operation after removing the relationship and it returns but of course with wrong results.

[gerrit_meier]

Could you provide the debug log auf the Cypher statements (org.springframework.data.neo4j.cypher) and tell me the generated statement for this query?

[upesh_j]

Thank you for quick response, Gerrit! Here is the query that got generated. Please note that Component is replaced with Part here and of course there are many more relationships that are modeled.

RUN "MATCH (n:Part) WHERE n.sku = $__id__ WITH n, id(n) AS __internalNeo4jId__ RETURN n{.allowCustomerReviews, .brand, .creationDate, .description, .externalUrl, .featured, .height, .hsnCode, .images, .length, .make, .manufacturedBy, .manufacturerPartNo, .name, .preferredRating, .published, .shortDescription, .sku, .taxClass, .taxStatus, .visibleInCatalog, .weight, .width, .ylInventory, __nodeLabels__: labels(n), __internalNeo4jId__: id(n), __paths__: [p = (n)-[:BELONGS_TO_CATEGORY|BELONGS_TO_SUB_CATEGORY|SOLD_BY|GROUPED_WITH|IS_ON_SALE|FITS_INTO|PART_OF|PRICED_AT]->()-[:MANUFACTURED_BY*0..1]->()-[:EQUIPMENTS*0..1]->()-[:EQUIPMENTS|MANUFACTURED_BY*0..]-()-[:BELONGS_TO_CATEGORY|PARTS_IN_CATEGORY|BELONGS_TO_SUB_CATEGORY|SOLD_BY|SELLS_PARTS|PRIMARILY_SOLD_BY|SELLS_AT|SECONDARY_SOLD_BY|PARTS_ON_SALE|GROUPED_WITH|PRIMARY_CATEGORIES|IS_ON_SALE|PARTS_IN_SUB_CATEGORY|FITS_INTO|PART_OF|MANUFACTURED_BY|GROUPED_PARTS|SECONDARY_CATEGORIES|CATEGORY|PRICING_LIST|EQUIPMENTS|PRICED_AT|SUB_CATEGORIES*0..]->()-[:BELONGS_TO_CATEGORY|PARTS_IN_CATEGORY|BELONGS_TO_SUB_CATEGORY|SOLD_BY|SELLS_PARTS|PRIMARILY_SOLD_BY|SELLS_AT|SECONDARY_SOLD_BY|PARTS_ON_SALE|GROUPED_WITH|PRIMARY_CATEGORIES|IS_ON_SALE|PARTS_IN_SUB_CATEGORY|FITS_INTO|PART_OF|MANUFACTURED_BY|GROUPED_PARTS|SECONDARY_CATEGORIES|CATEGORY|PRICING_LIST|EQUIPMENTS|PRICED_AT|SUB_CATEGORIES*0..]-()-[:PARTS_IN_CATEGORY|BELONGS_TO_CATEGORY|PRIMARILY_SOLD_BY|BELONGS_TO_SUB_CATEGORY|SOLD_BY|SELLS_PARTS|SELLS_AT|SECONDARY_SOLD_BY|PARTS_ON_SALE|GROUPED_WITH|PRIMARY_CATEGORIES|IS_ON_SALE|FITS_INTO|PART_OF|PARTS_IN_SUB_CATEGORY|MANUFACTURED_BY|GROUPED_PARTS|SECONDARY_CATEGORIES|CATEGORY|PRICING_LIST|EQUIPMENTS|PRICED_AT|SUB_CATEGORIES*0..]->()-[:PARTS_ON_SALE*0..1]->()-[:BELONGS_TO_CATEGORY|PARTS_IN_CATEGORY|PRIMARILY_SOLD_BY|SELLS_PARTS|BELONGS_TO_SUB_CATEGORY|SOLD_BY|SELLS_AT|SECONDARY_SOLD_BY|PARTS_ON_SALE|GROUPED_WITH|PRIMARY_CATEGORIES|IS_ON_SALE|FITS_INTO|PARTS_IN_SUB_CATEGORY|PART_OF|MANUFACTURED_BY|GROUPED_PARTS|SECONDARY_CATEGORIES|CATEGORY|PRICING_LIST|EQUIPMENTS|PRICED_AT|SUB_CATEGORIES*0..]-()<-[:PARTS_IN_CATEGORY|BELONGS_TO_CATEGORY|PRIMARILY_SOLD_BY|SELLS_PARTS|BELONGS_TO_SUB_CATEGORY|SOLD_BY|SELLS_AT|SECONDARY_SOLD_BY|PARTS_ON_SALE|GROUPED_WITH|PRIMARY_CATEGORIES|IS_ON_SALE|FITS_INTO|PARTS_IN_SUB_CATEGORY|PART_OF|MANUFACTURED_BY|GROUPED_PARTS|SECONDARY_CATEGORIES|CATEGORY|PRICING_LIST|EQUIPMENTS|PRICED_AT|SUB_CATEGORIES*0..]-()-[:GROUPED_PARTS*0..1]->()-[:BELONGS_TO_CATEGORY|PARTS_IN_CATEGORY|PRIMARILY_SOLD_BY|SELLS_PARTS|BELONGS_TO_SUB_CATEGORY|SOLD_BY|SELLS_AT|SECONDARY_SOLD_BY|PARTS_ON_SALE|GROUPED_WITH|PRIMARY_CATEGORIES|IS_ON_SALE|FITS_INTO|PARTS_IN_SUB_CATEGORY|PART_OF|MANUFACTURED_BY|GROUPED_PARTS|SECONDARY_CATEGORIES|CATEGORY|PRICING_LIST|EQUIPMENTS|PRICED_AT|SUB_CATEGORIES*0..]-()-[:BELONGS_TO_CATEGORY|PARTS_IN_CATEGORY|PRIMARILY_SOLD_BY|SELLS_PARTS|BELONGS_TO_SUB_CATEGORY|SOLD_BY|SELLS_AT|SECONDARY_SOLD_BY|PARTS_ON_SALE|GROUPED_WITH|PRIMARY_CATEGORIES|IS_ON_SALE|FITS_INTO|PARTS_IN_SUB_CATEGORY|PART_OF|MANUFACTURED_BY|GROUPED_PARTS|SECONDARY_CATEGORIES|CATEGORY|PRICING_LIST|EQUIPMENTS|PRICED_AT|SUB_CATEGORIES*0..]-() | p]}" {__id__="SKU0000192209"} {}

[upesh_j]

Hi @gerrit.meier , So after struggling quite a bit with this, there are a few things I found:

1. It doesn't appears to be only a problem related to Component class self-referencing. OUTGOING from class and INCOMING to the other is an issue. I saw it in the documentation but there is no remedy given for this. The use case is simple: I want to find out components that fit into a machine and I also want to find machines that a component can fit into. How do I model this scenario with mentioning OUTGOING from component to machine and INCOMING in Machine.
2. I removed all incoming relationships in all classes considering the documentation. This opens up a question for users: How do enforce or remember that I cannot establish an INCOMING relationship? Every developer needs to remember the graph model. What I would think is that developers should be free to model their new nodes/relationships the way they want and support the use case.

Am I missing something? I was wondering if there is a query depth parameter that is supported so that I can say, go only up to 1 level deep while fetching data so that we don't get into an infinite loop.

Much thanks for reading through this!

[gerrit_meier]

At the moment we are working on a solution for the self-references / cycles.
You are right that in general you should get as much freedom in defining the domain as you like but please keep in mind that the generic query generator behind the curtain can only follow the relationships you are defining blindly.
I just link to an answer I just posted in another thread regarding this situation.
tl;dr; We are eagerly working on a solution for this problem.

Cyclic references can't be saved (with auditing) Spring Data Neo4j & Neo4j-OGM

We are very aware of the path problem in certain domain-model scenarios and try to mitigate the problematic query creation by substituting it with cascading (data-driven) matches. Please also understand that the query might seems "insane" but this is exactly what we can derive from the metadata provided by the annotation in the domain model. Loading the N3PMVOrder will follow all reachable relationships by your definition. When SDN detects that there will be a circle because of the relationship…