Importing from a legacy UML business model

[paolodipietro58]

• neo4j 3.5.3, desktop 1.2.1-dev.3, browser 3.2.20

Well, before being an import/export question, this is a question of data representation (modeling) to support the correct data import result.

I have a UML model representing government services and object.

It is represented with UML Packages, containing classes with attributes and services and a lot of relationships with other classes.

The model is mainly hierarchical, with a lot of objects reused, so a graph would be a best representation.

Before writing code to transform UML to a correct set of data to feed Neo4j, I tried to depict the following scenario, but cannot find the correct syntax:

---

    WITH  [ "GovernmentSystems"
       	["Demography"
       		[ "Registry Services"
      			[ { UID : "1", name : "Change of dwelling" }
              [ "Request" 
                [{ UID : "1A", name : "Request" }
                  [ "Owner" 
                    [{ Type: "Rel", Object : "Person", RelName : "String", Optional : False, Multiplicity : 1, OtherStuffs : "Stuffs" }]
                  ]
                  [ "HasCurrentAddress" 
                    [{ Type: "Rel", Object : "Address"}]
                  ]
                  [ "HasFutureAddress" 
                    [{ Type: "Rel", Object : "Address"}]
                  ]
                  [ "Persons" 
                    [{ Type: "Rel", Object : "Person", RelName : "String", Optional : "Boolean", Multiplicity : "String"}]
                  ]
                ]
              ]
              [ "Response"
                [{ UID : "1B", name : "Response" }
                  [
                    "HasBeenAccepted"
                  ]
                  [
                    "HasBeenRejected"
                   [ { name : "reason", Type : "Rel", Object : "ErrorCode"}
                  ]
                ]
              ]
            ]
      			[ { UID : "2", name : "Self-certification" }]
      		 	[ { UID : "3", name : "Discordance of personal data" }]
      			[ { UID : "4", name : "Immigration" }]
      		]
      		[ "Civil Status Services"
      			[ { UID : "10", name : "Birth report" }]
      			[ { UID : "11", name : "Marriage" }]
      			[ { UID : "12", name : "Death report" }]
      		]
      		[ "Electoral services"
      			[ { UID : "100", name : "Cancellation from the electoral roll" }]
      		]
      	]
      	["Taxation"
      		[ { UID : "1000", name : "Payment of general advertising tax" }]
      	]
      ]
 

      [ "Objects"  
          [ "Address"  
            [{ UID : "O0001", Street : "String", Number : "Number", Zip : "Number(5)" , City : "String", State : "String", Country :   "String"}]
          ]
          [ "ErrorCode"
           [{ UID : "O0002", Code : "String", Description : "String" }]

          ]
          [ "Person"  
            [{ UID : "O0003", Firstname : "String", Lastname : "String", DateOfBirth : "Date" }]
          ]
      ]

    ] as Universe

UNWIND Universe AS parts
WITH parts[0]  as GovServ, parts[1] AS Objects
UNWIND GovServ AS Areas
WITH GovServ, Areas, Objects
RETURN GovServ, Areas, Objects;

---

Please, don't look at the formal errors, but help me to give a correct organization to the overall structure, keeping in count that this is represents a metamodel, and there are data not instances now but only the structure of the data.

I hope someone can help me.

Thank you

Paolo

[michael_hunger]

Draw your model on a whiteboard first and then you can import the data-source INTO that model, putting the right data in the right places.

[michael_hunger]

The formal errors are missing commas and such.

[paolodipietro58]

It's not easy to draw a model: data bring the model with them, and I don't want to change it.

These are the data I extracted from UML, with a 2008 program. The result are 8500 lines of code, where each one build a node

MERGE ( n :Model:Resource { name:'Servizi Anagrafici (BSL)' , ns:'ki' , uid:'400d41e40096' }) RETURN n;

and/or a relationship with another node

MATCH ( from {uid:'400d41e40096'})   MERGE (from)-[r:_HAS ]->(n:_relationship {   name : 'RicevutadiCambiodiAbitazione' , cardinality : '1' , uid : '40b8a25a0396' , complex :'true' }) return r;

and/or a relationship between two nodes:

MATCH (from {uid:'419b75640182'}),(to {uid:'3f4f2dd60303'}) CREATE (from)-[r:_IS_A]->(to) RETURN r;

in this latter case, the system returns a warning about a potential cartesian product.

Now I want to transform them in graph, but it hangs forever. So I need to transform in a different way to input. But don't know which one and how. I don't have to design a graph because I already have, hidden in the data from the UML.

[michael_hunger]

that cartesian product is not a problem.
you just should have (if you have more data) an index on :Label(uid)

if you leave off labels in MATCH/MERGE then indexes won't be used

[paolodipietro58]

Do you mean I need to create an :Label(uid) index for each label!!!

Some years ago we speack about the ability to have a global index.

How can I create a global(uid) index, valid for each label?

Because if I have

MATCH (from {uid:'468e2a2f020e'}),(to {uid:'463346dc0174'})

I have no Idea of the node it will return and of its labels !!!

[ameyasoft]

Here is my solution.
Remove 'RETURN n;' at the end of MERGE statments and make sure that each MERGE will have a different variable:

Create constraints on uid:
CREATE CONSTRAINT ON ( p:Model) ASSERT p.uid IS UNIQUE;
CREATE CONSTRAINT ON ( p1:Zona ) ASSERT p1.uid IS UNIQUE;
CREATE CONSTRAINT ON ( n:Zona_Comunale ) ASSERT n.uid IS UNIQUE;
CREATE CONSTRAINT ON ( n1:Zona_Censuaria ) ASSERT n1.uid IS UNIQUE;
CREATE CONSTRAINT ON ( n2:Web ) ASSERT n2.uid IS UNIQUE;
CREATE CONSTRAINT ON ( r:_relationship ) ASSERT r.uid IS UNIQUE;

MERGE ( n :Model:Zona { name:'Zona' , ns:'ki' , uid:'406444be0140' })
MERGE ( n1 :Model:Zona_Comunale { name:'Zona Comunale' , ns:'ki' , uid:'406445f40008' })
MERGE ( n2 :Model:Zona_Censuaria { name:'Zona Censuaria' , ns:'ki' , uid:'406445ff01d1' })
MERGE ( n3 :Model:Web { name:'Web' , ns:'ki' , uid:'4012534e010a' })
..................
MERGE ( n2000 :Model:Zona { name:'Zona' , ns:'ki' , uid:'xxxxxxxx' })

//_relationship nodes.....Copy these from 4500 statements

MERGE (r:_relationship { name : 'ClasseCatastale' , cardinality : '1' , uid : '4694b1eb00d4' , complex :'true' })

MERGE (r1:_relationship { name : 'CategoriaCatastale' , cardinality : '1' , uid : '4694b25301c9' , complex :'true' })

MERGE (r2:_relationship { name : 'IdentificativodiIscrizioneCameradiCommercio' , cardinality : '1' , uid : '468e2a8e010e' , complex :'true' })

//Add relationships.....

MERGE (n)-[:_HAS]->(r)
MERGE (n)-[:_HAS]->(r1)
MERGE (n1)-[:_HAS]->(r2)

MERGE (n)-[:_IS_A]->(n1)
MERGE (n2)-[:_IS_A]->(n3)

Here is an example:

MERGE ( n :Model:Resource { name:'Servizi Anagrafici (BSL)' , ns:'ki' , uid:'400d41e40096' })
MERGE ( n1 :Model:Resource { name:'Servizi Anagrafici (BSL2)' , ns:'ki' , uid:'3f4f2dd60303' })
MERGE (r:_relationship { name : 'RicevutadiCambiodiAbitazione' , cardinality : '1' , uid : '40b8a25a0396' , complex :'true' })
MERGE (n)-[:_HAS]->(r)
MERGE (n)-[:_IS_A]->(n1)
RETURN n, n1, n2;

Result:

[paolodipietro58]

Just to see the things from another point of view, I have more than 2000 MERGE statement (sequential) like these:

MERGE ( n :Model:Zona { name:'Zona' , ns:'ki' , uid:'406444be0140' }) RETURN n;
MERGE ( n :Model:Zona_Comunale { name:'Zona Comunale' , ns:'ki' , uid:'406445f40008' }) RETURN n;
MERGE ( n :Model:Zona_Censuaria { name:'Zona Censuaria' , ns:'ki' , uid:'406445ff01d1' }) RETURN n;
MERGE ( n :Model:Web { name:'Web' , ns:'ki' , uid:'4012534e010a' }) RETURN n;

then 4500 statements like these:

MATCH ( from {uid:'4694b1c6019b'})   MERGE (from)-[r:_HAS ]->(n:_relationship {   name : 'ClasseCatastale' , cardinality : '1' , uid : '4694b1eb00d4' , complex :'true' }) return r;
MATCH ( from {uid:'4694b1c6019b'})   MERGE (from)-[r:_HAS ]->(n:_relationship {   name : 'CategoriaCatastale' , cardinality : '1' , uid : '4694b25301c9' , complex :'true' }) return r;
MATCH (from {uid:'468e2a2f020e'}),(to {uid:'463346dc0174'}) CREATE (from)-[r:_IS_A]->(to) RETURN r;
MATCH ( from {uid:'468e2a2f020e'})   MERGE (from)-[r:_HAS ]->(n:_relationship {   name : 'IdentificativodiIscrizioneCameradiCommercio' , cardinality : '1' , uid : '468e2a8e010e' , complex :'true' }) return r;
MATCH (from {uid:'468ba8c8030d'}),(to {uid:'401fc02a0042'}) CREATE (from)-[r:_IS_A]->(to) RETURN r;

which is the best approach to make this working using WITH and UNWIND?

[paolodipietro58]

Thanks to Ameyasoft!

But I don't think it will run. I already had a similar experience, and all the records were in memory.

With a 32 GB machine, this approach filled the heap in a short time, stopping for heap error.

Michael Hunger remember me that the cypher environment is not built to manage so much variables, and suggest an approach based on a set of data followed by unwind, but I'm not able to accomplish this task in this case!

But I'll try it and let you know how it will finish!.

If you have time, please, give a look at another post I made on a related problem!!!

It returned the following error just after trying to pass the first MERGE statements (without the ex MATCH-MERGE):

[thomasf]

Allow me to turn your attention to my book " Metadata Recycling into Graph Data Models - The Guide to Data Model Recycling using Neo4j". Several UML representations and other oldies may be converted into Neo4j load templates using Cypher scripts.

Leanpub

Metadata Recycling into Graph Data Models

Recycle, Reuse and Reduce also for data models! Why waste time remodeling the same data, just because you go to graph? Learn how to auto-generate graph data models (for Neo4j) from legacy data models in UML, XML, ERD, concept maps and other formats. ...

Best
Thomas