Dynamic nodes and building relationships dynamically by Importing data from json files

[poojanarayan080]

I have 3 json files in which there is information about set of nodes I wanted to know if I could dynamically create nodes and relationship dynamically by writing a query using loops

If possible can you please explain with an example "like consider there are 3 json files
file1 contain all information of an individual
file2 contain information about previous job experience
file2 contain information about his education
we have to create a graphdb where file 1 will have relationship with file 2 and file 3 based on similarity"
Not an issue if you consider any other example

Thanks in Advance

[michael_hunger]

see if this helps you

Neo4j Graph Database Platform

Importing JSON Data from a REST API into Neo4j - Developer Guides

This article demonstrates some techniques for loading data from JSON-based REST APIs into Neo4j.

[poojanarayan080]

31m
https://community.neo4j.com/t/dynamic-relationship-by-importing-files-from-json/32996?u=poojanarayan...

I have 3 JSON files "DataStore.json", "Network.json", "VMDetails.json"
I wanted to create nodes and relationships dynamically by importing data
from JSON using apoc.load.json function.
I used the below-mentioned queries
To Create DataStore nodes

CALL apoc.load.json("file:///DataStore.json") YIELD value AS data
MERGE (d:UserD{DataStore: data.DataStore})
SET d.Name = data.Name,
d.Type = data.Type,
d.Capacity = data.Capacity,
d.Free = data.Free

To create Network nodes

CALL apoc.load.json("file:///Network.json") YIELD value AS net
MERGE(n:UserN{Network:net.Network})
SET n.Name =net.Name

Everything works as expected till here

I wanted to create a relationship with DataStore node and VM node and
another relationship with Network node and VM node while creating VM node
I used the below-mentioned query

CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm
MERGE(n:UserN{Network:vm.Network})
MERGE(d:UserD{DataStore:vm.DataStore})
MERGE(v:UserV{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,
v.PowerState = vm.PowerState,
v.RAM = vm.RAM,
v.Uuid = vm.Uuid
MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

This query is creating new DataStore and Network nodes instead of creating
a relationship with the existing nodes. Can anyone tell me where I am going
wrong? It would be very helpful. Thanks in advance

[michael_hunger]

Hi,

as it's not really clear what's contained in the JSON files, it's hard to tell.

Odd that you use UserN etc. as labels instead of Network, DataStore,VM.

You don't need MERGE for the user and datastore as you created them already upfront.

Did you check that your row contains the correct data?
If not your lookup data is wrong and MERGE will create new data.
Sometimes there are spelling errors or surrounding spaces or casing.

i.e.

CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm
return vm
limit 1

[poojanarayan080]

Thank you very much for your reply
If I didn't MERGE the datastore and network node in while creating VM node some mal(new and unwanted)nodes get createdMy row contains correct data.
I have attached the JSON files in this mail.
please go through it and let me know what me the mistake is
Thank you

with Reguards
Pooja

(Attachment DataStore.json is missing)

(Attachment VmDetails.json is missing)

(Attachment network.json is missing)

[michael_hunger]

Seems those fields are arrays do you need to use head(vm.Datastore) etc to get the first element

[poojanarayan080]

Yes Sir
If I use those only I can get the elements

[ameyasoft]

poojanarayan0805:

CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm
MERGE(n:UserN{Network:vm.Network})
MERGE(d:UserD{DataStore:vm.DataStore})
MERGE(v:UserV{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,
v.PowerState = vm.PowerState,
v.RAM = vm.RAM,
v.Uuid = vm.Uuid
MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

Try this:

Replace these two lines:

MERGE(n:UserN{Network:vm.Network})
MERGE(d:UserD{DataStore:vm.DataStore})

with:

MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore

[poojanarayan080]

No sir this doesn't seem to work
I am getting this when I replace
MERGE(n:UserN{Network:vm.Network})
MERGE(d:UserD{DataStore:vm.DataStore})

with this MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore
Before using MERGE for VM nodes

I used match after MERGE command before creating relationship like this

CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm
MERGE(v:VM{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,
v.PowerState = vm.PowerState,
v.RAM = vm.RAM,
v.Uuid = vm.Uuid
MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore

MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

I am getting this error

[ameyasoft]

poojanarayan0805:

MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore

MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

Add WITH condition:

WITH is required between MATCH and MERGE.

MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore

WITH n, d

MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

[poojanarayan080]

Even this doesn't work
I am getting this error

[ameyasoft]

Try this:

 CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm

MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore

WITH n, d

MERGE(v:VM{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,
v.PowerState = vm.PowerState,
v.RAM = vm.RAM,
v.Uuid = vm.Uuid


MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

[poojanarayan080]

I tried this but it gives me a syntax error. But we have defined vm in line-1

[clem]

Cypher is pretty fussy about needing WITH to carry variables along through the query.

My guess is that you need more WITH statements. It's probably one of the finer points of Cypher that's not always obvious until you get used to it.

[poojanarayan080]

Where can I use WITH statement in the above query
I am not able to know where it can be used

[ameyasoft]

ameyasoft:

Try this:

 CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm

MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore

WITH n, d

MERGE(v:VM{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,

Highlighted in Bold.

CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm
//Use WITH here......
WITH vm

MATCH(n:UserN)
WHERE n.Network in vm.Network

MATCH(d:UserD)
WHERE d.DataStore in vm.DataStore

//Second WITH here......
WITH n, d

MERGE(v:VM{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,

[poojanarayan080]

Thank you for your time
But even this does not work
Hence I came up with this query

CALL apoc.load.json("file:///DataStore.json") YIELD value AS data
MERGE (d:DataStore{DataStore: data.DataStore})
SET d.Name = data.Name,
d.Type = data.Type,
d.Capacity = data.Capacity,
d.Free = data.Free
WITH data as data
CALL apoc.load.json("file:///Network.json")YIELD value AS net
MERGE(n:Network{Network:net.Network})
SET n.Name =net.Name
WITH net AS nw
CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm
MERGE(n:Network{Name:vm.Network})
MERGE(d:DataStore{Datastore:vm.DataStore})
MERGE(v:UserV{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,
v.PowerState = vm.PowerState,
v.RAM = vm.RAM,
v.Uuid = vm.Uuid
MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

Here in this query, I am able to create a relationship with vmnode & network node correctly but in the case of datastore node it is creating a new node and creating a relationship with them instead of creating the relationship with the existing nodes

[ameyasoft]

Check to see if data.DataStore = vm.DataStore. Looks like they are not matching. Also check to see the Datastore value of the blue nodes with 'related' connections.
Update: the property names are not matching.
MERGE (d:DataStore{DataStore: data.DataStore})
MERGE(d:DataStore{Datastore:vm.DataStore})
's' is lower case in vm and is upper case in data.
Correct this in your vm code and this should work fine!

[poojanarayan080]

This is the output I am getting if I used this query

CALL apoc.load.json("file:///DataStore.json") YIELD value AS data
MERGE (d:DataStore{DataStore: data.DataStore})
SET d.Name = data.Name,
d.Type = data.Type,
d.Capacity = data.Capacity,
d.Free = data.Free
WITH data as data
CALL apoc.load.json("file:///Network.json")YIELD value AS net
MERGE(n:Network{Network:net.Network})
SET n.Name =net.Name
WITH net AS nw
CALL apoc.load.json("file:///VmDetails.json") YIELD value AS vm
MERGE(n:Network{Name:vm.Network})
MERGE(d:DataStore{DataStore:vm.DataStore})
MERGE(v:UserV{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore,
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network,
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,
v.PowerState = vm.PowerState,
v.RAM = vm.RAM,
v.Uuid = vm.Uuid
MERGE (n)-[:connected]->(v)
MERGE (v)-[:related]->(d)

DataStore..txt (699 Bytes) Network.txt (166 Bytes) VMDetails.txt (17.4 KB)

These are the data in the json file
I have uploaded in text format and removed IP Address for confidential reason
Thanks in advance

[ameyasoft]

Thanks for the data files and will look into these files.

[ameyasoft]

1. Edit DataStore file to make this change:
Change "DataStore":"Datastore-652" to "DataStore":"DataStore-652"

2. Use this script for UserV. Just create these nodes without any relationships.

CALL apoc.load.json("file:///VMDetails.json") YIELD value AS vm
MERGE(v:UserV{vmName:vm.vmName})
SET v.CPU= vm.CPU,
v.ConnectionState = vm.ConnectionState,
v.DataStore = vm.DataStore[0],
v.Hostname = vm.Hostname,
v.InstanceUuid = vm.InstanceUuid,
v.IpAddress = vm.IpAddress,
v.Network = vm.Network[0],
v.NumEthernetCards = vm.NumEthernetCards,
v.NumVirtualDisks = vm.NumVirtualDisks,
v.OverallStatus = vm.OverallStatus,
v.PowerState = vm.PowerState,
v.RAM = vm.RAM,
v.Uuid = vm.Uuid

Network and DataStore values are presented as arrays in VMDetails.json file

3. Create the relationships:
match (a:UserV)
match (b:Network) where b.Network = a.Network
merge (b)-[:connected]->(a)
return a, b

match (a:UserV)
match (b:DataStore) where b.DataStore = a.DataStore
merge (b)-[:related]->(a)
return a, b

Result:

[poojanarayan080]

Thank you very much
This works perfectly