Struggling with apoc.periodic.iterate to load data from RDBMS to Neo4j

[nitin_s_saxena]

Hi Team,

I am new to Neo4J and trying to load RDBMS data into Neo4J using apoc.load.jdbc to direct connect to RDBMS and create nodes,relationships and parameters .
While it was working fine and i successfully loaded my sample data into Neo4j.
But as data size is huge that will take time so we planned to use apoc.periodic.iterate to do this activity in batches that was starting point of my problem. initially there were so many syntax issues that i encountered during using apoc.periodic.ierate that got resolved . Now statement ran successfully but not inserting any data in Neo4j.
I uploaded both success load statement ( without apoc.periodic.iterate) and failure load statement(with apoc.periodic.iterate) cases along with output screen shot of failure case.

Success case
success_case.txt (108.4 KB)
Failure Case
failure_case.txt (108.8 KB)

Please let me know what i am doing wrong and guide what to correct

[david_allen]

You've got quite a lot going on here in this example. I think I can give you a couple of suggestions though that can make this drastically easier -- the code you've got is working hard, but there are some better methods I think which would simplify this.

There's so much code here, I need to suggest you approach this a different way. The tactic of looking through the success & failure cases and spotting the tiny syntax difference isn't a good use of time.

Define a SQL view if possible.

Your actual SQL query is doing a lot of text parsing and replacement. I bet this is a big source of your syntax errors, because you're trying to put nested strings inside of strings inside of Cypher, and this is very hard to read and get right. So first, on your SQL database, ideally do this:

CREATE VIEW myGraphView AS (big select statement here)

Then inside of your Cypher code, your SQL query is easy: "SELECT * FROM myGraphView"

Redo your Maps

You have a lot of code that looks like this:

(CASE WHEN row.attrname = 'enodebattributes_enodebname' then n END).enodebattributes_enodebname = row.attrvalue,

There are like 2 dozen of these? It looks like what you're doing is mapping a key/value structure (row.attrname/row.attrvalue) into a set of cypher properties. There's a much easier way to do this with way less syntax. It would work like this:

WITH {
   enodebattributes_enodebname: 'enodebattributes_enodebname'
  (...)
} AS mappings
MERGE (n:EQUIPMENT { EQUIP_INST_ID: row.EQUIP_INST_ID })
ON MATCH SET 
   n[coalesce(mappings[row.attrname], "defaultProp")] = row.attrvalue

Basically this would have you focus on writing the mappings between your "key" fields coming from SQL and your "property names" in your graph nodes. Then the way you set those values is always the same using the simple mapping. The coalesce is there to set a default property if no mapping is found.

If you make these two changes, your code will get way cleaner, way shorter, and any syntax errors you have will be easy to spot

[nitin_s_saxena]

david.allen:

n[coalesce(mappings[row.attrname], "defaultProp")] = row.attrvalue

Hi Davin,

Thanks for revert.
Issue got resolved by changing attrname with ATTRNAME and attrvalue with ATTRVALUE as from the query result column name are in ATTRNAME & ATTRVALUE.

I tried to rewrite the query as suggested by you but not successed as error is shown on below piece of code
n[coalesce(mappings[row.ATTRNAME], 'defaultProp')] = row.ATTRVALUE.
I attached the error code also here

and the complete statement here
load_data_script.txt (62.7 KB)

It seems there is some syntax error, can you please help