How to schedule a Cypher instruction for import data automaticaly?

[eulervicente]

Hello friends!

I would like some tips about an ETL process that I'm working for. I have create a Cypher statment using APOC to read data from SQL SERVER and write it in Neo4j. The statment is working fine, but I have to run it manually every time that I need to update the Neo4j database.

I would like to create a JOB that every day at 10:00 PM executes a Cypher statment that update the Neo4j database automaticaly. Is there some way to create a batch file that I can run a Cypher statment via comand line?

[koji]

Hi eulervicente,

This is a sample if you can write bash.

#!/bin/bash

readonly MYCYPHER="/dir/to/yourcypher.txt"
readonly NEO4J_HOME="/dir/to/neo4jhome"
readonly BOLT_ADDRESS="bolt://localhost:"
readonly BOLT_PORT="7687"
readonly NEO4J_USER="neo4j"
readonly NEO4J_PW="yourpassword"

cat ${MYCYPHER} | ${NEO4J_HOME}/bin/cypher-shell -a ${BOLT_ADDRESS}${BOLT_PORT} -u ${NEO4J_USER} -p ${NEO4J_PW}

exit 0

[eulervicente]

Hi Koji! Thank you very much!
Could you help me to do a version for Windows ? I am trying to do it but without success. I run the batch file but nothing happens. I have tested the Cypher statment in the Neo4j web browser and it is working correctly.

Check it out my batch file code:

set MYCYPHER="mycypher.txt"
set NEO4J_HOME="D:\NEO4J\neo4j-community-3.5.11-windows"
set BOLT_ADDRESS="bolt://localhost:7687"
set NEO4J_USER="neo4j"
set NEO4J_PW="mypassword"

D:
cd D:\NEO4J
type %MYCYPHER% | %NEO4J_HOME%\bin\cypher-shell -a %BOLT_ADDRESS% -u %NEO4J_USER% -p %NEO4J_PW%

[koji · Accepted solution]

Hi eulervicente,

I ran slow Windows 10 on my Mac with Parallels.
And I created create_node.txt like this.

CREATE (:Person)

And I wrote this simple code.

type .\import\create_node.txt | .\bin\cypher-shell.bat -a bolt://localhost:7687 -u neo4j -p abcde

There were no errors, but no nodes were created.
So I added a semicolon to create_node.txt.

CREATE (:Person);

Maybe I think your error is the same cause.
You should be able to add a semicolon to the end of the last line and it will work.

[eulervicente]

Hello Koji!

Thanks! It's working now.

I found out an alternative way to do this same task using an APOC procedure: apoc.cypher.runFiles.

call apoc.cypher.runFiles(["ETL_NodeLicitacao.txt","ETL_RelacaoLicitou.txt","ETL_RelacaoCotou.txt"],{}) yield row, result;

[mike_r_black]

Now you're getting into the world of ETL. It can be as simple as @koji gave an example doing or you can use ETL software such as Pentaho, Apache AirFlow, IBM DataStage, Kafka, etc... ETL software bridges the gap between systems, extracts data from a source and inserts into a destination.