Neo4j Connector with Spark

[deepika]

I want to load the data into neo4j using spark. I have installed apache spark and adding neo4j connector using python code.

df.write.format("org.neo4j.spark.DataSource")
.option("url","url" )
.option("authentication.type", "basic").option("authentication.basic.username", "xx")
.option("authentication.basic.password","xxxx")
.option("query", "MERGE (city:City{locationId:event.LOCALITY_ID}) ON CREATE SET city.city = event.LOCALITY_NAME")
.mode("Overwrite")
.save()

I am using neo4j apache connector.
neo4j-connector-apache-spark_2.12-4.0.1_for_spark_3.jar

However when i am executing , i am getting below error.

21/08/06 11:29:48 ERROR SchemaService: Query not compiled because of the following exception:
org.neo4j.driver.exceptions.ClientException: Variable event not defined (line 2, column 29 (offset: 71))
"MERGE (city:City{locationId:event.LOCALITY_ID}) ON CREATE SET city.city = event.LOCALITY_NAME"

Can some one assist here.

[poem_daga]

Hello,

I am facing the same issue while executing following Query:

MERGE (account:AccountSS {entityKey: event.entityKey, communityId: event.communityId, appId: event.appId, nativeType: event.nativeType})   MERGE (ap:DBAccessProfileSS {dataCollection: event.dataCollection, name: event.name, description: event.description, displayName: event.displayName})  MERGE (ap)<-[:IS_RELATED_TO_SS]-(account)

in the middle of logs

21/08/19 15:20:11 ERROR SchemaService: Query not compiled because of the following exception:
org.neo4j.driver.exceptions.ClientException: Variable event not defined (line 2, column 38 (offset: 80))
"MERGE (account:AccountSS {entityKey: event.entityKey, communityId: event.communityId, appId: event.appId, nativeType: event.nativeType}) MERGE (ap:DBAccessProfileSS {dataCollection: event.dataCollection, name: event.name, description: event.description, displayName: event.displayName}) MERGE (ap)<-[:IS_RELATED_TO_SS]-(account)"
^
at org.neo4j.driver.internal.util.Futures.blockingGet(Futures.java:143)
at org.neo4j.driver.internal.InternalResult.blockingGet(InternalResult.java:128)
at org.neo4j.driver.internal.InternalResult.consume(InternalResult.java:117)

Though query is executed successfully at the end. But there are exceptions in the log.

at the end of logs

21/08/19 15:20:12 INFO Neo4jDataWriter: Writing a batch of 11 elements to Neo4j,
for jobId=236e007e-85e5-4103-875c-0bc46f21f60f and partitionId=0
with query: WITH $scriptResult AS scriptResult
UNWIND $events AS event
MERGE (account:AccountSS {entityKey: event.entityKey, communityId: event.communityId, appId: event.appId, nativeType: event.nativeType}) MERGE (ap:DBAccessProfileSS {dataCollection: event.dataCollection, name: event.name, description: event.description, displayName: event.displayName}) MERGE (ap)<-[:IS_RELATED_TO_SS]-(account)
21/08/19 15:20:12 INFO DataWritingSparkTask: Committed partition 0 (task 4, attempt 0, stage 4.0)

[deepika]

poem.daga:

ugh query is executed successfully at t

Neo4j told this issue is related to the driver. we have used the latest connector jar file and this error is gone.