Head's Up! These forums are read-only. All users and content have migrated. Please join us at community.neo4j.com.
12-23-2022 07:58 AM
Hi there,
I am trying to import CSV data using Neo4j Desktop. This is my code below.
:auto USING PERIODIC COMMIT
LOAD CSV WITH HEADERS
FROM 'https://docs.google.com/spreadsheets/d/e/2PACX-1vRJ6RaDEnH2VGyPCb01o2QxRmcyVfOVuJOyf7as8NeMSCPzcjgNUsPOaFy3zXpMF-DM6mfa5f8PqeQR/pub?gid=149005180&single=true&output=csv'
AS row
MERGE (p:Place {uniqueAddressReference: toInteger(row.uniqueAddressReferenceNumber)})
ON CREATE SET
p.fullAddress = CASE row.formated_full_address WHEN "" THEN null ELSE row.formated_full_address END, p.isHospitality = true, p.primaryDescriptionText = CASE row.primaryDescriptionText WHEN "" THEN null ELSE row.primaryDescriptionText END, p.firmsName = CASE row.firmsName WHEN "" THEN null ELSE toLower(row.firmsName) END, p.houseNumberOfName = CASE row.formated_numberOfName WHEN "" THEN null ELSE row.formated_numberOfName END, p.subStreetLevelLineThree = CASE row.formated_subStreetLevelThree WHEN "" THEN null ELSE row.formated_subStreetLevelThree END, p.subStreetLevelLineTwo = CASE row.formated_subStreetLevelTwo WHEN "" THEN null ELSE row.formated_subStreetLevelTwo END, p.subStreetLevelLineOne = CASE row.formated_subStreetLevelOne WHEN "" THEN null ELSE row.formated_subStreetLevelOne END, p.street = CASE row.formated_street WHEN "" THEN null ELSE row.formated_street END, p.town = CASE row.formated_town WHEN "" THEN null ELSE row.formated_town END, p.postalDistrict= CASE row.formated_postalDistrict WHEN "" THEN null ELSE row.formated_postalDistrict END, p.county = CASE row.formated_county WHEN "" THEN null ELSE row.formated_county END, p.postCode = CASE row.postCode WHEN "" THEN null ELSE row.postCode END, p.totalArea = CASE row.totalArea WHEN "" THEN null ELSE row.totalArea END
RETURN p
I got this error below when I try to import.
Neo.DatabaseError.Statement.ExecutionFailed. premature eof
Can you please tell me what is causing this error ?
12-23-2022 08:47 AM
I was able to execute the query to completion. Looks like it may be an issue related to your database files.
12-23-2022 10:05 AM - edited 12-23-2022 10:06 AM
I created a new database to test my file and query. I didn't get the error in the new database. There is a problem with the other database. Some of the data from the files already exists in the old database because I tried to import the data before but it didn't finish the job because my DB got disconnected. Could the problem be that I can't re-import the files again as some of the data already exists in the old database?
12-23-2022 10:42 AM
I don't have intimate knowledge of the neo4j structure, but if you look at the folder/file structure of the database instance's root folder, it has a data/databases folder. In that folder are separate folders for reach db hosted on the instance. I think the data is entirely isolated.
To answer your question, I don't think so. You query uses merge, so it should deal just fine with existing data from the same import. It looks like the data file may have got corrupted, which is a big concern. This is beyond my knowledge, so I don't want to speculate.
It's obvious your import file and query are fine. Both you and I executed it.
There is a consistency check tool in the the admin tool. Maybe that will give you a hint of what the issue is.
https://neo4j.com/docs/operations-manual/current/tools/neo4j-admin/consistency-checker/
BTW- are you using the browser to import this large file? If so, maybe cypher-shell would be better.
12-23-2022 11:47 AM
I am using Neo4j Desktop to import this large file.
12-23-2022 11:54 AM
I try to use the admin tool to check what the problem is . I try this command.
bash-3.2$ bin/neo4j-admin database check Remote
Unmatched arguments from index 0: 'database', 'check', 'Remote'
The database is called Remote. I got this error
Unmatched arguments from index 0: 'database', 'check', 'Remote'
I try replacing the database name "Remote" with the connection URL but I still got the error.
12-23-2022 12:11 PM
I successfully ran this. This is the syntax for version 4.4
./neo4j-admin check-consistency --check-graph=true --check-indexes=true --check-index-structure=true --database=neo4j --verbose
The syntax you tried is for version 5.
12-23-2022 01:49 PM
Thank you. My neo4j database is hosted on Oracle Cloud. So I can't use that syntax to define the pathway for my database. How do I reference my remote DB ?
12-23-2022 02:24 PM
From the manual, it looks like it expects a path.
12-23-2022 08:24 PM
If it can help, the error means that your csv parser reached the end of file too soon.
It was expecting to parse something before the end of it.
12-27-2022 04:31 AM
Do you think the problem is I ran out of memory? I have allocated 12GB to my instance
12-27-2022 08:18 AM - edited 12-27-2022 08:20 AM
No,
With the informations I have,
I think the structure of your file is wrong, it's not able to interpret it's content properly as rows and columns to the end. This might be due to encoding, unfitting columns on a row, etc.
12-27-2022 08:55 AM - edited 12-27-2022 08:55 AM
But when i tested this file
https://docs.google.com/spreadsheets/d/e/2PACX-1vRJ6RaDEnH2VGyPCb01o2QxRmcyVfOVuJOyf7as8NeMSCPzcjgNUsPOaFy3zXpMF-DM6mfa5f8PqeQR/pub?gid=149005180&single=true&output=csv
on this website (https://www.papaparse.com/demo)...there is no error. Does that mean it has nothing to do with the file ?
All the sessions of the conference are now available online