Getting an error while importing with JSON

[mustran]

I have a file structured like this with 111609 records.
(record1)
{
"key": "tr/dec/SRC1997-018",
"editor": [
"Paul R. McJones"
],
"title": [
"The 1995 SQL Reunion: People, Project, and Politics, May 29, 1995."
],
"journal": [
"Digital System Research Center Report"
],
"volume": [
"SRC1997-018"
],
"year": [
"1997"
],
"ee": [
"db/labs/dec/SRC1997-018.html",
"http://www.mcjones.org/System_R/SQL_Reunion_95/"
],
"cdrom": [
"decTR/src1997-018.pdf"
]
},

(record 2)
{
"key": "tr/gte/TR-0263-08-94-165",
"ee": [
"db/labs/gte/TR-0263-08-94-165.html"
],
"author": [
"Frank Manola"
],
"title": [
"An Evaluation of Object-Oriented DBMS Developments: 1994 Edition."
],
"journal": [
"GTE Laboratories Incorporated"
],
"volume": [
"TR-0263-08-94-165"
],
"month": [
"August"
],
"year": [
"1994"
],
"url": [
"db/labs/gte/index.html#TR-0263-08-94-165"
],
"cdrom": [
"GTE/MANO94a.pdf"
]
}, (record n)
With smaller files it just works fine, but when it comes to large files it shows me this error:
"Property values can only be of primitive types or arrays thereof"

This cypher qyery does not work:
WITH "file:///articleFormatted.json" AS url
CALL apoc.load.json(url) YIELD value as article
MERGE (a:Article {a_id: article.key})
SET a.title = article.title

But when I try it this way it works:
This cypher qyery does not work:
WITH "file:///articleFormatted.json" AS url
CALL apoc.load.json(url) YIELD value as article
MERGE (a:Article {a_id: article.key})

I did a test and created the file with only first 3 records. The first query works perfectly!

[david_allen]

I can't see enough of your input data to pinpoint where the problem is, but the error message is clear. What's happening is that you're setting a property to an array with mixed types. For example if in your json you have:

"title": ["Something", "Something Else"]

That will work. If you have in your JSON

title: ["Something", 3.1415, true]

That will fail with the error you are seeing, because if you have a property value (title) that has an array, the array has to be consistently typed.

[mustran]

If that is the case, is there any way to set only the first element of the array. Because the file is so big I cannot check every single record to see if there is a record which contains a mixed data types, even though I think that every record has exactly one String (logically one title of the given article)?

[david_allen]

Yep, easy peasy.

Instead of this:

SET a.title = article.title

Do this:

SET a.title = article.title[0]

[mustran]

Thank you for your answer David. I would be very happy if there is a way to find on which record/line the given title is not as it should be (String or Array)?

[david_allen]

you can use the jq command line tool to query JSON files on disk. By pulling out only the title attribute, and then filtering a bit more you should be able to find which one it is.

https://stedolan.github.io/jq/

[mustran]

Thank you a lot for your clear answers!

[mustran]

I extracted all of the title keys and filtered them to display only those which have the length of 1, which means that there is a single title and no other data types. Unfortunately, I'm still getting the same error on neo4j. It is worth mentioning that the articleFormatted.json file is formatted as well. Validated it with jsonlint. To be more correct I made a function which checks every title value:
Short code sample:

articles.forEach(article=> {
//check if the type of title is string and only one item in the array
    if(typeof ((article.title) == 'string') && (article.title.length === 1)){
        title.push(article.title)
    }
});

As a result I got 111609 items, which is the same number as my records. Mentioned in the first comment.

[mustran]

@david.allen could you please read my last comment? Thank you!

[david_allen]

The reason I suspected the "title" attribute is because the snippet of Cypher you posted, that was the only attribute you were setting. If you're using different cypher setting different properties, then check those properties.

I can't see your data so I'm not certain what to tell you other than what that error means, which is that you're trying to set a property with a complex type, or an array of mixed types. The fix therefore is to clean up the input, either before it gets to cypher, or with cypher.

I can't help much beyond that because it requires the entire JSON and also a larger cypher example to see which properties you're setting.

[mustran]

Hi @david.allen.
Thank you for your reply. Im just trying to set the title as a property of a given node. With other properties (see first comment for the structure of the JSON) such as volume, journey etc it works. The problem appears only with the title property.
Also, I am encountering the same issue in another large JSON only with the title property. But in another JSON file with 845 records I am able to set the title property with no issues!