Hi guys I am trying to retrieve about 200k documents from Elasticsearch and create nodes using them in Neo4j.
I followed the instructions on this page: https://neo4j.com/labs/apoc/4.2/database-integration/elasticsearch/
And my query looked something like this:
"CREATE INDEX ON :Document(docId)
CALL apoc.es.query('localhost','ref','_search','size=1000&scroll=10m',null) yield value with value._scroll_id as scrollId, value.hits.hits as hits
UNWIND hits as hit
CREATE (doc:Document {docId: hit._source.docId, DOB: hit._source.DOB, Email: hit._source.Email, Address: hit._source.Address, `Full Name`: hit._source.`Full Name`, Gender: hit._source.Gender, `Log ID`: hit._source.`Log ID`, Source: hit._source.Source, Passport: hit._source.Passport, ingestJobId: hit._source.ingestJobId, `Source Id`: hit._source.`Source Id`, Age: hit._source.Age, `Medicare Number`: hit._source.`Medicare Number`, `Mobile Phone`: hit._source.`Mobile Phone`})
WITH range(2,10,1) as list, scrollId
UNWIND list as count
CALL apoc.es.get("localhost","_search","scroll",null,{scroll:"10m",scroll_id:scrollId},null) yield value with value._scroll_id as scrollId, value.hits.hits as nextHits
UNWIND nextHits as hit
CREATE (doc:Document {docId: hit._source.docId, DOB: hit._source.DOB, Email: hit._source.Email, Address: hit._source.Address, `Full Name`: hit._source.`Full Name`, Gender: hit._source.Gender, `Log ID`: hit._source.`Log ID`, Source: hit._source.Source, Passport: hit._source.Passport, ingestJobId: hit._source.ingestJobId, `Source Id`: hit._source.`Source Id`, Age: hit._source.Age, `Medicare Number`: hit._source.`Medicare Number`, `Mobile Phone`: hit._source.`Mobile Phone`})
return scrollId, doc"
But It doesn't work, It doesn't give any error it just says ((no changes, no records)
If I run only the First part of the command I can create 10k nodes(tried 100k, it caused error).
"
call apoc.es.get("localhost","ref","_search",null,{size:10000},null) yield value with value.hits.hits as hits Unwind hits as hit
CREATE (doc:Document {docId: hit._source.docId, DOB: hit._source.DOB, Email: hit._source.Email, Address: hit._source.Address, `Full Name`: hit._source.`Full Name`, Gender: hit._source.Gender, `Log ID`: hit._source.`Log ID`, Source: hit._source.Source, Passport: hit._source.Passport, ingestJobId: hit._source.ingestJobId, `Source Id`: hit._source.`Source Id`, Age: hit._source.Age, `Medicare Number`: hit._source.`Medicare Number`, `Mobile Phone`: hit._source.`Mobile Phone`})
"
Tried changing Range(1001,200000,1000), but the result was same
My neo4j version is 4.4.15 and apoc plugin is also compatible.
Kindly tell me what I am doing wrong and how can I make it work.
