Hi I'm new to Neo4j, very very excited on the possibilities but struggling with performance within the very first datasets

[marcos_valentin]

Hi Folks,

I'm Marcos from Brazil.
I work on research regarding agricultural commodities
I'am long time SQL user and now I'm tremendously excited with neo4j

I installed the enterprise version and modeled and uploaded a 600 relationship, some 80 nodes model, with 5 simple hops.
Everything worked fine, but in a Windows 10, corei7, 16 GB RAM creating the relationships and some queries were surprisingly slow to such a small dataset.
I ran the same model in the SQL Server 2017 Graph (nodes and edges) and everything was pretty fast.

I'm quite sure I'm doing something (or many things) wrong.

P.S, while running Bolt, my C: Drive space is eaten away.
Every time I finish, I have to run a C: chkdsk /f/v/x to cleanup.

If anyone can give me a hint, I'll be very grateful.

[david_allen]

Hi Marcos, please post your code, and tell us what it does. Without this we can't see your model or what the cause of the problem might be.

[marcos_valentin]

CREATING RELATIONSHIPS CODE:

MATCH (n1:Product {node_id:'Uréia'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUL', delivered_ton:160293}]->(n2);
MATCH (n1:Product {node_id:'Misturas Fertilizantes Complexos'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUL', delivered_ton:5251}]->(n2);
MATCH (n1:Product {node_id:'SSP - Superfosfato Simples'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUL', delivered_ton:225209}]->(n2);
MATCH (n1:Product {node_id:'TSP - Superfosfato Triplo'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUL', delivered_ton:443}]->(n2);
MATCH (n1:Product {node_id:'HDAN - Nitrato de Amônio (NH4)(NO3)'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:97951}]->(n2);
MATCH (n1:Product {node_id:'MAP - Fosfato monoamônico'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:116028}]->(n2);
MATCH (n1:Product {node_id:'Misturas Fertilizantes Complexos'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:12680}]->(n2);
MATCH (n1:Product {node_id:'Fosfato Natural Reativo'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:8411}]->(n2);
MATCH (n1:Product {node_id:'SSP - Superfosfato Simples'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:400798}]->(n2);
MATCH (n1:Product {node_id:'TSP - Superfosfato Triplo'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:385381}]->(n2);
MATCH (n1:Product {node_id:'MAP - Fosfato monoamônico'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:589646}]->(n2);
MATCH (n1:Product {node_id:'Misturas Fertilizantes Complexos'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'SUDESTE', delivered_ton:7574}]->(n2);
MATCH (n1:Product {node_id:'MAP - Fosfato monoamônico'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'CENTRO-OESTE', delivered_ton:13603}]->(n2);
MATCH (n1:Product {node_id:'SSP - Superfosfato Simples'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'CENTRO-OESTE', delivered_ton:159808}]->(n2);
MATCH (n1:Product {node_id:'TSP - Superfosfato Triplo'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'CENTRO-OESTE', delivered_ton:20303}]->(n2);
MATCH (n1:Product {node_id:'MAP - Fosfato monoamônico'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'CENTRO-OESTE', delivered_ton:64294}]->(n2);
MATCH (n1:Product {node_id:'Misturas Fertilizantes Complexos'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'CENTRO-OESTE', delivered_ton:31133}]->(n2);
MATCH (n1:Product {node_id:'Uréia'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'NORTE-NORDESTE', delivered_ton:312036}]->(n2);
MATCH (n1:Product {node_id:'Sulfato de Amônio (NH4)2SO4'}),(n2:Nutrient_Mist{node_id:'N'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'NORTE-NORDESTE', delivered_ton:52803}]->(n2);
MATCH (n1:Product {node_id:'Fosfato Natural Reativo'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'NORTE-NORDESTE', delivered_ton:3479}]->(n2);
MATCH (n1:Product {node_id:'SSP - Superfosfato Simples'}),(n2:Nutrient_Mist{node_id:'P2O5'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'NORTE-NORDESTE', delivered_ton:124402}]->(n2);
MATCH (n1:Product {node_id:'Cloreto de Potássio (KCl)'}),(n2:Nutrient_Mist{node_id:'K2O'}) CREATE (n1)-[:DELIVERS_BY_PROD {by_production:'NORTE-NORDESTE', delivered_ton:279139}]->(n2);

[david_allen]

Your code will already be a lot faster if you do this:

CREATE INDEX ON :Nutrient_Mist(node_id);
CREATE INDEX ON :Product(node_id);

Do you have these?

[marcos_valentin]

No, I haven't created indexes yet.

[david_allen]

You should do that. Every time you do a MATCH based on a property, make sure you have an index on it. This makes the lookup much, much faster and will speed your import.

The other way you can make your import a lot faster is by doing more work with each query. You do the match on each and every line. Meaning that some nodes you have to match are needing to get matched over and over again, which is a lot slower than necessary. If you did something like LOAD CSV (you can google the docs for cypher load csv) this could be a big speed improvement, or simply matching a node once, and then creating all of the things that you need to in a single query off of that, rather than 1 create per query.

[marcos_valentin]

Thank you very much david.
I'm having some issues with the neo4j python package. Is this forum the right one to post a question?

Best Regards.

[david_allen]

This is the right forum, but please try opening a new thread in the Python Driver Topic

[neo4j_devrel]

What part of Brazil are you from?

[marcos_valentin]

Sorry for the delay.
Rio de Janeiro