How to auto-import csv content into Neo4j

[kasperknudz]

Hi! We are trying to receive a CSV file from frontend and create nodes from the CSV content. How do we get the neo4J to load the CSV code from javascript automatically when we receive a new CSV-file?

Currently we are thinking something like this to receive the file in frontend:

import fileUpload from "express-fileupload";
app.use(fileUpload());

//Upload endpoint
app.post('/upload', (req, res) => {
  if(req.files === null){
    return res.status(400).json({ msg: 'No file uploaded'});
  }
  const file = req.files.file;
  file.mv(${__dirname}/client/public/uploads/${file.name}, err => {
    if(err){
      console.error();
      return res.status(500).send(err);
    }
    res.json({ fileName: file.name, filePath: `/uploads/${file.name}`});
  })
})

[tony_chiboucas · Accepted solution]

You'll need to trigger a Cypher command, somehow. That could be an API method you add to Neo4j via a plugin. Though that's much trickier than you might think (I'm almost ready to publish an open plugin like this). Before you go too deep, check the import guide:

Neo4j Graph Database Platform

CSV Import Guide: How to Use LOAD CSV Command for Neo4j

Learn how to import CSV data into Neo4j for medium-to-huge datasets and learn to navigate potential issues that might arise during the import process.

Your options:

• Plugin to tweak the api, and handle the upload.
• Driver on the server-side. Running Node.js as a middleman to provide the resource (server-side to handle the POST request) to receive and save the csv, then trigger the Cypher import command.
• Use the Bolt driver in the client, to trigger the Cypher import command.
  • You'd have to build a way to make the CSV file accessible to Neo4j. That can be a url to the file (with some Neo configuration tweaks), or the file can be put in the $NEO_HOME/import directory.
  • Alternatively, you could parse the CSV in the client, and convert it into a set of native Cypher commands that would not require uploading the file.

Cypher IMPORT

The neo4j-javascript driver (docs) (github) can be used server-side, or client-side. There are two Cypher utilities that can import CSV:

• LOAD CSV - Native Cypher. Best for files < 2MB in size.
• apoc.load.csv, or apoc.load.json - Part of the Apoc plugin. Can handle just about anything when combined with apoc.periodic.

However, both of these utilities require the file to already exist somewhere Neo can access.

Client Affect

If you're only building a javascript client, and you want the user to be able to create and upload the csv, then you're going to have to work some magic, and you have a few options:

• Use a intermediary storage location, have the client send the CSV there, resulting in a URL that Neo4j can access, then run the Cypher import command through the neo4j-javascript driver.
• Parse the csv in the client, and built the Cypher commands to MERGE/CREATE Nodes.

[kasperknudz]

We are thinking of the option:

Use a intermediary storage location, have the client send the CSV there, resulting in a URL that Neo4j can access, then run the Cypher import command through the neo4j-javascript driver.

and sessions would be the container that handles execution of cypher-queries into Neo4J when we drag and drop a CSV?

[jggomez]

Hi, I think that an easy solution could be to create a cloud function (for example with Firebase). This function can be triggered when a file is uploaded to storage. This function receives the file, processes and creates the nodes in neo4j. The function will run in background.

https://firebase.google.com/docs/functions/gcp-storage-events