Importing OSM for dummies!

[edurodmic001]

Hi everyone,

first time posting here... sorry if this is a repeat question, but I've searched without any luck so far 😞
I've been battling for a while now trying to import an OSM file into a DB using neo4j-desktop.
The closes command I can see as being applicable is the "CALL spatial.importOSM()",
(Going Spatial: Leverage Geospatial Data with Neo4j 3.4 - YouTube, found here, thanks to Craig and William for your eagerness to share your wealth of knowledge!!)

Is there a trick to importing OSM Data? Can it be done in Desktop or only using Java?
Any help will be greatly appreciated

Thanks in advanced
Mike

[dominicvivek06]

Hi, if you need a docker based solution, i can point to my blog.

http://www.dominickumar.com/blog/neo4j-spatial-docker/

The neo4j-spatial-0.26.2 doesn't work on Neo4j 4.x, it works only on Neo4j 3.x (x = 2,3,4)

Let me know if you need assistance.

[edurodmic001]

Hi Dominic,

thanks for your reply. Because my use-case requires using live data, Docker would end up creating too many inconsistencies. I intend to keep the DB running on a server and reference it using an app running in docker.

BR
Mike

[dominicvivek06]

Hi @edurodmic001 , I usually use Linux and not windows. So, the below steps are for Linux.

1. Download Neo4j Enterprise Edition 3.5.27 Neo4j 3.5.27 (tar) from the download center.

2. Download spatial plugin from https://github.com/neo4j-contrib/spatial/releases/download/0.26.2-neo4j-3.5.2/neo4j-spatial-0.26.2-n... and copy it to the plugins folder. (The link to jar file is changed now after comment from Craig. Wrong link was pasted here before).

3. Update the neo4j.conf file as
   dbms.security.procedures.unrestricted=algo.,apoc.,spatial.*
   dbms.security.procedures.whitelist=algo.,apoc.,spatial.*

4. Start the neo4j insance

5. Download an area from https://extract.bbbike.org/ with the format type -> OSM.

6. Extract the OSM and place it in a folder with .osm format.

7. Open the cypher-shell or browser and execute -> CALL spatial.importOSM("/maps/india.osm");

8. It will take a while depending upon the size of the OSM file. My maximum file size was 50MB, so it took around 5-10 mins.

The above steps were tested on Neo4j 3.x. From the release notes, they have mention that spatial pluging will work on neo4j 4.x also, but i haven't tested. I have created my own custom loader for loading osm files.

Last but not least, try this one also -> https://openmaptiles.org/

Let me know how this works.

Thanks,
Dominic KUMAR

[craig_taverner]

Dominic's advice above is good, although you should take care to match the correct versions of Neo4j. So for the plugin above you must download Neo4j 4.2. It will not work on Neo4j 3.5. Each plugin has the version of neo4j for which it was built included in the plugin name. So neo4j-spatial-0.28.0-neo4j-4.2.3-server-plugin.jar was built with Neo4j 4.2.3. It should work with any version of Neo4j 4.2, but because of the use of internal API's, it might not work with all Neo4j 4.2.x. But it most certainly will give problems on Neo4j 4.1, or 4.0, and definitely will not work at all on Neo4j 3.5.

If you have an already running version of Neo4j, then look at the spatial releases on the page at Releases · neo4j-contrib/spatial · GitHub, and choose one that is closest to the version of Neo4j you are using.

Also, take note that the procedure spatial.importOSM can only import one OSM file. If you plan to import more than one file, then you need to use the 0.28 version, and the procedure spatial.importOSMToLayer.

And finally, if you wish to import large OSM files quickly, it is better to use the external offline import tool at GitHub - neo4j-contrib/osm: OSM Data Model for Neo4j. It is much faster and can import much larger OSM files.

[dominicvivek06]

Hi @craig.taverner , just now I reviewed my post. The plugin is https://github.com/neo4j-contrib/spatial/releases/download/0.26.2-neo4j-3.5.2/neo4j-spatial-0.26.2-n... . I think I copied the wrong release link

[edurodmic001]

Hi @craig.taverner and @dominicvivek06,

thanks a lot for your support. The error I was getting was a FileNotFoundException, no matter what file-path I entered or where I put the .osm file.
For example, using Neo4jDesktop Enterprise running on Windows 10:

The two files shown here are saved in my
C:\<User>\.Neo4jDesktop\relate-data\projects\<SomeLongProjectNumber>\
folder. I tried using the complete file path, the URL, and even just the file name but always received the same error:

Failed to invoke procedure `spatial.importOSM`: Caused by: java.io.FileNotFoundException: map.osm (The system cannot find the file specified)

I'm guessing I'm just referencing the file wrong? Or am I storing it in the wrong place?
I did, however, have more success with apoc.load.xml with the file in the same directory, but I'm guessing importosm would require a lot less post-processing to get the info from the nodes than load.xml?

I'll try your suggestions tonight and see how they go

Mike

[edurodmic001 · Accepted solution]

Ok, and now I think I have it.. Put the files in the right folder (..\relate-data\dbmss\dbmss-xxx\import)
and reference it with CALL spatial.importosm(".\import\map.osm")
That seems to have worked. I definitely imported something anyway 😄

[dominicvivek06]

hope it worked. enjoy the graph database.