Gephi integration - update and warning

[llpree]

If visualization is your interest, there are a number of tools to explore graphs. Neo's own Bloom, now integrated with the free desktop version is a great place to start. Another interesting option is Gephi. They've been around for a while and their latest evolution is worth a post to make sure we can avoid some frustrations getting setup.

Gephi is really a presentation tool for a graph database and has always had a nice relationship with Neo. They might be in a transitional point as you can learn from videos from their founders. But my reason for posting here, is to update information that's stale and sometimes misleading on the Neo4j.com website. If you are following the active links, it will confound you as it will also NOT work unless you're using Gephi 0.8 (today's version is 0.9).

In older versions of Gephi/Neo, there was a now-defunct way to integrate Gephi with Neo. You would use an apoc procedure titled: apoc.gephi.add. It required some configuration and this is all spelled out on the current Neo site. The problem is that this doesn't work on Gephi's latest releases (0.9) but does on 0.8 which is a real step down - so don't waste your time.

We have a terrific coder Oliver Rode to thank for a really nice integration approach that is way more comfortable to use for business users as it no longer depends on APOC being installed nor do you need anything but Neo to be running (you need to know Neo's BOLT config for setting things up).

Just go to gephi-neo4j-url-importer-plugin on GitHub and pull down the plugin and read the README and you'll be "painting" pretty views of your graph in no time.

NOTE: It works for Neo 3.x (I've tested for 3.5 and above but NOT for 4.x. I think this is probably a simple fix to the driver. Also, this plugin is for Gephi v0.9.

Enjoy!

PS: if anyone from Neo is reading this, this link was my first hint as to the need for Neo to update their web pages related to Gephi. It's a deadend but at least helped me get to the Git site

Please update the docs on Gephi - either adding the integration for 0.9 or just replace the APOC ref as that's really not the version of Gephi folks will use. Thanks!

[Joel]

Hi Llewellyn,

First, thank you for posting about a Gephi neo4j URL importer plugin, I'm always looking for better, faster, easier ways to do things, I appreciate the share and will check it out. Is it OK with special characters?

Second, I'd let you know that I personally use apoc.gephi.add (Neo4j 3.5.x) with Gephi 0.9.2, and it works great. Another viable option is .graphml export from Neo4j, and then import into Gephi, however I prefer to use apoc.gephi.add and stream it over.

I would be interested in knowing what issue you ran into. Can you provide details?

Syntax issue, or did the stream break?

There is one issue, special characters in the stream link [here] are a problem (https://stackoverflow.com/questions/52318140/streaming-data-from-neo4j-to-gephi-invalid-utf-8-start-...) for more information on that.

EDIT: In my case special characters are a data cleaning issue, they shouldn't be there, however this issue could be a show stopper if working with text in any language other than English. Characters like ö cause the stream to break.

[llpree]

Thanks Joel for getting back. I simply want to make sure we’re giving folks the best chance for success with Neo and the supporting “cast” like Gephi. I’ll repro things when I’ve got a few. I’m heads down and knee deep on a project I’m hoping to leverage Neo for so it’ll need to wait until after the dust settles on that.

I did initially test it on three distinct machines, in one case with a new install of Neo. The error/issue seems more to do with the TCPIP/socket but I was only guessing based on the error messages I was getting – consistent on all 3 machines. Once I found the working (better?) mousetrap, I tested a few times, side by side, and again, it was consistent. I didn’t see entries on StackEx, etc., but was confident I’d uncovered something. Now I’m wondering if there’s something I missed. I do respond as a “ninja” to the community and noticed others with issues. Will be happy to share once I can get to it.

Finally, I will say that the process I outlined with the updated approach, is clean and will be easier for a junior Neo-phyte but I did like the APOC option because of the ability it offered.

I’ll get to this once my current insanity subsides and reply back.

LP

PS: if you are not a big Gephi person, I encourage looking into their recent talks about how they intend to dive into a dependency on the graphics processor over the CPU and doing with via javascript vs java. It’s the right direction and given the talks I’ve had with some Neo architects, Neo should be interested in this topic as well as it touches on what might be useful for Bloom. I see Bloom and Gephi as overlapping a bit but the Gephi team has a lot to offer. Just an FYI for ya.

[Joel]

Yes, I think it can be hard to find the information, and I have posted here on it before.

Neo4j to Gephi import : Failed to invoke procedure Invalid UTF-8 Import / Export

I tried to import my data from Neo4j into Gephi but it doesn't work. I have the following result in Neo4j : Failed to invoke procedure apoc.gephi.add : Caused by: com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0xfb at [Source: (apoc.export.util.CountingInputStream); line: 1, column: 136]

EDIT: So, I saw your recent post more than a week after I answered this other post, but just now I noticed the post dates, it looks like your post was online a day before I answered the other post, so you couldn't have seen my response. I know I did some searches on the topic at the time to see if someone else had already answered it, but never ran across your post, or I would have responded to it as well. Curious..

So that is great news! As far as I knew Gephi was dead (it has been stale for a long time now), exciting to hear there is work on it, I've long felt it is one of, if not the best viz tool, (but with a lot of room for improvement) It can viz larger graphs, and I really enjoy using it to get a sense of the graph neighborhood characteristics. Cytoscape (if you have a lot of patience and time) is pretty nice as well. The first time I tried Cytoscape I gave up, it can be that frustrating, but once you get going it can create some beautiful visualizations. It has two neo4j plugins, so far both work equally well, you can query neo4j directly and display the results in Cytoscape. They each take a little different configuration information, and you need to get one of the plugins from GIT (to patch an issue), so if you try them and run into any issues let me know, I use both plugins.