Using APOC Procedures on an Embedded Database from Java

[Ciro]

I'm a novice to Neo4j, but have familiarity with graph databases. I've created a simple Java app that creates and queries an embedded Neo4j database. I'd like to use APOC graph analytics procedures to compute typical graph metrics, and be able to export the graph in JSON format to Gephi to visualize my database. I'm doing this with this small, example database before scaling to a larger database in a GCP environment.

I'm using:
-- iMac running macOS V10.15
-- Java 1.8
-- JetBrains IntelliJ IDE IntelliJ IDEA 2019.2.3 (Community Edition)
-- Neo4j V3.5.8
-- Neo4j Java driver V1.7.5
-- Neo4j Desktop app V1.2.1 (I don't use this because I haven't been able to get it to read the contents of my embedded database)

My head is spinning from having read blog posts, Neo4j Community questions & answers, StackOverflow questions and answers, and Neo4j documentation, all of which appear to my untrained eyes as not self-consistent. I'm really confused!

My simple questions are:
-- How do I call an APOC procedure from a Java app?
-- Do I need to install a Neo4j server to gain access to the APOC procedures from a Java app?
-- Do I need to include Maven dependencies to build my Java app?

Could you please either point me to documentation that answers my questions or some source code of a simple Java app that built a graph database and applied APOC procedures to its contents?

Thank you in advance for your help.
Ciro Pinto-Coelho

[stefan_armbrust]

I wonder why you want to use embedded instead of a "normal" deployment.

Regardless on the "why" you do have 2 options for using APOC functionality in embedded. Both require to have

<dependency>
    <groupId>org.neo4j.procedure</groupId>
    <artifactId>apoc</artifactId>
    <version>3.5.0.5</version>
</dependency>

as a dependecy in your project.

Option 1: registering procs

Here you have to explicitly register all procedures and functions explicitly. You identify the class containing the stuff you need and use that snippet: https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/bb6055d43e5536e99917ae5ee92c0ecaecba8c08...
Once registered you can simply use these procedures/functions just like in server mode.

Option 2: use procedures directly from java code

Here you just do a new MyProcedureClass() and explicitly set all its field being annotated with @Context, e.g.

   Periodic p = new Periodic();
   p.db = <mygraphdbservice>;
   p..... = ...;

[Ciro]

Stefan, thanks so much for your reply.

I chose an embedded deployment because my intention is develop the code on my iMac and then move it to a Google Cloud Platform environment to capitalize on the available machine learning, sentiment analysis, and chatbot services as well as to allow the processing to scale over time. If an embedded deployment isn't the right choice, I'm open to any suggestions you have about database deployment.

Thanks for the Maven dependency. Thanks also for describing the two options. I assume I use one of these options to access existing APOC procedures, is that correct?

Do I also use one of these options to access Neo4j graph algorithms described here? Do I need an additional Maven dependency to access the Neo4j graph algorithms?

Thanks in advance for your help, it is much appreciated.

[Ciro]

stefan.armbruster:

org.neo4j.procedure apoc 3.5.0.5

Stefan, I included the Maven dependency you provided into the simple Java app I wrote to create a test database. Here is what I have in my POM file.

        <dependency>
            <groupId>org.neo4j</groupId>
            <artifactId>neo4j</artifactId>
            <version>3.5.8</version>
        </dependency>

        <dependency>
            <groupId>org.neo4j.driver</groupId>
            <artifactId>neo4j-java-driver</artifactId>
            <version>1.7.5</version>
        </dependency>

        <dependency>
            <groupId>org.neo4j.procedure</groupId>
            <artifactId>apoc</artifactId>
            <version>3.5.0.5</version>
        </dependency>

The app worked fine. When I added the Neo4j APOC dependency the app produced the following error message at runtime.

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/io/output/WriterOutputStream
	at apoc.util.ApocUrlStreamHandlerFactory.createURLStreamHandler(ApocUrlStreamHandlerFactory.java:14)
	at java.net.URL.getURLStreamHandler(URL.java:1154)
	at java.net.URL.<init>(URL.java:606)
	at java.net.URL.<init>(URL.java:497)
	at java.net.URL.<init>(URL.java:446)
	at java.net.JarURLConnection.parseSpecs(JarURLConnection.java:175)
	at java.net.JarURLConnection.<init>(JarURLConnection.java:158)
	at sun.net.www.protocol.jar.JarURLConnection.<init>(JarURLConnection.java:81)
	at sun.net.www.protocol.jar.Handler.openConnection(Handler.java:41)
	at java.net.URL.openConnection(URL.java:991)
	at java.net.URL.openStream(URL.java:1057)
	at java.util.ServiceLoader.parse(ServiceLoader.java:304)
	at java.util.ServiceLoader.access$200(ServiceLoader.java:185)
	at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:357)
	at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393)
	at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474)
	at java.util.Iterator.forEachRemaining(Iterator.java:115)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at org.neo4j.configuration.LoadableConfig.allConfigClasses(LoadableConfig.java:106)
	at java.util.Optional.orElseGet(Optional.java:267)
	at org.neo4j.kernel.configuration.Config$Builder.build(Config.java:306)
	at org.neo4j.kernel.configuration.Config.defaults(Config.java:375)
	at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:210)
	at org.neo4j.graphdb.factory.GraphDatabaseFactory.newEmbeddedDatabase(GraphDatabaseFactory.java:79)
	at BaselineGitHubToGraphDb.main(BaselineGitHubToGraphDb.java:98)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.output.WriterOutputStream
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 30 more

Process finished with exit code 1

Lines 97 and 98 in my code (BaselineGitHubToGraphDb) are as follows:

        gdbFactory = new GraphDatabaseFactory();
        gdbService = gdbFactory.newEmbeddedDatabase(new File(gdbDirectory));

What am I missing?

[stefan_armbrust]

does

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.6</version>
</dependency>

do the trick?

[Ciro]

Stefan, that dependency eliminated the error message, thanks.

I've rewritten my code to create a database in the Community Edition server instead of creating an embedded database. How do I call an APOC procedure to work on the data in this "normally deployed" database?

Thanks in advance for your help!

[stefan_armbrust]

Not sure I understood your question correctly. I guess you're aware that you simply do a call apoc.<whatever>() to invoke a procedure.

[nar_kendre]

Experts,
We are using the inventory application with an embedded graph database.

APOC plugin jar apoc-3.5.0.3-all.jar is deployed in against neo4j Release 3.5.5. It is working fine on default graph database.
If same is connected with Embedded Database on bolt protocol 7070, unable to execute the APOC

Similarly

APOC plugin jar apoc-3.4.0.4-all is deployed in against neo4j Release 3.4.18. It is working fine on default graph database.
If same is connected with Embedded mode on bolt protocol 7070, unable to execute the APOC.
Please help to analyse.

Thanks