Neo4j

tiwari_milind · ‎07-25-2019

Hi,

I am trying to incorporate machine learning procedures into my Neo4J databsee by following the steps given in the following link below. But as soon as I am doing them my database stops working and I have to delete it. Could anyone advise me how I can incorporate the machine learning procedures into graph algorithm and run the queries.

github.com

neo4j-graph-analytics/ml-models/blob/master/README.adoc

= Neo4j Statistics/Machine Learning Procedures (WIP)

This project uses http://commons.apache.org/proper/commons-math/[Apache Commons Math] to build, update, and store linear regression models in Neo4j.

[NOTE]
This project requires Neo4j 3.x+ 

== Installation

Download the https://github.com/neo4j-graph-analytics/ml-models/releases[latest relase-jar].
Copy it into `$NEO4J_HOME/plugins` and restart your database.

Check out the https://neo4j-graph-analytics.github.io/ml-models/[docs] for details on how to use the procedures.

== Building Manually

This project uses maven. To build a jar-file with the procedure in this
project, simply package the project with maven and place the package in your local repo:

    mvn clean package

This file has been truncated. show original

leonard_panichi · ‎07-26-2019

Hi,
I'm afraid you are not giving enough details. First you should tell which version of Neo4j you are using and which version of neo4j-ml-model.
Then, explain step by step what you did ? Did you build the jar yourself or downloaded an already packaged jar ?
Did you

Stop your database
Copy the jar in the plugins folder
Start your dataabse

in this order ?
Finally Copy/Paste the logs of Neo4j. What happens when you boot the database ?

tiwari_milind · ‎07-26-2019

Hi,

The details are as follows:
My version of Neo4J - Neo4j Desktop 1.2.1
Version of Neo4j ML Model - neo4j-ml-models-1.0.0.jar

I downloaded the already packaged jar.

Yes, I did stop my database and copied the jar in the plugins folder. And then restarted the database.
It is then I am getting the error - "Database Failed To Start".

Anything which I may be doing wrong?

leonard_panichi · ‎07-27-2019

So far it seems right to me. Just a question, if you delete the jar in the plugins directory, the database starts fine ? (Let's make sur that the jar is the problem).
I'm totally ignorant about neo4j desktop, do you mind trying this with neo4j 3.5.8 and neo4j-ml-models 1.0.3?
One more point, I had a look at https://github.com/neo4j-graph-analytics/ml-models/releases
It seems that you need to add dbms.security.procedures.whitelist=regression.*, embedding.* to neo4j.conf, did you ?

I tried to start a database with a dockerised neo4j 3.5.8, neo4j-ml 1.0.3 and the line I mentioned before in neo4j.conf and the database starts fine and all the methods of regression.linear seems to be available.

In my opninion just run neo4j throught docker as I said and you will be all good.

tiwari_milind · ‎07-27-2019

Dear Leonard,

I did add the line for dbms.security when I tried it the first time around.

Sorry about a silly question but what is docker? I get lost with the jargons.

Secondly, so after installing the ml.jar I should delete it?

Sincerely,

Milind

leonard_panichi · ‎07-27-2019

A bout tje .jar, no you should not delete it if you want to use the regressions methods but I wanted you to try to start your database after deleting it. If the database starts after you remove the jar, this confirms that the problem is with the jar file (I think).

About docker, I'm talking about this : https://www.docker.com/ You can very start your database in a container named a "dock", this is what I did when I tried to install neo4j-ml-models, but it's not required so let's skip this.

I think, your best shot grom here is to

Remove neo4j desktop,
install neo4j with one of the methods in https://neo4j.com/docs/operations-manual/current/installation/ , if you can install neo4j 3.5.8 it's better.
add the required line to neo4j.conf
add the neo4j-ml-models 1.0.3 in the plugins folder
start up the all thing and copy paste any error

Side question : which OS are you using ?

tiwari_milind · ‎07-27-2019

Hi Leonard,

I shall try that and let you know. In the meantime, the attached log for your reference if in case anything that I may be doing wrong with the current setting. I am quite scared to update to the new version because the last time I did that, I ended up corrupting my database and the entire project but even then I shall give it a try. By the way, the browser version is 3.5.6. But I will try the latest one as well.

Thanks once again for all the suggestions.

PS: I am using Windows 10.

2019-07-27 22:54:55.152+0000 INFO ======== Neo4j 3.5.6 ========
2019-07-27 22:54:55.161+0000 INFO Starting...
2019-07-27 22:54:56.769+0000 INFO Initiating metrics...
2019-07-27 22:55:03.107+0000 INFO Sending metrics to CSV file at C:\Users\13565987.Neo4jDesktop (backup)\neo4jDatabases\database-2fe08b4e-33d1-44f2-a15b-5baef9bfc96f\installation-3.5.6\metrics
2019-07-27 22:55:03.731+0000 INFO Bolt enabled on 127.0.0.1:7687.
2019-07-27 22:55:05.872+0000 INFO Started.
2019-07-27 22:55:06.097+0000 INFO Mounted REST API at: /db/manage
2019-07-27 22:55:06.121+0000 INFO Mounted unmanaged extension [org.neo4j.graphql] at [/graphql]
2019-07-27 22:55:06.157+0000 WARN Server thread metrics not available (missing neo4j.server.threads.jetty.all)
2019-07-27 22:55:06.172+0000 WARN Server thread metrics not available (missing neo4j.server.threads.jetty.idle)
2019-07-27 22:55:06.210+0000 INFO Server thread metrics have been registered successfully
2019-07-27 22:55:06.707+0000 WARN The following warnings have been detected with resource and/or provider classes:
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResourceExperimental.get(java.lang.String,java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResourceExperimental.options(javax.ws.rs.core.HttpHeaders), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResourceExperimental.executeOperation(java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.ManagementResource.get(java.lang.String,java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.ManagementResource.options(javax.ws.rs.core.HttpHeaders), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.ManagementResource.executeOperation(java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResource.get(java.lang.String,java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResource.options(javax.ws.rs.core.HttpHeaders), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResource.executeOperation(java.lang.String), with URI template, "", is treated as a resource method
2019-07-27 22:55:07.125+0000 INFO Remote interface available at http://localhost:7474/
2019-07-27 22:56:05.515+0000 INFO ======== Neo4j 3.5.6 ========
2019-07-27 22:56:05.523+0000 INFO Starting...
2019-07-27 22:56:06.445+0000 INFO Initiating metrics...
2019-07-27 22:56:12.350+0000 INFO Sending metrics to CSV file at C:\Users\13565987.Neo4jDesktop (backup)\neo4jDatabases\database-2fe08b4e-33d1-44f2-a15b-5baef9bfc96f\installation-3.5.6\metrics
2019-07-27 22:56:12.689+0000 INFO Bolt enabled on 127.0.0.1:7687.
2019-07-27 22:56:14.701+0000 INFO Started.
2019-07-27 22:56:14.947+0000 INFO Mounted REST API at: /db/manage
2019-07-27 22:56:14.967+0000 INFO Mounted unmanaged extension [org.neo4j.graphql] at [/graphql]
2019-07-27 22:56:15.027+0000 INFO Server thread metrics have been registered successfully
2019-07-27 22:56:15.461+0000 WARN The following warnings have been detected with resource and/or provider classes:
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResourceExperimental.get(java.lang.String,java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResourceExperimental.options(javax.ws.rs.core.HttpHeaders), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResourceExperimental.executeOperation(java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.ManagementResource.get(java.lang.String,java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.ManagementResource.options(javax.ws.rs.core.HttpHeaders), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.ManagementResource.executeOperation(java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResource.get(java.lang.String,java.lang.String), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResource.options(javax.ws.rs.core.HttpHeaders), with URI template, "", is treated as a resource method
WARNING: A sub-resource method, public final javax.ws.rs.core.Response org.neo4j.graphql.GraphQLResource.executeOperation(java.lang.String), with URI template, "", is treated as a resource method
2019-07-27 22:56:15.857+0000 INFO Remote interface available at http://localhost:7474/
2019-07-27 22:56:21.672+0000 WARN The client is unauthorized due to authentication failure.
2019-07-27 23:04:57.347+0000 INFO ======== Neo4j 3.5.6 ========
2019-07-27 23:04:57.355+0000 INFO Starting...
2019-07-27 23:04:58.401+0000 INFO Initiating metrics...
2019-07-27 23:05:18.625+0000 INFO Sending metrics to CSV file at C:\Users\13565987.Neo4jDesktop (backup)\neo4jDatabases\database-2fe08b4e-33d1-44f2-a15b-5baef9bfc96f\installation-3.5.6\metrics
2019-07-27 23:05:19.044+0000 INFO Bolt enabled on 127.0.0.1:7687.
2019-07-27 23:05:20.738+0000 INFO Started.
2019-07-27 23:05:20.845+0000 INFO Mounted REST API at: /db/manage
2019-07-27 23:05:20.873+0000 INFO Mounted unmanaged extension [org.neo4j.graphql] at [/graphql]
2019-07-27 23:05:20.957+0000 INFO Server thread metrics have been registered successfully
2019-07-27 23:05:21.110+0000 INFO Stopping...
2019-07-27 23:05:21.525+0000 INFO Stopped.
2019-07-27 23:05:21.526+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@16ceabce' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.getContextClassLoaderPA()Ljava/security/PrivilegedAction;". Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@16ceabce' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.getContextClassLoaderPA()Ljava/security/PrivilegedAction;".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@16ceabce' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.getContextClassLoaderPA()Ljava/security/PrivilegedAction;".
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:45)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:187)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:124)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
at com.neo4j.server.enterprise.CommercialEntryPoint.main(CommercialEntryPoint.java:22)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@16ceabce' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.getContextClassLoaderPA()Ljava/security/PrivilegedAction;".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:180)
... 3 more
Caused by: java.lang.NoSuchMethodError: com.sun.jersey.core.reflection.ReflectionHelper.getContextClassLoaderPA()Ljava/security/PrivilegedAction;
at com.sun.jersey.spi.scanning.AnnotationScannerListener.(AnnotationScannerListener.java:94)
at com.sun.jersey.spi.scanning.PathProviderScannerListener.(PathProviderScannerListener.java:59)
at com.sun.jersey.api.core.ScanningResourceConfig.init(ScanningResourceConfig.java:79)
at com.sun.jersey.api.core.PackagesResourceConfig.init(PackagesResourceConfig.java:104)
at com.sun.jersey.api.core.PackagesResourceConfig.(PackagesResourceConfig.java:78)
at com.sun.jersey.api.core.PackagesResourceConfig.(PackagesResourceConfig.java:89)
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:696)
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:674)
at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:205)
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:394)
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:577)
at javax.servlet.GenericServlet.init(GenericServlet.java:241)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:671)
at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:428)
at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312)
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:743)
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:369)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:852)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.server.Server.start(Server.java:418)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.Server.doStart(Server.java:382)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.neo4j.server.web.Jetty9WebServer.startJetty(Jetty9WebServer.java:329)
at org.neo4j.server.web.Jetty9WebServer.start(Jetty9WebServer.java:159)
at org.neo4j.server.AbstractNeoServer.startWebServer(AbstractNeoServer.java:290)
at org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter.start(AbstractNeoServer.java:543)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 5 more
2019-07-27 23:05:21.528+0000 INFO Neo4j Server shutdown initiated by request

tiwari_milind · ‎07-27-2019

I did try 3.5.8 and it did not work. Another thing with 3.5.8, it does not use GraphQL-Endpoint extension for Neo4J which is strange.

leonard_panichi · ‎07-28-2019

Hi Tiwari,
you tried 3.5.8 with neo4j-ml 1.0.3 ?
Any way I'm looking at your logs and I see what is, I think, a compatibility issue between unmanaged extensions...
You are using a graph-ql extension am I right ? What jar have you in your plugins directory ?
Could you remove all your jar from the plugins directory excpet neo4j-ml and start again ?

tiwari_milind · ‎07-28-2019

Hi Leonard,

I tried it. But it did not work.

2019-07-28 22:09:23.933+0000 INFO ======== Neo4j 3.5.8 ========
2019-07-28 22:09:23.940+0000 INFO Starting...
2019-07-28 22:09:24.853+0000 INFO Initiating metrics...
2019-07-28 22:09:42.206+0000 INFO Sending metrics to CSV file at C:\Users\13565987.Neo4jDesktop (backup)\neo4jDatabases\database-6ed746b1-d634-4902-a7d1-f747ea1ade81\installation-3.5.8\metrics
2019-07-28 22:09:43.120+0000 INFO Bolt enabled on 127.0.0.1:7687.
2019-07-28 22:09:45.073+0000 INFO Started.
2019-07-28 22:09:45.205+0000 INFO Mounted REST API at: /db/manage
2019-07-28 22:09:45.300+0000 WARN Server thread metrics not available (missing neo4j.server.threads.jetty.all)
2019-07-28 22:09:45.303+0000 WARN Server thread metrics not available (missing neo4j.server.threads.jetty.idle)
2019-07-28 22:09:45.363+0000 INFO Server thread metrics have been registered successfully
2019-07-28 22:09:45.730+0000 INFO Stopping...
2019-07-28 22:09:46.161+0000 INFO Stopped.
2019-07-28 22:09:46.162+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@76f42a10' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.classForNamePA(Ljava/lang/String;)Ljava/security/PrivilegedAction;". Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@76f42a10' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.classForNamePA(Ljava/lang/String;)Ljava/security/PrivilegedAction;".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@76f42a10' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.classForNamePA(Ljava/lang/String;)Ljava/security/PrivilegedAction;".
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:45)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:187)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:124)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
at com.neo4j.server.enterprise.CommercialEntryPoint.main(CommercialEntryPoint.java:22)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@76f42a10' was successfully initialized, but failed to start. Please see the attached cause exception "com.sun.jersey.core.reflection.ReflectionHelper.classForNamePA(Ljava/lang/String;)Ljava/security/PrivilegedAction;".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:180)
... 3 more
Caused by: java.lang.NoSuchMethodError: com.sun.jersey.core.reflection.ReflectionHelper.classForNamePA(Ljava/lang/String;)Ljava/security/PrivilegedAction;
at com.sun.jersey.server.impl.modelapi.annotation.IntrospectionModeller.workOutPostConstructPreDestroy(IntrospectionModeller.java:225)
at com.sun.jersey.server.impl.modelapi.annotation.IntrospectionModeller.createResource(IntrospectionModeller.java:132)
at com.sun.jersey.server.impl.application.WebApplicationImpl.getAbstractResource(WebApplicationImpl.java:769)
at com.sun.jersey.server.impl.application.WebApplicationImpl.createAbstractResourceModelStructures(WebApplicationImpl.java:1595)
at com.sun.jersey.server.impl.application.WebApplicationImpl._initiate(WebApplicationImpl.java:1356)
at com.sun.jersey.server.impl.application.WebApplicationImpl.access$700(WebApplicationImpl.java:180)
at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:799)
at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:795)
at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:795)
at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:790)
at com.sun.jersey.spi.container.servlet.ServletContainer.initiate(ServletContainer.java:509)
at com.sun.jersey.spi.container.servlet.ServletContainer$InternalWebComponent.initiate(ServletContainer.java:339)
at com.sun.jersey.spi.container.servlet.WebComponent.load(WebComponent.java:605)
at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:207)
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:394)
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:577)
at javax.servlet.GenericServlet.init(GenericServlet.java:241)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:671)
at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:428)
at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312)
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:743)
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:369)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:852)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167)
at org.eclipse.jetty.server.Server.start(Server.java:418)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.Server.doStart(Server.java:382)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.neo4j.server.web.Jetty9WebServer.startJetty(Jetty9WebServer.java:329)
at org.neo4j.server.web.Jetty9WebServer.start(Jetty9WebServer.java:159)
at org.neo4j.server.AbstractNeoServer.startWebServer(AbstractNeoServer.java:290)
at org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter.start(AbstractNeoServer.java:543)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 5 more
2019-07-28 22:09:46.164+0000 INFO Neo4j Server shutdown initiated by request

leonard_panichi · ‎07-29-2019

Well, I'm sorry Tiwari
Your error looks a lots like https://markhneedham.com/blog/2015/08/11/java-jersey-java-lang-nosuchmethoderror-com-sun-jersey-core...
But I can confirm that there is no such library incompatibility between neo4j 3.5.8 and neo4l-ml 1.0.3 (I tried it)
When your remove all extensions from the plugins folder did you also comment all the lines related to thoses extensions in neo4j.conf ? This can be an other reason for the database not to start.
An other thing you can try : use neo4j 3.5.8 enterprise. Who knows...

Usually, when I encounter such problem, I create a database perfectly clean, with the bare minimum, and I try thing in this config.

I'm afraid your problem is beyond my abilities.

tiwari_milind · ‎07-29-2019

Dear Leonard,

Thanks for all the efforts. I shall keep on trying and figure out a way to use ML procedures in Neo4J. If I succeed, I shall let you know what worked for me.

But once again, thanks a lot.

sriram_chittath · ‎08-12-2019

I had the exact same issue with Neo4j desktop. So as suggested i switched to running this as a Docker container -- This works
I am now able to see the below procedures when i do -- CALL dbms.procedures()

embedding.deepWalk
embedding.deepWalk.stream
embedding.deepgl
embedding.deepgl.stream

Used Neo4j 3.5.8 docker container and neo4j-ml 1.0.3

@leonard.panichi

I see these warnings in the log is this okay ?

WARN [o.n.k.i.p.Procedures] Failed to load embedding.Pruning from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: org/neo4j/graphalgo/core/utils/dss/DisjointSetStruct$Result

WARN [o.n.k.i.p.Procedures] Failed to load org.apache.hadoop.http.HttpServer2 from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: org/mortbay/util/MultiException

WARN [o.n.k.i.p.Procedures] Failed to load org.apache.hadoop.record.compiler.ant.RccTask from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: org/apache/tools/ant/Task

WARN [o.n.k.i.p.Procedures] Failed to load org.apache.log4j.net.SMTPAppender from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: javax/mail/MessagingException

WARN [o.n.k.i.p.Procedures] Failed to load com.thoughtworks.xstream.io.xml.Dom4JXmlWriter from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: org/dom4j/Element

WARN [o.n.k.i.p.Procedures] Failed to load com.thoughtworks.xstream.io.xml.JDomDriver from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: org/jdom/JDOMException

WARN [o.n.k.i.p.Procedures] Failed to load embedding.Pruning from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: org/neo4j/graphalgo/core/utils/dss/DisjointSetStruct$Result

WARN [o.n.k.i.p.Procedures] Failed to load org.apache.commons.configuration.tree.xpath.ConfigurationNodeIteratorAttribute from plugin jar /var/lib/neo4j/plugins/neo4j-ml-models-1.0.3.jar: org/apache/commons/jxpath/ri/model/NodeIterator

leonard_panichi · ‎08-12-2019

Hello Sriram,
Thanks for posting your logs, it might be useful for other people.
I'm not a dev of this library, neither a very competent java dev, so it's hard for me to tell, but I think you are ok, I see only warnings. Maybe it could be wise to try the prunning options of the algorithms you intend to use, for those with such options ? I see also some library related to xml and smtp, I wonder what is their purpose here...

One thing : have you compiled ml-models yourself or have you use the build on github ?

Any way, I thought it might be usefull for people struggling to post the docker compose I used to run neo :

version: '3.3'
services:
  neo4j:
    restart: always
    image: neo4j:3.5.7-enterprise
    ports:
      - 7474:7474
      - 6477:6477
      - 7687:7687
    volumes:
      - ./neo4j/conf:/conf
      - ./neo4j/plugins:/plugins
      - ./neo4j/data:/data
      - ./neo4j/logs:/logs
    environment:
      - NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
      - NEO4J_AUTH=neo4j/neo
      - NEO4J_dbms_connectors_default__listen__address=0.0.0.0
      - NEO4J_dbms_memory_heap_initial__size=4G
      - NEO4J_dbms_memory_heap_max__size=4G
      - NEO4J_dbms_memory_pagecache_size=2G
      - NEO4J_dbms_security_procedures_unrestricted=algo.*,apoc.*
      - NEO4J_dbms_tx__log_rotation_retention__policy=3G size
      - NEO4J_dbms_backup_enabled=true
      - NEO4J_dbms_backup_address=0.0.0.0:6362
      - NEO4J_dbms_logs_query_enabled=true
      - NEO4J_dbms_logs_query_threshold=1000ms
      - NEO4J_dbms_logs_query_parameter__logging__enabled=true
      - NEO4J_metrics_csv_enabled=false
      - NEO4J_metrics_prometheus_enabled=true
      - NEO4J_metrics_prometheus_endpoint=neo4j:2004
    networks:
      - my-network

networks:
  my-network:
    external: true

tiwari_milind · ‎08-12-2019

Dear Sriram,

Thanks for letting me know about that. But one question (sorry for my lack of knowledge) - How can I use a docker container?

If you could just explain me the process I can give it a try.

Thanks once again to you and @LeonardPanichi

tony_chiboucas · ‎08-15-2019

There are a few things happening here, but most importantly are those warnings telling you that components of neo4j-ml-models failed to load. When one of those are missing, the other parts of the plugin are erroring because they're calling functions that don't exist.

A quick look at the neo4j-ml-models repo:

neo4j-ml-models 1.0.2
neo4j.version 3.4.7
neo4j.graphalgos.version 3.4.7.0

So, to get this running:

use neo4j 3.4.7
install the Graph Algorithms plugin 3.4.7.0
add dbms.security.procedures.whitelist=regression.*, embedding.* to neo4j.conf
add dbms.security.procedures.whitelist=embedding.* to neo4j.conf
restart the database

That should get Neo4j running with all the requirements. Once you can get neo4j up and running on it's own, with all those dependencies loaded without warning, get back into following that guide.

Remeber: Warnings are your friends, and errors always tell you what's happening if you carefully read them.

sriram_chittath · ‎08-12-2019

@tiwari.milind

You could follow this to install docker on ubuntu -
https://docs.docker.com/install/linux/docker-ce/ubuntu/
https://neo4j.com/developer/docker-run-neo4j/

@leonard.panichi

I am suspecting that the warnings are due to the fact that i used the pre-built jar from here https://github.com/neo4j-graph-analytics/ml-models/releases

Should i compile the ml-models from source ?

leonard_panichi · ‎08-13-2019

You can try to compile it yourself to see if that changes anything, but it's not the first time I see warnings in the log file, as long as the database starts I think it's ok.

sriram_chittath · ‎08-14-2019

I am trying to follow this article https://tbgraph.wordpress.com/2018/10/17/feature-extraction-on-a-peer-to-peer-network-with-deepgl-em...

algo.pagerank works

When i do deepgl as mentioned in the article
CALL embedding.deepgl("Host","CONNECTION", {
nodeFeatures: ['pagerank'],
iterations: 2
})

Error
Neo.ClientError.Procedure.ProcedureCallFailed
Neo.ClientError.Procedure.ProcedureCallFailed: Failed to invoke procedure embedding.deepgl: Caused by: java.lang.NoSuchMethodError: org.neo4j.graphalgo.core.heavyweight.HeavyGraph.degree(ILorg/neo4j/graphdb/Direction;)

Is this because the pre-built jar does not have the right dependencies -- how can such issues be solved. Does the above article work in your setup ?

sriram_chittath · ‎08-16-2019

@tony.chiboucas

Thanks it worked. I used the exact versions you mentioned without issues. Was able to run both the deepgl and deepWalk -- now proceeding with other steps.

Does this mean that to use the latest Neo4j 3.58 the ml-models library need to be fixed to suport that version. Hence we need to wait.

tony_chiboucas · ‎08-16-2019

Yay! Glad I could help.

Does this mean that to use the latest Neo4j 3.58 the ml-models library need to be fixed to suport that version. Hence we need to wait.

That's why versioning and dependency management are a thing.

No need to wait, just use Neo4j 3.4.7. When a newer version of ml-models is released, update and check if everything is still working.

If you wait until ml-models supports Neo4j 3.5, by that time there'll probably be a Neo4j 4.

tiwari_milind · ‎08-21-2019

Hi to all,

Sorry if I am repeating what you have written so using the versions you mentioned above would allow me to run ML models in Neo4J. The versions would have to be exactly:
Neo4J 3.4.7, Graph Algorithms plugin 3.4.7.0 and neo4j ml models 1.0.2? Even Graph Algorithms version 3.4.8 would not work?

Also on a different note, with Neo4J 3.5.8, has the GraphQL plugin been discontinued?

tony_chiboucas · ‎08-21-2019

ml models was last released to support neo4j 3.4.7
It is not likely that ml 1.0.2 will work with neo4j 3.5.x

That said, it GraphAlg 3.4.8 is likely to work fine, but you can always check the repository. Just look in the pom.xml file for whatever version you're curious about. It also might be worth reading up on semantic versioning.

These libraries and plugins are all developed separately, and as such will always lag a little behind the latest version of neo4j. This is because a version of neo4j must be released before developers can even start adjusting the libraries and plugins to support it.

GraphQL recently released 3.5.04, which supports Neo4j 3.5.7.

sai_saidireddym · ‎12-19-2019

Hello Team,

I got an error(below mentioned) while creating a model in Neo4j
I added all three plugins like Apoc, Graph Algorthims, GraphQL
And i added ml model jar file(neo4j-ml-models-1.0.3) in plugins.

Can you please check and provide me the feasible solution.

Neo.ClientError.Procedure.ProcedureNotFound
There is no procedure with the name regression.linear.create registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed.

Thanks
Saidireddy

tony_chiboucas · ‎12-19-2019

Hello Sai,

Please start a new topic for your question. Adding a new separate question onto an existing discussion makes it very difficult to keep things organized, and makes it much harder for others to make use of the discussion.

Issues like yours, with missing procedures or dependencies, are almost always caused by version mismatching between the different plugins you've installed. Dependency management is a pain, and Neo4j is so new that the community hasn't yet come up with a clean solution.

Please read my previous response in this thread.

Sounds to me like you're trying to add 4 different plugins:

Apoc
Graph Algorithms
GraphQL
ML Models

You'll need to carefully check the repository for version compatibility for every plugin you're attempting to add. Just look in the pom.xml file for whatever version you're curious about. It also might be worth reading up on semantic versioning.

If you check the GraphQL release notes for 3.5.0.4, you'll see that it is meant for Neo4j 3.5.6
For GraphAlogorithms 3.5.4.0 it's Neo4j 3.5.3

Keep checking all the plugins you want to use until you find the lowest version of Neo4j (perhaps Neo4j 3.5.3), then you'll need to find the version of each plugin which is meant for that version of Neo4j.

Yeah, it's gonna take some time.