Neo4j 3.5.3 instance keep restarting

[akhil6095]

I'm installing a Neo4j instance over AWS using community AMI (ami-0118d82e9da26d491) which keeps failing with below warning message

 Failed to load `apoc.couchbase.CouchbaseManager` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: com/couchbase/client/java/env/CouchbaseEnvironment
2019-08-02 03:57:46.102+0000 WARN [o.n.k.i.p.Procedures] Failed to load `apoc.couchbase.CouchbaseConnection` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: com/couchbase/client/java/Cluster
2019-08-02 03:57:46.119+0000 WARN [o.n.k.i.p.Procedures] Failed to load `apoc.couchbase.document.CouchbaseUtils` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: com/couchbase/client/core/message/kv/MutationToken
2019-08-02 03:57:46.813+0000 WARN [o.n.k.i.p.Procedures] Failed to load `com.jayway.jsonpath.spi.json.GsonJsonProvider` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: com/google/gson/JsonElement
2019-08-02 03:57:46.815+0000 WARN [o.n.k.i.p.Procedures] Failed to load `com.jayway.jsonpath.spi.json.JsonOrgJsonProvider` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: org/json/JSONException
2019-08-02 03:57:46.833+0000 WARN [o.n.k.i.p.Procedures] Failed to load `com.jayway.jsonpath.spi.mapper.GsonMappingProvider$1` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: com/google/gson/Gson
2019-08-02 03:57:46.834+0000 WARN [o.n.k.i.p.Procedures] Failed to load `com.jayway.jsonpath.spi.mapper.GsonMappingProvider$2` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: com/google/gson/Gson
2019-08-02 03:57:47.860+0000 WARN [o.n.k.i.p.Procedures] Failed to load `org.neo4j.driver.internal.shaded.io.netty.handler.ssl.ConscryptAlpnSslEngine` from plugin jar `/var/lib/neo4j/plugins/apoc-3.5.0.2-all.jar`: org/conscrypt/BufferAllocator

I've verified that corresponding apoc jar apoc-3.5.0.2-all.jar is present at location /var/lib/neo4j/plugins but still it's complaining of unavailability of jar file.

Please find OS and Neo4j versions below:

AWS AMI : ami-0118d82e9da26d491
Neo4j version: 3.5.3

[andrew_bowman]

You may want to use apoc-3.5.0.4-all.jar, I'm not sure if 3.5.0.2 is compatible with Neo4j 3.5.3.

Also most of those failed to load messages will come up anyway, as there are parts of APOC that require other jars to use those procs, such as Couchbase for the couchbase procs, but if you don't intend to use those particular procs then you can ignore those warnings.

[akhil6095]

Updating jar file apoc-3.5.0.4-all.jar did not helped, also I noticed that while restarting neo4j i get below warning

Neo4j not running
Starting Neo4j.
WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
/usr/share/neo4j/bin/neo4j: line 451: /var/run/neo4j/neo4j.pid: No such file or directory
/usr/share/neo4j/bin/neo4j: line 450: /var/log/neo4j/neo4j.log: Permission denied

[andrew_bowman]

Ah, okay I think I see what's going on.

Here's our documentation on increasing your open file limit for linux

[akhil6095]

I've increased the file limit size and after running neo4j using elevated permission i get below error

 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@295d8b70' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687". Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@295d8b70' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@295d8b70' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687".
        at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:45)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:184)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:123)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:90)
        at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:32)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter@295d8b70' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687".
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:177)
        ... 3 more
Caused by: org.neo4j.graphdb.config.InvalidSettingException: Unable to construct bolt discoverable URI using '' as hostname: Expected hostname at index 7: bolt://:7687
        at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.add(DiscoverableURIs.java:133)
        at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.lambda$addBoltConnectorFromConfig$1(DiscoverableURIs.java:155)
        at java.util.Optional.ifPresent(Optional.java:159)
        at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.addBoltConnectorFromConfig(DiscoverableURIs.java:145)
        at org.neo4j.server.rest.discovery.CommunityDiscoverableURIs.communityDiscoverableURIs(CommunityDiscoverableURIs.java:38)
        at org.neo4j.server.CommunityNeoServer.lambda$createDBMSModule$0(CommunityNeoServer.java:99)
        at org.neo4j.server.modules.DBMSModule.start(DBMSModule.java:59)
        at org.neo4j.server.AbstractNeoServer.startModules(AbstractNeoServer.java:246)
        at org.neo4j.server.AbstractNeoServer.access$800(AbstractNeoServer.java:101)
        at org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter.start(AbstractNeoServer.java:528)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
        ... 5 more
Caused by: java.net.URISyntaxException: Expected hostname at index 7: bolt://:7687
        at java.net.URI$Parser.fail(URI.java:2848)
        at java.net.URI$Parser.failExpecting(URI.java:2854)
        at java.net.URI$Parser.parseHostname(URI.java:3390)
        at java.net.URI$Parser.parseServer(URI.java:3236)
        at java.net.URI$Parser.parseAuthority(URI.java:3155)
        at java.net.URI$Parser.parseHierarchical(URI.java:3097)
        at java.net.URI$Parser.parse(URI.java:3053)
        at java.net.URI.<init>(URI.java:673)
        at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.add(DiscoverableURIs.java:128)
        ... 15 more

It's strange that I've bolt configuration in place but it's complaining about bolt URI