Looking for appropriate plugin for Netty dependency for SSL provider

[david_strayhorn]

Hello all,

I am in the process of upgrading to Neo4j 4.0 for my hosted website (Linux RedHat, details below) and am stuck on step 11.2.4 of the operations manual, Choosing an SSL provider. The first step is the following:

1. Install a suitable dependency into the plugins/ folder of Neo4j. Dependencies can be downloaded from https://netty.io/wiki/forked-tomcat-native.html.

The problem is that when I follow the link, I cannot a suitable dependency. I assume I am looking for a file with .jar suffix and that I will be putting that file in my plugins folder, but I cannot find any jar files in https://netty.io/wiki/forked-tomcat-native.html. Is there something I am missing?

Thank you,

David

Neo4j version 4.0.2 (in process of upgrading from 3.5.13)
Neo4j Desktop Version 1.2.3.1074
Neo4j Browser Version 4.0.5

Linux version 3.10.0-1062.9.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Fri Dec 6 15:49:49 UTC 2019

[kkris1983]

Hi david.strayhorn4
did you manage to find solution for the issue?
I am in the same spot. I would appreciate learning from your experience.

[jggomez]

Hi, You can download the jar file in Maven Central

https://repo1.maven.org/maven2/io/netty/netty-tcnative/2.0.9.Final/

Thanks

[david_strayhorn]

What worked for me was to go to:
https://netty.io/wiki/forked-tomcat-native.html
then go to the "Downloads" drop-down menu at the top and download
https://dl.bintray.com/netty/downloads/netty-4.1.48.Final.tar.bz2
(looks like it's been updated to 4.1.50 in the interim)
After extracting I copied both files (netty-all-4.1.48.Final.jar and netty-all-4.1.48.Final-sources.jar) from the all-in-one folder to my neo4j plugins folder.
At first it wasn't working for me but then I realized I had enabled the connector
dbms.connector.https.enabled=true
but I had neglected to enable the policy:
dbms.ssl.policy.https.enabled=true
in my conf file and once I did that it worked.

[kkris1983]

Thank you for your helpful replies.
However, when starting server I receive following error

2020-07-06 16:54:16.837+0000 INFO  Starting...
2020-07-06 16:54:24.804+0000 INFO  ======== Neo4j 4.1.0 ========
2020-07-06 16:54:25.162+0000 ERROR Failed to start Neo4j on dbms.connector.http.listen_address, a socket address. If missing port or hostname it is acquired from dbms.default_listen_address. Error starting Neo4j database server at /Users/xxx/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-abc65513-dd91-4140-b69c-201fc2ec2aa8/installation-4.1.0/data/databases
java.lang.RuntimeException: Error starting Neo4j database server at /Users/xxx/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-abc65513-dd91-4140-b69c-201fc2ec2aa8/installation-4.1.0/data/databases
	at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:198)
	at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:158)
	at com.neo4j.server.enterprise.EnterpriseManagementServiceFactory.createManagementService(EnterpriseManagementServiceFactory.java:38)
	at com.neo4j.server.enterprise.EnterpriseBootstrapper.createNeo(EnterpriseBootstrapper.java:20)
	at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:117)
	at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:87)
	at com.neo4j.server.enterprise.EnterpriseEntryPoint.main(EnterpriseEntryPoint.java:25)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.bolt.BoltServer@29dcdd1c' failed to initialize. Please see the attached cause exception "io.netty.internal.tcnative.SSLPrivateKeyMethod".
	at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:424)
	at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:65)
	at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:101)
	at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:189)
	... 6 more
Caused by: java.lang.NoClassDefFoundError: io/netty/internal/tcnative/SSLPrivateKeyMethod
	at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:471)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:571)
	at org.neo4j.ssl.SslPolicy.nettyServerContext(SslPolicy.java:82)
	at org.neo4j.bolt.BoltServer.createSslContext(BoltServer.java:286)
	at org.neo4j.bolt.BoltServer.createExternalProtocolInitializer(BoltServer.java:252)
	at org.neo4j.bolt.BoltServer.init(BoltServer.java:154)
	at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:403)
	... 9 more
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLPrivateKeyMethod
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	... 16 more
2020-07-06 16:54:25.162+0000 INFO  Neo4j Server shutdown initiated by request

Server fails to start when line dbms.connector.bolt.tls_level=OPTIONAL is uncommented.

If I am not mistaken it is related to io/netty/internal/tcnative/SSLPrivateKeyMethod
I spent much time trying to figure out what is the problem and I have no clue how to proceed anymore.
I kindly ask you for help.

I am using neo4j 4.1.0 Enterprise

I generated self-signed certificate with command:

openssl req \
       -newkey rsa:2048 -nodes -keyout private.key \
       -x509 -days 365 -out public.crt

certificate folder structure is following

└── default
    ├── private.key
    ├── public.crt
    ├── revoked
    └── trusted

plugins are following:

netty-all-4.1.50.Final-sources.jar
netty-all-4.1.50.Final.jar

My configuration neo4j.conf is following:

#*****************************************************************
# Neo4j configuration
#
# For more details and a complete list of settings, please see
# https://neo4j.com/docs/operations-manual/current/reference/configuration-settings/
#*****************************************************************

# This setting constrains all `LOAD CSV` import files to be under the `import` directory. Remove or comment it out to
# allow files to be loaded from anywhere in the filesystem; this introduces possible security problems. See the
# `LOAD CSV` section of the manual for details.
dbms.directories.import=import

# Whether requests to Neo4j are authenticated.
# To disable authentication, uncomment this line
dbms.security.auth_enabled=true


#********************************************************************
# Memory Settings
#********************************************************************

# Java Heap Size: by default the Java heap size is dynamically calculated based
# on available system resources. Uncomment these lines to set specific initial
# and maximum heap size.
dbms.memory.heap.initial_size=512m
dbms.memory.heap.max_size=1G

# The amount of memory to use for mapping the store files.
# The default page cache memory assumes the machine is dedicated to running
# Neo4j, and is heuristically set to 50% of RAM minus the Java heap size.
dbms.memory.pagecache.size=512m

# Transaction state location. It is recommended to use ON_HEAP.
dbms.tx_state.memory_allocation=ON_HEAP


#*****************************************************************
# Network connector configuration
#*****************************************************************

# With default configuration Neo4j only accepts local connections.
# To accept non-local connections, uncomment this line:
dbms.default_listen_address=0.0.0.0

# Bolt connector
dbms.connector.bolt.enabled=true
dbms.connector.bolt.tls_level=OPTIONAL
#dbms.connector.bolt.listen_address=:7687
#dbms.connector.bolt.advertised_address=localhost:9000

# HTTP Connector. There can be zero or one HTTP connectors.
dbms.connector.http.enabled=true
#dbms.connector.http.listen_address=:7474

# HTTPS Connector. There can be zero or one HTTPS connectors.
dbms.connector.https.enabled=true
#dbms.connector.https.listen_address=:7473


#*****************************************************************
# SSL policy configuration
#*****************************************************************

# Bolt SSL configuration
dbms.ssl.policy.bolt.enabled=true
dbms.ssl.policy.bolt.base_directory=certificates/default
dbms.ssl.policy.bolt.private_key=private.key
dbms.ssl.policy.bolt.public_certificate=public.crt
dbms.ssl.policy.bolt.client_auth=NONE
dbms.ssl.policy.bolt.trust_all=true

# Https SSL configuration
dbms.ssl.policy.https.enabled=true
dbms.ssl.policy.https.base_directory=certificates/default
dbms.ssl.policy.https.private_key=private.key
dbms.ssl.policy.https.public_certificate=public.crt
dbms.ssl.policy.https.client_auth=NONE
dbms.ssl.policy.https.trust_all=true


#*****************************************************************
# Miscellaneous configuration
#*****************************************************************

# A comma separated list of procedures and user defined functions that are allowed
# full access to the database through unsupported/insecure internal APIs.
dbms.security.procedures.unrestricted=apoc.*


#********************************************************************
# JVM Parameters
#********************************************************************

# G1GC generally strikes a good balance between throughput and tail
# latency, without too much tuning.
dbms.jvm.additional=-XX:+UseG1GC

# Have common exceptions keep producing stack traces, so they can be
# debugged regardless of how often logs are rotated.
dbms.jvm.additional=-XX:-OmitStackTraceInFastThrow

# Make sure that `initmemory` is not only allocated, but committed to
# the process, before starting the database. This reduces memory
# fragmentation, increasing the effectiveness of transparent huge
# pages. It also reduces the possibility of seeing performance drop
# due to heap-growing GC events, where a decrease in available page
# cache leads to an increase in mean IO response time.
# Try reducing the heap memory, if this flag degrades performance.
dbms.jvm.additional=-XX:+AlwaysPreTouch

# Trust that non-static final fields are really final.
# This allows more optimizations and improves overall performance.
# NOTE: Disable this if you use embedded mode, or have extensions or dependencies that may use reflection or
# serialization to change the value of final fields!
dbms.jvm.additional=-XX:+UnlockExperimentalVMOptions
dbms.jvm.additional=-XX:+TrustFinalNonStaticFields

# Disable explicit garbage collection, which is occasionally invoked by the JDK itself.
dbms.jvm.additional=-XX:+DisableExplicitGC

#Increase maximum number of nested calls that are can be inlined from 9 (default) to 15
dbms.jvm.additional=-XX:MaxInlineLevel=15

# Restrict size of cached JDK buffers to 256 KB
dbms.jvm.additional=-Djdk.nio.maxCachedBufferSize=262144

# More efficient buffer allocation in Netty by allowing direct no cleaner buffers.
dbms.jvm.additional=-Dio.netty.tryReflectionSetAccessible=true

# Expand Diffie Hellman (DH) key size from default 1024 to 2048 for DH-RSA cipher suites used in server TLS handshakes.
# This is to protect the server from any potential passive eavesdropping.
dbms.jvm.additional=-Djdk.tls.ephemeralDHKeySize=2048

# This mitigates a DDoS vector.
dbms.jvm.additional=-Djdk.tls.rejectClientInitiatedRenegotiation=true

# Increase the default flight recorder stack sampling depth from 64 to 256, to avoid truncating frames when profiling.
dbms.jvm.additional=-XX:FlightRecorderOptions=stackdepth=256

# Allow profilers to sample between safepoints. Without this, sampling profilers may produce less accurate results.
dbms.jvm.additional=-XX:+UnlockDiagnosticVMOptions
dbms.jvm.additional=-XX:+DebugNonSafepoints

#********************************************************************
# Wrapper Windows NT/2000/XP Service Properties
#********************************************************************
# WARNING - Do not modify any of these properties when an application
#  using this configuration file has been installed as a service.
#  Please uninstall the service before modifying this section.  The
#  service can then be reinstalled.

# Name of the service
dbms.windows_service_name=neo4j

#********************************************************************
# Other Neo4j system properties
#********************************************************************

dbms.netty.ssl.provider=OPENSSL

[kkris1983]

What worked for me was replacing

netty-all-4.1.50.Final-sources.jar
netty-all-4.1.50.Final.jar

with

netty-tcnative-2.0.31.Final-osx-x86_64.jar
netty-tcnative-2.0.31.Final-sources.jar