Server certificate is not trusted

[seanB]

I have just set up my Neo4j server on a VM on Google Cloud, I'm using Enterprise version 4.1.1, and I've have finished following the great post (here) by @david.allen about how to get a certificate with LetsEncrypt.

This has all worked perfectly and I now have a fully secure Neo4j server that I can access through the browser (MYDOMAIN.COM:7473/browser) using my hostname. However, I am now having issues getting my application to connect to the server using the javascript driver.

I have keep getting the following error:
Failed to connect to server. Please ensure that your database is listening on the correct host and port and that you have compatible encryption settings both on Neo4j server and driver. Note that the default encryption setting has changed in Neo4j 4.0. Caused by: Server certificate is not trusted. If you trust the database you are connecting to, use TRUST_CUSTOM_CA_SIGNED_CERTIFICATES and add the signing certificate, or the server certificate, to the list of certificates trusted by this driver using neo4j.driver(.., { trustedCertificates:['path/to/certificate.crt']}). This is a security measure to protect against man-in-the-middle attacks. If you are just trying Neo4j out and are not concerned about encryption, simply disable it using encrypted="ENCRYPTION_OFF" in the driver options. Socket responded with: ERR_TLS_CERT_ALTNAME_INVALID

I have read through the driver documentation (here) and I have added both the trust: "TRUST_CUSTOM_CA_SIGNED_CERTIFICATES" and trustedCertificates:[] settings. I downloaded all of the certificates from my server (cert.pem, chain.pem, fullchain.pem and privacy.pem) and linked to them in the trustedCertificates setting.

Unfortunately I'm still getting the same error. For reference, this is how my driver is currently configured:

// This module can be used to serve the GraphQL endpoint
// as a lambda function

const { ApolloServer } = require('apollo-server-lambda')
const { makeAugmentedSchema } = require('neo4j-graphql-js')
const neo4j = require('neo4j-driver')

// This module is copied during the build step
// Be sure to run `npm run build`
const { typeDefs } = require('./graphql-schema')

const driver = neo4j.driver(
  process.env.NEO4J_URI,
  neo4j.auth.basic(
    process.env.NEO4J_USER,
    process.env.NEO4J_PASSWORD
  ),
  {
    encrypted: process.env.NEO4J_ENCRYPTED ? 'ENCRYPTION_ON' : 'ENCRYPTION_OFF',
    trust: "TRUST_CUSTOM_CA_SIGNED_CERTIFICATES",
    trustedCertificates: ['../../certificates/cert.pem', '../../certificates/chain.pem', '../../certificates/fullchain.pem', '../../certificates/privkey.pem'],
    logging: {
      level: 'debug',
      logger: (level, message) => console.log(level + ' ' + message)
    },
  }
)

const server = new ApolloServer({
  schema: makeAugmentedSchema({ typeDefs }),
  context: { driver, neo4jDatabase: process.env.NEO4J_DATABASE },
  introspection: true,
  playground: true,
})

exports.handler = server.createHandler()

I'm using the latest build of the driver, v2.14.4 and have enabled full logging but I'm not getting any more information than the above. I just can't figure out what I'm doing wrong - does anyone have any ideas?

[david_allen · Accepted solution]

if you have trusted certificates set up, and you can log into browser and your Chrome or Firefox is not complaining about the SSL, then your certificates should be good.

A thing to check is your NEO4J_URI. If you consult our drivers manual, there was a recent change which introduced new schemes like neo4j+s, neo4j+ssc, and so on. Those schemes can affect the way that certificates are processed, so it's something to look into.

Another thing to double check is that Neo4j separates the SSL policy of bolt and https. In the article, you'll notice settings like dbms.ssl.policy.bolt distinct from dbms.ssl.policy.https. This makes it possible to get https right, but still not have trusted certs in place for bolt, if things aren't set up correctly. You have to have the certificates set correctly for bolt in order for the client connection to work.

Note that if you use a URI scheme like neo4j+s://localhost in combination with a driver flag (trust all certificates) the URI scheme you're using still wins.

[seanB]

Thanks David, that was really useful. I had a look through the documentation (here) and changed my NEO4J_URI from bolt://SO.ME.IP.ADDRESS:7687 to neo4j://MYDOMAIN.COM:7687 and all is working as expected now.

The only issue that I now have is that Neo4j Browser has stopped working (I'm using Chrome). It loads and connects (to neo4j:+s//SO.ME.IP.ADDRESS:7687) but there are multiple errors in the console that state:

Neo4j driver is configured to use secure WebSocket on a HTTP web page. WebSockets might not work in a mixed content environment. Please consider configuring driver to not use encryption.

The Browser also crashes anytime I run a query. Any ideas what might be causing this?

[david_allen]

I can't be sure, but this suggests that something about your SSL setup on the bolt protocol specifically (not HTTPS) is likely wrong. When you grab the browser page (via HTTPS) you get HTML + CSS, and that works, but Neo4j Browser makes a connection on port 7687 to the machine, and it sounds like the Web Browser (Chrome) is rejecting the security specifics of that 7687 connection.

[david_allen]

@seanB the default address that Neo4j Browser will connect to is whatever you configure for the default advertised address of the database.

neo4j+s:// always implies that you're encrypted. The "mixed mode" message you get is that if you're on HTTP (port 7474) and you're trying to make a secure connection (neo4j+s) from an insecure context (HTTP) then your browser will warn you about that. If you have certs in place just use HTTPS all the time, it's better practice for sure, and then your comms won't be "mixed mode" (some encrypted, some unencrypted)

I don't know why the browser would crash; not even sure if you mean "Neo4j Browser" or the "Web Browser". Try checking your JS console to see if it gives any hints if it reoccurs

[seanB]

Thanks David, part of the issue is that I shouldn't be getting the "mixed mode" message as I'm on HTTPS (port 7473) (I've disabled the HTTP connector on port 7474) and trying to make a secure connection from what I believe is a secure context.

Apologies if I wasn't clear in my reply above, but I'm referring to the Neo4j browser crashing. I've been checking the console beyond the warnings I've mentioned above I get the following error every time I run a query on the graph:

react_devtools_backend.js:2273 Error: Reference.set failed: First argument contains a string greater than 10485760 utf8 bytes in property 'users.auth0|5f12e86734250500150753c9.grass.0.data'. So it looks like something to do with being signed in to Neo4j browser sync maybe? I've tried clearing local data and that hasn't helped.

I've also noticed that whilst Neo4j browser is working for me in Safari (for questing etc.) that Neo4j browser sync is not working for me in Safari, so maybe there is a clash happening?

[seanB]

Thanks for taking the time to reply again. I've checked my neo4j.conf file and can't see any issues with my SSL set up. For reference these are the settings I have for the bolt protocol:

Bolt connector

dbms.connector.bolt.enabled=true
dbms.connector.bolt.tls_level=REQUIRED
dbms.connector.bolt.listen_address=0.0.0.0:7687

Bolt SSL configuration

dbms.ssl.policy.bolt.enabled=true
dbms.ssl.policy.bolt.base_directory=certificates/bolt
dbms.ssl.policy.bolt.private_key=/var/lib/neo4j/certificates/bolt/neo4j.key
dbms.ssl.policy.bolt.public_certificate=/var/lib/neo4j/certificates/bolt/neo4j.cert
dbms.ssl.policy.bolt.client_auth=NONE

On further testing, I think this is an issue with the Chrome browser as I can get the Neo4j Browser working as expected in Safari. I have done some more reading around it and according to this stackoverflow post:

Chrome doesn't allow unsecure websocket (ws) connections to localhost (only wss, so you should setup a TLS certificate for your local web/websocket server)

If I'm understanding the issue correctly, this is the similar to the issue I'm experiencing and explains why it's an issue in Chrome but not in Safari.

[seanB]

Has anyone had a chance to look into this to see whether there are any issues with Neo4j browser sync?