Neo4j

david_allen · ‎10-11-2018

This summer, Neo4j launched the ability to create graph clusters in managed Google Kubernetes instances. Check it out!

I'm also adding this as a thread on the community forum so that later folks can find it. You can feel free to post follow-up questions on this topic here, or in the #neo4j-graph-platform:cloud topic.

greta · ‎10-11-2018

Thanks for submitting!

I’ve added a tag that allows your blog to be displayed on the community home page!

greta · ‎10-11-2018

Hi! To find out what I can do, say @greta display help.

brendan · ‎11-04-2018

Hi @david.allen I've gone through this setup and your posts a few times but I'm missing a key detail... how do I connect to the cluster from my code? Normally I give py2neo a bolt or http address and a db username/password. I can't figure out what the address should be in Kubernetes.

david_allen · ‎11-05-2018

@brendan please check the limitations section here:

github.com

neo-technology/neo4j-google-k8s-marketplace/blob/master/user-guide/USER-GUIDE.md#limitations

# Neo4j on Google Kubernetes Engine User Guide

## Overview

Neo4j on GKE allows users to deploy multi-node Neo4j Enterprise Causal Clusters to GKE instances, with configuration options for the most common scenarios.  It represents a very rapid way to get started running the world leading native graph database on top of Kubernetes.

This guide is intended only as a supplement to the [Neo4j Operations Manual](https://neo4j.com/docs/operations-manual/3.4/?ref=googlemarketplace).   Neo4j on GKE is essentially a docker container based deploy of Neo4j Causal Cluster.  As such, all of the information in the Operations Manual applies to its operation, and this guide will focus only on kubernetes-specific concerns and GKE-specific concerns.

## Licensing & Cost

Neo4j on GKE is available to any existing enterprise license holder of Neo4j in a Bring Your Own License (BYOL) arrangement.  Neo4j on GKE is also available under evaluation licenses, contact Neo4j in order to obtain one.   There is no hourly or metered cost associated with using Neo4j on GKE for current license holders; you will pay only for the google compute infrastructure necessary to run the software.

## One time Setup

Before installing Neo4j into your GKE cluster, confirm the following:
- You should have docker and kubectl installed locally from the machine where you want to use neo4j
- You have authenticated google’s CLI tools (gcloud) locally to your account.
- You have run gcloud container clusters get-credentials to configure your local kubectl client to interact with your GKE cluster.
- You should verify that you hold an existing Neo4j Enterprise license, whether purchased, via the startup program, or on an evaluation basis.

This file has been truncated. show original

The cluster is exposed primarily inside of the kubernetes network, not outside. You can set up additional kubernetes things like NodePorts to expose individual pods if you like - one thing you might also want to do is check the SSH port forwarding section in those docs linked above to forward bolt.

Bolt+routing from outside of the kubernetes cluster is a bit problematic at the moment because of the way kubernetes networking works. We weren't able to provide default templates for this because much depends on your local network setting, but those docs there should help.

giriraj_bhojak · ‎08-02-2019

Hello David,

We are trying to use Neo4j on GCP by deploying the causal cluster. We have a set of microservices deployed in another GKE cluster that would like to connect to Neo4j instance that sits in its own cluster.

Since this was posted in November last year, I would like to know if there has been any update to make it easier to connect to the Neo4j cluster.
Would it make sense to expose a cluster IP that manages set of Neo4j nodes and use that IP from the microservices cluster to connect to Neo4j.
We would like to avoid dealing with individual pods in Neo4j for the sake of connection.
Would really appreciate any help in this regard.

Thanks,
Giriraj

david_allen · ‎08-02-2019

There is some more information and suggested solutions around the limitations you can find here:

We don't set this up for users directly though, because it depends on too many configuration aspects of the GKE cluster that we can't anticipate ahead of time in the packaging (like how you do DNS management).

From the outside you could create a DNS name that has multiple A records to point to all of the other cluster members, and then use bolt+routing to that. There are frankly a lot of ways to do it -- but it's for each organization to choose how they want to do this given their security posture and other configuration bits.

A core challenge here is that neo4j uses a smart client-based routing approach (bolt+routing) and Kubernetes really wants to treat all pods as indistinguishable from one another and front them with LBs, and these two approaches do not match well. It's a common situation for other databases in kubernetes as well that differentiate between cluster member roles in their architecture.

giriraj_bhojak · ‎08-19-2019

Hi David,

Thank you for your detailed response earlier.
For our dev environment on a GKE cluster in GCP, we have installed Neo4j Causal Cluster in the same cluster as our app microservices.

When trying to insert some seed data in the database, we connected to the cluster thru kubectl as follows:

kubectl run -it --rm cypher-shell --image=gcr.io/cloud-marketplace/neo4j-public/causal-cluster-k8s:3.5 --restart=Never --namespace=default --command -- ./bin/cypher-shell -u neo4j -p "$NEO4J_PASSWORD" -a $APP_INSTANCE_NAME-neo4j.default.svc.cluster.local

Following is what we get as the after the connection succeeds:

Connected to Neo4j 3.5.1 at bolt://causal-cluster-k8s-1-neo4j.default.svc.cluster.local:7687 as user neo4j.

But when trying to insert some dummy data, we keep getting following:

No write operations are allowed directly on this database. Writes must pass through the leader. The role of this server is: FOLLOWER

Not sure what we are doing wrong here, it does seem like we are using bolt to connect to cypher shell.

We have three nodes in the dev envionment.
Upon running dbms.cluster.overview(), I do see one of the nodes as the Leader and the other two as followers.

Could you please let me know if I am doing something wrong?
Thank you once again for being thorough in your explanation of DNS in the GKE cluster.

Regards,
Giriraj

david_allen · ‎08-19-2019

When you do this in that command:

-a $APP_INSTANCE_NAME-neo4j.default.svc.cluster.local

You are connecting to a DNS service name with a default bolt driver. Essentially your client is looking up the first available node (which happens to be a FOLLOWER) and then connecting you to that. Your writes then fail.

Change it to this:

-a bolt+routing://$APP_INSTANCE_NAME-neo4j.default.svc.cluster.local

This will have your client use a "Routing Driver" which will assure that the application itself routes the right query to the right machine.

https://neo4j.com/docs/driver-manual/1.7/client-applications/#routing_drivers_bolt_routing

giriraj_bhojak · ‎08-19-2019

You are so responsive, thank you very much for that !
That routing change worked like a charm.
Really excited to use Neo4j for our product.

Regards,
Giriraj

Neo4j

Launching Neo4j on Google Kubernetes Marketplace