Scheduling Neo4j Backups while running in Kubernetes

[david_allen]

I was talking to a customer about this topic earlier today and realized we hadn't written anything up on this, and thought I'd drop a short post here for folks to find as it's relevant.

If you're running Neo4j on Kubernetes, either via the Google Kubernetes Marketplace entry, the public helm chart, or other -- you're probably going to have questions about how to do common maintenance operations in Kubernetes.

As part of the Google Kubernetes Marketplace offering of Neo4j, we engineered a special container to take a backup and copy it to Google Storage, which you can find here:

github.com

neo-technology/neo4j-google-k8s-marketplace/blob/master/backup/README-BACKUP.md

# Backing up Neo4j Containers

This directory contains files necessary for backing up Neo4j Docker containers
to google storage.

See backup.yaml for an example.   

The "credentials.json" file must be a base64-encoded version of a service key JSON that has permissions to write to the targeted google storage bucket.  The example provided is non-functional, and you must substitute your own.  To determine an appropriate value, perform the following:

- Create a service account with appropriate permissions to write to the google
storage bucket
- Save the key in JSON format to your local disk
- `cat my-key.json | base64`
- Use that resulting value in your `backup.yaml` file
- Finally, after adjusting parameters in backup.yaml, run `kubectl apply -f backup.yaml --namespace my-neo4j-deployed-namespace`

This container can be pretty easily adapted, say if you wanted to copy the backup to S3 instead of google storage. The concepts are pretty straight forward in the README, the idea is simply to schedule a new container into your kubernetes cluster which runs neo4j-admin backup, tars/gzips the backup set, and then sends it to the cloud storage provider of your choice.

Because kubernetes gives you the ability to do things like set CronJobs to do full backups on a weekly basis, with something like incrementals nightly, for example.

In a future post when the container is ready, I'll cover how to restore from these same backups. Taken together, this lets you stand up new clusters from existing backups quickly.

[aguyhasnoname]

Are you still going to post about the restore how to you mention in the last paragraph? If so, a link to it would be lovely.

[david_allen]

That article was posted quite some time ago -- here you go:

Medium – 28 Nov 18

How to Restore Neo4j Backups on Kubernetes and GKE – Google Cloud Platform -...

It’s a good idea to have a backup in case things go wrong. So now something has gone wrong. How will you recover?

Reading time: 5 min read

[aguyhasnoname]

Not sure why I didn't poke around for it. Many thanks!