Dumping database from jenkins job

[apanza]

(Not sure if this is the best category for this question.)

My main motivation here is to preserve the graph database itself as an artifact of an automated test launched by Jenkins.

I am running neo4j-3.4.5 community edition, installed on RHEL 7.3 via RPM. On that same machine, I am also running jenkins-2.129.

I would like to have a Jenkins job periodically, via neo4j-admin dump, backup the neo4j graph database. I added the jenkins user to the neo4j group: sudo usermod -G neo4j,dockerroot jenkins

ls -l output:

$ ls -l data/databases/
total 4
drwxr-xr-x. 4 neo4j neo4j 4096 Sep  4 16:24 graph.db

$ ll data/databases/graph.db/
total 3693724
drwxr-xr-x. 2 neo4j neo4j        10 Aug 21 18:45 index
-rw-r--r--. 1 neo4j neo4j      8192 Sep  4 16:40 neostore
-rw-r--r--. 1 neo4j neo4j       896 Sep  4 16:25 neostore.counts.db.a
-rw-r--r--. 1 neo4j neo4j       896 Sep  4 16:40 neostore.counts.db.b
-rw-r--r--. 1 neo4j neo4j         9 Sep  5 10:19 neostore.id
[Truncated]

So it looks like members of the neo4j group have read access to the graph.db database. But when I try to dump the database in the Jenkins job, it fails:

[backup_graph_db] $ /bin/sh -xe /tmp/jenkins169767805842862013.sh
+ whoami
jenkins
+ groups
perfsw neo4j dockerroot jenkins
+ systemctl stop neo4j
Failed to stop neo4j.service: Interactive authentication required.
See system logs and 'systemctl status neo4j.service' for details.
+ true
+ systemctl status neo4j
● neo4j.service - Neo4j Graph Database
   Loaded: loaded (/usr/lib/systemd/system/neo4j.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Wed 2018-09-05 10:20:02 PDT; 2h 16min ago
  Process: 1267 ExecStart=/usr/share/neo4j/bin/neo4j console (code=exited, status=0/SUCCESS)
 Main PID: 1267 (code=exited, status=0/SUCCESS)
+ true
++ date +%Y-%m-%d
+ DATE_STAMP=2018-09-05
+ BACKUP_NAME=b9_2018-09-05_d964e4b4411c957fe08f301c4798f423514ba309.dump
+ neo4j-admin dump --database=graph.db --to=b9_2018-09-05_d964e4b4411c957fe08f301c4798f423514ba309.dump
org.neo4j.commandline.admin.CommandFailed: you do not have permission to dump the database -- is Neo4j running as a different user?
	at org.neo4j.commandline.dbms.DumpCommand.execute(DumpCommand.java:100)
	at org.neo4j.commandline.admin.AdminTool.execute(AdminTool.java:127)
	at org.neo4j.commandline.admin.AdminTool.main(AdminTool.java:51)
Caused by: org.neo4j.commandline.dbms.CannotWriteException: Could not write to: /app/neo4j-community/data/databases/graph.db/store_lock
	at org.neo4j.commandline.dbms.StoreLockChecker.check(StoreLockChecker.java:84)
	at org.neo4j.commandline.dbms.DumpCommand.execute(DumpCommand.java:86)
	... 2 more
command failed: you do not have permission to dump the database -- is Neo4j running as a different user?
Build step 'Execute shell' marked build as failure
Finished: FAILURE

I thought that, at the very least, since jenkins user is in the neo4j group, and the graph.db database is group readable, then the jenkins user should be able to read and dump that database, even if jenkins is not the owner. Is there something else I have to do to make non-owners be able to dump the database? Or is only the owner permitted to do so? (I'm aware that Enterprise Edition has other options for backups.)

Is there some alternative way to periodically perform a dump/backup of the database? The only potential option I can see right now is to make the jenkins user a passwordless sudoer. (There is also a secondary issue to contend with: stopping and restarting the neo4j service within the jenkins job.)

[michael_hunger · Accepted solution]

That exception is thrown if it cannot write the file, can you check the permissions on
/app/neo4j-community/data/databases/graph.db/store_lock

Or actually all files under: /app/neo4j-community/data/databases/graph.db/ that they are group-writeable?
Perhaps your dump script has to set the permissions first as the file is recreated.

If that doesn't help, please raise an issue here: github.com/neo4j/neo4j/issues

[apanza]

michael.hunger:

/app/neo4j-community/data/databases/graph.db/

Thanks, making all the files under /app/neo4j-community/data/databases/graph.db/ group writeable worked.

It's a little surprising that a backup process would need write access, but I suppose needing to write to the store_lock file at least partially explains it.