Database "bridges" is unavailable, its status is "offline"

[mckenzma]

I have been working on a Neo4j 4.1.1 database. I created a new database called bridges and set as default. Everything has been working fine until today. While the databased was offline I did make a copy of the data folder that contains this database as I am trying something else. However, after making a copy of this folder on an external drive, I started up the database and got the error Database "bridges" is unavailable, its status is "offline."

Has anyone else come across this error? @david.allen do you have any insights per chance?

The only thing that I did was make a copy of the data directory onto an external drive and then I can't connect to bridges data base anymore.

[david_allen]

Changes in the 4.1 series added some metadata to the system database that make file copying in the way you're doing it not a safe thing to do. I've run into the problem that you've seen, and when you do file operations (treating "directory as database") you can end up messing up the metadata in systemdb - this leading to problems re-mounting the database when you bring it back online.

If you want to bring a new database online, you can move the files in place and then issue a CREATE DATABASE newThing and it should work as you expect. But avoid swapping a database out that already exists because of the systemdb metadata thing. In this case, if you wanted to swap out an existing database, it's best to DROP DATABASE foo; move the files in place and then CREATE DATABASE foo to mount your new files

[mckenzma]

I wasn't copying a database into the Neo4j Desktop folder structure. Instead I was trying to make a copy of the database from my Neo4j Desktop directory onto an external drive. Would that have the same effect as you described?

[mckenzma]

I am working on deploying a GRANDstack app entirely on a Raspberry Pi and wanted to use my existing db from the desktop to test it out.

[david_allen]

Copying the database out to an external drive wouldn't have that effect, but when you bring it back in somewhere else, it would. The point is that file-based operations aren't safe with 4.1, because the database may be coupled with systemdb internals that can be confused by moving files around. The tooling (dump & backup) as well as the cypher commands (CREATE/DROP database) are preferred to avoid those issues

[mckenzma]

Ok. Well then that would make sense why I can't find the database on the Pi, but not entirely sure it explains why I lost the connection locally on my MacBook Pro. However, I can try those steps you mentioned and see if that does the trick.

So then the steps would be as follows:

1. Create new directory newThing in the databases directory
2. Copy file from current bridges directory into newThing directory
3. From Neo4j Browser: CREATE DATABASE newThing
4. Start Database and verify it works
5. Then run DROP DATABASE bridges

Is that correct?

[david_allen]

That should mostly work, but I want to reiterate -- the neo4j system itself doesn't promise that works or suggest that it's going to work in the future. The issue is the copying files around. The preferred method is to use the tooling, i.e. don't ever copy a neo4j database directory, instead do neo4j-admin dump / neo4j-admin load. Or neo4j-admin backup / neo4j-admin restore.

The reason for this is that there's more going on than just the files you're trying to copy. For example, /var/lib/neo4j/data/databases/foo is one thing, but /var/lib/neo4j/data/transactions/foo is another thing. And that's apart from whatever is in systemdb about that database. For example, if you did fine-grained permissions where you GRANT'd some permission on foo, or REVOKE'd some permission, you can easily end up with a database that's out of sync with systemdb.

tl;dr what you're suggesting should work, but still I wouldn't recommend doing it that way for reasons previously assigned.

[mckenzma]

Ah ok. Well then I definitely didn't think copying out of the database folder would cause any issues. That makes a lot of sense about copying a database into another location.

I will have to read over those toolings then to see if I can get my database back. Definite unintended consequences...

[mckenzma]

I'll return with a summary of my steps once I get it figured out.

[mckenzma · Accepted solution]

After trying several steps to recover and dump I found the following message in my debug.log

It looks like the best and easiest way forward is to rebuild from scratch. For me and what I am working on this is perfectly fine.

If anyone else finds there way to this thread after encountering a similar error and issue, be advised avoid trying to copy a database directory. I was trying to copy an existing one onto an external drive. I neglected to research the appropriate ways of accomplishing this and it appears I have corrupted my database. Not the end of the world but a real pain

TLDR: DON'T TRY USING FILE SYSTEM COPY-AND-PASTE on databases. Follow the correct steps

Marking this as solved

[hayden_freedman]

I'm not sure if anyone else has had a similar problem, but I was having trouble importing a database from elsewhere into a my instance of neo4j 4.2.5 community edition. I kept getting the "database is unavailable, its status is offline" error. I was able to hack-ily fix this by adding this line to neo4j.conf:

dbms.recovery.fail_on_missing_files=false

It seems like the startup was failing due to missing transaction logs in the database I was importing. This command forced creation of new transaction logs.