Mismatching Store ID on neo4j-admin Import

[ericawood]

Hello,
My name is Erica and I am using Neo4j v4.0.5 Community Edition on an Ubuntu Linux 18.04 EC2 instance. When I run neo4j-admin import, the data imports successfully, the database fails to start when I run sudo service neo4j start: (below is an excerpt of my debug.log)

2020-06-23 16:56:55.713+0000 ERROR [o.n.d.d.DefaultDatabaseManager] Failed to start database: neo4j An error occurred! Unable to start database with name `neo4j`.
org.neo4j.dbms.api.DatabaseManagementException: An error occurred! Unable to start database with name `neo4j`.
        at org.neo4j.dbms.database.AbstractDatabaseManager.startDatabase(AbstractDatabaseManager.java:191)
        at org.neo4j.dbms.database.DefaultDatabaseManager.startDatabase(DefaultDatabaseManager.java:132)
        at org.neo4j.dbms.database.DefaultDatabaseManager.initialiseDefaultDatabase(DefaultDatabaseManager.java:64)
        at org.neo4j.dbms.database.DefaultDatabaseInitializer.start0(DefaultDatabaseInitializer.java:39)
        at org.neo4j.kernel.lifecycle.SafeLifecycle.transition(SafeLifecycle.java:124)
        at org.neo4j.kernel.lifecycle.SafeLifecycle.start(SafeLifecycle.java:138)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
        at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:160)
        at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:147)
        at org.neo4j.server.database.CommunityGraphFactory.newDatabaseManagementService(CommunityGraphFactory.java:36)
        at org.neo4j.server.database.LifecycleManagingDatabaseService.start(LifecycleManagingDatabaseService.java:88)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:157)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:114)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:89)
        at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:35)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Mismatching store id. Store StoreId: StoreId{creationTime=1592930762152, randomId=-7903980276561267047, storeVersion=3471765337752883975, upgradeTime=1592930762152, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1592930741648, randomId=-1125231581799010715, storeVersion=3471765337752883975, upgradeTime=1592930741648, upgradeTxId=1}
        at org.neo4j.kernel.database.Database.start(Database.java:478)
        at org.neo4j.dbms.database.AbstractDatabaseManager.startDatabase(AbstractDatabaseManager.java:187)
        ... 17 more
Caused by: java.lang.RuntimeException: Mismatching store id. Store StoreId: StoreId{creationTime=1592930762152, randomId=-7903980276561267047, storeVersion=3471765337752883975, upgradeTime=1592930762152, upgradeTxId=1}. Transaction log StoreId: StoreId{creationTime=1592930741648, randomId=-1125231581799010715, storeVersion=3471765337752883975, upgradeTime=1592930741648, upgradeTxId=1}
        at org.neo4j.kernel.recovery.Recovery.validateStoreId(Recovery.java:388)
        at org.neo4j.kernel.database.Database.start(Database.java:377)
        ... 18 more

When I run cypher-shell -d system and run show databases;, I receive the following:

+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
| name     | address          | role         | requestedStatus | currentStatus | error                                                            | default |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
| "neo4j"  | "localhost:7687" | "standalone" | "online"        | "offline"     | "An error occurred! Unable to start database with name `neo4j`." | TRUE    |
| "system" | "localhost:7687" | "standalone" | "online"        | "online"      | ""                                                               | FALSE   |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------+

I have looked in the following locations for advice:

• https://github.com/neo4j/neo4j/issues/12388
  • The suggestion at the bottom of removing the transaction data, store lock, and system data worked once for me. However, I received the error Caused by: java.lang.NoClassDefFoundError: org/neo4j/graphdb/event/TransactionEventHandler in subsequent attempts, which also prevented the database from starting, making this solution inviable.
• Neo4j v4.0 `Mismatching store id`
  • The advice here seemed to be that you shouldn't copy the database. I do not have this issue because I am using neo4j-admin import. Also, another piece of advice was to follow migration guides. To mitigate this, I uninstalled and reinstalled neo4j completely using sudo apt purge neo4j (then I deleted the directories that weren't empty) and used sudo apt install neo4j, installing 4.0.5. I wasn't looking to transfer any data from 4.0.5, rather to start fresh on 4.0.5.
• Getting an error "Mismatching store id"
  • This also recommended avoiding copying the database from version 3.5. Once again, I used neo4j-admin import.
• https://neo4j.com/developer/kb/how-to-solve-store-copy-failed-due-to-store-id-mismatch/
  • This did not apply because its steps were for Enterprise Edition.

Please let me know if you have any suggestions or questions. My team wants version 4.0 to work because its neo4j-admin import allows for the flag --read-buffer-size.

Thank you,
Erica Wood

[malik]

I'll be watching this issue with you, Erica. I keep making regular attempts to migrate from 3.5.14 that always fail miserably. One of the posts you've mentioned is mine.

Latest thing I've tried was to use APOC to export to a Neo4j Browser-friendly all-plain.cypher file, spent time downloading this huge file only to find out that dragging it to the Neo4j browser window results in a "Something went wrong" error. Useless.
Tried to launch a non-plain text APOC export to an all.cypher file but after a full day of exporting the process was still running. This is prohibitive for me as, if it takes that long, by the time it's over, I'm already a out of sync and should restart the export. Having a platform offline for a day (or more?) is also not an option right now. I know one might suggest I switch to Entreprise but that would be too early for us as well, unfortunately.

Let's hope for best.

[cborromeo]

I have tried using the neo4j-admin import command to import a very simple csv file. I am seeing the same issue as Erica. Here is my setup:

Ubuntu 18.04
Neo4j 4.1.3 community edition

I consulted the same links as Erica. I also deleted any prior neo4j files/versions before installing 4.1.3 using apt-get. I am NOT trying to migrate any data. I am trying to load the csv data into a new neo4j install. Before I run the 'neo4j-admin import' command, I stop neo4j and then delete the files under the /var/lib/data/databases/neo4j directory. This is a very simple file containing 4 node IDs CUIs.csv.txt (48 Bytes). Here is my command:

sudo neo4j-admin import --nodes=Concept="import/CUIs.csv" --skip-bad-relationships --skip-duplicate-nodes

Results:
The output from the import command indicates the data loaded successfully.neo4j-admin-import-output.txt (4.0 KB). However, starting neo4j throws a "Mismatching store id" error in debug.log debug.log.txt (135.9 KB).

Thanks for any help,
Chuck

[cborromeo]

I think I found my solution courtesy of this link (thanks to ogronome). Here are the steps I used:

1. Stop neo4j (if running)
2. Add dbms.recovery.fail_on_missing_files=false to the neo4j.conf file
3. sudo rm -r /var/lib/neo4j/data/databases/*
4. sudo rm -r /var/lib/neo4j/data/transactions/*
5. Perform the import command
6. Start neo4j

I removed all the directories under /neo4j/data/databases because I need to remove the default "neo4j" database files as part of the import process. These additional steps allowed me to import the data into neo4j 4.1.3.

Another somewhat related issue: although the Community Edition allows you to import data into a new database (using the --database=<new database> option in the neo4j-admin import command), you cannot use a different database. In order to switch to the new database you must execute the CREATE DATABASE <new database> command in cypher. The CREATE DATABASE command is not available in the Community Edition. See this link for more details.

[gene_mishchenko]

@cborromeo, does that mean that you remove the data and the transaction sub-directories for the system database as well?

I did that initially, as your solution suggests, because the system database data and transactions sub-directories are co-located within the databases and the transactions directories with others, but this solution required setting the neo4j user password after each instance start.

So I decided to remove not all the databases and transactions sub-directories, but instead just the ones for the database being re-created using neo4j-admin import. I was already deleting the databases sub-directory for my DB, so the only additional step was to remove the transactions sub-dir for my DB.
It worked.