Neo4j

mayassia · ‎01-01-2023

Hi everyone, first I wish you all hapyy new year. I have an old version of neo4j database which is 3.5 with the name graph.db. I want to copy data on version 4.4.11 using neo4j-admin but the problem that it does not work. I also tried to copy the folder directly into data folder and change the name of the default database in conf but it does not work

the command I used is: neo4j-admin copy --from-database=graph.db --to-database=neo4j
where graph.db is located in bin folder

dana_canzano · ‎01-01-2023

>> I want to copy data on version 4.4.11 using neo4j-admin but the problem that it does not work.

how so? it fails? it hangs? it gives error?

>> also tried to copy the folder directly into data folder and change the name of the default database in conf but it does not work

this is not a supported method

mayassia · ‎01-02-2023

I tried this command:

`neo4j-admin restore --from=/import/olddatabase --database=neo4j --force`

but the database does not start. Here is a part of neo4j file:

2023-01-02 08:41:00.024+0000 INFO Starting...
2023-01-02 08:41:05.619+0000 INFO ======== Neo4j 4.1.0 ========
2023-01-02 08:41:09.320+0000 INFO [neo4j] Starting upgrade of database
2023-01-02 08:41:12.489+0000 INFO [neo4j] Migrating Store files (1/6):
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ff8f8d60d93, pid=25904, tid=28584
#
# JRE version: OpenJDK Runtime Environment Zulu11.54+25-CA (11.0.14.1+1) (build 11.0.14.1+1-LTS)
# Java VM: OpenJDK 64-Bit Server VM Zulu11.54+25-CA (11.0.14.1+1-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# V [jvm.dll+0x740d93][thread 7996 also had an error]
[thread 20128 also had an error]
[thread 28008 also had an error]

#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\hs_err_pid25904.log
#
# Compiler replay data is saved as:
# C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\replay_pid25904.log
#
# If you would like to submit a bug report, please visit:
# http://www.azul.com/support/
#

Here is a part of my debug file, may be the problem due to missing indices:

2023-01-02 08:41:08.524+0000 INFO [o.n.i.d.DiagnosticsManager] --------------------------------------------------------------------------------
2023-01-02 08:41:08.525+0000 INFO [o.n.i.d.DiagnosticsManager] [ Transaction log ]
2023-01-02 08:41:08.525+0000 INFO [o.n.i.d.DiagnosticsManager] --------------------------------------------------------------------------------
2023-01-02 08:41:08.528+0000 INFO [o.n.i.d.DiagnosticsManager] Transaction log files stored on file store: NTFS
2023-01-02 08:41:08.537+0000 INFO [o.n.i.d.DiagnosticsManager] Oldest transaction 2 found in log with version 0

2023-01-02 08:41:08.577+0000 INFO [o.n.i.d.DiagnosticsManager] [ Store versions ]
2023-01-02 08:41:08.578+0000 INFO [o.n.i.d.DiagnosticsManager] --------------------------------------------------------------------------------
2023-01-02 08:41:08.578+0000 INFO [o.n.i.d.DiagnosticsManager] ArrayPropertyStore[neostore.nodestore.db.labels] SF4.0.0
2023-01-02 08:41:08.578+0000 INFO [o.n.i.d.DiagnosticsManager] NodeStore[neostore.nodestore.db] SF4.0.0
2023-01-02 08:41:08.579+0000 INFO [o.n.i.d.DiagnosticsManager] StringPropertyStore[neostore.propertystore.db.index.keys] SF4.0.0
2023-01-02 08:41:08.579+0000 INFO [o.n.i.d.DiagnosticsManager] PropertyIndexStore[neostore.propertystore.db.index] SF4.0.0
2023-01-02 08:41:08.579+0000 INFO [o.n.i.d.DiagnosticsManager] StringPropertyStore[neostore.propertystore.db.strings] SF4.0.0
2023-01-02 08:41:08.580+0000 INFO [o.n.i.d.DiagnosticsManager] ArrayPropertyStore[neostore.propertystore.db.arrays] SF4.0.0
2023-01-02 08:41:08.580+0000 INFO [o.n.i.d.DiagnosticsManager] PropertyStore[neostore.propertystore.db] SF4.0.0
2023-01-02 08:41:08.581+0000 INFO [o.n.i.d.DiagnosticsManager] RelationshipStore[neostore.relationshipstore.db] SF4.0.0
2023-01-02 08:41:08.581+0000 INFO [o.n.i.d.DiagnosticsManager] StringPropertyStore[neostore.relationshiptypestore.db.names] SF4.0.0
2023-01-02 08:41:08.581+0000 INFO [o.n.i.d.DiagnosticsManager] RelationshipTypeStore[neostore.relationshiptypestore.db] SF4.0.0
2023-01-02 08:41:08.582+0000 INFO [o.n.i.d.DiagnosticsManager] StringPropertyStore[neostore.labeltokenstore.db.names] SF4.0.0
2023-01-02 08:41:08.582+0000 INFO [o.n.i.d.DiagnosticsManager] LabelTokenStore[neostore.labeltokenstore.db] SF4.0.0
2023-01-02 08:41:08.582+0000 INFO [o.n.i.d.DiagnosticsManager] SchemaStore[neostore.schemastore.db] SF4.0.0
2023-01-02 08:41:08.583+0000 INFO [o.n.i.d.DiagnosticsManager] RelationshipGroupStore[neostore.relationshipgroupstore.db] SF4.0.0
2023-01-02 08:41:08.584+0000 INFO [o.n.i.d.DiagnosticsManager] NeoStore[neostore] SF4.0.0
2023-01-02 08:41:08.584+0000 INFO [o.n.i.d.DiagnosticsManager]
2023-01-02 08:41:08.676+0000 INFO [o.n.i.i.l.LabelScanStore] [system] Label index cleanup job registered
2023-01-02 08:41:08.677+0000 INFO [o.n.i.i.l.LabelScanStore] [system] Label index cleanup job started
2023-01-02 08:41:08.685+0000 INFO [o.n.i.i.l.LabelScanStore] [system] Label index cleanup job finished: Number of pages visited: 3, Number of tree nodes: 2, Number of cleaned crashed pointers: 0, Time spent: 1ms
2023-01-02 08:41:08.686+0000 INFO [o.n.i.i.l.LabelScanStore] [system] Label index cleanup job closed
2023-01-02 08:41:08.793+0000 INFO [o.n.k.i.i.s.GenericNativeIndexProvider] [system] Schema index cleanup job registered: descriptor=org.neo4j.internal.schema.IndexDescriptor@1, indexFile=C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\data\databases\system\schema\index\native-btree-1.0\1\index-1
2023-01-02 08:41:08.793+0000 INFO [o.n.k.i.i.s.GenericNativeIndexProvider] [system] Schema index cleanup job started: descriptor=org.neo4j.internal.schema.IndexDescriptor@1, indexFile=C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\data\databases\system\schema\index\native-btree-1.0\1\index-1
2023-01-02 08:41:08.797+0000 INFO [o.n.k.i.i.s.GenericNativeIndexProvider] [system] Schema index cleanup job finished: descriptor=org.neo4j.internal.schema.IndexDescriptor@1, indexFile=C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\data\databases\system\schema\index\native-btree-1.0\1\index-1 Number of pages visited: 3, Number of tree nodes: 2, Number of cleaned crashed pointers: 0, Time spent: 2ms
2023-01-02 08:41:08.798+0000 INFO [o.n.k.i.i.s.GenericNativeIndexProvider] [system] Schema index cleanup job closed: descriptor=org.neo4j.internal.schema.IndexDescriptor@1, indexFile=C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\data\databases\system\schema\index\native-btree-1.0\1\index-1
2023-01-02 08:41:08.806+0000 INFO [o.n.k.i.a.i.IndexingService] [system] IndexingService.init: indexes not specifically mentioned above are ONLINE
2023-01-02 08:41:08.821+0000 INFO [o.n.k.a.DatabaseAvailabilityGuard] [system] Requirement `Database unavailable` makes database system unavailable.
2023-01-02 08:41:08.823+0000 INFO [o.n.k.a.DatabaseAvailabilityGuard] [system] Database system is unavailable.
2023-01-02 08:41:08.869+0000 INFO [o.n.k.i.a.i.IndexingService] [system] IndexingService.start: indexes not specifically mentioned above are ONLINE
2023-01-02 08:41:09.008+0000 INFO [o.n.k.a.DatabaseAvailabilityGuard] [system] Fulfilling of requirement 'Database unavailable' makes database system available.
2023-01-02 08:41:09.009+0000 INFO [o.n.k.a.DatabaseAvailabilityGuard] [system] Database system is ready.
2023-01-02 08:41:09.011+0000 INFO [c.n.k.i.p.PageCacheWarmer] [system] Page cache warmup started.
2023-01-02 08:41:09.012+0000 INFO [o.n.m.DatabaseHealth] [system] Database health set to OK
2023-01-02 08:41:09.230+0000 INFO [c.n.d.DbmsReconciler] Database neo4j is requested to transition from EnterpriseDatabaseState{databaseId=DatabaseId{ad50ea0d[neo4j]}, operatorState=INITIAL, failed=false} to EnterpriseDatabaseState{databaseId=DatabaseId{ad50ea0d[neo4j]}, operatorState=STARTED, failed=false}
2023-01-02 08:41:09.231+0000 INFO [c.n.d.d.EnterpriseMultiDatabaseManager] Creating 'neo4j' database.
2023-01-02 08:41:09.235+0000 INFO [o.n.k.i.f.StatementLocksFactorySelector] No services implementing StatementLocksFactory found. Using SimpleStatementLocksFactory
2023-01-02 08:41:09.241+0000 INFO [c.n.d.d.EnterpriseMultiDatabaseManager] Starting 'neo4j' database.
2023-01-02 08:41:09.261+0000 INFO [o.n.k.i.s.f.RecordFormatSelector] [neo4j] Selected RecordFormat:StandardV3_4[v0.A.9] record format from store C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\data\databases\neo4j
2023-01-02 08:41:09.264+0000 INFO [o.n.k.i.s.f.RecordFormatSelector] [neo4j] Selected format 'RecordFormat:StandardV4_0[SF4.0.0]' for existing store C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\data\databases\neo4j with format 'RecordFormat:StandardV3_4[v0.A.9]'
2023-01-02 08:41:09.321+0000 INFO [o.n.k.i.s.DatabaseMigrator] [neo4j] Starting upgrade of database
2023-01-02 08:41:09.331+0000 INFO [c.n.k.i.p.PageCacheWarmer] [system] Page cache warmup completed. 113 pages loaded. Duration: 320ms.
2023-01-02 08:41:12.490+0000 INFO [o.n.k.i.s.DatabaseMigrator] [neo4j] Migrating Store files (1/6):

dana_canzano · ‎01-02-2023

@mayassia

your initial report indicates

>the command I used is: neo4j-admin copy --from-database=graph.db --to-database=neo4j

but your last date reports

>neo4j-admin restore --from=/import/olddatabase --database=neo4j --force`

also the stack reports

>>2023-01-02 08:41:05.619+0000 INFO ======== Neo4j 4.1.0 ========
>>2023-01-02 08:41:09.320+0000 INFO [neo4j] Starting upgrade of database

indicating a start on 4.1.0 whereas I thought per the post description `Migration from 3.5 to 4.4.11 and so was expecting a start on 4.4.x and not 4.1.0

Also the stack reports

>># An error report file with more information is saved as:
# C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\hs_err_pid25904.log
#
# Compiler replay data is saved as:
# C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\replay_pid25904.log
#

anything meaningful in either .log.

mayassia · ‎01-02-2023

Thanks for your reply. In fact, I changed the version 4.4.11 to the earliest version in neo4j desktop which is 4.1.0 to test if it works.

I found this:

#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ff8f8d60d93, pid=25904, tid=28584
#
# JRE version: OpenJDK Runtime Environment Zulu11.54+25-CA (11.0.14.1+1) (build 11.0.14.1+1-LTS)
# Java VM: OpenJDK 64-Bit Server VM Zulu11.54+25-CA (11.0.14.1+1-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# V [jvm.dll+0x740d93]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
# http://www.azul.com/support/
#

--------------- S U M M A R Y ------------

Command Line: -Xms512m -Xmx1G -XX:+UseG1GC -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:+UnlockExperimentalVMOptions -XX:+TrustFinalNonStaticFields -XX:+DisableExplicitGC -XX:MaxInlineLevel=15 -Djdk.nio.maxCachedBufferSize=262144 -Dio.netty.tryReflectionSetAccessible=true -Djdk.tls.ephemeralDHKeySize=2048 -Djdk.tls.rejectClientInitiatedRenegotiation=true -XX:FlightRecorderOptions=stackdepth=256 -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -Dfile.encoding=UTF-8 com.neo4j.server.enterprise.EnterpriseEntryPoint --config-dir=C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee\conf --home-dir=C:\Users\USER\.Neo4jDesktop\relate-data\dbmss\dbms-c03fffba-cd61-4e25-b471-d74041b559ee

Host: Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz, 8 cores, 15G, Windows 10 , 64 bit Build 19041 (10.0.19041.2364)
Time: Mon Jan 2 09:41:16 2023 Maroc (heure d’été) elapsed time: 18.171812 seconds (0d 0h 0m 18s)

--------------- T H R E A D ---------------

Current thread (0x000002981dbe9800): JavaThread "C2 CompilerThread0" daemon [_thread_in_vm, id=28584, stack(0x000000e3dc500000,0x000000e3dc600000)]

Current CompileTask:
C2: 18171 4530 4 org.neo4j.io.pagecache.impl.muninn.PageList::fault (119 bytes)

mayassia · ‎01-02-2023

I forgot to mention that the database was in Linux. Actually, I'm using windows.

ameyasoft · ‎01-02-2023

This worked for me:
Step 1: Take a backup of 3.5 database:
./neo4j-admin backup --backup-dir=/tmp/3.5.24 --name=graphdbbackup

Step 2: On 4.x version create a folder 'dumps' under 'data' folder. Copy graphdbbackup file and paste it in the 'dumps' folder.

Step 3: Run this: ./neo4j-admin copy --from-path=/dumps/graphdbbackup --to-database=graphdbnew

Step 4: Once it's successful start your db in windows version. Select 'system' database from the drop down list in the left pane. Run CREATE DATABASE graphdbnew

Last Step: Run SHOW DATABASES and you should see the newly create db name and the status column should show 'online'. Finally you can select the newly created database in the dropdown list in the left pane.

These steps worked for me. Here is a link: https://neo4j.com/docs/upgrade-migration-guide/current/version-4/tutorials/online-backup-copy-database/