Location where Neo4j keeps the index data

[alberto_rubio]

Hello! 🙂

I'm doing tests with the version 3.5.0 and I've noticed that the index information is not stored in the same directory that I specify as graph.db.

I have seen that the directory of the logs of the transactions can be changed: https://neo4j.com/docs/operations-manual/3.5-preview/reference/configuration-settings/#config_dbms.d... but I don't find a property for the index information, has something escaped me?

I don't want to have the information of the database inside my neo4j.server directory and now the information of the indexes is stored there.

Can you help me?

Thanks, Alberto.

[david_allen]

Indexes should be present under graph.db/schema/index, like this:

$ ls -l graph.db/schema/index/native-btree-1.0/
total 0
drwxr-xr-x  4 davidallen  staff  128 Oct 29 09:47 1
drwxr-xr-x  4 davidallen  staff  128 Oct 29 09:47 3

The "native-btree-1.0" is the index provider/backend. So you may have multiple different subdirectories there, depending on how the index is implemented. Within that subdirectory, you end up with 1 directory per index. Underneath of those are the actual data files that back the index.

As far as I'm aware, you can't change the location of the index storage. Since it's in a subdirectory underneath of graph.db, I think it pretty much is stored in the graph.db folder, albeit not side-by-side in the same directory. This could get pretty messy over time, for example if you had a large number of indexes.

[alberto_rubio]

Hi David,

Thank you for answering me so quickly

Yep, you're right, my problem is with the unsupported.dbms.directories.database property.

I put in my neo4j.conf file the line: unsupported.dbms.directories.database=/tmp/graph.db for specify the databases folder (https://neo4j.com/developer/kb/manually-migrating-configuration-settings-from-neo4j-2x-to-neo4j-3x/)

After, When I start neo4j, it ignores this property and create the database in the data folder:

/home/admin/neo4j-enterprise-3.5.0-beta03/data/:
databases  dbms

/home/admin/neo4j-enterprise-3.5.0-beta03/data/databases:
graph.db  store_lock

/home/admin/neo4j-enterprise-3.5.0-beta03/data/databases/graph.db:
index                        neostore.labeltokenstore.db.id        neostore.nodestore.db.labels.id      neostore.propertystore.db.index.id       neostore.relationshipgroupstore.db.id    neostore.relationshiptypestore.db.names.id
neostore                     neostore.labeltokenstore.db.names     neostore.propertystore.db            neostore.propertystore.db.index.keys     neostore.relationshipstore.db            neostore.schemastore.db
neostore.counts.db.a         neostore.labeltokenstore.db.names.id  neostore.propertystore.db.arrays     neostore.propertystore.db.index.keys.id  neostore.relationshipstore.db.id         neostore.schemastore.db.id
neostore.id                  neostore.nodestore.db                 neostore.propertystore.db.arrays.id  neostore.propertystore.db.strings        neostore.relationshiptypestore.db        neostore.transaction.db.0
neostore.labelscanstore.db   neostore.nodestore.db.id              neostore.propertystore.db.id         neostore.propertystore.db.strings.id     neostore.relationshiptypestore.db.id     profiles
neostore.labeltokenstore.db  neostore.nodestore.db.labels          neostore.propertystore.db.index      neostore.relationshipgroupstore.db       neostore.relationshiptypestore.db.names

/home/admin/neo4j-enterprise-3.5.0-beta03/data/databases/graph.db/index:

/home/admin/neo4j-enterprise-3.5.0-beta03/data/databases/graph.db/profiles:
neostore.3.cacheprof                     neostore.labeltokenstore.db.names.3.cacheprof  neostore.propertystore.db.arrays.3.cacheprof      neostore.relationshipgroupstore.db.3.cacheprof       neostore.schemastore.db.3.cacheprof
neostore.counts.db.a.3.cacheprof         neostore.nodestore.db.3.cacheprof              neostore.propertystore.db.index.3.cacheprof       neostore.relationshipstore.db.3.cacheprof
neostore.labelscanstore.db.3.cacheprof   neostore.nodestore.db.labels.3.cacheprof       neostore.propertystore.db.index.keys.3.cacheprof  neostore.relationshiptypestore.db.3.cacheprof
neostore.labeltokenstore.db.3.cacheprof  neostore.propertystore.db.3.cacheprof          neostore.propertystore.db.strings.3.cacheprof     neostore.relationshiptypestore.db.names.3.cacheprof

/home/admin/neo4j-enterprise-3.5.0-beta03/data/dbms:
auth  roles

In the /tmp/graph.db folder, Neo4J creates only transactions:

/tmp/graph.db/:
neostore.transaction.db.0

My question is, how could I have all the files used by neo4j in the folder /tmp/graph.db?

Thanks, Alberto.

[david_allen]

I'm not sure if this changed in 3.5, I'll ask around and see what I can find. One thing I'm likely to hear though is that since it has the prefix "unsupported", between v2 and v3 (in the docs you cite) a number of legacy configuration parameters were renamed "unsupported". This is typically done when Neo4j doesn't provide support any more and/or has a plan to deprecate or remove the configuration field. So it wouldn't surprise me if it had been removed.

Now, on "what can you do about it" - can you describe a bit more about what you're trying to accomplish? You want the files in /tmp. Would a symlink suffice, that would get the job done. Or do you want them in /tmp because you want them to be regularly deleted and you want a "transient database"? Or is it for some other reason?

In the absence of this configuration parameter, you could install a copy of neo4j with a root directory under /tmp, and in this way you'd get the property you want without configuration. Or the symlink too. It depends no whether your'e trying to host the data on that specific device for IO reasons, or what.

[alberto_rubio]

Hi David, how are you?!

Did you find out if Neo4J had erased this property? I was just waiting to know what to do.

I used the directory /tmp to simplify the example, I use this parameter to save the db directory on a different device from the root of the machine. On the other hand, using this property I can use the same configuration in different dbs with only changing the property value.

Thanks!

[Benoit]

Hi,

Why are you not using the config dbms.directories.data to set where your want to store the data folder of neo4j ?

Cheers.

[alberto_rubio]

Hi Benoit!

Thanks for reply!

I didn't want to use it because according to the documentation it indicates where you want to put the folder databases (folder that I don't want to have either).

Best, Alberto.

[Benoit]

Hi Alberto,

But this folder is needed by Neo4j and contains all the data of the service and the configuration is supported.

For the database configuration, like you see in your config file, it's an unsupported configuration, and so it can disappears and changes in time.

But if you really want it, can you test this : unsupported.dbms.directories.databases.root=/tmp/graph.db

[alberto_rubio]

Hi Benoit,

It was just what I wanted.

Many thanks,
Alberto.