Memory Allocation for Queries not Populating

[mike_blum_neo4j]

It appears in 3.4.4` that enabling # Uncomment this line to include bytes allocated by the executed queries being logged: dbms.logs.query.allocation_logging_enabled=true

When reviewing the query.log the memory allocation for a query doesn't populate:

I have all the other query profiling properties enabled and here is the output:

2018-09-22 00:46:36.676+0000 INFO  87 ms: (planning: 1, waiting: 0) - bolt-session	bolt	neo4j	neo4j-javascript/1.6.1		client/0:0:0:0:0:0:0:1:55269	server/0:0:0:0:0:0:0:1:7687>	neo4j - CALL dbms.queryJmx("org.neo4j:*") - {} - {}

[tom_geudens]

Hello Mike,

You are right, I just tested with the same configuration on 3.3.6 and 3.4.7. It's definitely there in 3.3.x, it's not there in 3.4.x. I've pinged our specialists on the subject and I'll get back to you with an update as soon as I get feedback !

Regards,
Tom

[tom_geudens · Accepted solution]

Hello Mike,

Problem solved. Version 3.4.1 introduced two new parameters that - for performance reasons - by default turn off all cpu and memory allocation tracking of queries. You need to turn these on as well :

dbms.track_query_cpu_time=true
dbms.track_query_allocation=true

Add those and you'll get the results in the query.log again.

Regards,
Tom

[mike_blum_neo4j]

Great! I'm trying to resolve some performance issues in production. Is it advisable to have those flags enabled in production long term?

[tom_geudens]

Hello Mike. As you may deduce ... no. Nor is having query log itself enabled. It (and those settings) are exactly for what you are doing, resolving (production) performance issues, they do cause overhead and should be turned off after the analysis.

Regards,
Tom

[mike_blum_neo4j]

Oh so we should have the query log disabled entirely in production? Right now we have both the query log running along with on the application server we're logging the Bolt queries. What is the best practice for tracking queries in production long term?

[tom_geudens]

Hello Mike,

Now, opinions may differ on this but this is how I understand and see it. The query.log is a tool for getting information about queries that take longer than expected or ... to find such queries. Preferably you do this on a pre-production environment but obviously there may be cases where you can only do that on production. For a limited time.

It should not be used for audit purposes. Several reasons :

1. It was not designed for that, the impact is much bigger than an audit tool should have. While this is obviously relative (read: you probably hardly notice it in most situations) ... it still holds true.
2. It does not have all the information. An audit tool has the who, when, what information, query.log does not.

If you are interested in full audit, ping me and I'll explain your options there (as that would be off-topic in this thread).

Regards,
Tom