How to monitor Neo4j with Prometheus

[dana_canzano]

Commencing with the release of Neo4j 3.4, one can now use the open source monitoring tool Prometheus to monitor Neo4j.
The following article details a basic Prometheus implementation to demonstrate the connectivity between Prometheus v2.2.1 and Neo4j.

After installing Prometheus, Neo4j needs to be configured via its neo4j.conf file with the following parameters:

# Enable the Prometheus endpoint. Default is 'false'.
metrics.prometheus.enabled=true
# The IP and port the endpoint will bind to in the format <hostname or IP address>:<port number>.
# The default is localhost:2004.
metrics.prometheus.endpoint=localhost:2004

and the prometheus.yml file should be configured as follows:

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'Neo4j-prometheus'

    # metrics_path: /metrics
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['localhost:2004']

In the above configuration snippets, Prometheus has been installed on the same server as Neo4j. If you choose to install it on a different server, simply replace references to localhost above with the IP address of the Neo4j instance.

Launching the Prometheus browser at http://:9090 and clicking the menu option Status -> Targets should display:

Returning back to the 3rd menu choice of Graphs one can define a graph to monitor a Neo4j metric. For example, in the detail below the
graph represents the number of transactions started (neo4j_transaction_started):

[mike_blum_neo4j]

Given the available prometheus metrics (below). What are some good metrics to be alerting on from an operations standpoint? For example - getting an estimate for how long queries are taking to plan and execute or JVM memory utilization.

# TYPE neo4j_bolt_accumulated_processing_time gauge
neo4j_bolt_accumulated_processing_time 0.0
# HELP neo4j_bolt_accumulated_queue_time Generated from Dropwizard metric import (metric=neo4j.bolt.accumulated_queue_time, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$462/581364120)
# TYPE neo4j_bolt_accumulated_queue_time gauge
neo4j_bolt_accumulated_queue_time 0.0
# HELP neo4j_bolt_connections_closed Generated from Dropwizard metric import (metric=neo4j.bolt.connections_closed, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$455/939829672)
# TYPE neo4j_bolt_connections_closed gauge
neo4j_bolt_connections_closed 0.0
# HELP neo4j_bolt_connections_idle Generated from Dropwizard metric import (metric=neo4j.bolt.connections_idle, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$457/1601800698)
# TYPE neo4j_bolt_connections_idle gauge
neo4j_bolt_connections_idle 0.0
# HELP neo4j_bolt_connections_opened Generated from Dropwizard metric import (metric=neo4j.bolt.connections_opened, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$454/1684802151)
# TYPE neo4j_bolt_connections_opened gauge
neo4j_bolt_connections_opened 0.0
# HELP neo4j_bolt_connections_running Generated from Dropwizard metric import (metric=neo4j.bolt.connections_running, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$456/292890110)
# TYPE neo4j_bolt_connections_running gauge
neo4j_bolt_connections_running 0.0
# HELP neo4j_bolt_messages_done Generated from Dropwizard metric import (metric=neo4j.bolt.messages_done, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$460/2121573248)
# TYPE neo4j_bolt_messages_done gauge
neo4j_bolt_messages_done 0.0
# HELP neo4j_bolt_messages_failed Generated from Dropwizard metric import (metric=neo4j.bolt.messages_failed, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$461/1842463725)
# TYPE neo4j_bolt_messages_failed gauge
neo4j_bolt_messages_failed 0.0
# HELP neo4j_bolt_messages_received Generated from Dropwizard metric import (metric=neo4j.bolt.messages_received, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$458/1539477312)
# TYPE neo4j_bolt_messages_received gauge
neo4j_bolt_messages_received 0.0
# HELP neo4j_bolt_messages_started Generated from Dropwizard metric import (metric=neo4j.bolt.messages_started, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$459/1184248953)
# TYPE neo4j_bolt_messages_started gauge
neo4j_bolt_messages_started 0.0
# HELP neo4j_bolt_sessions_started Generated from Dropwizard metric import (metric=neo4j.bolt.sessions_started, type=org.neo4j.metrics.source.db.BoltMetrics$$Lambda$453/1755634886)
# TYPE neo4j_bolt_sessions_started gauge
neo4j_bolt_sessions_started 0.0
# HELP neo4j_check_point_events Generated from Dropwizard metric import (metric=neo4j.check_point.events, type=org.neo4j.metrics.source.db.CheckPointingMetrics$$Lambda$436/623446986)
# TYPE neo4j_check_point_events gauge
neo4j_check_point_events 0.0
# HELP neo4j_check_point_total_time Generated from Dropwizard metric import (metric=neo4j.check_point.total_time, type=org.neo4j.metrics.source.db.CheckPointingMetrics$$Lambda$437/1985363238)
# TYPE neo4j_check_point_total_time gauge
neo4j_check_point_total_time 0.0
# HELP neo4j_cypher_replan_events Generated from Dropwizard metric import (metric=neo4j.cypher.replan_events, type=org.neo4j.metrics.source.db.CypherMetrics$$Lambda$447/1535454025)
# TYPE neo4j_cypher_replan_events gauge
neo4j_cypher_replan_events 0.0
# HELP neo4j_cypher_replan_wait_time Generated from Dropwizard metric import (metric=neo4j.cypher.replan_wait_time, type=org.neo4j.metrics.source.db.CypherMetrics$$Lambda$448/215690645)
# TYPE neo4j_cypher_replan_wait_time gauge
neo4j_cypher_replan_wait_time 0.0
# HELP neo4j_ids_in_use_node Generated from Dropwizard metric import (metric=neo4j.ids_in_use.node, type=org.neo4j.metrics.source.db.EntityCountMetrics$$Lambda$440/401792389)
# TYPE neo4j_ids_in_use_node gauge
neo4j_ids_in_use_node 534.0
# HELP neo4j_ids_in_use_property Generated from Dropwizard metric import (metric=neo4j.ids_in_use.property, type=org.neo4j.metrics.source.db.EntityCountMetrics$$Lambda$442/553846997)
# TYPE neo4j_ids_in_use_property gauge
neo4j_ids_in_use_property 1657.0
# HELP neo4j_ids_in_use_relationship Generated from Dropwizard metric import (metric=neo4j.ids_in_use.relationship, type=org.neo4j.metrics.source.db.EntityCountMetrics$$Lambda$441/509104428)
# TYPE neo4j_ids_in_use_relationship gauge
neo4j_ids_in_use_relationship 590.0
# HELP neo4j_ids_in_use_relationship_type Generated from Dropwizard metric import (metric=neo4j.ids_in_use.relationship_type, type=org.neo4j.metrics.source.db.EntityCountMetrics$$Lambda$443/581383895)
# TYPE neo4j_ids_in_use_relationship_type gauge
neo4j_ids_in_use_relationship_type 33.0
# HELP neo4j_log_rotation_events Generated from Dropwizard metric import (metric=neo4j.log_rotation.events, type=org.neo4j.metrics.source.db.LogRotationMetrics$$Lambda$438/1032867484)
# TYPE neo4j_log_rotation_events gauge
neo4j_log_rotation_events 0.0
# HELP neo4j_log_rotation_total_time Generated from Dropwizard metric import (metric=neo4j.log_rotation.total_time, type=org.neo4j.metrics.source.db.LogRotationMetrics$$Lambda$439/1351247295)
# TYPE neo4j_log_rotation_total_time gauge
neo4j_log_rotation_total_time 0.0
# HELP neo4j_network_master_network_store_writes Generated from Dropwizard metric import (metric=neo4j.network.master_network_store_writes, type=org.neo4j.metrics.source.cluster.NetworkMetrics$$Lambda$445/2047770317)
# TYPE neo4j_network_master_network_store_writes gauge
neo4j_network_master_network_store_writes 0.0
# HELP neo4j_network_master_network_tx_writes Generated from Dropwizard metric import (metric=neo4j.network.master_network_tx_writes, type=org.neo4j.metrics.source.cluster.NetworkMetrics$$Lambda$444/1198515856)
# TYPE neo4j_network_master_network_tx_writes gauge
neo4j_network_master_network_tx_writes 0.0
# HELP neo4j_network_slave_network_tx_writes Generated from Dropwizard metric import (metric=neo4j.network.slave_network_tx_writes, type=org.neo4j.metrics.source.cluster.NetworkMetrics$$Lambda$446/1424188812)
# TYPE neo4j_network_slave_network_tx_writes gauge
neo4j_network_slave_network_tx_writes 0.0
# HELP neo4j_page_cache_eviction_exceptions Generated from Dropwizard metric import (metric=neo4j.page_cache.eviction_exceptions, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$433/1249070084)
# TYPE neo4j_page_cache_eviction_exceptions gauge
neo4j_page_cache_eviction_exceptions 0.0
# HELP neo4j_page_cache_evictions Generated from Dropwizard metric import (metric=neo4j.page_cache.evictions, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$428/1551350415)
# TYPE neo4j_page_cache_evictions gauge
neo4j_page_cache_evictions 0.0
# HELP neo4j_page_cache_flushes Generated from Dropwizard metric import (metric=neo4j.page_cache.flushes, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$432/2061316668)
# TYPE neo4j_page_cache_flushes gauge
neo4j_page_cache_flushes 221.0
# HELP neo4j_page_cache_hit_ratio Generated from Dropwizard metric import (metric=neo4j.page_cache.hit_ratio, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$434/639542871)
# TYPE neo4j_page_cache_hit_ratio gauge
neo4j_page_cache_hit_ratio 0.7703462562888429
# HELP neo4j_page_cache_hits Generated from Dropwizard metric import (metric=neo4j.page_cache.hits, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$431/455036946)
# TYPE neo4j_page_cache_hits gauge
neo4j_page_cache_hits 5206.0
# HELP neo4j_page_cache_page_faults Generated from Dropwizard metric import (metric=neo4j.page_cache.page_faults, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$427/576222471)
# TYPE neo4j_page_cache_page_faults gauge
neo4j_page_cache_page_faults 1552.0
# HELP neo4j_page_cache_pins Generated from Dropwizard metric import (metric=neo4j.page_cache.pins, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$429/6688524)
# TYPE neo4j_page_cache_pins gauge
neo4j_page_cache_pins 6758.0
# HELP neo4j_page_cache_unpins Generated from Dropwizard metric import (metric=neo4j.page_cache.unpins, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$430/2047799833)
# TYPE neo4j_page_cache_unpins gauge
neo4j_page_cache_unpins 6537.0
# HELP neo4j_page_cache_usage_ratio Generated from Dropwizard metric import (metric=neo4j.page_cache.usage_ratio, type=org.neo4j.metrics.source.db.PageCacheMetrics$$Lambda$435/901852282)
# TYPE neo4j_page_cache_usage_ratio gauge
neo4j_page_cache_usage_ratio 0.011887163854443517
# HELP neo4j_server_threads_jetty_all Generated from Dropwizard metric import (metric=neo4j.server.threads.jetty.all, type=org.neo4j.metrics.source.server.ServerMetrics$$Lambda$469/1444635117)
# TYPE neo4j_server_threads_jetty_all gauge
neo4j_server_threads_jetty_all 8.0
# HELP neo4j_server_threads_jetty_idle Generated from Dropwizard metric import (metric=neo4j.server.threads.jetty.idle, type=org.neo4j.metrics.source.server.ServerMetrics$$Lambda$468/79094208)
# TYPE neo4j_server_threads_jetty_idle gauge
neo4j_server_threads_jetty_idle 2.0
# HELP neo4j_transaction_active Generated from Dropwizard metric import (metric=neo4j.transaction.active, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$413/1778994610)
# TYPE neo4j_transaction_active gauge
neo4j_transaction_active 0.0
# HELP neo4j_transaction_active_read Generated from Dropwizard metric import (metric=neo4j.transaction.active_read, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$414/1706939736)
# TYPE neo4j_transaction_active_read gauge
neo4j_transaction_active_read 0.0
# HELP neo4j_transaction_active_write Generated from Dropwizard metric import (metric=neo4j.transaction.active_write, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$415/498748934)
# TYPE neo4j_transaction_active_write gauge
neo4j_transaction_active_write 0.0
# HELP neo4j_transaction_committed Generated from Dropwizard metric import (metric=neo4j.transaction.committed, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$416/1273689789)
# TYPE neo4j_transaction_committed gauge
neo4j_transaction_committed 2.0
# HELP neo4j_transaction_committed_read Generated from Dropwizard metric import (metric=neo4j.transaction.committed_read, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$417/1798194863)
# TYPE neo4j_transaction_committed_read gauge
neo4j_transaction_committed_read 2.0
# HELP neo4j_transaction_committed_write Generated from Dropwizard metric import (metric=neo4j.transaction.committed_write, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$418/1613912455)
# TYPE neo4j_transaction_committed_write gauge
neo4j_transaction_committed_write 0.0
# HELP neo4j_transaction_last_closed_tx_id Generated from Dropwizard metric import (metric=neo4j.transaction.last_closed_tx_id, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$426/342740966)
# TYPE neo4j_transaction_last_closed_tx_id gauge
neo4j_transaction_last_closed_tx_id 771.0
# HELP neo4j_transaction_last_committed_tx_id Generated from Dropwizard metric import (metric=neo4j.transaction.last_committed_tx_id, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$425/1221759982)
# TYPE neo4j_transaction_last_committed_tx_id gauge
neo4j_transaction_last_committed_tx_id 771.0
# HELP neo4j_transaction_peak_concurrent Generated from Dropwizard metric import (metric=neo4j.transaction.peak_concurrent, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$412/262878641)
# TYPE neo4j_transaction_peak_concurrent gauge
neo4j_transaction_peak_concurrent 1.0
# HELP neo4j_transaction_rollbacks Generated from Dropwizard metric import (metric=neo4j.transaction.rollbacks, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$419/792195577)
# TYPE neo4j_transaction_rollbacks gauge
neo4j_transaction_rollbacks 0.0
# HELP neo4j_transaction_rollbacks_read Generated from Dropwizard metric import (metric=neo4j.transaction.rollbacks_read, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$420/863286716)
# TYPE neo4j_transaction_rollbacks_read gauge
neo4j_transaction_rollbacks_read 0.0
# HELP neo4j_transaction_rollbacks_write Generated from Dropwizard metric import (metric=neo4j.transaction.rollbacks_write, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$421/720803395)
# TYPE neo4j_transaction_rollbacks_write gauge
neo4j_transaction_rollbacks_write 0.0
# HELP neo4j_transaction_started Generated from Dropwizard metric import (metric=neo4j.transaction.started, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$411/662433937)
# TYPE neo4j_transaction_started gauge
neo4j_transaction_started 2.0
# HELP neo4j_transaction_terminated Generated from Dropwizard metric import (metric=neo4j.transaction.terminated, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$422/1484397819)
# TYPE neo4j_transaction_terminated gauge
neo4j_transaction_terminated 0.0
# HELP neo4j_transaction_terminated_read Generated from Dropwizard metric import (metric=neo4j.transaction.terminated_read, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$423/1213367860)
# TYPE neo4j_transaction_terminated_read gauge
neo4j_transaction_terminated_read 0.0
# HELP neo4j_transaction_terminated_write Generated from Dropwizard metric import (metric=neo4j.transaction.terminated_write, type=org.neo4j.metrics.source.db.TransactionMetrics$$Lambda$424/1578959831)
# TYPE neo4j_transaction_terminated_write gauge
neo4j_transaction_terminated_write 0.0
# HELP vm_gc_count_g1_old_generation Generated from Dropwizard metric import (metric=vm.gc.count.g1_old_generation, type=org.neo4j.metrics.source.jvm.GCMetrics$$Lambda$450/794476802)
# TYPE vm_gc_count_g1_old_generation gauge
vm_gc_count_g1_old_generation 0.0
# HELP vm_gc_count_g1_young_generation Generated from Dropwizard metric import (metric=vm.gc.count.g1_young_generation, type=org.neo4j.metrics.source.jvm.GCMetrics$$Lambda$450/794476802)
# TYPE vm_gc_count_g1_young_generation gauge
vm_gc_count_g1_young_generation 7.0
# HELP vm_gc_time_g1_old_generation Generated from Dropwizard metric import (metric=vm.gc.time.g1_old_generation, type=org.neo4j.metrics.source.jvm.GCMetrics$$Lambda$449/304966690)
# TYPE vm_gc_time_g1_old_generation gauge
vm_gc_time_g1_old_generation 0.0
# HELP vm_gc_time_g1_young_generation Generated from Dropwizard metric import (metric=vm.gc.time.g1_young_generation, type=org.neo4j.metrics.source.jvm.GCMetrics$$Lambda$449/304966690)
# TYPE vm_gc_time_g1_young_generation gauge
vm_gc_time_g1_young_generation 159.0
# HELP vm_memory_buffer_direct_capacity Generated from Dropwizard metric import (metric=vm.memory.buffer.direct.capacity, type=org.neo4j.metrics.source.jvm.MemoryBuffersMetrics$$Lambda$467/1040172900)
# TYPE vm_memory_buffer_direct_capacity gauge
vm_memory_buffer_direct_capacity 221184.0
# HELP vm_memory_buffer_direct_count Generated from Dropwizard metric import (metric=vm.memory.buffer.direct.count, type=org.neo4j.metrics.source.jvm.MemoryBuffersMetrics$$Lambda$465/2013613908)
# TYPE vm_memory_buffer_direct_count gauge
vm_memory_buffer_direct_count 6.0
# HELP vm_memory_buffer_direct_used Generated from Dropwizard metric import (metric=vm.memory.buffer.direct.used, type=org.neo4j.metrics.source.jvm.MemoryBuffersMetrics$$Lambda$466/213383793)
# TYPE vm_memory_buffer_direct_used gauge
vm_memory_buffer_direct_used 221185.0
# HELP vm_memory_buffer_mapped_capacity Generated from Dropwizard metric import (metric=vm.memory.buffer.mapped.capacity, type=org.neo4j.metrics.source.jvm.MemoryBuffersMetrics$$Lambda$467/1040172900)
# TYPE vm_memory_buffer_mapped_capacity gauge
vm_memory_buffer_mapped_capacity 4020.0
# HELP vm_memory_buffer_mapped_count Generated from Dropwizard metric import (metric=vm.memory.buffer.mapped.count, type=org.neo4j.metrics.source.jvm.MemoryBuffersMetrics$$Lambda$465/2013613908)
# TYPE vm_memory_buffer_mapped_count gauge
vm_memory_buffer_mapped_count 3.0
# HELP vm_memory_buffer_mapped_used Generated from Dropwizard metric import (metric=vm.memory.buffer.mapped.used, type=org.neo4j.metrics.source.jvm.MemoryBuffersMetrics$$Lambda$466/213383793)
# TYPE vm_memory_buffer_mapped_used gauge
vm_memory_buffer_mapped_used 4020.0
# HELP vm_memory_pool_code_cache Generated from Dropwizard metric import (metric=vm.memory.pool.code_cache, type=org.neo4j.metrics.source.jvm.MemoryPoolMetrics$$Lambda$464/1845137754)
# TYPE vm_memory_pool_code_cache gauge
vm_memory_pool_code_cache 1.7128256E7
# HELP vm_memory_pool_compressed_class_space Generated from Dropwizard metric import (metric=vm.memory.pool.compressed_class_space, type=org.neo4j.metrics.source.jvm.MemoryPoolMetrics$$Lambda$464/1845137754)
# TYPE vm_memory_pool_compressed_class_space gauge
vm_memory_pool_compressed_class_space 1.1684392E7
# HELP vm_memory_pool_g1_eden_space Generated from Dropwizard metric import (metric=vm.memory.pool.g1_eden_space, type=org.neo4j.metrics.source.jvm.MemoryPoolMetrics$$Lambda$464/1845137754)
# TYPE vm_memory_pool_g1_eden_space gauge
vm_memory_pool_g1_eden_space 3.93216E8
# HELP vm_memory_pool_g1_old_gen Generated from Dropwizard metric import (metric=vm.memory.pool.g1_old_gen, type=org.neo4j.metrics.source.jvm.MemoryPoolMetrics$$Lambda$464/1845137754)
# TYPE vm_memory_pool_g1_old_gen gauge
vm_memory_pool_g1_old_gen 1.5728664E7
# HELP vm_memory_pool_g1_survivor_space Generated from Dropwizard metric import (metric=vm.memory.pool.g1_survivor_space, type=org.neo4j.metrics.source.jvm.MemoryPoolMetrics$$Lambda$464/1845137754)
# TYPE vm_memory_pool_g1_survivor_space gauge
vm_memory_pool_g1_survivor_space 4.5088768E7
# HELP vm_memory_pool_metaspace Generated from Dropwizard metric import (metric=vm.memory.pool.metaspace, type=org.neo4j.metrics.source.jvm.MemoryPoolMetrics$$Lambda$464/1845137754)
# TYPE vm_memory_pool_metaspace gauge
vm_memory_pool_metaspace 8.0334168E7
# HELP vm_thread_count Generated from Dropwizard metric import (metric=vm.thread.count, type=org.neo4j.metrics.source.jvm.ThreadMetrics$$Lambda$451/143999341)
# TYPE vm_thread_count gauge
vm_thread_count 40.0
# HELP vm_thread_total Generated from Dropwizard metric import (metric=vm.thread.total, type=org.neo4j.metrics.source.jvm.ThreadMetrics$$Lambda$452/1906335777)
# TYPE vm_thread_total gauge
vm_thread_total 43.0

[ben_butler-cole]

Hello, Mike.

Page cache faults and GC time would be a good place to start.

-Ben

[toby]

Hi @ben.butler-cole
Thanks for this, I'm also setting up Prometheus monitoring for our Neo4j servers.

I'm confused about the choice of metric type you expose. Everything is exposed as a guage. For some metrics this makes sense, but particularly with the neo4j_bolt_messages* and neo4j_connections*, I believe these would be much better exposed as a counter.

Gauges can go up as well as down, however the count of messages done for instance will only ever increase.
Also the Prometheus rate functions are not valid with gauges however they can be used with counters.

I think all this means I can't create a nice sexy graph of bold connections opened per minute etc.

I'd welcome your thoughts and insight on the design considerations to make these gauges instead of counters.

Thanks

[ben_butler-cole]

@toby Those metric types look like a bug to me. Do you have a support contract?

[toby]

Thanks for the prompt reply @ben.butler-cole
Yes, we've recently signed up and have an on-boarding call with your support folks soon, so will raise it there.

[ben_butler-cole]

@toby Please do raise a support ticket. I just want to make sure this is getting tracked somewhere useful.

[toby]

https://support.neo4j.com/hc/en-us/requests/5879

[mike_r_black]

I'm in the same situation as the rest of you. I've newly setup Prometheus + Grafana but now I'm in the conudrum of all the metrics to monitor, what should I be looking at and how? It would be cool to have a "Showroom" where people could share a metric/dashboard. Prometheus is new to me so I'm still learning how to query it (range vs. vectors and beyond).

[toby]

I'm using these queries:

avg({__name__=~"neo4j_bolt_messages_.*"}) by (__name__)

avg({__name__=~"neo4j_transaction_(active|committed|peak|started|terminated).*"}) by (__name__)

max(neo4j_page_cache_page_faults) by (name)

max({__name__=~"vm_gc_time_g1_.*_generation"}) by (__name__)

Of these 4 the first 3 would be better if Neo4j exported them as counters. Having the GC time as a gauge makes sense.

[mike_blum_neo4j]

toby:

avg({name=~"neo4j_transaction_(active|committed|peak|started|terminated).*"})

Is there any way to get the results of CALL dbms.listTransactions() or CALL dbms.listQueries() in the Prometheus metrics? It appears that running the query.log in production is considered harmful and I'm wondering how we can alert on hung transactions / blocking queries in production.

[albertodelazzar]

As far as I know, the list of current queries is not published as a metric regardless it's Prometeus or Graphite.
Of course, using the query.log could produce a lot of records (and files) if you have a low threshold and a lot of transactions.

[anubhuti_lucky]

when i am putting this on prometheus.yml .

scrape_configs:

The job name is added as a label job=<job_name> to any timeseries scraped from this config.

• job_name: 'Neo4j-prometheus'

  metrics_path: /metrics

  scheme defaults to 'http'.

  static_configs:

  • targets: ['localhost:2004']

It is showing down in Prometheus . .