Neo4j - python driver, Service unavailable

[phanindhar_bodl]

Am using neo4j on python 2.7.6 application using neo4j-driver(1.7.0b3).
Am facing service unavailable using proper credentials. Is there something that needs to be done to get it connected.

In [7]: graph_utils.get_neo4j_connection().session().run('Match (n) return count(n)')
---------------------------------------------------------------------------
ServiceUnavailable                        Traceback (most recent call last)
/home/apps/haygot/django/core/management/commands/shell.pyc in <module>()
----> 1 graph_utils.get_neo4j_connection().session().run('Match (n) return count(n)')

/usr/local/lib/python2.7/dist-packages/neo4j_driver-1.7.0b3-py2.7.egg/neo4j/__init__.py in run(self, statement, parameters, **kwparameters)
    432
    433         if not self._connection:
--> 434             self._connect()
    435         cx = self._connection
    436         protocol_version = cx.protocol_version

/usr/local/lib/python2.7/dist-packages/neo4j_driver-1.7.0b3-py2.7.egg/neo4j/__init__.py in _connect(self, access_mode)
    364                 return
    365             self._disconnect(sync=True)
--> 366         self._connection = self._acquirer(access_mode)
    367         self._connection_access_mode = access_mode
    368

/usr/local/lib/python2.7/dist-packages/neobolt-1.7.0rc3-py2.7-linux-x86_64.egg/neobolt/direct.py in acquire(self, access_mode)
    669
    670     def acquire(self, access_mode=None):
--> 671         return self.acquire_direct(self.address)
    672
    673

/usr/local/lib/python2.7/dist-packages/neobolt-1.7.0rc3-py2.7-linux-x86_64.egg/neobolt/direct.py in acquire_direct(self, address)
    541         """
    542         if self.closed():
--> 543             raise ServiceUnavailable("Connection pool closed")
    544         with self.lock:
    545             try:

ServiceUnavailable: Connection pool closed

Thanks in Advance,
Phanindhar Bodla

[phanindhar_bodl]

Any clue or suggestion will be helpful.

[michael_hunger]

I guess it's related to your graph_utils.get_neo4j_connection()

do you have the same if you use the plain driver?

[phanindhar_bodl]

Am just returning the plain driver object from the plain driver, since its frequently used , I used a method for the same.

[michael_hunger]

returning or creating a new one each time?
how is the lifecycle handled.

[phanindhar_bodl]

def get_neo4j_connection():
return GraphDatabase.driver(NEO4J_URL,auth=(settings.NEO_USER,settings.NEO_PASSWORD))

This is what am using, just saw that we are creating new instance each time.

And was working well for long and did stop since 10 days.

[technige]

It's impossible to tell from that snippet but it's probably how you're using the Driver object. As per the docs, a Driver object should be created on application startup and destroyed on shutdown. I expect the scope of yours is somehow different and is being closed before you try to use it.

[phanindhar_bodl]

I think we were creating driver object each time. But this has been working since a month and stopped since 10 days.

[michael_hunger]

Yes you should not do that

Create the driver once at application start and then make it available to the application.

[phanindhar_bodl]

Hi @michael.hunger

I've tried it the way you have suggested. I created a single instance of driver during app start and tried to use this.

Additionally i've create few loggers to verify the state of driver.

As soon as the driver instance is created, I logged driver state and it is open. Even before I use that connection driver is getting closed.

Is there anything else that I might need to change.

Thanks,
Phanindhar Bodla

[technige]

Can you share some of your updated code?

[phanindhar_bodl]

This is the code which am using,

import logging

from neo4j.v1 import GraphDatabase

from django.conf import settings

log = logging.getLogger('graph_driver')


class GraphDriver:
    driver = None

    def __init__(self):
        if not GraphDriver.driver:
            GraphDriver.driver = GraphDatabase.driver(settings.NEO4J_URL, auth=(settings.NEO_USER, settings.NEO_PASSWORD))
            print GraphDriver.driver.closed()

    @staticmethod
    def get_instance():
        if not GraphDriver.driver:
            GraphDriver()
        print GraphDriver.driver
        return GraphDriver

def get_neo4j_connection():
    return GraphDriver.get_instance().driver

On creation of driver object, I can see this is driver obj open. Just before i can use this I can see this to be closed.

[technige]

In order to implement a singleton correctly, you probably want to override __new__ instead of __init__. To manage the full driver lifecycle correctly, you'll also want some kind of shutdown method that can be called when your application closes. So, your code probably needs to look something like this:

class GraphDriver(object):

    __instance = None
    __driver = None
    
    def __new__(cls):
        if cls.__instance is None:
            inst = cls.__instance = object.__new__(cls)
            inst.__driver = GraphDatabase.driver(settings.NEO4J_URL, auth=(settings.NEO_USER, settings.NEO_PASSWORD))
        return cls.__instance

    @classmethod
    def shutdown(cls):
        if cls.__instance:
            cls.__instance.__driver.close()
            cls.__instance = None

This will lazily create a Driver instance but make the same instance available to all callers. So if you run a = GraphDriver(); b = GraphDriver() then a and b will end up as the same wrapper to the same Driver object.

When your application closes completely (not after every use), you'll need to call GraphDriver.shutdown() to close the connection pool correctly.

Note that this code is just illustrative 🙂

[phanindhar_bodl]

@technige Thanks so the detailing. Issue seems to be with the shutdown in my case. Used https://docs.python.org/2/library/atexit.html based shutdown hook to to get this is done correctly.