Database 'neo4j' is unavailable - using cypher-shell & Docker

[andy_mcshane]

I am running neo4j using Docker, all is good. I am now customising a neo4j image and loading data after the docker container has started. Part of this process involves executing a cypher script file using the cypher-shell. I have both a dockerfile and a docker-compose.yml file.

If I use the dockerfile on it's own and just do build and then run everything is working exactly as expected and my container spins with a pre-populated neo4j database.

If I use the docker-compose.yml with the 'build: .' option that then executes the dockerfile when it gets to the point of executing the cypher script using cypher-shell I get the error ' Database 'neo4j' is unavailable.' and that is all the message that I see.

This is my dockerfile

FROM neo4j:4.2.2

ENV NEO4J_HOME="/var/lib/neo4j"
ENV NEO4J_PASSWD=password
ENV NEO4J_apoc_import_file_enabled=true
ENV APOC_VERSION=4.2.0.1

# SHould look at copying this from a shared resource folder
ADD https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/${APOC_VERSION}/apoc-${APOC_VERSION}-all.jar ${NEO4J_HOME}/plugins

# Copy over generated dependency graph json file from shared resource folder
COPY Neo4j/Dependencies_Tree_Full_UnSorted.json ${NEO4J_HOME}/import/
# copy over cypher script from shared resource folder
COPY Neo4j/data_loader.cypher ${NEO4J_HOME}/import/

# set initial-password to start loading the data
# sleep for 20 secs for neo4j to start without any overlapping
CMD bin/neo4j-admin set-initial-password ${NEO4J_PASSWD} && \
    bin/neo4j start && sleep 6  0 && \
    if [ -f "${NEO4J_HOME}/import/data_loader.cypher" ]; then \
        echo "starting data load ..." && \
        cat ${NEO4J_HOME}/import/data_loader.cypher | NEO4J_USERNAME=neo4j NEO4J_PASSWORD=${NEO4J_PASSWD} bin/cypher-shell --fail-fast && rm ${NEO4J_HOME}/import/* && \
        echo "data load finished ..."; \
    fi && tail -f /logs/neo4j.log

and this is my docker-compose.yml file

version: '3.4'

services:
  neo4j:
    container_name: neo4j_dg
    #image: neo4j:latest
    restart: unless-stopped
    build: .
    ports:
      - 7474:7474
      - 7687:7687
    volumes:
      # Set volumes and paths as required
      - ./conf:/conf
      - ./data:/data
      - ./import:/import
      - ./logs:/logs
      - ./plugins:/plugins
    environment:
      # Raise memory limits
      - NEO4J_dbms_memory_pagecache_size=1G
      - NEO4J_dbms.memory.heap.initial_size=1G
      - NEO4J_dbms_memory_heap_max__size=1G
      - NEO4J_dbms.default_listen_address=0.0.0.0
      - NEO4J_dbms.connector.bolt.listen_address=:7687

I was initially getting the error ' Unable to connect to localhost:7687, ensure the database is running and that there is a working network connection to it.' but this went away when I added

      - NEO4J_dbms.default_listen_address=0.0.0.0
      - NEO4J_dbms.connector.bolt.listen_address=:7687

to my docker-compose.yml file to now be replaced with the 'Database 'neo4j' is unavailable.' error message. As you can see in the dockerfile I have also set a 60 second sleep period for the cypher-shell to wait in case it was a case of playing catchup but still no good.

As I say this is only happening when I use 'docker-compose up', everything is fine if I only use 'docker run'

As always any guidance is very much appreciated.

[andy_mcshane]

andy.mcshane:

Database 'neo4j' is unavailable

This is starting to get very frustrating now, no idea what I did but the original error is now back again!
'Unable to connect to localhost:7687, ensure the database is running and that there is a working network connection to it.'

[sam_gijare]

Please look at the logs and read the exact error that is happening and then analyze it to understand what exactly is the problem. After that look into possible ways of workaround that will eliminate the errors.

[dominicvivek06]

hello @andy.mcshane , welcome back

can you post the docker-compose up log file ...

[andy_mcshane]

Hi Dominic, yes me again, Making progress though!

This is the output which is now back to original 'not available' error.

C:\Platinum\Git\DataMigration\Deltafs.DataMigration.Dependency>docker-compose up
Building neo4j
Step 1/9 : FROM neo4j:4.2.2
4.2.2: Pulling from library/neo4j
a076a628af6f: Already exists                                                                                            943d8acaac04: Already exists                                                                                            b9998d19c116: Already exists                                                                                            eba5b958e041: Already exists                                                                                            b8d0884b547f: Already exists                                                                                            4b3572cb5079: Already exists                                                                                            e743ea4f2800: Already exists                                                                                            020ba241c011: Already exists                                                                                            Digest: sha256:c7f24de1dc1d2020ab24a884b8a39538937c1b14bc0ca1da3ddb2573b6fc412f
Status: Downloaded newer image for neo4j:4.2.2
 ---> 9edee9e153ab
Step 2/9 : ENV NEO4J_HOME="/var/lib/neo4j"
 ---> Running in 693fd32d3cef
Removing intermediate container 693fd32d3cef
 ---> 9b17fffbf659
Step 3/9 : ENV NEO4J_PASSWD=password
 ---> Running in 5786dd086b7c
Removing intermediate container 5786dd086b7c
 ---> bc63a359f760
Step 4/9 : ENV NEO4J_apoc_import_file_enabled=true
 ---> Running in 469a6e49bc5a
Removing intermediate container 469a6e49bc5a
 ---> 5c8a09636fd0
Step 5/9 : ENV APOC_VERSION=4.2.0.1
 ---> Running in 5276762657b6
Removing intermediate container 5276762657b6
 ---> baf859ef2e84
Step 6/9 : ADD https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/${APOC_VERSION}/apoc-${APOC_VERSION}-all.jar ${NEO4J_HOME}/plugins

 ---> b4cc06e3d519
Step 7/9 : COPY Neo4j/Dependencies_Tree_Full_UnSorted.json ${NEO4J_HOME}/import/
 ---> 39efe19d1c45
Step 8/9 : COPY Neo4j/data_loader.cypher ${NEO4J_HOME}/import/
 ---> 7a7a6e94c743
Step 9/9 : CMD bin/neo4j-admin set-initial-password ${NEO4J_PASSWD} &&     bin/neo4j start && sleep 20 &&     if [ -f "${NEO4J_HOME}/import/data_loader.cypher" ]; then         echo "starting data load ..." &&         cat ${NEO4J_HOME}/import/data_loader.cypher | NEO4J_USERNAME=neo4j NEO4J_PASSWORD=${NEO4J_PASSWD} bin/cypher-shell --fail-fast && rm ${NEO4J_HOME}/import/* &&         echo "data load finished ...";     fi && tail -f /logs/neo4j.log
 ---> Running in 3fd3ed3b511e
Removing intermediate container 3fd3ed3b511e
 ---> 4e92a89d5349

Successfully built 4e92a89d5349
Successfully tagged deltafsdatamigrationdependency_neo4j:latest
WARNING: Image for service neo4j was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating neo4j_dg ... done                                                                                              Attaching to neo4j_dg
neo4j_dg | grep: /var/lib/neo4j/conf/neo4j.conf: No such file or directory
neo4j_dg | Fetching versions.json for Plugin 'apoc' from https://neo4j-contrib.github.io/neo4j-apoc-procedures/versions.json
neo4j_dg | Installing Plugin 'apoc' from https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/4.2.0.1/apoc-4.2.0.1-all.jar to /plugins/apoc.jar
neo4j_dg | Applying default values for plugin apoc to neo4j.conf
neo4j_dg | Changed password for user 'neo4j'.
neo4j_dg | Directories in use:
neo4j_dg |   home:         /var/lib/neo4j
neo4j_dg |   config:       /var/lib/neo4j/conf
neo4j_dg |   logs:         /logs
neo4j_dg |   plugins:      /plugins
neo4j_dg |   import:       /import
neo4j_dg |   data:         /var/lib/neo4j/data
neo4j_dg |   certificates: /var/lib/neo4j/certificates
neo4j_dg |   run:          /var/lib/neo4j/run
neo4j_dg | Starting Neo4j.
neo4j_dg | Started neo4j (pid 397). It is available at http://localhost:7474/
neo4j_dg | There may be a short delay until the server is ready.
neo4j_dg | See /logs/neo4j.log for current status.
neo4j_dg | starting data load ...
neo4j_dg | Database 'neo4j' is unavailable.

[conker84 · Accepted solution]

May I suggest a simpler approach?

This works for me:

my docker-compose.yml

version: '3'

services:
  neo4j:
    image: neo4j:4.2.2
    hostname: neo4j
    container_name: neo4j
    ports:
      - 7474:7474
      - 7687:7687
    environment:
      NEO4J_AUTH: "neo4j/apoc"
      NEO4J_dbms_memory_heap_max__size: 8G
      NEO4J_dbms_security_procedures_unrestricted: "apoc.*"
      NEO4J_apoc_import_file_enabled: "true"
      NEO4J_dbms_logs_debug_level: DEBUG
      NEO4J_apoc_initializer_cypher: "CALL apoc.cypher.runFile('file:///myfile.cypher')"
      NEO4JLABS_PLUGINS: '["apoc"]'
    volumes:
      - ./neo4j/import:/import

from the same directory I created a subdir neo4j/import inside import I have myfile.cypher with the following content:

CREATE (:Actor {name:'Tom Hanks'})-[:ACTED_IN {roles:'Forrest'}]->(:Movie {title:'Forrest Gump'});

[andy_mcshane]

This is fantastic and seems to work great, thank you very much!