Running integration tests against neo4j db in CI/CD gitlab

[maros_zeleny]

Hi,
I would like to ask what is right approach of running integration tests against neo4j db in CI/CD ?

Here is my case:
We have backend implemented in node.js and using neo4j-driver for connection to db. We use gitlab for CI/CD. We are running in CI/CD the test stage with neo4j database and our neo4j backend service defined as services in docker-compose. The problem is when database is started, we seed data into it using cypher-shell but somehow our service started integration test before and test stage failed.
We tried using sleep command or official script wait-for-it.sh (https://github.com/vishnubob/wait-for-it) to seed db before running tests but it does not work. This problem happen only in CI/CD, locally it works fine (f.e. running npm run test).

What is the right approach or what are you doing if you want to perform integration tests in CI/CD ? It's good idea ?

below you can see our docker compose file

version: "3"

services:
  database-neo4j-test:
      image: neo4j:3.5.3
      container_name: database-neo4j-test
      ports:
      - "7474:7474"
      - "7687:7687"
      volumes:
        - ../database-neo4j-service/src/scripts/init.cql:/var/lib/neo4j/import/init.cql
      environment:
        - NEO4J_AUTH=neo4j/neo4j  
        - NEO4J_dbms_connector_bolt_thread__pool__min__size=100
        - NEO4J_dbms_connector_bolt_thread__pool__max__size=5000
        - NEO4J_dbms_connector_bolt_thread__pool__keep__alive=5m
        - NEO4J_dbms_threads_worker__count=1000
        - NEO4J_dbms_memory_heap_max__size=5G
        - NEO4J_dbms_memory_heap_initial__size=1G
        - NEO4J_dbms_memory_heap_max__size=2G
        - NEO4J_dbms_memory_heap_initial__size=2G
        - NEO4J_relationship__auto__indexing=true
        - NEO4J_relationship__keys__indexable=arch_id
        - NEO4J_relationship__keys__indexable=type
      command: >
            sh -c "bin/neo4j-admin set-initial-password neo4j || true &&
            bin/neo4j start &&
            sleep 20 &&
            cat /var/lib/neo4j/import/init.cql | NEO4J_USERNAME=neo4j NEO4J_PASSWORD=neo4j cypher-shell --fail-fast > init_output.log 2>&1 &&
            tail -f /dev/null"
            
  neo4j-service-test:
    container_name: neo4j-service-test
    build:
      context: .
      dockerfile: Dockerfile.gitlab.test
      args:
        - NPM_CONFIG=${NPM_CONFIG}
    command: sh -c "./scripts/wait-for-it.sh database-neo4j-test:7687; npm run test:gitlab"
    environment:
      NEO4J_URL: bolt://archmesh-neo4j-test:7687
    depends_on:
      - database-neo4j-test

[krisgeus]

I think the problem is that you start both processes as a service. For seeding the database that’s fine but the test should be run as a task or step in your gitlab pipeline.
I think gitlab will make sure the build step will start after the service is ready