Neo4j

amanjain1527 · ‎11-12-2019

Hi , I am trying to load data from AWS s3 bucket using a Pre signed URL and getting the below error

Command -
LOAD CSV WITH HEADERS FROM '<presigned_url>' AS ROW RETURN ROW LIMIT 10;

Error -
Neo.ClientError.Statement.ExternalResourceFailed: Couldn't load the external resource at: https://.s3.amazonaws.com/temp.csv?AWSAccessKeyId=***********&Signature=****security-token=

My Neo4j is running on Kubernetes. Should my file have to be publicly accessible ?? What other options I can check ? I have also set dbms.security.allow_csv_import_from_file_urls=true

dana_canzano · ‎11-12-2019

a KB was created for this sometime ago and as described at

can you review and see if it fills in the missing details

amanjain1527 · ‎11-13-2019

I saw two probable solutions there -

set dbms.security.allow_csv_import_from_file_urls=true
neo4j.conf, comment out or delete dbms.directories.import=import

how to comment out or delete dbms.directories.import=import during Neo4j installation through Kubernetes.

david_allen · ‎11-13-2019

In Kubernetes, neo4j runs within docker containers. Use these docs and use the "environment variable syntax" for passing the config you need to your container, and the docker/kubernetes install works just like the regular Neo4j install method.

https://neo4j.com/docs/operations-manual/current/docker/configuration/

amanjain1527 · ‎11-13-2019

David,
We are using helm command to install neo4j. I am not sure how to set properties neo4j.conf while installing through heml. I am new to Kubernetes. Could you please help? Also do we need check our bucket policies/ IAM roles ? Can this be a reason for this error ?

amanjain1527 · ‎11-13-2019

David
Below are the kubernetes commands used in our .gitlab-ci.yaml file

export KOPS_STATE_STORE=s3://<bucket_name>
- export NAME=<cluster_name>
- export KUBECONFIG=$HOME/.kube/config
- kops export kubecfg --name=$NAME
- kubectl apply -f ./k8s/ingress-bolt.yaml
- kubectl apply -f ./k8s/ingress.yaml
- helm upgrade --install neo4j stable/neo4j -f k8s/values.yaml --namespace dev-tem
- kubectl get services
- kubectl get pods --all-namespaces

Let me know if you need any other information.
Thanks

david_allen · ‎11-13-2019

What you've got here is not the right approach. What you rather need to do is get a copy of the helm chart templates and adjust the environment variables passed to the pod to add your own custom configuration. Any variables you do with export here will have no effect.

When you type helm upgrade --install stable/neo4j this repo is where that code resides. You should grab the code this repo, and adjust the environment variables right here:

github.com

helm/charts/blob/master/stable/neo4j/templates/core-statefulset.yaml#L26


    heritage: {{ .Release.Service | quote }}
    release: {{ .Release.Name | quote }}
    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
    app: {{ template "neo4j.name" . }}
    component: core
spec:
  containers:
  - name: {{ template "neo4j.fullname" . }}
    image: "{{ .Values.image }}:{{ .Values.imageTag }}"
    imagePullPolicy: "{{ .Values.imagePullPolicy }}"
    env:
      - name: NEO4J_ACCEPT_LICENSE_AGREEMENT
        value: "{{ .Values.acceptLicenseAgreement }}"
      - name: NEO4J_dbms_mode
        value: CORE
      - name: NUMBER_OF_CORES
        value: "{{ .Values.core.numberOfServers }}"
      - name: AUTH_ENABLED
        value: "{{ .Values.authEnabled }}"
      - name: NEO4J_causal__clustering_discovery__type
        value: DNS

Neo4j

Cannot Load data into Neo4j using AWS S3 pre signed URL