Unable to use apoc.load.csv in Neo4j 4.1.1

[kalyan_b_aninda]

Hello recently I have install Neo4j 4.1.1 My previous version was 3.5.11 . The bellow query worked well in oldversion but now I am getting this error

Failed to invoke procedure apoc.periodic.iterate: Caused by: org.neo4j.internal.kernel.api.exceptions.ProcedureException: There is no procedure with the name apoc.load.csv registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed.

the query was

CALL apoc.periodic.iterate('CALL apoc.load.csv("a.csv") yield map as row return row','CREATE (p:SHOPS) SET p = row', {batchSize:1000000, iterateList:true, parallel:true})

I have used default apoc library which has come default with new neo4j version
changed the neo4j.conf file into

apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=true

but still I am getting the error. can anyone helping me out why this is happening

I tried Using period commit approach too but have got
'executing queries that use periodic commit in an open transaction is not possible.' . So previous query was my alternate query to avoid this error . A good help is well appreciated

[koji]

Hi

This is my simple a.csv

A,B,C
1,2,3
4,5,6

Environment

macOS Catalina 10.15.6
Neo4j Desktop 1.3.4
Neo4j 4.1.1
APOC 4.1.0.2

I added this line to neo4j.conf.

apoc.import.file.enabled=true

Your Cypher works well on my Mac.

[kalyan_b_aninda]

Thanks @koji for your reply but I didnt get it why this is happening. the query should perform properly. Same query works on yours computer but my one is giving error. I have given same configuration in my .conf file 😞 weird

[koji]

Please check your /plugin folder.
Is the APOC version correct?

If you are using a Neo4j Server, Please add this line to your neo4j.conf and reboot the Neo4j.

dbms.security.procedures.unrestricted=apoc.*

[ameyasoft]

Open any text editor and create a new file. Add this line:
set apoc.import.file.enabled=true

Save this as apoc.conf and copy this file to 'conf' folder in your database installation. This is the same folder where neo4j.conf file resides.

[koji]

I removed this line from neo4j.conf and create apoc.conf with this line.

apoc.import.file.enabled=true

It works, too.

[kalyan_b_aninda]

Thanks for the help . I did all the things . I copied apoc library jar from lab folder to plugins folder. I created apoc. conf file and then copied it conf folder . It is not working

Failed to invoke procedure apoc.periodic.iterate: Caused by: org.neo4j.internal.kernel.api.exceptions.ProcedureException: There is no procedure with the name apoc.load.csv registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed.

dbms.security.procedures.whitelist=apoc.coll., apoc.load., apoc. *,gds. *
apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=true
apoc.export.file.enabled=true

these are my. neo4j.conf file

[Cobra]

Hello @kalyan.b.aninda

Is it a local database or a remote server?

If it's remote server, did you restart it?

Regards,
Cobra

[kalyan_b_aninda]

Hi @Cobra nice to see you again. it is in my local database. just started neo4j start in my local pc

[Cobra]

So you are using Neo4j Desktop or you installed a Neo4j Server version on your PC (as service)?

[kalyan_b_aninda]

neo4j server version on pc ( as service)

[Cobra]

So did you restarted the service after you put the APOC plugin the plugin folder?

[kalyan_b_aninda]

yes I have done it before . and for assurance I applied it right now

[Cobra]

Can you share the neo4j.file ?

[kalyan_b_aninda]

are implying. the conf file here it is

#*****************************************************************

Neo4j configuration

For more details and a complete list of settings, please see

https://neo4j.com/docs/operations-manual/current/reference/configuration-settings/

#*****************************************************************

The name of the default database

#dbms.default_database=neo4j

Paths of directories in the installation.

dbms.directories.data=data
dbms.directories.plugins=/usr/local/Cellar/neo4j/4.1.1/libexec/plugins
dbms.directories.logs=logs
#dbms.directories.lib=lib
#dbms.directories.run=run
#dbms.directories.transaction.logs.root=data/transactions

This setting constrains all LOAD CSV import files to be under the import directory. Remove or comment it out to

allow files to be loaded from anywhere in the filesystem; this introduces possible security problems. See the

LOAD CSV section of the manual for details.

dbms.directories.import=import

Whether requests to Neo4j are authenticated.

To disable authentication, uncomment this line

dbms.security.auth_enabled=false

Enable this to be able to upgrade a store from an older version.

#dbms.allow_upgrade=true

#********************************************************************

Memory Settings

#********************************************************************

Memory settings are specified kilobytes with the 'k' suffix, megabytes with

'm' and gigabytes with 'g'.

If Neo4j is running on a dedicated server, then it is generally recommended

to leave about 2-4 gigabytes for the operating system, give the JVM enough

heap to hold all your transaction state and query context, and then leave the

rest for the page cache.

Java Heap Size: by default the Java heap size is dynamically calculated based

on available system resources. Uncomment these lines to set specific initial

and maximum heap size.

#dbms.memory.heap.initial_size=512m
#dbms.memory.heap.max_size=512m

The amount of memory to use for mapping the store files.

The default page cache memory assumes the machine is dedicated to running

Neo4j, and is heuristically set to 50% of RAM minus the Java heap size.

#dbms.memory.pagecache.size=10g

Limit the amount of memory that all of the running transaction can consume.

By default there is no limit.

#dbms.memory.transaction.global_max_size=256m

Limit the amount of memory that a single transaction can consume.

By default there is no limit.

#dbms.memory.transaction.max_size=16m

Transaction state location. It is recommended to use ON_HEAP.

dbms.tx_state.memory_allocation=ON_HEAP

#*****************************************************************

Network connector configuration

#*****************************************************************

With default configuration Neo4j only accepts local connections.

To accept non-local connections, uncomment this line:

#dbms.default_listen_address=0.0.0.0

You can also choose a specific network interface, and configure a non-default

port for each connector, by setting their individual listen_address.

The address at which this server can be reached by its clients. This may be the server's IP address or DNS name, or

it may be the address of a reverse proxy which sits in front of the server. This setting may be overridden for

individual connectors below.

#dbms.default_advertised_address=localhost

You can also choose a specific advertised hostname or IP address, and

configure an advertised port for each connector, by setting their

individual advertised_address.

By default, encryption is turned off.

To turn on encryption, an ssl policy for the connector needs to be configured

Read more in SSL policy section in this file for how to define a SSL policy.

Bolt connector

dbms.connector.bolt.enabled=true
#dbms.connector.bolt.tls_level=DISABLED
#dbms.connector.bolt.listen_address=:7687

HTTP Connector. There can be zero or one HTTP connectors.

dbms.connector.http.enabled=true
#dbms.connector.http.listen_address=:7474

HTTPS Connector. There can be zero or one HTTPS connectors.

dbms.connector.https.enabled=false
#dbms.connector.https.listen_address=:7473

Number of Neo4j worker threads.

#dbms.threads.worker_count=

#*****************************************************************

SSL policy configuration

#*****************************************************************

Each policy is configured under a separate namespace, e.g.

dbms.ssl.policy..*

can be any of 'bolt', 'https', 'cluster' or 'backup'

The scope is the name of the component where the policy will be used

Each component where the use of an ssl policy is desired needs to declare at least one setting of the policy.

Allowable values are 'bolt', 'https', 'cluster' or 'backup'.

E.g if bolt and https connectors should use the same policy, the following could be declared

dbms.ssl.policy.bolt.base_directory=certificates/default

dbms.ssl.policy.https.base_directory=certificates/default

However, it's strongly encouraged to not use the same key pair for multiple scopes.

N.B: Note that a connector must be configured to support/require

SSL/TLS for the policy to actually be utilized.

see: dbms.connector.*.tls_level

SSL settings (dbms.ssl.policy..*)

.base_directory Base directory for SSL policies paths. All relative paths within the

SSL configuration will be resolved from the base dir.

.private_key A path to the key file relative to the '.base_directory'.

.private_key_password The password for the private key.

.public_certificate A path to the public certificate file relative to the '.base_directory'.

.trusted_dir A path to a directory containing trusted certificates.

.revoked_dir Path to the directory with Certificate Revocation Lists (CRLs).

.verify_hostname If true, the server will verify the hostname that the client uses to connect with. In order

for this to work, the server public certificate must have a valid CN and/or matching

Subject Alternative Names.

.client_auth How the client should be authorized. Possible values are: 'none', 'optional', 'require'.

.tls_versions A comma-separated list of allowed TLS versions. By default only TLSv1.2 is allowed.

.trust_all Setting this to 'true' will ignore the trust truststore, trusting all clients and servers.

Use of this mode is discouraged. It would offer encryption but no security.

.ciphers A comma-separated list of allowed ciphers. The default ciphers are the defaults of

the JVM platform.

Bolt SSL configuration

#dbms.ssl.policy.bolt.enabled=true
#dbms.ssl.policy.bolt.base_directory=certificates/bolt
#dbms.ssl.policy.bolt.private_key=private.key
#dbms.ssl.policy.bolt.public_certificate=public.crt
#dbms.ssl.policy.bolt.client_auth=NONE

Https SSL configuration

#dbms.ssl.policy.https.enabled=true
#dbms.ssl.policy.https.base_directory=certificates/https
#dbms.ssl.policy.https.private_key=private.key
#dbms.ssl.policy.https.public_certificate=public.crt
#dbms.ssl.policy.https.client_auth=NONE

Cluster SSL configuration

#dbms.ssl.policy.cluster.enabled=true
#dbms.ssl.policy.cluster.base_directory=certificates/cluster
#dbms.ssl.policy.cluster.private_key=private.key
#dbms.ssl.policy.cluster.public_certificate=public.crt

Backup SSL configuration

#dbms.ssl.policy.backup.enabled=true
#dbms.ssl.policy.backup.base_directory=certificates/backup
#dbms.ssl.policy.backup.private_key=private.key
#dbms.ssl.policy.backup.public_certificate=public.crt

#*****************************************************************

Logging configuration

#*****************************************************************

To enable HTTP logging, uncomment this line

#dbms.logs.http.enabled=true

Number of HTTP logs to keep.

#dbms.logs.http.rotation.keep_number=5

Size of each HTTP log that is kept.

#dbms.logs.http.rotation.size=20m

To enable GC Logging, uncomment this line

#dbms.logs.gc.enabled=true

GC Logging Options

see https://docs.oracle.com/en/java/javase/11/tools/java.html#GUID-BE93ABDC-999C-4CB5-A88B-1994AAAC74D5

#dbms.logs.gc.options=-Xlog:gc*,safepoint,age*=trace

Number of GC logs to keep.

#dbms.logs.gc.rotation.keep_number=5

Size of each GC log that is kept.

#dbms.logs.gc.rotation.size=20m

Log level for the debug log. One of DEBUG, INFO, WARN and ERROR. Be aware that logging at DEBUG level can be very verbose.

#dbms.logs.debug.level=INFO

Size threshold for rotation of the debug log. If set to zero then no rotation will occur. Accepts a binary suffix "k",

"m" or "g".

#dbms.logs.debug.rotation.size=20m

Maximum number of history files for the internal log.

#dbms.logs.debug.rotation.keep_number=7

#*****************************************************************

Miscellaneous configuration

#*****************************************************************

Enable this to specify a parser other than the default one.

#cypher.default_language_version=3.5

Determines if Cypher will allow using file URLs when loading data using

LOAD CSV. Setting this value to false will cause Neo4j to fail LOAD CSV

clauses that load data from the file system.

dbms.security.allow_csv_import_from_file_urls=true

Value of the Access-Control-Allow-Origin header sent over any HTTP or HTTPS

connector. This defaults to '*', which allows broadest compatibility. Note

that any URI provided here limits HTTP/HTTPS access to that URI only.

#dbms.security.http_access_control_allow_origin=*

Value of the HTTP Strict-Transport-Security (HSTS) response header. This header

tells browsers that a webpage should only be accessed using HTTPS instead of HTTP.

It is attached to every HTTPS response. Setting is not set by default so

'Strict-Transport-Security' header is not sent. Value is expected to contain

directives like 'max-age', 'includeSubDomains' and 'preload'.

#dbms.security.http_strict_transport_security=

Retention policy for transaction logs needed to perform recovery and backups.

dbms.tx_log.rotation.retention_policy=1 days

Only allow read operations from this Neo4j instance. This mode still requires

write access to the directory for lock purposes.

#dbms.read_only=false

Comma separated list of JAX-RS packages containing JAX-RS resources, one

package name for each mountpoint. The listed package names will be loaded

under the mountpoints specified. Uncomment this line to mount the

org.neo4j.examples.server.unmanaged.HelloWorldResource.java from

neo4j-server-examples under /examples/unmanaged, resulting in a final URL of

http://localhost:7474/examples/unmanaged/helloworld/{nodeId}

#dbms.unmanaged_extension_classes=org.neo4j.examples.server.unmanaged=/examples/unmanaged

A comma separated list of procedures and user defined functions that are allowed

full access to the database through unsupported/insecure internal APIs.

dbms.security.procedures.unrestricted=my.extensions.example,my.procedures.,apoc.,gds.*

A comma separated list of procedures to be loaded by default.

Leaving this unconfigured will load all procedures found.

dbms.security.procedures.whitelist=apoc.coll.,apoc.load.,apoc.,gds.
apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=true
apoc.export.file.enabled=true

#********************************************************************

JVM Parameters

#********************************************************************

G1GC generally strikes a good balance between throughput and tail

latency, without too much tuning.

dbms.jvm.additional=-XX:+UseG1GC

Have common exceptions keep producing stack traces, so they can be

debugged regardless of how often logs are rotated.

dbms.jvm.additional=-XX:-OmitStackTraceInFastThrow

Make sure that initmemory is not only allocated, but committed to

the process, before starting the database. This reduces memory

fragmentation, increasing the effectiveness of transparent huge

pages. It also reduces the possibility of seeing performance drop

due to heap-growing GC events, where a decrease in available page

cache leads to an increase in mean IO response time.

Try reducing the heap memory, if this flag degrades performance.

dbms.jvm.additional=-XX:+AlwaysPreTouch

Trust that non-static final fields are really final.

This allows more optimizations and improves overall performance.

NOTE: Disable this if you use embedded mode, or have extensions or dependencies that may use reflection or

serialization to change the value of final fields!

dbms.jvm.additional=-XX:+UnlockExperimentalVMOptions
dbms.jvm.additional=-XX:+TrustFinalNonStaticFields

Disable explicit garbage collection, which is occasionally invoked by the JDK itself.

dbms.jvm.additional=-XX:+DisableExplicitGC

#Increase maximum number of nested calls that are can be inlined from 9 (default) to 15
dbms.jvm.additional=-XX:MaxInlineLevel=15

Restrict size of cached JDK buffers to 256 KB

dbms.jvm.additional=-Djdk.nio.maxCachedBufferSize=262144

More efficient buffer allocation in Netty by allowing direct no cleaner buffers.

dbms.jvm.additional=-Dio.netty.tryReflectionSetAccessible=true

Exits JVM on the first occurrence of an out-of-memory error. Its preferable to restart VM in case of out of memory errors.

dbms.jvm.additional=-XX:+ExitOnOutOfMemoryError

Remote JMX monitoring, uncomment and adjust the following lines as needed. Absolute paths to jmx.access and

jmx.password files are required.

Also make sure to update the jmx.access and jmx.password files with appropriate permission roles and passwords,

the shipped configuration contains only a read only role called 'monitor' with password 'Neo4j'.

For more details, see: http://download.oracle.com/javase/8/docs/technotes/guides/management/agent.html

On Unix based systems the jmx.password file needs to be owned by the user that will run the server,

and have permissions set to 0600.

For details on setting these file permissions on Windows see:

http://docs.oracle.com/javase/8/docs/technotes/guides/management/security-windows.html

#dbms.jvm.additional=-Dcom.sun.management.jmxremote.port=3637
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.authenticate=true
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.ssl=false
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.password.file=/absolute/path/to/conf/jmx.password
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.access.file=/absolute/path/to/conf/jmx.access

Some systems cannot discover host name automatically, and need this line configured:

#dbms.jvm.additional=-Djava.rmi.server.hostname=$THE_NEO4J_SERVER_HOSTNAME

Expand Diffie Hellman (DH) key size from default 1024 to 2048 for DH-RSA cipher suites used in server TLS handshakes.

This is to protect the server from any potential passive eavesdropping.

dbms.jvm.additional=-Djdk.tls.ephemeralDHKeySize=2048

This mitigates a DDoS vector.

dbms.jvm.additional=-Djdk.tls.rejectClientInitiatedRenegotiation=true

Enable remote debugging

#dbms.jvm.additional=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005

This filter prevents deserialization of arbitrary objects via java object serialization, addressing potential vulnerabilities.

By default this filter whitelists all neo4j classes, as well as classes from the hazelcast library and the java standard library.

These defaults should only be modified by expert users!

For more details (including filter syntax) see: https://openjdk.java.net/jeps/290

#dbms.jvm.additional=-Djdk.serialFilter=java.;org.neo4j.;com.neo4j.;com.hazelcast.;net.sf.ehcache.Element;com.sun.proxy.;org.openjdk.jmh.**;!

Increase the default flight recorder stack sampling depth from 64 to 256, to avoid truncating frames when profiling.

dbms.jvm.additional=-XX:FlightRecorderOptions=stackdepth=256

Allow profilers to sample between safepoints. Without this, sampling profilers may produce less accurate results.

dbms.jvm.additional=-XX:+UnlockDiagnosticVMOptions
dbms.jvm.additional=-XX:+DebugNonSafepoints

#********************************************************************

Wrapper Windows NT/2000/XP Service Properties

#********************************************************************

WARNING - Do not modify any of these properties when an application

using this configuration file has been installed as a service.

Please uninstall the service before modifying this section. The

service can then be reinstalled.

Name of the service

dbms.windows_service_name=neo4j

#********************************************************************

Other Neo4j system properties

#********************************************************************
wrapper.java.additional=-Djava.awt.headless=true
wrapper.java.additional.4=-Dneo4j.ext.udc.source=homebrew
dbms.directories.data=/usr/local/var/neo4j/data
dbms.directories.logs=/usr/local/var/log/neo4j

[Cobra]

dbms.security.procedures.unrestricted=apoc.*,gds.*

[kalyan_b_aninda]

Tried it but noting happened 😞

Failed to invoke procedure apoc.periodic.iterate: Caused by: org.neo4j.internal.kernel.api.exceptions.ProcedureException: There is no procedure with the name apoc.load.csv registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed.

[Cobra]

dbms.security.procedures.whitelist=apoc.*, gds.*

[kalyan_b_aninda]

I figured it out @cobra the apoc 4.1 core version jar which is come up with apoc 4.1 does not have apoc.load.csv class. i replaced it with apoc-4.1.0.2-all.jar. and it worked smoothly

[deepak_tanna]

Unfortunately teven though i downloaded apoc-4.3.0-rc01-all.jar this didn't work for me and I continue to get the error. Do I have to reinstall the plugin after replacing the jar

[Cobra]

Did you restart the database?

[deepak_tanna]

Yes i did but i think i was facing an issue due to settings in my neo4j.conf - fortunately i got help from the support team and we were able to get it to run.