APOC jar not included in yum update for neo4j-enterprise

[tms]

I run neo4j-enterprise on a reasonably up-to-date CentOS 7 system running on an AWS EC2 "t2-large" instance.

I've just successfully updated neo4j-enterprise to v4.4.3, the current version.

# yum update neo4j-enterprise
...
  Updating   : neo4j-enterprise-4.4.3-1.noarch                                                                  1/2 
  Cleanup    : neo4j-enterprise-4.4.2-1.noarch                                                                  2/2 
  Verifying  : neo4j-enterprise-4.4.3-1.noarch                                                                  1/2 
  Verifying  : neo4j-enterprise-4.4.2-1.noarch                                                                  2/2 

Updated:
  neo4j-enterprise.noarch 0:4.4.3-1                                                                                 

Complete!

I notice that although I got a new version of "labs", the apoc jar has not been updated:

# ls -l labs
total 15692
-rwxr-xr-x 1 neo4j neo4j 16049698 Jan  5 04:06 apoc-4.4.0.1-core.jar
-rwxr-xr-x 1 neo4j neo4j    11330 Jan  5 04:06 LICENSE
-rwxr-xr-x 1 neo4j neo4j     1131 Jan  5 04:06 README.txt

I had expected to see something like apoc-4.4.3.1-core.jar or something with "4.4.3" in its name.

I'm debugging a failed query that uses apoc, and seeing 38 complaints about failures in apoc-4.4.0.1-core.jar. I don't know if these are consequential or not, the version mismatch between the apoc jar and neo4j has my attention.

Here's what the complaints look like (I'm in /var/log/neo4j as root):

# grep Failed debug.log
2022-01-27 19:29:28.924+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.s3.S3Aws` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/amazonaws/auth/AWSCredentials
2022-01-27 19:29:28.925+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.s3.S3Aws$1` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/amazonaws/services/s3/model/S3ObjectInputStream
2022-01-27 19:29:28.926+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.s3.S3URLConnection` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/amazonaws/ClientConfiguration
2022-01-27 19:29:28.938+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.google.cloud.GCStorageURLConnection` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/google/cloud/storage/Storage
2022-01-27 19:29:28.941+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.hdfs.HDFSUtils` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/hadoop/fs/FSDataOutputStream
2022-01-27 19:29:28.942+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.hdfs.HDFSUtils$1` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/hadoop/fs/FSDataInputStream
2022-01-27 19:29:29.043+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.antlr.v4.gui.TreeViewer$VariableExtentProvide` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/abego/treelayout/NodeExtentProvider
2022-01-27 19:29:29.044+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.antlr.v4.gui.TreePostScriptGenerator` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/abego/treelayout/Configuration
2022-01-27 19:29:29.192+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.antlr.v4.gui.TreeViewer` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/abego/treelayout/TreeForTreeLayout
2022-01-27 19:29:29.244+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.antlr.v4.gui.TreePostScriptGenerator$VariableExtentProvide` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/abego/treelayout/NodeExtentProvider
2022-01-27 19:29:29.282+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.antlr.v4.unicode.UnicodeDataTemplateController` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/ibm/icu/text/UnicodeSet
2022-01-27 19:29:29.535+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.antlr.v4.gui.TreeLayoutAdaptor` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/abego/treelayout/TreeForTreeLayout
2022-01-27 19:29:29.715+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `com.jayway.jsonpath.spi.json.GsonJsonProvider` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/google/gson/JsonElement
2022-01-27 19:29:29.741+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `com.jayway.jsonpath.spi.json.JsonOrgJsonProvider` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/json/JSONException
2022-01-27 19:29:29.745+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `com.jayway.jsonpath.spi.mapper.GsonMappingProvider$1` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/google/gson/Gson
2022-01-27 19:29:29.746+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `com.jayway.jsonpath.spi.mapper.GsonMappingProvider$2` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/google/gson/Gson
2022-01-27 19:29:29.873+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.ConfigurationNodePointer` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/ri/model/NodePointer
2022-01-27 19:29:29.873+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.ConfigurationNodeIteratorBase` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/ri/model/NodeIterator
2022-01-27 19:29:29.874+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.ConfigurationNodeIteratorAttribute` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/ri/model/NodeIterator
2022-01-27 19:29:29.875+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.ConfigurationNodeIteratorChildren` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/ri/model/NodeIterator
2022-01-27 19:29:29.875+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.ConfigurationNodePointerFactory` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/ri/model/NodePointerFactory
2022-01-27 19:29:29.876+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.XPathExpressionEngine` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/ri/model/NodePointerFactory
2022-01-27 19:29:29.876+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.XPathContextFactory` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/JXPathContext
2022-01-27 19:29:29.877+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.tree.xpath.ConfigurationAttributePointer` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jxpath/ri/model/NodePointer
2022-01-27 19:29:29.922+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.io.VFSFileSystem` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/vfs2/FileSystemException
2022-01-27 19:29:29.996+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.interpol.ExprLookup` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/jexl2/JexlContext
2022-01-27 19:29:30.001+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.reloading.VFSFileHandlerReloadingDetector` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/commons/vfs2/FileSystemException
2022-01-27 19:29:30.033+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.resolver.CatalogResolver` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/xml/resolver/CatalogManager
2022-01-27 19:29:30.034+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.resolver.CatalogResolver$Catalog` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/xml/resolver/Catalog
2022-01-27 19:29:30.035+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.resolver.CatalogResolver$CatalogManager` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/xml/resolver/CatalogManager
2022-01-27 19:29:30.036+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.spring.ConfigurationPropertySource` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/springframework/core/env/EnumerablePropertySource
2022-01-27 19:29:30.036+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.configuration2.spring.ConfigurationPropertiesFactoryBean` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/springframework/beans/factory/InitializingBean
2022-01-27 19:29:30.473+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `module-info` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: module-info is not a class because access_flag ACC_MODULE is set
2022-01-27 19:29:32.512+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.xerces.util.XMLCatalogResolver` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/xml/resolver/readers/CatalogReader
2022-01-27 19:29:33.225+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.antlr.runtime.tree.DOTTreeGenerator` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/antlr/stringtemplate/StringTemplate
2022-01-27 19:29:33.372+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.logging.impl.AvalonLogger` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/avalon/framework/logger/Logger
2022-01-27 19:29:33.373+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.logging.impl.Log4JLogger` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/log4j/Priority
2022-01-27 19:29:33.375+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `org.apache.commons.logging.impl.LogKitLogger` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: org/apache/log/Logger

Have I missed a manual step to download an appropriate apoc jar?

[tms]

Is this really so boring? Has nobody else encountered this?

If there is no difference between the apoc jar for 4.4.3-1 and 4.4.0.1, and someone please confirm that?

Thanks,
Tom S.

[dana_canzano]

@tms

1. this isnt just a yum concern. A Neo4j Enterprise 4.4.3 and via tar also includes

 ls -al
total 15700
drwxr-xr-x  2 neo4j neo4j     4096 Jan  5 08:53 .
drwxr-xr-x 15 neo4j neo4j     4096 Jan 18 23:20 ..
-rw-r--r--  1 neo4j neo4j 16049698 Jan  5 08:53 apoc-4.4.0.1-core.jar

2. the WARN messages decribed and as described at

# grep Failed debug.log
2022-01-27 19:29:28.924+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.s3.S3Aws` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/amazonaws/auth/AWSCredentials
2022-01-27 19:29:28.925+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.s3.S3Aws$1` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/amazonaws/services/s3/model/S3ObjectInputStream
2022-01-27 19:29:28.926+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load apoc.util.s3.S3URLConnection` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/amazonaws/ClientConfiguration
2022-01-27 19:29:28.938+0000 WARN  [o.n.k.a.p.GlobalProcedures] Failed to load `apoc.util.google.cloud.GCStorageURLConnection` from plugin jar `/var/lib/neo4j/plugins/apoc-4.4.0.1-core.jar`: com/google/cloud/storage/Storage

can be ignored. These will be generated/logged on any/all APOC installs. See excessive logging of WARN statements in debug.log on startup · Issue #1117 · neo4j-contrib/neo4j-apo...

If there is no difference between the apoc jar for 4.4.3-1 and 4.4.0.1

Releases · neo4j-contrib/neo4j-apoc-procedures · GitHub describes each of the releases

[tms]

I appreciate your attention and the information provided here.

I reviewed the apoc github material before posting. I may have been unclear in my comment.

My perhaps incorrect understanding is that an apoc jar release is included in each each neo4j release, and I think I recall earlier material that strongly encouraged updating both simultaneously.

I've spent more than a week now trying to sort out the same issue, and I'm hoping to rule out 4.4.0.1-core.jar as a contributor. It appears that there is not yet a successor to this jar, and so I'll continue using it.

[dana_canzano]

@tms

fwiw

Release 4.4.0.2 · neo4j-contrib/neo4j-apoc-procedures · GitHub was released 16 days ago

[tms]

Fabulous!

I see two jars to download:

apoc-4.4.0.2-all.jar
apoc-4.4.0.2-core.jar

Until now, the "plugins" directory has always used a "...-core.jar". Is that the one I want?

Sorry to be dense, I'm a bit sleep-deprived.

[dana_canzano]

@tms

labs\ has always only included apoc-*core.jar and never included apoc-*all.jar. Introduction - APOC Documentation

apoc-*all.jar is all of APOC whereas apoc-*core.jar is a subset and represents the most commonly used apoc procedures

Procedures & Functions - APOC Documentation list all the procedures and describes if it is part of core or all/full

[tms]

I appreciate this. I reread the documentation, and learned the same. I guess "RTFM" is always a good thing.

I'm rerunning the problematic queries to see if this helps. I also did a yum update that changed some other things, and I've set the AWS option to allow long periods of high CPU activity.