Neo4j APOC Release — Export Streaming support, Delete custom procedures, New string functions

[mark_needham]

Neo4j APOC Release — Export Streaming support, Delete custom procedures, New string functions

The latest release of APOC introduces streaming support for exporting to JSON and GraphML, as well as other features.

This week we released version 3.5.0.6 of APOC, Neo4j’s standard library.

APOC

tl;dr

This release contains the following features and bug fixes:

• streaming support for export JSON and GraphML
• string multiplication and fill functions
• removing custom procedures and functions
• null issues with apoc.coll.containsAll and Mongo procedure
• apoc.periodic.repeat has improved reporting for bad queries
• apoc.graph.fromDocument considers whitelist property mappings

You can install the latest release directly from the Neo4j Desktop in the ‘Plugins’ section of your project. Jennifer Reif also has a detailed post explaining how to install plugins if you haven’t used any yet.

Installing the APOC Library

If you’re installing the library on a server installation of Neo4j you can download the JAR from the GitHub releases page. This page also includes the release notes.

GitHub releases page

Streaming support for export JSON and GraphML

APOC supports exporting data to a variety of different formats, including JSON, GraphML, CSV, and Cypher script.


It can export data in those formats into files or as a stream of results.

Exporting to a file is a useful feature, but only works if we are able to write to the file system on the server where Neo4j is hosted. If we don’t have that ability then the streaming approach is the way to go.

This release adds missing stream functionality for the JSON and GraphML export formats.

Assuming that we’ve imported the movies graph (:play movies), the following query exports the nodes and relationships around Tom Cruise and the movies that he acted in:

CALL apoc.export.json.query(
  "MATCH path = (p:Person {name: $name})-[:ACTED_IN]->(movie) 
   RETURN path",
  null,
  {stream: true, params: {name: "Tom Cruise"}})
YIELD data
RETURN data;

If we execute that query, we’ll see the following output:

Export streaming JSON

We could then write that stream to disk on the client, use the data to update our application, or send it into a tool like jq to do further filtering and analysis.

Repeating Text

This release also adds a couple of new functions that can be used to generate arrays or strings of repeated text.

apoc.text.repeat concatenates a string a certain number of times. For example, the following function call will create a string with the value Aura repeated 5 times:

RETURN apoc.text.repeat("Aura", 5) AS result;

╒══════════════════════╕
│"result"              │
╞══════════════════════╡
│"AuraAuraAuraAuraAura"│
└──────────────────────┘

apoc.coll.fill is similar, but creates an array of repeated values. For example, the following function call will create a list with the value Neo4j Aura repeated 3 times:

RETURN apoc.coll.fill("Neo4j Aura", 3) AS result;

╒════════════════════════════════════════╕
│"result"                                │
╞════════════════════════════════════════╡
│["Neo4j Aura","Neo4j Aura","Neo4j Aura"]│
└────────────────────────────────────────┘

Deleting custom procedures and functions

Let’s declare a function that tells us how long it is until 2020:

CALL apoc.custom.asFunction("countdown", 
  "RETURN duration.inDays(date(), date('2020-01-01')) AS duration",
  "Duration", 
  [], 
  true);

We can now list all the custom functions and procedures on our database:

CALL apoc.custom.list();

List of functions and procedures

Let’s run the function to see how many days until the new year:

RETURN custom.countdown().days AS days;

╒══════╕
│"days"│
╞══════╡
│43    │
└──────┘

Not long to go! Now let’s use the new apoc.custom.removeFunction to remove the function:

CALL apoc.custom.removeFunction("countdown");

Now if we try to run it we won’t be able to:

RETURN custom.countdown();

Unknown function 'custom.countdown'

If we want to recreate a function with the same name, we’ll also need to flush the query cache to make sure references to any old version of the function are removed:

CALL apoc.custom.asFunction("countdown", 
  "RETURN duration.inDays(date(), date('2020-01-01')) AS duration",
  "Duration", 
  [], 
  true);

CALL dbms.clearQueryCaches();

Bug Fixes

This release also contains several bug fixes, including:

apoc.graph.fromDocument

The apoc.graph.fromDocument procedure now considers the mappings field when creating a graph structure from a JSON document. The following query creates the graph structure (:User)<-[:USER]-(:Tweet):

CALL apoc.graph.fromDocument('{
 "id": 1,
 "text": "Very interesting tweet",
 "data": "02-11-2019",
 "user": { "id": 1, "screenName": "conker84", "name": "Andrea" },
 "geo": { "latitude": 11.45, "longitude": -12.3 }
 }', {
  mappings: { `$`: "Tweet{!id, text}", 
              `$.user`: "User{!screenName, name}" },
  write: true
})

Executing this query will result in the following graph:

Small twitter graph

Before this release the user field would not have been properly processed.

apoc.periodic.repeat

The apoc.periodic.repeat procedure also has better error reporting when the query provided has syntax errors. For example, the following query submitted to this procedure passes a String rather than a Map to the apoc.periodic.iterate procedure:

CALL apoc.periodic.repeat(
    'brokenJob',
    'CALL apoc.periodic.iterate(
      "MATCH (n) RETURN n", 
      "WITH n SET n.foo = \'mark\'",
       "")',
    120)

Failed to invoke procedure `apoc.periodic.repeat`: Caused by: org.neo4j.cypher.internal.v3_5.util.SyntaxException: Type mismatch: expected Map, Node or Relationship but was String (line 4, column 9 (offset: 108))

There are also some fixes around handling of null values for the apoc.mongo.* and apoc.coll.containsAll procedures.

We hope these changes improve your experience with the library, and if you have any questions or suggestions please let us know in the APOC category of community.neo4j.com.

Cheers,
Mark and Jennifer Reif


---

Neo4j APOC Release — Export Streaming support, Delete custom procedures, New string functions was originally published in Neo4j Developer Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.