Submit query and pick up the results later?

[chrszrkl]

I have long running queries which create/change a lot of nodes and relationships and take they usually take a few minutes to complete. I run these queries from a serverless function on AWS Lambda, so I'm charged on the execution time and also limited on how long each function can run at most.

The function creates the connection to the database and submits the query, but then all heavy work is done by the database and the function actually just waits for it to return the result. I was wondering if it is possible to submit the query as a unit of work, then close the connection and stop the function, and later pick up the results?

[mike_r_black]

I haven't tried this but I'm wondering if the query to execute was a script file, then in Aws your just telling the DB to execute the file but the lambda doesn't actually need to stay connected.

[chrszrkl]

Do you mean to start the script with apoc.cypher.runFile(file or url,[{config}]) as query from AWS Lambda, or to start the script with cypher-shell?

In the first case, my expectation was that query would be cancelled if Lambda closes the database connection, and I wouldn't be able to retrieve the results of the query.

[mike_r_black]

It's nothing I've researched but just brainstorming, but you're on the track that I'm thinking. Is it possible to invoke asynchronous the execution of query, then if you need the results of the output, you cutie always dunno the results to S3 and use S3 file creation triggers to tell you when the file arrives.

Otherwise you're just going to have to pay for the lambda to stay on until results are delivered. Or another idea, ssh of the ec2 hosting the DB to execute a script from the ec2 which could keep the active connection to the DB for however long it takes for the query to run.

[chrszrkl]

Interesting idea! I did a quick search and found the [apoc.export.*.query] procedures (apoc.export.json.query - APOC Documentation) to export the result of a query in JSON or CSV to a given file.

I'd prefer to connect via drivers to the database instead of ssh-ing directly into the EC2 instance. I would need to figure out if I can execute the query and close the connection without terminating the submitted query.