How to run 100's of rules on Graph data?

[ckayay]

I have historical transaction data. The requirement is to run rules to evaluate if the new transaction is evaluating as true based on the historical data.

For eg: A new Payment transaction will evaluate the following rules:

• If payment is made by credit card
• If there were multiple payment failures (considering historical data)
• If the credit card in the payment is used by some other account.

I will need to know all the rules that is evaluated as true.

There are above 100 relationship based rules that need to run on such historical data (for several years). Thus, I was assuming to use a Graph DB to store the data, but not sure how to run 100s of rules on the full data set each time performantly?

One option was: Store the data in Graph-based DB, Run complex SparQL (covering all the rules) each time, but in this case, I would not know which rule returned true.

Any ideas, please? Also is it possible to create such rules graphically (by end-users instead of hard coding such as Drools)

[jasperblues]

Hello @ckayay

With a graph database you can look up an initial point of interest, such as a credit card, and then traverse related entities in O(1) time complexity. This is called non-index adjacency and it is very fast.

Theoretically you could apply 100 rules, that each investigate 1000 related entities in a reasonable amount of time - that's 100,000 items with no additional index hits. (Hitting an index will perform in something like O(log n) time complexity, which is slower).

However, care must be taken to structure your data in such a way that traversals are reasonable, and to avoid 'super nodes' <--- That's a node that has so many relationships that traversing them will no longer happen expediently.

Max Demarzi writes a very good blog post on that topic: https://maxdemarzi.com/2015/08/26/modeling-airline-flights-in-neo4j/

These are some initial thoughts to start the discussion. Hope it helps.

EDIT: Forgot to add. Storing rules in the DB, if possible, affords the opportunity to apply which ones are applicable and when.