I have two huge files. The driver file contains 30 Million shipment details. The other file contains the correction details of each Shipment. Based on a specific correction criteria of a Shipment, I have to build the before and after image correction details.
To identify the specific correction criteria, I will have to read through all the previous corrections for the Shipment and then I have to build the before and after image.
Currently I built this logic in Python with Pandas dataframe. This logic works well with 300K records and it takes less than 5 mins for the same. But if I want to try it out for the real production file with 30+ million records, it runs for a very long time in my VDI machine.
Is this something that could be achieved in Neo4j?
