Modeling large dataset with high frequency timestamps

[EleniAp]

Hi, I am looking to model a dataset that contains measurements, lets say around 30 parameters, such as pressure, temperature etc., for every millisecond over the past 3 years. The time is present asa timestamp. Is Neo4J the appropriate platform for this? I have seen people suggesting using the timestamp as a property for each measurement but I feel this will lead to an excessive number of nodes. The data is stored in hourly change log files. Would it be more better to keep the data in the files and just create a node for each file?

Thanks

[glilienfield]

Have you considered a time-series database instead? This article references a few.

https://www.trustradius.com/time-series-databases

[EleniAp]

That was my initial thought but I need to model relationships to other data, such as an event that took place during two timestamps. Most timeseries databases I've seen don't have foreign key capabilities.

[glilienfield]

Got it. Neo4j can do that. There are 31,536,000,000 ms per year and you have three years of data, so 90 billion nodes.

The website states trillions of relationships, but it doesn’t seem to be achieved by a simple deployment. Maybe a neo4j staffer or someone with experience with scaling can provide feedback.

https://neo4j.com/product/neo4j-graph-database/scalability/