I often deal with reasonable sized amount of time series data, 50-200 million doubles with associated time stamps and would like to visualize them dynamically.
Is there existing software to do this effectively? How about libraries and data formats? Zoom-cache is one example of library focusing on large time series. In Zoom-cache the data is summarized at several resolutions to make it easier to view at different resolutions.
Edit: Also if there is anywhere else I should ask this question or seek an answer please let me know.
Best Answer
Sorry for self-ad, but Thunderplot (thunderplot.com) would be good candidate for it. I wrote it exactly for interactive plotting of large datasets. I didn't test it with 200KK rows datasets, but it works fine with ~10KK rows. Also there's "read every Nth row" feature, so you can reduce amount of data to store/visualize. I can send you the registration key in exchange to one of those large datasets. 🙂
Similar Posts:
- Solved – Clustering data into bins of variable sizes
- Solved – Impractically long running time PCA command in R RStudio
- Solved – Impractically long running time PCA command in R RStudio
- Solved – Pros and Cons: Methods for Detrending Time Series Data
- Solved – Pros and Cons: Methods for Detrending Time Series Data