How to organize a datastore with multiple devices for multiple days?

3 次查看（过去 30 天）
Magsud Hasanov
Magsud Hasanov 2022-5-22
回答： Magsud Hasanov 2022-6-14
So, I have a data from 75 smart meters for range of 17 days.
Each day all 75 smart meters did some measurements.
How it is better to organize the data? I really like the idea of datastore() in Matlab. But don't know how to do it automatically for so many variables(tables)
If I create a timetable for each day of each smart meter, it would require me to create 75*17 =1275 separate tables.
And also I would like to analyze each day (with all smart meter measurements combined) so it will make the total number 1275 +17 = 1292.
Can you suggest a good option of organizing such big amount of data?
I am analyzing it for load forecasting using Machine Learning.
Thank you very much in advance!
  1 个评论
dpb
dpb 2022-5-22
That's really not much data at all in terms of observations -- unless there are gigabytes of data which it wouldn't seem there could be but a few measurements from each device.
I'd just collect the whole dataset in memory in a timetable and use retime and similar functions to do operations over the data. memapfile is just going to get in the way here.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Magsud Hasanov
Magsud Hasanov 2022-6-14
I just used array with all paths to all files and made datastore(), then did readall()

更多回答（1 个）

Seth Furman
Seth Furman 2022-5-30
Provided your data are already stored in files on disk in a common format, you can create a tall timetable and perform operations on it as if it were a large timetable.
https://www.mathworks.com/help/matlab/import_export/tall-arrays.html
  1 个评论
Magsud Hasanov
Magsud Hasanov 2022-6-13
Hello, after making path into all files, I just used datastore() and then readall(). It created the big dataset. The amount of data was enough small to use readall(). Thank you very much for your time!

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Descriptive Statistics 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by