General question, should I use indexing

2 次查看（过去 30 天）
Brandy
Brandy 2023-7-11
评论： Steven Lord 2023-7-11
I have a lot of field data and will be getting more in the future. I'm not great at matlab but I'm trying to get better and improve my code past the terrible (call in each file I want for 50 or more days of data) code I currently have. My question (in example form) is this: I have over 40 days of radiometer data, now I need to read in each day file and then manipulate the data, for example there are 4 columns with "flags" I need to write some code saying that when certain flags=1 then only grab (1:2:end,:) data. Would it be better to collate all of the days of data into one large matrix and try indexing by the date and messing with the data that way, or would it be better to write a for loop which reads in each day and manipulates the data in each day sequentially? Does it matter? I don't really know how to do either of these methods and I'm just trying to decide which method to spend my time on. So any advice would be appreceated.
  2 个评论
Stephen23
Stephen23 2023-7-11
Generally it is easier to process data when it is stored in fewer arrays. More arrays just means more loops.
Perhaps a table might be suitable storage, then you can use inbuilt functions to process the groups of data:
https://www.mathworks.com/help/matlab/matlab_prog/perform-calculations-by-group-in-table.html
Steven Lord
Steven Lord 2023-7-11
Because of this sentence fragment I'd suggest a timetable instead of a table.
"try indexing by the date"
They're very similar, and you can do most if not all of what you can do to a table with a timetable, but timetable also lets you do some time-based processing (like retime and synchronize) that would be trickier with a table.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Neev
Neev 2023-7-11
Hey Brandy
So. whenever dealing with a large amount of field data, you will always have two options:-
1. Collate the data into one large matrix:
- Read each day's data and concatenate them vertically into a single matrix.
- Manipulate the data using indexing and logical operations to filter based on the flags.
- This approach allows for efficient indexing and manipulation of the entire dataset, but it can be memory-intensive.
2. Use a for loop to process each day's data sequentially:
- Read each day's data and process it within a loop.
- Manipulate the data within each iteration, filtering based on the flags.
- This approach is memory-efficient but may be slower compared to the first approach.
So, I don't want to focus your mind on any one out of the two, but you can go through how optimised your work might become if you implement either of the approach from above. Now, it is for you to choose the approach based on factors such as memory constraints, computational efficiency, and code organization. Optimize your code by vectorizing operations and using MATLAB's built-in functions effectively. Start with a smaller subset of data to test and optimize your code before applying it to the entire dataset.
  1 个评论
Brandy
Brandy 2023-7-11
Hey Neev,
Thanks for the answer and advice. I'm glad I was at least right that these are the two main ways to approach it and I'll have to just see which one works better for my setup. Also, I didn't know about vectorising so thanks and I'll look into that too.
Cheers,
Brandy

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Language Fundamentals 的更多信息

标签

产品


版本

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by