Readtable For Loop for importing large quantities of csv files in to Matlab
14 次查看(过去 30 天)
显示 更早的评论
Hello,
I am currently working on a project which requires me to import sequential data into matlab so that it can be processed and analysed as one data set. There is multiple csv files because the data logger creates a new file every 40mb or so.
I have put together the following code to import the data.
if true
% code
end
clear all
close all
d = uigetdir();
filePattern = fullfile(d, '*.csv');
file = dir(filePattern);
x={};
for k = 1 : numel(file)
baseFileName = file(k).name;
fullFileName = fullfile(d, baseFileName)
x{k} = readtable(fullFileName);
end
There are 4 things I have not done yet.
1. Preallocate memory for x - I don't know how much to preallocate.
2. I am not appending the data in to one matrix. Rather, a cell array combining each set of data. For Example 3 sets of data creates a 1x3 cell array each including a table of the data.
3. Added a status print in cmd window i.e: "read file xyz"
4. Not added datetime format because the first column has cotents like this: "31/03/2017 07:58:31"
Here is an example of the output.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/197554/image.png)
In this case what I actually need is a 64,803 x 191 table (Need to keep variable names - each row is a specific variable).
How do I achieve this?
For any given data set I will always have the same amount of columns but the number of rows may vary in each file.
0 个评论
采纳的回答
Guillaume
2018-10-10
1. Replace
x = {};
by
x = cell(1, numel(file)); %better name for that variable advised. How about: tables
Note that unless the number of files is significant, this won't have any noticeable impact as it only allocates memory for the cells, not for the tables that go into the cells. This is normal and how cell arrays work.
2. That's fine and is the proper way to do it
3. After the readtable:
fprintf('read file %s\n', fullFileName);
4. Assuming all the tables have the same variable names, to concatenate them all into one table, after the loop:
bigtable = vertcat(x{:});
4 个评论
更多回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!