Readtable For Loop for importing large quantities of csv files in to Matlab

13 次查看(过去 30 天)
Hello,
I am currently working on a project which requires me to import sequential data into matlab so that it can be processed and analysed as one data set. There is multiple csv files because the data logger creates a new file every 40mb or so.
I have put together the following code to import the data.
if true
% code
end
clear all
close all
d = uigetdir();
filePattern = fullfile(d, '*.csv');
file = dir(filePattern);
x={};
for k = 1 : numel(file)
baseFileName = file(k).name;
fullFileName = fullfile(d, baseFileName)
x{k} = readtable(fullFileName);
end
There are 4 things I have not done yet.
1. Preallocate memory for x - I don't know how much to preallocate.
2. I am not appending the data in to one matrix. Rather, a cell array combining each set of data. For Example 3 sets of data creates a 1x3 cell array each including a table of the data.
3. Added a status print in cmd window i.e: "read file xyz"
4. Not added datetime format because the first column has cotents like this: "31/03/2017 07:58:31"
Here is an example of the output.
In this case what I actually need is a 64,803 x 191 table (Need to keep variable names - each row is a specific variable).
How do I achieve this?
For any given data set I will always have the same amount of columns but the number of rows may vary in each file.

采纳的回答

Guillaume
Guillaume 2018-10-10
1. Replace
x = {};
by
x = cell(1, numel(file)); %better name for that variable advised. How about: tables
Note that unless the number of files is significant, this won't have any noticeable impact as it only allocates memory for the cells, not for the tables that go into the cells. This is normal and how cell arrays work.
2. That's fine and is the proper way to do it
3. After the readtable:
fprintf('read file %s\n', fullFileName);
4. Assuming all the tables have the same variable names, to concatenate them all into one table, after the loop:
bigtable = vertcat(x{:});
  4 个评论

请先登录,再进行评论。

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Data Type Conversion 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by