Reading a variable that have different names in different data files

MA
2021 10 2
2 个回答
回答已采纳
更新时间：2021 10 12
6 次查看（30 天）

0 个投票

assuming I am using the following lines to read some data files that have a table structure
for k = 1 : nfiles
fullFileName = fullnames{k};
%reading the file as a table
opts = detectImportOptions(fullFileName);
getvaropts(opts,{'YEAR','MONTH','DAY','HOUR','MIN','GDALT','NE8'});
opts.SelectedVariableNames = {'YEAR','MONTH','DAY','HOUR','MIN','GDALT','NE8'};
t = readtable(fullFileName,opts);
end
later on I have discovred that some files do not have the variable NE8 or GDALT, or they have them under different names, how can I exclude the files that does not have these variables and not read them and how can I read them from different files under differnet names ?

2 个评论
显示 无 隐藏 无

Walter Roberson
Walter Roberson 2021-10-2
detectImportOptions and then examine the variable names returned in the options structure, to see whether it has the variables you need
getvaropts(opts,{'YEAR','MONTH','DAY','HOUR','MIN','GDALT','NE8'});
That line is not doing anything useful for you -- not unless you remove the semi-colon so that you can display the output.
MA
MA 2021-10-2
编辑：MA 2021-10-2
This is the error I am getting
Error using matlab.io.ImportOptions/getNumericSelection (line 518)
Unknown variable name: 'POP'.
Error in matlab.io.ImportOptions/set.SelectedVariableNames (line 178)
rhs = getNumericSelection(obj,rhs);
Error in extractingtables (line 30)
opts.SelectedVariableNames = {'YEAR','MONTH','DAY','HOUR','MIN','GDALT','NE8','POP'};
where in one of the files we have such variable as seen in the inage below:
There are files that contain the NE8 variables , files that contain it under the name POP and files that does not have it at all. I am getting errors for both the second case and third case since not all the files have 'POP' and sone files does not have 'NE8'

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

Walter Roberson
Walter Roberson 2021-10-2

0 个投票

The assumption below is that if GDALT is missing then you do not want the file, but that if NE8 is missing then you can use POP interchangably, and that if both NE8 and POP are missing then you do not want the file.
needed_vars = {'YEAR','MONTH','DAY','HOUR','MIN','GDALT'};
wanted_one_of_vars = {'NE8', 'POP'};
opts = detectImportOptions(fullFileName);
vars = opts.VariableNames;
if all(ismember(needed_vars, vars)) && any(ismember(wanted_one_of_vars, vars))
if ismember(wanted_one_of_vars{1}, opts)
NE8_varname = wanted_one_of_vars{1};
else
NE8_varname = wanted_one_of_vars{2};
end
else
continue; %or whatever you need to do for files that do not have all the variables
end
opts.SelectedVariableNames = [needed_vars, {NE8_varname}];
t = readtable(fullFileName,opts);

3 个评论
显示 1更早的评论 隐藏 1更早的评论

MA
MA 2021-10-10
I am getting the following error :
Error using ismember>ismemberR2012a (line 186)
Input A of class char and input B of class matlab.io.text.DelimitedTextImportOptions must be the same class, unless one is double.
Error in ismember (line 95)
lia = ismemberR2012a(A,B);
Error in extractingtables (line 38)
if ismember(wanted_one_of_vars{1}, opts)
Walter Roberson
Walter Roberson 2021-10-10
needed_vars = {'YEAR','MONTH','DAY','HOUR','MIN','GDALT'};
wanted_one_of_vars = {'NE8', 'POP'};
opts = detectImportOptions(fullFileName);
vars = opts.VariableNames;
if all(ismember(needed_vars, vars)) && any(ismember(wanted_one_of_vars, vars))
if ismember(wanted_one_of_vars{1}, vars)
NE8_varname = wanted_one_of_vars{1};
else
NE8_varname = wanted_one_of_vars{2};
end
else
continue; %or whatever you need to do for files that do not have all the variables
end
opts.SelectedVariableNames = [needed_vars, {NE8_varname}];
t = readtable(fullFileName,opts);

请先登录，再进行评论。

更多回答（1 个）

0 个投票

Have you tried this ways of reading data, e.g.:
for k = 1 : nfiles
fullFileName = fullnames{k};
%reading the file as a table
opts = detectImportOptions(fullFileName);
t{k} = readtable(fullFileName,opts);
end
That results in k number of tables residing in a cell array variable called t that can be separated via another step.

3 个评论
显示 1更早的评论 隐藏 1更早的评论

MA
MA 2021-10-2
Thank you for the quick reply. but this will read the whole file for me, isn't it? because I need only specific variables to be read from a file, (there are 18 variables at least in each files and I need anly 7 variables) and reading a whole file would not be effecient when taknig into account that we have more 11k files. correct me if I am wrong.
Yes, indeed what you are saying is correct. Maybe in this case, you had better import just part of the data by indexes and ignore the rest without using the variable names since they are not consistent for all data files.
MA
MA 2021-10-2
yes I thought of that as well but the issue is the columns of the variables 'GDALT'and 'NE8' are having different places in different files, at one file they may be the 6th and 8th column at another they may be the 10th and 15th, that is why I am using there names in the first place, but now NE8 appear in a deffirent name as well in some files.

请先登录，再进行评论。

类别

在 帮助中心 和 File Exchange 中查找有关 Tables 的更多信息

产品

版本

R2021b

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by