Ideal way to import csv data and create column vectors that will be variables i want to work with

Question

Ines Shekhovtsov 2023-1-23

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1899115-ideal-way-to-import-csv-data-and-create-column-vectors-that-will-be-variables-i-want-to-work-with

评论： Ines Shekhovtsov 2023-1-24

Hello,

I have a .csv file that i wish to import and create column vecotrs (variables) that i can manipulate further with my code. Below is my code. I also wish to remove NaN values from all columns that have them. In additon i want to muliply all of the EMG data columns by 1000 which is why that number is there towards the end of my code. This code seems to work but i was told to not use eval in code writing and just wanted to see if someone can check it and offer any other suggestions to code it differently.

Thank you in advance!

filename='RESP_Trial_3.csv';
column_names={'RLT','RLTx','RLTy','RLTz','RUT','RUTx','RUTy','RUTz','RIC','RICx','RICy','RICz',...
    'RPEC','RPECx','RPECy','RPECz','RPS','RPSx','RPSy','RPSz','RD','RDx','RDy','RDz',...
    'RRA','RRAx','RRAy','RRAz','ROB','ROBx','ROBy','ROBz','Analog21','Analog22','Analog23','Sync'};
EMGVariables={'RLT','RUT','RIC','RPEC','RPS','RD','RRA','ROB','Sync'};
%read the imported matrix table and create data struct to store variables
%Create variables in the workspace with the same names as column names and
%assign the corresponding columns of the matrix as their values
matrix = readmatrix(filename,'HeaderLines', 7);
data = struct();
for i = 1:numel(column_names)
    variable_name = column_names{i};
    assignin('base',variable_name, matrix(:,i));
end
%Remove NaN
for i = 1:numel(EMGVariables)
    variable_name = EMGVariables{i};
    variable = 1000*eval(variable_name);
    variable(isnan(variable))=[];
    assignin('base', variable_name, variable);
end

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Stephen23 2023-1-23

"This code seems to work but i was told to not use eval in code writing..."

EVAL and ASSIGNIN and various others:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

"...and just wanted to see if someone can check it and offer any other suggestions to code it differently"

Use a table.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

dpb 2023-1-23

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1899115-ideal-way-to-import-csv-data-and-create-column-vectors-that-will-be-variables-i-want-to-work-with#answer_1154735

在 MATLAB Online 中打开

filename='RESP_Trial_3.csv';
column_names={'RLT','RLTx','RLTy','RLTz','RUT','RUTx','RUTy','RUTz','RIC','RICx','RICy','RICz',...
              'RPEC','RPECx','RPECy','RPECz','RPS','RPSx','RPSy','RPSz','RD','RDx','RDy','RDz',...
              'RRA','RRAx','RRAy','RRAz','ROB','ROBx','ROBy','ROBz','Analog21','Analog22','Analog23','Sync'};
EMGVariables={'RLT','RUT','RIC','RPEC','RPS','RD','RRA','ROB','Sync'};
opt=detectImportOptions(filename);                  % get an import options object
opt.MissingRule='omitrow';                          % skip any incomplete rows
opt.ImportErrorRule='omitrow';                      % ditto any bad conversions...
tData=readtable(filename,opts,'HeaderLines', 7);    % and bring in the table
tData.Properties.VariableNames=column_names;        % and name variables as desired
tData{:,EMGVariables}=1000*tData{:,EMGVariables};   % and scale given subset by variable names

As the above illustrates, you can dynamically address any/all variables by name without there being a million different variables nor using very slow and difficult to debug eval expressions by the simple expedient of using a table.

Using the import object also lets you clean the table of any bad data on import with the important feature missing in the previous that the data as independent variables before was not necessarily consistent in length/location between the samples -- a missing value in one was not being compensated for in another column.

10 个评论
显示 8更早的评论隐藏 8更早的评论

Ines Shekhovtsov 2023-1-24

编辑：Ines Shekhovtsov 2023-1-24

EDIT: Seems i cannot attach the file as it is too large. Can you please let me know where i can drop the file in here? Thank you

Thank you, yes i would of course love to utilize the full potential of Matlab. I will upload the sample file here for you to see. I will outline what i am trying to do with the code below. I have written most of the code other than smoothing gyroscope data. It works yet i'm sure that i could have written it in a much more elegant and easier way.

First about the .csv file. Data starts on line 8. It contains EMG data, gyroscope data in x, y and z directions and a sync pulse column that helps us synchronize events with data. This data was collected using Trigno Discover software. EMG is sampled at 1925.9259Hz, Gyro data at 74.0741Hz, and analog sync pulse data at 2222.2222 Hz. (As a side note i am not sure why these sampling frequencies are not whole numbers). This sampling data can be found in header line 7. Emg variable names are on line 4, however they contain numbers after and i just want the variable name.

We plan to do much more with this data but here is a list of things my code currently does:

Remove NaN value from each data column since it is all sampled at different frequencies

bandpass filter all of the emg data (6th order butterworth filter 30-300Hz).

Using sync pulse i need to extract region of interest from it and align all of the EMG data and gyroscope data with it. So essentially, i would want to take a region of interest between two sync pulses and plot how the EMG looked as well as the correstponding gyroscope data all during the same time interval. This was a bit tricky to do because of the different sampling rates.

The plot should have EMG data on left and gyroscope data on right. The gyroscope data should be smoothed and plotted on same graph. Hope that makes sense.

So for example the first row on plot would have emg data plotted for bandpassed RUT variable on the left side and to the right side it will have RUTx, RUTy and RUTz variables (that have been smoothed) on one plot exactly to the right of the EMG plot.

Thank you for your time and willingness to help!

dpb 2023-1-24

"...Seems i cannot attach the file as it is too large..."

Would only need first 100 lines or less to illustrate...open it in the editor and save just a piece of it to a new file and attach that. Or, as .csv it probably would zip down to small amount although somewhat less convenient.

"...Remove NaN value from each data column since it is all sampled at different frequencies..."

That may need some consideration of how you import the data, then. Probably the first thing to do after import would be to resample all to a common time base. But it 'splains what wasn't evident in the original post about NaN and independently removing NaN data. Would need to see the file to really tell the "ideal" way given the additional information; the above was based on erroneous (or at least incomplete) assumptions about the file structure.

Ines Shekhovtsov 2023-1-24

Could i use websave to upload file to mathworks?

请先登录，再进行评论。

Ideal way to import csv data and create column vectors that will be variables i want to work with

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

10 个评论
显示 8更早的评论隐藏 8更早的评论

更多回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Ideal way to import csv data and create column vectors that will be variables i want to work with

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

10 个评论 显示 8更早的评论隐藏 8更早的评论

更多回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

10 个评论
显示 8更早的评论隐藏 8更早的评论