ASCII (.csv) file data extraction

6 次查看（过去 30 天）
Lukas
Lukas 2012-4-6
I am just relearning MATLAB coding for my work environment to streamline data analysis. I am trying to extract numerical values from a .csv that has a larger header of string information in it. Along with the data, there is information such as the frequency of which the measurements were taken at within the header. I am trying to write an extract function tailored for our needs but I am having difficulty getting started. I am not sure whether or not to use textread, csvread, etc. Everything that I have tried thus far doesnt seem to work as errors come up. Below is an example of the header followed by the data
SetupTitle, CV US Protocol
PrimitiveTest, C-V Sweep
TestParameter, Channel.UnitTyp, CMU
TestParameter, Channel.Unit, CMU1:MF/SC
TestParameter, Channel.IName,
TestParameter, Channel.VName, VBias
TestParameter, Channel.Mode,
TestParameter, Channel.Func, VAR1
TestParameter, Channel.Index,
TestParameter, Channel.Time,
TestParameter, Measurement.Primary.Locus, Single
TestParameter, Measurement.Primary.Scale, LINEAR
TestParameter, Measurement.Primary.Start, -10
TestParameter, Measurement.Primary.Stop, 20
TestParameter, Measurement.Primary.Step, 0.1
TestParameter, Measurement.Aborting.Condition, CONTINUE AT ANY
TestParameter, Measurement.PostOutput.Value, START
TestParameter, Measurement.PostOutput.Retention, OFF
TestParameter, Measurement.Secondary.ACLevel, 0.03
TestParameter, Measurement.Secondary.FName, Freq
*TestParameter, Measurement.Secondary.Frequency, 1000000, 100000, 10000, 1000*
...
AnalysisSetup, Analysis.Setup.Preference.ScalarVisible, true
AnalysisSetup, Analysis.Setup.Title, CV US Protocol
Dimension1, 301, 301, 301
Dimension2, 4, 4, 4
DataName, VBias, C, G
DataValue, -10, 7.5466E-11, 8.1216499999999987E-06
DataValue, -9.9, 7.61209E-11, 7.47184E-06
DataValue, -9.8, 7.65261E-11, 9.57089E-06
So as you can see, the header is full of useless information until the bold line that consists of the frequencies that each measurement was taken at. The subsequent data, as shown above, then reads as shown and the only interesting pieces I need at the 'VBias" and the 'C'. The other funny issue is that these measurements were saved in the same file consecutively so the order that the frequencies were listed in is the order of the sweep data...meaning the data is continuous from one frequency to the next without any header and the only way you know when this happens is the VBias goes from +10V to -10V in the sweep. Anyway, I would appreciate any help extracting this info into a useful array that I can then write manipulate. Thank you in advance.
[Merged information from duplicate question]
I have a .csv file with a large header. On a line in the header, I need to extract the frequencies at which I performed a measurement. For example:
TestParameter, Measurement.Secondary.Frequency, 1000000, 100000, 10000, 1000
I need to be able to extract the 1000000, 100000, 10000 and 1000 each as a variable but the other side of this is that the number or frequencies needs to be flexible. Some measurements, I will only have one frequency, others I might have 10 and I need to be able to put them in an array of variables that I can assign their correlating data to. Any help with this would be appreciated.

请先登录，再回答此问题。

采纳的回答

per isakson
per isakson 2012-4-6
There are several alternatives. However, if the file fits in memory I would
  1. read the complete file with textscan (textread will become obsolete)
  2. extract the information I need
fid = fopen( file_spec, 'r' );
cac = textscan( fid, '%s', 'Whitespace','', 'Delimiter','\n' );
sts = fclose( fid );
cac = cac{:};
is_data = strncmp( 'DataValue', cac, 9 );
str = transpose( char( cac(is_data) ) );
str = cat( 1, str, repmat( ',', 1, size(str,2) ) );
num = textscan( str, '%*s%f%f%f', 'Delimiter', ',', 'Whitespace', '' )
This is a start that I haven't tested. See Data extraction from .txt file
  5 个评论
显示 3更早的评论
per isakson
per isakson 2012-4-6
"isolate the various frequency measurements". M according to previous comment. Mm98 = M( abs(M(:,1)-(-9.8))<0.01, : );
Lukas
Lukas 2012-4-18
Yes, ultimately what I would like to do is have a cellarray called Freq with all the frequencies listed and then assign each discrete frequency the corresponding data of only the VBias and C columns....if that is possible.

请先登录，再进行评论。

更多回答（4 个）

Walter Roberson
Walter Roberson 2012-4-6
If you use textscan() with 'CommentStyle', {'SetupTitle', 'Frequency,'} and 'Delimiter', ',' and number of lines set to 1 and format set to '%f%f%f%f' then you would read the frequencies and be positioned to start reading with the next line. You can then repeat the same sort of CommentStyle trick to read just the Dataname line.
  1 个评论
Walter Roberson
Walter Roberson 2012-4-18
If you switch the format to a single '%f' in the above textscan() construct, then you would read the first numeric value in the Frequency line. Then fgetl() the rest of that line, and textscan() with that one string variable in place of a file identifier, using 'Delimiter', ',' and a format of '%f'; that will get you the rest of the values on the line.

请先登录，再进行评论。

Walter Roberson
Walter Roberson 2012-4-18
Your existing question is still open and active here, and waiting for your responses. Please do not open duplicate questions; you can edit your existing question or reply to the responses you received. Your duplicate question has been deleted.
  1 个评论
Lukas
Lukas 2012-4-18
Sorry, I just needed to add that the number of variables to extract will vary. I still am not clear as how it works. I am trying to understand the more detailed workings of textscan in order to figure out how to start the extraction of variables and have it continue to the endofline but I dont see how to do it.

请先登录，再进行评论。

Lukas
Lukas 2012-4-19
I still havent been able to figure out if it is possible to extract the data (VBias and Capacitance) as a 2 column array that I can separate into separate arrays by using boolean logic of looking at the VBias step size and when the steps size of the next point exceeds the previous point, start a new variable. Can anyone help with this point?
  1 个评论
Walter Roberson
Walter Roberson 2012-4-19
When you are positioned at the beginning of the first DataValue line:
celldata = textscan(fid, 'DataValue,%f%f%*f', 'Delimiter', ',', 'CollectData', 1);
YourArray = celldata{1};

请先登录，再进行评论。

Lukas
Lukas 2012-4-19
fid = fopen(uigetfile({'*.csv';'*.txt';'*.dat'}, 'Select the CV File'));
Is this a valid way to select and open a file? I get a -1 response from fopen indicating it cannot open the file. I dont know if thats a file format problem or a coding problem
  2 个评论
Walter Roberson
Walter Roberson 2012-4-19
In order for that to work, the file would have to be in the same directory.
It would be more robust to test the returned value from uigetfile() in order to determine whether the user canceled the uigetfile() operation.
It would be clearer to specify the 'r' or 'rt' flag in the fopen() call.
Lukas
Lukas 2012-4-23
It would have to be in the same directory even though the uigetfile prompt lets you change directory to select the file? And isnt 'r' the default if nothing else is specified?

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Text Files 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by