Using regexp to create dataset
3 次查看(过去 30 天)
显示 更早的评论
I have imported a large database using textscan(). Now I have data with 12 variables. Each observation looks like this:
5,573346285,746540138,NA,1341119065,NA,7,0,2,1341111281,"-1,-1,-1,0,-1",-0.8
These are cell data and I would like to convert them in dataset type, but my problem is that the 11th variable is a string that may contain several numbers separated by commas. I cannot use something like this regexp(datacell{1,1}{6,1}, ',\s*', 'split') because it will split the 11th variable in many different parts. Can you please suggest me a code that can make it? Thank you.
2 个评论
采纳的回答
Stephen23
2016-5-21
编辑:Stephen23
2016-5-21
fmt = [repmat('%f',1,10),'%q','%f'];
opt = {'CollectOutput',true, 'Delimiter',',', 'TreatAsEmpty','NA'};
fid = fopen('test.txt','rt');
C = textscan(fid,fmt,opt{:});
fclose(fid);
D = cellfun(@(s)sscanf(s,'%f,'),C{2},'UniformOutput',false);
This returns all of the numeric values in C{1} and C{3}:
>> C{1}
ans =
5 573346285 746540138 NaN 1341119065 NaN 7 0 2 1341111281
6 573346286 746540139 NaN 1341119066 NaN 8 1 3 1341111282
7 573346287 746540140 NaN 1341119067 NaN 9 0 4 1341111283
and those quoted strings are in C{2}:
>> C{2}
ans =
'-1,-1,-1,0,-1'
'-1,0,-1'
'-1'
The quoted strings are simply converted to numeric using sscanf (no regexp is required):
>> D{:}
ans =
-1
-1
-1
0
-1
ans =
-1
0
-1
ans =
-1
The sample file that I used is attached here (I had to create my own as you did not provide us with a sample file to work with):
6 个评论
Stephen23
2016-5-23
编辑:Stephen23
2016-5-23
@Sebastiano delre: the error you are getting "I get this error: "Error using textscan. Invalid file identifier. Use fopen to generate a valid file identifier." " has nothing to do with my algorithm at all.
That error occurs when MATLAB cannot open the file that you requested, most likely because you are passing a wrong filepath to fopen.
This commonly occurs when beginners:
- try to access a file in some folder that is not the current directory, but pass only the filename without the filepath.
- define a filepath to a file that does not exist.
- spell the filename incorrectly.
The solution is (almost always) to pass the correct filepath. You should make this change and tell me what the error message msg is:
[fid,msg] = fopen('test.txt','rt'); % for your filename and filepath!
You will also find hundreds of threads on this forum that explain this exact error message, if you want to read more information about it.
However you have also changed the file format from what you explained in your question, which will then cause my code to not work. Your question did not mention that the file has a header! You can fix this by adding 'HeaderLines',1 to the textscan options.
Or you could try this code, which generates a structure using those header names, which lets you access the data using the fieldnames:
fmt = [repmat('%f',1,10),'%q','%f'];
opt = {'CollectOutput',true, 'Delimiter',',', 'TreatAsEmpty','NA'};
[fid,msg] = fopen('test.txt','rt');
H = regexp(fgetl(fid),'[^,"]+','match');
C = textscan(fid,fmt,opt{:});
fclose(fid);
M = strrep(H,'.','');
C{1} = num2cell(C{1});
C{3} = num2cell(C{3});
C{2} = cellfun(@(s)sscanf(s,'%f,'),C{2},'UniformOutput',false);
M(2,:) = num2cell(horzcat(C{:}),1);
S = struct(M{:});
and access the data like this:
>> S(9).Sentiment
ans =
-1.3333
>> S(2).Sentiment
ans =
-0.5000
>> S(2).startingtime
ans =
1.3411e+09
PS: Sorry about the date mixup! PPS: This new code was tested on your sample file:
更多回答(2 个)
Walter Roberson
2016-5-21
If you are using one of the more recent versions of textscan then you can use the %q format to read the double-quoted string as a single item.
0 个评论
Azzi Abdelmalek
2016-5-21
编辑:Azzi Abdelmalek
2016-5-21
a='5,573346285,746540138, NA ,1341119065,NA,7,0,2,1341111281,"-1,-1,-1,0,-1",-0.8'
b=regexp(a,'\<".+\>"\,','match');
c=strrep(a,b,'');
data1=regexp(c,'[\s\,]+','split');
data2=regexp(b{1}(2:end-2),'[\s\,]+','split');
data=[data1{:} data2{:}]
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 String Parsing 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!