How do I ensure that the number of delimiters on each row are constant when reading a hige csv file

Question

S968 2018-9-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/418855-how-do-i-ensure-that-the-number-of-delimiters-on-each-row-are-constant-when-reading-a-hige-csv-file

评论： dpb 2020-4-18

I have a huge csv file using comma. Using the readtable function, I keep getting the error message: ''All lines of a text file must have the same number of delimiters.''.

rawdata = readtable(filename, 'Delimiter', ',');

How can I ensure that each row has the same number of delimiters?

4 个评论
显示 2更早的评论隐藏 2更早的评论

dpb 2018-9-13

That's the job of the writer of the file; do you have any control over that?

Is it possible the problem is that there are headers or multiple sections within the file that needs must be handled specifically? Or are there strings with embedded punctuation perhaps that weren't quote-delimited and that's causing interpretation as multiple fields?

So many possibilities, so little fact on which to work...as noted earlier, need to see a portion of the file that contains the problem to be able to know anything at all for sure.

You can always brute-force check, of course --

fid=fopen('yoursuspectfile.csv','r');  % open the file
l=fgetl(fid);                          % read first line
n1=sum(l==',');                        % count delimiters first record
i=1;                                   % record counter
disp(sprintf('Record %5d has %5d delimiters',i,n1))
while ~feof(fid)                       % rest of file
  i=i+1;                               % increment counter
  l=fgetl(fid);
  if sum(l==',')~=n1                   % not same as first record
    disp(sprintf('Record %5d has %5d delimiters',i,n1))
  end
end
fid=fclose(fid)

Masood Anzar 2020-4-18

How to handle if the any row of column is blank

I have a table containing data like 't=1139692479071' in each row of first column.

But some of the row are blank. I want to split the data using '=' as separator.

When I run this command ===> split(T{:,1},'=')

This error is coming : Element 2687 of the text contains 0 delimiters while the previous elements have 3.

All elements must contain the same number of delimiters.

Please advice me how to apply delimiter if the data have blank elements. I cannot remove blank rows as there is data in other columns also.

dpb 2020-4-18

You'll still have to either

Insert a dummy text into blank rows like 'T=nan'
Or only select the rows that have the delimiter to split

logical addressing/indexing is your friend in such cases.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Viren Gupta 2018-9-24

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/418855-how-do-i-ensure-that-the-number-of-delimiters-on-each-row-are-constant-when-reading-a-hige-csv-file#answer_338045

Making sure that each line of the input file has same number of delimiters is what the person who created this file has to do. You can only check if each line of the file has same number of delimiters. For this, you can use 'fgets()' function which gives you a string and then use 'strsplit' function based on ',' as your delimiter and then check the size of the cell array returned. If all such cells have same size, then each row has same number of delimiters else not.