Import a thousand of AscII files, combine them in one XYZ text file and export them again as one text file?

Question

Majid Mohamod 2017-5-4

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/338750-import-a-thousand-of-ascii-files-combine-them-in-one-xyz-text-file-and-export-them-again-as-one-tex

编辑： Walter Roberson 2019-7-18

HI,

I have a thousands of ASCII files which start with the name "CDR_19830101z.asc" and end with the name "CDR_20160101z.asc". I am trying to call them all in Matlab, and export them as one combined XYZ text or excel file.

These files are averages rainfall per day, of a specific area. The X and Y value are the same of all the files. The only value which changes is the Z value which consider the the average rainfall per day. So I am trying to build a graph between the time and rainfall of each square within this study area.

The file CDR_19830101z.asc format is:

ncols     6
nrows    4
xllcorner 41.500
yllcorner 35.500
cellsize 0.25
NODATA_value -99
0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00

The last file "CDR_20160101z.asc" format:

ncols     6
nrows    4
xllcorner 41.500
yllcorner 35.500
cellsize 0.25
NODATA_value -99
1.19 2.14 0.89 2.86 3.18 5.77
1.24 4.41 1.39 2.42 2.87 4.51
1.27 1.82 1.29 1.60 1.24 2.88
1.37 2.43 3.26 2.44 2.80 2.13

I appreciate your help and time.

Thanks, Majid

...[cleaned up/formatted data--dpb. NB: Use the {}Code button to format such lines in future]...

2 个评论
显示无隐藏无

dpb 2017-5-4

Are all dates present or are there missing files? IOW, do you need to store the date info in the file or can it be inferred from the position knowing first entry date/time?

Just to be certain we know what we're talking about, I presume the 24 entries are annual totals for a given geographical subsection within the total area as given by the LL corner coordinates?

Majid Mohamod 2017-5-9

编辑：Majid Mohamod 2017-5-9

Raster2XYZ.m

There is no missing in the data. Actually I found the script which load the Raster ASCII and convert it to XYZ.csv file. But I want to modify this script to be as the following:

Apply this script on different files (12000 files) and auto save with any name (It's not important the name for now) with format csv.

I was thinking that if we can use the loop fucntion in MatLab and apply this script on each file one after one!

The script which I use is:

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2017-5-9

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/338750-import-a-thousand-of-ascii-files-combine-them-in-one-xyz-text-file-and-export-them-again-as-one-tex#answer_266365

在 MATLAB Online 中打开

projectdir = uigetdir('Select a directory');
dinfo = dir( fullfile(projectdir, '*.asc') );
filenames = fullfile( projectdir, {dinfo.name} );
for K = 1 : length(filenames)
  this_file = filenames{K};
    now convert the one file named by this_file
end

21 个评论
显示 19更早的评论隐藏 19更早的评论

Majid Mohamod 2017-5-10

编辑：Majid Mohamod 2017-5-10

在 MATLAB Online 中打开

The code work very well, but the problem is that I have to select the file and save the output. There is a way to select and save the output automatically? I appreciate your help and time.

I merged your code with the script which I have as shown bellow:

projectdir = uigetdir('Select a directory');
dinfo = dir( fullfile(projectdir, '*.asc') );
filenames = fullfile( projectdir, {dinfo.name} );
for K = 1 : length(filenames)
  this_file = filenames{K};
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                       RASTER TO XYZ
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                 Produced by Joseph Wheaton 
%                   Last Updated: 18 September 2007                 
%                                                                %
%                                                                %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 
%
% This script reads in an ASCII raster file and then outputs a CSV file in
% XYZ format with a point (row) for each grid cell. It is slow because
% most of the IO part of the code can not be vectorized, but it does
% the trick.
%------Read in Data -------------------------------------------------
% Prompts User to select the most recent DEM file name
[filename, pathname]=uigetfile('*.asc',strcat('Load the ASCII raster you wish to convert.'));    
filename_NewDEM=[pathname filename];
fid=fopen(filename_NewDEM,'r');      % opens file to fid in read only mode
dum1=fscanf(fid,'%s',1);            % assigns first part of header (ncols) to dummy variable dum1 (%s is string notation)
nx=fscanf(fid,'%u',1);              % stores actual number of collumns to nx (%u is decimal notation)
dum2=fscanf(fid,'%s',1);            % assigns second line header info (nrows) to dummy variable dum2
ny=fscanf(fid,'%u',1);              % stores actual number of rows to ny
dum3=fscanf(fid,'%s',1);            % assigns third line header info (xllcorner) to dummy variable dum3
xll=fscanf(fid,'%f',1);             % stores actual lower left x coordinate corner to xll (%f is fixed point notation)
dum4=fscanf(fid,'%s',1);            % assigns fourth line header info (yllcorner) to dummy variable dum4
yll=fscanf(fid,'%f',1);             % stores actual lower left y coordinate corner to yll
dum5=fscanf(fid,'%s',1);            % assigns fifth line header info (cellsize) to dummy variable dum5
lx=fscanf(fid,'%f',1);              % stores cell size in lx
dum6=fscanf(fid,'%s',1);            % assigns sixth line header info (nodata) to dummy variable dum6
nodata=fscanf(fid,'%f',1);          % stores the no data ARC tag to nodata
Z=fscanf(fid,'%f',[nx,ny]);      % stores all of the cell data in a double array dimensioned according to nx and ny
fclose(fid);                        % closes the file
fprintf('Done reading ARC data.\n');
fprintf('\n')
%------Make the X,Y,Z Arrays -------------------------------------------------
xleft = xll + (0.5*lx);
ytop =  yll + (ny*lx) - (0.5*lx);
xPos = xleft;
yPos = ytop;
fprintf('Processing points... be patient. \n');
for i=1:ny;              % Begin main gridcell loop (rows)
    for j=1:nx;          % Begin gridcell loop (collumns)
        if ((Z(j,i) == nodata) | (Z(j,i) == 0));
            X(j,i) = nodata;  
            Y(j,i) = nodata;  
        else    
            X(j,i) = xPos;  
            Y(j,i) = yPos;  
        end
        % Advance for next collmn
        xPos = xPos + lx;
    end                 % End gridcell loop (collumns)
    % At end of row reset xPos
    xPos = xleft;
    % and advance yPos to next row
    yPos = yPos - lx;
end  
%------Write the Output File ---------------------
% SORT OUT FILE NAME AND PATH:
[filename,pathname]=uiputfile('*.csv','Choose a name to save the XYZ file to');    % Select final file name
csv_file_name=[pathname filename '.csv'];   
fprintf('Creating file... be patient. \n');
%Write DoD data to a temporary file in ARC format
fid1= fopen(csv_file_name, 'w');
% Write Header
fprintf(fid1,'Easting,Northing,Elevation\n');
    i=0;
    j=0;
    for i=1:ny;              % Begin main gridcell loop (rows)
        for j=1:nx;          % Begin gridcell loop (collumns)
            if (Z(j,i) ~= nodata);
                fprintf(fid1, '%12.3f,%12.3f,%6.3f\n', X(j,i), Y(j,i), Z(j,i));  
            end
        end                 % End gridcell loop (collumns)
    end                     % End main gridcell loop (rows)
fclose(fid1);
fprintf('It worked... imagine that. Done writing %s', csv_file_name);
% DONE
end

Majid Mohamod 2017-5-15

编辑：Majid Mohamod 2017-5-15

在 MATLAB Online 中打开

Please, there is a small problem it's just showed up. When I try to convert the file with the following format:

ncols     11
nrows    15
xllcorner 41.000
yllcorner 33.500
cellsize 0.25
NODATA_value -99
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00
-99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00 -99.00