Unpredictable execution time of custom function using ncread

2 次查看（过去 30 天）
N/A
N/A 2015-8-3
编辑： N/A 2015-8-3
Hello everybody,
I am currently working on my Bachelors Thesis. I am reading medium to large .nc files. These files contain a global data set of wave parameters on a 180 (longitude) x 170 (latitude) grid with a 6 hour temporal resolution.
I have written a couple funtions to make the code more readable for outsiders (the company I'm working for). So I use BA_test.m which contains several sections. Each of these sections contains the needed variables to run the desired funtion; Example:
%%Hs vs Tp plot of any coordiante tuple
tic
if(exist('cnt','var')==0);
cnt = 1;
end
path = 'E:\Tobias\Daten\';
output_path = 'Z:\Studienarbeiten\Kleine_Tobias\';
kind = 'WAVE'; % WAVE , SEA , SWELL
lon = -37; % deg East; real, phys. coordinate
lat = 46; % deg North; real, phys. coordiante
[cnt] = BA_HS_TP(lon,lat,kind,path,output_path,cnt);
toc
This test program defines the counter variable cnt (used for multiple figures), the input data path, output data path, the kind of variable to be read with ncread (here 'WAVE'), lon, lat.
BA_HS_TP is this:
function [cnt] = BA_HS_TP(lon,lat,kind,path,out_path,cnt)
cur_dir = pwd;
if strcmp(cur_dir,path)==0;
cd(path);
% display(['Cd to ' path])
end
lon = lon + 181;
lat = lat + 81;
switch kind
case 'WAVE'
tmp1 = squeeze(squeeze(ncread([path 'HS_1981-2011.nc'],'Hs',[lon,lat,1],[1,1,inf])));
tmp2 = squeeze(squeeze(ncread([path 'PP_1981-2011.nc'],'WAVE_PP',[lon,lat,1],[1,1,inf])));
x_string = 'Combined wave peak period [s]';
y_string = 'Combined wave significant wave height [m]';
case 'SEA'
tmp1 = squeeze(squeeze(ncread([path 'SEA_HS_1981-2011.nc'],'SEA_Hs',[lon,lat,1],[1,1,inf])));
tmp2 = squeeze(squeeze(ncread([path 'SEA_PP_1981-2011.nc'],'SEA_PP',[lon,lat,1],[1,1,inf])));
x_string = 'Wind sea peak period [s]';
y_string = 'Wind sea significant wave height [m]';
case 'SWELL'
tmp1 = squeeze(squeeze(ncread([path 'SWELL_HS_1981-2011.nc'],'SWELL_Hs',[lon,lat,1],[1,1,inf])));
tmp2 = squeeze(squeeze(ncread([path 'SWELL_PP_1981-2011.nc'],'SWELL_PP',[lon,lat,1],[1,1,inf])));
x_string = 'Swell peak period [s]';
y_string = 'Swell significant wave height [m]';
end
cmap = load('Histogram_cmap.mat'); % load custom colormap, in order to show the low probability variations in the Hist
tmpp = double([tmp1,tmp2]);
figure(cnt);
hist3(tmpp);
view(70,40);
[n,c] = hist3(tmpp); cnt = cnt + 1;
Anzahl = sum(sum(n));
figure(cnt);
contourf(c{2}, c{1}, n/Anzahl,500);
set(gca,'ydir','reverse');
set(gca,'xaxislocation','top');
shading flat;
caxis([0 1]);
colormap(cmap.map);
cb = colorbar;
ylabel(cb,'relative frequency');
xlabel(x_string);
ylabel(y_string);
% output to excel
xlswrite([out_path num2str(lon-181) 'East_' num2str(lat-81) 'North.xlsx'],n/Anzahl);
cnt = cnt + 1;
end
The problem is, if I run this script from within the BA_test.m (I actually evalute one of several sections), the program runs very slow. Using the "Run and Time" I get around 250-360 seconds of runtime. Running the exact same code from a new script (creating a new .m-file, copy paste) yields a faster execution (~3 seconds). I could understand this by believing there is something dumb I do within the section of the BA_test.m, BUT! When I run this new .m-file for a different lon, lat configuration this new .m-file shows the same, ultra slow execution.
I hope I was able to describe my problem accurately. Of course I have checked the time-consuming code line using the profiler. It turns out if the function runs slow, it is due to the "+netcdf\private\netcdflib (MEX-file)". I googled for anything concerning this topic and only found issues where people were dealing with differences in execution time between for example Matlab and R.
Since this slowdown greatly influences my workflow I would be very very thankful for any hints or potentially useful forums threads!
Cheers Tobias
PS: I have atteched the two files and for your information, the .nc-files to be read are about 11GB, but thats not an issue if I only read specified parts of the .nc-file by using ncread('filename','var_name',[start_x,start_y,start_z],[count_x,count_y,count_z]);
EDIT: added profiler output as pdf.

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Fourier Analysis and Filtering 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by