Data processing for different size matrixes

Question

amberly hadden 2016-4-12

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/278634-data-processing-for-different-size-matrixes

评论： amberly hadden 2016-4-20

Hello everyone, I was wondering if someone could guide me for “from where to start?” I have almost 20 data files each one is of different size (e.g., 13000*6 2000*6 5000*6 etc). Now I want to produce a 3D plot from this data. The first value in each data set is a time which is different in a sense that each data file has different time interval for readings e.g., at 0.1s, some at 0.5s interval some at 1s some at 2 etc. I tried rounding of time and to pick one data set at round figures (e.g., 1s interval) but it is affecting results because readings or at very small time intervals? I would appreciate it very much if someone could help me pass this step with possible accuracy.

2 个评论
显示无隐藏无

Star Strider 2016-4-12

‘The first value in each data set is a time ...’

Do you mean the first row, the first column or something else?

You imply that you want to combine them. You might be able to interpolate them to the same times for all of them, depending on the data.

How would you want to plot them?

amberly hadden 2016-4-12

编辑：amberly hadden 2016-4-13

Time is the first column and next columns are parameters i.e., results of experiments for different locations . I'm interested in producing a 3D model out of this data as I have time (or depth), lat, lon, parameter. So I need one size for all matrixes to mesh and then produce a model. Thank you star once again for being so helpful.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Star Strider 2016-4-12

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/278634-data-processing-for-different-size-matrixes#answer_217642

‘So I need one size for all matrixes to mesh and then produce a model.’

I’m still not quite certain where you’re going with your data, but if you want them all to have the same time base (and all time bases begin with zero or some specific number), I would choose the smallest absolute time interval, defined by ‘t(end)-t(1)’ for each data set, define the sampling interval you want (the time difference between any two consecutive samples), and then use the interp1 function to limit them to a common time base without extrapolating.

Read the documentation for the interp1 function, experiment with it on created (not actual) data to understand how it works, then use it on your data.

If all goes well, this will work!

22 个评论
显示 20更早的评论隐藏 20更早的评论

Star Strider 2016-4-14

在 MATLAB Online 中打开

I apologise for the delay. Life interferes.

I interpolated the data by depth, but the depth vector is rather narrow to avoid extrapolating the shorter depth records, and non-overlapping depth records. It is easier to limit the longer depth records to the shortest depth interval, in order to avoid extrapolating and inserting a series of NaN values at the ends. Extrapolating is bad regardless.

There is now one common depth vector, ‘dep_intrp’, with the rest of your ‘xyz’ values interpolated to match those depths in the ‘xyz’ matrix:

[D,S,R] = xlsread('amberly hadden inpolygon.xlsx');
dxyzn = D(~isnan(D));
dxyz = reshape(dxyzn, 14, 4, []);                                       % Remove ‘NaN’ Values
dep_limm = squeeze([min(dxyz(:,1,:))  max(dxyz(:,1,:))]);
dep_intrp = linspace(max(dep_limm(1,:)),  min(dep_limm(2,:)), 20);      % Depth Interpolation Vector
for k1 = 1:3
    xyz(:,:,k1) = interp1(squeeze(dxyz(:,1,k1)), dxyz(:,2:end,k1), dep_intrp, 'linear');
end
figure(1)
for k1 = 1:3
    subplot(1,3,k1)
    plot3(xyz(:,1,k1), xyz(:,2,k1),xyz(:,3,k1), '.-b')
    grid on
end
set(gcf, 'Position', [300 400  946 288])
figure(2)
for k1 = 1:3
    subplot(3,1,k1)
    plot(dep_intrp, xyz(:,1,k1), '.-',   dep_intrp, xyz(:,2,k1), '.-',  dep_intrp, xyz(:,3,k1), '.-')
    grid
    if k1 == 1
        legend('x', 'y', 'z', 'Location','northoutside', 'Orientation','horizontal')
    end
end
xlabel('Depth')

The ‘figure(1)’ plot simply plots ‘x’, ‘y’, and ‘z’. I have no idea what you want to plot, or how you want to plot it. I just did these plots to be sure the interpolation yielded what appear to be decent data.

Star Strider 2016-4-15

在 MATLAB Online 中打开

My pleasure!

The data in your file were 14 rows by 12 columns (after I eliminated the NaN values) that, owing to their format, I reshaped into a (14x4x3) matrix.

I’m not certain what your actual data set is, or how it is organised. I’m having problems analysing the numerical data to attempt to deduce the structure of your file. Your original matrix in ‘D’ has 1314866 elements. The difference between that and ‘dxyzn’ are the number of NaN values. Since ‘dxyzn’ has 397256 elements, divided by 4 (the number of columns in each ‘depth|x|y|z’ matrix segment), you should be able to reshape it to a (99314x4) matrix. I would do this reshape call:

dxyz = reshape(dxyzn, 99314, 4);

The problem then is to split them up into the appropriate number of depth segments. I have to leave that to you, because I have no idea how your data are organised. If you have the same number of rows in each depth segment (as in the file you posted), then you can change the reshape call to do that easily, just as I did with your posted file. Otherwise, you need to create a series of cell arrays. That isn’t difficult from a programming perspective, but does complicate your analysis.

If you can provide a bit more detail as to how your file is organised, I might be able to provide more exact coding for how to use the reshape function with it.

amberly hadden 2016-4-20

Star hello - here is the more refine idea (Figure in attachment). I have x-y and z values and parameter to model. I want to break x and y in 0.005 intervals and z in 0.1m interval. This will generate very small tiny cells for x-y and z. I'm then trying to use my data file and trying to get values on each 0.005 and 0.1 intervals. So let say I have an area and depth first I want to divide area in very small celss and then depth intervals. Any data which falls in or on boundery of cell will be part of that specific cell. That is why I start breaking x and y in 0.005 intervals and z in 0.1 intervals. But I did my best to get out put which is again nothing else but zero unfortunately. [

a = xlsread('test.xlsx'); all = a(:,1:4); %% % space = 0.005; x = (min(a(:,1)):0.005:max(a(:,1))); y = (min(a(:,2)):0.005:max(a(:,2))); zcells =(min(a(:,3)):0.1:max(a(:,3))); %% dimx = length(x) - 1; dimy = length(y) - 1; dimz = length(zcells) - 1; dimmodel = dimx*dimy*dimz; [X,Y]=meshgrid(x,y);

clear count clear i clear space %% Sort the Data into Cells and out put the mesh file for parameter percentile = 1.0; model = zeros(dimmodel,1); % clc timet = 0; if percentile > 0.9 && percentile <= 1.0 C = 1; elseif percentile > 1.0 percentile <= 0.0 errordlg('The percentile must be between 0 and 1.0!'); return else C = 0; end for i = 1:dimy tic; disp(['y-index ',num2str(i),' of ',num2str(dimy)]); for j = 1:dimx for k = 1:dimz cellno = dimx*dimz*(i-1)+dimz*(j-1)+k; emin = x(j); emax = x(j+1); nmin = y(i); nmax = y(i+1); zmin = zcells(k); zmax = zcells(k+1); % m = 1; % while m <= length(all) indices = find(all(:,1) >= emin & all(:,1) <= emax &... all(:,2) >= nmin & all(:,2) <= nmax &... all(:,3) >= zmin & all(:,3) <= zmax); % m = m + 1; % end if ~isempty(indices) addition(1,:) = all(indices,4); addition = sort(addition); start = length(addition)-ceil(length(addition)*percentile)+C; if start < 1 start = 1; end useadd = addition(start:length(addition)); count = length(useadd); model(cellno,1) = sum(useadd)/count; else model(cellno,1) = 0; end clear addition clear useadd clear count end end time = toc; timet = timet + time; clc disp(['Time for run: ',num2str(floor(time/60)),' minutes and ',num2str(((time/60)-floor(time/60))*60),' seconds']); disp(['Estimated time remaining: ',num2str(floor(time*(dimy-i)/3600)),' hours and ',... num2str(((time*(dimy-i)/3600)-floor(time*(dimy-i)/3600))*60),' minutes']); end

dlmwrite([num2str(dimx),'x',num2str(dimy),'x',num2str(dimz),'_test.txt'],... model,'precision',7,'newline','pc'); ]

Star Strider 2016-4-20

在 MATLAB Online 中打开

I still cannot follow your code. (Also, please format it. Use the {}Code button at the top of the text entry window.)

1. Please do not do this:

all = a(:,1:4);

The all function is a logical function that is important in doing vector comparisons. You are ‘overshadowing’ it, so any call to it will reference the variable and not the function. The solution is simply to rename it to something that is meaningful in the context of your code. (Calling it ‘Data’ is usually my choice.) The MATLAB Editor makes a global-search-and-replace relatively easy.

2. Again, please check this line to be certain it does what you want:

indices = find(all(:,1) >= emin & all(:,1) <= emax &... all(:,2) >= nmin & all(:,2) <= nmax &... all(:,3) >= zmin & all(:,3) <= zmax);

Even though you ‘overshadow’ the all function earlier, you might actually need it (or the related any function) here. Experiment by choosing one comparison, looking at the contents of the column, the value you are comparing it to, and the result of that comparison.

For instance:

Comparison_1 = all(:,1) >= emin
Comparison_2 = all(:,1) <= emax
Cmp_1_and_Cmp_2 = Comparison_1 & Comparison_2

Do the same for the others, and examine the results of these operations. Go through all the logic in your find call systematically to see what the results are, and then make the appropriate changes if the results are not what you want.

3. After you get your logic sorted, read through the documentation on the various interpolation functions, since it seems to me that you want to do some sort of interpolation. See particularly the interpolation functions: interp2, interp3, scatteredInterpolant, and choose whatever interpolation function is best for your data.

Regardless, do not extrapolate! Extrapolating will destroy the credibility of your data.

Star Strider 2016-4-20

My pleasure.

It’s really not that much, but it does require that you code those comparisons to do what you want them to do. I have no idea what you want to do, so I can’t offer any specific suggestions.

The first thing I would do would be to rename your ‘all’ matrix to something that does not overshadow the all function, since you may need it. You can do that in the next five minutes.

You really do need to see what your comparisons are doing, and if they are doing what you want them to. This is especially relevant to the find call. You may want to break up the find call into several different find calls, then use the appropriate set functions, such as intersect, to select the indices you want.

Best of luck on your finals! I’m certain you will do well!

amberly hadden 2016-4-20

Thank you star I needed this to produce results for a presentation and if I spend time on it I would not be able to prepare for exams. The surprising thing is the program I sent you with the exception of 'all' is working for some other data set which means I'm having some error in defining intervals of may be missing something perhaps mesh(x,y,z)? or something else may be. I would work on it after exam. Best regards, Amb

请先登录，再进行评论。

Data processing for different size matrixes

2 个评论
显示无隐藏无

采纳的回答

22 个评论
显示 20更早的评论隐藏 20更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

Data processing for different size matrixes

2 个评论 显示 无隐藏 无

采纳的回答

22 个评论 显示 20更早的评论隐藏 20更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

2 个评论
显示无隐藏无

22 个评论
显示 20更早的评论隐藏 20更早的评论