Iterate to filter signal in a parfor

Question

Luca Pecoriello 2019-12-11

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/496202-iterate-to-filter-signal-in-a-parfor

评论： Ridwan Alam 2019-12-13

LowPassFilter.mat

在 MATLAB Online 中打开

Hi everyone

I am using MATLAB 2015 and I am using a student version with all toolboxes installed.

I have a third-dimensional order tensor containing the data coming from a thermographic measurement (in the form of pixel-pixel-time) and I want to apply a custom made low pass filter which I have created with the matlab filter designer. I want filter every signal (so the signal corresponding to every pixel in time) and I want to use a parfor.

In one .m code I am defining the following command:

test_hyper_cube = zeros(512,640,3000)
hyper_cube_filtered_lowpass = lowPassHyperCube(test_hyper_cube,LP); %% LP is a filter object

%% Then I define the following code in another external function that should process each signal from each pixel in time. The function receives 2 inputs: inputdata which is a 3rd order tensor and filterVar which is filter object

function [outputData] = lowPassHyperCube(inputData,filterVar)
outputData = [];
parfor i=1:size(inputData,1)
    tempvar = [];
    for j=1:size(inputData,2)
        newvar = filter(filterVar,squeeze(inputData(i,j,:))-mean(squeeze(inputData(i,j,:))))+mean(squeeze(inputData(i,j,:)));
        newvar = reshape(newvar,1,[]);
        tempvar = [tempvar;newvar];
    end
    outputData(i,:,:) = tempvar;
    disp(['Iteration n.', num2str(i)])
end
end

But if I run it I get the following error:

Error using lowPassHyperCube>(parfor body) (line 9)
An UndefinedFunction error was thrown on the workers for 'filter'.  This might be because the file containing 'filter' is not
accessible on the workers.  Use addAttachedFiles(pool, files) to specify the required files to be attached.  See the documentation
for 'parallel.Pool/addAttachedFiles' for more details.
Error in lowPassHyperCube (line 6)
parfor i=1:size(inputData,1)
Caused by:
    Undefined function 'filter' for input arguments of type 'struct'.

Any idea why?

Let me know you can try this out. Attached you have the mat file containing the filter object I have created but you shouldn't need any specific data, you can just try with fake ones.

Greetings

Luca

3 个评论
显示 1更早的评论隐藏 1更早的评论

Luca Pecoriello 2019-12-13

编辑：Luca Pecoriello 2019-12-13

在 MATLAB Online 中打开

Hi.

Sorry for my late reply, but I was doing some more in-depth tests.

I am still not 100% sure. I have noticed that the computation gets stuck at some point. Maybe you can help me out understanding some points.

Here is the code, following the integration of your suggestions. Because in a parfor it is not possible to use the "clear" command to clear a variable, I am freeing some memory after each iteration with the following lines

inputVector = [];

and

newvar = [];

This is the following code. Maybe can you try it and tell me what could be eventually improved?

function [outputData] = lowPassHyperCube(inputData,filterObj)
tic
outputData = [];
filterVar = filterObj.Numerator;
disp('Preparation for filtering data with parallelized loop');
parfor i=1:size(inputData,1)
    
    tempvar = [];
    disp(['Filtering data at outer loop step: ', num2str(i)]);
    
    for j=1:size(inputData,2)
        
        inputVector = squeeze(inputData(i,j,:));        
        newvar = filter(filterVar,1,inputVector-mean(inputVector))+mean(inputVector);
        newvar = reshape(newvar,1,[]);
        tempvar = [tempvar;newvar];
        disp(['Filtering data at internal loop step: ', num2str(j)]);
        inputVector = [];
        newvar = [];
        
    end
    outputData(i,:,:) = tempvar;
    tempvar = []; %% In a parfor the variable cannot be cleared
    disp(['Filtering iteration n.', num2str(i)])
    
end
toc
disp('EOP. Data have been filtered');
end

For your information, this is the PC I am using. I am actually surprised the code gets stuck, seen the powerful hardware I have

Name: ASUS DESKTOP PC G11CB_G11CD Series. DESKTOP-I3TV9GG
Processor: Intel(R) Core(TM) i7-6700 CPU @ 3.40 GHz - 3.40 GHz
RAM 32.0 GByte

Greetings

Luca

Ridwan Alam 2019-12-13

Luca, I don't think it has much to do with freeing up those temp memory. You can certainly try to not use newvar, inputVector at all. I used those only for ease of reading.

Maybe you can try to upgrade the Matlab version. Earlier versions needed something called matlabpool. I am not sure you need that or are already using that.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Ridwan Alam 2019-12-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/496202-iterate-to-filter-signal-in-a-parfor#answer_406048

编辑：Ridwan Alam 2019-12-12

在 MATLAB Online 中打开

The error is caused by the way global digital filter objects are handled by filter() in parfor loop.

function [outputData] = lowPassHyperCube(inputData,filterObj)
    outputData = [];
    filterVar = filterObj.Numerator;
     
    parfor i=1:size(inputData,1)
        tempvar = [];
        for j=1:size(inputData,2)
            inputVector = squeeze(inputData(i,j,:));
            
            newvar = filter(filterVar,1,inputVector-mean(inputVector))+mean(inputVector);
            newvar = reshape(newvar,1,[]);
            tempvar = [tempvar;newvar];
            
        end
        outputData(i,:,:) = tempvar;
        disp(['Iteration n.', num2str(i)])
    end
end

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Iterate to filter signal in a parfor

3 个评论
显示 1更早的评论隐藏 1更早的评论

回答（1 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Iterate to filter signal in a parfor

3 个评论 显示 1更早的评论隐藏 1更早的评论

回答（1 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

3 个评论
显示 1更早的评论隐藏 1更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论