PCT GPU Computing: CUDA error was: setting the device when a process is active is not allowed

4 次查看（过去 30 天）
Evgeny Pr
Evgeny Pr 2013-1-25
编辑： Matt J 2016-12-21
Hi, everyone!
It seems to me that there is a problem in Parallel Computing Toolbox (PCT) with CUDA operation. The problem is that the setting of CUDA device in PCT fails, if CUDA device has been set up earlier within the same process by someone else, therefore, I cannot use PCT functionality, working with CUDA in my application which is also used by CUDA.
I will give you a simple example. Here it is:
% Starting MATLAB Environment...
% 1. Load a DLL file used by CUDA (or MEX file, it does not matter, only for example)
loadlibrary('MyComputingLib.dll', 'MyComputingLib.h', 'alias', 'cuda_lib')
% 2. perform calculations...
% 3. try to use PCT CUDA...
gpuDevice()
% ... and the error message appears:
An unexpected error occurred during CUDA execution. The CUDA error was: setting the device when a process is active is not allowed.
If the aforementioned actions are performed in the reversed order, the error does not occur (My software works correctly with CUDA device setting). Why cannot MATLAB set up CUDA device, when, starting with CUDA 4.0 version, it is possible to set up CUDA device a multiple number of times within one process?
I think this limitation is the rudiment of what has been left from the old CUDA versions (old derivers), when it was impossible to set up a device a multiple number of times after the process start.
Thank you in advance for your answer.

请先登录，再回答此问题。

采纳的回答

Joss Knight
Joss Knight 2016-12-20
In MATLAB versions up to R2016b, MATLAB's built-in commands must be the first activity on each device within the MATLAB process after a device reset. If you are calling into code that makes a CUDA API call on any device before MATLAB does, MATLAB will error unless you call cudaDeviceReset() on that device before MATLAB tries to access it.
This happens because MATLAB sets device flags when switching to or resetting a device, and this is only allowed before any kernels have run after a reset.
As some have pointed out, if you have only one GPU you can work around this by making sure MATLAB (or any MATLAB workers in a pool) uses the GPU before any of your custom functionality. The more robust solution for your own multi-gpu custom code is to call cudaDeviceReset() on each device you have used before you return control to MATLAB.
This behaviour will no longer occur in the next version of MATLAB.
  3 个评论
显示 1更早的评论
Joss Knight
Joss Knight 2016-12-21
That's a surprise. I wouldn't have expected that to happen. Is this the same "Cannot set device on active process" error?
Matt J
Matt J 2016-12-21
编辑：Matt J 2016-12-21
I did not keep track of the error messages, but I don't recall seeing that particular one.
I've been surmising that the GPU isn't distinguishing between MATLAB instances. If external CUDA code in MATLAB session #1 uses the GPU first, then subsequently MATLAB built-ins in session #2 use it, it is as if the same order of events took place in the same session.
Anyway, I would be grateful for some detailed documentation/guidelines on how external CUDA code (using loadlibrary etc...) can interact safely with the Optimization Toolbox and maybe also communicate gpuArray data to each other. Our lab has regular need of this.

请先登录，再进行评论。

更多回答（1 个）

Matt J
Matt J 2013-11-26
编辑：Matt J 2013-11-26
I seem to be experiencing the same thing. Hope TMW can comment...
However, if what I'm seeing is the same as you, a simple workaround is to do something like this in startup.m
a=gpuArray(1); clear a
I find that as long as gpuArray is the first thing to use the GPU in the MATLAB session, things seem to work normally.
  1 个评论
David Pate
David Pate 2016-12-5
编辑：David Pate 2016-12-5
I also had this problem in Matlab 2016a, and I'm using a solution similar to yours. In my startup file I have:
try
gpu = gpuDevice;
clear gpu;
catch
disp('No GPU device found during startup');
end

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Get Started with GPU Coder 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by