Matlab does not see CUDA GPU
7 次查看(过去 30 天)
显示 更早的评论
Colleagues, hello!
After updating of the Nvidia driver 390.65 to 419.69 Matlab (R2018b) ceased to see video cards (Quadro K6000 + 3xTesla K40c).
The CUDA Toolkit updating from v.9.1 to v.10.1, NVIDIA_Nsight_Visual_Studio, FrameWork 4.7, Visual Studio Tools, VS community & msys2 installations, and Matlab replacement to R2019a did not help.
CPU: 2x Xeon E5-2697v2
Operating System: Windows 8.1
>> gpuDeviceCount
ans =
0
32 个评论
Joss Knight
2019-4-4
Can nvidia-smi see your card?
(At Windows Command Prompt)
cd "C:\Program Files\NVIDIA Corporation\NVSMI"
nvidia-smi
Alexander Sonin
2019-4-5
Joss, thank you very much for responding to my problem. I tried to enter your magic commands. Before their first entering, driver 419.69 (optimal for Tesla k40c) and CUDA Toolkit 10.1 was installed.
And after the entering of your commands, Windows saw three Tesla k40c video cards:
Matlab also saw these video cards:
>> gpuDeviceCount
ans =
3
But the very first Quadro K6000 video card (which is connected to monitors) was not visible neither in Command Prompt nor in Matlab.
Therefore, I installed the driver 419.67 (optimal for the Quadro K6000) on top.
As a result, Windows in the Command Prompt saw all 4 video cards:
but Matlab did not see one:
>> gpuDeviceCount
ans =
0
Removing drivers 419.67, 419.69 and a clean installation 419.67 did not change the result: in the Command Prompt video cards are visible, but not in Matlab.
Alexander Sonin
2019-4-8
And finally, I again installed the newest recommended driver 419.69 from above (in the hope that at least 3 video cards will be visible). But as a result, Matlab did not see a no one video cards, while all 4 video cards were visible in Command Prompt.
Joss Knight
2019-4-8
"infoROM is corrupted"? This does not sound good. Was MATLAB running when you did this? If so, see if you get this error without MATLAB running. If you do, this is definitely nothing to do with MATLAB. I suggest contacting NVIDIA support.
Otherwise the next step is to contact MathWorks support. Something is broken.
Alexander Sonin
2019-4-9
What is "infoROM"? Is it ROM on Quadro K6000 GPU?
I get the warning " infoROM is corrupted" without MATLAB running.
NVIDIA Support bounce me to MathWorks Support
Walter Roberson
2019-4-9
Alexander Sonin
2019-4-10
Walter, thanks for the links.
At the
https://bitcointalk.org/index.php?topic=1854250.1580
recommended reflashing (reinstall? update?) the GPUs ROM (BIOS?) with NVFlash.
I am download a latest BIOS for Quadro K6000:
https://www.techpowerup.com/vgabios/164411/nvidia-quadrok6000-12288-131024
and NVIDIA NVFlash:
https://www.techpowerup.com/download/nvidia-nvflash/
but don’t know, how apply NVFlash to reflashing BIOS into GPU.
The NVFlash folder include nvflash.exe- and nvflash64.exe- files:
If to run any of them, command window will appear, similar to the Windows command prompt:
However, the entering of any command in this window is not possible.
Where can I get detailed instructions on the use of NVFlash?
Alexander Sonin
2019-4-10
编辑:Alexander Sonin
2019-4-10
I am reflashed BIOS by means of NVFlash.
Instructions for NVFlash usage:
Now GPU-Z for Quadro K6000 indicate a new latest BIOS (80.80.35.00.01) and nv-smi is without any WARNINGs:
But MATLAB still does not see the any video cards:
>> gpuDeviceCount
ans =
0
Alexander Sonin
2019-4-10
Yes, in GPU-Z the tick opposite CUDA is not present in both Quadro K6000 and Tesla K40c.
Walter Roberson
2019-4-10
If it were my system I would probably try activating the tick and then rebooting and see if anything worked.
Alexander Sonin
2019-4-11
These boxes opposite CUDA and DirectCompute (for Tesla K40c) are not clickable and the checkboxes cannot be set:
On another computer (with Win10Pro x64, Titan (also GK110) and GTX780 (also GK110)) there are ticks in all the squares (and GPUs is visible in MATALAB):
As I understand it, this is the reason for the invisibility of video cards in MATLAB. Is it possible to activate CUDA in another way, for example from the command prompt or nv-smi?
Walter Roberson
2019-4-11
Sorry, at present I have no ideas as to why you cannot enable CUDA on those devices. (Well, other than vague ideas of "firmware problems".)
Would you be able to experiment by swapping one of the cards between systems, to see whether the problem stays with the system or stays with the card ?
Alexander Sonin
2019-4-11
Yes, I be able to experiment by swapping one of the cards between systems, but first I will try to install the old driver, like 39x.xx (optimal for CUDA Toolkit 9.1 or 9.2), which present earlier, at which GPU was visible. If CUDA returns and the GPUs become visible, does it make sense to change the GPUs?
Walter, how do you suggest changing GPUs?
1. Replace one of the GPUs in a faulty system (Quadro K6000 and Tesla K40c) with another type of GPU from a working system (for example, GTX780)?
2. Remove all GPUs from a faulty system (Quadro K6000 and Tesla K40c) and put one of the same (Tesla K40c) from the available stock?
3. Take video cards one at a time out of the faulty system and see what happens?
4. In any other way?
Walter Roberson
2019-4-11
If CUDA returns when you downgrade drivers, I do not think swapping cards would help.
I would suggest exchanging one faulty K40c with a working system, so that both systems have a mix of suspected faulty and known good K40c.
Alexander Sonin
2019-4-17
Hello, Walter!
The devil pulled my leg and I did the opposite: I started by rearranging the videocards.
1. Remove one bottom Tesla K40c
In PC are present: Quadro K6000 + Tesla K40c + Tesla K40c
NVIDIA Driver 419.67
CUDA in GPU-Z not visible (both for Quadro K6000 and Tesla K40c)
In MATLAB gpuDeviceCount=0
2. GTX 780 inserted in bottom slot instead removed Tesla K40c
In PC are present: Quadro K6000 + Tesla K40c + Tesla K40c + GTX 780
Unknown where did driver 388.19 come from!!!
I did not install it!
In GPU-Z CUDA became visible for all GPUs (Quadro K6000, 2xTesla K40c, GTX 780)
And in MATLAB all GPUs became visible, but not usable:
>> gpuDeviceCount
ans =
4
>> gpuDevice(1)
Error using gpuDevice (line 26)
There is a problem with the graphics driver or with this
GPU device. Be sure that you have a supported GPU and
that the latest driver is installed.
Caused by:
The graphics driver was found, but it is too old. The
graphics driver on your system supports CUDA version
9.1. The required CUDA version is 10 or greater.
Alexander Sonin
2019-4-17
3. Video cards from all slots except the first one are extracted.
In PC are present: only Quadro K6000
388.19 NVIDIA driver stayed
In MATLAB Quadro GPU is also visible, but not usable:
>> gpuDevice()
Error using gpuDevice (line 26)
There is a problem with the graphics driver or with this GPU
device. Be sure that you have a supported GPU and that the
latest driver is installed.
Caused by:
The graphics driver was found, but it is too old. The
graphics driver on your system supports CUDA version
9.1. The required CUDA version is 10 or greater.
4. Removed 388.19 driver.
nv-smi does not work
>> gpuDeviceCount
ans =
0
Alexander Sonin
2019-4-17
5. 419.67 driver is installed (optimal for Quadro K6000 and CUDA Toolkit 10.1)
The nv-smi sees the Quadro K6000
In GPU-Z is not present CUDA.
In MATLAB Quadro GPU is not visible:
>> gpuDeviceCount
ans =
0
Alexander Sonin
2019-4-17
6. CUDA Toolkit 10.1 has been installed.
In GPU-Z PhysX added, but is not present CUDA.
In MATLAB Quadro GPU is not visible:
>> gpuDeviceCount
ans =
0
Alexander Sonin
2019-4-17
7. The Quadro K6000 was removed and the GTX 780 was installed instead.
419.67 driver disappeared.
NVSMI sees the 382.05 driver, but in Windows Programs & Components ony 3D Vision 390.41 driver is present.
In GPU-Z CUDA and everything else is present!
In MATLAB GTX 780 is visible, but not usable:
>> gpuDeviceCount
ans =
1
>> A=gpuArray([1 2 3])
There is a problem with the graphics driver or with this GPU
device. Be sure that you have a supported GPU and that the
latest driver is installed.
Caused by:
The graphics driver was found, but it is too old. The
graphics driver on your system supports CUDA version 8.
The required CUDA version is 10 or greater.
8. Driver 425.31 (optimal for Win8.1 and GTX 780) is installed on top of the existing one.
GTX 780 in MATLAB is visible and usable, with newest driver:
>> gpuDeviceCount
ans =
1
>> A=gpuArray([1 2 3])
A =
1 2 3
>> B=gather(A.^2)
B =
1 4 9
>> whos
Name Size Bytes Class Attributes
A 1x3 4 gpuArray
B 1x3 24 double
ans 1x1 8 double
>> gpuDevice
ans =
CUDADevice with properties:
Name: 'GeForce GTX 780'
Index: 1
ComputeCapability: '3.5'
SupportsDouble: 1
DriverVersion: 10.1000
ToolkitVersion: 10
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 6.4425e+09
AvailableMemory: 6.1924e+09
MultiprocessorCount: 12
ClockRateKHz: 941000
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1
All is well!
But, Quadro and Tesla video cards must be installed in the system.
9. Therefore, the GTX 780 was extracted, and the Quadro K6000 was installed again in the system. Also 398.75 driver is installed (optimal for CUDA ToolKit 9.2).
CUDA is not visible in GPU-Z.
In MATLAB:
>> gpuDeviceCount
ans =
0
Alexander Sonin
2019-4-17
10. CUDA ToolKit 9.2 with patches was installed.PhysX added in GPU-Z, but CUDA is not visible.
In MATLAB:
>> gpuDeviceCount
ans =
0
Alexander Sonin
2019-4-17
11. Reinstal NVIDIA driver to 391.29 (optimal for CUDA ToolKit 9.1)
CUDA is not visible in GPU-Z.
In MATLAB:
>> gpuDeviceCount
ans =
0
Alexander Sonin
2019-4-17
12. CUDA Toolkit 9.1 was installed instead CUDA Toolkit 9.2
And CUDA appeared in GPU-Z!
But where did the 391.29 driver go?
Instead, there is a 388.19 driver!
Apparently, this is an internal driver from the CUDA Toolkit 9.1?
And thanks to him CUDA is appeared?
In MATLAB:
>> gpuDeviceCount
ans =
1
13. Then CUDA Toolkit 9.1 patches were installed.
The situation has not changed
Alexander Sonin
2019-4-17
14 Then I began to return the Teslas into the system.
Two Tesla K40c were installed into the system.
398.75 driver is resurrected, and CUDA has disappeared in GPU-Z.
In the MATLAB video cards are not visible:
>> gpuDeviceCount
ans =
0
15. Installation of an additional fourth TeslaK40x did not change the situation.
Alexander Sonin
2019-4-17
16. Then I installed the 390.65 driver (suitable for CUDA Toolkit 9.1)
17. And reinstalled CUDA Toolkit 9.1 with patches.
388.19 driver activated and CUDA returned.
In MATLAB R2019a all GPUs are visible, but not usable:
>> gpuDeviceCount
ans =
4
>> gpuDevice(1)
Error using gpuDevice (line 26)
There is a problem with the graphics driver or with this GPU device. Be sure that you
have a supported GPU and that the latest driver is installed.
Caused by:
The graphics driver was found, but it is too old. The graphics driver on your system
supports CUDA version 9.1. The required CUDA version is 10 or greater.
>> A=gpuArray([1 2 3])
There is a problem with the graphics driver or with this GPU device. Be sure that
you have a supported GPU and that the latest driver is installed.
Caused by:
The graphics driver was found, but it is too old. The graphics driver on your
system supports CUDA version 9.1. The required CUDA version is 10 or greater.
Alexander Sonin
2019-4-17
18. Then I try enter same commands in МАТLAB R2018b, and after that (oh, wonders!) the GPU devices was earned:
>> gpuDeviceCount
ans =
4
>> gpuDevice(1)
ans =
CUDADevice with properties:
Name: 'Tesla K40c'
Index: 1
ComputeCapability: '3.5'
SupportsDouble: 1
DriverVersion: 9.1000
ToolkitVersion: 9.1000
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 1.1976e+10
AvailableMemory: 5.3635e+09
MultiprocessorCount: 15
ClockRateKHz: 745000
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1
>> gpuDevice(2)
ans =
CUDADevice with properties:
Name: 'Tesla K40c'
Index: 2
ComputeCapability: '3.5'
SupportsDouble: 1
DriverVersion: 9.1000
ToolkitVersion: 9.1000
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 1.1976e+10
AvailableMemory: 5.3635e+09
MultiprocessorCount: 15
ClockRateKHz: 745000
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1
>> gpuDevice(3)
ans =
CUDADevice with properties:
Name: 'Quadro K6000'
Index: 3
ComputeCapability: '3.5'
SupportsDouble: 1
DriverVersion: 9.1000
ToolkitVersion: 9.1000
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 1.2885e+10
AvailableMemory: 5.7037e+09
MultiprocessorCount: 15
ClockRateKHz: 901500
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1
>> gpuDevice(4)
ans =
CUDADevice with properties:
Name: 'Tesla K40c'
Index: 4
ComputeCapability: '3.5'
SupportsDouble: 1
DriverVersion: 9.1000
ToolkitVersion: 9.1000
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 1.1976e+10
AvailableMemory: 5.3802e+09
MultiprocessorCount: 15
ClockRateKHz: 745000
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1
>> A=gpuArray([1 2 3])
A =
1 2 3
At link
I was disappointed to find that the R2019a only works with a CUDA version no lower than 10.
Alexander Sonin
2019-4-17
Thus, by experience, I found:
1. Installing CUDA Toolkit on top of the installed driver can solve the GPU visibility problem (which differs from the statements made on mathworks.com).
2. In order for MATLAB to see GPUs, CUDA must be visible in GPU-Z.
3. In order for MATLAB to use GPUs, the version of CUDA Toolkit, installed on a PC, must be at least specified in the “GPU Support by Release” section.
But, I did not understand, why GPU-Z never saw CUDA when installing CUDA Toolkit 10.x.
回答(1 个)
Walter Roberson
2019-4-8
Those are Kepler architecture. The 10 series drivers dropped support for Kepler if I recall correctly.
2 个评论
Walter Roberson
2019-4-8
No, this turned out to be wrong. The Fermi architecture was dropped last year.
End of support for the Kepler mobile gpus was announced last month; https://www.techspot.com/news/79115-nvidia-end-support-mobile-kepler-gpus-starting-april.html but they should still have drivers until next year. Support for the desktop Kepler will continue after that.
Alexander Sonin
2019-4-9
Walter, thank you very much for reviewing my problem!
The Titan and GeForce 780 (Kepler GK110) GPUs are installed on another personal computer. OS Win7, driver 419.67. MATLAB2018b has seen these video cards.
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 GPU Computing 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!发生错误
由于页面发生更改,无法完成操作。请重新加载页面以查看其更新后的状态。
您也可以从以下列表中选择网站:
如何获得最佳网站性能
选择中国网站(中文或英文)以获得最佳网站性能。其他 MathWorks 国家/地区网站并未针对您所在位置的访问进行优化。
美洲
- América Latina (Español)
- Canada (English)
- United States (English)
欧洲
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
亚太
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)