CUSOLVER_S​TATUS_INTE​RNAL_ERROR with RTX 4080 Super in MATLAB 2022a C MEX Function

21 次查看（过去 30 天）
Stefanos
Stefanos 2024-6-20
评论： Stefanos 2024-6-26
Hello,
I'm encountering a problem when running a CUDA-based C MEX function in MATLAB 2022a on a machine equipped with an RTX 4080 Super GPU. The specific error occurs when executing the line:
cusolverStatus_t status = cusolverDnCreate(&handle);
Here are the details:
  • The MEX function compiles successfully using the mexcuda command.
  • The same code operates correctly on this machine when using an RTX 2080Ti GPU with exactly the same software setup.
  • When the RTX 4080 Super is used, this error appears, even though the function runs without issues in Visual Studio, indicating that CUDA (Version 12.5) is properly configured on my system.
  • Upgrading to MATLAB 2024a resolves the issue, suggesting a compatibility problem with MATLAB 2022a. However, I am restricted to using MATLAB 2022a due to dependencies on other external software.
Additional context:
  • NVIDIA Driver Version: 555.99
  • CUDA Toolkit Version: 12.5
Has anyone else experienced similar issues with newer NVIDIA GPUs in MATLAB 2022a? Any insights or suggestions on how to resolve or circumvent this error with the RTX 4080 Super would be greatly appreciated.
Thank you!
  1 个评论
Umar
Umar 2024-6-21
Hi Stefanos, It seems that the compatibility issue lies between MATLAB 2022a and the RTX 4080 Super GPU. Upgrading to MATLAB 2024a resolves the problem, indicating a version-specific conflict. Since you are constrained to MATLAB 2022a, consider checking for any MATLAB updates or patches that address GPU compatibility issues. Additionally, ensure that all CUDA libraries and drivers are up to date. You may also try adjusting compiler options or CUDA configurations specific to the RTX 4080 Super GPU. Seeking support from MathWorks or NVIDIA forums could provide further insights or workarounds tailored to your setup.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Joss Knight
Joss Knight 2024-6-24
编辑：Joss Knight 2024-6-25
This is a known issue with CUDA 11.2, Ada cards, and cusolver. The solution is to upgrade MATLAB to R2023a or above, because it requires a new version of CUDA which cannot be provided in an update release.
It is possible that you can be walked through dropping a fixed version of cusparse and cusolver from CUDA 11.3, that would be compatible with MATLAB R2022a and not broken. However, it would require you to edit your MATLAB installation. (SPOILER: This turned out to be the only solution that worked.) Upgrading is definitely preferable.
Since this is a mex function, you have the option to compile using your own CUDA toolkit rather than the one shipped with MATLAB. To do this you will need to make some edits to mex options in order to allow mexcuda to use a version of the toolkit that doesn't match the version of MATLAB. I can walk you through this if this is your preferred route. At the moment mexcuda is not using the 12.5 tookit you installed because the version is wrong.
EDIT: Actually, I think I know how to force mexcuda to compile with your toolkit without any options file changes:
mexcuda -v CUDA_LIB_PATH='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\lib\x64' CUDA_ROOT='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5' LIBDEVICE='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\nvvm\libdevice' cusolver_handle_mex.cu
The -v will help you work out whether the correct version of nvcc was used. What I can't be sure of is that using CUDA 12.5 alongside CUDA 11.2 won't have unforeseen problems. If it does you might be better off installing CUDA 11.3 instead, which will be compatible with R2022a but won't have the cusolver bug.
  18 个评论
显示 16更早的评论
Stefanos
Stefanos 2024-6-26
Yes, it would have been nice to not have to hack it this way but since it works it is a good solution for the time being until we can use a newer Matlab release. Thank you again!

请先登录，再进行评论。

更多回答（1 个）

Joss Knight
Joss Knight 2024-6-24

It's probably necessary for you to provide your code, since any number of things could have gone wrong prior to intializing cusolver which could prevent it from intializing successfully. For instance, did you start your mex function by calling mxGPUInit?

  2 个评论
Stefanos
Stefanos 2024-6-24
I am not sure what mxGPUInit as I cannot seem to find any documentation on it. Could you point me to a page or reference? Also, it seems weird that the same code works on the 2022a version with another GPU or with the actual GPU I have the issue with the 2024a version. In addition, I have never had to use this previously mxGPUInit and have been working with MEX GPU code for quite a while.
Here is a simplified version of my MEX code that produces the error in question:
#include "mex.h"
#include <cusolverDn.h>
#include <cuda_runtime.h>
#include <cstdio>
// Function to convert cuSOLVER status to a string
const char *cusolverGetErrorString(cusolverStatus_t status)
{
switch (status)
{
case CUSOLVER_STATUS_SUCCESS:
return "CUSOLVER_STATUS_SUCCESS";
case CUSOLVER_STATUS_NOT_INITIALIZED:
return "CUSOLVER_STATUS_NOT_INITIALIZED";
case CUSOLVER_STATUS_ALLOC_FAILED:
return "CUSOLVER_STATUS_ALLOC_FAILED";
case CUSOLVER_STATUS_INVALID_VALUE:
return "CUSOLVER_STATUS_INVALID_VALUE";
case CUSOLVER_STATUS_ARCH_MISMATCH:
return "CUSOLVER_STATUS_ARCH_MISMATCH";
case CUSOLVER_STATUS_MAPPING_ERROR:
return "CUSOLVER_STATUS_MAPPING_ERROR";
case CUSOLVER_STATUS_EXECUTION_FAILED:
return "CUSOLVER_STATUS_EXECUTION_FAILED";
case CUSOLVER_STATUS_INTERNAL_ERROR:
return "CUSOLVER_STATUS_INTERNAL_ERROR";
case CUSOLVER_STATUS_MATRIX_TYPE_NOT_SUPPORTED:
return "CUSOLVER_STATUS_MATRIX_TYPE_NOT_SUPPORTED";
default:
return "Unknown cuSOLVER error";
}
}
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
// Print initial diagnostic information
mexPrintf("Starting cuSOLVER handle initialization.\n");
// Declare a cuSOLVER handle
cusolverDnHandle_t handle;
// Initialize the cuSOLVER handle
cusolverStatus_t status = cusolverDnCreate(&handle);
if (status != CUSOLVER_STATUS_SUCCESS)
{
mexErrMsgIdAndTxt("cusolver:initializationFailed",
"CUSOLVER initialization failed: %s", cusolverGetErrorString(status));
}
mexPrintf("cuSOLVER handle initialized successfully.\n");
// Destroy the cuSOLVER handle
status = cusolverDnDestroy(handle);
if (status != CUSOLVER_STATUS_SUCCESS)
{
mexErrMsgIdAndTxt("cusolver:destructionFailed",
"CUSOLVER destruction failed: %s", cusolverGetErrorString(status));
}
mexPrintf("cuSOLVER handle destroyed successfully.\n");
}
In addition here is the compilation command used:
% Set CUDA_PATH to the appropriate path
% setenv('MW_CUDA_FORWARD_COMPATIBILITY', '1');
setenv('CUDA_PATH', 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5');
% Add CUDA binaries to the system PATH
setenv('PATH', [getenv('CUDA_PATH') '\bin;' getenv('PATH')]);
% Compile the MEX function
mexcuda -v cusolver_handle_mex.cu ...
-L"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\lib\x64" ...
-I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\include" ...
-lcusolver -lcudart -lcublas

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 GPU Computing 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by