parfeval with background pool fails when using Java classes

24 次查看（过去 30 天）
Jan Kappen
Jan Kappen 2024-5-22
评论： Jan Kappen 2024-5-23
I'm trying to either use an existing database connection, based on com.mysql.cj.jdbc.Driver or to get that created in a parfeval call using the background worker.
Minimal examples using all the workarounds with and without function handles, parallel.pool.Constant(), external functions, etc.
db = com.mysql.cj.jdbc.Driver
fun1 = @()db.acceptsURL("asdf")
fun2 = @(db)db.acceptsURL("asdf")
fun3 = @(c)c.Value.acceptsURL("asdf")
C1 = parallel.pool.Constant(db);
getDriver = @()com.mysql.cj.jdbc.Driver;
C2 = parallel.pool.Constant(@getDriver);
C3 = parallel.pool.Constant(@getDriverExt);
fun1() % no error, direct call, outputs false
fun2(db) % no error, direct call, outputs false
fetchOutputs(parfeval(backgroundPool, fun1, 1)) % error serializing
fetchOutputs(parfeval(backgroundPool, fun2, 1, db)) % error serializing
fetchOutputs(parfeval(backgroundPool, fun3, 1, C1)) % error serializing, The 'Value' field cannot be retrieved because the parallel.pool.Constant is invalid.
fetchOutputs(parfeval(backgroundPool, fun3, 1, C2)) % Unrecognized function or variable 'getDriver'.
fetchOutputs(parfeval(backgroundPool, fun3, 1, C3)) % Unable to resolve the name 'com.mysql.cj.jdbc.Driver'.
getDriverExt.m
function conn = getDriverExt()
conn = com.mysql.cj.jdbc.Driver;
end
Here are the results of all runs:
>> fun1() % no error, direct call, outputs false
ans =
logical
0
>> fun2(db) % no error, direct call, outputs false
ans =
logical
0
>> fetchOutputs(parfeval(backgroundPool, fun1, 1)) % error serializing
Warning: com.mysql.cj.jdbc.Driver@3662c887 is not serializable
Error using parallel.BackgroundPool/parfeval
Internal error occurred when passing the input argument of type "function_handle" to the thread-based worker:
MATLAB:Serialize
Error during serialization
>> fetchOutputs(parfeval(backgroundPool, fun2, 1, db)) % error serializing
Warning: com.mysql.cj.jdbc.Driver@3662c887 is not serializable
Error using parallel.BackgroundPool/parfeval
Internal error occurred when passing the input argument of type "com.mysql.cj.jdbc.Driver" to the thread-based worker:
MATLAB:Serialize
Error during serialization
>> fetchOutputs(parfeval(backgroundPool, fun3, 1, C1)) % error serializing, The 'Value' field cannot be retrieved because the parallel.pool.Constant is invalid.
Error using parallel.Future/fetchOutputs
One or more futures resulted in an error.
Caused by:
Error using parallel.pool.Constant/get.Value (line 191)
The 'Value' field cannot be retrieved because the parallel.pool.Constant is invalid.
>> fetchOutputs(parfeval(backgroundPool, fun3, 1, C2)) % Unrecognized function or variable 'getDriver'.
Error using parallel.Future/fetchOutputs
One or more futures resulted in an error.
Caused by:
Error using parallel.pool.Constant/get.Value (line 196)
Error occurred while creating parallel.pool.Constant on the workers.
Unrecognized function or variable 'getDriver'.
>> fetchOutputs(parfeval(backgroundPool, fun3, 1, C3)) % Unable to resolve the name 'com.mysql.cj.jdbc.Driver'.
Error using parallel.Future/fetchOutputs
One or more futures resulted in an error.
Caused by:
Error using parallel.pool.Constant/get.Value (line 196)
Error occurred while creating parallel.pool.Constant on the workers.
Unable to resolve the name 'com.mysql.cj.jdbc.Driver'.
Is it impossible to use Java classes in the workers? If no, how to do it properly?
Thanks!

请先登录，再回答此问题。

回答（1 个）

Harald
Harald 2024-5-22
Hi Jan,
for functionality to work in a background pool, it needs to be explicitly supported. For more information, please see here:
https://www.mathworks.com/help/matlab/matlab_prog/run-functions-on-threads.html#mw_1cce412a-b5cb-42a4-b0b3-335654833576
Have you tried the desired functionality in a process-based parallel pool, and would that be an alternative for your application?
p = parpool("Processes");
Best wishes,
Harald
  3 个评论
显示 1更早的评论
Harald
Harald 2024-5-22
Hi,
Have you tried the desired functionality in a process-based parallel pool to rule out other issues? For example, Unrecognized function or variable 'getDriver' does not sound like a problem specific to thread-based pools.
I suspect the problematic part is External Language Interfaces, which includes Java.
While I understand that thread-based pools would be preferable, I do not yet understand why process-based pools are not a suitable alternative. You'd admittedly have to live with the initial time it takes to open the pool.
It may be a long shot, but could timers be an alternative?
Best wishes,
Harald
Jan Kappen
Jan Kappen 2024-5-23
Hi Harald,
> While I understand that thread-based pools would be preferable, I do not yet understand why process-based pools are not a suitable alternative. You'd admittedly have to live with the initial time it takes to open the pool.
Not all users have parallel computing toolbox. backgroundPool was introduced for these reasons if I understand correctly.
> I suspect the problematic part is External Language Interfaces, which includes Java.
yes... but there's an example that uses python (an external language) which works fine via constant:
pass Python object as argument to function in "parfeval" - MATLAB Answers - MATLAB Central (mathworks.com)
(edit: this seems to use processes, not threads :()
Two cross links:
Is it possible to use parfeval and backgroundPool to write to a Mongo database in a background thread? - MATLAB Answers - MATLAB Central (mathworks.com)
(Why) Is Java not available in a thread-based environment? - MATLAB Answers - MATLAB Central (mathworks.com)
> It may be a long shot, but could timers be an alternative?
yes, maybe.
Is a Matlab AppDesigner GUI running on a different thread per default? I just realized, that GUI elements remain responsive (or at least animations still work). How does that work?

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Startup and Shutdown 的更多信息

标签

产品


版本

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by