Simulink: database toolbox connection fails in Parallel Computing parfor loop?

2 次查看（过去 30 天）
Hua Kun Tan
Hua Kun Tan 2023-9-29
评论： Raymond Norris 2023-9-29
I am trying to use mutlithreaded database updates in Simulink using the update function inside parfor
My code is something like this:
parfor i = 1:numLoad
updateQuery = strcat("{""$set"":", jsonencode(document(i)), "}");
findQuery = strcat("{""name"": """, document(i).name, """}");
temp_conn = dbConnections{i};
update(temp_conn, "collection", findQuery, updateQuery);
end
where dBConnections is initialised like this:
for i = 1:5
dbConnections{i} = mongoc("localhost",27017,"database"); % Replace with your connection setup
end
However, this crashes.
It will not crash if I either (1) change parfor to for, or (2) remove the update function.
Is the mongoc connection just not usable in parfor?
The error messages are:
Warning: A worker aborted during execution of the parfor loop. The parfor loop will now run again on the remaining workers. > In distcomp/remoteparfor/handleIntervalErrorResult (line 246) In distcomp/remoteparfor/getCompleteIntervals (line 396) In parallel_function>distributed_execution (line 746) In parallel_function (line 578) In my_model>Outputs (line 118)
An error occurred while running the simulation and the simulation was terminated
Caused by:
  • Error evaluating registered method 'Outputs' of MATLAB S-Function 'my_model' in 'my_model/Level-2 S-Function For MongoDB'. The following is the MATLAB call stack (file names and line numbers) that produced this error: ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [195] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [259] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [396] ['C:\Users\me\MATLAB\Projects\learning_simulink\work\my_model.m'] [118]
  • All workers aborted during execution of the parfor loop
Any ideas for what the issue is?

请先登录，再回答此问题。

回答（1 个）

Raymond Norris
Raymond Norris 2023-9-29
To summarize:
  • You're running a single Simulink model (my_model), in which you have a block (Level-2 S-Function for MongoDB) that's running MATLAB code
  • In this MATLAB code, you have serial code that runs a for-loop to create an array of database connections (but really all the same db connection)
  • And you then run a parfor-loop, such that each loop iteration updates a collection with a set of documents a Mongo DB
A couple of thoughts:
  2 个评论
Hua Kun Tan
Hua Kun Tan 2023-9-29
Hi, thank you, you have summarised it quite well.
The array of db connections was an attempt to figure out if the issue was caused by multiple threads using the same connection.
I attempted to use parallel.pool.Constant to solve my issue, however, the same errors persist.
conn = mongoc("localhost",27017,"database"); % ip, port, database name
constant_conn = parallel.pool.Constant(conn);
parfor i = 1:numLoad
updateQuery = strcat("{""$set"":", jsonencode(document(i)), "}");
findQuery = strcat("{""name"": """, document(i).name, """}");
update(constant_conn.Value, "collection", findQuery, updateQuery);
end
An error occurred while running the simulation and the simulation was terminated
Caused by:
  • Error evaluating registered method 'Outputs' of MATLAB S-Function 'my_model' in 'my_model/Level-2 S-Function For MongoDB'. The following is the MATLAB call stack (file names and line numbers) that produced this error: ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [195] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [259] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [396] ['C:\Users\me\MATLAB\Projects\learning_simulink\work\my_model.m'] [122]
  • All workers aborted during execution of the parfor loop.
The baffling thing is that replacing parfor with for runs perfectly.
Do you have any luck attempting to run mongodb updates in a parfor loop?
Raymond Norris
Raymond Norris 2023-9-29
Change the creation, as such
constant_conn = parallel.pool.Constant(@() mongoc("localhost",27017,"database")); % ip, port, database name
This will create the connection on the workers, instead of creating it from the client and then passing it to the workers.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Reporting and Database Access 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by