Receiving I/O error 32 when opening another program (EnergyPlan) inside parfor

MD
2018 8 9
1 个回答
回答已采纳
更新时间：2025 1 6
9 次查看（30 天）

0 个投票

Hello everyone,
I am trying to use parralel computing to run the same program (EnergyPlan) simultaneously. EnergyPlan is a energy system tool that computes whether a given energy system is able to supply the given demand. I would like to run different energy system configurations (e.g. different amounts of wind and solar power) and use an optimization algorithm to find the best setup for the given demand.
The program is set up that it opens EnergyPlan using the "system" command to execute EnergyPlan. Afterwards the results of each run are retrieved compared to see, which system scores highest. When I run the program in a simple for-loop, it works without problems. However, when I change the for-loop to a parfor-loop, the opened EnergyPlan displays the error "I/O error 32". According to sources I found this error indicates a sharing violation. The first EnergyPlan program is executed (unfortunately the program pops up so I can always see when a run is performed) just fine and it closes again. However, there is a second EnergyPlan opened while the first one is still open and this one shows the beforementioned error. According to EnergyPlan.eu the tool can be run in parallel and other researchers have done so succesfully using python or java. Therefore, I don't know why this error occurs with Matlab.
Does anyone know how to circumvent this issue? Any help would be much appreciated.
Regards,
Markus
Minimum working example (provided EnergyPlan and toolbox for Matlab are installed):
%%Paths and Folder definitions
%Path where it is located EnergyPLAN executable.
dir.energyPlanPath = 'energyPLAN.exe';
%Path of reference file.
dir.inputFilePath = 'energyPlan Data\Data\Denmark2030Alternative.txt';
%Folder where will be output resulf of EnergyPLAN.
dir.outputFolder = 'Outputs\';
REvalue = 1e4*rand(10,2);
result = zeros(size(REvalue,1),1);
parfor i = 1:length(REvalue)
% Call EnergyPlan
annualData = energyPlan(dir.energyPlanPath,...
dir.inputFilePath,dir.outputFolder,...
'input_RES1_capacity=', REvalue(i,1),... % 1st variable to be changed
'input_RES2_capacity=', REvalue(i,2)); % 2nd variable to be changed
result(i) = annualData(55); % random value taken out
end

19 个评论
显示 17更早的评论 隐藏 17更早的评论

OCDER
OCDER 2018-8-9
https://www.energyplan.eu/useful_resources/matlab-toolbox-for-energyplan/
Is it a toolbox, or a standalone .exe that you are using via system command? If it's a matlab program, it should work WITHOUT invoking something like system('EnergyPlan.exe')
Can you copy & paste the help info for the function you want to use? Also provide the code for using parfor, minus any extra code that isn't needed.
MD
MD 2018-8-9
编辑：Walter Roberson 2018-8-13
Hey OCDER,
Thanks for your response! That is the toolbox I am using. Inside the main function "energyPlan" is the command "system(executionString)". EnergyPlan itself is a standalone program. The toolbox was created to be able to access the program via Matlab.
This is the info of the function "energyPlan":
% ENERGYPLAN Call EnergyPLAN tool from Matlab and save results.
% [A, M, H] = ENERGYPLAN('energyPlanPath','inputFilePath', 'outputFolder', param,value, ...)
% ENERGYPLAN is useful for executing one or several configurations of
% models in EnergyPLAN, save the results and generate vectors with
% annual, monthly and hourly data.
% USER CONFIGURABLE OPTIONS
% Possible param/value options are:
% 'input_RES1_capacity=' - RES1 installed capacity
% 'input_RES2_capacity=' - RES2 installed capacity
% 'input_RES1_factor=' - factor correction of RES1
% .
% .
% .
% 'Input_FOM_ind_CHP_heat='
% Other m-files required: none
% Subfunctions: changeInputEnergyPlan, readOutputEnergyPlan
% MAT-files required: none
% See also CHANGEINPUTENERGYPLAN, READOUTPUTENERGYPLAN.
I put a minimum working example (provided both EnergyPlan and the toolbox are installed) in the question.
OCDER
OCDER 2018-8-9
I guess I'm still not seeing the system(executionString). Is this something you wrong into your code, or it's a part of EnergyPlan?
What do you get if you do:
which energyPlan
When you use system, it's the same as running from the cmd of windows, and thus it's not really running from inside Matlab. Can you copy paste the line where system is used, and tell what .m file it's in?
MD
MD 2018-8-9
Hey OCDER,
I get "C:\Users\MD\Documents\MATLAB\Add-Ons\Toolboxes\EnergyPLAN Toolbox\code\energyPlan.m"
Inside that file there are the following lines:
%%Function that changes input reference file.
%Call to function that changes inputs.
changeInputEnergyPlan(inputFilePath,newInputFilePath,varargin);
%%EnergyPLAN is executed.
executionString=['"',energyPlanPath,'" -i "', newInputFilePath, '" -ascii "', outputFilePath,'"'];
system(executionString); %execution of EnergyPLAN.
%%Annual, monthly and hourly results are obtained from the output file.
[annualData, monthlyData, hourlyData]=readOutputEnergyPlan(outputFilePath);
Under "%%EnergyPlan is executed" you find
system(executionString); %execution of EnergyPLAN.
OCDER
OCDER 2018-8-10
编辑：OCDER 2018-8-10
I see what you mean now in the initial post. Hmmm... how about adding a "start" in that system command of that .m file?
https://stackoverflow.com/questions/1449188/running-windows-batch-file-commands-asynchronously
So something like this:
system(['start ' executionString])
MD
MD 2018-8-13
编辑：MD 2018-8-13
Thanks again for your response! Unfortunately, when I add start to the beginning of the command as recommended in your link, I receive a different error.
According to https://www.energyplan.eu/training/faqs/#eleven I need to write "C:\kt\EnergyPLAN\run\energyPLAN.exe -i c:\MyInputFolder\Inputfile.txt -ascii c:\MyResultFolder\MyResult.txt" to run the program from cmd. This works just fine. However, when I add "start" it shows the error "'-i' could not be found. Ensure that the name is typed correctly and repeat the process". It seems that sees "-i" as directory.
Unfortunately, I am not very familiar with using cmd and I do not know how to change the command line accordingly. Would you know? Maybe it has to do with the fact that I add quotes to my directories:
start "C:\Users\...\EnergyPlan\energyPLAN.exe" -i "C:\Users\...\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt" -ascii "C:\Users\...\EnergyPlan\Outputs\out_Denmark2030Alternative.txt"
OCDER
OCDER 2018-8-13
so this doesn't work?
start C:\kt\EnergyPLAN\run\energyPLAN.exe -i c:\MyInputFolder\Inputfile.txt -ascii c:\MyResultFolder\MyResult.txt
I see a space in your command.
-i c:\MyInputFolder\ Inputfile.txt
^
Or modifying that matlab code as suggested?
system(['start ' executionString]) %Note the space after the word start.
MD
MD 2018-8-13
The first example is just a copy of the text in the link. The command at the end is the one that I use (without the full directory as it is unnecessary and a bit long).
I tried it without "start" and as normal for-loop and it works without a problem. When I add the "'start '" to the command, I receive the command about not finding "-i". Instead of interpreting it as command, cmd seems to think that "-i" is a file and not the instruction on what to do with EnergyPlan. I inserted a screenshot of the error (unfortunately in German but the translation is in the previous comment).
OCDER
OCDER 2018-8-13
I see. If you have spaces in the full file path to the .exe and have to use quotes, then you need to also specify the title for the start window it seems.
start "title1" "C:\Users\...\EnergyPlan\energyPLAN.exe" -i "C:\Users\...\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt" -ascii "C:\Users\...\EnergyPlan\Outputs\out_Denmark2030Alternative.txt"
^^^^^^^^ title of this window
MD
MD 2018-8-13
Thanks! It seems like we are getting closer to the solution. However, I am not quite sure what you mean with title of this windows. Is there a specific title name that I have to add? When I just use "title1", I get the following error:
Warning: File 'C:\Users\...\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt' not found.
> In energyPlan (line 66)
In parallel_function>make_general_channel/channel_general (line 917)
In remoteParallelFunction (line 41)
I tried giving each run and individual temporary file (the Matlab toolbox creates a "..._tmp.txt" file for the evaluation of the system) but it still didn't work.
I also tried to eliminate all spaces to do without the qotes. Thus my command line was
start C:\Users\...\EnergyPlan\energyPLAN.exe -i C:\Users\...\EnergyPlan\Data\Denmark2030Alternative_tmp.txt -ascii C:\Users\...\EnergyPlan\Outputs\out_Denmark2030Alternative.txt
However, this gave me the error that it could not find the file since it thinks that the file is not only one directory but all of them combined:
I also receive the error I/O 103.
Once I remove "start" from the command line again, the program works just fine with a normal for-loop.
C:\Users\...\EnergyPlan\energyPLAN.exe -i C:\Users\...\EnergyPlan\Data\Denmark2030Alternative_tmp.txt -ascii C:\Users\...\EnergyPlan\Outputs\out_Denmark2030Alternative.txt
OCDER
OCDER 2018-8-13
Wait, are you using "..." in your file path, or you're just doing this to condense the file path for the posting here?
Try this in matlab, filling in the "..." with the full path to the files. The title is any title name for the window it's going to open - not important:
system('start "random_title" "c:\...\energyPlan.exe" -i "C:\Users\...\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt" -ascii "C:\Users\...\EnergyPlan\Outputs\out_Denmark2030Alternative.txt"')
Also note the apostrophe encasing EVERYTHING.
system('....')
^ ^
In my system, something like this works in Matlab:
parfor j = 1:4
system('start "blah" "C:\somePath\someFile.exe"')
end
OCDER
OCDER 2018-8-13
Just wondering, have you asked them directly how to do use their energy plan in Matlab parfor loops? They might have a document for that somewhere that's not published.
https://www.energyplan.eu/contact/
Walter Roberson
Walter Roberson 2018-8-13
The line
changeInputEnergyPlan(inputFilePath,newInputFilePath,varargin);
looks to me as if it is likely doing an assignin('caller') or assignin('base') or changing global variables.
If so then you will need to toss your code into a function that you call from the parfor loop: poofing variables into existence directly inside a parfor block gets you a sharing violation.
MD
MD 2018-8-13
I am using "..." in my file path since they contain spaces. However, in my previous post I tried to get rid of the spaces in the file paths to be able to also get rid of the quotes.
These are the combinations that work. I took out the dots now to make sure that they don't cause confusion.
1. Quotes are used. Simple for-loop.
executionString=['"',energyPlanPath,'" -i "', newInputFilePath, '" -ascii "', outputFilePath,'"'];
system(executionString); %execution of EnergyPLAN.
The executionString yields:
"C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPLAN.exe" -i "C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt" -ascii "C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\Outputs\out_Denmark2030Alternative.txt"
2. Quotes are not used (spaces are not used in file path). Simple for-loop
executionString=[energyPlanPath, ' -i ', newInputFilePath, ' -ascii ', outputFilePath];
system(executionString); %execution of EnergyPLAN.
The executionString yields:
C:\Users\HumptyDumpty\Desktop\Programmeetc\Studium\4oSemestre\MasterThesis\EnergyPlan\energyPLAN.exe -i C:\Users\HumptyDumpty\Desktop\Programmeetc\Studium\4oSemestre\MasterThesis\EnergyPlan\Data\Denmark2030Alternative_tmp.txt -ascii C:\Users\HumptyDumpty\Desktop\Programmeetc\Studium\4oSemestre\MasterThesis\EnergyPlan\Outputs\out_Denmark2030Alternative.txt
These are the combinations that don't work.
1. Quotes are used. Either for-loop or parloop. I used num2str(i) here to ensure that every title is different but also when I simply use "title1", it doesn't work.
executionString=['start "title',num2str(i),'" "',energyPlanPath,'" -i "', newInputFilePath, '" -ascii "', outputFilePath,'"'];
system(executionString); %execution of EnergyPLAN.
The executionString yields:
start "title1" "C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPLAN.exe" -i "C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt" -ascii "C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\Outputs\out_Denmark2030Alternative.txt"
The error I get says that it cannot the file. You can see the error in the screenshot above that one of the error I/O 103.
2. Quotes are not used. Either simple for-loop or parfor. This time I don't use "title1" as this is apparently only necessary when using quotes. I changed the folder names again so they don't contain blanks.
executionString=['start ', energyPlanPath, ' -i ', newInputFilePath, ' -ascii ', outputFilePath];
system(executionString); %execution of EnergyPLAN.
The executionString yields:
start C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPLAN.exe -i C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt -ascii C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\Outputs\out_Denmark2030Alternative.txt
The error is the same as before that it cannot find the file as the command seems to mix up which is simply information about what to treat the file as (like -i and -ascii) and the actual directory paths.
Both ways - quotes or without - work just fine when I don't add "start" before it and use a simple for-loop. As soon as "start" is added it cannot find the file anymore.
MD
MD 2018-8-13
I have contacted them and this was their response:
I am not sure you can run it parallel if they use they rely on the same input/outputs. If not, it should not be a problem to run to versions next to each other.
However, this is the case. There are other studies that compute simultaneously but not using Matlab.
I also contacted the creator of the toolbox. However, he has not responded yet.
MD
MD 2018-8-13
@Walter: Thanks for your input. This is the function:
function changeInputEnergyPlan(inputFilePath,outputFilePath, varargin)
if iscell(varargin{1,1})==1; varargin=varargin{1,1}; end
warning('off','all')
% Read txt into cell A
fid = fopen(inputFilePath,'rt', 'n', 'UTF16LE');
i = 1;
tline = fgetl(fid);
A{i,1} = tline;
while ischar(tline)
i = i+1;
tline = fgetl(fid);
A{i,1} = tline;
end
fclose(fid);
A{1,1}='EnergyPLAN version';
A{2,1}=[];
A=A(~cellfun('isempty',A));
% % Change cell A
for j=1:length(varargin)
if mod(j,2)== 0 % solo cojo los valores par de las entradas
tf=strcmp(A,varargin(j-1));
tf = circshift(tf,1);
if iscellstr(varargin(j))==1
A{tf}=varargin{j};
else
A{tf}=sprintf('%1.2f',varargin{j});
end
end
end
%A{18} = sprintf('%d',100);
% Write cell A into txt
fid = fopen(outputFilePath, 'wt');
for i = 1:numel(A)
if A{i+1,1} == -1
fprintf(fid,'%s', A{i,1});
break
else
fprintf(fid,'%s\n', A{i,1});
end
end
fclose(fid);
warning('on','all')
end
It is used to change the values in the .txt file that EnergyPlan reads in. So yes there could be an issue as all agents try to access this file. I changed the filename for each run using "i" to change the name of each input file and output file:
newInputFilePath =[inputFilePath(1:end-4),num2str(i),'_tmp.txt'];
outputFileName = ['out_',num2str(i),inputFilePath(k(end)+1:end)];
When I run it with a simple for-loop there is no problem. I run it and it simply creates more temporary files. However, when I run it with parfor it shows the same error that it cannot find the file as shown in the screenshot. So there might also be an assignment issue but I would only know, if the program would correctly identify the file names.
OCDER
OCDER 2018-8-13
energyPlanPath = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPLAN.exe';
origInput = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt';
inputPre = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp';
outputPre = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\Outputs\out_Denmark2030Alternative';
%
assert(~isempty(dir(energyPlanPath)), 'energyPlanPath is not valid. Check file name.');
assert(~isempty(dir(origInput)), 'origInput file is not valid. Check file name.');
%
%To prevent parfor from accessing and writing to same file, make copies and specify separate outputs
Inputs = cell(4, 1);
Outputs = cell(4, 1);
for j = 1:4
Inputs{j} = [inputPre num2str(j) '.txt'];
[Success, Msg] = copyfile(origInput, Inputs{j});
assert(Success, Msg);
Outputs{j} = [outputPre num2str(j) '.txt'];
end
%Attempt to run in parallel
parfor j = 1:4
executionString = sprintf('start "title" "%s" -i "%s" -ascii "%s"', energyPlanPath, Inputs{j}, Outputs{j});
system(executionString); %execution of EnergyPLAN.
end
If this fails, it seems like the author of the code did not program this to run in parallel if it is going to share a common file. The other people might be using a workaround to contain each instance of the energy plan in a separate container and then run multiple versions, similar to running it on multiple computers.
MD
MD 2018-8-13
@OCDER: It worked!!! The strange thing is that the loop doesn't wait for the system command to finish but continues with the readOutputEnergyPlan. Then it opens the EnergyPlan instances afterwards while Matlab has already given me an error (which makes sense since the output cannot be read when it is not there). It actually opens even more instances of EnergyPlan than there are logical cores at the same time. Furthermore, it even opens more than 4 instances in your example. I guess, I will still have to investigate a bit more.
However, it does not have problems finding the file anymore. The computation is run and it obtains an output. Thank you very much for solving the issue! I would have never found it without you.
OCDER
OCDER 2018-8-13
Nice! If that works, can you go ahead an accept the answer below? That way we know the question is answered with a working solution, and I get some reputation points :) Play around with removing start so that it waits for it to finish per parallel worker. I think the file naming and having multiple input file copies was the real issue, not start. If you do need start, then you might need another step to wait for the output file to be generated before continuing.

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

OCDER
OCDER 2018-8-13
编辑：OCDER 2018-8-13

1 个投票

energyPlanPath = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPLAN.exe';
origInput = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp.txt';
inputPre = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\energyPlan Data\Data\Denmark2030Alternative_tmp';
outputPre = 'C:\Users\HumptyDumpty\Desktop\Programme etc\Studium\4o Semestre\Master Thesis\EnergyPlan\Outputs\out_Denmark2030Alternative';
%
assert(~isempty(dir(energyPlanPath)), 'energyPlanPath is not valid. Check file name.');
assert(~isempty(dir(origInput)), 'origInput file is not valid. Check file name.');
%
%To prevent parfor from accessing and writing to same file, make copies and specify separate outputs
Inputs = cell(4, 1);
Outputs = cell(4, 1);
for j = 1:4
Inputs{j} = [inputPre num2str(j) '.txt'];
[Success, Msg] = copyfile(origInput, Inputs{j});
assert(Success, Msg);
Outputs{j} = [outputPre num2str(j) '.txt'];
end
%Attempt to run in parallel
parfor j = 1:4
executionString = sprintf('start "title" "%s" -i "%s" -ascii "%s"', energyPlanPath, Inputs{j}, Outputs{j});
system(executionString); %execution of EnergyPLAN.
%to wait for output file to be generated
while isempty(dir(Outputs{j}))
pause(0.1); %check again in 0.1 sec
end
... rest of your codes
end

3 个评论
显示 1更早的评论 隐藏 1更早的评论

MD
MD 2018-8-13
I managed to run it smoothly without any problems now. The reason for the other problems was actually the use of "start". This forces the program to open regardless of whether Matlab is ready or not. With this executionString it works perfectly since a new instance of EnergyPlan is only opened when the old one was computed:
executionString = sprintf('"%s" -i "%s" -ascii "%s"', energyPlanPath, Inputs{j}, Outputs{j});
Now I only need to find a way to force the program to run in the background and not pop up every single time it runs.
Thanks again!
Filippo
Filippo 2025-1-5
Helo MD, did you find a solution in order to let EnergyPlan run in background and not pop up for each model simulation?
Thank you!
Walter Roberson
Walter Roberson 2025-1-6
You could potentially use the .NET interface System.Diagnostics.Process
https://www.mathworks.com/matlabcentral/answers/583955-get-the-status-using-system-command-when-program-has-been-closed#answer_485480

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

产品

版本

R2017b

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by