Find more time-efficient way to extract data from a .txt file

Question

Albert Garcia 2020-3-7

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/509624-find-more-time-efficient-way-to-extract-data-from-a-txt-file

评论： dpb 2020-3-8

I have a code that takes over 4minutes to converge to a solution, and I need to get convergence about 30 times for synthesis purposes, resulting in over 2hrs of run time. I found that what is slowing my code down so much is the fopen function.

The files with the function name "intermediate_GW" and "intermediate_GW2" read a single data point with the format #####.# from a text file outputted from my gross weight solver (not listed above), and then, they output this value to the functions FF, DISK, and SW_TOTAL. "intermediate_GW2" stays the same per iteration of this WEIGHT_FUEL program while "intermediate_GW" changes per call. Is there a way for me to have the values from these two programs in a better format for this purpose? Below is a rough flowchart of how my programs are going to run. I can't have the TOTAL_WEIGHT program directly output to my other programs because this causes an infinite recursion, and I had to bypass that by using txt files.

If it could be done such that the below loop of intermediate_GW2, DISK, and SW_TOTAL can be called only once instead of 40000 times, this alone would drastically improve the runtime of my code.

9 个评论
显示 7更早的评论隐藏 7更早的评论

Albert Garcia 2020-3-8

在 MATLAB Online 中打开

You're right. I've been thinking about this, and I think all my problems arise because I just don't know how to properly structure my code. So this is what I want to do...

I have this main code, and I can have this program run quickly if I just had all my calculations within one .m file. However, I've been told that this is poor coding and that I should write my functions and such into different files so that it's easier to digest; this makes sense, and I wouldn't want to have thousands of lines of code in 1 .m file. This, however, leads to a very slow program. I guess what I really want is for my seperate functions to run as if they were embeded in this main code without having all this code in one program.

Below, I threw in my convergence code and one of the functions that this code needs.

This here is my major code that runs my other functions.

function [GW,W] = WEIGHT_SIZING(H,IT,x,y)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%   Function:   [W_total,W] = WEIGHT_ALLOCATION()
%   Program:    Weight Allocation
%   Input:      
%               H:          Altitude [ft]               -FOR USAGE IN STD_ATM
%               IT:         Sea Level Temperature [°F]  -FOR USAGE IN STD_ATM
%               x:          Gear Ratio Combination      -FOR USAGE IN TRANSMISSION
%               y:          Mission                     -FOR USAGE IN SW_TOTAL, FF, WEIGHT_FUEL
%   Output:     
%               W_total:    Sum of Component Weights [lbf]
%               W:          Component Weights [lbf]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
W_ammo  = .32*2.205;            %Ammo Weight [lbf]
W_hefi  = 100;                  %Hellfire Weight [lbf]
W_PL_AR = W_ammo*500+4*W_hefi;  %Armed Recon Payload weight [lbf]
W_PL_AT = W_ammo*500+8*W_hefi;  %Attack Payload weight [lbf]
W_PL    = [W_PL_AR W_PL_AT];    %Payload Weight Conditional [lbf]
epsilon = 2;    %Used for convergence
i       = 1;    %Used for iterations
while epsilon > 1
    
    if i == 1   %Sets gross weight to guess gross weight
        GW  = 12180;     %Guess gross weight [lbf]
    else
	%Opens gross weight text file and pulls data from it
        file	= fopen('intermediate_GW.txt','r');
        GW      = fscanf(file,'%f');        fclose(file);
        GW2     = GW;
    %Gross weight calculations
        W_F             = WEIGHT_FUEL(H,IT,x,y);    %Fuel weight from WEIGHT_FUEL
        [SW]            = SW_TOTAL(y);              %Wing area from SW_TOTAL
        [HPmast,~,Qm]   = TRANSMISSION(x);          %Power and torque transmitted to mast from TRANSMISSION
        EW_GW           = .6776;                    %Empty weight to gross weight ratio from HS data
        W_E             = GW / EW_GW;               %Empty weight [lbf]
        Nr              = 2;                        %Number of rotors
        Lf              = 38.38;                    %Fuselage length [ft]
        HPeng           = 3000;                     %Engine power [HP]
        v_ktas          = [220 200];                %Forward flight dash speed [ktas]
        [~,Tmr]         = FF(H,IT,v_ktas(y),x,y,0); %Thrust of main rotor from FF
        %        1   2   3    4   5   6   7  8     9 
        W_SYS = [175 100 1050 700 525 100 20 143.6 15];
        W(1)  = 20.906 * SW^.601;                       %Wing
        W(2)  = 30.537 * (Tmr/1000)^1.042 * Nr^1.136;   %Main Rotor
        W(3)  = GW*.0114;                               %Tail
        W(4)  = 7.5986*Lf^.946*(Tmr/1000)^.636;         %Body
        W(5)  = .0127*GW^1.095;                         %Landing Gear
        W(6)  = .05418*HPmast^1.22;                     %Engine Section
        W(7)  = .01703*HPeng^1.053;                     %Air Induction
        W(8)  = 456*.8;%5.1403*HPeng^0.702;                     %Propulsion
        W(9)  = 46.1558*(Qm/1000)^0.921*Nr;             %Drive
        W(10) = GW * 0.0427;                            %Flight Controls
        W(11) = W_SYS(1);                               %Auxiliary Power Unit
        W(12) = W_SYS(2);                               %Instruments
        W(13) = GW * .0104;                             %Hydraulics
        W(14) = GW * .0333;                             %Electrical
        W(15) = W_SYS(3);                               %Avionics
        W(16) = W_SYS(4);                               %Armament
        W(17) = W_SYS(5);                               %Furniture and Equipment
        W(18) = W_SYS(6);                               %Air Conditioning
        W(19) = W_SYS(7);                               %Anti-Ice
        W(20) = W_SYS(8);                               %Fluids
        W(21) = W_SYS(9);                               %Load and Hand
        W(22) = W_E * .03;                              %Contingency
        W(23) = 400;%480;                               %Crew
        W(24) = W_PL(y);                                %Payload
        W(25) = W_F;                                    %Fuel
        GW  = sum(W);               %Calculating new gross weight
        epsilon = abs(GW-GW2);      %Difference in new and old gross weights to determine if reiteration is necessary
    end
    i   = i + 1;                    %Just to make i not equal to 1. 
%Opens gross weight text file and edits data for this iteration
    fileID = fopen('intermediate_GW.txt','w');
    fprintf(fileID,'%.1f\n',GW);
    fclose(fileID);
    fileID = fopen('intermediate_GW2.txt','w');
    fprintf(fileID,'%.1f\n',GW);
    fclose(fileID);
end

This here is a child function

function [Sw_w,b,CL,CD] = SW_TOTAL(y)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%   Function:   [Sw_t] = SW_TOTAL(y)
%   Program:    Wing Sizing
%   Input:      
%               y:          Mission
%   Output:     
%               Sw_w:       Total Usable Wing Area [ft^2]
%               b:          Total Span of Wing [ft]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[~,~,~,Den,~] = STD_ATM(4000,95);
GW  = intermediate_GW2;
v_ktas  = [220 200];        %Objective dash for AR and AT [ktas]
v       = v_ktas(y)*1.68781;%Design point velocity [f/s]
Lw      = GW * .2;          %Lift produced by wing [lbs]
q       = .5 * Den * v^2;   %Design point dynamic pressure [lbs/ft^2]
ao      = 2*pi;             %2-D Lift-Curve Slope 
AR      = 5;
a       = ao / (1+pi*ao/(pi*AR));
aoa     = 5*pi/180;         %Design angle of attack of wing [rad]
c_r     = 6;                %Root chord [ft]
w_f     = 6.25;             %Fuselage width [ft]
tr      = .5;               %Taper Ratio 
CL      = a*aoa;            %Lift coefficient
CD      = .00847;           %Drag coefficient
Sw_w    = Lw / (q*CL) ;     %Wetted area of wing [ft^2]
b_w     = Sw_w/(c_r*(1+tr));%Span of each wetted wing [ft]
b   = w_f + 2 * b_w;        %Total Wing Span
h   = c_r*(1-tr)/b_w*w_f;   %Height of Triangle
Sw_t    = Sw_w + c_r*w_f + 2*h*(w_f/2)/2;

Albert Garcia 2020-3-8

The values of GW, SW, W_F, and Tmr all change (calculated in the functions SW_TOTAL, WEIGHT_FUEL, and FF), but there are interrelations between these functions, too, as seen in the profiler screenshot above.

Well, I am saving the last computed value and iterating. I do set both W and W2 to the same value; the SW_TOTAL function has the wings sized to match the initial weight of the aircraft since the wings won't change size during flight, and this program along with others so use this W2 since it's meant to be a constant value. The W value is used for the FF program since it calculates how much power the aircraft needs to fly, and this W value decreases as fuel is burnt off, making the aircraft lighter, allowing it to fly at the same velocities using less power/faster with the same power.

As for the converging part, me setting it twice doesn't really do any harm. I have the W2 term, the weight calculated in the previous iteration, compared to the new GW term. Then, I go convert the data in both text files to the newest GW term so that the aircraft's geometries are sized appropriately for the weight, with one text file changing and the other remaining constant.

Yeah, I would have to restructure the whole code if I wanted to try fzero. I guess that's what I would have to do regardless.

dpb 2020-3-8

在 MATLAB Online 中打开

"...with one text file changing and the other remaining constant."

That doesn't seem to be so...

  ...
  i=i+1;
  fileID = fopen('intermediate_GW.txt','w');
  fprintf(fileID,'%.1f\n',GW);
  fclose(fileID);
  fileID = fopen('intermediate_GW2.txt','w');
  fprintf(fileID,'%.1f\n',GW);
  fclose(fileID);
end

writes the same value to both files every iteration.

Presuming the

GW = intermediate_GW2;

function is reading the file of that name, the new GW value in SW_TOTAL(y) is the same one just written.

In which case, just pass it (new GW) in the argument list and bypass the file.

When you get to the bottom and GW=sum(W), THEN you do have a difference again that you can test for convergence.

I can't see any reason for the file at all if you just pass the information as an additional argument to the functions that need it.

Albert Garcia 2020-3-8

In the programs above, it doesn't show the GW file changing values within the first text file, but it does change values when it is used by other functions.

That's how I first tried it, having the main file pass the information to the beneath arguments, but I couldn't figure out how to do that without infinite recursions, so I ended up with this inconvenient solution.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

dpb 2020-3-8

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/509624-find-more-time-efficient-way-to-extract-data-from-a-txt-file#answer_419095

在 MATLAB Online 中打开

While I'm convinced you really don't need the files at all and probably with only a minor amount of rearranging if it works as is, one really expedient workaround would be to not open/close the files every time...

function [GW,W] = WEIGHT_SIZING(H,IT,x,y)
  ...
  i       = 1;    %Used for iterations
  fID1 = fopen('intermediate_GW.txt','w');
  fID2 = fopen('intermediate_GW2.txt','w');
  ...
  while epsilon > 1
    ...
    i=i+1;
    frewind(fID1)
    fprintf(fID1,'%.1f\n',GW);
    frewind(fID1)
    frewind(fID2)
    fprintf(fID2,'%.1f\n',GW);
    frewind(fID2)
  end
  fID1=fclose(fID1);       % release file handles only when completed iteration loop
  fID2=fclose(fID2);
end

You then either add fID1, fID2 to the argument list to the functions that need them, or, (ugly, but expedient) make them GLOBAL variables.

5 个评论
显示 3更早的评论隐藏 3更早的评论

Albert Garcia 2020-3-8

Well, I will be using this program quite a bit for the next month, but I don't think I should invest more time in making this code better since it's good enough now. I do need to learn how to properly structure my code when I write it though; my terrible structure skills are definitely what have made me waste so much time fixing this code when I should've written it write the first time around.

dpb 2020-3-8

Your call, of course. If it's fast enough this way, then perhaps there's no real justification for spending the time now to refactor it.

请先登录，再进行评论。

Find more time-efficient way to extract data from a .txt file

9 个评论
显示 7更早的评论隐藏 7更早的评论

采纳的回答

5 个评论
显示 3更早的评论隐藏 3更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

Find more time-efficient way to extract data from a .txt file

9 个评论 显示 7更早的评论隐藏 7更早的评论

采纳的回答

5 个评论 显示 3更早的评论隐藏 3更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

9 个评论
显示 7更早的评论隐藏 7更早的评论

5 个评论
显示 3更早的评论隐藏 3更早的评论