How specific lines in a text file can be updated many times ?

Question

ND 2015-5-22

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/218102-how-specific-lines-in-a-text-file-can-be-updated-many-times

评论： Walter Roberson 2015-5-29

Dear all, Please how can I update specific lines or specific matrix which found in the middle of a text file. Actually, I have jacobian matrix needs to be updated at every iteration and I should save the changes in the same file.

Many thanks for your help in advance.

2 个评论
显示无隐藏无

Cedric 2015-5-23

Why not computing/updating the Jacobian matrix directly in Fortran?

ND 2015-5-26

As my job should be done within Matlab, using Matlab just for calling and running other software packages.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

dpb 2015-5-22

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/218102-how-specific-lines-in-a-text-file-can-be-updated-many-times#answer_180079

Simple answer is "you can't, at least easily". Sequential text files are, well, "sequential". Even with the complex ways you could only make changes that were precisely the same length in the revised data as that changed.

A far better solution would be to make the changes in memory (which you have to do, anyway, to make anything happen) and then if you really, really, must save these data every iteration, use save to save the array in a more efficient manner than text. If you then must look at it at some point in text, then you can convert but there's really no need otherwise.

21 个评论
显示 19更早的评论隐藏 19更早的评论

Walter Roberson 2015-5-22

编辑：Walter Roberson 2015-5-22

When you are using one file as a template to create a new version, it is a poor idea to overwrite the original file. Suppose something goes wrong during the process? A bug in the code? A power glitch? Your cat knocks the drive cable so the write fails?

Therefore, you should do one of the following:

create a new file each time, preserving the previous ones, leaving it for the user to clean them up as desired
create a new file, rename the old file to ".bak" (or similar), rename the new file to the old name; in this way if the new file creation fails, there is always the .bak to go back to
create a new file, rename it to the old name once it is fully created; keeping a backup like the above would be better, but at the very least, do not put the new file into the place of the old file until the new file has been completely created and verified to have worked
in cases where preserving file dates and permissions (and alternative resource forks) is important and those cannot be easily copied from one file to another, then create a new file, write the data to it, verify it is all correct, then take the contents and use them to overwrite the old file in-place (because overwriting preserves dates and access permissions and resource forks.) In this set-up, there is always a file with valid data contents in existence in case something goes wrong while overwriting the file. Note: you cannot reliably do this in MATLAB, because MATLAB does not allow you to truncate existing files if they should shrink due to the new content being smaller
in the more general case, use a "transactional database" that is able to roll-back failed or incomplete transactions; or if your file-system supports transactions (for example the native Mac OS-X file system does), find a way to make the update take only a single transaction for file-system purposes so that you can recover if something goes wrong.

Those of us who used to do tape backups know the rule of three. Always use three generations of tapes for doing backups. The first of the tapes is the one being backed up to, and that is going to be the one that is corrupted when the system fails during backup and fouls itself needing restoration. The second of the tapes is the one you go to do the restore from to recover from the failure during the previous backup. And the reason why you need a third tape is that you forgot to put the write protect ring on the second tape and it promptly got clobbered by the tape drive management system (or by you accidentally entering the wrong command) when what you wanted to do was restore from it. The third tape is the one you remember to protect before starting the restore. And if your tape drive mangles that one too, it's your own fault for not having fixed the drive before you risked your last remaining copy.

dpb 2015-5-22

You could, except--it's going to be slower than either of the above and there's no need to reread constant data every pass plus the major factor that when you write into a sequential file it is truncated from that point on -- hence Walter's suggestion of writing the three pieces or my alternative of only writing the changing part.

It would be possible if Matlab supported formatted direct-access record files, but it doesn't unless you "roll your own" which is what I alluded to in my original response of the harder way--namely, yes, if the file is a fixed format file you could determine the offset into the file of the location you're interested in, read the whole thing into memory as Walter says as a character array, and then do an internal formatted equivalent of Fortran internal WRITE (sprintf or the like) into that buffer at the correct point in memory and then write the modified file back out.

All in all, is it really worth that effort over the simpler way? Are you going to be doing this more than just a time or two so it would be worth the extra effort?

Alternatively, can your other package read anything other than a text file? A stream (so-called 'binary') file could be modified more readily knowing only the size of the arrays as the locations then are simply a direct multiple of the size (REAL or DOUBLE) times the array size.

Walter Roberson 2015-5-22

It is not possible to "delete" lines in a text file. It is only possible to overwrite with new text that is at least as long as before (not shorter), or else to copy the lines before into a new file, write the new lines, and then copy the lines after from the old file to the new file.

When you are overwriting in-place, the limitation that the final file cannot get any shorter is due to a short-coming in MATLAB; operating systems do provide a way to say "truncate the file here" but MATLAB does not provide access to that.

The limitation that you cannot write a shorter chunk into the middle of a file and have the rest of the file "fall down" in relative position, is a limitation on all filesystems that store files as streams or as blocks. The last operating system that I know of that supported storing files as a list of variable-length records, was VAX VMS when using the RMS (Record Management System) I/O interface. The internal structure of those kinds of files did not fit in to the C/Unix model of a file as a consecutive stream of bytes, and eventually VMS RMS became obsolete even though it had some very good ideas.

There was a point at which MATLAB was supported on VMS, but my understanding is that MATLAB never provided support or interface to the VMS RMS services.

I don't know if there are any modern operating systems that support line-oriented files. It isn't impossible, I suppose: IBM's VM operating system supported datasets with variable-length records, and it would not entirely surprise me if there are still Big Data customers on mainframes who demand them.

Walter Roberson 2015-5-29

在 MATLAB Online 中打开

Use an editor to divide the file into partA.f (fixed part before), partB.f (the part that will be changing), partC.f (fixed part after.) It isn't worth doing it by program since you only need to do it once and you would need to examine the file carefully to determine exactly how to detect the boundaries by program. Since you are saving a copy of the file in this way, my discussion about generations of backups does not apply.

Then take the suggested code but with very minor modifications:

fid=fopen('partB.f','rt');  % open the middle part
fprintf(fid,fmt,array)  % w/ appropriate format
fid=fclose(fid);       % close partB
!'copy partA.f+partB.f+partC.f yourFile.f' % concatenate

but change the word yourFile.f to the exact file name that is expected by the other package, and change "array" to be the name of your variable that holds the computed jacobian, and change "fmt" to be a format specifier that formats the expression in a way that Fortran understands.

Creating a new file this way is the easy part. The harder part is in rewriting the expression for Fortran, as you might need to rewrite vector notation and you might need to do "word wrap", and whether you use .f or .F or .f77 or .F77 as your file extension can matter as to whether the file is interpreted as Fixed Format (72 column) or Free Format (I do not recall the maximum line length for that.)

dpb 2015-5-29

编辑：dpb 2015-5-29

在 MATLAB Online 中打开

Again we repeat--you do NOT need to read the file every time unless the previous ground rule that the rest is invariant with the exception of this section, and in fact, it's a waste of resources to do so unless that is so. Just open one of them that you currently have in your favorite text editor and save the first and last parts as outlined.

I (and I presume Walter as well) was presuming the array to be written was numeric not a functional form but I suppose it does makes sense as input to have it as above.

Now it becomes a question of what actually is going to change; the functional form or only the numeric constants? The former is potentially fairly difficult depending on just what the form could be; the latter is pretty simple; you simply use fprintf with the expression as part of the format string and substitute in an appropriate numeric format string where the numbers that change are.

If I presume it's only the floating point numbers and not the exponents in the first since the first term is really SQRT() it probably is fixed, then

fmt1='DDSDDE(1,1) = X1**(-0.5D0)*X5*X6**2*%.3E+%.3E\n';

Then you use this with the constants you need/want something like

fprintf(fid,fmt1,coeff(1),coeff(2))

where coeff are the values you've decided you need next, from whatever method you have to generate those. Clearly, you can use whatever array features of Matlab that are convenient, I simply showed the two elements specifically to illustrate the idea clearly.

When you've done the appropriate modifications for all sections of PartB as above, fclose it and follow with the concatenation as illustrated.

dpb 2015-5-29

Well, I don't know what "the Fortran function" is in Matlab, but surely you've seen enough pieces of how to write whatever it is you want to write expounded upon here.

Try some code and see where you then actually fail and with a specific example of that problem we can answer a specific question. I don't really follow what you're lacking at this point...

Walter Roberson 2015-5-29

在 MATLAB Online 中打开

Using the fortran() call makes it really simple. Instead of fopen/fprintf/etc to write out the new file, just use

fortran(DDSDDE,'file','partB.f');

Here, the variable that stores your symbolic expression should be a symbolic matrix, and it should be the same name as what you want to appear in the file. If you use a regular MATLAB matrix of symbolic expressions then there is the possibility that the output variable name will be "t0" instead of the name you want.

Side note: when you use fortran() in this form, there is the bonus that it will generate optimized code, using temporary variables to avoid computing the same thing multiple times. That's good for computation purposes, but does tend to make the code more difficult to understand.

请先登录，再进行评论。