Exclude System or Hidden Folders from Mac Dir List

Question

Matt O'Brien 2023-4-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1950983-exclude-system-or-hidden-folders-from-mac-dir-list

回答： Matt O'Brien 2023-5-9

在 MATLAB Online 中打开

Here is the attributes for the System Volume Information directory on a Mac disk.

How can I get a Dir listing of all folders on a Mac drive but exclude hidden or system folders.

Note (see 2) ... System Attribute for a System file is Nan... So, if I cannot use the attribute function how can I establish (reliably) what are system or hidden files.

AVF_INFO and 'System...' are hidden directories . DCIM and PRIVATE are 'normal' dirs I expect to find in a dir list.

Note 2. I have already eliminated lots of directories which I also needed to exclude from my Dir list.

MyDirList = HD_FullList([HD_FullList.isdir]);  %remove files from list
MyDirList=MyDirList(~ismember({MyDirList.name},{'.','..'}));% remove . & ..
MyDirList=MyDirList(~ismember({MyDirList.name},{'$RECYCLE.BIN'}));% remove recycle bin
MyDirList=MyDirList(~ismember({MyDirList.name},{'.Trashes'}));% remove trash dirs
MyDirList( startsWith({MyDirList.name}, '.') ) = [];  %exclude hidden files ie starting with .

And excuse me for venting a little rant here.... Why is it so ridulously difficult to get dir lists of files and directories on Mac and Win. This is excruciatingly painful... and seriously error prone. This might be my last MatLab project.

25 个评论
显示 23更早的评论隐藏 23更早的评论

Matt O'Brien 2023-4-22

移动：Walter Roberson 2023-4-22

Thank you for your feedback. Appreciated. We are completely on the same page.

I think the absense of a proper MatLab dir function which works on both Mac and Win, with switches to manage System and Hidden files is a massive own goal.

My only workaround is to hard code for files or directories I find which do not have the attribute set as per their real world status. This makes producing future proof or bug free code not only impossible, but implies code is flawed from the start.

To me, this signals the end of my adoption of Matlab and my current project is now relegated to the status of prototype.

There is a lot to like about MatLab.... but there are too many barriers to overcome.

I will no longer recommend Matlab as a small footprint IDE as I have been doing.

I have tried to write my own Dir function, but if I cannot properly identify system or hidden files on a Mac.... what is the point. I am disappointed, because I have invested so much time on this and now must committ to washing all this effort down the sink.

I will log a ticket with Matlab... in case there is something obvious I am missing..

Sad, but so be it.

.

Matt O'Brien 2023-4-22

Two examples included in the list above.

'System Volume Information' is a directory included when I use Dir on a Mac. I assume this is both a Sytem file and perhaps hidden file. I just need to make sure that I can remove it from my Dir list (eg. I do not want to copy this dir by mistake, or include this dir for a user to select.. and subsequently encounter uncontrollable error conditions.
AVF_INFO. This is a hidden directory created by Sony when they format Sd cards in their cameras. It contains a database of the images and video files stored on the SD card and allows the Sony cameras display thumnnails and other key features on a modern digital camera. Again, this is invisible, by default in most scenarios.... but it is included in the contents of a Dir command. What is worse..... the attributes function returns a value of NaN for this directory.... so other than hard coding this directory name, I cannot exclude it from a dir list... so my app will fail at some stage when it tries to take inappropriate actions on these directories.

If you search many of the queries re flaws with the Dir command on Mac... they recommend you exclude any file or directory which starts with a full stop (ie '.').

That will remove a lot of hidden files... but it is not a safe strategy... as I have tried and it does not exclude the 2 examples referenced here. What is worse... I have no way of knowing what other system or hidden directories my app might bump into in the future...with totally and unacceptably unpredictable results.

Thanks for your query and allowing me to explain this further.

Matt O'Brien 2023-4-22

You are correct .... these are SD cards, which conform to DCIM standards, used globally by the camera / video industry, which includes Fat and ExFat standards. Millions of users around the world are using these cards in Macs and Windows machines, directly in a Mac SD reader or via backups of these cards taken in the field. It is an outdated standard, I strongly would be in favour of this standard been upgraded, but we can only use what we have.

There are large number of MatLab users reading their image files from such cards.

The following screen grab is from a Mac screen.

Note that Mac treats the System / Hidden files differently (faded font in the Yellow boxes) ... so the Mac OS knows the difference.

I am not asking that MatLab should recognise and deal with Windows file attributes ... but MatLab should recognise and allow an app to distinguish between these directory / file types.

If MatLab are not prepared to do that then it reinforces in my mind that MatLab is not a place I want to invest time and money.

I will accept a workaround... but so far I have not got one.

I have posted a support case to MatLab. Lets see what MatLab say...

I am currently hard coding my app to deal with this .... but that is guaranteed to break down at some point in the future .. so it would be immoral for me to share my apps as a result.

I do appreciate that you have taken the time and effort to respons and I am grateful for that.

dpb 2023-4-22

编辑：dpb 2023-4-22

What does the MacOS system 'ls' return for those files? Unless it returns some attribute or has some switch to differentiate them, then you're pretty-much hosed independently of MATLAB.

The MATLAB dir() function on Windows does return hidden/system files whereas the normal OS CMD dir does not; the "/a:hs" switches are negated by default for those two attributes whereas TMW has chosen to set them for their internal dir() function. That has never made any sense to me; particularly when they won't pass a switch and then don't return the attributes field to let one screen by. Of course, Walter just reminded me a day or so ago that even the .Folder property is a moderately recent addition as it came up in another Q? of a poster using R2013x and running into that problem.

I don't know what MacOS does, but I would think the workaround has to be to use the OS-supplied functions and that you'll run into the same issue on whatever platform you develop if it doesn't identify those as eventually it gets passed down to the underlying OS APIs that are fundamental.

As noted, I began a crusade some 30 years ago by submitting enhancement requests; after 10 years of no action, finally just gave up that quest as hopeless. But I still fail to see why TMW is so apparently klewless on this front...

dpb 2023-4-22

"I will accept a workaround... but so far I have not got one."

The obvious workaround is to use the underlying OS equivalent -- as noted above, however, unless it can tell the difference, then you're pretty-much out of luck no matter what; the differences in the above screen image would seem to imply there is some attribute that is distinguishable or else the mounted file system has installed its own extension into the UI as well and is bypassing the standard OS API.

I never used a Mac and have never done anything with the digital camera stuff even on Windows, so I've not direct knowledge about what/how it is done, but anything you do inside or outside of MATLAB would seem to have similar issues no matter what programming language you choose.

I found a perhaps helpful <Apple link on the file system>; in it is the following --

Creating and Handling Invisible Files

In macOS, you make a file invisible to the user by prepending its name with a period character. Adding this character signals to the Finder and other display-related interfaces that they should treat the file as invisible under normal conditions. You should use this capability sparingly, though, and only in very specific cases. For example, the process for localizing a custom directory name involves placing hidden files with the localized names at the top-level of the directory. You should never use this technique to hide user documents or other data your app creates.

Even if you mark a file as invisible, there are still ways for the user to see it. Users can always view the true file system contents using the Terminal app. Unlike the Finder, the command line in the Terminal window displays the actual names of items in the file system and not their display names or localized names. You can also show hidden files in the Open and Save panels (using the setShowsHiddenFiles: method) in cases where you want the user to be able to select them for viewing or editing.

Regardless of whether a file contains a leading period, your app’s code can always see hidden files. Methods that enumerate the contents of directories generally include hidden files in the enumeration. The only exceptions to this rule is the meta directories . and .. that represent the current and parent directories. Therefore, any code that enumerates the contents of a directory should be prepared to handle hidden files and deal with them appropriately, usually by just ignoring them.

From the above, I presume that probably the particular files of interest may begin with the dot in the filename for hidden files; not sure about system files although I presume they would also be hidden.

Good luck, but looks to me like you'll have similar issues no matter what you use to write the application in; doesn't appear this is anything that will be unique to MATLAB although I still agree they could supply more regarding what switches are set/not and let the user control). The ls man page has the list of switches; looks as though it should be feasible to return the limited list desired and/or use the annotation of the type. The big issue with unix-like filenames is that such abominations as \n are allowable legal characters in a filename so you can't rely 100% on parsing a text listing since it uses \n to separate records -- and one of those could just have been part of a filename--Catch 22!

Matt O'Brien 2023-4-22

A few commemts on the above.

I have started to look at the ls command... but not optimistic. Mac documentation for this command is extremely sparse and refers to general linux to get more info... which so far has not revealed a magic option... but have not given up on this.
In the code window above... you can see that I have already removed all the dot and dot dot dirs, everything beginning with a dot. Sony have obviously found a way to flag a folder as hidden (AVF_INFO). No doubt Canon and Nikon and Fuji will all use similar techniques to keep their SD card housekeeping directories away from casual misuse. I know in Sony's case, if the AVF_INFO is missing the camera will give you a message... pls wait ... while it rebuilds it. My fear is that my code may need to be hardcoded for every brand of camera.
There is a Mac keyboard combo of Cmd Shift + which reveals / hides System / Hidden files. I would love to be able to replicate this in code.
I intend to investigate how Apple Swift / Xcode handles hidden files. The danger is that if I go that far it might be easier for me to write an Apple native app.
The problem I have is that I have the app working for Windows but running aground trying to resolve this issue on the Mac.

Thanks for posting the above info. I will explore further.

Walter Roberson 2023-4-22

Hiding dot files is a convention, not something enforced by the OS.

MacOS HFS+ and APFS file systems support "flags" ( https://www.maketecheasier.com/use-file-flags-modify-file-behavior-macos/ ) that can be set with chflags https://ss64.com/osx/chflags.html -- but not queried with that command. At the moment all I see for querying the flags is ls -lO (lower-case L, capital Oh), but I would imagine there are other ways to get the flags. Ah, I see they can be read with stat see https://unix.stackexchange.com/questions/640422/api-syscall-to-read-or-list-bsd-macos-file-flags

The supported flags do include hidden but not system

It would not be unreasonable for a MacOS filesystem layer reading a Windows system to examine the windows attributes and fill in the hidden attribute of the MacOS stat() call, but MacOS does not have a system attribute in stat(), so in order to code the system attribute MacOS would have to go even further afield, such as synthesizing an extended attribute https://ss64.com/osx/xattr.html .

I would suggest that even if MacOS went to that trouble, that expecting MATLAB's dir() call to examine that kind of information would be much the same as expecting MATLAB's dir() call to fill out the USERREAD USERWRITE USEREXECUTE fields based upon interpreting Access Control List https://www.techrepublic.com/article/introduction-to-os-x-access-control-lists-acls/ . For example some might say that for a file, USEREXECUTE should be false if the file has com.apple.quarintine associated with it. But other people would say that the results of dir() should reflect only the basic attributes associated with the file -- and on MacOS, going to look in the flags for hidden or (hypothetically) looking in extended attributes to see if some kind of system attribute has been synthesized is not basic attributes associated with a file.

Question: is there a standard for how third party filesystem drivers such as FUSE (for Linux ext* filesystems) or Paragon's NTFS drivers should import non-MACoS filesystem flags and attributes and access control lists? A standard that Mathworks could follow so that it always gets the flags in dir() "right" ?

Matt O'Brien 2023-4-22

编辑：Matt O'Brien 2023-4-22

Our messages are overlapping...so apologies if my comments are slightly out of synch.

Thanks for the previous link. I did start to look at more generic scenarios and reading directly from the card in a device, whether it be a camera or a card reader is an option.

However, for my needs, I need to be able to read the image or video data from a card reader (most frequent situation) or (less regularly, but important) a portable storage device (ie Sdd) which contains backups of multiple cards. The scenario I am catering for usually involves 10,000's of 50MB images per card, so it is high volume processing and thus the reason to streamline / automate aspects of this process.

However, these tools may lead to some cluses or work arounds.

I started off in the 70's writing Fortran iv (Civil Engineering apps). I then began coding directly in machine code, advanced to assembler and then on to 3rd gen languages (Cobol was my fav) before moving to mainframes. At that stage I left coding behind as I became responsible for teams, projects, enterprise computing, etc..

Thanks for all your contributions.

Typical,,, what starts off as a 50 line script and a basic idea becomes a good idea and a monster....

I always had a healthy respect for Digital and the PDP/11. I remember reading The Soul Of A New Machine (Data General) as it was the early days when alternative to mainframe computing started to emerge.

https://en.wikipedia.org/wiki/The_Soul_of_a_New_Machine

Regards.

dpb 2023-4-23

编辑：dpb 2023-4-23

We're not that far apart, then although I predate you by probably about 10 years or so...first mainframe was a Philco 2000; the class scientific computing machine of its era -- no disk or durm, but 27(!) 7-track tape drives. Fortran II and assembler for it running the proprietary versions of the nuclear design codes from Bettis for a commercial nuclear power vendor. The Philco was already in the process of being retired when I arrived but it took over two years to get the CDC 6600 installed and able to run all the needed codes before could pull the plug on the old Philco. Other than the commercial timeshare still in Philly, I think it well may have been the last one in a production environment. Fairly shortly after it was replaced by a Cyber 76...after only a couple years with the latter, I moved on to the consulting gig and the other end of embedded/standalone systems, mostly around the M6809E running Forth. Quite a number of various specific-purpose "instruments" built with this configuration from loose-parts monitoring to neutron-noise systems looking for things like inner liner vibration/movement, etc., ...

Walter Roberson 2023-4-23

@Matt O'Brien

Note that Mac treats the System / Hidden files differently (faded font in the Yellow boxes) ... so the Mac OS knows the difference.

At most MacOS knows about hidden, not about system.

'System Volume Information' is a directory included when I use Dir on a Mac. I assume this is both a Sytem file and perhaps hidden file.

System Volume Information is a Windows artifact; on Windows it is system folder (possibly hidden too, not sure right now) used to contain restore points.

Are you expecting that the MacOS built-in FAT filesystem drivers will convert the DOS system attribute into some kind of flag or attribute or extended attribute that MATLAB should know to check? If so, then what are you expecting that MacOS already converts the Windows system attribute into, that MATLAB is being negligent in checking? Or are you expecting that MATLAB should have its own FAT filesystem drivers, that it should detect that it is a Windows filesystem and should do its own block-level reading of the files and interpret the binary attributes so that it can fill in the system attribute for the dir() call ? Or are you saying that Mathworks has been irresponsible in not already having (successfully) pushed Apple to convert the DOS FAT hidden attribute into something that MATLAB could have gotten ahold of for the purposes of filling out the dir() information ?

Matt O'Brien 2023-4-23

Yes... I noticed the behaviour of the 'Hidden' files on Mac.... When I discovered 'hidden' folders using the MatLab dir function on a Mac... I immediately learned how to hide / reveal these directories using the keyboard short cut .. which told me that Apple know how to identify these files / directories.

I had already discovered the attrib features, from testing my app on Windows. I assumed (dangerous) that this feature would also work on Mac.... but learned very quickly that it does not. This brings me to the start of my query (ie first post/query above.

I am not expecting MatLab or Mac to change any of its behaviour in relation to how it handles such cards / directories / files. My app has an option to copy from card to disk (where backup cards are retained within a project structure). I have used Beyond Compare to compare the source card and the destination on a hard drive. They match perfectly. I have checked this with multiple cards, multiple times on multiple machines (and also on Windows).

So parts of my app are delighted these System / Hidden files are available through the Dir command. However, I also need to analyse the contents of the directories. Unfortunately, while DCIM standards exist, every Camera maker has their own naming convention for raw or jpg files and there does not seem to be any convention for video files. Also video codecs change all the time as new standards and the limits of hardware continues to advance.

So it is not feasible for me to build an app that can cater for all the naming conventions of every camera / video maker in the world... so I cannot use Folder names or file extensions as a filtering mechanism.

Most people can build apps to process these images either by hardcoding the file types or be selective in terms of the file extensions they use.

In my case I cannot do that. At the same time I cannot have system files or hidden files appearing in reports or analyses or as part of any image processing routines.

Also.... inceasingly .... lots of monitoring and measuring systems are storing data on embedded SD cards ... so the ability to be able to select only files which are not system or hidden becomes increasingly important.

The key answer to the question you asked is ....

I do not need any special processing or handling ...... the way the system handles the files (eg copy from SD to disk is perfect) but I do need a means to recognise what files are system or hidden, so they can be excluded in certain cases and included in other scenarios. I am not too concerned as to what method is used to do that (ie I do not need the attrib function/feature modified)

Hopefully that makes sense. I am happy to clarify any points.

When I get a chance, I am going to look for a Mac utility which will list all dirs and files on an SD card and can include / exclude system or hidden files. Maybe I can get Beyond Compare or similar to create a text file of hidden or system files. My app can run that app, store the results and I can then use the information within the logic of my app.

PS... I suspect the system files are also hidden files.

Walter Roberson 2023-4-23

在 MATLAB Online 中打开

Why is it so ridulously difficult to get dir lists of files and directories on Mac and Win. This is excruciatingly painful... and seriously error prone. This might be my last MatLab project.

Use the ** directory specification, such as

dinfo = dir('/Volumes/Card52312/**');

This will return everything from Card52312 and later, all in one list. dinfo.folder will give the names of the containing folders and dinfo.name will give you the names inside the folder, and dinfo.isdir will tell you whether the entry is a directory or not.

Is dir() a flexible function that will allow you to filter the returned information based on various criteria? No, it is not. It is closer to the unix stat() call, which returns a list of whatever is there, expecting you to do any post-processing that might be appropriate.

Is MATLAB being especially negligent in not returning information that other programming languages makes readily available? No: C++ (2017) std::filesystem does not provide any facility for searching directories for folders/files with particular attributes, and the set of permissions supported by C++ does not include system or hidden or any analogues; https://en.cppreference.com/w/cpp/filesystem/perms

People who need filtered file information on Unix-type systems code the search themselves, or find a pre-written library that will do the search for them -- or use the standard Unix utility find https://ss64.com/osx/find.html . For example,

find . -flags +hidden -print

./DCIM/114NIKON/._DSCN2398.JPG

./DCIM/114NIKON/._DSCN2399.JPG

./DCIM/114NIKON/._DSCN2406.JPG

./NIKON001.DSC

./._.Trashes

./.Trashes

./.fseventsd

./System Volume Information

find . -type d -flags +hidden -print

./.Trashes

./.fseventsd

./System Volume Information

... which still does not permit you to look for system

(Side note: Using ** or using the find utility might potentially cause MacOS to "materialize" dataless files and directories; See https://github.com/fish-shell/fish-shell/issues/8399 )

Matt O'Brien 2023-4-25

编辑：Matt O'Brien 2023-4-25

I did a very quick Google search for options to identify hidden files in Swift / Xcode (ie native App on Mac).

On my first search I found the following .... I highlighted the most interesting sections of this example.

I cannot be sure this will work... but it looks promisiong.

https://stackoverflow.com/questions/27721418/getting-list-of-files-in-documents-folder

At this stage, I am not spending any more time on this. I will put my use of MatLab on hold... but expect I will abandon MatLab completely.

I will also warn my colleagues and interested parties in relation to this major black hole when trying to process image data from Media cards. My purpose in writing the app was to be able to help others manage high volume image processing. As I know how to manage this stuff myself I see no point in providing an app which is doomed to failure in due course. I can hard code my app if I wish to deal with cameras I use myself to test if I have identifed and coded for all system and hidden files.

I am going to take a rest from this.

My next step will be to see if I can find a Mac Utility which will identify hidden / system files, but I have spent way too much time on this (and learning MatLab) already.

Thanks to Walter for his time and efforts.

Regards.

Matt O'Brien 2023-4-25

编辑：Matt O'Brien 2023-4-25

Can some one show me how to execute the following 'ls' commands from inside MatLab, along the following lines.

MyLsList = ls -a MyDirString

MyLsList = ls MyDirString

so I can capture the related ls output in the variable MyLsList and MyDirString contains the target directory of my choice.

I can get the Ls command to work but unable to add in the -a parameter. If I can get these two Ls commands to work, one should contain all all entries incl hidden and the second may be the list I am looking for.

Or perhaps I need to recognise the difference between these lists to find the hidden entries.

This would be a possible workaround for me.

dpb 2023-4-25

编辑：dpb 2023-4-25

在 MATLAB Online 中打开

I don't have a non-Windoes OS to illustrate with, but the format should suffice to see syntax.

>> [~,s]=system('dir /a:h /b c:\*.sys')
s =
    'hiberfil.sys
     pagefile.sys
     swapfile.sys
     '
>> 

For a variable string use something like (be sure to be careful to keep embedded spaces where needed between command switches and the filename, etc., ...). For example,

cmd=strcat('dir /a:h /b ',fn);

looks just right, doesn't it? But, it has the nasty behavior that it does an internal strtrim() on the arguments before doing the catenation so you end up with the filename string being directly appended to the last command switch and Boom! the call fails and you'll spend a long time looking to figure out why... :( To make the behavior even more obtuse, if you use a cellstr or string class, then it doesn't trim--go figure.

>> fn=fullfile('c:','*.sys');
>> cmd=['dir /a:h /b ' fn];
>> [~,s]=system(cmd)
s =
    'hiberfil.sys
     pagefile.sys
     swapfile.sys
     '
>> 

NOTA BENE: the above is one string with embedded \n -- remember the caveat that hopefully nobody was nasty enough to have used one in an actual filename on your system...that's a nasty.

>> split(s,newline)
ans =
  4×1 cell array
    {'hiberfil.sys'}
    {'pagefile.sys'}
    {'swapfile.sys'}
    {0×0 char      }
>>

That's with MS CMD processor that MATLAB uses "out-of-the-box" -- another aberration is that TMW ignores the COMSPEC environment variable so one has to go thru gyrations to get another shell. I use the JPSoft (shameless plug for super product; not related, just 20+ year user) replacement which has more functionality included a "/f" switch besides "/b" to remove the header/trailer info; it returns the fully-qualified name for all.

@Walter Roberson is bound to come along before long and can/will give a specific example, undoubtedly! :)

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt O'Brien 2023-4-25

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1950983-exclude-system-or-hidden-folders-from-mac-dir-list#answer_1223044

在 MATLAB Online 中打开

Thanks for the syntax.

I got the following to work ... in principle ...

-a includes hiddenitems

-R recursive mode ...

clc
MyLsString= strcat('ls -a -R /Volumes/MoB_WD_5TB/DeeAprilTravel2023');
[~,s]=system(MyLsString);
MyList = splitlines(s);
MyLen = length(MyList);
z=0;    

But I am not sure I have the skills or trust my efforts to untangle the output.

Splitlines.

I used split lines to get lines raather than a single long text string. This seems to work ... but not sure I can trust the output....

More troubling....

Data Returned.

No doubt this data can be unscrambled... but I would not trust myself to do this.

Also ... I see lots of funny scenarios.

eg. Line 7 DCIM is a very important sub directory ... but it is on the same line as 'fseventsd-uuid'

Maybe this is a splitline issue.

So ... my summary ... Good to know how to issue system commands from Matlab .... but looks scary to try and convert this into useful data to resolve my issue.

4 个评论
显示 2更早的评论隐藏 2更早的评论

dpb 2023-4-25

在 MATLAB Online 中打开

Well, you get what the OS tools will give you...first thing I'd do would be to compare to the output format at the command line to see how it formats the output; as noted before there are "issues" with OS'es that allow such abominations as a \n as an allowable character in a filename -- or, perhaps some of the problems may be there are other nonprinting characters embedded in those -- only more spelunking will let you know what it is that is in some of those filename strings for certain.

Good luck; it reminds me why I an always glad despite its warts in other areas to not have to have dealt with the --ix file system.

From the stuff you posted before, it would look to me still like the more sure path would be to use a mex gateway into the SWIFT OS calling stuff you found above, although I'd definitely to some more exhaustive looking at the details of what ls returns and the actual returned formatted output. I've been know to use output redirection and capture a file in the past, too...what does the output of

ls -a -R /Volumes/MoB_WD_5TB/DeeAprilTravel2023 >lsOut.txt

look like? Is it unwrapped differently?

BTW in

MyLsString= strcat('ls -a -R /Volumes/MoB_WD_5TB/DeeAprilTravel2023');

the strcat function is totally superfluous...but, you probably already knew that...

8 个评论
显示 6更早的评论隐藏 6更早的评论

dpb 2023-5-8

I still agree with the initial point that TMW should have either incorporated the common OS switches in the existing dir() function LONG ere now (like in the beginning) or have produced alternate functionality. It is, after all, the user-friendly rapid-development environment that is what it's all about; leaving this to the user to deal with is, simply put, "just plain rude!" as one of my granddaughters is wont to opine...it's doable, yes, but why can't we simply just use "/h" or "/-h" in some form other than calling the OS and parsing it ourselves?

Back years ago, I had done for Windows; unfortunately, it was another of the utilities that got left on the corporate machine when retired to the farm and forgot to offload that stuff before leaving...what I really miss is the mex extension that took a Fortran FORMAT string and variables then passed on the the Fortran i/o library --a MUCH nicer solution than writing those ugly C formatting strings with no repeat counts, etc., ...without paying consulting client, it hasn't been enough of an issue to have bothered to regenerate although at the moment I can't even get a Fortran compiler to link mex; not having the "high-priced spread" of the Intel compiler. Since they support gcc, there's no reason the Fortran compiler shouldn't be supported as well, but I've been unable to solve the name-mangling issues to link. Anyways, sorry for the side rant; we old fogeys tend to wander off...

Matt O'Brien 2023-5-8

Aside. I cut my teeth using Fortran. There is a gas pipeline in Ireland whose concrete coating thickness (to keep submerged when crossing rivers, lakes and estuaries) I calculated with a few lines of Fortran ... which I believe is still doing the job it was designed to do.

I strongly believe the fundimental point that the dir() function needs the required switches or an sdir (Super Dir) function is implemented.

My dream language to use was Cobal for so many reasons. Brilliant for combining logic and data structures (but geared for large scale data processing).

请先登录，再进行评论。

Answer 3

Matt O'Brien 2023-5-8

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1950983-exclude-system-or-hidden-folders-from-mac-dir-list#answer_1231279

I am keen to supply feedback asap .... realised I needed to run the script in a Mac environment...

I copied the struct output from your script above into a table and saved to Excel.

With the test data I currently have on the card / card reader ... I have approx 700 lines of files/folders...

My initial dive into this is encouraging... with a little twist...

The folders, listed in these screen grabs, I am keen to exclude .... It looks like the script has successfully excluded these folders but left the files (or some of them) from within these folders. I only found these 3 examples.

I need to do a more thorough job of checking the test data and maybe testing a few other scenarios (eg different cameras). I may be missing some outliers in the other 700 rows, but initial inspection is looking very promising.

No need to try and tweak the script at this point. I will revert when I am comfortable I have digested the results.

I may need to get the list of 'hidden directories' and then specifically exclude / ignore the content of this directories.

(Ps1. Just FYI. there were dot and double dot folders included in the list ... which I am comfortable getting rid of.)

(Ps2. I will try to acquire a better grasp of the system 'find' command).

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 4

Matt O'Brien 2023-5-9

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1950983-exclude-system-or-hidden-folders-from-mac-dir-list#answer_1231534

Just a quick update.

I am getting very confusing results. While the system Find results were initially promising, the MatLab dir command is now giving me identical results as using the System find command, as per the the script above. Exactly the same number of files / directories returned by both (ie 676, so a decent volume of data). I expect the MatLab dir command would include all the hidden folders.... but now... they are excluded ?????

My suspicision (worry) is that setting the option (in Finder) to display or hide hidden objects is having an effect on the respective find and dir commands when executed from MatLab. I need to check this further. If true, this is a disaster.

The second point of confusion is the following scenario, hinted in a note above.

Sony creates a hidden AV-Info directory (might be spelled wrong, writing on my ipad). This contains several files. Both Dir and Find lists do not include the folder name, but do include the folder files. My simple assumption is that the folder is flagged on the system as hidden but the respective files in the folder are flagged as visible. If so, it means that I have to specifically identify folders which are hidden and then remove all their contents from the list supplied via Dir or Find.

I need to dig deeper.

This is horrendously time consuming... I am not sure at what point I give up. I will continue for the moment.. but growing weary.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 5

Matt O'Brien 2023-5-9

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1950983-exclude-system-or-hidden-folders-from-mac-dir-list#answer_1231674

I am posting this... just in case there is interest ...I do not expect anyone to dig thru this stuff.

I found a glitch above and have completed a detailed comparison of the List genberated by MatLab Dir and System Find.

The result is close to the desired ... but there are a few glitches.

I am attaching an Excel spreadsheet which has a detailed line by line comparison of both lists, where I have commented on the differences (most at top and bottom of the list).

I am also enclosing a pdf with a few screen grabs, incl a screen grab of most of the folders expaned on the SD card, using Finder (and showing hidden items).

I need to digest this myself.... but looks like the Find list is close but will need to exclude visible files/folders stored in hidden folders (which may be several folders deep... ie the parent folder maybe visible but the parent of the parent may be hidden).

I will leave this for a few hours now and will review myself with a fresher pair of eyes.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Exclude System or Hidden Folders from Mac Dir List

25 个评论
显示 23更早的评论隐藏 23更早的评论

回答（5 个）

4 个评论
显示 2更早的评论隐藏 2更早的评论

8 个评论
显示 6更早的评论隐藏 6更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Exclude System or Hidden Folders from Mac Dir List

25 个评论 显示 23更早的评论隐藏 23更早的评论

回答（5 个）

4 个评论 显示 2更早的评论隐藏 2更早的评论

8 个评论 显示 6更早的评论隐藏 6更早的评论

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

25 个评论
显示 23更早的评论隐藏 23更早的评论

4 个评论
显示 2更早的评论隐藏 2更早的评论

8 个评论
显示 6更早的评论隐藏 6更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论