You can set all the Mask properties using the Simulink.Mask class object from the command line. Use these links for Simulink.Mask class and How to Control Masks programatically: -
I hope when you mentioned ‘visibility flag’, you meant the Icon Transparency tab in Mask editor, which corresponds to IconOpaque property in Simulink.Mask class.
If by ‘Inside a Masked-Subsystem’, you meant about displaying an image on the Mask Icon then you can go to Mask->Edit Mask(Ctrl+M) and in the Icon Drawing pane, you can write the code for your Image. For example :-
Mobjt = Simulink.Mask.get('test/Sub');%Creating a Mask object for required Subsystem
if strcmp(Mobjt.IconOpaque,'on')%When the Icon Transparency is Opaque
A = imread('1.jpg');%Read the required image
image(A);%Displaying it on the Mask
else %When the Icon Transparency is Transparent
B = imread('2.jpg'); %Read the required image
image(B);%Displaying it on the Mask
end
But, if you want to display your Image inside the Subsystem then you can create another Subsystem(inside the Subsystem) and repeat the same steps as above.
You can also look in the following link below on How to Draw Mask Icon : -
Hope this helps!
