When you are creating the mask, right click on the block and choose "edit MASK" then in the "Parameters" pane of the "Mask Editor" you will see your parameters defined. When you click on each of the parameters, you can deinf e callback function for each of the parameters by writing an m-file's name in Dialog callback. Or you can just simply write pieces of code there in Dialog callback. This code runs as a callback for that parameter.
In that code you can change the visibility of the mask parameters(as well as enabling or disabling them) with commands like this:
set_param(gcb,'MaskVisibilities',{'on';'off'})
where I make the first parameter visible but the second one invisible, and
set_param(gcb,'MaskEnables',{'on';'off'})
where I make the first parameter enabled but the second one disabled.
