For SSH:
The password dialog will be shown if public key authentication failed but additional authentication methods are available on the server. One possible reason for this public-key authentication failure is MATLAB finding ~/.ssh/id_rsa or ~/.ssh/id_dsa or ~/.ssh/identity file(s) and attempting to authenticate over SSH with them, but none of these files contained a key which MATLAB was able to work with. This may for example occur if your ~.ssh/id_rsa key is in OpenSSH format instead of the RSA format.
You can find a detailed way to proceed in the following link: