Set MATLAB Job Scheduler Cluster Security

Set Security Level

Set the MATLAB^® Job Scheduler security level with the SECURITY_LEVEL parameter in the mjs_def file before starting the mjs service on your cluster nodes. The mjs_def file indicates what values you can set and briefly describes each security level.

This table describes the available security levels for accessing MATLAB Job Scheduler and its jobs.

Security Level	Description	User Restrictions
0	No security. Any user can access any job. Tasks are associated with the user who started the mjs process on the worker machines (typically root or Local System). This is the default level and is the security level in all releases prior to R2010b. Jobs are associated with the default username, but the software provides no protection.	None
1	Jobs are associated with the submitting user. Any user can access any job. A dialog box warns if the accessed job belongs to another user. Tasks are associated with the user who starts the `mjs` process on the worker machines (typically root or Local System).	A dialog box prompts you to specify a username when you first access the job manager. Your MATLAB Job Scheduler username does not have to match your system or network username. You do not require a password.
2	Jobs have MATLAB Job Scheduler password protection. Jobs and tasks are associated with the submitting user and are password protected. The submitting user can authorize other users to access their jobs and tasks. In this case, an authorized user can enter their own password to access the jobs and tasks. Other unauthorized users cannot access your jobs. Tasks are associated with the user who started the `mjs` process on the worker machines (typically root or Local System).	When you start MATLAB Job Scheduler, you must provide a new password for the job manager administrator account. You can use this account to access all jobs and tasks. If you use LDAP server authentication, you must provide the LDAP server password of the administrator account when MATLAB Job Scheduler prompts you. A dialog box prompts you to specify a username and password when you first access MATLAB Job Scheduler from your MATLAB client session. Your MATLAB Job Scheduler username and password do not have to match your system or network username and password. If you use LDAP server authentication, your MATLAB Job Scheduler username and password must match the username and password in the LDAP server.
3	In addition to the security of level 2, tasks are associated with the submitting user on worker machines. Jobs and tasks are associated with the submitting user and are password protected. Other unauthorized users cannot access your jobs. Tasks are associated with the user who submitted the job.	MATLAB Job Scheduler must use encrypted communication with the workers. For more information, see Set Encrypted Communication. When you start MATLAB Job Scheduler, you must provide a new password for the job manager’s administrator account. You can use this account to access all jobs and tasks. If you use LDAP server authentication, you must provide the LDAP server password of the administrator account when MATLAB Job Scheduler prompts you. A dialog box prompts you to specify a user name and password when you first access MATLAB Job Scheduler from your MATLAB client. Your job manager MATLAB Job Scheduler username and password must be the same as your system or network username and password because the parallel workers must log you in to run the task as you. Read and write permissions to the `CHECKPOINTBASE` folder and all its subfolders must be restricted to the user who starts the `mjs` process. On UNIX systems, the root user must start the `mjs` process on the cluster nodes. On Windows systems, the submitting user must be able to log on locally to every worker machine to successfully run jobs on the cluster. You must grant each submitting user account the `"Allow log on locally"` right. If you disable this right, all the jobs you submit will fail when the cluster starts them. To enable this right, change the `SeInteractiveLogonRight` constant for the user in the User Rights Assignment security policy settings for each machine in the cluster.

Tip

Run the job manager and the workers at the same security level. The job manager does not register a worker running at a lower security level.

Local, MATLAB Job Scheduler, and Network Passwords

For any security level above level 0, when you start MATLAB Job Scheduler (for example, with the startjobmanager command), the software creates a cluster administrator account with the username specified in the ADMIN_USER parameter in the mjs_def file. If you do not specify a username, the administrator account username defaults to admin. The software prompts you to provide a password for the new administrator account. The administrator account has all the necessary permissions for accessing the cluster and all its jobs. To use LDAP server authentication, the username specified in ADMIN_USER must be in the LDAP server.

For any security level, MATLAB Job Scheduler associates every job with the user who submits it. Therefore, whenever you access MATLAB Job Scheduler or a job, MATLAB Job Scheduler must verify your identity.

At security level 0, the software sets the Username property to the login name of the person who creates the job. You can change this value at any time. For all higher security levels, the first time you access MATLAB Job Scheduler, a dialog box prompts you for your username. If the security level is 2 or 3, you must also provide a password. The username and password you provide for MATLAB Job Scheduler must match your network username and password if you are using security level 3 or if the MATLAB Job Scheduler cluster has LDAP server authentication configured. Otherwise, you can create a new username and password for MATLAB Job Scheduler. For convenience, you can choose to save your username and password on the local computer so you do not need to enter them every time you access your job.

For information about changing a password and logging out of a MATLAB Job Scheduler cluster, see changePassword (Parallel Computing Toolbox) and logout (Parallel Computing Toolbox). For more information about LDAP server authentication for MATLAB Job Scheduler clusters, see Configure LDAP Server Authentication for MATLAB Job Scheduler.

Authorize Users for Job and Task Access

This example uses:

Parallel Computing Toolbox Parallel Computing Toolbox

Open Live Script

This example shows how to authorize users to access your job on a MATLAB Job Scheduler cluster with security level 2 or 3. When you create a job and submit it to a MATLAB Job Scheduler cluster, jobs and tasks are associated with the submitting user. These jobs and tasks are password protected so unauthorized users cannot access your jobs.

Use parcluster (Parallel Computing Toolbox) to create a cluster object using the cluster profile 'MyMJSCluster'. Replace 'MyMJSCluster' with the name of your cluster profile. Then, use batch (Parallel Computing Toolbox) to create and submit a job on the cluster.

c = parcluster('MyMJSCluster');
j = batch(c,@rand,1,{2});

You can set the AuthorizedUsers property of a job to authorize user access to that job and its tasks. Each user that you specify must have already used the MATLAB Job Scheduler cluster. Authorize access to a job for users "user1" and "user2".

j.AuthorizedUsers = ["user1","user2"];

Create Shared Secret File

The secret file establishes trust between the processes on different machines.

To create this file, run one of these scripts:

matlabroot/toolbox/parallel/bin/createSharedSecret on Linux^® operating systems
matlabroot\toolbox\parallel\bin\createSharedSecret.bat on Windows^® operating systems

Specify the location of the secret file in the SHARED_SECRET_FILE parameter in the mjs_def file to enable MATLAB Job Scheduler to find it. The shared secret file contains sensitive data and must be read-only for the user who starts the mjs process.

In a shared file system, all the nodes can point to the same secret file. The nodes can also share the same mjs_def file.
In a nonshared file system, create a secret file with the provided script, then copy the file to each node and make sure the mjs_def file of each node indicates the location of its secret file.

Set Encrypted Communication

To set encrypted communication between MATLAB Job Scheduler, the client, and the workers, set these values in the mjs_def file:

USE_SECURE_COMMUNICATION = true
ALL_SERVER_SOCKETS_IN_CLUSTER = true

Encrypted communication is provided using TLSv1.3.

Before R2023a: Encrypted communication is provided via an SSLSocket using TLSv1.2.

Note

Before R2023a: If you specify ALL_SERVER_SOCKETS_IN_CLUSTER as false in the mjs_def file, then the mjs service establishes encrypted communication between MATLAB Job Scheduler and workers only. Communication between workers is never encrypted. If communication between a worker and the client is sent via another worker, only the communication between that worker and the client is encrypted.

Additionally, all hosts that run job managers or workers require the secret file at the location specified by the SHARED_SECRET_FILE parameter in the mjs_def file. To create the secret file, see Create Shared Secret File.

Note

Encrypted communication is required when you use MATLAB Job Scheduler security level 3.

Set MATLAB Client Verification

Verify whether a MATLAB client can connect to your MATLAB Job Scheduler cluster.

You must use the same secret file as the cluster to create a certificate file. Use the certificate when you start the job manager and to create a certified cluster profile. To create the secret file, see Create Shared Secret File.

Connections between the MATLAB client and MATLAB Job Scheduler cluster are verified using mutual TLS (mTLS).

Configure MATLAB Job Scheduler Cluster

In the mjs_def file, set REQUIRE_CLIENT_CERTIFICATE to true.

Navigate to one of these folders:

matlabroot\toolbox\parallel\bin on Windows operating systems
matlabroot/toolbox/parallel/bin on Linux operating systems

When you have a location for the secret file, use the generateCerticate command to generate the certificate. Specify the path to the secret file and the name of the certificate.

generateCertificate -secretfile path_to_shared_secret_file/secret -certfile mjsClusterClientCert

To start the job manager, specify the file path to the certificate to the startjobmanager command using the -certificate flag.

startjobmanager -certificate mjsClusterClientCert

Create Certified Cluster Profile

The MATLAB client must also have a cluster profile with the correct certificate to connect to the job manager.

To create the certified cluster profile, use the createProfile command. Specify the name and hostname of the cluster and the path of the certificate file. For example, create a cluster profile for the cluster clusterName, host name mjsHost, and certificate file mjsClusterClientCert. The command creates a cluster profile file clusterName with the extension .mlsettings. This file contains the certificate that the MATLAB client needs to connect to the job manager.

createProfile -name clusterName -host mjsHost -certfile mjsClusterClientCert

The certificate and associated cluster profile control which users can connect to the job manager. You must store this data securely and distribute the cluster profile to users through a secure channel. You can use the Cluster Profile Manager to import a profile into the MATLAB client. For more information, see Discover Clusters and Use Cluster Profiles (Parallel Computing Toolbox).

Set Cluster Command Verification

MATLAB Job Scheduler cluster administrators can restrict use of MATLAB Job Scheduler cluster commands to only specified users. Restrict command use to prevent unauthorized users from sending privileged commands to the cluster. Privileged commands are commands that can change the state of the cluster.

Privileged Commands

This table lists the privileged commands that require verification. You can find the executables for these commands in these folders:

matlabroot\toolbox\parallel\bin on Windows operating systems
matlabroot/toolbox/parallel/bin on Linux operating systems

Command	Description
`pausejobmanager`	Pause a job manager that is running under the `mjs` service.
`resize`	Determine or update resizing information for job manager processes under the `mjs` service.
`resumejobmanager`	Resume a job manager that is running under the `mjs` service.
`startjobmanager`	Start a job manager process and the associated job manager lookup process under the `mjs` service.
`startworker`	Start a MATLAB worker process under the `mjs` service.
`stopjobmanager`	Stop a job manager process and the associated job manager lookup process under the `mjs` service.
`stopworker`	Stop a MATLAB worker process under the `mjs` service.
`util/clusterlogs`	Set or get the log level of the `mjs` service.
`util/workerRegisterWithJobManager`	Register a MATLAB worker to a specified job manager.

Note

Command verification is enabled by default when you set MATLAB Job Scheduler to Security Level 3.

Set Command Verification in `mjs_def` File

To require verification before the mjs service executes a privileged command on the cluster, set the REQUIRE_SCRIPT_VERIFICATION parameter to true in the mjs_def file.

You must also set the SHARED_SECRET_FILE parameter to the location of the secret file used by the mjs process you are trying to send commands to.

Specify the secret file using one of these options.

Provide a secret file at the command line. You can provide the path to the secret file when you send a privileged command to the cluster.
For example, to stop a worker in the cluster, enter this command in a Windows or Linux command prompt:
```
stopworker -name worker1 -secretfile path_to_shared_secret_file/secret
```
Provide a path in the mjs_def file. If you are using the same cluster host as the location of the secret file, you can provide a path to the secret file in the mjs_def file.
Only users that have read access to the secret file can run privileged commands on the cluster.