Main Content

Define MATLAB Job Scheduler Startup Parameters

MATLAB® Job Scheduler service runs using several parameters. These parameters set the process name, the user name, log file location, ports, and so on of the MATLAB Job Scheduler cluster.

You can edit the parameters in the mjs_def before installing or starting the mjs service. Find this file in these locations:

  • matlabroot\toolbox\parallel\bin\mjs_def.bat on Microsoft® Windows® operating systems

  • matlabroot/toolbox/parallel/bin/mjs_def.sh on Linux® operating systems.

The mjs service reads the mjs_def file when you start or stop the mjs service, run the mjs service with the arguments console, or install or uninstall the mjs service on a Windows operating system. When you start and stop workers and job managers, they contact the mjs service they are running under to obtain the values of the definitions and defaults stored in this file. Thus, the mjs service does not read the mjs_def file again when starting and stopping workers and job managers.

For a working cluster setup, the mjs_def files must use consistent settings across all hosts of the cluster.

Note

If you want to run more than one job manager on the same host, they must all have unique names. Specify the names using flags with the startup commands.

The user-configurable parameters in the mjs_def file are defined below.

MATLAB Job Scheduler Process and Logging

Parameter

Description

HOSTNAME

Name of the host reading the mjs_def file.

The processes under the mjs advertise themselves with the host name, HOSTNAME. The job manager must be able to resolve the host name that the MATLAB workers advertise, and all MATLAB workers and clients must be able to resolve the host name that the job manager advertises.

On Windows operating systems, the default HOSTNAME is the fully qualified domain name (FQDN) of the host. The mjs service can use the FQDN if the client resolves the short name of the host to a different host. The mjs service uses this command to determine the FQDN of the host.

for /F "usebackq skip=2 tokens=4" %%A in (`reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters /v "NV Hostname" 2^>nul`) do (
	    set strHostname=%%A
)

for /F "usebackq skip=2 tokens=4" %%A in (`reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters /v "NV Domain" 2^>nul`) do (
	    set strDomain=%%A
)
HOSTNAME=%strHostname%.%strDomain%

On Linux operating systems, the mjs service determines the default HOSTNAME with this command.

HOSTNAME=`hostname -f`

Change this setting only if:

  • the code or command above returns an incorrect host name, or

  • the command above is not available, or

  • the code or command above returns a host name that cannot be resolved by all the other computers that the processes under the mjs service communicates with

BASE_PORT

Base port of the mjs service.

The mjs service uses as many ports as required, starting with BASE_PORT. By default, BASE_PORT is 27350.

If you use a host that runs a total of nJ job managers and nW workers, the mjs service reserves a total of 7+2*nJ+4*nW consecutive ports for its own use. All job managers and workers, even those on different hosts, that are going to work together must use the same base port. Otherwise the job managers and workers will not be able to contact each other. In addition, MPI communication occurs on ports starting at BASE_PORT+1000 and use 2*nW consecutive ports.

For example, if you use a host with 1 job manager and 16 workers, then you need the following ranges of ports to be open:

  • 27350 – 27422 for the mjs service.

  • 28350 – 28382 for MPI communication.

To connect from MATLAB to a cluster with a non-default BASE_PORT, you must append the value of BASE_PORT to the 'Host' property in the MATLAB Job Scheduler cluster profile. You must do this in the form Hostname:BASE_PORT, for example myMJSHost:44001.

MJSUSER

Username to start the mjs service.

By default, all the processes that the mjs service manages are associated with the user who starts the mjs service. If MJSUSER is not specified, the mjs service runs as LocalSystem on Windows operating systems, or as root user on Linux operating systems.

You can set this parameter to run the mjs service as a different user from the user who starts the service. On Windows operating systems, set the value before installing the service; on Linux operating systems, set the value before starting the service.

The MJSUSER must have access to the MATLAB installation folder. You can specify MJSUSER as DOMAINNAME\USERNAME, or as .\USERNAME if the user is a local user.

For more information about the privileges and requirements for the MJSUSER, see Set the User.

MJSPASS

Password for username specifies in the MJSUSER parameter (Windows operating systems only).

To run the mjs service as MJSUSER, you can specify a password for that user. You can either set the password in the mjs.def file or, if unset, the mjs service prompts you for a password when you start the mjs service.

If you specify the password in the mjs_def file and the password uses characters that are treated as special characters in a batch file, they must be replaced with an appropriate escape sequence:

  • A single exclamation point (!) in the password must be given as ^^!.

  • A single percent (%) in the password must be given as %%.

  • A single caret (^) in the password must be given as ^^^^.

  • Other special characters & < > [ ] { } = ; ' + , ` ~ must be escaped with a single preceding ^.

PIDBASE and LOCKBASE

Folder used to store PID and lock files. (Linux operating systems only.)

The PIDBASE and LOCKBASE folders contain the PID and lock files for the mjs service. The default for PIDBASE is /var/run and the default for LOCKBASE is /var/lock/subsys.

You must ensure that the user the mjs service runs as has write access to this folder.

LOGBASE

Folder used to store mjs log files.

The LOGBASE folder contains the log files the mjs service generates during the normal course of operation.

  • On Microsoft Windows operating systems, the default LOGBASE folder path is %PROGRAMDATA%\MJS\Log, where %PROGRAMDATA% is the system program data environment variable. For example, if %PROGRAMDATA% is set to C:\ProgramData, the log files are placed in C:\ProgramData\MJS\Log. You must not enclose the folder name in double quotes and the folder name must not contain parenthesis.

  • On Linux operating systems, the default is /var/log/mjs.

You must ensure that the user the mjs service runs as has write access to this folder. For security level 3 worker hosts, the user the mjs service runs must also be able to grant write privileges to this folder to all users who run jobs and tasks on the host.

CHECKPOINTBASE

Folder used to store mjs checkpoint folders.

The CHECKPOINTBASE folder contains persistent information related to the mjs service.

  • On Microsoft Windows operating systems, the default CHECKPOINTBASE folder path is %PROGRAMDATA%\MJS\Checkpoint, where %PROGRAMDATA% is the system program data environment variable. For example, if %PROGRAMDATA% is set to C:\ProgramData, the checkpoint folders are placed in C:\ProgramData\MJS\Checkpoint. You must not enclose the folder name in double quotes and the folder name must not contain parenthesis.

  • On Linux operating systems, the default is /var/lib/mjs.

On the host that runs the job manager, the checkpoint folder stores database information related to the job manager and can require substantial diskspace.

On the hosts that runs the workers, the checkpoint folder stores data transferred with tasks for workers.

You must ensure that the user the mjs service runs as has write access to this folder.

MJS_JAVA

Set the Java® Runtime Environment (JRE™) path for the mjs service.

By default, the mjs service uses the Java installation that ships with MATLAB. To use your own JRE installation for the mjs service, specify a path to an installed JRE. For information about Java versions compatible with MATLAB, see MATLAB Interfaces to Other Languages.

Job Manager Security

Parameter

Description

SECURITY_LEVEL

Security level for the cluster.

You can specify the security level for your MATLAB Job Scheduler cluster. Select from one of these levels:

  1. Level 0 – No security features are enabled. This is the default security level. Jobs are not protected.

  2. Level 1 – this level warns users when they try to access other users' jobs and tasks. However, all users can still perform all actions.

  3. Level 2 – users must enter a password to access their jobs and tasks. Other users do not have access unless it is granted by the job owner through the AuthorizedUsers job property.

  4. Level 3 – same as level 2 except this level runs jobs and tasks on the workers as the user to which they belong. The password must be the system or network password used to log on to a worker host. Only trusted workers (i.e. workers with an identical shared secret file) can connect to the job manager, see SHARED_SECRET_FILE below.

    You must set the USE_SECURE_COMMUNICATION parameter to true. On Linux operating systems, you must run the mjs service as the root user.

For more information, see Set Security Level.

ADMIN_USER

Username of the cluster administrator account.

At Security Level 2 and 3, the administrator can access and manipulate all users' jobs and tasks. They can also change users' passwords if LDAP server authentication is not enabled.

The mjs service prompts you to provide a password for the administrator account when you start the job manager or perform administrator operations.

If LDAP server authentication is enabled, the username specified in the ADMIN_USER parameter must be a valid username in the LDAP server. To authenticate the administrator's username against the LDAP server, you must provide the password for the username specified in ADMIN_USER when you start the job manager.

If you do not specify a username, the cluster administrator account username defaults to admin.

USE_SECURE_COMMUNICATION

Use encrypted communication between services.

Set up encrypted communication between job manager, client and workers.

By default, job managers and workers communicate over non-encrypted channels. This can be useful when you do not have a requirement to protect the data or the connection is already protected from unauthorized access (for example, if your cluster network is isolated and has no access to the Internet). Additionally, encrypted communication can cause a performance decrease due to the extra overhead incurred when encrypting messages.

When you set USE_SECURE_COMMUNICATION to true, communication between the MATLAB client, job managers, and workers is encrypted.

Encrypted communication requires a shared secret file on each participating host (excluding the MATLAB client). For more information about the shared secret file, including how to generate one, see the description for SHARED_SECRET_FILE below.

USE_SECURE_COMMUNICATION is enabled by default when you set the cluster to Security Level 3. To establish encrypted communication between MATLAB Job Scheduler, client and workers, see Set Encrypted Communication.

REQUIRE_SCRIPT_VERIFICATION

Require verification for privileged commands sent to the cluster.

By default, REQUIRE_SCRIPT_VERIFICATION is set to false and the mjs service will run commands sent to it without any verification. This allows anyone to modify the state of the cluster, for example, by starting or stopping worker or job manager processes.

Privileged commands include any command which can modify the cluster state. When you set REQUIRE_SCRIPT_VERIFICATION to true, the mjs service verifies each command with the secret file before execution. Command verification is enabled by default when you set the cluster to Security Level 3.

For more information, see Set Cluster Command Verification.

SHARED_SECRET_FILE

Location of the shared secret file.

To establish encrypted connections between job managers and workers, a shared secret file is required on all participating hosts. You can create a shared secret file with the createSharedSecret script. For more information, see Create Shared Secret File.

If the shared secret file does not already exist when you start the mjs, the mjs service creates one in the default location: %CHECKPOINTBASE%\security\secret.

When you set the cluster to Security Level 3, the mjs uses the shared secret file to ensure only trusted workers can connect to the job manager.

Note

Secret files contain sensitive data and you must protect it against unauthorized access. Anyone that can access the secret file might be able to eavesdrop on the connections between services.

ALLOW_CLIENT_PASSWORD_CACHE

Remember user passwords for future sessions.

If set to true, this option allows users to let the MATLAB client remember their login information for future client sessions. Users can still choose to not store any information at the password prompt in MATLAB.

ALLOWED_USERS

List of users allowed to log on to the job manager.

This property defines a list of users that are allowed to access the job manager. Multiple usernames must be separated by commas. By default, ALLOWED_USERS is set to ALL.

To allow any user to access the job manager, use the keyword ALL instead of a list of usernames.

REQUIRE_CLIENT_CERTIFICATE

Require MATLAB clients to present a certificate to connect to the job manager when using encrypted communication.

By default, REQUIRE_CLIENT_CERTIFICATE is set to false. If you set REQUIRE_CLIENT_CERTIFICATE to true, cluster users must have a cluster profile containing the certificate to connect to the job manager. For more information, see Set MATLAB Client Verification.

WORKER_DOMAIN

Windows domain workers use when logging on at security level 3 (Windows operating systems only).

To run tasks as the user that submitted them, Windows requires a domain in addition to the user name. By default, if a task belongs to USER, the software runs the task as %WORKER_DOMAIN%\USER.

In most circumstances the default value is correct and must not be altered.

Note

This property is required only when running on Windows at security level 3.

USE_LDAP_SERVER_AUTHENTICATION

Option to use an LDAP server to authenticate user credentials. (since R2023a)

By default, USE_LDAP_SERVER_AUTHENTICATION is set to false. To enable LDAP server authentication for a Security Level 2 or 3 cluster, set this parameter to true. For more information, see Configure LDAP Server Authentication for MATLAB Job Scheduler.

Cluster users must use their LDAP server username and password to connect to the cluster.

LDAP_URL

URL of the LDAP server to authenticate users. (since R2023a)

If you have configured the cluster to use LDAP server authentication, you must set the LDAP_URL. Specify the LDAP_URL as:

ldap://HOST:PORT

If you have configured your LDAP server over SSL, specify the URL as:

ldaps://HOST:PORT

For more information, see Configure LDAP Server Authentication for MATLAB Job Scheduler.

LDAP_SECURITY_PRINCIPAL_FORMAT

Format of a security principal for the LDAP server. (since R2023a)

You can specify the format of a security principal the LDAP server uses for authentication.

Common formats include "[username]@domain.com" and "cn=[username],ou=Users,dc=domain,dc=com". The username of the principal replaces the [username] token during authentication.

LDAP_SYNCHRONIZATION_INTERVAL_SECS

Frequency at which the cluster synchronizes with the LDAP server. (since R2023a)

The default value is 1800 seconds, which will synchronize the cluster with the LDAP server every half hour.

You can specify one of the following:

  • A positive number corresponding to the number of seconds between synchronizations. Access to the cluster might be granted for old/expired credentials until the next synchronization.

  • 0. The cluster synchronizes with the LDAP server on every cluster access requiring authentication.

EXPORT_METRICS

Option to export cluster monitoring metrics from job manager. (since R2024b)

You can export metrics from the job manager for integration with cluster monitoring tools. For more information, see Configure Metrics for MATLAB Job Scheduler.

METRICS_PORT

Port for exporting metrics. (since R2024b)

The job manager uses an HTTP(S) server at the port you specify to export metrics. By default, this port number is 8001.

USE_SECURE_METRICS

Option to use encrypted communication to export metrics. (since R2024b)

By default, USE_SECURE_METRICS is set to true and the job manager exports metrics on an encrypted HTTPS server. To disable encryption and export metrics on an HTTP server, set USE_SECURE_METRICS to false.

METRICS_CA_FILE

Job manager's trusted certificate authority (CA) for encrypted metrics. (since R2024b)

Path to a CA certificate to verify client certificates. If USE_SECURE_METRICS is true, the job manager only forms encrypted connections to clients that present certificates signed by this CA. You must set this parameter if USE_SECURE_METRICS is true.

You can use the mjssetup tool to generate the job manager's CA certificate. To learn more, see Configure Metrics for MATLAB Job Scheduler.

METRICS_CERT_FILE

Job manager's certificate for encrypted metrics. (since R2024b)

Path to the job manager's public certificate. The job manager presents this certificate to clients that want to form encrypted connections for metrics. You must set this parameter if USE_SECURE_METRICS is true.

You can use the mjssetup tool to generate the job manager's public certificate. To learn more, see Configure Metrics for MATLAB Job Scheduler.

METRICS_KEY_FILE

Job manager's private key for encrypted metrics. (since R2024b)

Path to the private key associated with the certificate file you specify for METRICS_CERT_FILE. You must set this parameter if USE_SECURE_METRICS is true.

You can use the mjssetup tool to generate the job manager's private key. To learn more, see Configure Metrics for MATLAB Job Scheduler.

Job Manager and Worker Settings

Parameter

Description

DEFAULT_JOB_MANAGER_NAME

Default name of the job manager.

When you start a new job manager, it needs to be identified by a name on the network, and when a new worker is started, it needs to know the name of the job manager to register with. The values specified with the DEFAULT_JOB_MANAGER_NAME parameter is the default job manager name used in both cases.

To override the default job manager name, use the startjobmanager command with the -name flag. To change the job manager name on a worker host, use the startworker command with the -jobmanager flag.

JOB_MANAGER_HOST

Host on which the job manager lookup process runs.

The MATLAB worker processes and the job manager process contact the job manager lookup process on the specified host.

You can override the job manager host using the startworker command with the -jobmanagerhost flag.

SAVE_JOB_HISTORY

Save job and task accounting information to disk. (since R2024a)

By default, SAVE_JOB_HISTORY is set to true and the job manager stores job and task information in job history database located in the CHECKPOINTBASE folder. The job history files have a 10GB limit. You can extract and analyze the job history data for accounting purposes. To disable the saving of job history data, set this parameter to false. For more information, see Manage and Access MATLAB Job Scheduler Job History.

JOB_MANAGER_MAXIMUM_MEMORY

Maximum heap size of the job manager Java process.

You can adjust the amount of heap memory for the job manager Java process. By default, JOB_MANAGER_MAXIMUM_MEMORY is set to 1024m. For recommendations on when to adjust the heap memory for the job manager, see Increase Heap Memory

DEFAULT_WORKER_NAME

Default name of the worker.

You can override the default worker by calling the startworker command with the -name flag. Worker names must be unique on each host, therefore, you must use the -name flag with the startworker command if you want to start more than one worker on a single host.

WORKER_MAXIMUM_MEMORY

Maximum heap size of all worker Java process.

You can adjust the amount of heap memory for the worker Java process. By default, WORKER_MAXIMUM_MEMORY is set to 1024m. The WORKER_MAXIMUM_MEMORY does not impact the amount of memory the MATLAB worker can use.

WORKER_START_TIMEOUT

Number of seconds for worker sessions wait for MATLAB to start before detecting a stall.

The default value is 600s. You must select a value greater than the time it takes for a MATLAB session to start.

MATLAB_SHELL

Shell workers use in system calls (Linux operating systems only).

You can specify the shell that is spawned when a worker invokes a MATLAB system command. For example, to use the Bourne shell, set MATLAB_SHELL to /bin/sh.

MATLAB checks internally for the MATLAB_SHELL variable first and, if empty or not defined, then checks SHELL. If SHELL is also empty or not defined, MATLAB uses the Bourne shell, /bin/sh. The value of MATLAB_SHELL must be an absolute path, that is /bin/sh, not simply sh.

Note that some shells (for example tcsh) might choose to interpret a system command in MATLAB as a request to start up a new login shell, rather than a subshell, which might have consequences for environment variable changes you have made in MATLAB, for example to the PATH environment variable that would allow other applications to run.

USE_ONLINE_LICENSING

Use a license that is managed online.

Set USE_ONLINE_LICENSING to true to enable on-demand licensing or to use a license that is managed online.

When enabled, users must log in to their MathWorks® account to connect to the cluster, and their account must be linked to a MATLAB Parallel Server™ license that is managed online.

ALL_SERVER_SOCKETS_IN_CLUSTER

Initiate connections from the client to the cluster.

To make all connections outbound from client, set ALL_SERVER_SOCKETS_IN_CLUSTER to true. To allow inbound connections to the client from the job manager and workers in addition to outbound connections from the client, set ALL_SERVER_SOCKETS_IN_CLUSTER to false.

MJS_ADDITIONAL_MATLABROOTS

Path to additional installations of previous MATLAB releases.

MJS_ADDITIONAL_MATLABROOTS is required only when specifying installations of MATLAB that are different from the MATLAB Parallel Server version in use.

You must list the path to the MATLAB installation without quotes. Multiple entries are allowed separated by semi-colons, for example:

MJS_ADDITIONAL_MATLABROOTS=C:\Program Files\MATLAB\R2016a;C:\Program Files\MATLAB\R2016b

Only versions of MATLAB released prior to the MATLAB Parallel Server version you are using are supported, starting with R2016a.

MATLAB clients can only use this cluster if a path to their version is specified.

MAX_LINUX_WORKERS

The maximum number of Linux workers the cluster can scale up to on demand. Only set this value if you have set up your cluster for auto-resizing. For more details, see Set up MATLAB Job Scheduler Cluster for Auto-Resizing.

MAX_WINDOWS_WORKERS

The maximum number of Windows workers the cluster can scale down to on demand. Only set this value if you have set up your cluster for auto-resizing. For more details, see Set up MATLAB Job Scheduler Cluster for Auto-Resizing.

SCHEDULING_ALGORITHM

Scheduling algorithm for the job manager. (since R2023b)

Select from one of these options:

  • standard: (default) Spreads communicating jobs across as few worker machines as possible to reduce communication overheads and fills in unused spaces on worker machines with independent jobs. This scheduling algorithm provides good behaviour for a wide range of uses including clusters with auto-resizing or autoscaling enabled.

  • loadBalancing: Distributes load evenly across the cluster to give as many resources as possible to running jobs and tasks when the cluster is underutilized.

See Also

| |

Related Topics