Manage Server Logs

Each microservice in MATLAB^® Online Server™ outputs log information as it runs. Checking log information can help debug issues that occur while the server is running.

Check Logs

You can get the content of server logs by using the file system, dumping log information to system output, or by streaming information to system output.

Access Logs Using the File System

Access the single node on which you installed MATLAB Online Server (for example, using ssh), and then access the folder /var/log/containers/. This folder contains logs for all the MATLAB Online Server microservices.

Issue the following command, replacing <namespace-name> with the namespace you used for MATLAB Online Server:

ls /var/log/containers/<namespace-name>

Examples:

<namespace-name>-core-ui-6679d5ff7f-rdtfb_<namespace-name>_core-ui-165cfda8ff5c4d2b01838d0acb573df881e04a08c329afcd.log
<namespace-name>-gateway-65697586f7-vsvls_<namespace-name>_gateway-78afff6773618381f1fc6ba05e63eceaf3f3a6fd5de3defd.log
<namespace-name>-matlab-pool-apparmor-loader-zdqwk_<namespace-name>_apparmor-loader-87115017a99407c3bf647712c61426d.log
<namespace-name>-matlab-pool-helpsearch-5689dd4cdf-t77rh_<namespace-name>_helpsearch-963806b3d47815d8c7df217770b38b.log
<namespace-name>-matlab-pool-ui-689fb5d9ff-tsrh4_<namespace-name>_webgui-ec90a23bbd0db048115f6fae4113084a8df6d0ef86.log

Dump Logs to System Output

Use the kubectl command to access the logs for individual pods. The following example demonstrates the command to dump the content of the logs belonging to the matlab-pool-helpsearch pod. Replace <namespace-name> with the namespace you used for MATLAB Online Server.

kubectl --namespace=<namespace-name> logs <namespace-name> -matlab-pool-helpsearch-5689dd4cdf-t77rh

If a pod has more than one container, for example, matlab-pool, you can first extract the list of its containers, and then dump the content of its container logs by specifying the name of the container.

For example, to get the list of containers for the MATLAB Pool pod, enter the following command, replacing <namespace-name> with the namespace you used for MATLAB Online Server:

kubectl --namespace=<namespace-name> get pods <namespace-name> -matlab-pool-5858d796b9-lqslb -o jsonpath='{.spec.containers[*].name}'

The results list four containers inside the MATLAB Pool pod. The containers are listed left to right with a space between each container name:

matlab resource-proxy display windowmanager

To get logs for the MATLAB container that is running inside the MATLAB Pool pod, run the following command, replacing <namespace-name> with the namespace you used for MATLAB Online Server:

kubectl --namespace= <namespace-name> logs <namespace-name> -matlab-pool-5858d796b9-lqslb -c matlab

Stream Logs to System Output

To continuously monitor the contents of logs, stream their contents by adding the -f flag to the kubectl command as follows, replacing <namespace-name> with the namespace you used for MATLAB Online Server:

kubectl --namespace=<namespace-name> logs -f <namespace-name> -matlab-pool-helpsearch-5689dd4cdf-t77rh
kubectl --namespace=<namespace-name>logs -f <namespace-name> -matlab-pool-5858d796b9-lqslb -c matlab

Configure Log Shipping

MATLAB Online Server does not come with any log shipping capabilities. To persist the MATLAB Online Server application logs for debugging and future use or in integrating with Splunk^®, follow the procedures described earlier.

Option 1: Persistence of Logs on Host

MATLAB Online Server comprises various services. Logs from multiple services can be fetched using kubectl and stored in files on disk.

For logs of MATLAB pods that are in use, perform the following steps:

Run the following command to get the status of in-use pods:

kubectl get pods -l inUse=true

This action returns the status of pods in use:

NAME                                         READY   STATUS    RESTARTS   AGE
<namespace-name>-matlab-pool-f479b5b96-d6kfk   2/2     Running   0          90m

To retrieve the logs of a particular pod, run the following commands, replacing your-namespace with the namespace you used for MATLAB Online Server, depending on which logs you want to retrieve:

License service logs

kubectl logs <license-pod-name> -n <namespace-name> >> <path/to/license/log/file>

authnz service logs

kubectl logs <authnz-pod-name> -n <namespace-name> >> <path/to/authnz/log/file>

MATLAB logs

kubectl logs <matlab-pod-name> -n <namespace-name> -c matlab >> <path/to/matlab/log/file>

Resource proxy logs

kubectl logs <resource-proxy-pod-name> -n <namespace-name> -c resource-proxy >> <path/to/proxy/log/file>

These examples illustrate storing log files on the host. You can persist logs on any other location, for example, an NFS server of your organization, where they can be exported with logging tools of your choice for more analysis and for setting up alerts.

Option 2: Integrate with Splunk (Advanced)

Follow these instructions to integrate Splunk with MATLAB Online Server.

Note

Depending on the platform you have chosen, you might have to make changes for Splunk integration to work. For resources, check the documentation from your product vendor.

Create a minimum of two Splunk indices:
- One events index, which handles logs and objects (you can also create two separate indices for logs and objects)
- One metrics index
If you do not configure these indices, Splunk Connect for Kubernetes^® uses the defaults created in your HEC (HTTP Event Collector) token.
Create an HEC token if you do not already have one.
Create a namespace for Splunk Connect for Kubernetes with the following command:
```
kubectl create ns mos-splunk-connect-k8s
```

Create the file values.yaml with the following contents (type lines exactly as shown):

kubernetes:
 clusterName: "<cluster-name>"
logLevel: "warn"
 global:
  splunk:
   hec:
    token: "<hec-token>"
    host: "<splunk-host>"
    port: <splunk-port>
    protocol: "http"
    insecureSSL: true
    indexName: "<splunk-index-name>"
journalLogPath: "/var/log/journal"
logs:
 matlab-pool:
  from:
   pod: "*-matlab-pool"
   container: "matlab"
  multiline:
   firstline: "/^\[#\|\d{4}-\d{1,2}/"
  bootstrap:
   from:
    journald:
     unit: "kubeadm.service"
   timestampExtraction:
    regexp: "\w(?<time>[0-1]\d[0-3]\d [^\s]*)"
     format: "%m%d %H:%M:%S.%N"
    sourcetype: "kube:kubeadm"
  resource-proxy:
   from:
    pod: "*-matlab-pool"
    container: "resource-proxy"
   multiline:
    firstline: "/^\[#\|\d{4}-\d{1,2}/"
   bootstrap:
    from:
     journald:
      unit: "kubeadm.service"
     timestampExtraction:
      regexp: "\w(?<time>[0-1]\d[0-3]\d [^\s]*)"
      format: "%m%d %H:%M:%S.%N"
     sourcetype: "kube:kubeadm"

Install Splunk Connect for Kubernetes.

helm install --name mos-splunk-connect --namespace mos-splunk-connect-
k8s -f values.yaml https://github.com/splunk/splunk-connect-for-
kubernetes/releases/download/1.3.0/splunk-connect-for-kubernetes-1.3.0.tgz

Note

This procedure is just one method to integrate Splunk with Kubernetes. There are other methods that you can choose based on your requirements.

There are also various log shipping options with Kubernetes that you can choose to run, depending on your organization's needs and availability of tools.

Configure Log Rotation

Kubernetes does not rotate logs automatically. You can configure log rotation manually. For example, you can deploy logrotate as a daemon service on a MATLAB Online Server node and then configure it to run each hour.

MATLAB Online Server logs are under the folder /var/log/containers/. Use the logrotate configuration file to configure the logs. By default, this file is /etc/logrotate.conf. For more information, see Check Pod Status.