Main Content

Resolve Evicted or Terminated Pod Issues

How Pods Become Evicted

When available compute resources are low, some pods can get into an Evicted state. When resources become available on the node, these pods are automatically rescheduled.

To see pod status, run the following command, replacing your-namespace with the namespace you used for MATLAB® Online Server™:

kubectl get pods --namespace your-namespace

How to Clean Up Evicted Pods

  • Run a cron job to delete Evicted pods at regular intervals.

  • Manually delete evicted pods using the following command, replacing your-namespace with the namespace you used for MATLAB Online Server:

    kubectl get pods --namespace your-namespace | grep Evicted | \
       awk '{print $1}' | xargs kubectl \
       delete pod --namespace your-namespace
    

How Pods Become Terminated

A Terminated state indicates that the container completed its execution and has stopped running. A pod enters the Terminated state when it has successfully completed execution or when it has failed.

Kubernetes® cleans up Terminated pods when the number of pods exceeds the configured threshold, as determined by terminated-pod-gc-threshold in the kube-controller-manager. For a single-node installation, terminated-pod-gc-threshold is set to 10.

To see the status of pods, run the following command, replacing your-namespace with the namespace you used for MATLAB Online Server:

kubectl get pods --namespace your-namespace

How to Clean Up Terminated Pods

You can clean up pods in a Terminated state using one of these methods:

  • Run a cron job to delete Terminated pods at regular intervals.

  • Manually delete terminated pods using the following command, replacing your-namespace with the namespace you used for MATLAB Online Server.

    kubectl get pods --namespace your-namespace | grep Terminated | \
       awk '{print $1}' | xargs kubectl \
       delete pod --namespace your-namespace

Occasionally, manual deletion does not actually delete the terminated pod. In this case, you might have to forcibly delete the pod. Run the following command, replacing your-namespace with the namespace you used for MATLAB Online Server:

kubectl get pods --namespace your-namespace | grep Terminated | \
   awk '{print $1}' | xargs kubectl \
   delete pod --namespace your-namespace --force --grace-period=0

For more about the kubectl command, see the kubectl Cheat Sheet at kubernetes.io.

Related Topics