Rerun Failed Tests

If a test failure is caused by incorrect or incomplete code, it is useful to rerun failed tests quickly and conveniently. When you run a test suite, the test results include information about the test suite and the test runner. If there are test failures in the results, when MATLAB displays the test results there is a link to rerun the failed tests.

Totals:
   1 Passed, 1 Failed (rerun), 0 Incomplete.
   0.25382 seconds testing time.

This link allows you to modify your test code or your code under test and quickly rerun failed tests. However, if you make structural changes to your test class, using the rerun link does not pick up the changes. Structural changes include adding, deleting, or renaming a test method, and modifying a test parameter property and its value. In this case, recreate the entire test suite to pick up the changes.

Create the following function in your current working folder. The function is meant to compute the square and square root. However, in this example, the function computes the cube of the value instead of the square.

function [x,y] = exampleFunction(n)
    validateattributes(n,{'numeric'},{'scalar'})
    
    x = n^3;     % square (incorrect code, should be n^2)
    y = sqrt(n); % square root
end

Create the following test in a file exampleTest.m.

function tests = exampleTest
    tests = functiontests(localfunctions);
end

function testSquare(testCase)
    [sqrVal,sqrRootVal] = exampleFunction(3);
    verifyEqual(testCase,sqrVal,9);
end

function testSquareRoot(testCase)
    [sqrVal,sqrRootVal] = exampleFunction(100);
    verifyEqual(testCase,sqrRootVal,10);
end

Create a test suite and run the tests. The testSquare test fails because the implementation of exampleFunction is incorrect.

suite = testsuite('ExampleTest.m');
results = run(suite)

Running exampleTest

================================================================================
Verification failed in exampleTest/testSquare.

    ---------------------
    Framework Diagnostic:
    ---------------------
    verifyEqual failed.
    --> The values are not equal using "isequaln".
    --> Failure table:
            Actual    Expected    Error    RelativeError
            ______    ________    _____    _____________
        
              27         9         18            2      
    
    Actual Value:
        27
    Expected Value:
         9

    ------------------
    Stack Information:
    ------------------
    In C:\Work\exampleTest.m (testSquare) at 7
================================================================================
..
Done exampleTest
__________

Failure Summary:

     Name                    Failed  Incomplete  Reason(s)
    =====================================================================
     exampleTest/testSquare    X                 Failed by verification.
    

results = 

  1×2 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   1 Passed, 1 Failed (rerun), 0 Incomplete.
   0.24851 seconds testing time.

Update the code in exampleFunction to fix the coding error.

function [x,y] = exampleFunction(n)
    validateattributes(n,{'numeric'},{'scalar'})
    
    x = n^2;     % square
    y = sqrt(n); % square root
end

Click the (rerun) link in the command window to rerun the failed test. You cannot rerun failed tests if the variable that stores the test results is overwritten. If the link is no longer in the Command Window, you can type results at the prompt to view it.

Running exampleTest
.
Done exampleTest
__________


ans = 

  TestResult with properties:

          Name: 'exampleTest/testSquare'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 0.0034
       Details: [1×1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   0.0033903 seconds testing time.

MATLAB stores the TestResult array associated with tests that you rerun in the ans variable. results is a 1x2 array that contains all the tests in exampleTest.m, and ans is a 1x1 array that contains the rerun results from the one failed test.

whos

   Name         Size            Bytes  Class                         Attributes

  ans          1x1               664  matlab.unittest.TestResult              
  results      1x2              1344  matlab.unittest.TestResult              
  suite        1x2                96  matlab.unittest.Test

To programmatically rerun failed tests, use the Failed property on the TestResult object to create and run a filtered test suite.

failedTests = suite([results.Failed]);
result2 = run(failedTests);

Running exampleTest
.
Done exampleTest
__________

To ensure that all passing tests continue to pass, rerun the full test suite.

Rerun Failed Tests

Related Topics