Create Advanced Parameterized Test

This example shows how to create a test that is parameterized in the TestClassSetup, TestMethodSetup, and Test methods blocks. The example test class tests random number generation.

Create TestRand Test Class

In a file in your current folder, create the TestRand class to test various aspects of random number generation. Define the properties used for parameterized testing.

classdef TestRand < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        generator = {'twister','combRecursive','multFibonacci'};
    end
    
    properties (MethodSetupParameter)
        seed = {0,123,4294967295};
    end
    
    properties (TestParameter)
        dim1 = struct('small',1,'medium',2,'large',3);
        dim2 = struct('small',2,'medium',3,'large',4);
        dim3 = struct('small',3,'medium',4,'large',5);
        type = {'single','double'};
    end
end

In the TestRand class, each properties block corresponds to parameterization at a particular level. The class setup-level parameterization defines the type of random number generator. The method setup-level parameterization defines the seed for the random number generator, and the test-level parameterization defines the data type and size of the random values.

Define Test Class and Test Method Setup Methods

Define the setup methods at the test class and test method levels. These methods register the initial random number generator state. After the framework runs the tests, the methods restore the original state. The classSetup method defines the type of random number generator, and the methodSetup method seeds the generator.

classdef TestRand < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        generator = {'twister','combRecursive','multFibonacci'};
    end
    
    properties (MethodSetupParameter)
        seed = {0,123,4294967295};
    end
    
    properties (TestParameter)
        dim1 = struct('small',1,'medium',2,'large',3);
        dim2 = struct('small',2,'medium',3,'large',4);
        dim3 = struct('small',3,'medium',4,'large',5);
        type = {'single','double'};
    end
    
    methods (TestClassSetup)
        function classSetup(testCase,generator)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(0,generator)
        end
    end
    
    methods (TestMethodSetup)
        function methodSetup(testCase,seed)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(seed)
        end
    end
end

Define Test Method with Sequential Parameter Combination

Define the testSize method in a methods block with the Test and ParameterCombination = 'sequential' attributes.

classdef TestRand < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        generator = {'twister','combRecursive','multFibonacci'};
    end
    
    properties (MethodSetupParameter)
        seed = {0,123,4294967295};
    end
    
    properties (TestParameter)
        dim1 = struct('small',1,'medium',2,'large',3);
        dim2 = struct('small',2,'medium',3,'large',4);
        dim3 = struct('small',3,'medium',4,'large',5);
        type = {'single','double'};
    end
    
    methods (TestClassSetup)
        function classSetup(testCase,generator)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(0,generator)
        end
    end
    
    methods (TestMethodSetup)
        function methodSetup(testCase,seed)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(seed)
        end
    end
    
    methods (Test, ParameterCombination = 'sequential')
        function testSize(testCase,dim1,dim2,dim3)
            testCase.verifySize(rand(dim1,dim2,dim3),[dim1 dim2 dim3])
        end
    end
end

The method tests the size of the output for each corresponding parameter value in dim1, dim2, and dim3. For a given TestClassSetup and TestMethodSetup parameterization, the framework calls the testSize method three times — one time for each of the 'small', 'medium', and 'large' values. For example, to test with all of the 'medium' values, the framework uses testCase.verifySize(rand(2,3,4),[2 3 4]).

Define Test Method with Pairwise Parameter Combination

Define the testRepeatable method in a methods block with the Test and ParameterCombination = 'pairwise' attributes.

classdef TestRand < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        generator = {'twister','combRecursive','multFibonacci'};
    end
    
    properties (MethodSetupParameter)
        seed = {0,123,4294967295};
    end
    
    properties (TestParameter)
        dim1 = struct('small',1,'medium',2,'large',3);
        dim2 = struct('small',2,'medium',3,'large',4);
        dim3 = struct('small',3,'medium',4,'large',5);
        type = {'single','double'};
    end
    
    methods (TestClassSetup)
        function classSetup(testCase,generator)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(0,generator)
        end
    end
    
    methods (TestMethodSetup)
        function methodSetup(testCase,seed)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(seed)
        end
    end
    
    methods (Test, ParameterCombination = 'sequential')
        function testSize(testCase,dim1,dim2,dim3)
            testCase.verifySize(rand(dim1,dim2,dim3),[dim1 dim2 dim3])
        end
    end
    
    methods (Test, ParameterCombination = 'pairwise')
        function testRepeatable(testCase,dim1,dim2,dim3)
            state = rng;
            firstRun = rand(dim1,dim2,dim3);
            rng(state)
            secondRun = rand(dim1,dim2,dim3);
            testCase.verifyEqual(firstRun,secondRun)
        end
    end
end

The method verifies that the random number generator results are repeatable. For a given TestClassSetup and TestMethodSetup parameterization, the framework calls the testRepeatable method 10 times to ensure testing with each pair of parameter values specified by dim1, dim2, and dim3. If the parameter combination were exhaustive, the framework would call the method 3³ = 27 times.

Define Test Method with Exhaustive Parameter Combination

Define the testClass method in a methods block with the Test attribute. Because the ParameterCombination attribute is not specified, the parameter combination is exhaustive by default.

classdef TestRand < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        generator = {'twister','combRecursive','multFibonacci'};
    end
    
    properties (MethodSetupParameter)
        seed = {0,123,4294967295};
    end
    
    properties (TestParameter)
        dim1 = struct('small',1,'medium',2,'large',3);
        dim2 = struct('small',2,'medium',3,'large',4);
        dim3 = struct('small',3,'medium',4,'large',5);
        type = {'single','double'};
    end
    
    methods (TestClassSetup)
        function classSetup(testCase,generator)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(0,generator)
        end
    end
    
    methods (TestMethodSetup)
        function methodSetup(testCase,seed)
            orig = rng;
            testCase.addTeardown(@rng,orig)
            rng(seed)
        end
    end
    
    methods (Test, ParameterCombination = 'sequential')
        function testSize(testCase,dim1,dim2,dim3)
            testCase.verifySize(rand(dim1,dim2,dim3),[dim1 dim2 dim3])
        end
    end
    
    methods (Test, ParameterCombination = 'pairwise')
        function testRepeatable(testCase,dim1,dim2,dim3)
            state = rng;
            firstRun = rand(dim1,dim2,dim3);
            rng(state)
            secondRun = rand(dim1,dim2,dim3);
            testCase.verifyEqual(firstRun,secondRun)
        end
    end
    
    methods (Test)
        function testClass(testCase,dim1,dim2,type)
            testCase.verifyClass(rand(dim1,dim2,type),type)
        end
    end
end

The method verifies that the class of the output from rand is the same as the expected class. For a given TestClassSetup and TestMethodSetup parameterization, the framework calls the testClass method 3×3×2 = 18 times to ensure testing with each combination of dim1, dim2, and type parameter values.

Create Suite from Test Class

At the command prompt, create a suite from the TestRand class.

suite = matlab.unittest.TestSuite.fromClass(?TestRand)

suite = 

  1×279 Test array with properties:

    Name
    ProcedureName
    TestClass
    BaseFolder
    Parameterization
    SharedTestFixtures
    Tags

Tests Include:
   17 Unique Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

For a given TestClassSetup and TestMethodSetup parameterization, the framework creates 3+10+18 = 31 test elements. These 31 elements are called for each TestMethodSetup parameterization (resulting in 3×31 = 93 test elements for each TestClassSetup parameterization). There are three TestClassSetup parameterizations; therefore, the suite has a total of 3×93 = 279 test elements.

Query the name of the first test element.

suite(1).Name

ans =

    'TestRand[generator=twister]/[seed=0]testClass(dim1=small,dim2=small,type=single)'

The name of the first element is composed of these parts:

Test class — TestRand
Class setup property and parameter name — [generator=twister]
Method setup property and parameter name — [seed=0]
Test method name — testClass
Test method property and parameter names — (dim1=small,dim2=small,type=single)

Run Suite Created Using Selector

At the command prompt, create a selector to select test elements that test the 'twister' generator for 'single' precision. Create a suite by omitting test elements that use properties with the 'large' parameter name.

import matlab.unittest.selectors.HasParameter
s = HasParameter('Property','generator','Name','twister') & ...
    HasParameter('Property','type','Name','single') & ...
    ~HasParameter('Name','large');

suite2 = matlab.unittest.TestSuite.fromClass(?TestRand,s)

suite2 = 

  1×12 Test array with properties:

    Name
    ProcedureName
    TestClass
    BaseFolder
    Parameterization
    SharedTestFixtures
    Tags

Tests Include:
   9 Unique Parameterizations, 0 Shared Test Fixture Classes, 0 Tags.

If you first generate the full suite from the TestRand class, you can construct the same filtered suite using the selectIf method.

suite = matlab.unittest.TestSuite.fromClass(?TestRand);
suite2 = selectIf(suite,s);

Run the filtered test suite.

suite2.run;

Running TestRand
.......... ..
Done TestRand
__________

Run Suite from Method Using Selector

Create a selector that omits test elements that use properties with the 'large' or 'medium' parameter names. Limit results to test elements from the testRepeatable method.

import matlab.unittest.selectors.HasParameter
s = ~(HasParameter('Name','large') | HasParameter('Name','medium'));

suite3 = matlab.unittest.TestSuite.fromMethod(?TestRand,'testRepeatable',s);
{suite3.Name}'

ans =

  9×1 cell array

    {'TestRand[generator=twister]/[seed=0]testRepeatable(dim1=small,dim2=small,dim3=small)'               }
    {'TestRand[generator=twister]/[seed=123]testRepeatable(dim1=small,dim2=small,dim3=small)'             }
    {'TestRand[generator=twister]/[seed=4294967295]testRepeatable(dim1=small,dim2=small,dim3=small)'      }
    {'TestRand[generator=combRecursive]/[seed=0]testRepeatable(dim1=small,dim2=small,dim3=small)'         }
    {'TestRand[generator=combRecursive]/[seed=123]testRepeatable(dim1=small,dim2=small,dim3=small)'       }
    {'TestRand[generator=combRecursive]/[seed=4294967295]testRepeatable(dim1=small,dim2=small,dim3=small)'}
    {'TestRand[generator=multFibonacci]/[seed=0]testRepeatable(dim1=small,dim2=small,dim3=small)'         }
    {'TestRand[generator=multFibonacci]/[seed=123]testRepeatable(dim1=small,dim2=small,dim3=small)'       }
    {'TestRand[generator=multFibonacci]/[seed=4294967295]testRepeatable(dim1=small,dim2=small,dim3=small)'}

Run the test suite.

suite3.run;

Running TestRand
.........
Done TestRand
__________

Run All Double Precision Tests

At the command prompt, run all of the test elements from the TestRand class that use the 'double' parameter name.

runtests('TestRand','ParameterName','double');

Running TestRand
.......... .......... .......... .......... ..........
.......... .......... .......... .
Done TestRand
__________