Types of Code Coverage for MATLAB Source Code
When you run tests, you can collect and access code coverage
information for your MATLAB® source code by adding an instance of the CodeCoveragePlugin
class to the test runner. As the tests run, the plugin
collects information that shows the parts of the source code that were executed by the
tests. You can access this information either programmatically or as a code coverage
report.
If you create a
CodeCoveragePlugin
instance using a CoverageResult
, CoverageReport
, or StandaloneReport
format object, the plugin
supports the following coverage types, which let you perform a detailed analysis of the
source code covered by your tests.
Statement Coverage
Statement coverage identifies the source code statements that execute when the tests run. Use this type of coverage to determine whether every statement in your source code is executed at least once.
To report statement coverage, MATLAB divides the source code into statements that are separated by a comma, semicolon, or newline character. For example, this code has three statements.
b = 1, a = 2 * (b + 10); x = (b ~= 0) && (a/b > 18.5)
MATLAB divides control flow statements into smaller units for code coverage
reporting. For example, in the following control flow statement code, MATLAB reports coverage for these five units: if x > 0
,
elseif x < 0
, and three calls to the disp
function. To achieve 100% statement coverage, you need tests that execute each of these
units.
if x > 0 disp("x is positive.") elseif x < 0 disp("x is negative.") else disp("x is either zero or NaN.") end
A keyword followed by an expression forms a unit for which MATLAB reports statement coverage. Among keywords that do not require an
expression, MATLAB reports coverage for break
,
catch
, continue
,
return
, and try
. It
ignores keywords such as else
,
end
, and
otherwise
.
In general, MATLAB reports coverage for statements that perform some action on program data
or affect the flow of the program. It ignores code that defines functions, classes, or
class members, such as function [y1,...,yN] = myfun(x1,...,xM)
or
classdef MyClass
.
Function Coverage
Function coverage identifies the functions defined in the source code that execute when the tests run. Use this type of coverage to determine whether every function in your source code was called at least once.
For example, this code contains three defined functions: f
,
root
, and square
. To achieve 100% function
coverage, you need tests that result in each of these functions being called.
function f(x) if x >= 0 root else square end disp(x) function root x = sqrt(x); end function square x = x.^2; end end
Decision Coverage
Decision coverage identifies the decision outcomes achieved by the tests. Use this type of coverage to determine whether decisions in your source code, including branching and control flow statements, have been tested.
Typically, a decision is an expression that affects the flow of the program with the
MATLAB keywords if
,
elseif
, switch
,
for
, or while
.
Additionally, if modified condition/decision coverage (MC/DC) reporting is enabled, any
assignment statement that contains the short-circuiting logical operators
&&
or ||
is considered a
decision.
For example, this code includes three decisions if MC/DC reporting is enabled, and two
decisions otherwise. To achieve 100% decision coverage, you need tests that evaluate
each included decision to both true
and
false
.
function y = f(x) z = (x > -3) || (x == -5); % Decision #1 (if MC/DC reporting is enabled) if x > 0 % Decision #2 y = 1; elseif (x < 0) && z % Decision #3 y = -1; else y = 0; end
Note
When you create a plugin with the MetricLevel
name-value
argument specified as "decision"
or
"condition"
, the plugin ignores the decisions in the source
code that do not follow a branching or control flow keyword. The plugin analyzes
such decisions only when you specify MetricLevel
as
"mcdc"
.
Condition Coverage
Condition coverage identifies the condition outcomes achieved by the tests. Use this
type of coverage to determine whether every condition in your source code has been
evaluated to both true
and false
.
Conditions are logical expressions that do not contain short-circuiting operators, but
that are part of an assignment statement formed by such operators. For example, this
code has two conditions. To achieve 100% condition coverage, you need tests that
evaluate each condition to both true
and
false
.
y = (x > -3) || (x == -5); % An assignment statement with two conditions z = x ~= 0; % Not a condition if x == 0 % A decision but not a condition disp("x is zero.") else disp("x is nonzero.") end
Modified Condition/Decision Coverage (MC/DC)
Modified condition/decision coverage (MC/DC) identifies how tests independently exercise conditions within decisions. To achieve 100% MC/DC, you need tests that satisfy both of these statements:
All conditions within decisions are evaluated to both
true
andfalse
.Every condition within a decision independently affects the outcome of the decision.
Instead of testing for all possible combinations of condition outcomes, satisfying
these statements requires fewer tests. For example, consider the decision
result = x || (y && z)
, which contains three conditions
x
, y
, and z
. This table
shows how different condition outcomes affect the decision outcome.
Combination | x | y | z | result |
---|---|---|---|---|
1 | false | false | false | false |
2 | false | false | true | false |
3 | false | true | false | false |
4 | false | true | true | true |
5 | true | false | false | true |
6 | true | false | true | true |
7 | true | true | false | true |
8 | true | true | true | true |
To achieve 100% MC/DC for result = x || (y && z)
, testing
for combinations 2, 3, 4, and 6 is sufficient because:
Combinations 2 and 6 show that
x
can take both possible values and also affect the decision outcome independently.Combinations 2 and 4 show that
y
can take both possible values and also affect the decision outcome independently.Combinations 3 and 4 show that
z
can take both possible values and also affect the decision outcome independently.
How to Use Coverage Types
To perform a code coverage analysis using the supported coverage types, specify the
MetricLevel
name-value argument when you create a plugin using
one of the static methods of the CodeCoveragePlugin
class. For example, run
your tests and generate an interactive HTML code coverage report that includes all
supported metrics for the source code in a folder.
import matlab.unittest.plugins.CodeCoveragePlugin import matlab.unittest.plugins.codecoverage.CoverageReport suite = testsuite("MyTestClass"); runner = testrunner("textoutput"); format = CoverageReport; plugin = CodeCoveragePlugin.forFolder("myFolder", ... Producing=format,MetricLevel="mcdc"); runner.addPlugin(plugin) results = runner.run(suite);
The MetricLevel
argument specifies which coverage types to
include in the analysis. This table shows the possible values of
MetricLevel
and the corresponding coverage types.
Value of MetricLevel | Types of Coverage Included |
---|---|
"statement" (default) | Statement and function coverage |
"decision" | Statement, function, and decision coverage |
"condition" | Statement, function, decision, and condition coverage |
"mcdc" | Statement, function, decision, condition, and modified condition/decision coverage |
See Also
Classes
matlab.unittest.plugins.CodeCoveragePlugin
|matlab.coverage.Result
|matlab.unittest.plugins.codecoverage.CoverageResult
|matlab.unittest.plugins.codecoverage.CoverageReport
|matlabtest.plugins.codecoverage.StandaloneReport
|matlab.unittest.plugins.codecoverage.CoberturaFormat