Simulink Code Coverage Metrics
If you have a Simulink® Coverage™ license, you can run a SIL or PIL simulation that produces code coverage metrics for generated model code. The simulation performs several types of code coverage analysis.
Statement Coverage
Statement coverage measures the number of source code statements that execute when the code runs. Use this type of coverage to determine whether every statement in the program has been invoked at least once.
The percentage of statement coverage is represented by the following equation:
Statement coverage = (Number of executed statements / Total number of statements) *100
Statement Coverage Example
This code snippet contains five statements. To achieve 100% statement coverage, you need at least one test with positive x values, one test with negative x values, and one test with x values of zero.
if (x > 0) printf( "x is positive" ); else if (x < 0) printf( "x is negative" ); else printf( "x is 0" );
Condition Coverage
Condition coverage analyzes statements that include conditions in source code. Conditions are
C/C++ Boolean expressions that contain relation operators (<
,
>
, <=
, or >=
), equation
operators (!=
or ==
), or logical negation operators
(!
), but that do not contain logical operators
(&&
or ||
). This type of coverage
determines whether every condition has been evaluated to all possible outcomes at least
once. Conditions that are inside branching constructs, such as if
,
while
, and do-while
, report decision coverage
instead of condition coverage.
The percentage of condition coverage is represented by the following equation:
Condition coverage = (Number of executed condition outcomes / Total number of condition outcomes) *100
Condition Coverage Example
In this expression:
y = x<=5 || x!=7;
there are two conditions:
x<=5 x!=7
To achieve 100% condition coverage, your test cases need to demonstrate a true and
false outcome for both conditions. For example, a test case where x
is equal to 4
demonstrates a true case for both conditions, and a
case where x is equal to 7 demonstrates a false case for both
conditions.
Boolean Assignment Statements
Code coverage analyzes assignment statements that contain Boolean values and reports
them as a condition. Model coverage only analyzes logical expressions in assignment
statements, meaning only expressions that contain a logical operator such as the logical
AND
(&&
) or the logical
OR
(||
). This difference can result in a
discrepancy between model and code coverage results and can sometimes result in
unsatisfiable condition outcomes in the code coverage analysis.
For example, consider the following statement:
bool A = true;
During code coverage analysis, Simulink Coverage analyzes this statement for condition coverage. The statement is true at every time step, so the result is that you get 50% condition coverage on this statement because the false case cannot occur. Model coverage does not analyze this statement, so it also creates a discrepancy between model and code coverage results.
Decision Coverage
Decision coverage analyzes statements that represent decisions in source code. Decisions are
Boolean expressions composed of conditions and one or more of the logical C/C++ operators
&&
or ||
. Conditions within branching constructs
(if/else, while, and do-while) are decisions. Decision coverage determines the percentage of the
total number of decision outcomes the code exercises during execution. Use this type of coverage
to determine whether all decisions, including branches, in your code are tested.
Note
The decision coverage definition for DO-178C compliance differs from the Simulink
Coverage definition. For decision coverage compliance with DO-178C, in the Configuration
Parameters, set the Structural Coverage Level to Condition
Decision
for Boolean expressions not containing && or ||
operators.
The percentage of decision coverage is represented by the following equation:
Decision coverage = (Number of executed decision outcomes / Total number of decision outcomes) *100
Decision Coverage Example
This code snippet contains three decisions:
y = x<=5 && x!=7; // decision #1 if( x > 0 ) // decision #2 printf( "decision #2 is true" ); else if( x < 0 && y ) // decision #3 printf( "decision #3 is true" ); else printf( "decisions #2 and #3 are false" );
To achieve 100% decision coverage, your test cases must demonstrate a true and false outcome for each decision.
Modified Condition/Decision Coverage (MCDC)
Modified condition/decision coverage (MCDC) analyzes whether the conditions within decisions independently affect the decision outcome during execution. To achieve 100% MCDC, your test cases must demonstrate:
All conditions within decisions have been evaluated to all possible outcomes at least once.
Every condition within a decision independently affects the outcome of the decision.
The percentage of MCDC is represented by the following equation:
MCDC coverage = (Number of conditions evaluated to all possible outcomes affecting the outcome of the decision / Total number of conditions within the decisions) *100
Modified Condition/Decision Coverage Example
For this decision:
X || ( Y && Z )
the following set of test cases delivers 100% MCDC coverage.
X | Y | Z | |
---|---|---|---|
Test case #1 | 0 | 0 | 1 |
Test case #2 | 0 | 1 | 0 |
Test case #3 | 0 | 1 | 1 |
Test case #4 | 1 | 0 | 1 |
In order to demonstrate that the conditions Y
and
Z
can independently affect the decision outcome, the condition
X
must be false for those test cases. If the condition
X
is true, then the decision is already known to be true.
Therefore, the conditions Y
and Z
would not affect
the decision outcome.
Limitations
For code coverage, MCDC is limited to expressions that contain 1000 or less logical combinations. If an expression contains more than 1000 logical combinations, Simulink Coverage analyzes MCDC until reaching the 1000 logical combinations limit, then displays a warning that MCDC analysis is incomplete.
Cyclomatic Complexity
Cyclomatic complexity measures the structural complexity of code by using the McCabe complexity measure. To compute the cyclomatic complexity of code, code coverage uses this formula:
N is the number of decisions in the code. on is the number of outcomes for the nth decision point. Code coverage adds 1 to the complexity number for each C/C++ function.
Coverage Example
For this code snippet, the cyclomatic complexity is 3:
void evalNum(int x) { if (x > 0) // decision #1 printf( "x is positive" ); else if (x < 0) // decision #2 printf( "x is negative" ); else printf( "x is 0" ); }
The code contains one function that has two decision points. Each decision point has two outcomes. Using the preceding formula, N is 2, o1 is 2, and o2 is 2. Code coverage uses the formula with these decisions and outcomes and adds 1 for the function. The cyclomatic complexity for this code snippet is:
c = (o1 − 1) + (o2 − 1) + 1 = (2 − 1) + (2 − 1) + 1 = 3
Relational Boundary Coverage
Relational boundary code coverage examines code that has relational operations. Relational boundary code coverage metrics align with those for model coverage, as described in Relational Boundary Coverage (Simulink Coverage). Fixed-point and enum values in your model are integers during code coverage.