Relational Boundary Coverage

Percentage of relational operations that are tested by current test cases

Since R2023b

expand all in page

Description

Relational boundary is a metric that records whether a test examines a relational operation with equal values and values that differ slightly. Consider an arbitrary relational operation operator(left,right). Depending on the data type of left and right, this operation has two or three boundaries.

IntegerFloat

  • left-right == 0

  • left-right == -1

  • left-right == +1

  • left-right ∈ [0..tolerance]

  • left-right ∈ [-tolerance..0]

The inclusivity of the limits 0..tolerance and -tolerance..0 depends on the relational operation.

To obtain 100% relational boundary coverage, add test cases that can examine all the appropriate boundaries. For instance, consider this code:

void foo(int a, int b){
    //...
    if(a>b){
        //...
    }
}
To cover all relational boundaries of a>b, test the function foo with these values of a and b:

  • a == b

  • a == b+1

  • a == b-1

Note that in the Polyspace Platform user interface, you do not see relational boundary coverage in a separate column on the Results List pane. If you select a decision containing a relational operator (==, <=, and so on), the Result Details pane shows this metric. To see more details on how Polyspace® Test™ calculates this metric, generate an HTML report from your coverage results. See Structure of HTML Reports Generated from C/C++ Code Profiling Results.

Polyspace Implementation

To calculate the relational boundary coverage, Polyspace calculates the total number of boundaries (B_total) in your code and the number of boundaries that are tested (B_tested) by current test cases:

Boundary Coverage = 100*B_tested/B_total
For each integer relational operation, Polyspace adds 3 to B_total. For each floating-point relational operation, Polyspace adds 2 to B_total.

For floating-point boundaries, use either a predefined tolerance or specify your own tolerance. The tolerance value depends on the data type of both the operands. If both operands have the same data type, the tolerance follows these rules.

Data Type of OperandTolerance
Floating-point

max(absTol, relTol × max(|lhs|,|rhs|))

  • absTol is an absolute tolerance value you specify. The default value is 1e-05.

  • relTol is a relative tolerance value you specify. The default value is 0.01.

  • lhs is the left operand and rhs is the right operand.

  • max(x,y) returns x or y, whichever is greater.

Integer1
BooleanN/A

If the two operands have different types, the tolerance follows the rules for the stricter type. The float type is the strictest type, followed by int and bool.

Polyspace Test checks for different relational boundaries when analyzing different floating-point relational operations. See this table when checking a relational operation between two floating-point variables left and right.

Relational OperationChecked BoundariesComment
left == rightleft-right ∈ [-tolerance..0)left - right == 0 is excluded.
left-right ∈ (0..tolerance]
left != rightleft-right ∈ [-tolerance..0)left - right == 0 is excluded.
left-right ∈ (0..tolerance]
left <= rightleft-right ∈ [-tolerance..0]left - right == 0 is included in the region below the relational boundary.
left-right ∈ (0..tolerance]
left < rightleft-right ∈ [-tolerance..0)left - right == 0 is included in the region above the relational boundary.
left-right ∈ [0..tolerance]
left >= rightleft-right ∈ [-tolerance..0)left - right == 0 is included in the region above the relational boundary.
left-right ∈ [0..tolerance]
left > rightleft-right ∈ [-tolerance..0]left - right == 0 is included in the region below the relational boundary.
left-right ∈ (0..tol]

Polyspace includes the boundary left - right == 0 below the relational boundary for <=, but above the relational boundary for <. This rule is consistent with decision coverage. For instance:

  • For the relation left <= right, the decision is true if left is less than or equal to right. Polyspace groups < and = together. Therefore, left - right == 0 lies in the region below the relational boundary.

  • For the relation left < right, the decision is true only if left is less than right. Polyspace groups > and = together. Therefore, left - right == 0 lies in the region above the relational boundary.

Examples

expand all

Consider this code:

int foo(int x, int y)
{
    

    if (x == y)
        return 1; 
    else if (x < 2)
        return 2;
	else return -1;
   
}

This function contains two relational operations, and six boundaries.

Boundaries of x==yBoundaries of x<2
x-y == 0x-2 == 0
x-y == -1x-2 == -1
x-y == +1x-2 == +1

Consider these test cases.

Test Case 1Test Case 2Test Case 3

  • Input — Parameter value = (1,1)

  • Assessment — Return value = 1.0

  • Coverage — (1/6)×100 or 17%

  • Coverage Details — This case tests the boundary x-y == 0.

  • Input — Parameter value =(0,1)

  • Assessment — Return value = 2

  • Coverage — (1/6)×100 or 17%

  • Coverage Details — This case tests the boundary x-y == 1.

  • Input — Parameter value = (1,0)

  • Assessment — Return value = 3

  • Coverage — (2/2)×100 or 33%

  • Coverage Details — This case tests the boundary x-y == -1 and x-2 == -1.

The three tests cover four out of the six relational boundaries, resulting in 67% relational boundary coverage. To obtain 100% coverage for this metric, test these boundaries with additional test cases:

  • x - 2 == 0

  • x - 2 == 1

Correction — Add New Test Case to Obtain 100% Coverage

To obtain complete coverage, add two new test cases.

Test Case 1Test Case 2

  • Input — Parameter value = (2,-1)

  • Assessment — Return value = -1.0

  • Coverage — (1/6)×100 or 17%

  • Coverage Details — This case tests the boundary x-2 == 0.

  • Input — Parameter value =(3,-1)

  • Assessment — Return value = -1

  • Coverage — (1/6)×100 or 17%

  • Coverage Details — This case tests the boundary x-2 == 1.

After adding these test cases, the relational boundary coverage increases to 100%.

Consider this code:

double foo(double x, double y)
{
    

    if (x > y)
        return 1; 
    else if (x < 2.3)
        return 2;
	else return -1;
   
}

This function contains two relational operations and four boundaries.

Boundaries of x>yBoundaries of x<2.3
x-y ∈ (0..tolerance]x-2.3 ∈ [0..tolerance]
x-y ∈ [-tolerance..0]x-2.3 ∈ [-tolerance..0)

Here, tolerance = max(1e-05, 0.01× max(|lhs|,|rhs|)). When choosing test cases, use values of x and y so that the result of the relational operation falls within the boundary. For instance, to cover the boundaries of (x < 2.3), use values of x, such as 2.31 and 2.29. To cover the boundaries of (x > y), use values of (x,y), such as (2.29,2.31) and (2.31,2.29). If you use values that are outside the boundary tolerance, the test cases cannot cover the relational boundary.

Consider these cases.

Test Case 1Test Case 2

  • Input — Parameter value = (2.29,2.31)

  • Assessment — Return value = 2.0

  • Coverage — (2/4)×100 or 50%

  • Coverage Details — This case tests the boundary x-y ∈ [-tolerance..0] and x-2.3 ∈ [-tolerance..0).

  • Input — Parameter value =(2.31,2.29)

  • Assessment — Return value = 1.0

  • Coverage — (1/4)×100 or 25%

  • Coverage Details — This case tests the boundary x-y ∈ [-tolerance..0].

In combination, these two test cases cover 75% of the relational boundaries.

Correction — Add New Test Case to Obtain 100% Coverage

To obtain complete coverage, add a new test case that tests the boundary x-2.3 ∈ [0..tolerance]. Choose x = 2.31 to test the boundary. To avoid the return after the x > y statement, pick a value of y that is greater than x, such as 2.5:

  • Input — Parameter value = (2.31,2.5)

  • Assessment — Return value = -1.0

  • Coverage — (1/4)×100 or 25%

  • Coverage Details — This case tests the boundary x-2.3 ∈ [0..tolerance].

After you add this test case, the relational boundary coverage increases to 100%.

Version History

Introduced in R2023b

See Also

| |

Topics