Collect Coverage in Tests

Coverage refers to determining the testing completeness of models and generated code by analyzing how much of the model has been exercised. To collect coverage using the Simulink^® Test™ Test Manager or sltest.testmanager.CoverageSettings, you must have Simulink Coverage™ installed. Although you can set up and run test cases using only Simulink Coverage, Simulink Test provides additional test creation and test management features. For tests with coverage collection turned on, the Test Manager includes the coverage of each metric you choose to collect in the results. If you have Requirements Toolbox™ installed, you can also use the Test Manager to verify that coverage results are traced to specific requirements.

Note

Coverage is supported for Model Reference blocks, atomic Subsystem blocks, and top-level models configured for Software-in-the-Loop (SIL) or Processor-in-the-Loop (PIL). Coverage is not supported for SIL or PIL models in subsystems.

For information on considerations when collecting coverage in a test harness, see Test Harness Considerations in Test Harness and Model Relationship.

Set Up Coverage Collection Using the Test Manager

In the Test Manager, you can enable coverage and select the coverage metrics at the test file level. Test suites and test cases inherit the coverage settings from the test file. You can turn off coverage collection for individual test suites and test cases. However, you cannot turn off coverage at the test suite or test case level for MATLAB^®-based Simulink tests. For information about MATLAB-based Simulink tests, see Test Models Using MATLAB-Based Simulink Tests.

Note

The Test Manager uses its coverage settings and not the coverage settings from the model Configuration Parameters.

To set up the Test Manager to include coverage collection:

Create a test file and set up a test case for your model.
Select the test file and expand the Coverage Settings section. Under Coverage to Collect, select the coverage:
- Record coverage for system under test — Collects coverage for the model or, when included, the component specified in the System Under Test section for each test case. If you are using a test harness, the system under test is the component for which the harness is created. The test harness is not the system under test.
  - For a block diagram, the system under test is the whole block diagram.
  - For a Model block, the system under test is the referenced model.
  - For a subsystem, the system under test is that subsystem.
- Record coverage for referenced models — Collects coverage for models that are referenced from within the specified system under test. If the test harness references another model, the coverage results are included for that model, too.
The selected coverage settings propagate from the test file to the test suites and test cases in the test file.
Optionally, to add or remove existing coverage filter files, click Add or Remove, respectively, in the Coverage Filters section and select the filter file. More than one filter file can be applied at the same time.
Select the coverage metrics to collect. For information on metrics, see Types of Model Coverage (Simulink Coverage) and Model Objects That Receive Coverage (Simulink Coverage).
Run the test. Coverage is collected for the test suites and test cases in the test file.

To turn off coverage collection for a test suite or test case, select the test suite or test case and then, deselect the Coverage to Collect option. To turn off one or more types of coverage collection, deselect them from the Coverage Metrics section. For MATLAB-Based Simulink tests, you can change the coverage collection and coverage metrics only at the test file level.

View Coverage Results in the Test Manager

View Aggregated Coverage Results and Metrics

After you collect coverage, use the Results and Artifacts pane in the Test Manager to view the results. Coverage results aggregated for all test cases are reported in results sets. Filtered coverage results are shown only at the Results level, and not at the test file, test suite, or test case level. When analyzing filtered coverage results, viewing them at the Results level provides overall coverage information. For example, if coverage is less than 100% for a specific test case, the missing coverage might be included in a different test case.

Select a Results item in the pane and expand the Aggregated Coverage Results section. The coverage percentage is shown for each metric and the colors summarize the coverage results.

Dark blue — Satisfied coverage
Red — Unsatisfied coverage
Light blue — Justified coverage

Summary and aggregated coverage results for a Result Set in the Test Manager.

To aggregate results from different test files into a single result set, select the separate results in the Results and Artifacts list. Then, from the context menu, select Merge Coverage Results. A results set that contains the combined coverage results appears in the list.

Scoping Coverage for Requirements-Based Tests

For requirements-based design and testing, such as for compliance to DO-178B, enable Scope coverage results to linked requirements to check that your model design is executing the requirements and that the tests are verifying those requirements. Both Simulink Coverage and Requirements Toolbox licenses are required. This option is available only if the results set contains more than one simulation, such as multiple test cases or iterations.

When the Scope coverage results to linked requirements check box is selected, coverage results include only tests that are directly linked to requirements and are explicitly tested. Requirements linked to steps in a Test Sequence block are not included in the coverage results. The aggregated results update automatically without having to resimulate the model. If you have tests that touch a model component but are not directly linked to a requirement, your aggregated coverage results percentages might decrease when you enable scoping. To obtain 100% coverage to your requirements, you might need to update your tests, add requirements links, or justify or exclude some items from coverage.

Trace Coverage Results to the Model

To navigate from the test coverage results in the Test Manager to the model, click the model name in the Aggregated Coverage Results table.

The model opens, and its Coverage Report opens in the Coverage Details pane of the model window. In this sample model, the model elements are red because they have less than 100% coverage.

Coverage report with highlighted model elements.

Point to a model element to see a summary of its metrics and block execution.

Hover pop up showing decision at 50% and execution at 100% coverage

Click a model element to scroll to its detailed coverage results information in the Coverage Details pane.

Coverage details with model element highlights

Create a Coverage Report

To create a report of the coverage for a model, click the arrow in the Report column of the Aggregated Coverage Results table.

Add Tests for Missing Coverage

If you have a Simulink Design Verifier™ license, you can generate additional test cases to increase the coverage in your models.

In the Test Manager,

In the Test Manager, select the Results and Artifacts pane.
Select the Results item for which you want to collect more coverage.
In the right pane, in the Aggregated Coverage Results section, select the line with incomplete coverage in the table.
At the bottom of the Aggregated Coverage Results section, click Add Tests for Missing Coverage.
In the Add Test for Missing Coverage dialog box,
- Harness — Select whether to use the existing harness or create a new harness
- Source — Select the source of the inputs to the harness. If you use an existing harness, the Source field is read only.
- Test Case — Select whether to use the existing test case or create a new test case. If you create a new harness, the only option is to use a new test case.
- Test Type — Select the type of test to use for the new test case. This field is displayed if you select to create either a new harness or a new test case.
- Test File — Select whether to use the existing test file or create a new test file. This field is displayed if you select to create either a new harness or a new test case.
- Location — If you select to create a new test file, specify the path and name of the test file.
Click OK to generate test cases that add the missing coverage.
If you created a new test case or new harness, in the Test Browser pane, drag and drop the test case into the test suite that contains the original test case.
Rerun the test suite.

For a complete example of how to increase test coverage in the Test Manager, see Increase Test Coverage for a Model.

Alternatively, you can create and use tests to increase coverage programmatically by using sltest.testmanager.addTestsForMissingCoverage and sltest.testmanager.TestOptions.

Coverage Filtering Using the Test Manager

Coverage filter rules specify one or more model objects or lines of generated code to exclude from coverage collection or for which you want to justify the coverage results. A set of coverage filter rules is contained in a filter file, which can be applied to the model or code being tested. You can apply more than one filter file to a test and also, reuse filter files for different models. When you apply a new or updated filter, the aggregated coverage results, which are shown for a result set, update automatically. You do not have to resimulate your model. For more information, see Coverage Filtering (Simulink Coverage).

To view the filtered coverage results, select a result set (that is, a Results item) in the Results and Artifacts pane.

From the Test Manager, you can:

Add or remove an existing coverage filter file — In the Test Browser pane, select the test file and expand the Coverage Settings section. Click Add or Remove at the bottom of the Coverage Filters or Applied Coverage Filters table and select the coverage filter file to add or remove, respectively. More than one coverage filter file can be applied to the coverage results.
Edit or create a filter file, define a filter rule, and justify or exclude coverage — From a Coverage Report or the Coverage Details pane of a model, open the Simulink Coverage Filter Editor by clicking on a justify icon or a Justify or Exclude link. When the Filter Editor is open, the Test Manager is locked. When you close the Filter Editor, the Test Manager is enabled and the results and applied filters list are updated with your changes. For information on using the Filter Editor, see Creating and Using Coverage Filters (Simulink Coverage) and Create, Edit, and View Coverage Filter Rules (Simulink Coverage).
Append currently applied coverage filters to the test file — Click Update Test File.
View coverage results — Select a Results item in the Results and Artifacts pane and expand the Aggregated Coverage Results section.

For more information on coverage filters, rules, and files, see the Coverage Filtering topics in Analyze Coverage and View Results (Simulink Coverage).