Main Content

Include Results from External Sources in Verification Status

Requirements Toolbox™ allows you to include the verification status of results from external sources in the Requirements Editor. You can summarize requirements verification status, author your custom domain registration, and write custom logic to fetch the results. For more information, see Review Requirements Verification Status.

You can also include test results from:

  • Continuous integration (CI) servers such as Jenkins

  • Custom results updated manually or with test scripts

You can create custom document interfaces that interpret test results from the external environment into language specific to your development environment. For more information, see Define Custom Document Interface for Direct Linking to Requirements.

You can use built-in verification services to interpret result files for most common cases, such as JUnit and TAP (Test Anything Protocol), to include external test results in the requirements verification status.

When you include the verification status of external test results in your requirements:

  • The external results are listed in the Verified column of the Requirements Editor, along with results from other sources, such as Model Verification blocks and Simulink® Test™ test files.

  • Pass/fail indication is reflected in requirement links.

  • Result status is automatically aggregated across requirement hierarchies.

  • Result status automatically updates as requirements are added or removed.

How to Populate Verification Results from External Sources

Commonly, external test results are run and managed outside of the MATLAB® environment. Test results can be the product of:

  • Running test scripts or other programs that generate a result file

  • Running a MATLAB Unit Test test case with a custom TestRunner object, with or without a CI server

You can create links to the test results by either:

  • Linking directly to a result file. The external result artifact is used as the link destination and the requirements are used as the links source. To create the custom document interface, you need:

    1. The file location

    2. The file format (for example, JUnit or Excel®)

    For details, see Linking to a Result File.

  • Linking to a test script and providing code that fetches results based on that test location. The external test artifacts are used as the link destination and the requirements are used as the link source. Your custom logic in the GetResultFcn function should locate the result artifact that corresponds to the test artifact and fetch results from that result artifact. See Linking to a Test Script.

The following steps are used to create the links from external sources and populate verification statuses from them:

  1. Create and register a custom document interface: Create and register a custom document interface. For more information, see Define Custom Document Interface for Direct Linking to Requirements.

  2. Link from the requirement to the test result file or test script: Use the Outgoing Links Editor or slreq.createLink to link from the requirements to the results file.

    Note

    In order to avoid stale results, make sure that the timestamp of result file is latest than the timestamp of link creation. If the timestamp is not mentioned or set to NaT in the result returned by GetResult function, then the staleness detection gets skipped.

  3. Display the verification status: In the Requirements Editor, view the Verified column to view the verification status. For details, see Display Verification Status.

  4. Refresh the requirements view: After the tests run, refresh the verification status by clicking the Refresh button.

You can include the verification status from external sources in your requirements report by clicking Report > Generate Report from the Requirements Editor.

When populating verification results from external sources:

  • Test the GetResultFcn code before integrating the code with rmi register. For more information about GetResultFcn, see Define Custom Document Interface for Direct Linking to Requirements.

  • Confirm the custom link type registration in the Outgoing Links Editor.

  • Use caching to improve the performance for cases where a single file contains a result for many links.

  • Insert break points into the GetResultFcn code and use the Refresh button to re-execute it.

  • When using Projects, register and unregister the custom link type when using in project startup or shutdown scripts.

Related Examples

More About