Execute C/C++ Tests Written Using Polyspace Test xUnit API

You can write C/C++ unit tests using the Polyspace® Test™ xUnit API and build them using your own compilation toolchain. When you build these tests, you create a test runner executable that can be used to run the tests later. The test runner executable prints test results (test pass/fail information) on the console. You can optionally pipe the console output to a file for further processing.

This topic shows the various options available for test execution. To see the list of options available for test execution, at the command line, enter the following:

testRunner -h
Where testRunner is the test runner executable, for example, the executable testrunner.exe created in Write C/C++ Unit Tests Using Polyspace Test xUnit API and Run Tests at Command Line. This topic goes over some of the options that you see in the list.

Prerequisites

This topic describes test authoring using the Polyspace Test xUnit API. To compile these tests, you are required to know some file paths in advance. For your convenience, you can define environment variables to stand for the file paths, or otherwise include the file paths in your build. For more information, see Set Up C/C++ Testing and Code Profiling Using Self-Managed Builds.

Execute Specific Tests

If you run the test executable testRunner without additional options, all tests are executed.

  • To run a test that is not part of a test suite, add the option -test followed by the test name. For example, to run the test myTest, enter:

    testRunner -test myTest
    Enter multiple test names separated by comma-s. For example, to run the tests myTest1 and myTest2, enter:
    testRunner -test myTest1,myTest2

  • To run tests in a specific suite, add the option -test followed by the suite name. For example, to run all tests in the suite mySuite, enter:

    testRunner -test mySuite
    To run a specific test in a specific suite, add the option -test followed by the suite and test name, separated by a /. For example, to run a test myTest in the suite mySuite, enter:
    testRunner -test mySuite/myTest
    For more information on test suites, see Group C/C++ Tests into Suites with Common Setup and Teardown Code.

  • To run a parameterized test with specific values of the test parameters, add the option -test followed by the comma-separated parameter value numbers in parenthesis.

    For example, suppose a test myTest in the mySuite has two parameters:

    • param1 takes its values from the array data1.

    • param2 takes its values from the array data2.

    To run this test with param1 taking the first value of data1 and param2 taking the first value of data2, enter:

    testRunner -test mySuite/myTest(1,1)

    For more information on parameterized tests, see:

Check for Test Execution Success

You can check the overall status of test execution by checking the exit code of the test runner executable. The exit code is 0 if all tests pass and 1 if at least one test fails or there is an error in test execution.

You can check the exit code in different ways on various operating systems. For instance,

  • On Windows®, you can check the exit code of the last executed command as follows:

    echo %ERRORLEVEL%

  • On Linux®, you can check the exit code of the last executed command as follows:

    echo $?

Define Test Results Format

By default, the test runner executable displays results in a human-readable tabular format and shows assessment details only for failing tests. You can change the result format using the option -format followed by a comma-separated list of arguments.

For example, to display results in the Test Anything Protocol (TAP) format and log assessments for passing tests, enter:

testRunner -format tap,passed

For more information, see Results Format of Tests Written Using Polyspace Test API.

You can also generate test results in the .pstestr format and open the results in the Polyspace Platform user interface, or generate an HTML report from the test results. To generate results in this format, you have to follow these steps:

  1. Generate results in an intermediate machine-readable (.mrf) format using the test runner executable. You can either append the flag -format mrf to the test runner executable or define the configuration macro PST_OUTPUT as follows:

    #define PST_OUTPUT PST_OUTPUT_MRF

  2. Generate a .pstestr file from this intermediate format using the command polyspace-test -convert. You can open this file in the Polyspace Platform user interface. See also Open Polyspace Results in Polyspace Platform User Interface.

    When generating the .pstestr file, you can add the option -compilation-working-dir compileFolder. This option specifies the folder from which the compiler command was executed when creating the test runner. The compiler folder specification allows the polyspace-test command to find the source files starting from Polyspace Test results.

  3. Generate an HTML report from this .pstestr file using the command polyspace-test -report.

For example, the following commands generate a file resultsFile.pstestr in the folder resultsFolder and then generate an HTML report from the results:

testRunner -format mrf > resultsFile.mrf
polyspace-test -convert -results-dir resultsFolder resultsFile.mrf
polyspace-test -report -report-dir reportFolder -html resultsFolder/resultsFile.pstestr
After report generation, navigate to the file index.html in a subfolder of reportFolder. This file shows a breakdown of test results by test suite. You can click on links in this file to drill down to individual tests in the suite.

See Also

|

Topics