Calculate C/C++ Code Profiling Metrics by Using xUnit API-Based Tests on Target

Using Polyspace® Test™, you can calculate C/C++ code profiling metrics, such as code coverage, execution time, and memory use:

  • The metric Code Coverage measures how much of your code is covered by the existing test cases. Measure the value of different code coverage metrics by using Polyspace Test and add appropriate test cases to achieve your desired level of coverage. Polyspace Test calculates code coverage for standard metrics. See Review Code Profiling Results.

  • The metric Execution Time computes the time required to execute the different callable entities of your code. See Execution Time.

  • The metric Memory Use measures how the callable entities use the available stack memory. See Memory Use.

To calculate these metrics on a target, you can follow one of these two possible workflows:

  • Using a Polyspace Platform project — If you have a Polyspace Platform project (.psprjx) containing your source files and tests, you can perform these calculations on your target hardware by registering a target in the project. To register a target, use a registration file, which is a MATLAB® file that sets up the connection between your target and Polyspace Test. You can execute this workflow either in the Polyspace Platform user interface or by using the Polyspace Test command-line interface.

  • Without using a Polyspace Platform project — If you have xUnit API-based tests that you link with your source files using your own toolchain, then you can perform these calculations on your target hardware without requiring a Polyspace Platform project. In this workflow, you cross-compile your source and tests for the hardware board using your own cross-compiler.

This example shows how to calculate the code coverage and execution profile of your source by cross-compiling your source and xUnit tests for your target. To do these calculations using a Polyspace Platform project, see;

Prerequisite

To follow this topic, you must have a toolchain correctly configured to run C/C++ code on your specific target. You leverage this preexisting toolchain to generate test executables and to run these executables on your target.

Code Profiling Modes

Code profiling on a target involves instrumenting your source code, building the instrumented source code along with tests using your toolchain, and running the resulting executable on the target. When you instrument source code, you essentially inject additional code around the instrumentation points that is used later to collect code profiling data. This additional code uses macros that are defined in files available with a Polyspace Test installation and precompiled into static libraries for specific compilers.

Polyspace Test supports two modes for code profiling:

  • Monitor mode – In this mode, the coverage data is streamed from the target to the host during test execution and converted to a format suitable for viewing in the Polyspace Platform user interface or generating a report.

  • Manual conversion mode – Instead of monitoring test execution, you can also write the coverage data to a binary file that you convert later.

If you can stream data from your target through a serial or TCP/IP port, you can use either the monitor mode or manual conversion mode for code profiling. The monitor mode is recommended, as it automatically manages receiving of data from the target and handles the subsequent data conversion. If your port is not supported for the monitor mode, use the manual conversion mode. For more information on the ports supported and the port-related information used by monitor mode, see polyspace-test -monitor.

Monitor Mode

Define configuration files with data streaming functions, instrument your source files for code coverage computation, compile the instrumented source files together with test and configuration files, and finally run the test executable on your target while monitoring the test execution.

Configure Code Profiling

Prior to compilation, define a configuration file that specifies that the tests will be executed on a target and specifies streaming functions to stream coverage data from the target to the host:

  1. In a header file pstunit_config.h, set appropriate configuration macros to modify the default test execution. In particular, you must set these macros:

    #ifndef PSTUNIT_CONFIG_H
#define PSTUNIT_CONFIG_H

/* Enable monitoring of test execution */
#define PST_MONITOR_MODE_WITH_PROFILING 1
/* Disable execution time calculation, otherwise requires timer function definition */
#define PST_ENABLE_EXECUTION_TIME   0

extern void pst_write(const char * str, unsigned long len); 

/* Transfer string of specific length */
#define PST_WRITE(str, len) pst_write(str, len)

    For more information on the configuration macros, see Configuration Macros in Polyspace Test API for C/C++ Code.

  2. In a header file psprofile_config.h, include the previous header pstunit_config.h:

    #ifndef PSPROFILE_CONFIG_H
#define PSPROFILE_CONFIG_H

#include <pstunit_config.h>

#endif

  3. Define the implementation of the function mapped to the macro PST_WRITE in a source file pstunit_config.c.

    For example, this source file defines the function pst_write() to send a string of characters through the UART interface using the function HAL_UART_Transmit() from the STM32 HAL library.

    #include "pstunit_config.h"
#include "stm32746g_discovery.h"
extern UART_HandleTypeDef huart1;

void pst_write(const char * str, unsigned long len) {
      HAL_UART_Transmit(&huart1, (const uint8_t*)str, len, HAL_MAX_DELAY);
}

Create Executable from Instrumented Source Code

On the host computer, cross-compile the following files for your target:

  • The source files

  • The test files

  • The configuration file pstunit_config.c

  • The files pstunit.c and psprofile.c available with a Polyspace Test installation (see file location below).

In addition:

For example, if your source file is named src.c and the test file is tests.c, you can cross-compile using the GNU® toolchain for an Arm target (arm-none-eabi-gcc executable) as follows:

  1. Instrument the source files, and then compile the instrumented sources along with other files in a single command:

    polyspace-code-profiler -instrument -instrum-dir <instrumFolder> -limit-instrumentation-to <srcFolder> -cov-metric-level mcdc --  arm-none-eabi-gcc -c src.c tests.c pstunit_config.c <pstunit_source> <psprofile_source> -I <pstunit_include> -I <psprofile_config_dir> -I <psprofile_include> -D PST_USER_CONFIG_FILE

    Here:

    • <srcFolder> is the folder containing the source file src.c and <instrumFolder> is the folder that contains the results of instrumentation.

    • <pstunit_source> is the file <polyspaceroot>\polyspace\pstest\pstunit\src\pstunit.c, where <polyspaceroot> is the Polyspace installation folder, for example, C:\Program Files\Polyspace\R2026a.

    • <psprofile_source> is the file <polyspaceroot>\polyspace\psprofile\src\psprofile.c.

    • <pstunit_include> is the folder <polyspaceroot>\polyspace\pstest\pstunit\include.

    • <psprofile_include> is the folder <polyspaceroot>\polyspace\psprofile\include.

    • <psprofile_config_dir> is the folder containing the header files pstunit_config.h and psprofile_config.h that you created in an earlier step.

  2. Link the object files created in the previous step.

    arm-none-eabi-gcc src.o tests.o pstunit_config.c pstunit.o psprofile.o

    This step produces an executable that you can launch on the target.

Collect Profiling Data on Target

After creating the test executable on the host, run the executable on the target. You can either stream data to the host during execution (monitor mode) or to gather all coverage data on the target and transfer it later for manual conversion.

Run the command polyspace-test -monitor on your host to monitor the execution of tests, retrieve the code profiling results, and convert them to a format appropriate for review.

For example, suppose that you are running tests in Windows® on an STM32 board using the STM32_Programmer_CLI.exe command and suppose your executable name is unittests.hex:

"C:\Program Files\STMicroelectronics\STM32Cube\STM32CubeProgrammer\bin\STM32_Programmer_CLI.exe" --connect port=swd --erase all --download unittests.hex --go
You can monitor the execution of this test launch command by prepending the polyspace-test -monitor command to your test launch command as follows:
polyspace-test -monitor -com serial -read-timeout 15 -com-timeout 30 -port COM6 -baud-rate 115200 -parity none -results-dir resultsFolder -- "C:\Program Files\STMicroelectronics\STM32Cube\STM32CubeProgrammer\bin\STM32_Programmer_CLI.exe" --connect port=swd --erase all --download unittests.hex --go

This command monitors the test execution and retrieves code profiling results in binary form from the target. The host-target communication happens over a serial port COM6 with a read timeout of 15 seconds and a communication timeout of 30 seconds. For more information on the command, see polyspace-test -monitor.

After retrieving the code profiling results, the polyspace-test -monitor command attempts to convert the results into a .psprof format and stores the results in the folder specified for the option -results-dir.

Create Report from Collected Data

Generate a report from the *.psprof file by using the polyspace-code-profiler command. For example, at the command line, enter:

polyspace-code-profiler -report -html -report-dir <reportFolder> <conFolder>
Here:

  • <reportFolder> is the folder containing the output report.

  • <conFolder> is the folder containing the *.psprof file you produced in the previous step.

After you generate the readable report, review your code coverage or execution profile metrics in the report. See Structure of HTML Reports Generated from C/C++ Code Profiling Results. Review the report to:

  • Identify which code coverage metrics have an unacceptably low value.

    You can add tests to increase code coverage or justify missing coverage. See Improve or Justify Missing Code Coverage Results.

  • Identify the bottlenecks and memory inefficiency in your code.

Manual Conversion Mode

Define configuration files with data streaming functions, instrument your source files for code coverage computation, compile the instrumented source files together with test and configuration files, and run the test executable on your target. Gather the coverage data and convert from binary format to a format suitable for viewing in the Polyspace Platform user interface or generating reports.

Configure Code Profiling

Prior to compilation, define a configuration file that specifies that the tests will be executed on a target and specifies streaming functions to stream coverage data from the target to the host:

  • If your target contains a filesystem, define a header file psprofile_config.h as follows:

    #ifndef PSPROFILE_CONFIG_H
#define PSPROFILE_CONFIG_H

#define PSPROFILE_WORKING_BUFFER_SIZE       65536
#define PSPROFILE_HOST_EXECUTION                0 //Execution is on target
#define PSPROFILE_PROFILING_COUNTER_WIDTH      32 //32bit target
#define PSPROFILE_SENDING_TYPE              PSPROFILE_SENDING_FILE

#endif

  • If your target does not contain a filesystem:

    1. In a header file psprofile_config.h, set appropriate configuration macros to handle data sending and related functions. In particular, you must define these macros:

      #ifndef PSPROFILE_CONFIG_H
#define PSPROFILE_CONFIG_H

#include <stdint.h>

#define PSPROFILE_WORKING_BUFFER_SIZE         512

#define PSPROFILE_HOST_EXECUTION              0   // Execution is on target

#define PSPROFILE_PROFILING_COUNTER_WIDTH     32  // 32bit target

// Set the macro PSPROFILE_SENDING_TYPE to PSPROFILE_USER_DEFINED to indicate that
// you will implement the data sending and related functions
#define PSPROFILE_SENDING_TYPE                PSPROFILE_USER_DEFINED

// Macro definition required by the PSPROFILE_SENDING_TYPE == PSPROFILE_USER_DEFINED.
#define PSPROFILE_SENDING_INIT()              psprofile_sending_init()
#define PSPROFILE_SENDING_DATA_ASYNC(ptrData, numData)  \
                                             psprofile_sending_data_async(ptrData, numData)

// Prototypes needed to avoid warning
#define PSPROFILE_CONFIG_PROTOTYPES           \
    int psprofile_sending_init(void);         \
    int psprofile_sending_data_async(const char* ptrData, uint32_T numData);

#endif

    2. Define the implementation of the functions mapped to the macros PSPROFILE_SENDING_INIT and PSPROFILE_SENDING_DATA_ASYNC in a configuration file psprofile_config.c.

      For example, this source file defines the function psprofile_init(), psprofile_sending_data_async() and other functions to send a string of characters through the UART interface using the function HAL_UART_Transmit() from the STM32 HAL library.

      #include "psprofile.h"
#include "stm32746g_discovery.h"
 
// In this example, the UART is used to communicate with the ST-Link COM Port on the computer
static UART_HandleTypeDef huart1;

// Implement the USART1 initialization
// Return value: PSPROFILE_SUCCESS: init went well
//               PSPROFILE_ERROR: error occurred
int psprofile_sending_init(void) {
    huart1.Instance = USART1;
    huart1.Init.BaudRate = 115200;
    huart1.Init.WordLength = UART_WORDLENGTH_8B;
    huart1.Init.StopBits = UART_STOPBITS_1;
    huart1.Init.Parity = UART_PARITY_NONE;
    huart1.Init.Mode = UART_MODE_TX_RX;
    huart1.Init.HwFlowCtl = UART_HWCONTROL_NONE;
    huart1.Init.OverSampling = UART_OVERSAMPLING_16;
    huart1.Init.OneBitSampling = UART_ONE_BIT_SAMPLE_DISABLE;
    huart1.AdvancedInit.AdvFeatureInit = UART_ADVFEATURE_NO_INIT;

    BSP_COM_Init(COM1, &huart1);
    return PSPROFILE_SUCCESS;
}

// This method is called for each frame that is sent to the host. 
// Here, the data are sent directly to the UART
// Return value: PSPROFILE_SUCCESS: sending went well
//               PSPROFILE_ERROR: error occurred
int psprofile_sending_data_async(const char* ptrData, uint32_T numData) {
    if (
        HAL_OK != HAL_UART_Transmit(
            &huart1,
            (const uint8_t*)ptrData,
            numData,
            HAL_MAX_DELAY
        )
    ) {
        return PSPROFILE_ERROR;
    }
    return PSPROFILE_SUCCESS;
}

Create Executable from Instrumented Source Code

On the host computer, cross-compile the following files for your target:

  • The source files

  • The test files

  • The optional configuration file psprofile_config.c

  • The files pstunit.c and psprofile.c available with a Polyspace Test installation (see file location below).

In addition:

For example, if your source file is named src.c and the test file is tests.c, you can cross-compile using the GNU toolchain for an Arm target (arm-none-eabi-gcc executable) as follows:

  1. Instrument the source files, and then compile the instrumented sources along with other files in a single command:

    polyspace-code-profiler -instrument -instrum-dir <instrumFolder> -limit-instrumentation-to <srcFolder> -cov-metric-level mcdc --  arm-none-eabi-gcc -c src.c tests.c psprofile_config.c <pstunit_source> <psprofile_source> -I <pstunit_include> -I <psprofile_config_dir> -I <psprofile_include> -D PST_USER_CONFIG_FILE

    Here:

    • <srcFolder> is the folder containing the source file src.c and <instrumFolder> is the folder that contains the results of instrumentation.

    • <pstunit_source> is the file <polyspaceroot>\polyspace\pstest\pstunit\src\pstunit.c, where <polyspaceroot> is the Polyspace installation folder, for example, C:\Program Files\Polyspace\R2026a.

    • <psprofile_source> is the file <polyspaceroot>\polyspace\psprofile\src\psprofile.c.

    • <pstunit_include> is the folder <polyspaceroot>\polyspace\pstest\pstunit\include.

    • <psprofile_include> is the folder <polyspaceroot>\polyspace\psprofile\include.

    • <psprofile_config_dir> is the folder containing the configuration file header psprofile_config.h.

  2. Link the object files created in the previous step.

    arm-none-eabi-gcc src.o tests.o psprofile_config.o pstunit.o psprofile.o

    This step produces an executable that you can launch on the target.

Collect Profiling Data on Target

After creating the test executable on the host, run the executable on the target. You can either stream data to the host during execution (monitor mode) or to gather all coverage data on the target and transfer it later for manual conversion.

Instead of monitoring the execution of tests, you can manually run the test executable on your target and retrieve the coverage results in a .bin format.

  1. Transfer the executable you created in the previous step to your target.

    • On targets with a filesystem, you can create a writable folder to store the *.bin file containing the data and specify the complete path to the *.bin file as the environment variable PSPROFILE_RESULTS_FILE. For example, specify <writable_directory>/<filename.bin> as the value for the variable PSPROFILE_RESULTS_FILE. When you run the executable, the .bin file in the predefined location contains the code profiling data. Transfer this .bin file to the host to convert to an appropriate format.

    • On targets without a filesystem, when you run the executable, the coverage data is streamed in a .bin file to your host. The data is transmitted using the functions you specified in the file psprofile_config.c earlier. You can read the transmitted data on your host computer over a serial port.

      For a complete example of this workflow using an STM32 Discovery Board, see Running C/C++ Tests and Collecting Coverage on STM32 Discovery Board Using Polyspace Test.

  2. Convert the data in the *.bin file into a *.psprof file by using polyspace-code-profiler. For example, at the command line, enter:

    polyspace-code-profiler -convert -instrum-dir <instrumFolder> -results-dir <conFolder> <filename.bin>
    Here:

    • <conFolder> is the folder where polyspace-code-profiler stores the output *.psprof file.

    • <instrumFolder> is the folder containing the instrumented code that you created when generating the test executable.

    After you execute this command, the folder <conFolder> contains the *.psprof file.

Create Report from Collected Data

Generate a report from the *.psprof file by using the polyspace-code-profiler command. For example, at the command line, enter:

polyspace-code-profiler -report -html -report-dir <reportFolder> <conFolder>
Here:

  • <reportFolder> is the folder containing the output report.

  • <conFolder> is the folder containing the *.psprof file you produced in the previous step.

After you generate the readable report, review your code coverage or execution profile metrics in the report. See Structure of HTML Reports Generated from C/C++ Code Profiling Results. Review the report to:

  • Identify which code coverage metrics have an unacceptably low value.

    You can add tests to increase code coverage or justify missing coverage. See Improve or Justify Missing Code Coverage Results.

  • Identify the bottlenecks and memory inefficiencies in your code.

See Also

|

Topics

External Websites