AEB Euro NCAP Testing with RoadRunner Scenario

Introduction

In the realm of automotive safety, the Euro NCAP sets important safety standards for vehicles, including tests for AEB systems. AEB is an advanced active safety system that helps drivers avoid or mitigate collisions with other vehicles. The Euro NCAP has established a comprehensive set of testing protocols to rigorously assess the effectiveness of AEB systems. This example enables you to assess the functionality of an AEB application using AEB tests per Euro NCAP testing protocols.

This example shows how to generate Euro NCAP scenarios for Safety Assist (SA) and Vulnerable Road User (VRU) AEB scenarios and their variants with RoadRunner Scenario and simulate them using the test bench model. The example also shows how to use the iterative testing methodology in Simulink® Test Manager to test the generated variants. In this example, you:

This example generates, simulates, and analyzes Safety Assist AEB Car-to-Car Front turn-across-path scenario (SA AEB CCFtap) scenarios for demonstration purposes.

Configure Euro NCAP Test Environment

This section shows how to set up the test environment and specify the necessary inputs to generate AEB test scenarios per Euro NCAP protocol.

Set Up Environment

Start RoadRunner interactively using the roadrunnerSetup function. In the RoadRunner Setup dialog box, specify the RoadRunner Installation Folder and RoadRunner Project Folder locations. You must use a RoadRunner Project created with Base + Add On Assets because this example uses assets from RoadRunner Asset Library.

rrApp = roadrunnerSetup;

The function returns a roadrunner object, rrApp, that provides functions for performing basic workflow tasks such as opening, closing, and saving scenes and projects. After setting up the required paths, close RoadRunner. The subsequent steps in this example assume the RoadRunner application is closed.

After setting up the required paths, close RoadRunner. The subsequent steps in this example use rrApp, and assume the RoadRunner application is closed.

close(rrApp)

Specify Inputs

Specify the Euro NCAP test scenario.

testName = {"SA AEB CCFtap"};

Specify the ego asset file path.

egoAssetFile = "Vehicles/ADT Vehicles/SK_Sedan.fbx_rrx";

Specify the ego behavior file.

egoBehaviorFile = "Behaviors/DrivingTestBench.rrbehavior.rrmeta";

Set the simulation step size for RoadRunner Scenario simulation.

stepSize = 0.03;

Set the vehicle fidelity model to either 3DOF or 14DOF.

vehicleModel = "3DOF";

Set the required paths for the RoadRunner application AEB test bench project by using the helperInitializeEuroNCAPTestSession helper function.

sessionTestConfig = helperInitializeEuroNCAPTestSession( ...
    testName,egoAssetFile,egoBehaviorFile,stepSize,vehicleModel);

Warning: Name is nonexistent or not a directory:
C:\Work\EuroNCAP\ExampleManager\user.Bdoc.j2747277\variantgenerator-ex33428524\toolbox\driving\supportpackages\variantgenerator\drivingApplication 
Session test folder exists for the current configuration.
Loading available session test configuration...

Examine the sessionTestConfig structure.

display(sessionTestConfig)

sessionTestConfig = 

  struct with fields:

                      Id: "24A9643B127C2C49EA1966DE1BD48CAC"
              WorkFolder: "C:\Work\EuroNCAP\DrivingTest\Work\24A9643B127C2C49EA1966DE1BD48CAC"
            TestTypeList: {["SA AEB CCFtap"]}
                TestSpec: {[9×11 table]}
                TestInfo: [9×7 table]
          PathToEgoAsset: "Vehicles/ADT Vehicles/SK_Sedan.fbx_rrx"
       EgoAssetDimension: [1×1 struct]
       PathToEgoBehavior: "Behaviors/DrivingTestBench.rrbehavior.rrmeta"
    PathToTargetBehavior: "Behaviors/Target.rrbehavior.rrmeta"
        DrivingConfigObj: [1×1 configureDrivingTestBench]
               TestBench: "DrivingTestBench.slx"
        GenerateVariants: 0
                StepSize: 0.0300
                 Vehicle: [1×1 struct]

The helperInitializeEuroNCAPTestSession helper function creates the required folders for the current test session and opens the DrivingTest project that contains the AEB test bench model for cosimulation with RoadRunner Scenario. The function also creates a session folder within the working folder WorkFolder. This session folder contains the Scene, Scenarios, and Simulation logs folders for the current session. This function outputs the sessionTestConfig structure, which contains the information required to generate test scenarios and to set up the test bench. The GenerateVariants field of the sessionTestConfig strucure specifies whether to generate scenarios for the specified test or to reuse scenarios from existing session based on their availability.

Generate Euro NCAP Scenarios

In this section, you generate Euro NCAP test scenarios for the SA AEB CCFtap test.

Review SA AEB CCFtap Scenario

The Euro NCAP specification designates the ego vehicle as the vehicle under test (VUT) and the target vehicle as a global vehicle target (GVT). In the SA AEB CCFtap scenario, a VUT turns across the path of an oncoming GVT that is traveling at a constant speed, causing a collision in which the frontal structure of the GVT strikes the frontal structure of the VUT. The VUT and GVT follow predefined paths specified in the scenario. The GVT travels on a straight-line path. The VUT travels straight, makes a left turn, and then travels straight again per Euro NCAP Test Protocol – AEB Car-to-Car systems version 4.2. You can generate nine different scenario variations by using these speed values for the VUT and GVT:

Generate SA AEB CCFtap Scenarios

This example uses the helperGenerateEuroNCAPScenarios helper function to generate the Euro NCAP scenarios. For more information on generating AEB Car-to-Car scenarios, see the Generate Variants for Testing AEB Car-to-Car Scenarios example. The helper function does not regenerate the scenarios if they are already available and you do not modify any input parameters. Depending on the number of tests and the number of variants for each test, the helper function can take some time to generate variants.

The helper function also copies the generated scenarios to the RoadRunner project and updates the TestInfo field of the sessionTestConfig structure with scenario descriptions, to help you understand the scenario parameters.

sessionTestConfig = helperGenerateEuroNCAPScenarios( ...
    sessionTestConfig,egoAssetFile);

To verify the generated scenario information, examine the TestInfo field of the sessionTestConfig structure.

disp(sessionTestConfig.TestInfo)

    Test ID       Test Type       RRScenario File Name    Test Start Time    Test End Time               Scenario Description               Target Dimension
    _______    _______________    ____________________    _______________    _____________    __________________________________________    ________________

       1       "SA AEB CCFtap"     "SA_AEB_CCFtap_1"          27.741            35.741        "VUT Speed = 10 km/h; GVT Speed = 30 km/h"      {1×1 struct}  
       2       "SA AEB CCFtap"     "SA_AEB_CCFtap_2"          27.741            35.741        "VUT Speed = 10 km/h; GVT Speed = 45 km/h"      {1×1 struct}  
       3       "SA AEB CCFtap"     "SA_AEB_CCFtap_3"           27.74             35.74        "VUT Speed = 10 km/h; GVT Speed = 60 km/h"      {1×1 struct}  
       4       "SA AEB CCFtap"     "SA_AEB_CCFtap_4"          17.923            25.923        "VUT Speed = 15 km/h; GVT Speed = 30 km/h"      {1×1 struct}  
       5       "SA AEB CCFtap"     "SA_AEB_CCFtap_5"          17.244            25.244        "VUT Speed = 15 km/h; GVT Speed = 45 km/h"      {1×1 struct}  
       6       "SA AEB CCFtap"     "SA_AEB_CCFtap_6"          17.244            25.244        "VUT Speed = 15 km/h; GVT Speed = 60 km/h"      {1×1 struct}  
       7       "SA AEB CCFtap"     "SA_AEB_CCFtap_7"          18.207            26.207        "VUT Speed = 20 km/h; GVT Speed = 30 km/h"      {1×1 struct}  
       8       "SA AEB CCFtap"     "SA_AEB_CCFtap_8"            13.8              21.8        "VUT Speed = 20 km/h; GVT Speed = 45 km/h"      {1×1 struct}  
       9       "SA AEB CCFtap"     "SA_AEB_CCFtap_9"          12.123            20.123        "VUT Speed = 20 km/h; GVT Speed = 60 km/h"      {1×1 struct}  

Review One SA AEB CCFtap Scenario

In this example, you use the AEB test bench model to simulate the first generated scenario, SA_AEB_CCFtap_1.rrscenario, with RoadRunner Scenario. Open the scenario.

rrApp = roadrunner;
testID = 1;
scenarioFileName = fullfile(sessionTestConfig.Id,sessionTestConfig.TestInfo(testID,:).("RRScenario File Name"));
openScenario(rrApp,scenarioFileName)

Note that the example saves scenarios that include behaviors for the ego and target vehicles.

The ego vehicle has the DrivingTestBench.rrbehavior.rrmeta behavior, which contains the Simulink behavior DrivingTestBench.slx. In this example, the ego vehicle must start from rest and attain a constant speed to meet the test conditions per Euro NCAP protocol. The target vehicle has the MATLAB behavior Target.rrbehavior, which contains the TargetBehavior.m MATLAB System object™. The target vehicle continuously monitors the ego vehicle speed to adjust its timing to ensure a collision with the ego vehicle.

Review and Simulate AEB Test Bench

In this example, you use a DrivingTestBench.slx model to simulate and test the performance of the AEB controller.

Open the AEB test bench model.

open_system(sessionTestConfig.TestBench)

The test bench model contains these modules:

For more information on the AEB test bench model, see the AEB Test Bench for Euro NCAP Scenarios example.

Simulate Test Bench Model

This example uses one of the generated variants, SA_AEB_CCFtap_1.rrscenario, to simulate the AEB test bench model. You can use the configureDrivingTestBench object to configure the AEB test bench. Simulate the test bench model and plot the results.

Connect to the RoadRunner Scenario server for cosimulation by using the createSimulation function, and enable data logging.

rrSim = createSimulation(rrApp);
set(rrSim,PacerStatus="Off")
set(rrSim,Logging="on")
set(rrSim,StepSize=sessionTestConfig.StepSize)
set(rrSim,MaxSimulationTime=sessionTestConfig.TestInfo(testID,:).("Test End Time"));

Connection status: 1
Connected to RoadRunner Scenario server on localhost:60731, with client id {9669219d-29fd-4e0a-8d5c-c6eaac494ed9}

Configure the AEB test bench model with AEB enabled.

drivingConfigObj  = sessionTestConfig.DrivingConfigObj;
setup(drivingConfigObj,AEBEnable=1,...
        TestStartTime=sessionTestConfig.TestInfo(testID,:).("Test Start Time"),...
    Vehicle=sessionTestConfig.Vehicle);
fprintf("\nSetup of DrivingTestBench complete.\n")

Setup of DrivingTestBench complete.

Simulate the AEB test bench model.

set(rrSim,SimulationCommand="Start")
while strcmp(get(rrSim,"SimulationStatus"),"Running")
    pause(1)
end

Plot the simulation results by using the helperPlotAEBResults helper function.

helperPlotAEBResults(out.logsout, ...
    sessionTestConfig.TestInfo(testID,:).("RRScenario File Name"));

The simulation results contain these charts:

When the ego vehicle attains the steady-state speed, this example sets the AEB Test Enable parameter to true, indicating the start of the test. Once the test starts, the ego vehicle applies brakes based on the TTC value to avoid collision with the target vehicle.

Perform Iterative Testing

This example enables you to perform automatic testing using table iterations from the test manager file. Use the helperGenerateTestManagerFile function to generate the test suite to perform automatic testing.

Generate a test manager file.

testManagerFileName = helperGenerateTestManagerFile(sessionTestConfig,rrApp);

Close the active RoadRunner Scenario instance.

close(rrApp)

Review the generated test suites and test cases by opening Simulink Test Manager and running them programmatically.

Open and observe the test cases.

sltestmgr
testFileObj = sltest.testmanager.load(testManagerFileName);

Run the test file and get the results.

run(testFileObj);

Get the latest result object.

resultSet = sltest.testmanager.getResultSets();
resultsObj = resultSet(end);

% Export the results to work folder.
sltest.testmanager.exportResults(resultsObj, testManagerFileName + '_Results.mldatx');

You also can run the tests for all the variants of the SA AEB CCFtap scenario by clicking Run in the Test Manager.

You can select or deselect scenarios in the Table Iterations section to simulate one or many scenarios.

You can observe the completion status of the SA AEB CCFtap scenario simulations in the Results and Artifacts tab.

Review Euro NCAP Test Report

In this section, you learn how to programmatically create a consolidated report after successfully simulating the AEB test bench model with all the SA AEB CCFtap scenarios, and how to observe this information in the Test Manager UI.

Collate the test results from the simulation runs by using the helperCollateTestResults helper function.

pathToSimulationLogs = fullfile(sessionTestConfig.WorkFolder,"SimulationLogs");
testResults = helperCollateTestResults(resultsObj,pathToSimulationLogs);

Display the collated collision status results for the SA AEB CCFtap scenarios.

testType = testName{1};
disp(testType)
fieldName = strrep(testType,' ','_');
disp(testResults.(fieldName))

SA AEB CCFtap
    CollisionAvoidanceStatus: [3×3 double]

Get the Euro NCAP assessment score using the helperGetEuroNCAPScore helper function.

ncapScore = helperGetEuroNCAPScore(testResults,testType);

Plot the consolidated results.

helperEuroNCAPReport(testResults,ncapScore,testType);

Observe the consolidated report generated from the Test Manager after the successful completion of AEB testing with all the variants. Note the table in the report that shows the test status for all variants, in this case titled Car-to-Car Front turn across path (CCFtap) scenarios: Collision Avoidance Status. A green field signifies that AEB has triggered as expected, in that variant scenario, and full points for avoiding the collision have been awarded. If AEB fails to trigger, and if a collision occurs, the field for that variant scenario appears in red.

You can click a field in this table, to get information about the variant name, as well as the Replay Simulation and AEB Simulation Result options.

Replay Simulation — Replay the simulation using simulation data logged from the RoadRunner application.

AEB Simulation Result — Display the AEB simulation results from the test bench model.

Alternatively, you can find this consolidated report and AEB Simulation result in Test Manager after the successful completion of all SA AEB CCFtap scenarios.

To find the consolidated report, on the Results and Artifacts tab, select the folder containing the SA AEB CCFtap test and, in the MATLAB Figures section, under Cleanup, select EURO NCAP SA AEB CCFtap scenario variation results & scoring.

If you select an individual test case, the Results and Artifacts tab instead contains the simulation results for only that test case.

Explore Further

In this example, you have generated scenarios for SA AEB CCFtap test scenarios for the Sedan vehicle. You have simulated the test bench model and analyzed the performance using Test Manager.

To use the same process for other Euro NCAP AEB scenarios, change the testName value. You can specify any test scenario from this list.

You can also specify sequence of test scenarios as well. testName = {"SA AEB CCRs"; "SA AEB CCRm"; "SA AEB CCRb"; "SA AEB CCFtap";... "SA AEB CCCscp"; "SA AEB CCFhol"; "SA AEB CCFhos";... "VRU AEB Crossing CBFA"; "VRU AEB Crossing CBNA";... "VRU AEB Crossing CPFA";"VRU AEB Crossing CPNA";... "VRU AEB Crossing CPNCO"; "VRU AEB Crossing CBNAO";... "VRU AEB Longitudinal CBLA"; "VRU AEB Longitudinal CMRb";... "VRU AEB Longitudinal CMRs" ; "VRU AEB Longitudinal CPLA";... "VRU AEB Turning CBTAf"; "VRU AEB Turning CBTAn";"VRU AEB Turning CMFtap";... "VRU AEB Turning CPTAfo";"VRU AEB Turning CPTAfs";"VRU AEB Turning CPTAno";... "VRU AEB Turning CPTAns"};

If you specify multiple tests, you can get the status of each test from Test Manager after successful simulation completion by selecting that test folder on the Results and Artifacts tab.

You can also observe the consolidated Euro NCAP report for each specified test. For example, a report for variants of the SA AEB CCRm scenario might look like this.

You can also modify the RoadRunner asset for the ego vehicle by changing the egoAssetFile value.

egoAssetFile = "Vehicles/ADT Vehicles/SK_MuscleCar.fbx_rrx";

For more information on how to generate variants by modifying the ego asset information, see the Generate RoadRunner Scenario Variants by Modifying Actor Assets example.

For more information on how to configure, simulate, and analyze the performance of the AEB test bench for one Euro NCAP scenario, see the AEB Test Bench for Euro NCAP Scenarios example.

Supporting Functions

helperGenerateEuroNCAPScenarios — Generates variants using the ncapScenario and generateVariants functions for specified tests when sessionTestConfig.GenerateVariants is True. If it is False, the function reuses the existing scenarios and copies them to the RoadRunner project path.

function sessionTestConfig = helperGenerateEuroNCAPScenarios(sessionTestConfig,egoAssetFile)

numTests = length(sessionTestConfig.TestTypeList);

if(sessionTestConfig.GenerateVariants)
    fprintf("Started generating Euro NCAP scenarios...\n")
    egoAssetDim = helperGetActorDimensions(egoAssetFile);
    for testIdx = 1:numTests
        testName = string(sessionTestConfig.TestTypeList(testIdx));

        % Get seed scenario descriptor
        fprintf("Generating seed scenario for %s (%d of %d)...\n",testName,testIdx,numTests)
        seedScenarioDescriptor = ncapScenario(testName);

        % Ego asset information
        egoAssetInfo = struct("Dimension",egoAssetDim,"AssetPath", egoAssetFile);

        % Get variation properties
        variationProperties = helperEuroNCAPVariationProperties(testName, "egoAssetInfo", egoAssetInfo,"seedScenarioDescriptor",seedScenarioDescriptor);

        % Generate scenario descriptors for the selected variations
        fprintf("Generating Euro NCAP scenario descriptors for %s (%d of %d)...\n",testName,testIdx,numTests)
        [ncapScenarioDescriptor,variantInfo] = generateVariants(seedScenarioDescriptor,variationProperties);

        % Save descriptor and variant info to sessionTestConfig
        sessionTestConfig.Variant(testIdx).Descriptor = ncapScenarioDescriptor;
        sessionTestConfig.Variant(testIdx).Info = variantInfo;
    end
    % Save the variant descriptors to sessionTestConfig before generating
    % RR scenario files.
    save(fullfile(sessionTestConfig.WorkFolder,"sessionTestConfig.mat"), ...
        "sessionTestConfig")

    fprintf("Completed generating Euro NCAP variation descriptors.\n")
end
% Generate or copy RoadRunner Scenario files based on
% sessionTestConfig.GenerateVariants. Populate test information like
% scenario file name, test start time, test end time and description
% required for testing, analysis, and visualization.
sessionTestConfig = helperPrepareTestScenarios(sessionTestConfig);
fprintf("Completed preparing Euro NCAP test scenarios for RoadRunner Scenario.\n\n")
end

Session test folder exists.
Copying existing scenarios from session test folder to RR project...
Completed preparing Euro NCAP test scenarios for RoadRunner Scenario.