Toyota Builds Virtual Proving Ground for ADAS
Creating Digital Assets for Realistic Virtual Testing
A sleek, white Toyota® SUV prototype gleamed at the automaker’s technical headquarters in Aichi, Japan, ready for a rigorous test drive. Engineers installed equipment so a robot driver could accurately assess the vehicle’s drivability and performance, including its advanced driver assistance systems (ADAS), on winding roads.
Only the winding roads in this test drive were actually a sophisticated vehicle-in-the-loop simulation (VILS) designed to evaluate ADAS and other dynamic functions. A large external screen displayed 3D scenes while a real-time driving simulator produced corresponding motions, vibrations, and sounds. Sensors, including lidar and cameras, provided live feedback.
Toyota’s VILS, the Advanced Driver Assistance Systems Real Car Simulator (ADAS-RCS), connects real and virtual worlds through a chassis dynamometer with a cosimulation block. The robots drive for hours, checking key performance indicators for potential weaknesses. Then artificial intelligence sets targets for addressing these weaknesses, based on the results.
Toyota’s VILS, the ADAS-RCS. (Image credit: Toyota)
“The simulator’s main feature is the ability to conduct multifunctional evaluations in one go,” says Daiki Miyata, a vehicle performance development engineer in the Model-Based Development X-in-the-Loop Simulations Group (MBD XILS) at Toyota, who also works in the Model-Based Design Platform Group. His team works to streamline the vehicle development process through focused simulations. Achieving lean vehicle development is the goal.
The realistic environments in Toyota’s ADAS-RCS enable testing on complex road surfaces that aren’t available at the automaker’s physical tracks. But developing 3D road data models requires a significant amount of time and effort. The team turned to MATLAB® and RoadRunner to replicate real-world off-site driving conditions.
Testing road conditions in RoadRunner with MATLAB. (Image credit: Toyota)
“We adopted RoadRunner because we were impressed by its ability to create 3D virtual scenes, read various map data from diverse environments, and analyze the map data through its integration with MATLAB,” Miyata says. “Being able to make adjustments intuitively in RoadRunner was also a deciding factor.”
The Toyota team's VILS primarily targets adaptive cruise control, but the engineers also plan to utilize the simulator for lane tracing assist, lane change assist, pre-collision safety systems, and new advanced safety features.
“The ADAS-RCS has only recently entered the operational phase. We see many opportunities to come,” Miyata says.
Front-Loading Risky Road Conditions
In the past, skilled drivers at Toyota, known as masters, spent long hours evaluating vehicle performance on test courses at the automaker’s facilities in Japan. Design constraints necessitated testing worst-case scenarios late in the automotive development process. Weather could affect on-site testing conditions, requiring retakes that added pressure to a tight schedule.
Automotive development has become increasingly complex with the advent of ADAS and software-defined vehicles. At Toyota, engineers are front-loading road tests for ADAS by replicating conditions that affect drivers’ responses, judgment, and vehicle operation, such as lane changes in heavy traffic or twisty mountain roads.
“Factors like weather, pedestrians, and moving objects cannot be captured by predefined scenarios,” Miyata states. “Finding and overcoming such edge case scenarios quickly is essential for enhancing vehicle safety and security.”
“Thanks to MATLAB and its toolboxes, I was able to focus on what I really wanted to do.”
The architecture of ADAS-RCS. (Image credit: Toyota)
Obstructed views and blind curves presented the biggest technical challenges. In traditional manual scene generation processes, creating 3D roads was arduous and required more than six months per course. Plus, these routes wouldn’t work in other simulators at Toyota, including model-in-the-loop and software-in-the-loop simulators.
“Toyota has various simulators, so preparing scenes that cannot be reused for each kind would incur enormous costs,” Miyata says. The team needed a faster and more efficient approach.
Early in Miyata’s role as a vehicle performance engineer at Toyota, he was responsible for improving vehicle comfort. He learned about MATLAB while developing a method for testing engine noise. “I deepened my understanding of signal processing,” he remembered. “Thanks to MATLAB and its toolboxes, I was able to focus on what I really wanted to do.”
He selected MATLAB and RoadRunner to generate 3D driving routes for the MBD XILS Group’s simulation bench. The team started by pulling latitude, longitude, and altitude data from a ZENRIN® DataCom standard definition map segment for the road generation stage.
Miyata’s experience with MATLAB enabled him to devise a method for automating tedious data preprocessing required for road generation, including point smoothing and curve fitting, to generate road banks. Then Miyata converted the map data to OpenDRIVE® format using the Driving Scenario Designer app in Automated Driving Toolbox™.
“The Driving Scenario Designer app was convenient and allowed me to do what I wanted more quickly than expected,” he says.
Generating Grounded Surroundings
Once they had generated roads, the engineers leveraged RoadRunner to construct the surrounding environment. At this time, the team encountered pitch problems. At first glance, the road in the interactive editor seemed smooth, but when Miyata drove along it in the simulator, the vehicle bobbed like it was on a bumpy road. RoadRunner created roads based on the OpenDRIVE file, but the roads had centimeter-level bumps due to discontinuous road altitude at connections.
Miyata used polynomial approximation for altitude info when creating roads in MATLAB, but using RoadRunner Scene Builder prevented similar jolting. “Now we can generate road models more efficiently and with greater scalability,” he says.
Another issue emerged when Miyata added elevation data to the simulator and took his colleague for a test drive. The road visible ahead on the monitor appeared to be floating in a bright blue sky scattered with wispy cirrus clouds.
“Without RoadRunner, it would have taken enormous time and effort to create the virtual courses.”
Road smoothing using polynomial approximation for the altitude of roads. (Image credit: Toyota)
The problem turned out to be missing elevation data at certain spots. The digital elevation data originated from the Geospatial Information Authority of Japan, which conducts national surveying and mapping. However, a third-party tool it was using showed a -9,999 elevation at the river in its selected map segment. Miyata turned to MATLAB for the solution.
His team created a tool that combined elevation and image data and converted them into a mosaic-merged GeoTIFF format for RoadRunner. With the new elevation data ready, Miyata zeroed in on a specific map area in the RoadRunner graphical user interface. He dragged and dropped images to the ground surface, checked the fit, and then added the OpenDRIVE road data. In a few easy clicks, he added a dotted line to give the road two lanes, made minor corrections, and pasted in the elevation.
“The RoadRunner graphical user interface is intuitive,” Miyata says. “It is easy to make adjustments for tricky situations like this.”
The final step involved checking road generation through a video playback in RoadRunner Scenario. One button offered a 360-degree, drone-like overhead view of the vehicle’s surroundings, and another allowed the team to see the road from the driver’s perspective. This time, the ADAS-RCS had accurately banked areas and undulating mountain scenes. No more floating. Adding shading and detail to the rolling landscape created a more realistic environment.
“Without RoadRunner, it would have taken enormous time and effort to create the virtual courses,” Miyata says. Miyata estimates that it would take up to at least six months for his team to create a similar driving scenario.
Scaling Up Lean Vehicle Development
New 3D courses that used to take Miyata’s team more than a day to create can now be implemented in under 30 minutes. RoadRunner increased the team’s overall productivity.
“The ability to output formats such as OpenDRIVE and OpenSceneGraph, which are suitable for various simulators, significantly reduces scene generation costs,” Miyata explains. “That’s a major benefit.”
Next, his group aims to automate and streamline their VILS process further. They also plan to collaborate with the mass production development team to bring the benefits of simulation to them. Miyata believes it’s essential to have others utilize the technology, identify issues, and implement improvements.
“We have made considerable progress and succeeded in incorporating real traffic flow using RoadRunner Scenario.”
Reproduction of the surroundings around Toyota’s Headquarters. (Image credit: Toyota)
During his presentation at MATLAB Expo Japan, Miyata discussed scaling up development. He highlighted how MATLAB live scripts, App Designer, and MATLAB Compiler™ help deploy code for multiple users to process geographic information system data. He said using Git™ allowed for version control and customization according to user needs. Live scripts, he continued, made it possible to write easy-to-understand manuals in environments like JupyterLab, enabling user-friendly app deployment.
Since the expo, his team has expanded the simulator’s functionality using the free editable global map OpenStreetMap® to reproduce simple buildings known as “tofu assets” after the rectangular soybean curd blocks. Toyota also has a department that specializes in generating precision assets. Miyata says that the ADAS-RCS initiative inspired them to examine new ways to utilize these resources.
Front-loading the evaluation of driving on public roads with their ADAS-RCS is still in the verification stage, but Miyata and his colleagues expect that it will ultimately reduce the days required for conducting the test driving by approximately 70%.
The group’s future plans involve using point cloud data from the field to automatically classify and place building assets in RoadRunner using AI. Mixed-reality technology is also being explored.
“In addition, we have made considerable progress and succeeded in incorporating real traffic flow using RoadRunner Scenario,” Miyata says. “You can’t test ADAS thoroughly without a realistic simulation environment.”
The team addressed many challenging cases while building digital assets for the simulation environment. Having capable and easy-to-use tools was essential.
“If we can effectively utilize digital assets, we expect the ADAS-RCS and XILS to level up,” he continues. “Deploying it as a toolchain will become possible, improving the entire vehicle development process.”
Read Other Stories
AUTOMATED DRIVING / ROBOTICS
Applying Proven Robotics Techniques to Automobile Development Processes
Renault Advances ADAS Prototyping with ROS Toolbox and Simulink Integration
AUTOMATED DRIVING / ROBOTICS
Designing WATonoBus, the Autonomous Shuttle Bus on Campus
Self-Driving Shuttle Is an Ideal Platform for Developing ADAS Technology
