ASTRI Accelerates Development of Robotic Manipulation System Using MBSE Digital Twin
“The integration of MATLAB, Simulink, and Deep Learning Toolbox gave us the confidence to move forward with the MBSE digital twin project.”
Challenge
Reduce development time, manual processes, and costs
Solution
Adopt model-based systems engineering and develop a digital twin with MATLAB, Simulink, and Deep Learning Toolbox
Results
- Integration time reduced by 40%
- Issues resolved in the design stage
- Teams worked collaboratively
ASTRI created a digital twin to design, build, and validate its robotic welding system.
For the Hong Kong Applied Science and Technology Research Institute (ASTRI), the move to smart manufacturing is key to meeting customer demand and addressing the rising costs of development and production. As part of this initiative, ASTRI has adopted model-based systems engineering (MBSE) in the engineering lifecycle of its robotic manipulation systems. Through MBSE, ASTRI designs, builds, and validates physical systems using digital twins, which they developed with MATLAB® and Simulink®.
Where traditional CAD-based simulations enable engineers to project how a system might perform, the digital and physical twin approach determines how a physical asset actually performs as it’s being designed, enabling iterative development and modifications. “The MBSE digital twin approach reduces development time by frontloading the system validation process,” says Dr. T. John Koo, Director of Cyber-Physical Systems at ASTRI. “We can identify issues as the physical system is being designed.”
Challenge
Often in conventional development and production processes, multiple engineering teams work independently on system subcomponents. It’s not until all the separate parts are integrated that issues can be identified and resolved, which drives up costs and delays production. For ASTRI, being able to validate and modify subcomponents at any stage in the development process would create a more agile and cost-effective approach to building robotic manipulation systems.
Solution
Starting with their robotic welding system, ASTRI created its digital twin embedded with computer vision and machine learning algorithms using MATLAB, Simulink, Computer Vision Toolbox™, Deep Learning Toolbox™, and ROS Toolbox. Unlike the conventional approach, the system is designed so that the path of the welder is based on the CAD model of the weld piece. Then, based on the estimated position and orientation of the weld piece provided by a vision sensor, a robotic manipulator conducts the welding process accordingly. This approach provides more design flexibility and shortens the setup time.
To produce the digital twin’s vision, ASTRI engineers generated a set of synthetic RBGD images of the weld piece with Computer Vision Toolbox. Using Deep Learning Toolbox, they developed and trained a deep neural network (DNN) with the synthetic RGBD images to provide an initial estimate of the position and orientation of the weld piece. Then, starting from the initial estimation, an iterative closest point (ICP) algorithm was deployed to produce a more accurate estimate.
For physical implementation, ASTRI engineers created a filter that can remove the background from each image taken by the vision sensor. Using a model of the robot manipulator, they deployed motion planning algorithms developed by the Robot Operating System (ROS) community to generate feasible trajectories for welding processes based on the estimation of the position and orientation of the weld piece.
The team also created a software-in-the-loop (SIL) simulation with MATLAB and ROS to validate the behavior of the digital twin before physical execution with the robot hardware. ROS Toolbox provides an interface that connects MATLAB and Simulink with ROS, which enables development and simulation of the robotic manipulator. MATLAB Deep Learning Container and NVIDIA® CUDA® Container ran together and were implemented within the NVIDIA DGX™ station for the simulation of the digital twin.
Once the models and algorithms were developed and validated through the simulation, ASTRI engineers integrated them into the physical twin. They used an Industrial PC to control the physical twin components and ran the vision sensor and the robotic manipulator with ROS. The C code and GPU CUDA code for run-time are generated automatically by MATLAB Coder™ and GPU Coder™ with NVIDIA TensorRT™, respectively. Using MATLAB and Simulink with open-source tools, models and algorithms can be created and modified without having to specify the C/C++, HDL, or GPU code at design time, but the functionality of the design can be validated via hardware platforms implemented with different combinations of CPUs, FPGAs and GPUs.
The MBSE digital twin was just the first milestone of AI-powered smart manufacturing for ASTRI. For one of its customers, the institute has gone on to build a more advanced Collaborative Mobile Manipulator (CMM), which requires additional technologies for autonomous navigation and manipulation in 3D space. For ASTRI, combining MATLAB and Simulink brought the digital and physical worlds closer together and inspired confidence in a model-based approach.
Results
- Integration time reduced by 40%. “The MBSE digital twin approach reduced integration time by 40% and development time by 30%,” says Dr. Koo. “MATLAB and Simulink offer us a level of automation and real-world simulation that accelerates engineering processes and improves the quality of the products.”
- Issues resolved in the design stage. "With the MBSE approach, we can trace the root causes of functionality issues as we’re designing the system,” says Dr. Koo. “The digital twin built with MATLAB and Simulink allows us to visualize and test how the system will perform before all the subcomponents are integrated.”
- Teams work collaboratively. “Developing robotic manipulation systems used to be a very siloed, sequential process,” says Dr. Koo. “With MATLAB and Simulink, engineers located in different areas can work together, in parallel, resolving issues and validating systems as they are being built.”
