Deploying Neural Network with Simulink Embedded Coder: Reducing Code Size and Improving Execution Speed

21 次查看（过去 30 天）
Julian
Julian 2023-1-2
评论： Prateek 2023-1-6
My goal is to deploy a fully connected neural network model to a STM32MP1 microcontroller using the Embedded Coder.
To this end, I've adapted the following workflow (any suggestions appreciated):
  1. Transform the neural network from .pth to .onnx file format (via torch.onnx.export),
  2. Bring the network to Matlab using importONNXNetwork(),
  3. Load this .mat file inside a Simulink Predict block (see image),
  4. Generate C-Code via the Embedded Coder App, and
  5. Build and run the model on the STM32MP1 microcontroller.
The constraints of the microcontroller (RAM) and the task (frequency) are the following:
  • Network Size: Given the training results the minimum size is a MLP with three hidden layers (256x128x64), I/O = 10/2
  • Code Size: The generated Code needs to be < 200kb in order to fit into the RAM
  • Network Speed: The generated code needs to run within < 1ms
Currently, with this workflow, I am able to generate and build C-Code for a smaller network (128x64x32) that is now running on the STM32MP1 with an execution time of around 0.9ms, while the actual network (256x128x64) fails to build on the microcontroller due to an overflow of 76kB.
Hence, the two related questions are (of this priority):
  1. How to reduce the size of the generated C-Code (from 276kb to <200kb)
  2. How to improve the execution speed for the generated C-Code

请先登录，再回答此问题。

回答（1 个）

Prateek
Prateek 2023-1-6
Hello Julian,
Optimizing code generation for memory is broad topic, and I can point you to some pages that contain relevant information about this:
  1. Performance - MATLAB & Simulink - MathWorks India
  2. Optimization Tools and Techniques - MATLAB & Simulink - MathWorks India
  3. Select objective / Prioritized objectives - MATLAB & Simulink - MathWorks India
  4. Minimize Computations and Storage for Intermediate Results at Block Outputs - MATLAB & Simulink - MathWorks India
  5. Enable and Reuse Local Block Outputs in Generated Code - MATLAB & Simulink - MathWorks India
  6. Inline Numeric Values of Block Parameters - MATLAB & Simulink - MathWorks India
  7. Inline Invariant Signals - MATLAB & Simulink - MathWorks India
  8. Vector Operation Optimization - MATLAB & Simulink - MathWorks India
These might help you figure out a good approach to optimizing your model.
Hope this helps.
Regards,
Prateek
  2 个评论
Julian
Julian 2023-1-6
Hi Prateek,
Thanks for offering help on this topic and providing some links to the docs!
Having checked your pointers, I've tested the model advisor (#2) and selecting different objectives (#3) for improving the generated code without success so far. While the docs on code generation in general seems comprehensive, I feel it rarely focusing on the deployment of neural networks so far - do you have specific pointers also for the code optimization of NN?
Another Idea to reduce the size of the generated C-code for the NN could be to use the float16 datatype (half-precision), instead of the default float32 (single precision), which would essentially cut the size by almost 50%. However, using onnxmltools.utils.float16_converter to convert the ONNX model to float16 and importing it to Matlab with importONNXNetwork() results in the following error:
"Error using nnet.internal.cnn.onnx.getDataFromTensorProto
The datatype of initializer 'model._model.a2c_network.sigma' ('FLOAT16') is not supported."
Is Matlab (and the Coder) even able to handle neural networks with float16 datatypes at this point in time? An older forum entry said it was under "active development".
Best, Julian

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Simulink Coder 的更多信息

标签

产品


版本

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by