Programming Drones with Simulink

Hi, everyone. My name is Greg, and I'm with Simulink Product Marketing at MathWorks. In this webinar, you will learn how Simulink lets you design, simulate, and deploy flight controllers for a palm-sized quadcopter using Simulink. Its support package for PARROT minidrones and Aerospace Blockset. This webinar is meant as a getting started resource and will introduce Simulink, a quadcopter hardware, and how you can use Simulink add-on tools to develop complete workflows from aircraft modeling and simulation to deployment and flight analysis. Let's get started.

We will achieve today's webinar objectives using this outline. There will be a brief introduction about Simulink, its add-on tools, the Simulink Support Package for PARROT minidrones, and the hardware itself. And I will mention how this webinar can be used to complement teaching and learning objectives in university-level courses and other training activities.

I will show you how you can use Aerospace Blockset to model and simulate aircraft and other airborne vehicles that are dependent on atmospheric and other environmental conditions. And then we'll cover what you need to do to bring this simulation to the real world and deploy flight control algorithms on PARROT minidrones. For this, I will show how to replace the aircraft firmware, how to fly the minidrone with the ready-to-sim deploy and fly example available in Aerospace Blockset, and how to record flight data using Simulink Coder. Then I'll conclude the webinar.

Let's take a look at the software tools and materials we use in this webinar. For this webinar, I use MATLAB^® Release 2017b, Simulink, and the Simulink Support Package for PARROT Minidrones, Aerospace Blockset, Simulink Coder, and Simulink 3D Animation™. Please note that a demo license for Simulink 3D Animation is included with Simulink, which lets us visualize simulations in 3D without the ability to modify the virtual reality files used.

For the hardware, I used a Rolling Spider minidrone from PARROT, safety goggles, a micro-USB cable included with the minidrone, a USB dongle that supports Bluetooth^® Low Energy with its drivers, additional batteries to the one included with the minidrone, and a charger. The materials presented in this webinar also apply to the PARROT Mambo minidrone, which is supported as of January 2018.

For this introduction, there are six questions that I would like to cover. This should help to bring everyone up to speed on the background needed for this webinar and invite you to investigate further.

All right. What is Model-Based Design? Model-Based Design is a process that enables fast and cost-effective development of dynamic systems, including control systems, signal processing, and communication systems. In Model-Based Design, a system model is at the center of the development process, from requirements development through design, implementation, and testing. The model is an executable specification that you continually refine throughout the development process.

And what is Simulink? Simulink is a block diagram environment for multidomain simulation and Model-Based Design. It supports system-level design, simulation, automatic code generation, and continuous test and verification of embedded systems. Simulink provides a graphical editor, customizable block libraries, and solvers for modeling and simulating dynamic systems.

Now what are PARROT minidrones? PARROT minidrones are ultra-compact flying dynamic systems with four propellers that make them a quadcopter, and they may be controlled from a smartphone or tablet. They can be made to be very stable, thanks to autopilot. They can combine signals from 3-axis gyroscopes and 3-axis accelerometers, pressure sensor for flight altitude, ultrasonic sensor for precision flight near the ground, and a down facing camera that can be used for optical flow and image processing.

Now knowing what is Simulink and what are PARROT minidrones, what is the Simulink Support Package for PARROT Minidrones? The support package is an official MathWorks hardware support add-on based on Simulink that lets you design, simulate, and deploy flight control algorithms to PARROT minidrones wirelessly over Bluetooth. It lets you use a combination of onboard sensors to develop, simulate, and test flight control algorithms. With a PARROT minidrone and the Simulink support package, you have a low-cost, palm-sized lab to perform feedback control experiments in the classroom or at home.

What are Simulink add-ons and what capabilities do they offer? Simulink add-ons are MathWorks tools that expand capabilities offered in Simulink. The Simulink product family includes add-on tools for event-based modeling, physical modeling, control systems, signal processing and wireless communications, code generation, real-time simulation and testing, verification and validation, 3D visualization, and reporting.

From more than 40 Simulink add-on tools, I'd like to focus on two that we will use in this webinar. Aerospace Blockset is a Simulink add-on tool that extends its features with blocks for modeling and simulating aircraft, spacecraft, rockets, propulsion systems, and unmanned airborne vehicles.

Simulink Coder is an add-on tool that generates and executes C and C++ code from Simulink diagrams, Stateflow^® charts, and MATLAB functions. The generated source code can be used for real-time and non-real time applications, including simulation acceleration, rapid prototyping, and hardware-in-the-loop simulation.

Why did MathWorks develop Simulink support for PARROT minidrones? The reasons are the following three. First, to help professors and instructors train students on Model-Based Design using drones as a popular hardware platform. Second, to help innovators in industry understand and adopt Model-Based Design using a higher education proven solution.

And third, to generate interest and awareness about the impact of Model-Based Design in critical, real-world applications. We would like you to take advantage of these capabilities, learn how these can be applied to your home classroom or projects, and help you get up and running in hours instead of weeks or months.

Beyond this webinar and in case of questions or troubleshooting steps for working with your minidrone, we provide detailed steps in documentation accompanied by MATLAB Answers™ that can be found from the hardware catalog page. Let's go ahead and see how we can use Aerospace Blockset to model and simulate the drone.

Aerospace Blockset provides a ready-to-sim build and fly example on which we based this webinar. In this section, you will learn how to find and open an instance of the quadcopter project from Aerospace Blockset examples. You will get a brief tour on how the example makes use of best practices for this application, show you how to run a simulation of the drone, ways to visualize results, how to modify the model to customize flight simulation inputs, and an overview of the flight controller provided with the Aerospace Blockset example.

You can find and open the simulation model by entering doc in the MATLAB command window, selecting Aerospace Blockset, clicking on examples, and scrolling to quadcopter project. On the documentation page, you can find details about this example. You can also open it directly by entering the following in the MATLAB command window.

Once a project environment is configured, there will be three new windows: the Simulink project name Quadcopter, the top level of the quadcopter flight simulation model, and a visualization window for the drone that employs Simulink 3D Animation. The Simulink project is environment that lets us organize files, settings, and user-defined tasks related to this example and contains an instance of them that we can modify locally on the user's Simulink project folder without overwriting the example files shaped in Aerospace Blockset.

Let's take a brief tour of the simulation model. The model is composed of six main subsystem blocks in orange that contain mathematical representations of the dynamic systems in them. There is a mathematical representation for the dynamics of the airframe, the sensors, the environment, and the flight control system. And there are also blocks used to provide input commands to the simulation and obtain a visualization output.

Four of the subsystems here are variant subsystems, which let us switch between various subsystem choices, hence their name variant. The block called FCS for Flight Control System is, in this case, not a variant subsystem, but it is a modeled subsystem, which means that the block contents make reference to another Simulink model. There are a couple of other blocks at the top level of this model to set the pace of the simulation and another one to make the simulation stop in case it reaches an undesired flight condition.

To run a simulation of the drone, all you need to do is press Play and you will see Simulink obtaining the time response of the system for the duration specified in T Final, and then stop. You can use the set pace block to change the speed at which the simulation runs.

For example, you can run only a tenth of a second of a simulation for each real second and increase the sample time to 100 microseconds to visualize a slow-motion animation of the quadcopter. When running the simulation, you can see the 3D model of the minidrone take off from the ground and hover.

If you double click on the visualization subsystem, you will find another couple of options. First, signals from states  are connected to standard cockpit fly instrument displays available with Aerospace Blockset. This instrument will show measurements of aircraft variables, in this case, artificial horizon, altimeter, airspeed indicator, heading indicator, climb-rate indicator, and percent RPM indicators for each of the four thrusters. The associated signals are found in the extract flight instruments subsystem.

Secondly, the visualization block includes for variant subsystems options to capture or display simulation results. The variant active by default is a Simulink 3D Animation option. The other three options controlled by the variable VSS visualization are scopes to visualize the time response of variables and states; workspace to save the time response variables on the MATLAB workspace and its means for post-flight simulation analysis, for example, with Aerospace Toolbox; and flight gear, a flight simulator to render aircraft dynamics in this third-party and open source simulator commonly used in the aerospace industry.

In this webinar, we will rely on the Simulink 3D Animation rendering option using the demonstration license enabled for all Simulink users. And we will see how changes in the flight simulation inputs result in changes in the aircraft simulation behavior.

To modify the quadcopter flight simulation inputs, you can use a command subsystem. This one has four other variants that let you choose the source of the input signals used for simulation. There are the Simulink builder approach, the joystick option, the data input variant, and reading from a spreadsheet file.

The default variant used for the command subsystem employs signal builder, an easy option we use in this webinar. This variant lets you define the XYZ and yawl, pitch, roll commands as reference signals of the flight controller. You can modify any of these signals on signal builder and observe the change in aircraft behavior in simulation. You will use one of the visualization options already covered in this webinar.

The flight controller is found in the FCS subsystem. When we double click on it, it opens another model to which this block makes reference. This is called a model subsystem, and it's included in the base Simulink library. In this case, this model subsystem contains and encapsulates the flight control algorithms that will eventually be deployed on the drone.

Keep in mind that the simulation model is at the center of Model-Based Design and helps to test designs in simulation before we ever try them on a real piece of hardware. This helps refine the control design before we're ready to test it on hardware and also helps to prevent all sorts of issues during testing, including damage and crashes. You can only imagine how important MBD is to the development and lifecycle of systems more critical than a minidrone.

Let's see what's inside our flight control system. The algorithm in the subsystem is an example of what our controller could include. This flight controller design is derived from work at MIT by Professor Sertac Karaman and Fabian Riether. It makes use of the aircraft commands sensory inputs containing the hardware abstraction layer and provides outputs to the motors and a crash predictor flag that helps us stop the simulation, or the flight, in case loss of stability is detected.

Please know that the flight controller in this webinar is part of the quadcopter project example shipped with Aerospace Blockset and is only intended for that purpose. If you're interested in designing your own state estimator and flight controller, you can start by exploring the controller and estimator model subsystems found in the flight control system block.

However, flight control design is beyond the scope of this webinar, but we invite you to use these subsystems to come up with your own designs. For example, students at MIT have used a controller subsystem to implement state feedback controllers and LQR controllers as part of feedback control courses for undergraduate and graduate students.

Once a flight controller has been designed and tuned in simulation, you may want to generate code and test it on the minidrone hardware. Now that we have a better understanding of the simulation model of the drone and how it works, we can go ahead and get more hands on with the hardware.

This section of the webinar takes you through the steps to set up the hardware and wireless communication needed to deploy flight controllers to the minidrones. In order to deploy the flight controller design on a PARROT minidrone, you will first need to download the hardware support package using the Add-On Explorer and follow the hardware set-up steps provided. This will help you install a custom firmware that PARROT has provided to let Simulink users design and integrate their own flight control algorithms with the aircraft software system. Firmware replacement takes place over a micro-USB connection and may take about five minutes.

After the firmware has been replaced, depending on your operating system, there are specific steps you should follow to enable wireless communication between the host computer and the minidrone. For computers running on Windows^®, you will need to enable a Bluetooth Low Energy interface with supported chipset and drivers or to use a USB dongle with those characteristics. For Windows, the recommended drivers are the CSR Harmony wireless software stack version 2.1.63.0.

For each one of the operating systems supported—Windows, Mac, and Linux^®—the support package documentation includes more detailed set-up and troubleshooting steps. Additionally, for some specific configurations, our teams at tech support have provided MATLAB Answers that can help address any persisting connectivity issue.

After the Bluetooth connection has been verified with a drone and before any flight tests with a controller have been designed, it is worth using a template included with the support package to perform a tabletop test in which we only move the propellers. To do this, you can open the template by entering parrot_gettingstarted in the MATLAB command window and clicking on Deploy to Hardware.

The diagnostic viewer window offers information about the code generation, compilation, file transfer, and execution of the drone process. At this point, the model is ready to run on a drone. To run it, first open the flight control user interface by clicking here on the diagnostic viewer, and optionally, setting up the duration for which you will like the model to execute. Be careful to not increase the power gain on the flight control UI at this time as the propellers may produce sufficient thrust to initiate an uncontrolled flight.

Next, by clicking on Start, you will execute your algorithms on the minidrone. In this case, there are no flight controller algorithms in the loop, and we're only sending signals to the motors to make sure that the toolchain and Bluetooth communication are working well.

After these have been verified, we can interrupt the execution of the algorithm by clicking on Stop. The flight control user interface lets us download the flight log and the MAT file with flight data from the minidrone to the current directory. It is a good practice to retrieve these files for the record and analysis. Once a co-generation process has been tested with a drone, we can now get ready to deploy the flight controller design from the Aerospace Blockset example and perform an actual test flight.

Having the flight controller design ready and the drone hardware set up and tested, we now cover the test flight and analysis section of the webinar. First, using the Aerospace Blockset example, we will set up the hardware target to deploy code specifically to the PARROT minidrone, and we will make sure that we have enabled data logging to leverage the capability provided by Simulink Coder to record sensor and flight controller signals for post-flight analysis.

Then, we will perform the first test flight, which with a support package in Aerospace Blockset, you should be able to reproduce in only a few hours or even less. I will then explain how to modify the command input to track an altitude reference signal defined with signal builder and how to obtain the flight data for analysis and visualization.

Going back to the Simulink project containing the Aerospace Blockset example to set up the hardware target from the project, all you need is to click on the shortcut available on the toolstrip. Then, on the quadcopter flight simulation model, double click on the FCS subsystem. To enable data logging on the aircraft, you can enter the following in the MATLAB command window.

When data logging is enabled, the Simulink model will try to allocate memory before flying to stored sensor flight controller signals. Please note that the minidrone can only store so much data each flight, and reducing the flight duration parameter, T Final, will let you control how much memory is allocated.

If the memory needed exceeds the drone capacity, the Simulink model will not execute when pressing Start on the flight control user interface. With these considerations, we're ready to try the flight controller ship with Aerospace Blockset. However, other important considerations are the following.

All flights are test flights. You should test flight controller designs and inputs that meet a performance criteria in simulation. Perform test flights in a room that is large enough, for example, a room that is at least 20 feet by 20 feet by 10 feet high. Take reasonable care to prevent damage to the aircraft, its surroundings, observers, and yourself.

On ultrasound sensitivity, small rooms may cause flight stability issues from ultrasound signals bouncing on walls and furniture. Ground materials, for example a carpet, may absorb ultrasound signals and affect flight stability.

On camera sensitivity, patterns on the ground may affect optical flow performance and flight stability. Lighting conditions may also affect optical flow and image processing performance and flight stability. Keeping these considerations in mind, let us try first the hover flight of the Aerospace Blockset example.

We click on Deploy to Hardware. Let Simulink build and transfer the flight controller to the drone. Open the flight control user interface and click on Start. Make sure that any other participants are aware of the test flight that is about to take place. For each flight, it is best practice to download the flight log and the MAT file with flight data for post-flight analysis.

Now, I will modify the flight control system a little and bring in the reference signals to deploy them to the drone as well. I will try changing the reference signal for the y-axis in the left-right direction and make the drone change position to a flight according to the square signal shown. For this webinar, I created a variant subsystem for the flight controller that lets me choose this as an option.

Once selected, I simulate, verify the expected behavior in simulation, and then deploy to the hardware and test. I deploy to the hardware up in the flight control UI. And here's the result: The drone takes off and holds to a center position, and then follows the given reference signal in y.

After the flight is complete, I retrieve the flight log and the MAT file from the minidrone. We can use the data stored in a MAT file to plot and visualize sensor signals, orientation, motor output, altitude, trajectory, position, velocities, optical flow velocities, and battery charge percentage.

Similarly, I have created another variant system that changes the reference signal for the z-axis and makes the drone change altitude with the flight profile shown. Once more, I simulate to verify the expected behavior and then deploy to the hardware for a test.

I deployed to the hardware. Open the flight control UI and here's the result. The drone takes off to one meter of altitude and climbs to two meters, drops one meter again, and then descends to the ground.

Finally, let's talk about providing altitude set points directly from Simulink. We call this running the flight controller on external mode, which builds and deploys the Simulink model on the hardware while allowing certain parameters to be updated during execution from the block diagram.

In this case, we use this variant of the flight controller and make use of this dashboard block to manually modify the altitude of the drone during flight. I deploy to the hardware. Open the flight control UI, and here is the result of the test flight. The drone takes off to about one meter of altitude and then follows the altitude commands I send from the Simulink editor.

You can try a few other changes to the reference signals on your own to test this flight controller. However, this design is only a starting point and does not make use of the atmospheric pressure sensor, for example. What changes would you introduce to the flight controller to make it more robust using the atmospheric pressure sensor?

The objective of this webinar was to provide a getting started resource on using Simulink to simulate build and fly a palm-sized drone that you can use to learn about control systems and Model-Based Design. Now you can dig into the flight control system and come up with your own flight control strategies, designed simulations, and test flights. Now, you can also start navigating the V diagram using Model-Based Design and develop techniques using any of the tools supported for simulation, code generation, and verification and validation workflows with Simulink.

Finally, here are some additional resources that you can use to learn more about Model-Based Design, Simulink, the Simulink Support Package for PARROT Minidrones, Aerospace Blockset, and how to join the community through MATLAB Central™, File Exchange, and MATLAB Answers. Thank you for joining us. We look forward to seeing your projects in the future.