Introduction

The Parrot^® minidrone uses various onboard sensors like Accelerometer and Gyroscope to stabilize itself. But, due to minor imperfections, the drone does not stay in place and tends to drift. This situation can be avoided by using images from the downward-facing camera of the drone. The camera continuously captures the images and checks for motion in X and Y directions. The flight controller uses the camera data, in addition to Accelerometer and Gyroscope, to stabilize itself in space and hold the x-y position of the drone.

Optical Flow

To understand optical flow, consider a case where the drone drifts by a distance of dx in x and dy in y direction, due to some noise in the sensors. In this case, the controller needs to know the distances moved in both x and y directions to bring it back to the original position.

This can be obtained using the downward-facing camera by comparing consecutive images as the drone moves. The downward facing camera fixes onto objects/patterns on the ground and if there is any change in position of that object/pattern, the camera reports it as a movement in terms of velocity – which is optical flow. This velocity measurement can be used to calculate the distance moved by the drone.

Troubleshooting Optical Flow Issues

There are cases in which the camera cannot detect the horizontal movement. If the texture of the surface below the drone is uniform with no patterns, the camera might not detect movement. Also, when the lighting is low, the camera might not detect movement. In such cases, the optical flow values will be zero resulting in drift and possible crash of the drone.

To avoid such situations, we have modelled a logic (crash prediction logic) in the new Simulink^® project template, Hover Parrot Minidrone. The parrotMinidroneHover project (which is part of the template) contains the Crash Predictor Flags subsystem (in the flightControlSystem Simulink model). This subsystem is used to shut down the drone if there is no optical flow measurement for 50 continuous sample times.

If the drone shuts-down due to optical flow issues, some fixes can be done to the surface to increase contrast. Increasing contrast on the surface ensures that your camera produces non-zero optical flow values.

The below image shows a scenario of a surface where you may see optical flow errors:

The region marked in red is a surface of uniform texture. When the drone is flying over the surface, it receives zero optical flow information. This is what the drone sees:

In this case, the drone will drift and eventually crash.

To increase contrast, add tapes on the surface over which the drone is flying. This change avoids the shutdown of drone due to optical flow issues.

The below image shows an example of adding tapes to the surface to increase contrast.

This is what the drone sees now:

The texture on the surface need not be as shown in the above image. It can be any random pattern as long as there is a contrast on the surface.

In cases where there are optical flow issues even though there is contrast on the surface, the lighting can be an issue. In this case, increase the room's lighting so that the drone can get non-zero optical flow values.

Simulink Model for Optical Flow

In the parrotMinidroneHover project (which is part of Hover Parrot Minidrone Simulink template), the crash prediction logic to shut down the drone is modeled in the Crash Prediction Flags subsystem in the flightControlSystem model.

In the Crash Prediction Flags subsystem, the <opticalFlow_data> is a vector obtained from VisionSensors, and it has the following fields: dx_OF (velocity in x-direction), dy_OF (velocity in y-direction) and OF_status. The OF_status is ‘0’ when the drone is unable to detect optical flow and ‘1’ when the drone detects optical flow.

The maximum number of cycles for which there is no optical flow can be changed by changing the value of the Compare to constant block, highlighted in purple. Once the specified value is reached, the drone shuts down.

To disable the Optical flow, perform one of the following options:

Removal of Enable Optical Flow Option from Configuration Parameters

The Enable Optical Flow option in the Configuration Parameters dialog box is now removed, and the functionality is exposed in the flightControlSystem model as shown above. The option is deprecated from R2019a onwards.

Limitations/ Backward Compatibility

For the older Simulink models (before R2019a), the optical flow behavior is unchanged (the Parrot drone shuts down after 50 cycles without optical flow). However, it is recommended to move to the new template (Hover Parrot Minidrone) introduced in R2019a.