Table of contents
Object Tracking with a Radio Controlled Quadcopter
The goal of this project is to implement a simple object tracking oriented by a predefinide color using the images captured by a USB camera onboard a quadcopter. The main ideia is to use a Intel Edison (Figure 1) board to do the image processing and to command the movimentation of the quadcopter when is necessary (general commands will received by a commercial radio control). The quadcopter will use the MultiWii controller as stabilization system (Figure 2). In the Figure 3 you can see a high level block diagram of the project.
Figure 1 - Intel Edison and the official Arduino Compatible Board
Figure 2 - MultiWii SE v2.5 Controller
The quadcopter will be controlled mainly by the radio control, but at the same time, the tracking of an object can be activated via a switch located at the radio control. The activation of this switch will be acknowledged by the receiver, that will enable the en_track signal (ADC input) at the Intel Edison board. Starting at this moment, the object tracking system at the Intel Edison board will seek a target of predefined color in the current image captured by the camera. When a target have been acquired, the track system will work to keep the target always in the center of the camera frame, this will be done sending movimentation commands to the quadcopter through the MultiWii serial interface, using the MultiWii Serial Protocol (MSP).
In this object tracking project, the camera will be installed under the quadcopter, capturing images of the ground, so it can move left / right and forward / backward, maintaining a constant height.
Figure 3 - Blocks diagram
The movimentation of the quadcopter is done by changing the throttle applied to each one of the four brushless motors, the throttle to each motor should be adequated for the desired movement, like you can see in the Figure 4. This will be managed but the MultiWii controller board.
Figure 4 - Quadcopter states ("+" configuration). (a) Forward motion; (b) backwards motion; (c) movement left; (d) movement right; (e) increase altitude; (f) decrease altitude; (g) leftwards rotation; (h) rightwards rotation.
- Radio Controller (TX + RX) (Borrowed by LISHA)
- MultiWii board (Borrowed by LISHA)
- Quadcopter frame with brushless motors and ESCs (Borrowed by LISHA)
- USB camera (To be acquired or borrowed by LISHA)
- LiPo battery 11.1 Volts >25C >3000mAh (To be acquired or borrowed by LISHA)
Utilized components details:
- 4x Brushless motor A2830-11 1000KV RCTimer Outrunner
- 4x ESC SK-30A Simonk Firmware ESC BEC 5V/2A 2-4S LiPo RCTimer
- 4x Propeller 10x4.5 (two reversed)
- The Quadcopter's battery should provide enough power for the motors, MultiWii and the Intel Edison board.
- The battery should supply enough power for the Quadcopter for at least 10 minutes and save power for landing if the battery is low (disabling the image processing function).
- The camera images should have the adequate resolution to be processed in the Intel Edison board (and/or the images could be optimized for performance by downscaling resolution if necessary).
- The processing time between a camera frame and another should be of the order of milliseconds.
- The response time to data received from MultiWii by Intel Edison board must be of the order of milliseconds.
- The Quadcopter must do smooth movements, because the camera is fixed (Note: Ideally, the camera should be installed on a gimbal, so the inclination caused by the movement of the Quadcopter is compensated and the camera keep the target on the frame).
- 620g Quadcopter frame + 4 ESCs + 4 brushless motors
- 66g 4 Propeller with spinner (16,5g each)
- 11g MultiWii board
- 22g RX module
- 7g Intel Edison module
- 46g Intel Edison Arduino board
- 32g USB Camera
- 193g Battery 2800mAh 3S 35C
- 361g Battery 5000mAh 3S 30C
- Just Intel Edison Kit + USB Camera: 85g
- Quadcopter without battery: 804g
- Quadcopter with 2800mAh battery: 997g
- Quadcopter with 5000mAh battery: 1165g
A simplified functional model was created in the Ptolemy II software, which can be seen in the Figure 5.This model simulates the object tracking and the quadcopter position correction. I used a series of images simulating the frames captured by a camera (you can see the frames sequence in the Figure 7), which are processed by an external program (described in the subsection 5.1). This program processes the frames, seeking for a target, the target is defined by a color range in the HSV color space. When a target is found, the movimentation command and the offset corrections for the X and Y coordinates are generated, this data would be used by the tracking controller (Intel Edison) for repositioning the quadcopter.
Figure 5 - Simulation Model
The simulation result can be seen in the Figure 6, the image processing program works to centralize the tracked object in the center of the camera frame, generating the corrections offsets and the movimentation commands.
Figure 6 - Ptolemy II simulation results
The frames sequence in the Figure 7, tries to simulate the desired behavior of the quadcopter in the real world, also considering the possibility of a rapid movement of the target (middle of the frames, when the quadcopter is on HOLD and the boat seems to move ahead).
Figure 7 - Simulation camera frames and correction commands
The program uses the OpenCV library to do the image processing. In this simplified model, it is not doing the tracking of objects properly, because each frame is processed independently, so, if there is more than one object in the scene with the same color, only the first to be located will be tracked, even if the first is not the object which was first in the previous frame. The source code can be seen here:
- Process 2 reads the en_track signal from the RX module and telemetry data from MultiWii board.
- Process 2 sends these data to Process 1 using the pipe channel.
- Process 1 evaluates the received data, do the necessary processing and send new data back to Process 2.
- Process 2 send commands to MultiWii board if applicable.
- http://www.mouser.com/images/microsites/Intel_EDI1ARDUINALK.jpg (Figure 1)
- http://artofcircuits.com/wp-content/uploads/2014/11/MultiWii-SE-2V5-5.jpg (Figure 2)
- http://www.mdpi.com/1424-8220/15/12/29785/htm (Figure 3)
- https://commons.wikimedia.org/wiki/File:USS_Hunley_%28AS-31%29_top_view_1980.jpeg (Figure 7)
- https://github.com/dch33/Quad-Sim (MATLAB Model)