Real Time Object Tracking using Simulink and Raspberry Pi

Hello Friends, 

In this project we will design algorithm for real time tracking of red colored objects using image processing techniques and implement the design on Raspberry pi hardware. 

Real time Object tracking has a number of applications like tracking trespassers, counting products on assembly line, Robotics applications etc.  

Here we built a simulink model for our application and deploy the model on the raspberry pi hardware. 

Simulink model takes live input from camera and provides output to the pan-tilt arrangement in the form of GPIO signals to move the pan-tilt in the horizontal plane

The system has been divided into four parts for the convenience of the reader

Part I - Download Matlab and Simulink hardware package for Raspberry Pi

Part II - Perform System setup and configuration of the hardware package for our Raspberry Pi hardware

Part III - Design Simulink model for colour based Real time object tracking.

Part IV - Build and deploy the system onto Raspberry Pi

Lets get started

Requirements:

Raspberry Pi 2 model B and above versions

MATLAB version 15a is used here

Computer/Laptop 4GB RAM

Part I: Download Matlab and Simulink hardware package for Raspberry Pi

Open MATLAB and in Home tab click Add-Ons as shown

This will display a drop down list from which select Get Hardware Support Packages

This may take few minutes

After few minutes a pop up window Initializing Support Package installer will appear as follows

After initialization, we get the installer window as follows

Select install from internet (if you have a good speed internet connection)/ Download from internet (for slow connections), and click next

It will then scan for the installed packages, this may take few minutes.

Finally, the list of available packages for different hardware is displayed as follows.

Select Raspberry Pi , it will show two packages, MATLAB and Simulink, Select both by clicking on install (here it shows reinstall as its already installed on my system) and then click next.

It then asks to login to your MathWorks account (Don't have one, well it's easy to create one). After login confirm the installation of the selected packages. Simply click on Install to confirm the installation. The Download for Raspbian Wheezy starts and this may take few minutes. Raspbian Wheezy is the Linux distribution that comes with this support package.

Download is followed by the installation procedure and therefore takes some time.

(Grab a cup of coffee in the meantime 😄 )

OK... So once we are done with the installation, we see a window that guides us further to perform the required setup task. Click Continue

Part II : Setup and Configure the hardware

After clicking continue, the setup window appears

Select the Support Package from the drop down list.

The nest window allows to select the hardware/Board. Select the board from the drop down list. Next step is to configure the network. Here you see a window as below

Here we have three options for performing network configuration of Raspberry Pi. The host computer uses this configuration to detect the hardware. You can either directly connect Pi to your host computer through RJ45 cable (which is the easiest way) or else you can connect it to your home network / Router or lastly you can manually enter network settings if you are connected to a LAN. When you manually enter the network setting make sure to enter the IP address in the same range (Preferably lesser) of that of your host computer. FOr example if your host computer has the address 192.168.15.105 then Raspberry Pi can be assigned 192.168.15.104.

Once done with the network setting click next. The system will start Enumerating Memory Card devices. Here you will be prompted to insert a 4GB or larger memory card into your host machine. After having the card inserted, select the drive from the drop down list.

If the drive is successfully selected then you can consider yourself to be Perfect.. 😄😎

But if you get a Pop Up saying sdwriter.exe has stopped working like below

then there are two work around 1. Restart MATLAB as administrator and type Targetupdater in the command window. You will be landed directly to the Support Package Installer window. Select the drive and click next, if the problem still exists then execute 2nd workaround.

2. You need to copy the mcf110u.dll into the same directory as the sdwriter.exe. The DLL is found in MATLAB/bin/win64. After this type Targetupdater in the command window. You will be landed directly to the Support Package Installer window. Select the drive and click next, now the problem should be solved.

Confirm the firmware write and the process will start, this may take some time.

Further you will be prompted to remove the card and insert it to the RPi hardware, connect the hardware to the host through Ethernet cable either directly or via some network. Turn the ON and continue

It starts detecting the board and will ask to confirm the configuration once detected. Finally, test the connection and click Finish

Part III : Design and Build Simulink Model

The Simulink model is available here

        Simulink model takes live input from the camera and triggers the DC motor connected to GPIO of RPi to move the camera left or right thereby tracking the object. 

Let peek into the Simulink model a bit.

The model starts with a user defined block that takes three inputs R,G and B from the video capture block of RPi and concatenates them to produce a single output that is then provided to the Simulink's inbuilt RGB to Intensity color space conversion block.    

 It then performs image processing operations like filtering and thresholding for colour segmentation. This segmented binary image is then given to Blob Analysis block which, using morphological operations, finds a group of 8 connected pixels of given size (here 600X3000). It provides centroid and the bounding box co-ordinates of these blobs. The centroid coordinated are used to move the motor in the appropriate direction whereas the bounding box coordinates are used to draw the rectangles over the original image and is provided to the display (monitor in our case).

Part IV: Deploy the model on the hardware

Once the Simulink model in built successfully, next step is to deploy it on to the hardware. To do this we have to set the simulation mode to external and select deploy to hardware as shown beow

Hardware Connections: The geared DC motor is connected to GPIO 17 and GPIO 27 through motor driver IC L293D. Our camera is mounted on this DC motor apparatus.