Algorithms for Passengers Movement Tracking

Tal Yahav and Idan Aharoni
Supervised by Johanan Erez


The final goal of this project was to develop an autonomous vehicle that uses a small camera and a Pocket PC computer to track and follow a walking person. In our work we dealt with finding a suitable algorithm for this mission. Several possible algorithms were examined theoretically, three of them were implemented in MATLAB and practically checked on sample movies.

The problem

The objectives of the algorithm:

  • "Target Recognition":Recognizing of movement of person's feet or body
  • "Target Tracking": Calculation of movement directions and tracking of the "target"
  • Taking into account the camera's self-movement
  • Taking into account background noises Low computational complexity in order to allow it to work on a computer with low resources
  • High robust performance

Few basic assumptions used in our methods:

  • Passenger's colors do not change during movement
  • While walking, passenger's legs appear diagonal to each other during most of their movement

The solution

The three algorithms that we implemented and tested are:

1. Movement estimation base block comparison

In this method each frame is divided to blocks of 16X16. For each block we then try to estimate movement by correlation with the near area in the following frame. With these results we produce a movement vectors diagram, which indicates which blocks have moved, and where to, so we can determine to which direction the figure moved.

Movement estimation between 2 frames

Estimation of motion vectors between 2 frames:

2. Feet tracking based on recognition of diagonals in the image

This algorithm was motivated from an article explaining a method of recognizing people walking in an urban environment. The method is based the assumption that in an urban surrounding most lines should be straight lines. This assumption may enable us to filter these lines and remain with the legs of walking persons. We implemented such an algorithm and tried to track pedestrian legs by this method.

3. Tracking according to colors characteristics

In this method, we first recognize the motion in the movie's frames by differentiating two concurrent frames. A new legal target is defined by as the most significant block of "moving pixels" in the current frame. In each frame , the algorithm draws a window around the " target". The window is adaptive so that its size changes when the "target" gets closer or farer to the camera. Once we found a target we analyze the color histogram of this block and determine the most dominant colors in this area. In the next frames we search for the areas that contain the same colors. When a target is lost, we search for new targets as described before.

In the above image, the red areas are the most common colors of the target , after filtering RGB components , we calculate the center of mass of these areas .

Sample movie of succesful tracking: [movie]

1. Movement estimation based block comparison

The algorithm was ineffective due to the following reasons:

  • It is unable to determine movements of monochromatic blocks
  • Too sensitive to noise
  • High time complexity

2. Feet tracking based on recognition of diagonals in the image

This algorithm may work well under certain circumstances, but it has some disadvantages:

  • We found that there were many diagonal lines in the scenes that we examined that don't belong to persons.
  • In case of noise in the picture, the algorithm finds fault diagonals that harms the tracking.

3.Tracking based on colors recognition

This algorithm worked the best for our example movies:


  • High robustness
  • Ability to follow any moving target, not only a person, according to color characteristics.


  • If the colors of the background is close to the colors of the person, the algorithm might lose tracking
  • In case of multiple moving objects in the movie, the algorithm tracks only one of them and ignores the other in case of similar colors of the targets and it might lose tracking if their colors are alike.


MATLAB 7. Digital Camera.


The Tracking according to colors characteristics method gave the best results. It shall be able to implemented it for use in projects that envolve autonomous vehicles with onboard camera due to its reliability and its reasonable time complexity .


We would like to thank our supervisor Johanan Erez , and the Laboratory staff for the help given us. We are also grateful to the Ollendorf Minerva Center Fund for supporting this project.