Traffic object tracking with multiple camera views
This work is used in some of these projects.
Overview:
In an increasingly automated and controlled world, in which the density of traffic particularly grows in urban areas continuously and which constantly rises associated density of relevant situation-describing information, for humans it will be more difficult to be aware of the whole situation without assistance systems.
In the dissertation "analysis of the traffic with distributed opto-electronic sensors" of Adrian Schischmanow a complete concept was presented, which makes a central collection of the entire traffic data and thus the purposeful guidance for example over control of light signal plants possible. Apart from the organization, storage and data transmission the collection of the traffic-relevant data plays a crucial role.
The collection of this data consists of seperate passes:
It will be show that Camera calibration plays an important role, especially for data fusion.
In this thesis solutions for the seperate steps will be introduced and realized.
In this step besides the determination of the interior orientation the determination of the exterior orientation is the main issue. For the application of traffic observation its important to use unambitious but effective algorithms without the need for any pre-estimations.
Therefor two methods are proposed:
- DLT - Direct Linear Transformation
- RansacLDP - Location Determination Problem
|
|
|
Figure 1: Ransac - Location Determination Problem |
In this thesis the mentioned methods and the appropriate algebraic fundamentals will be introduced and the realizations will be reviewed in the context of accuracy and therefor usability.
Object recognition in the context of this thesis concentrates on the extraction of object information on the basis of image data. Particulary, miscellaneous information of traffic objects which are pictured in video data will be derived. In this thesis different approaches will be introduced rudimental. Two of the possible algorithms are image differentiation and background estimation analysis.
|
|
|
Figure 2: Image subtraction of two pictures |
One important task is to show the problems and the limitations of the used object recognition algorithms. Thus, this has an important impact to the design and the realization of the object tracking algorithms. The data derived with the object recognition methods are time and space related object information, e.g. postition, shape, color and size of every pictured object.
Object tracking is the main task of the traffic data collecting concept. Therefor a single image processing is needed. The object recognition methods yields object related information for every single image and sensor. However, the relation between the single image data of different images is unknown. The task of the tracking algorithm is to link recognized objects for every image with each other as shown in figure 3.
|
|
|
Figure 3: Principal of the object tracking |
The aim is to descibe the whole-state of the shown scene for specific time period. The whole-state is the sum of the single states of every traffic object. In this thesis the requirements and one possible realization are shown .
|
|
|
Figure 4: Used algorithm Kalman-Filter |
|
|
|
Figure 5: Tracking results for real data |
Possible errors of the object recognition methods have to take account to when designing and realizing the tracking algorithm. Typical errors are
Bei dem Umsetzung und Gestaltung des Tracking-Algorithmus ist auf die potentiellen Fehler der Objekterkennung einzugehen. Typische Fehler sind zum Beispiel:
- Pseudo objects due to the image noise
- Segmented Objects
- Clustered Objects due to occlusion effects
- Nonrecognized objects due to occlusion effects
Often, it is not possible to picture one whole traffic area with just one sensor. Thus, different sensors are needed. On one hand it is because of the limited field of view (FOV) of the used sensors. On the other hand it is because of occlusion effects due to buildings or larger traffic objects. However, the derived object data is referenced to the whole area. In the result there may exist redundant data for one single object. Therefor it is required to fuse the gathered data of the different sensors to one object state information.
Is one object recognized by different sensors with non-overlapping FOV the data fusion has to colink this object data. This is because there is no information available that one identical object is pictured. Redundant single object data derived from overlapping FOV sensors have to be fused to one single object state information. This process is called data fusion. This fusion can be realized on different levels:
- Image data level
- Object data level
- Information level
In the related chapter the basics of data fusion are described.
|
|
|
Figure 6: Data fusion on object data level |
The introduced tracking algorithm based on the Kalman-Filter yields perfect results for perfect object data. There are some factors having negative impact on the quality of the results. Besides image noise objects with high velocity and acceleration changes and a low image capturing frequency reduces the quality of the tracking results. An accurate tracking especialy of high velocity objects requires a capturing frequency higher than 5 Hz. The implemented data fusion methods do not need a capturing sychronisation of the different sensors. In the contrary, a high asynchroniticity raises the virtual capturing frequency (see fig. 6) and thus the tracking result quality.
The evaluation of the implemented algorithms shows that the object tracking issue is doable. Perfect data yield perfect results in every case. However, in reality perfect is not available. The main problems of the object tracking is dued to the main problems of the object recognition algorithms, such as clustered or segmented objects or image noise. But, a low level reduction of these effects can achieved.