Traffic Sign Detection and Recognition using Features Combination and Random Forests

Introduction

Advanced Driver Assistance Systems (ADAS) are among the most rapidly growing fields in automotive electronics. These systems are designed to enhance driving safety by providing real-time data on road conditions, obstacles, and traffic signs. A key component of ADAS is Traffic Sign Detection and Recognition (TSDR), which aims to identify and classify road signs to assist drivers in making informed decisions. Road signs guide, warn, and regulate traffic. However, in real-world conditions, recognizing traffic signs can be challenging due to environmental factors such as low light, occlusion, or motion blur.

This paper proposes a fast and robust system for TSDR that works in three stages: image segmentation, shape detection, and sign recognition. The system is designed to handle common challenges such as varying lighting conditions and noise in urban traffic scenes. The authors present a comparison of four feature descriptors—HOG (Histogram of Oriented Gradients), Gabor filters, LBP (Local Binary Pattern), and LSS (Local Self-Similarity)—along with a Random Forest classifier, which is shown to outperform Support Vector Machines (SVM) in this application.

Previous Work and Challenges

The challenge of recognizing traffic signs is not new. Over the years, many approaches have been proposed to solve the problem, and these can be divided into two broad categories: color-based detection and shape-based detection.

• Color-based Detection: Road signs have specific colors like red, blue, and white, making color an obvious feature for detection. However, color-based methods are often affected by lighting conditions. To mitigate this, various color spaces, such as HSV (Hue, Saturation, and Value), YUV, and HSI, have been explored. The paper highlights the limitations of RGB, which is highly sensitive to illumination changes, and emphasizes HSV as the preferred space for segmentation in their proposed method.
• Shape-based Detection: Traffic signs have standard shapes, such as circles, triangles, and rectangles. Methods like Haar-like features, Radial Symmetry, and Generalized Hough Transform have been used to detect these shapes. In this paper, the authors employ a feature called Distance to Borders (DtBs), which measures the distance from the shape’s external edge to its bounding box. This feature is used as an input to the Random Forest classifier.

However, while color and shape provide valuable information, real-world applications encounter challenges such as motion blur, occlusion, deformation of signs, and changing lighting conditions. These factors must be addressed to develop a robust system.

Proposed Methodology

The proposed system consists of three key stages: image segmentation, shape classification, and traffic sign recognition.

1. Image Segmentation

The first step in detecting traffic signs involves segmenting the input image to extract Regions of Interest (ROIs). Given the variation in lighting and the presence of objects with similar colors to traffic signs, the authors use the HSV color space for segmentation, as it is less sensitive to changes in illumination than RGB.

• Enhancement: The authors enhance the image in the HSV space by applying luminance and contrast enhancements to the Value (V) component. This process improves the quality of the input image, making it easier to segment signs.
• Thresholding: After enhancement, fixed thresholds are applied to segment red and blue regions, which are typical colors for traffic signs. These thresholds were determined empirically based on traffic sign images. The segmentation result is a binary image where regions of interest (ROIs) are highlighted.

2. Shape Classification

After segmentation, the next step is to classify the detected ROIs based on their shapes. The proposed system is capable of recognizing circular, triangular, and rectangular shapes, which are standard shapes for traffic signs.

• Distance to Borders (DtBs): This feature represents the distance from the external edge of the shape to its bounding box, making it effective for shape classification. DtBs are computed for each ROI, and these feature vectors are used as inputs to a Random Forest classifier.
• Random Forest Classifier: The Random Forest classifier is an ensemble of decision trees where each tree votes for the most frequent class. Random Forests have shown to be robust against overfitting and noise. Moreover, they offer higher accuracy and better performance compared to single classifiers. In this system, the classifier is trained to distinguish between circular, triangular, and rectangular signs.

3. Traffic Sign Recognition

Once the ROIs are classified based on shape, the final step is to recognize the specific traffic signs. This involves extracting relevant features from the detected signs and feeding them to a classifier for final recognition.

The authors compare four feature descriptors:

• HOG (Histogram of Oriented Gradients): This descriptor captures the distribution of intensity gradients or edge directions, making it useful for detecting objects with distinct edges like traffic signs.
• Gabor Filters: Gabor filters are used to extract local spectral characteristics from images, which are useful in pattern recognition problems.
• LBP (Local Binary Pattern): LBP is a texture descriptor that compares each pixel with its neighbors to generate a binary pattern, which is then used to describe texture features.
• LSS (Local Self-Similarity): LSS measures self-similarities within local patches of an image and is useful for recognizing repetitive patterns in traffic signs.

The authors experimented with various combinations of these descriptors. The combination of HOG and LSS, along with the Random Forest classifier, yielded the best results.

Experimental Results

The proposed system was tested on the Swedish Traffic Signs Data set (STSD), which contains more than 20,000 images and 3,488 labeled traffic signs. The evaluation was performed using standard precision-recall metrics and analyzing the Correct Classification Rate (CCR).

Detection Results

The detection stage was evaluated using a Precision-Recall curve, where recall measures the fraction of true signs detected, and precision measures the fraction of detected signs that are correct. The system achieved a recall of 93.41% and a precision of 95.12%, resulting in an Area Under Curve (AUC) of 94.50%.

• Recall: recall=Number of correctly detected signsNumber of true signs×100\text{recall} = \frac{\text{Number of correctly detected signs}}{\text{Number of true signs}} \times 100recall=Number of true signsNumber of correctly detected signs​×100
• Precision: precision=Number of correctly detected signsNumber of detected signs×100\text{precision} = \frac{\text{Number of correctly detected signs}}{\text{Number of detected signs}} \times 100precision=Number of detected signsNumber of correctly detected signs​×100

Recognition Results

For the recognition stage, the authors compared the performance of the feature descriptors and classifiers in terms of CCR and execution time. The results are summarized in Table III.

• The combination of HOG and LSS features gave the highest Correct Classification Rate (CCR) of 96.13% with Random Forest, which is significantly higher than the CCR of individual feature descriptors.
• In terms of execution time, the Random Forest classifier was faster than SVMs, making it more suitable for real-time applications.

Qualitative Examples

The authors provided examples of successful and failed recognition cases. In cases of successful recognition, the system accurately detected and classified the signs. However, in some cases, the system failed to detect or misclassified signs due to factors like motion blur or occlusion.

Conclusion

The paper presents a robust and efficient system for Traffic Sign Detection and Recognition (TSDR). By leveraging a combination of HOG and LSS features and using the Random Forest classifier, the proposed system achieves a CCR of over 96%. The system is resilient to challenges like lighting conditions, occlusion, and shape deformations, making it suitable for real-world applications.

Future work could focus on improving the segmentation stage by using adaptive thresholds and incorporating temporal information to track traffic signs over time, thereby enhancing the detection process.