Smart cars are on the radar: Automatic object recognition and tracking using lidar

The automobile industry in China has been developing extremely rapidly over the past few decades, and more people have access to cars now than ever before. Because of this, many cities are facing traffic issues, such as traffic congestion and frequent accidents. One way to alleviate these problems, aside from improving public transportation, is to develop and implement intelligent transportation systems (ITS).

Smart cars are an essential component of ITSs. The idea is to enable cars to sense their surroundings while simultaneously communicating with the ITS. This combination can unlock a set of useful functions, such as automatic obstacle detection, dynamic cruise control, and self-driving. In turn, these capabilities can help prevent accidents and alleviate traffic congestion substantially.

However, modern smart cars struggle to make sense of their surroundings accurately for various reasons. First, they mostly rely on mounted cameras, which have very limited range and perform poorly in less-than-ideal conditions, such as in low light or when it is raining. Second, identifying vehicles, pedestrians, and other obstacles in images is computationally intensive, requiring a lot of time and processing power. Finally, most image processing algorithms tested in smart cars today can misclassify pedestrians, other vehicles, and static objects like trees. 

Fortunately, Xiangtian Zheng of Yunnan University, China, has been working on a different approach to sensing and image processing in smart vehicles. As described in his paper in the Journal of Electronic Imaging, Zheng decided to replace the usual cameras used for vehicle detection and tracking with a laser radar, or lidar. Through a carefully crafted scheme for signal acquisition and processing, his methodology makes smart cars better at identifying other vehicles and “understanding” their movements.

But how does lidar work and what are its benefits? “Put simply, the working principle of lidar involves irradiating a target with a laser and detecting the radiation that is echoed back through the processes of reflection, refraction, scattering, and transmission,” said Zheng. “Most importantly, by measuring the time it takes for the laser to return to the emitter after encountering an obstacle, we can accurately calculate the distance between the object and the device within a resolution of mere centimeters.”

Besides its accuracy and longer range, lidar data can be much simpler to process than images. In this study, Zheng made an innovative use of a grid map. The idea is to first make a spatial map of all the distances measured by the lidar. The premise is that consecutive points on this map, whose difference in distance is small, most likely correspond to the same object. This enables the algorithm to divide the map into regions that contain each individual object in the field of view. Through the extraction of other features from the lidar data and by combining certain filtering and classification techniques, Zheng’s model is capable of telling vehicles apart from pedestrians and static objects. It can also estimate the trajectories of the objects it perceives, which is a crucial feature for autonomous driving.

Zheng’s method achieved higher recognition rates than the traditional method. Credit: X. Zheng, doi: 10.1117/1.JEI.32.1.011203

Zheng put his model to the test by running a series of experiments. He compared the performance of his novel lidar approach based on a grid map versus a more conventional approach that also uses lidar. The experiments gauged the capacity of both systems to accurately identify multiple objects at different distances, reliably detect pedestrians, and track multiple moving vehicles simultaneously. “Compared with more conventional lidar-based methods, the approach proposed in this work achieved higher recognition rates for vehicles and pedestrians, while also ensuring the accuracy of the estimated tracking trajectories,” said Zheng. Moreover, the proposed system was significantly faster at recognizing objects, which is relevant for obvious safety reasons.

Overall, the results of this study highlight the capabilities of lidar for smart vehicles and point towards promising research directions. ITSs and smarter cars help us solve the current problem of traffic congestion in China and elsewhere, making transportation safer and more pleasant.

Read the article by Xianian Zheng, “Laser radar-based intelligent vehicle target recognition and detection system using image detection technology,” J. of Electronic Imaging, 32(1), 011203 (2022), doi 10.1117/1.JEI.32.1.011203.