Introduction:
During my internship, I had the opportunity to work on a series of projects, and the third project focused on developing a people counter using an overhead camera. This was also a project that's done under the supervision of Assoc Prof Yuen Chau. Our objective was to develop an accurate and cost-effective solution for counting the number of people entering In this article, I will share the key aspects and achievements of this very challenging project.
Project Background and Objectives:
In our project, we address the crucial task of accurately counting the number of people entering and exiting various areas, such as smart buildings and public transport systems. This count is vital for a range of applications, from optimizing energy usage and enhancing security to gaining insights into customer behavior in settings like shopping malls.
We examined a range of methods previously employed by researchers, encompassing sensor-based approaches, WiFi signal analysis, and image processing techniques. Each of these methods has its inherent limitations, whether it be sensitivity to lighting conditions or the hardware demands associated with deep learning applications.
Identifying a research gap in this area, we propose a novel approach to people counting that works effectively under any lighting condition while maintaining excellent performance on edge computing devices. Our solution combines an overhead IR camera for people detection and tracking using extracted features. We leverage transfer learning, efficient feature extraction, and a custom object tracking algorithm to enhance the accuracy and reliability of our people counting system.
This project represents a valuable contribution to the field, offering a solution that can be applied in a wide range of settings, including smart buildings and public transport systems.
Previous works:
In the field of people counting across doorways, researchers have explored various methods over the years, including sensor-based approaches, WiFi signal analysis, and image processing techniques. However, each method has its limitations.
For instance, some early attempts involved Laser Range Finders and ultra-wideband radar sensors, but they suffered from issues like sensor placement and sensitivity to the speed of people. WiFi-based solutions faced problems with phase distortion and architectural compatibility. [1],[2],[3]
Image processing-based methods attempted background subtraction and depth cameras, but these approaches struggled with issues such as sensitivity to lighting changes, false positives caused by other objects, and the limitations of predefined 3D human models. [5]
With the emergence of deep learning, researchers began experimenting with convolutional neural networks (CNN) and spatio-temporal context tracking. While promising, these methods often suffered from slow performance, high computational costs, and issues with head rotation.
To address the need for real-time performance and higher FPS, cluster pruning was introduced, but it still faced limitations in terms of counting people crossing a door in real-time.
The practicality of implementing these methods on edge devices raised additional challenges due to lower FPS and limited capabilities under low/no light conditions.
Recognizing the existing drawbacks, this research aims to bridge the gap by introducing a model for reliable people detection and object tracking using machine vision techniques. The objective is to optimize the model for edge computing devices, ensure it operates under any lighting condition, and provide a higher FPS for more efficient people counting across doorways.
Methodology:
Due to confidentiality agreements, I cannot disclose the specific code details. However, I can provide a general overview of our approach.
People Detection and Tracking: We detect people heads by transfer learning Single Shot MultiBox Detector.[6] Also we trained it to detect distractions like bags, trolleys and chairs. We developed a custom tracking algorithm to track individuals in the video feed captured by the overhead camera. This algorithm allowed us to accurately identify and track people as they moved through the monitored area.
FPS Optimization: One of the primary focuses of our project was to achieve a high Frames Per Second (FPS) value, as it plays a crucial role in accurately capturing and tracking fast-moving individuals. We optimized our implementation to achieve an average FPS value of approximately 25 on an Intel® NUC 12 Pro Mini PC, specifically the NUC12WSHi7 model. This improvement was significant, considering that previous attempts typically achieved lower FPS values, which were below 15.
Accuracy Enhancement: While ensuring a higher FPS value was critical, we also took measures to enhance the accuracy of the people counting system. We fine-tuned our algorithm to handle challenging scenarios, such as people carrying bags, using trolleys, and walking closely together. These enhancements resulted in an overall accuracy rate of around 96%, providing reliable and precise people counting.
Results and Achievements:
The project's notable achievements lie in achieving a higher FPS value for accurate people counting and being able to work under any lighting condition. By optimizing our implementation, we successfully increased the average FPS to approximately 25, surpassing the limitations of previous attempts. This improvement was crucial, as people often move quickly within a small fraction of a second. Additionally, our system demonstrated an impressive accuracy rate of around 96%, ensuring reliable tracking and counting of individuals.
References
[1] Jae Hoon Lee et al. “Security Door System Using Human Tracking Method with Laser Range Finders”. In: International Conference on Mechatronics and Automation (Aug. 2007). doi: https://doi.org/ 10.1109/icma.2007.4303868.
[2] Jeong Woo Choi, Xuanjun Quan, and Sung Ho Cho. “Bi-Directional Passing People Counting System Based on IR-UWB Radar Sensors”. In: IEEE Internet of Things Journal 5.2 (Apr. 2018), pp. 512–522. doi: https://doi.org/10.1109/jiot.2017.2714181.
[3] Liping Tian et al. “A People-Counting and Speed-Estimation System Using Wi-Fi Signals”. In: Sensors 21.10 (May 2021), p. 3472. doi: https://doi.org/10.3390/s21103472.
[4] Yanni Yang et al. “Door-Monitor: Counting In-and-out Visitors with COTS WiFi Devices”. In: IEEE Internet of Things Journal (2019), pp. 1–1. doi: https://doi.org/10.1109/jiot.2019.2953713.
[5] Shijie Sun et al. “Benchmark Data and Method for Real-Time People Counting in Cluttered Scenes Using Depth Sensors”. In: IEEE Transactions on Intelligent Transportation Systems 20.10 (Oct. 2019), pp. 3599–3612. doi: https://doi.org/10.1109/tits.2019.2911128
[6] Wei Liu et al. “SSD: Single Shot MultiBox Detector”. In: CoRR abs/1512.02325 (2015). arXiv: 1512. 02325. url: http://arxiv.org/abs/1512.02325.
Comments