Al-Rafidain Engineering Journal (AREJ)

FPGA-SoC Based Object Tracking Algorithms: A Literature Review

Document Type : Review Paper

Authors

1 Electrical Engineering Departement, College of Engineering, University of Mosul, Mosul, Iraq

2 Department of Mechatronics Engineering, College of Engineering, University of Mosul, Mosul, Iraq

Abstract

Systems for object detection and tracking are becoming increasingly important in practical applications today. Many research and development groups are interested in improving the performance of such systems, and numerous methods have been developed and proposed. Additionally, computer vision is constantly developing and implemented on reconfigurable and embedded systems. The purpose of this study is to present past and recent research works in the field of visual tracking systems that used FPGA and FPGA-SoC platforms. The study includes a brief description of several popular algorithms related to the main characteristics and in which field is preferred. Resource utilization was also considered in this study to present the most and the least resources used to implement different algorithms. The study found that flip-flops (FF) and lookup tables (LUT) are usually used, while BRAM, DSP, and multipliers had the lowest percentage utilization. Due to the recent development in the production of advanced processing systems, there is an increase focusing on employing FPGA-SoC platforms in visual surveillance systems. The reason behind that is their ability to implement complex processing using both hardware and software co-design to gain high performance in less design time compared with using only FPGA-based platforms.

Keywords

Main Subjects

References
  1. Sirpa Korhonen, “’Hardware Accelerated Visual Tracking Algorithms. A Systematic Literature Review,” Aug. 2015. [Online]. Available: https://www.researchgate.net/publication/281088238
  2. S. Gong, C. Liu, Y. Ji, B. Zhong, Y. Li, and H. Dong, Advanced Image and Video Processing Using MATLAB, vol. 12. in Modeling and Optimization in Science and Technologies, vol. 12. Cham: Springer International Publishing, 2019. doi: 10.1007/978-3-319-77223-3.
  3. Hongtu Jiang, H. Ardo, and V. Owall, “A Hardware Architecture for Real-Time Video Segmentation Utilizing Memory Reduction Techniques,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 2, pp. 226–236, Feb. 2009, doi: 10.1109/TCSVT.2008.2009244.
  4. R. Parekh, Fundamentals of image, audio, and video processing using MATLAB: with applications to pattern recognition, First edition. Boca Raton London New York: CRC Press, 2021.
  5. K. Blachut and T. Kryjak, “Real-Time Efficient FPGA Implementation of the Multi-Scale Lucas-Kanade and Horn-Schunck Optical Flow Algorithms for a 4K Video Stream,” Sensors, vol. 22, no. 13, p. 5017, Jul. 2022, doi: 10.3390/s22135017.
  6. B. M.  K.  Younis, B. Sh.  Mahmood, and F. H. Ali, “Reconfigurable Self-Organizing Neural Network Design and it’s FPGA Implementation,” (AREJ), vol. 17, no. 3, pp. 99–115, Jun. 2009, doi: 10.33899/rengj.2009.42925.
  7. R. Marzotto, P. Zoratti, D. Bagni, A. Colombari, and V. Murino, “A real-time versatile roadway path extraction and tracking on an FPGA platform,” Computer Vision and Image Understanding, vol. 114, no. 11, pp. 1164–1179, Nov. 2010, doi: 10.1016/j.cviu.2010.03.015.
  8. G. Conti, M. Quintana, P. Malagón, and D. Jiménez, “An FPGA Based Tracking Implementation for Parkinson’s Patients,” Sensors, vol. 20, no. 11, p. 3189, Jun. 2020, doi: 10.3390/s20113189.
  9. M. Amiri, F. M. Siddiqui, C. Kelly, R. Woods, K. Rafferty, and B. Bardak, “FPGA-Based Soft-Core Processors for Image Processing Applications,” J Sign Process Syst, vol. 87, no. 1, pp. 139–156, Apr. 2017, doi: 10.1007/s11265-016-1185-7.
  10. F. Kristensen, H. Hedberg, H. Jiang, P. Nilsson, and V. Öwall, “An Embedded Real-Time Surveillance System: Implementation and Evaluation,” J Sign Process Syst Sign Image Video Technol, vol. 52, no. 1, pp. 75–94, Jul. 2008, doi: 10.1007/s11265-007-0100-7.
  11. M. Tomasi, S. Pundlik, and G. Luo, “FPGA–DSP co-processing for feature tracking in smart video sensors,” J Real-Time Image Proc, vol. 11, no. 4, pp. 751–767, Apr. 2016, doi: 10.1007/s11554-014-0413-2.
  12. P. Hobden, S. Srivastava, and E. Nurellari, “FPGA-Based CNN for Real-Time UAV Tracking and Detection,” Front. Space Technol., vol. 3, p. 878010, May 2022, doi: 10.3389/frspt.2022.878010.
  13. A. Yilmaz, O. Javed, and M. Shah, “Object tracking: A survey,” ACM Comput. Surv., vol. 38, no. 4, p. 13, Dec. 2006, doi: 10.1145/1177352.1177355.
  14. Dr. F. Ali, “Transformation Matrix for 3D computer Graphics Based on FPGA(English),” (AREJ), vol. 20, no. 5, pp. 1–15, Oct. 2012, doi: 10.33899/rengj.2012.61024.
  15. S. Sajjanar, S. K. Mankani, P. R. Dongrekar, N. S. Kumar, Mohana, and H. V. Ravish Aradhya, “Implementation of real time moving object detection and tracking on FPGA for video surveillance applications,” in 2016 IEEE Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER), Mangalore: IEEE, Aug. 2016, pp. 289–295. doi: 10.1109/DISCOVER.2016.7806248.
  16. Dr. S. A. Dawwd and U. T. Salim, “Systolic Video Stream Object Detector Using FPGA-E,” (AREJ), vol. 22, no. 4, pp. 33–43, Sep. 2014, doi: 10.33899/rengj.2014.89977.
  17. A. Arif et al., “Performance and energy-efficient implementation of a smart city application on FPGAs,” J Real-Time Image Proc, vol. 17, no. 3, pp. 729–743, Jun. 2020, doi: 10.1007/s11554-018-0792-x.
  18. N. A. Mandellos, I. Keramitsoglou, and C. T. Kiranoudis, “A background subtraction algorithm for detecting and tracking vehicles,” Expert Systems with Applications, vol. 38, no. 3, pp. 1619–1631, Mar. 2011, doi: 10.1016/j.eswa.2010.07.083.
  19. R. Rodriguez-Gomez, E. J. Fernandez-Sanchez, J. Diaz, and E. Ros, “FPGA Implementation for Real-Time Background Subtraction Based on Horprasert Model,” Sensors, vol. 12, no. 1, pp. 585–611, Jan. 2012, doi: 10.3390/s120100585.
  20. D. B. K. Trieu and T. Maruyama, “Real-time color image segmentation based on mean shift algorithm using an FPGA,” J Real-Time Image Proc, vol. 10, no. 2, pp. 345–356, Jun. 2015, doi: 10.1007/s11554-012-0319-9.
  21. U. Ali and M. B. Malik, “Hardware/software co-  design of a real-time kernel based tracking system,” Journal of Systems Architecture, vol. 56, no. 8, pp. 317–326, Aug. 2010, doi: 10.1016/j.sysarc.2010.04.008.
  22. M. Genovese and E. Napoli, “FPGA-based architecture for real time segmentation and denoising of HD video,” J Real-Time Image Proc, vol. 8, no. 4, pp. 389–401, Dec. 2013, doi: 10.1007/s11554-011-0238-1.
  23. C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins, CO, USA: IEEE Comput. Soc, 1999, pp. 246–252. doi: 10.1109/CVPR.1999.784637.
  24. Artur Zawadzki and Marek Gorgon, “Automatically controlled pan–tilt smart camera with FPGA based image analysis system dedicated to real-time tracking of a moving object,” Journal of Systems Architecture, 2015.
  25. HARRIS, C and STEPHENS, “A combined corner and edge detector,” presented at the Vision Conference, 1988, pp. 147–151.
  26. I. Bravo, M. Mazo, J. L. Lázaro, A. Gardel, P. Jiménez, and D. Pizarro, “An Intelligent Architecture Based on Field Programmable Gate Arrays Designed to Detect Moving Objects by Using Principal Component Analysis,” Sensors, vol. 10, no. 10, pp. 9232–9251, Oct. 2010, doi: 10.3390/s101009232.
  27. C. Zhang, P. Li, G. Sun, Y. Guan, B. Xiao, and J. Cong, “Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks,” in Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey California USA: ACM, Feb. 2015, pp. 161–170. doi: 10.1145/2684746.2689060.
  28. L. Bai, Y. Zhao, and X. Huang, “A CNN Accelerator on FPGA Using Depthwise Separable Convolution,” IEEE Trans. Circuits Syst. II, vol. 65, no. 10, pp. 1415–1419, Oct. 2018, doi: 10.1109/TCSII.2018.2865896.
  29. Z. Wei, D.-J. Lee, and B. E. Nelson, “FPGA-based Real-time Optical Flow Algorithm Design and Implementation,” JMM, vol. 2, no. 5, pp. 38–45, Sep. 2007, doi: 10.4304/jmm.2.5.38-45.
  30. [Horn, B.K. and Schunck, “Determining optical flow,” 1981, pp. 185–203.
  31. F. Barranco, M. Tomasi, J. Diaz, M. Vanegas, and E. Ros, “Parallel Architecture for Hierarchical Optical Flow Estimation Based on FPGA,” IEEE Trans. VLSI Syst., vol. 20, no. 6, pp. 1058–1067, Jun. 2012, doi: 10.1109/TVLSI.2011.2145423.
  32. I. Ishii, T. Taniguchi, K. Yamamoto, and T. Takaki, “High-Frame-Rate Optical Flow System,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 1, pp. 105–112, Jan. 2012, doi: 10.1109/TCSVT.2011.2158340.
  33. H.-S. Seong, C. E. Rhee, and H.-J. Lee, “A Novel Hardware Architecture of the Lucas–Kanade Optical Flow for Reduced Frame Memory Access,” IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 6, pp. 1187–1199, Jun. 2016, doi: 10.1109/TCSVT.2015.2437077.
  34. C. Wang, E. D. Burnham-Fay, and J. D. Ellis, “Real-time FPGA-based Kalman filter for constant and non-constant velocity periodic error correction,” Precision Engineering, vol. 48, pp. 133–143, Apr. 2017, doi: 10.1016/j.precisioneng.2016.11.013.
  35. P. Babu and E. Parthasarathy, “FPGA implementation of multi-dimensional Kalman filter for object tracking and motion detection,” Engineering Science and Technology, an International Journal, vol. 33, p. 101084, Sep. 2022, doi: 10.1016/j.jestch.2021.101084.
  36. M. Genovese, E. Napoli, D. De Caro, N. Petra, and A. G. M. Strollo, “FPGA Implementation of Gaussian Mixture Model Algorithm for 47 fps Segmentation of 1080p Video,” Journal of Electrical and Computer Engineering, vol. 2013, pp. 1–8, 2013, doi: 10.1155/2013/129589.
  37. G. K. Gultekin and A. Saranli, “An FPGA based high performance optical flow hardware design for computer vision applications,” Microprocessors and Microsystems, vol. 37, no. 3, pp. 270–286, May 2013, doi: 10.1016/j.micpro.2013.01.001.
  38. S. Guo et al., “A system on chip-based real-time tracking system for amphibious spherical robots,” International Journal of Advanced Robotic Systems, vol. 14, no. 4, p. 172988141771655, Jul. 2017, doi: 10.1177/1729881417716559.
  39. J. G. Pandey, “An embedded FPGA-SoC framework and its usage in moving object tracking application,” Des Autom Embed Syst, vol. 25, no. 3, pp. 213–236, Sep. 2021, doi: 10.1007/s10617-021-09252-y.
  40. M. Genovese, E. Napoli, D. De Caro, N. Petra, and A. G. M. Strollo, “FPGA Implementation of Gaussian Mixture Model Algorithm for 47 fps Segmentation of 1080p Video,” Journal of Electrical and Computer Engineering, vol. 2013, pp. 1–8, 2013, doi: 10.1155/2013/129589.
Al-Rafidain Engineering Journal (AREJ)
Volume 28, Issue 2
September 2023
Page 284-295
Files
Share
Export Citation
Statistics