Silbernagel, Linus M
(2024)
Flexible FPGA Acceleration Architecture for Real-Time Neuromorphic Optical Flow.
Master's Thesis, University of Pittsburgh.
(Unpublished)
Abstract
Event-based vision sensors have become popular due to their high temporal resolution, low power consumption, and high dynamic range. These properties make the sensors more attractive compared to traditional cameras for many computer-vision tasks that require low latency in resource-constrained environments. An important aspect in many computer-vision tasks is optical flow, which is the estimation of an object's velocity. Traditional approaches use frame-by-frame displacement of an object for estimating optical flow. However, with event-based sensors, the paradigm has shifted to using pixel-wise event information. The implementation of these algorithms needs to be able to achieve high performance to keep up with the low-latency event streams from the sensors. Hardware such as field-programmable gate arrays (FPGAs) are commonly used as accelerator platforms because they allow for reconfigurable hardware to tailor the algorithmic implementation. Existing solutions for accelerating event-based optical flow are either too computationally expensive to run on embedded platforms or sacrifice precision in favor of enhanced speed. This research presents an FPGA-based acceleration architecture of a plane-fitting algorithm using a Savitzky-Golay filter for event-based optical-flow calculation implemented on an FPGA. This architecture uses a small history of recent events, unlike traditional methods that store a sensor frame. This optimization enables the architecture to be expanded to higher-resolution sensors without the limitation of requiring enough on-chip memory to store the entire frame. The developed architecture has multiple parameters that can be configured to allow for a tailored FPGA implementation. The architecture was tested on two datasets, Bar-Square and Shapes-Rotation. Our experimental results show that this design reaches real-time performance on an embedded platform. The architecture can achieve a throughput of 506.12~Kevts/s with an event-by-event latency of 1.98~$\mu$s. The architecture is agnostic to input sensor resolution and can be configured to optimize for accuracy or throughput performance.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
|
| Date: |
6 September 2024 |
| Date Type: |
Publication |
| Defense Date: |
19 June 2024 |
| Approval Date: |
6 September 2024 |
| Submission Date: |
20 June 2024 |
| Access Restriction: |
No restriction; Release the ETD for access worldwide immediately. |
| Number of Pages: |
47 |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
Swanson School of Engineering > Electrical and Computer Engineering |
| Degree: |
MS - Master of Science |
| Thesis Type: |
Master's Thesis |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
FPGA, high-level synthesis (HLS), Xilinx, Vitis, event-based sensing, event-based optical flow, neuromorphic sensors, hardware accelerator |
| Date Deposited: |
06 Sep 2024 19:56 |
| Last Modified: |
06 Sep 2024 19:56 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/46596 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}