Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Optimization and Hardware Acceleration of Event-Based Optical Flow for Real-Time Processing and Compression on Embedded Platforms

Stumpp, Daniel Charles (2022) Optimization and Hardware Acceleration of Event-Based Optical Flow for Real-Time Processing and Compression on Embedded Platforms. Master's Thesis, University of Pittsburgh. (Unpublished)

This is the latest version of this item.

[img]
Preview
PDF
Accepted Version

Download (3MB) | Preview

Abstract

Event-based vision sensors produce asynchronous event streams with high temporal resolution based on changes in the visual scene. The properties of these sensors allow for accurate and fast calculation of optical flow as events are generated. Existing solutions for calculating optical flow from event data either fail to capture the true direction of motion due to the aperture problem, do not use the high temporal resolution of the sensor, or are too computationally expensive to be run in real time on low-power embedded platforms. In this research, software optimization of the existing ARMS (Aperture Robust Multi-Scale flow) algorithm is performed. The new optimized software version (fARMS) significantly improves throughput on a traditional CPU. Further, we present hARMS, a hardware realization of the fARMS algorithm allowing for real-time computation of true flow on low-power, embedded platforms. The proposed hARMS architecture targets hybrid system-on-chip (SoC) devices and was designed to maximize configurability and throughput. The hardware architecture and fARMS algorithm were developed with asynchronous neuromorphic processing in mind, abandoning the common use of an event frame and instead operating using only a small history of relevant events, allowing latency to scale independently of the sensor resolution. This change in processing paradigm improved the estimation of flow directions by up to 73\% compared to the original ARMS algorithm and yielded a demonstrated hARMS throughput of up to 1.21 Mevent/s on the benchmark configuration selected. This throughput enables real-time performance and makes it the fastest known realization of aperture-robust, event-based optical flow to date. Finally, the fast event-based optical flow enabled by the fARMS and hARMS designs is leveraged for use in a novel flow-based event-stream compression algorithm. This compression method enables communication across low-bandwidth mediums, while allowing for accurate reconstruction of the event stream.


Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: University of Pittsburgh ETD
Status: Unpublished
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Stumpp, Daniel Charlesdcs98@pitt.edudcs980000-0002-1319-3871
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Thesis AdvisorGeorge, Alanalan.george@pitt.eduadg910000-0001-9665-2879
Committee MemberBenosman, Ryadbenosman@pitt.edu0000-0003-0243-944X
Committee MemberKubendran, Rajkumarrajkumar.ece@pitt.edurak1960000-0003-3066-4898
Date: 10 June 2022
Date Type: Publication
Defense Date: 7 April 2022
Approval Date: 10 June 2022
Submission Date: 15 March 2022
Access Restriction: 1 year -- Restrict access to University of Pittsburgh for a period of 1 year.
Number of Pages: 74
Institution: University of Pittsburgh
Schools and Programs: Swanson School of Engineering > Electrical and Computer Engineering
Degree: MS - Master of Science
Thesis Type: Master's Thesis
Refereed: Yes
Uncontrolled Keywords: Event-based, aperture robust, optical flow, neuromorphic computing, field programmable gate arrays, system-on-chip, parallel acceleration, real-time systems, compression
Date Deposited: 10 Jun 2022 19:06
Last Modified: 10 Jun 2023 05:15
URI: http://d-scholarship.pitt.edu/id/eprint/42396

Available Versions of this Item

  • Optimization and Hardware Acceleration of Event-Based Optical Flow for Real-Time Processing and Compression on Embedded Platforms. (deposited 10 Jun 2022 19:06) [Currently Displayed]

Metrics

Monthly Views for the past 3 years

Plum Analytics


Actions (login required)

View Item View Item