Heterogeneous Model to Heterogeneous System Mapping with Computation and Communication Awareness

Zhang, Xinyi (2022) Heterogeneous Model to Heterogeneous System Mapping with Computation and Communication Awareness. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Preview

PDF Download (7MB) | Preview

Abstract

While machine learning (ML) has been widely used in real-life applications, the complex nature of real-world problems calls for heterogeneity in both machine learning models and hardware systems. For the algorithm, the heterogeneity in ML models comes from the multi-sensor perceiving and multi-task learning, i.e., multi-modality multi-task (MMMT) models, resulting in diverse Deep Neural Networks (DNNs) with associated DNN layers. For the system, as the diverse DNN layers largely increase the heterogeneity of computing and dataflow patterns, heterogeneous computing becomes a promising solution to address the computation efficiency. it becomes prevailing to integrate dedicated acceleration components such as CPU, GPU, ASIC, and FPGA accelerators into one system to improve overall efficiency. It thus introduces a new problem, heterogeneous model to heterogeneous system mapping (H2H), in which both computation and communication efficiency need to be considered.

this dissertation proposes three aspects to enable an efficient heterogeneous model to heterogeneous system mapping. First, a Convolution accelerator design exploration based on FPGA is proposed to address the efficiency in CNN models. Second, the accelerator architecture exploration for LSTM and Transformer are proposed based on FPGA to address the computing efficiency for LSTM and Transformer based models. Third, aiming at system-level formulation, simulation, and optimization, a computation and communication aware heterogeneous model to heterogeneous system mapping algorithm with its associated system-level simulator is proposed. In solving the H2H system mapping, we take the multi-FPGA system as the multi-accelerator platform due to its flexibility in accelerator architecture exploration and fast prototyping procedure.

---

Share

Citation/Export:	Select format... Citation - Text Citation - HTML Endnote BibTex Dublin Core OpenURL MARC (ISO 2709) METS MODS EP3 XML Reference Manager Refer
Social Networking:	Share |

---

Details

Item Type:	University of Pittsburgh ETD
Status:	Unpublished
Creators/Authors:	Creators Email Pitt Username ORCID Zhang, Xinyi xiz173@pitt.edu xiz173 0000-0002-9307-1654
ETD Committee:	Title Member Email Address Pitt Username ORCID Committee Chair Hu, Jingtong jthu@pitt.edu Committee Member Jones, Alex akjonespitt@gmail.com Committee Member Dickerson, Samuel dickerson@pitt.edu Committee Member In, In Hee inhee.lee@pitt.edu Zhang, Youtao zhangyt@cs.pitt.edu Hao, Callie callie.hao@gatech.edu
Date:	10 June 2022
Date Type:	Publication
Defense Date:	28 February 2022
Approval Date:	10 June 2022
Submission Date:	19 February 2022
Access Restriction:	1 year -- Restrict access to University of Pittsburgh for a period of 1 year.
Number of Pages:	121
Institution:	University of Pittsburgh
Schools and Programs:	Swanson School of Engineering > Electrical and Computer Engineering
Degree:	PhD - Doctor of Philosophy
Thesis Type:	Doctoral Dissertation
Refereed:	Yes
Uncontrolled Keywords:	Heterogeneous Computation Communication Machine Learning DNN accelerator FPGA locality
Date Deposited:	10 Jun 2022 18:12
Last Modified:	10 Jun 2023 05:15
URI:	http://d-scholarship.pitt.edu/id/eprint/42359

Available Versions of this Item

• Heterogeneous Model to Heterogeneous System Mapping with Computation and Communication Awareness. (deposited 10 Jun 2022 18:12) [Currently Displayed]

---

Metrics

Monthly Views for the past 3 years

Plum Analytics

---

Actions (login required)

View Item