Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Distributed Machine Learning Framework: New Algorithms and Theoretical Foundation

Huo, Zhouyuan (2020) Distributed Machine Learning Framework: New Algorithms and Theoretical Foundation. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Download (2MB) | Preview


Machine learning is gaining fresh momentum, and has helped us to enhance not only many industrial and professional processes but also our everyday living. The recent success of machine learning relies heavily on the surge of big data, big models, and big computing. However, inefficient algorithms restrict the applications of machine learning to big data mining tasks. In terms of big data, serious concerns, such as communication overhead and data privacy, should be rigorously addressed when we train models using large amounts of data located on multiple devices. In terms of the big model, it is still an underexplored research area if a model is too big to train on a single device. To address these challenging problems, this thesis is focusing on designing new large-scale machine learning models, efficiently optimizing and training methods for big data mining, and studying new discoveries in both theory and applications.

For the challenges raised by big data, we proposed several new asynchronous distributed stochastic gradient descent or coordinate descent methods for efficiently solving convex and non-convex problems. We also designed new large-batch training methods for deep learning models to reduce the computation time significantly with better generalization performance. For the challenges raised by the big model, We scaled up the deep learning models by parallelizing the layer-wise computations with a theoretical guarantee, which is the first algorithm breaking the lock of backpropagation such that the large model can be dramatically accelerated.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Huo, Zhouyuanzhh39@pitt.eduzhh39
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairHuang,
Committee MemberMao,
Committee MemberGao,
Committee CoChairZhan,
Committee MemberChen,
Date: 31 July 2020
Date Type: Publication
Defense Date: 1 April 2020
Approval Date: 31 July 2020
Submission Date: 17 March 2020
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Number of Pages: 141
Institution: University of Pittsburgh
Schools and Programs: Swanson School of Engineering > Electrical and Computer Engineering
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: Distributed Machine Learning
Date Deposited: 31 Jul 2020 13:58
Last Modified: 31 Jul 2020 13:58


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item