Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Efficient Learning Algorithms for Training Large-Scale and High-Dimensional Machine Learning Models

Bao, Runxue (2023) Efficient Learning Algorithms for Training Large-Scale and High-Dimensional Machine Learning Models. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

[img] PDF
Restricted to University of Pittsburgh users only until 19 January 2025.

Download (2MB) | Request a Copy

Abstract

Machine learning has achieved tremendous successes and played increasingly essential roles in many application scenarios in the past decades. The recent advance in machine learning relies heavily upon the emergence of big data with both massive samples and numerous features. However, the computational inefficiency and memory burden of the learning algorithms restrict the capability of machine learning for large-scale applications. Therefore, it is important to design efficient learning algorithms for big data mining. In this dissertation, we propose several newly designed efficient learning algorithms to address the challenges of high dimensionality from the aspects of both samples and features for big data mining. First, we develop an efficient approximate solution path algorithm and introduce a safe screening rule to accelerate the model training of the Ordered Weighted L1 regression. We also formulate a unified safe variable screening rule for the family of ordered weighted sparse models, which can effectively accelerate the training algorithms. Second, we develop a new accelerated doubly stochastic gradient descent method for regularized loss minimization problems. The proposed method is able to simultaneously achieve a linear convergence rate and linear rate of explicit model identification. Finally, we design a novel distributed dynamic safe screening method in parallel and distributed computing to solve sparse models and apply the method to the shared-memory and distributed-memory architecture. The method can accelerate the training process without any loss of accuracy. The contributions of the thesis are expected to speed up the training of large-scale machine learning models through smart handling of the model sparsity and data sparsity.


Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: University of Pittsburgh ETD
Status: Unpublished
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Bao, Runxuerub26@pitt.edurub26
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairMao, Zhi-Hongzhm4@pitt.edu
Committee MemberHuang, Hengheng.huang@pitt.edu
Committee MemberCan, Azimeazime.cancimino@pitt.edu
Committee MemberDallal, AhmedAHD12@pitt.edu
Committee MemberSun, Minguidrsun@pitt.edu
Committee MemberZeng, Bobzeng@pitt.edu
Date: 19 January 2023
Date Type: Publication
Defense Date: 9 November 2022
Approval Date: 19 January 2023
Submission Date: 12 November 2022
Access Restriction: 2 year -- Restrict access to University of Pittsburgh for a period of 2 years.
Number of Pages: 142
Institution: University of Pittsburgh
Schools and Programs: Swanson School of Engineering > Electrical and Computer Engineering
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: Optimization, Sparse Learning, Safe Screening, Distributed Training, Model Identification
Date Deposited: 19 Jan 2023 19:23
Last Modified: 19 Jan 2023 19:23
URI: http://d-scholarship.pitt.edu/id/eprint/43816

Metrics

Monthly Views for the past 3 years

Plum Analytics


Actions (login required)

View Item View Item