Efficient Learning Algorithms for Training Large-Scale and High-Dimensional Machine Learning Models

Bao, Runxue (2023) Efficient Learning Algorithms for Training Large-Scale and High-Dimensional Machine Learning Models. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

PDF Restricted to University of Pittsburgh users only until 19 January 2025. Download (2MB) | Request a Copy

Abstract

Machine learning has achieved tremendous successes and played increasingly essential roles in many application scenarios in the past decades. The recent advance in machine learning relies heavily upon the emergence of big data with both massive samples and numerous features. However, the computational inefficiency and memory burden of the learning algorithms restrict the capability of machine learning for large-scale applications. Therefore, it is important to design efficient learning algorithms for big data mining. In this dissertation, we propose several newly designed efficient learning algorithms to address the challenges of high dimensionality from the aspects of both samples and features for big data mining. First, we develop an efficient approximate solution path algorithm and introduce a safe screening rule to accelerate the model training of the Ordered Weighted L1 regression. We also formulate a unified safe variable screening rule for the family of ordered weighted sparse models, which can effectively accelerate the training algorithms. Second, we develop a new accelerated doubly stochastic gradient descent method for regularized loss minimization problems. The proposed method is able to simultaneously achieve a linear convergence rate and linear rate of explicit model identification. Finally, we design a novel distributed dynamic safe screening method in parallel and distributed computing to solve sparse models and apply the method to the shared-memory and distributed-memory architecture. The method can accelerate the training process without any loss of accuracy. The contributions of the thesis are expected to speed up the training of large-scale machine learning models through smart handling of the model sparsity and data sparsity.

---

Share

Citation/Export:	Select format... Citation - Text Citation - HTML Endnote BibTex Dublin Core OpenURL MARC (ISO 2709) METS MODS EP3 XML Reference Manager Refer
Social Networking:	Share |

---

Details

Item Type:	University of Pittsburgh ETD
Status:	Unpublished
Creators/Authors:	Creators Email Pitt Username ORCID Bao, Runxue rub26@pitt.edu rub26
ETD Committee:	Title Member Email Address Pitt Username ORCID Committee Chair Mao, Zhi-Hong zhm4@pitt.edu Committee Member Huang, Heng heng.huang@pitt.edu Committee Member Can, Azime azime.cancimino@pitt.edu Committee Member Dallal, Ahmed AHD12@pitt.edu Committee Member Sun, Mingui drsun@pitt.edu Committee Member Zeng, Bo bzeng@pitt.edu
Date:	19 January 2023
Date Type:	Publication
Defense Date:	9 November 2022
Approval Date:	19 January 2023
Submission Date:	12 November 2022
Access Restriction:	2 year -- Restrict access to University of Pittsburgh for a period of 2 years.
Number of Pages:	142
Institution:	University of Pittsburgh
Schools and Programs:	Swanson School of Engineering > Electrical and Computer Engineering
Degree:	PhD - Doctor of Philosophy
Thesis Type:	Doctoral Dissertation
Refereed:	Yes
Uncontrolled Keywords:	Optimization, Sparse Learning, Safe Screening, Distributed Training, Model Identification
Date Deposited:	19 Jan 2023 19:23
Last Modified:	19 Jan 2023 19:23
URI:	http://d-scholarship.pitt.edu/id/eprint/43816

---

Metrics

Monthly Views for the past 3 years

Plum Analytics

---

Actions (login required)

View Item