Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

New Nonlinear Machine Learning Algorithms with Theoretical Analysis

Liu, Guodong (2023) New Nonlinear Machine Learning Algorithms with Theoretical Analysis. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Download (969kB) | Preview


Recent advances in machine learning have spawned progress in various fields. In the context of machine learning data, nonlinearity is an intrinsic property. Therefore, a nonlin- ear model will facilitate the flexibility of representation and fit the data properly. However, increasing flexibility usually means the higher complexity and less interpretability. Thus, there is a niche for designing feasible nonlinear machine learning models to handle the fore- mentioned challenges.
As a part of this work, a new method, called as sparse shrunk additive models (SSAM) is proposed. This model explores the structure information among features for high-dimensional nonparametric regression with the allowance of the flexible interactions among features. It bridges the sparse kernel regression and sparse feature selection. Theoretical results on the convergence rate and sparsity characteristics are established by the novel analysis techniques with integral operator and concentration estimate.
Most of the nonlinear models usually involve tuning multiple (up to thousands) hyper- parameters, which plays a pivotal role in model generalization. Another part of this work is a new hyperparameter optimization method with zeroth-order hyper-gradients (HOZOG). We proved the feasibility analysis of using HOZOG to achieve hyperparameter optimization under the condition of Lipschitz continuity. The extensive experiments verify the analysis.
For large-scale data, there remain computational challenges in implementing various al- gorithms. To address this issue, we propose a new regularized modal regression model with robust sampling strategy. Unlike conventional sampling for large-scale least squares, our sampling probabilities are dependent on the robust loss function for learning the conditional mode. We provide theoretical analysis to support the proposed model: the approximation bound is established by error analysis with Rademacher complexity, and the robustness characterization is provided based on the finite sample breakdown point analysis. The ex- periments are conducted on both synthetic and real-world data sets and the empirical results demonstrate the promising performance of resulting estimator.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Liu, Guodongmealsd@gmail.comgul250000-0001-9539-6439
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairHuang,
Committee MemberZhan,
Committee MemberMao,
Committee MemberGao,
Committee MemberChen,
Date: 14 September 2023
Date Type: Publication
Defense Date: 27 June 2023
Approval Date: 14 September 2023
Submission Date: 26 July 2023
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Number of Pages: 89
Institution: University of Pittsburgh
Schools and Programs: Swanson School of Engineering > Electrical and Computer Engineering
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: nonlinear models, machine learning, additive models, zeroth order optimization, modal regression.
Date Deposited: 14 Sep 2023 13:44
Last Modified: 14 Sep 2023 13:44


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item