Liu, Guodong
(2023)
New Nonlinear Machine Learning Algorithms with Theoretical Analysis.
Doctoral Dissertation, University of Pittsburgh.
(Unpublished)
Abstract
Recent advances in machine learning have spawned progress in various fields. In the context of machine learning data, nonlinearity is an intrinsic property. Therefore, a nonlin- ear model will facilitate the flexibility of representation and fit the data properly. However, increasing flexibility usually means the higher complexity and less interpretability. Thus, there is a niche for designing feasible nonlinear machine learning models to handle the fore- mentioned challenges.
As a part of this work, a new method, called as sparse shrunk additive models (SSAM) is proposed. This model explores the structure information among features for high-dimensional nonparametric regression with the allowance of the flexible interactions among features. It bridges the sparse kernel regression and sparse feature selection. Theoretical results on the convergence rate and sparsity characteristics are established by the novel analysis techniques with integral operator and concentration estimate.
Most of the nonlinear models usually involve tuning multiple (up to thousands) hyper- parameters, which plays a pivotal role in model generalization. Another part of this work is a new hyperparameter optimization method with zeroth-order hyper-gradients (HOZOG). We proved the feasibility analysis of using HOZOG to achieve hyperparameter optimization under the condition of Lipschitz continuity. The extensive experiments verify the analysis.
For large-scale data, there remain computational challenges in implementing various al- gorithms. To address this issue, we propose a new regularized modal regression model with robust sampling strategy. Unlike conventional sampling for large-scale least squares, our sampling probabilities are dependent on the robust loss function for learning the conditional mode. We provide theoretical analysis to support the proposed model: the approximation bound is established by error analysis with Rademacher complexity, and the robustness characterization is provided based on the finite sample breakdown point analysis. The ex- periments are conducted on both synthetic and real-world data sets and the empirical results demonstrate the promising performance of resulting estimator.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
|
| Date: |
14 September 2023 |
| Date Type: |
Publication |
| Defense Date: |
27 June 2023 |
| Approval Date: |
14 September 2023 |
| Submission Date: |
26 July 2023 |
| Access Restriction: |
No restriction; Release the ETD for access worldwide immediately. |
| Number of Pages: |
89 |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
Swanson School of Engineering > Electrical and Computer Engineering |
| Degree: |
PhD - Doctor of Philosophy |
| Thesis Type: |
Doctoral Dissertation |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
nonlinear models, machine learning, additive models, zeroth order optimization, modal regression. |
| Date Deposited: |
14 Sep 2023 13:44 |
| Last Modified: |
14 Sep 2023 13:44 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/45151 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}