Classification via Multiple Hyperplanes: Loss functions, Overparametrization, and Interpolation
dc.contributor.author | Wang, Yutong | |
dc.date.accessioned | 2022-09-06T16:06:03Z | |
dc.date.available | 2022-09-06T16:06:03Z | |
dc.date.issued | 2022 | |
dc.date.submitted | 2022 | |
dc.identifier.uri | https://hdl.handle.net/2027.42/174328 | |
dc.description.abstract | Many well-established classification algorithms such as support vector machines (SVM) are originally proposed as large-margin classifiers from a single hyperplane. This dissertation is divided into two halves, each half studying classification from the perspective of using multiple hyperplanes. The first half introduces a new framework for multiclass loss functions called the permutation-equivariant and relative margin-based (PERM) losses, inspired by multiclass classification with multiple hyperplanes. Using our framework, we establish statistical and optimization results on Weston-Watkins multiclass SVMs. Furthermore, we provide sufficient conditions for the classification-calibration of a general family of PERM losses. These sufficient conditions subsume all previously known and establish new classification-calibration results. The second half focuses on hyperplane arrangement classifiers (HACs). When implemented as neural networks, we show that the HACs can be overparameterized yet still have small VC dimensions and further achieve minimax optimality (assuming the empirical risk minimization can be solved to optimality). By using an ensemble of randomly initialized HACs, we demonstrate for the first time an interpolating ensemble method that is consistent for a broad class of distributions in arbitrary dimensions. We discuss the significance of these results in the context of recent advances in the theory of overparameterized learning. | |
dc.language.iso | en_US | |
dc.subject | machine learning | |
dc.subject | statistical learning theory | |
dc.subject | support vector machines | |
dc.subject | neural networks | |
dc.subject | ensemble methods | |
dc.title | Classification via Multiple Hyperplanes: Loss functions, Overparametrization, and Interpolation | |
dc.type | Thesis | |
dc.description.thesisdegreename | PhD | en_US |
dc.description.thesisdegreediscipline | Electrical and Computer Engineering | |
dc.description.thesisdegreegrantor | University of Michigan, Horace H. Rackham School of Graduate Studies | |
dc.contributor.committeemember | Scott, Clayton D | |
dc.contributor.committeemember | Tewari, Ambuj | |
dc.contributor.committeemember | Balzano, Laura | |
dc.contributor.committeemember | Qu, Qing | |
dc.subject.hlbsecondlevel | Electrical Engineering | |
dc.subject.hlbtoplevel | Engineering | |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/174328/1/yutongw_1.pdf | |
dc.identifier.doi | https://dx.doi.org/10.7302/6059 | |
dc.identifier.orcid | 0000-0001-7472-6750 | |
dc.identifier.name-orcid | Wang, Yutong; 0000-0001-7472-6750 | en_US |
dc.working.doi | 10.7302/6059 | en |
dc.owningcollname | Dissertations and Theses (Ph.D. and Master's) |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.