Sample Complexity Analysis and Self-regularization in Identification of Over-parameterized ARX Models
dc.contributor.author | Du, Zhe | |
dc.contributor.author | Liu, Zexiang | |
dc.contributor.author | Weitze, Jack | |
dc.contributor.author | Ozay, Necmiye | |
dc.date.accessioned | 2022-08-31T02:52:05Z | |
dc.date.available | 2022-08-31T02:52:05Z | |
dc.date.issued | 2022-08-30 | |
dc.identifier.uri | https://hdl.handle.net/2027.42/174145 | en |
dc.description.abstract | AutoRegressive eXogenous (ARX) models form one of the most important model classes in control theory, econometrics, and statistics, but they are yet to be understood in terms of their finite sample identification analysis. The technical challenges come from the strong statistical dependency not only between data samples at different time instances but also between elements within each individual sample. In this work, for ARX models with potentially unknown orders, we study how ordinary least squares (OLS) estimator performs in terms of identifying model parameters from data collected from either a single length-$T$ trajectory or $N$ i.i.d. trajectories. Our main results show that as long as the orders of the model are chosen optimistically, i.e., we are learning an over-parameterized model compared to the ground truth ARX, the OLS will converge with the optimal rate $\Ocal(1/\sqrt{T})$ (or $\Ocal(1/\sqrt{N})$) to the true (low-order) ARX parameters. This occurs without the aid of any regularization, thus is referred to as \emph{self-regularization}. Our results imply that the oracle knowledge of the true orders and usage of regularizers are not necessary in learning ARX models --- over-parameterization is all you need. | en_US |
dc.description.sponsorship | ONR grants N00014-18-1-2501 | en_US |
dc.description.sponsorship | ONR grants N00014-21-1-2431 | en_US |
dc.description.sponsorship | NSF grants EECS-1553873 | en_US |
dc.description.sponsorship | NSF grants CNS-1931982 | en_US |
dc.description.sponsorship | AFOSR YIP award FA9550-19-1-0026 | en_US |
dc.description.sponsorship | NSF CAREER award CCF-1845076 | en_US |
dc.language.iso | en_US | en_US |
dc.rights | Attribution 4.0 International | * |
dc.rights.uri | http://creativecommons.org/licenses/by/4.0/ | * |
dc.subject | System identification | en_US |
dc.subject | ARX models | en_US |
dc.subject | Ordinary least squares | en_US |
dc.title | Sample Complexity Analysis and Self-regularization in Identification of Over-parameterized ARX Models | en_US |
dc.type | Conference Paper | en_US |
dc.subject.hlbsecondlevel | Computer Science | |
dc.subject.hlbsecondlevel | Electrical Engineering | |
dc.subject.hlbtoplevel | Engineering | |
dc.description.peerreviewed | Peer Reviewed | en_US |
dc.contributor.affiliationum | Department of Electrical Engineering and Computer Science | en_US |
dc.contributor.affiliationumcampus | Ann Arbor | en_US |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/174145/1/CDC2022_ARXID_Selfregularization.pdf | |
dc.identifier.doi | https://dx.doi.org/10.7302/5876 | |
dc.identifier.orcid | 0000-0002-8245-9215 | en_US |
dc.identifier.orcid | 0000-0001-8020-1619 | en_US |
dc.identifier.orcid | 0000-0002-5627-5113 | en_US |
dc.identifier.orcid | 0000-0002-5552-4392 | en_US |
dc.description.depositor | SELF | en_US |
dc.identifier.name-orcid | Du, Zhe; 0000-0002-8245-9215 | en_US |
dc.identifier.name-orcid | Liu, Zexiang; 0000-0001-8020-1619 | en_US |
dc.identifier.name-orcid | Weitze, Jack; 0000-0002-5627-5113 | en_US |
dc.identifier.name-orcid | Ozay, Necmiye; 0000-0002-5552-4392 | en_US |
dc.working.doi | 10.7302/5876 | en_US |
dc.owningcollname | Electrical Engineering and Computer Science, Department of (EECS) |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.