期刊名称：Sankhya. Series A, mathematical statistics and probability
印刷版ISSN：0976-836X
电子版ISSN：0976-8378
出版年度：2017
卷号：79
期号：2
页码：159-200
DOI：10.1007/s13171-017-0111-9
语种：English
出版社：Indian Statistical Institute
摘要：Rates of convergence for empirical risk minimizers have been well studied in the literature. In this paper, we aim to provide a complementary set of results, in particular by showing that after normalization, the risk of the empirical minimizer concentrates on a single point. Such results have been established by Chatterjee ( The Annals of Statistics , 42(6):2340–2381 2014 ) for constrained estimators in the normal sequence model. We first generalize and sharpen this result to regularized least squares with convex penalties, making use of a “direct” argument based on Borell’s theorem. We then study generalizations to other loss functions, including the negative log-likelihood for exponential families combined with a strictly convex regularization penalty. The results in this general setting are based on more “indirect” arguments as well as on concentration inequalities for maxima of empirical processes.
关键词：Concentration ; Density estimation ; Empirical process ; Empirical risk minimization ; Normal sequence model ; Penalized least squares
Loading...
