Maximum Penalized Likelihood Estimation: Volume II: Regression

This is the second volume of a text on the theory and practice of maximum penalized likelihood estimation. It is intended for graduate students in s- tistics, operationsresearch, andappliedmathematics, aswellasresearchers and practitioners in the ?eld. The present volume was supposed to have a short chapter on nonparametric regression but was intended to deal mainly with inverse problems. However, the chapter on nonparametric regression kept growing to the point where it is now the only topic covered. Perhaps there will be a Volume III. It might even deal with inverse problems. But for now we are happy to have ?nished Volume II. The emphasis in this volume is on smoothing splines of arbitrary order, but other estimators (kernels, local and global polynomials) pass review as well. We study smoothing splines and local polynomials in the context of reproducing kernel Hilbert spaces. The connection between smoothing splines and reproducing kernels is of course well-known. The new twist is thatlettingtheinnerproductdependonthesmoothingparameteropensup new possibilities: It leads to asymptotically equivalent reproducing kernel estimators (without quali?cations) and thence, via uniform error bounds for kernel estimators, to uniform error bounds for smoothing splines and, via strong approximations, to con?dence bands for the unknown regression function. ItcameassomewhatofasurprisethatreproducingkernelHilbert space ideas also proved useful in the study of local polynomial estimators.

This is the second volume of a text on the theory and practice of maximum penalized likelihood estimation. It is intended for graduate students in statistics, operations research and applied mathematics, as well as for researchers and practitioners in the field. The present volume deals with nonparametric regression.

The emphasis in this volume is on smoothing splines of arbitrary order, but other estimators (kernels, local and global polynomials) pass review as well. Smoothing splines and local polynomials are studied in the context of reproducing kernel Hilbert spaces. The connection between smoothing splines and reproducing kernels is of course well-known. The new twist is that letting the innerproduct depend on the smoothing parameter opens up new possibilities. It leads to asymptotically equivalent reproducing kernel estimators (without qualifications), and thence, via uniform error bounds for kernel estimators, to uniform error bounds for smoothing splines and via strong approximations, to confidence bands for the unknown regression function.

The reason for studying smoothing splines of arbitrary order is that one wants to use them for data analysis. Regarding the actual computation, the usual scheme based on spline interpolation is useful for cubic smoothing splines only. For splines of arbitrary order, the Kalman filter is the most important method, the intricacies of which are explained in full. The authors also discuss simulation results for smoothing splines and local and global polynomials for a variety of test problems as well as results on confidence bands for the unknown regression function based on undersmoothed quintic smoothing splines with remarkably good coverage probabilities.

P.P.B. Eggermont and V.N. LaRiccia are with the Statistics Program of the Department of Food and Resource Economics in the College of Agriculture and Natural Resources at the University of Delaware, and the authors of Maximum Penalized Likelihood Estimation: Volume I: Density Estimation.