Doctor

Statistical inference in two non-standard
regression problems
Emilio Seijo

Submitted in partial fulfillment of the
requirements for the degree
of Doctor of Philosophy
in the Graduate School of Arts and Sciences

COLUMBIA UNIVERSITY
2012

c 2012
Emilio Seijo
All Rights Reserved

ABSTRACT
Statistical inference in two non-standard
regression problems

Emilio Seijo

Thisthesis analyzes two regression models in which their respective least
squares estimators have nonstandard asymptotics. It is divided in an introduction and two parts. The introduction motivates the study of nonstandard
problems and presents an outline of the contents of the remaining chapters.
In part I, the least squares estimator of a multivariate convex regression function is studied in greatdetail. The main contribution here is a proof of the
consistency of the aforementioned estimator in a completely nonparametric
setting. Model misspecification, local rates of convergence and multidimensional regression models mixing convexity and componentwise monotonicity
constraints will also be considered. Part II deals with change-point regression models and the issues that might arise whenapplying the bootstrap to
these problems. The classical bootstrap is shown to be inconsistent on a simple change-point regression model, and an alternative (smoothed) bootstrap
procedure is proposed and proved to be consistent. The superiority of the alternative method is also illustrated through a simulation study. In addition, a

version of the continuous mapping theorem specially suited forchange-point
estimators is proved and used to derive the results concerning the bootstrap.

Contents
List of Figures

vi

Acknowledgments

vii

Chapter 1 Introduction

1

I

6

Convex Regression

Chapter 2 Multivariate Convex Regression
2.1

7

Least squares estimation of a multivariate convex regression
function . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..

Existence and uniqueness

................

13

Finite sample properties . . . . . . . . . . . . . . . . .

16

2.2.3

Computation of the estimator . . . . . . . . . . . . . .

17

Consistency of the least squares estimator . . . . . . . . . . .

19

2.3.1

Fixed Design . . . . . . . . . . . . . . . . . . . . . . .

21

2.3.2

Stochastic Design . . . . . . . . . .. . . . . . . . . . .

22

2.3.3

Main results . . . . . . . . . . . . . . . . . . . . . . . .

23

2.3.4
2.4

12

2.2.2

2.3

Characterization and finite sample properties . . . . . . . . . .
2.2.1

2.2

7

Proof of the main results . . . . . . . . . . . . . . . . .

24

Proofs of auxiliary lemmas . . . . . . . . . . . . . . . . . . . .

30

i

2.4.1

Proofof Lemma 2.3.1 . . . . . . . . . . . . . . . . . . .

30

2.4.2

Proof of Lemma 2.3.2 . . . . . . . . . . . . . . . . . . .

35

2.4.3

Proof of Lemma 2.3.3 . . . . . . . . . . . . . . . . . . .

39

2.4.4

Proof of Lemma 2.3.4 . . . . . . . . . . . . . . . . . . .

39

2.4.5

Proof of Lemma 2.3.5 . . . . . . . . . . . . . . . . . . .

41

2.4.6

Proof of Lemma 2.3.6. . . . . . . . . . . . . . . . . . .

44

2.4.7

Proof of Lemma 2.3.7 . . . . . . . . . . . . . . . . . . .

44

2.4.8

Proof of Lemma 2.3.8 . . . . . . . . . . . . . . . . . . .

48

2.4.9

Proof of Lemma 2.3.9 . . . . . . . . . . . . . . . . . . .

50

2.4.10 Proof of Lemma 2.3.10 . . . . . . . . . . . . . . . . . .

51

Chapter 3 Additional topics regarding convexregression

53

3.1

The one-dimensional case . . . . . . . . . . . . . . . . . . . . .

53

3.2

Convex and componentwise monotone regression functions . .

54

3.2.1

The convex, α-monotone least squares estimator: computation and finite sample properties . . . . . . . . . .

3.2.2

55

The convex, α-monotone least squares estimator: consistency . . . . . . . . . . . . . ....