Estimation of Linear Functionals in High-Dimensional Linear Models: From Sparsity to Nonsparsity

Abstract

High dimensional linear models are commonly used in practice. In many applications, one is interested in linear transformations ${\beta }^{\top }x$ of regression coefficients $\beta \in R^p$ , where $x$ is a specific point and is not required to be identically distributed as the training data. One common approach is the plug-in technique which first estimates $\beta$ , then plugs the estimator in the linear transformation for prediction. Despite its popularity, estimation of $\beta$ can be difficult for high dimensional problems. Commonly used assumptions in the literature include that the signal of coefficients $\beta$ is sparse and predictors are weakly correlated. These assumptions, however, may not be easily verified, and can be violated in practice. When $\beta$ is non-sparse or predictors are strongly correlated, estimation of $\beta$ can be very difficult. In this paper, we propose a novel pointwise estimator for linear transformations of $\beta$ . This new estimator greatly relaxes the common assumptions for high dimensional problems, and is adaptive to the degree of sparsity of $\beta$ and strength of correlations among the predictors. In particular, $\beta$ can be sparse or non-sparse and predictors can be strongly or weakly correlated. The proposed method is simple for implementation. Numerical and theoretical results demonstrate the competitive advantages of the proposed method for a wide range of problems.

Keywords: Correlated predictors; eigenvalue sparsity; linear transformation; prediction.