Series regression is one of the most popular non-parametric regression techniques in practice. The most routinely used series estimator is based on ordinary least squares fitting, which is known to be minimax rate optimal in various settings, albeit under stringent restrictions on the basis functions. In this work, inspired by the recently developed Forster-Warmuth (FW) regression, we propose an alternative nonparametric series estimator that can attain minimax estimation rates under strictly weaker conditions imposed on the basis functions, than virtually all existing series estimators in the literature. Another contribution of this work generalizes the FW-regression to a so-called counterfactual regression problem, in which the response variable of interest may not be directly observed (hence, the name “counterfactual”) on all sampled units. Although counterfactual regression is not entirely a new area of inquiry, we propose the first-ever systematic study of this challenging problem from a unified pseudo-outcome perspective. In fact, we provide what appears to be the first generic and constructive approach for generating the pseudo-outcome (to substitute for the unobserved response) which leads to the estimation of the counterfactual regression curve of interest with small bias, namely bias of second order. Several applications are used to illustrate the resulting FW counterfactual regression including a large class of nonparametric regression problems in missing data and causal inference literature, for which we establish conditions for minimax rate optimality. This is joint work with Yachong Yang and Arun kuchibhotla.
Paper can be found here: https://arxiv.org/abs/2307.16798.