Section: New Results

Uniform regret bounds over $R^d$ for the sequential linear regression problem with the square loss

In [45] we consider the setting of online linear regression for arbitrary deterministic sequences, with the square loss. We are interested in obtaining regret bounds that hold uniformly over all vectors $R^d$. When the feature sequence is known at the beginning of the game, they provided closed-form regret bounds of $2d{B}^{2}lnT+O\left(1\right)$, where $T$ is the number of rounds and $B$ is a bound on the observations. Instead, we derive bounds with an optimal constant of 1 in front of the $d{B}^{2}lnT$ term. In the case of sequentially revealed features, we also derive an asymptotic regret bound of $d{B}^{2}lnT$ for any individual sequence of features and bounded observations. All our algorithms are variants of the online nonlinear ridge regression forecaster, either with a data-dependent regularization or with almost no regularization.