Zhang, Ceyao ; Zhang, Tianjian ; Yin, Feng ; Zoubir, Abdelhak M. (2023)
Data-Adaptive M-Estimators for Robust Regression via Bi-Level Optimization.
In: Signal Processing, 210
doi: 10.1016/j.sigpro.2023.109063
Artikel, Bibliographie

Kurzbeschreibung (Abstract)

M-estimators are widely used in robust regression to handle heavy-tailed data corrupted by outliers. Although they have been applied to a plethora of real scenarios, it remains a challenge to practitioners how to set the tuning parameters. Often, it is set by manual tuning or according to the asymptotic efficiency rule, being sub-optimal for a real dataset with finite size. In this paper, we explore a data-driven paradigm where the optimal tuning parameters are determined by the dataset itself. Specifically, we treat the tuning parameters as hyper-parameters in robust regression, formulate the tuning problem via a novel bi-level optimization framework, and solve the regression model parameters and the tuning parameters in a joint manner. To solve this problem efficiently, especially when using neural network as the regression model, we further employ an online approximation strategy to iteratively optimize the model parameters and the tuning parameters with a proven sub-linear convergence rate. Moreover, our proposed framework is generic for any parametric regression model and M-estimator with differentiable loss function. We instantiate this framework with two popular M-estimators (Huber’s and Tukey’s) and derive the corresponding data-adaptive M-estimators. In the experiment, we present positive simulation results compared with various salient benchmarks.

Frage zum Eintrag

Redaktionelle Details anzeigen