RLVI: Robust Learning via Variational Inference

Implementation of Adaptive Robust Learning using Latent Bernoulli Variables.

Our method, called RLVI, enables robust maximization of the likelihood. The learned parametric model is thus accurate even when the training data is corrupted. Additionally, RLVI is well-suited for online or stochastic optimization as it does not require estimating the total ratio of contaminated data and adaptively infers the probabilities of sample corruption.

Please cite our paper if you find this approach useful in your work

@inproceedings{karakulev2024adaptive,
  title={Adaptive Robust Learning using Latent Bernoulli Variables},
  author={Aleksandr Karakulev and Dave Zachariah and Prashant Singh},
  booktitle={Forty-first International Conference on Machine Learning},
  year={2024},
  url={https://openreview.net/forum?id=v6eaD7Wekw}
}

Standard parameter estimation

Benchmark, reproduced from (Osama et al., 2020), that compares robust learning algorithms on the four test problems with corrupted samples in the training data: linear and logistic regression, principal component analysis, covariance estimation.

cd standard-learning
python3 main.py

Online learning

Binary classification for Human Activity Recognition dataset (Amine El Helou, 2023), performed in batches with varying level of corruption to simulate the online learning setting.

cd online-learning
python3 main.py

Accuracy levels are higher with RLVI than with stochastic likelihood maximization (SGD).

Deep learning

Synthetic noise (MNIST, CIFAR10, CIFAR100)

Experiments with the datasets in which training data is corrupted with synthetic noise. There are four types of noise: symmetric, asymmetric, pairflip, and instance. Noise rate from 0 to 1 needs to be specified to corrupt the training set. For the method, one can use rlvi or one of the following: regular, coteaching (Han et al., 2018), jocor (Wei et al., 2020), cdr (Xia et al., 2020), usdnl (Xu et al., 2023), and bare (Patel & Sastry, 2023).

Example:

cd deep-learning
python3 main.py \
        --method=rlvi \
        --dataset=mnist \
        --noise_type=pairflip \
        --noise_rate=0.45

Real noise (Food101)

Experiments with the dataset in which training data is corrupted by nature: some of the training images are mislabeled and contain some noise. For the method, one can specify rlvi or one of the following: regular, coteaching, jocor, cdr, usdnl, bare.

Example:

cd deep-learning
python3 food.py --method=rlvi