Skip to content
/ loo Public
forked from stan-dev/loo

loo R package for approximate leave-one-out cross-validation (LOO-CV) and Pareto smoothed importance sampling (PSIS)

License

Notifications You must be signed in to change notification settings

kdubovikov/loo

 
 

Repository files navigation

loo

CRAN_Status_Badge RStudio_CRAN_mirror_downloads_badge codecov R-CMD-check

Efficient approximate leave-one-out cross-validation for fitted Bayesian models

loo is an R package that allows users to compute efficient approximate leave-one-out cross-validation for fitted Bayesian models, as well as model weights that can be used to average predictive distributions. The loo package package implements the fast and stable computations for approximate LOO-CV and WAIC from

  • Vehtari, A., Gelman, A., and Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing. 27(5), 1413--1432. doi:10.1007/s11222-016-9696-4. Online, arXiv preprint arXiv:1507.04544.

and computes model weights as described in

  • Yao, Y., Vehtari, A., Simpson, D., and Gelman, A. (2018). Using stacking to average Bayesian predictive distributions. In Bayesian Analysis, doi:10.1214/17-BA1091. Online, arXiv preprint arXiv:1704.02030.

From existing posterior simulation draws, we compute approximate LOO-CV using Pareto smoothed importance sampling (PSIS), a new procedure for regularizing importance weights. As a byproduct of our calculations, we also obtain approximate standard errors for estimated predictive errors and for comparing predictive errors between two models. We recommend PSIS-LOO-CV instead of WAIC, because PSIS provides useful diagnostics and effective sample size and Monte Carlo standard error estimates.

Resources

Installation

  • Install the latest release from CRAN:
install.packages("loo")
  • Install the latest development version from GitHub:
# install.packages("remotes")
remotes::install_github("stan-dev/loo")

We do not recommend setting build_vignettes=TRUE when installing from GitHub because some of the vignettes take a long time to build and are always available online at mc-stan.org/loo/articles/.

Python and Matlab/Octave Code

Corresponding Python and Matlab/Octave code can be found at the avehtari/PSIS repository.

About

loo R package for approximate leave-one-out cross-validation (LOO-CV) and Pareto smoothed importance sampling (PSIS)

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • R 100.0%