README.html

<h2 id="read-me">Read me</h2>
<p><a
href="https://mybinder.org/v2/gh/acpennlab/statistics-resampling-online/jammy-docker?urlpath=/lab/tree/statistics-resampling.ipynb"><img
src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a> <a
href="https://gnu-octave.github.io/statistics-resampling/"><img
src="https://img.shields.io/badge/docs-online-blue.svg"
alt="Documentation" /></a> <a
href="https://doi.org/10.5281/zenodo.3992392"><img
src="https://zenodo.org/badge/DOI/10.5281/zenodo.3992392.svg"
alt="DOI" /></a> <a
href="https://github.com/gnu-octave/statistics-resampling/"><img
src="https://img.shields.io/github/stars/gnu-octave/statistics-resampling?label=GitHub%20Repo&amp;style=social"
alt="GitHub" /></a></p>
<h3 id="package-maintainer">Package maintainer</h3>
<p>Andrew Penn (andy.c.penn@gmail.com)</p>
<h3 id="package-contributors">Package contributors</h3>
<p>Andrew Penn<br />
(More contributors are welcome!)</p>
<h3 id="citations">Citations</h3>
<p>If you use this package, please include the following
citation(s):</p>
<ul>
<li>Penn, Andrew Charles. (2020). Resampling methods for small samples
or samples with complex dependence structures. <em>Zenodo</em>. <a
href="https://doi.org/10.5281/zenodo.3992392">https://doi.org/10.5281/zenodo.3992392</a>
<a href="https://doi.org/10.5281/zenodo.3992392"><img
src="https://zenodo.org/badge/DOI/10.5281/zenodo.3992392.svg"
alt="DOI" /></a></li>
</ul>
<p>(Note that package versions 5.4.3 and below were named the
‘statistics-bootstrap’ package. The ‘statistics-resampling’ package is a
more developed version of the older <a
href="https://github.com/acp29/iboot">‘iboot’</a> package).</p>
<h3 id="description">Description</h3>
<p>The statistics-resampling package is an Octave package and Matlab
toolbox that can be used to perform a wide variety of statistics tasks
using non-parametric resampling methods. In particular, the functions
included can be used to estimate bias, uncertainty (standard errors and
confidence intervals), prediction error, information criteria, and
calculate <em>p</em>-values for null hypothesis significance tests.
Variations of the resampling methods are included that improve the
accuracy of the statistics for small samples and samples with complex
dependence structures.</p>
<h3 id="using-the-statistics-resampling-package-online">Using the
statistics-resampling package online</h3>
<p>Try out the statistics-resampling package in your browser at <a
href="https://mybinder.org/v2/gh/acpennlab/statistics-resampling-online/jammy-docker?urlpath=/lab/tree/statistics-resampling.ipynb">statistics-resampling-online</a>
<a
href="https://mybinder.org/v2/gh/acpennlab/statistics-resampling-online/jammy-docker?urlpath=/lab/tree/statistics-resampling.ipynb"><img
src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a>: a
ready-to-go implementation of statistics-resampling in a JupyterLab
Notebook with an Octave kernel. <em>Note that the first time (since the
last repository commit) that you use statistics-resampling online with
Binder you can expect it to take a while to build a docker image, but
subsequent access to statistics-resampling-online will take less than a
minute or so.</em></p>
<p>Collaborative student projects in GNU Octave can use the
statistics-resampling package at <a
href="https://octave-online.net/">Octave-Online</a>. Doing so requires
users to download the latest release of the Source code (tar.gz) from <a
href="https://github.com/gnu-octave/statistics-resampling/releases">here</a>
and follow steps 2-5 of these <a
href="https://octaveonline.uservoice.com/knowledgebase/articles/1078840-installing-custom-packages">instructions</a>.</p>
<p>Users who prefer Jupyter and have a workflow that is collaborative
and/or crosses over multiple programming languages may find it more
convenient to install and use the statistics-resampling package at <a
href="https://cocalc.com/">COCALC</a>. The approach described above (for
Octave-Online) also applies to installing the statistics-resampling
package via a Jupyter Notebook with an Octave kernel at COCALC.</p>
<p>Alternatively, if you have an account with MATLAB you can try out the
statistics-resampling package at <a
href="https://matlab.mathworks.com/open/github/v1?repo=gnu-octave/statistics-resampling&amp;file=README.md">Matlab-Online</a>
<a
href="https://matlab.mathworks.com/open/github/v1?repo=gnu-octave/statistics-resampling&amp;file=README.md"><img
src="https://www.mathworks.com/images/responsive/global/open-in-matlab-online.svg"
alt="Open in MATLAB Online" /></a> by either following the local
installation instructions below, or by adding the <em>toolbox</em> (not
collection) version of the package via Apps &gt;&gt; Get More Apps.</p>
<p>Follow the links in the ‘Quick start’ section below to obtain some
examples of data and code to try out with the package.</p>
<h3 id="requirements-and-dependencies">Requirements and
dependencies</h3>
<p>Users with greater computational demands may want to consider
installing and running the statistics-resampling package offline.
Installation of the statistics-resampling package has some software
requirements. The core functions in this package require, and are known
to be compatible with, <a href="https://octave.org/">GNU Octave</a>
(version &gt;= 4.4.0) and <a
href="https://uk.mathworks.com/products/matlab.html">Matlab</a> (version
&gt;= R2007a 7.4.0). Some optional features of this package have further
dependencies:</p>
<ul>
<li>All parallel computing options require either the <a
href="https://gnu-octave.github.io/packages/parallel/">parallel
package</a> (in Octave) or the <a
href="https://uk.mathworks.com/products/parallel-computing.html">Parallel
Computing Matlab Toolbox</a> (in Matlab). Note that the Parallel package
in Octave also requires the <a
href="https://gnu-octave.github.io/packages/struct/">struct
package</a>.<br />
</li>
<li>The optional jackknife functionality in <code>boot1way</code>
requires the <a
href="https://gnu-octave.github.io/packages/statistics/">statistics
package</a> (in Octave) or the <a
href="https://uk.mathworks.com/products/statistics.html">Statistics and
Machine Learning Toolbox</a> (in Matlab).</li>
</ul>
<h3 id="installation">Installation</h3>
<p>To install (or test) the statistics-resampling package in your
computer at a location of your choice, for either <strong>Matlab or
Octave</strong>, follow these steps:</p>
<ul>
<li>Download the latest package release from <a
href="https://github.com/gnu-octave/statistics-resampling/releases/">here</a>.
Extract (not just browse) the contents of the compressed file (.zip or
.tar.gz), and move the package directory to the desired location.</li>
<li>Open Octave or Matlab (command prompt).</li>
<li>Change directory (cd) into the package folder. (The directory
contains a file called ‘make.m’ and ‘install.m’, among others)</li>
<li>Type <code>make</code> to compile the MEX files from source (or use
the precompiled binaries if available. If suitable precompiled binaries
are not available for your platform, then Matlab/Octave will need access
to a C++11 compiler. Note that if you skip the make step, then the
package functions will still work, but some will run slower. This step
is interactive so check the command window.)</li>
<li>Type <code>install</code>. The package will load now (and
automatically in the future) when you start Octave/Matlab.</li>
</ul>
<p>If you want or need to uninstall the package, change directory (cd)
into the package folder and type uninstall.</p>
<p>Alternatively, <strong>Octave</strong> users can install the latest
release of the package just like any other Octave package by typing:</p>
<p><code>pkg install -forge statistics-resampling</code></p>
<p>Or for the most recent developmental version of the package:</p>
<p><code>pkg install "https://github.com/gnu-octave/statistics-resampling/archive/refs/heads/master.zip"</code></p>
<p>The package can then be loaded on demand in Octave with the following
commmand:</p>
<p><code>pkg load statistics-resampling</code></p>
<p>(Note that this isn’t necessary if you used the local installation
instructions first described in this section)</p>
<p>Alternatively, <strong>MATLAB</strong> users can conveniently install
the package functions as a toolbox by double-clicking the
‘statistics-resampling.mltbx’ file in the matlab subdirectory. The
toolbox installed in this way can be disabled or uninstalled via
MATLAB’s Add-On manager. MEX files are included with the toolbox
installation for Windows (32- or 64-bit), MacOS (Intel or Apple Silicon
64-bit) and Linux (64-bit).</p>
<h3 id="usage">Usage</h3>
<p>All help and demos are documented on the <a
href="https://gnu-octave.github.io/statistics-resampling/function_reference">‘Function
Reference’</a> page in the <a
href="https://gnu-octave.github.io/statistics-resampling/">manual</a>.
If you do not see the navigation pane on the manual web pages, please
enable javascript in your browser. If you need further help with using
any of the functions in this package, please post your questions on the
<a
href="https://github.com/gnu-octave/statistics-resampling/discussions">discussions
page</a>.</p>
<p>Function help can also be requested directly from the Octave/MATLAB
command prompt, by typing <code>help function-name</code> - substituting
in the actual function name.</p>
<p>In Octave only, you can get a basic overview of the package and it’s
functions by typing:
<code>pkg describe -verbose statistics-resampling</code>, or request
demonstrations of function usage by typing
<code>demo function-name</code>. Users can also request help with using
functions and programming in Octave at the <a
href="https://octave.discourse.group/c/help/6">discourse group</a>.</p>
<p><em>TIPS: You can now document and publish your statistics-resampling
analysis in Jupyter Notebooks (with an Octave kernel) at your GitHub
repository using the <a
href="https://nbgitpuller.readthedocs.io/en/latest/link.html?tab=binder">nbgitpuller
link generator</a> and the statistics-resampling-online Binder
environment <a
href="https://github.com/acpennlab/statistics-resampling-online/"><img
src="https://img.shields.io/github/stars/acpennlab/statistics-resampling-online?label=GitHub%20Repo&amp;style=social" /></a>.
Alternatively, you could fork the repository or use it as a template for
you own GitHub repository. Using Jupyter notebooks, you can also
integrate use of the statistics-resampling package into your analysis
workflow alongside other programming languages including Python, R and
Julia <a
href="https://mybinder.org/v2/gh/acpennlab/statistics-resampling-online/jammy-docker?urlpath=/lab/tree/statistics-resampling.ipynb"><img
src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a></em></p>
<h3 id="quick-start">Quick start</h3>
<p>Below are links to demonstrations of how the bootstrap or
randomization functions from this package can be used to perform
variants of some commonly used statistical tests, but without the
Normality assumption:</p>
<ul>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/boot1way.html#1">t-test
to compare two independent samples (assuming equal
variances)</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/boot1way.html#2">t-test
to compare two independent samples (assuming equal variances)</a> (but
also robust to outliers)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#1">t-test
to compare two independent samples</a> (but also robust to
heteroscedasticity, i.e. unequal variance)</p></li>
<li><p>Nested t-test to compare two independent samples. (See example
listed below for nested one-way ANOVA)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#2">t-test
to compare two paired or matching samples</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#3">One-way
ANOVA to compare two or more independent samples</a> (but also robust to
heteroscedasticity)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#13">Nested
one-way ANOVA to compare two or more independent samples</a> (but also
robust to heteroscedasticity and grouping of observations)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#6">Balanced
two-way factorial ANOVA</a> (but also robust to
heteroscedasticity)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#7">Unbalanced
two-way factorial ANOVA</a> (but also robust to
heteroscedasticity)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#8">Simple
linear regression</a> (but also robust to heteroscedasticity)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#9">One-way
ANCOVA</a> (but also robust to heteroscedasticity)</p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#4">One-way
repeated measures ANOVA</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/randtest1.html#1">Randomization
or permutation test to evaluate whether a sample comes from a population
with the specified mean</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/randtest2.html#1">Randomization
or permutation test to compare the distributions of two independent or
paired/matching samples</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/randtest2.html#5">Randomization
or permutation test to compare the distributions of two independent
samples with clustered observations</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/randtest.html#3">Randomization
or permutation test to evaluate the statistical significance of a
correlation coefficient</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootmode.html#1">Statistically
evaluate the number of modes (i.e. peaks) in a distribution</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/sampszcalc.html#6">Correcting
sample size calculations for a two-sample test with nested
design</a></p></li>
<li><p><a
href="https://gnu-octave.github.io/statistics-resampling/function/bootlm.html#15">Comparing
linear models using bootstrap estimates of prediction error, information
criteria and relative likelihood</a></p></li>
</ul>
<p>For examples of how to import data sets from a human-readable text
file, like a tab-separated-value (TSV) and comma-separated-value (CSV)
file, see the examples in the JupyterLab Notebook at <a
href="https://mybinder.org/v2/gh/acpennlab/statistics-resampling-online/jammy-docker?urlpath=/lab/tree/statistics-resampling.ipynb">statistics-resampling-online</a>
<a
href="https://mybinder.org/v2/gh/acpennlab/statistics-resampling-online/jammy-docker?urlpath=/lab/tree/statistics-resampling.ipynb"><img
src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a> and the
last demonstration listed on <a
href="https://gnu-octave.github.io/statistics-resampling/function/randtest2.html#5">this
page</a></p>
<h3 id="issues">Issues</h3>
<p>If you find bugs or have any suggestions, please raise an issue on
GitHub <a
href="https://github.com/gnu-octave/statistics-resampling/issues">here</a>.
If you have any problems specifically with Binder for
statistics-resampling online, please raise an issue on GitHub <a
href="https://github.com/acpennlab/statistics-resampling-online/issues">here</a>.
Please make sure that, when reporting a bug, you provide as much
information as possible for other users to be able to replicate it.</p>
<p>Package: <a
href="https://gnu-octave.github.io/statistics-resampling/">statistics-resampling</a></p>