Merge branch 'master' of https://github.com/jdhenshaw/scousepy into ML

docs/_build/html/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: cd116f10fadcc3141d4629e3c9a3cca0
+config: cf432b621010090e2bf9affeea60c5ba
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/_build/html/_sources/description.rst.txt

@@ -4,76 +4,141 @@
 A brief introduction to scousepy
 ********************************
 
-The method has been updated slightly from the `original IDL version of the
-code <https://github.com/jdhenshaw/SCOUSE>`_. It is now more interactive than
-before which should hopefully speed things up a bit for the user. The method
-is broken down into six stages in total. Each stage is summarised below.
+Spectral decomposition with ``scousepy`` is broken up into a sequence of distinct
+stages. Each stage is summarised below. More information on executing each of
+these steps can be found in the tutorials.
 
 .. image:: Figure_cartoon.png
    :width: 850px
    :align: center
 
+Stage 0: preparation step
+~~~~~~~~~~~~~~~~~~~~~~~~~
+The preparation step involves the creation of configuration files that ``scousepy``
+will use to run the fitting. The first is ``scousepy.config``. This contains the
+top-level information for fitting including information on the directory structure,
+the number of cpus for parallelised fitting, and optional keywords. The second is
+``coverage.config``. This contains all relevant information for the definition of
+the coverage (stage 1).
+
 Stage 1: defining the coverage
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Here ``scousepy`` identifies the spatial area over which to fit the data. It
-generates a grid of spectral averaging areas (SAAs). The user is required to
-provide the width of the spectral averaging area. Extra refinement of spectral
-averaging areas (i.e. for complex regions) can be controlled using the keyword
-`refine_grid`.
+Here ``scousepy`` identifies the spatial area over which to fit the data. There
+are two different ways this can be achieved: 1) via an interactive GUI; 2) via
+the configuration file ``coverage.config''.
+
+Both methods generate a grid of *spectral averaging areas (SAAs)*. In the
+interactive mode, ``scousepy`` will generate moment maps based on user defined
+variables that are supplied as keywords within a ``matplotlib`` GUI. These
+*optional* keywords include a mask threshold as well as ranges in the ``x``,
+``y``, and ``velocity`` axes. In the non-interactive mode, the keywords are supplied
+via ``coverage.config''. The user can also port a ready made mask for the
+coverage definition via the  `mask_coverage` keyword, which should be a path
+to a FITS file which will act as the mask.
+
+In defining the coverage the user must supply the size of the SAAs, which is
+provided via `wsaa` (corresponding to the width, in pixels, of the SAA). The
+user can also provide a filling factor via `fillfactor`. This keyword will
+allow ``scousepy`` to reject all SAAs, where the fractional number of significant
+pixels contained within a given SAA does not satisfy this constraint. Extra
+refinement of the SAAs (i.e. for complex regions) can be controlled using the
+keyword `refine_grid`. By default, the SAAs are Nyquist sampled. This means
+that any given spectrum may have multiple solutions.
 
 Stage 2: fitting the spectral averaging areas
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 User-interactive fitting of the spatially averaged spectra output from stage 1.
-``scousepy`` makes use of the `pyspeckit <http://pyspeckit.readthedocs.io/en/latest/>`_
-package and is fully interactive.
+Running stage 2 will launch a GUI for interactive fitting of the spectra extracted
+from the SAAs.
+
+``scousepy`` uses a technique called *derivative spectroscopy* to provide initial
+guesses for the decomposition of the SAA spectra. Derivative spectroscopy
+identifies peaks in the data by computing the derivatives of the spectrum
+(shown in the top-left panel of the GUI). The method is controlled by two
+parameters that can be adjusted using the sliders at the top of the GUI,
+`SNR` and `alpha`. The former is the signal-to-noise requirement for all identified
+peaks, and the latter controls the kernel size for smoothing of the spectrum.
+Smoothing is required to avoid noise amplification in the derivative spectra.
+
+The fit from derivative spectroscopy can be overruled by initiating the interactive
+fitter from `pyspeckit <http://pyspeckit.readthedocs.io/en/latest/>`_.
+
+The user can navigate through the spectra using the buttons at the bottom of the
+GUI. The user may also choose to apply derivative spectroscopy to all of the
+spectra using the default (or current) values of `SNR` and `alpha`.
 
 Stage 3: automated fitting
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Non user-interactive fitting of individual spectra contained within all SAAs.
+Non user-interactive fitting of the individual spectra contained within all SAAs.
 The user is required to input several tolerance levels to ``scousepy``. Please
 refer to `Henshaw et al. 2016 <http://adsabs.harvard.edu/abs/2016MNRAS.457.2675H>`_
-for more details on each of these.
+for more details on each of these. These are supplied via the ``scousepy.config``
+file.
+
+The Nyquist sampling of the SAAs means that a given spectrum may have multiple
+solutions. ``scousepy`` identifies the best-fitting solution via the Akaike
+Information Criterion (AIC). The AIC is a measure of relative fitting quality
+which is used for fitting evaluation and model selection. The decision is in
+favour of the model with the lowest AIC. The AIC is given
+
+.. math::
+
+  \mathrm{AIC}=2(k-L)
+
+in which :math:`k` is the number of free parameters, and :math:`L` is the log
+likelihood function of the model evaluated at the maximum likelihood estimate
+(i. e., the parameters for which L is maximized). More generally, ``scousepy``
+computes the AIC assuming that the observations are Gaussian distributed such
+that
+
+.. math::
+
+  \mathrm{AIC}=n\,\mathrm{ln}\bigg(\frac{SSR}{n}\bigg)+2k
+
+in which :math:`SSR` is the sum of the squared residuals and :math:`n` is the
+sample size. In the event that the sample size is not large enough :math:`n<40`,
+a correction is applied
+
+.. math::
+
+  \mathrm{AIC}=n\,\mathrm{ln}\bigg(\frac{SSR}{n}\bigg)+2k+\frac{2k(k+1)}{n-k-1}.
+
+The computation is handled via `astropy <https://docs.astropy.org/en/stable/api/astropy.stats.akaike_info_criterion_lsq.html>`_.
+
+To select the best-fitting solution, ``scousepy`` uses the following rule of
+thumb from Burnham and Anderson 2002, pg. 70:
+
+.. math::
 
-Stage 4: selecting the best fits
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+  \Delta \mathrm{AIC}_{i}=\mathrm{AIC}_{i}-\mathrm{AIC}_{min}
 
-Here ``scousepy`` selects the best-fits that are output in stage 3.
+.. math::
 
-Optional Stages
-~~~~~~~~~~~~~~~
+  \Delta \mathrm{AIC}_{i}<2\;\mathrm{substantial\;support\;for\;model}\;i
 
-Unfortunately there is no one-size-fits-all method to selecting a best-fitting
-solution when multiple choices are available (stage 4). SCOUSE uses the Akaike
-Information Criterion, which weights the chi-squared value of a best-fitting
-solution according to the number of free-parameters.
+.. math::
 
-While AIC does a good job of returning the best-fitting solutions, there are
-areas where the best-fitting solutions can be improved. As such the following
-stages are optional but *highly recommended*.
+  4<\Delta \mathrm{AIC}_{i}<7\;\mathrm{considerably\;less\;support\;for\;model}\;i
 
-This part of the process has changed significantly from the original code. The
-user is now presented with several diagnostic plots (see below), selecting
-different regions will display the corresponding spectra, allowing the user to
-check the fit quality.
+.. math::
 
-Depending on the data a user may wish to perform a few iterations of Stages 5-6.
+  \Delta \mathrm{AIC}_{i}>10\;\mathrm{essentially\;no\;support\;for\;model}\;i
 
-Stage 5: checking the best-fitting solutions
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+where :math:`\mathrm{AIC}_{min}` is the minimum :math:`\mathrm{AIC}` value out of
+the models compared.
 
-Checking the fits. Here the user is required to check the best-fitting
-solutions to the spectra. This stage is now fully interactive. The user is first
-presented with several diagnostic plots namely: `rms`, `residstd`, `redchi2`,
-`ncomps`, `aic`, `chi2`. These can be used to assess the quality of fits
-throughout the map. Clicking on a particular region will show the spectra
-associated with that location. The user can then select spectra for closer
-inspection or refitting as required.
 
-Stage 6: re-analysing the identified spectra
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Stage 4: quality control
+~~~~~~~~~~~~~~~~~~~~~~~~
 
-In this stage the user is required to either select an alternative solution or
-re-fit completely the spectra identified in stage 5.
+Quality control of the best-fitting solutions derived in stage 3. Running
+stage 4 will launch a GUI displaying various diagnostic plots of the goodness-of-fit
+statistics output by the decomposition. Clicking on this image will display a
+grid of spectra in the central panel for closer inspection. Clicking on one of
+those spectra will plot the selected spectrum in the right-hand panel. At this
+point the user has the option to select an alternative model solution (if
+available) or re-enter the fitting procedure, either using derivative spectroscopy
+or via the manual fitting procedure implemented in `pyspeckit <http://pyspeckit.readthedocs.io/en/latest/>`_.

docs/_build/html/_sources/index.rst.txt

@@ -26,6 +26,7 @@ Documentation
 
   installation.rst
   description.rst
+  tutorial_v2.0.0.rst
   tutorial_v1.0.0.rst
   license.rst
 
@@ -101,8 +102,9 @@ Please also consider acknowledgements to the required packages in your work.
 Papers using scousepy
 ~~~~~~~~~~~~~~~~~~~~~
 
-* Henshaw et al. 2021, MNRAS, accepted
-* Liu et al. 2021, MNRAS, submitted
+* Hu, Lazarian, & Wang 2021, MNRAS, submitted
+* Henshaw et al. 2021, MNRAS, 509, 4758
+* Liu, Hu, & Lazarian 2021, MNRAS, 501, 4952
 * Barnes et al. 2021, MNRAS, 503, 4601
 * Callanan et al. 2021, MNRAS, 505, 4310
 * Yue et al. 2021, RAA, 21, 24

docs/_build/html/_sources/tutorial_v1.0.0.rst.txt

@@ -53,8 +53,8 @@ is tuned to the n2h+ data, which has hyperfine structure. Here we implement the
 range 32-42 km/s since this allows us to focus on the isolated hyperfine
 component, which we model as a Gaussian.
 
-Stage 1
-~~~~~~~
+Stage 1: defining the coverage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 As discussed in the :ref:`description <description>` of the code, the purpose
 of stage 1 is to identify the spatial area over which to fit the data. Stage 1
@@ -105,8 +105,8 @@ the most complex line-profiles.
   :align: center
   :width: 400
 
-Stage 2
-~~~~~~~
+Stage 2: fitting the spectral averaging areas
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Stage 2 is where we will perform our manual fitting. It is simple to run using ::
 
@@ -169,8 +169,8 @@ perform bitesize fitting where the process is broken down into sessions and the
 user fits a fixed number of spectra at any one time. The number of spectra to
 fit in any one session can be controlled using the ``nspec`` keyword.
 
-Stage 3
-~~~~~~~
+Stage 3: automated fitting
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Stage 3 represents the automated decomposition stage. ``scousepy`` will take you
 best-fitting solutions from stage 2 and pass these to the individual spectra
@@ -219,8 +219,8 @@ case the fitting was not parallelised (``njobs=1``)...
   :align: center
   :width: 900
 
-Stage 4
-~~~~~~~
+Stage 4 - selecting the best-fitting solutions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Not much to say here - just set this stage running. Here the best-fitting
 solutions from stage 3 are compiled for each spectrum. Where the SAAs overlap
@@ -229,8 +229,8 @@ there will be multiple fits for each spectrum. Duplicates are removed and the
 
   s = scouse.stage_4(s, verbose=True)
 
-Stage 5
-~~~~~~~
+Stage 5 - quality control
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Now we want to check our work. Stage 5 works interactively and is run in the
 following way ::
@@ -296,8 +296,8 @@ lot of re-fitting (maybe >5-10% of the data), I will stop at this stage and twea
 the parameters of stage 3 and re-run from there. Once I'm happy that the majority
 of the fits are reasonable I will go through stage 5 in earnest.
 
-Stage 6
-~~~~~~~
+Stage 6 - refitting
+~~~~~~~~~~~~~~~~~~~
 
 Here we are going to look again at the spectra we selected during stage 5.
 Stage 6 is run using ::