Merge pull request scipy#10286 from mdhaber/dev_doc_overhaul

Development Documentation Overhaul

doc/API.rst.txt

@@ -1,5 +1,10 @@
-API - importing from SciPy
-==========================
+.. _scipy-api:
+
+SciPy API
+=========
+
+Importing from SciPy
+---------------------
 
 In Python the distinction between what is the public API of a library and what
 are private implementation details is not always clear.  Unlike in other
@@ -8,17 +13,20 @@ objects.  Occasionally this may be convenient, but be aware that if you do so
 your code may break without warning in future releases.  Some widely understood
 rules for what is and isn't public in Python are:
 
-  - Methods / functions / classes and module attributes whose names begin
-    with a leading underscore are private.
-  - If a class name begins with a leading underscore none of its members are
-    public, whether or not they begin with a leading underscore.
-  - If a module name in a package begins with a leading underscore none of
-    its members are public, whether or not they begin with a leading
-    underscore.
-  - If a module or package defines ``__all__`` that authoritatively defines the
-    public interface.
-  - If a module or package doesn't define ``__all__`` then all names that don't
-    start with a leading underscore are public.
+- Methods / functions / classes and module attributes whose names begin with a
+  leading underscore are private.
+
+- If a class name begins with a leading underscore none of its members are
+  public, whether or not they begin with a leading underscore.
+
+- If a module name in a package begins with a leading underscore none of
+  its members are public, whether or not they begin with a leading underscore.
+
+- If a module or package defines ``__all__`` that authoritatively defines the
+  public interface.
+
+- If a module or package doesn't define ``__all__`` then all names that don't
+  start with a leading underscore are public.
 
 .. note:: Reading the above guidelines one could draw the conclusion that every
           private module or object starts with an underscore.  This is not the
@@ -140,3 +148,49 @@ change is made.
 
   - ``scipy.stats.distributions``
   - `scipy.stats.mstats`
+
+
+SciPy structure
+---------------
+
+All SciPy modules should follow the following conventions. In the
+following, a *SciPy module* is defined as a Python package, say
+``yyy``, that is located in the scipy/ directory.
+
+* Ideally, each SciPy module should be as self-contained as possible.
+  That is, it should have minimal dependencies on other packages or
+  modules.  Even dependencies on other SciPy modules should be kept to
+  a minimum.  A dependency on NumPy is of course assumed.
+
+* Directory ``yyy/`` contains:
+
+  - A file ``setup.py`` that defines
+    ``configuration(parent_package='',top_path=None)`` function
+    for `numpy.distutils`.
+
+  - A directory ``tests/`` that contains files ``test_<name>.py``
+    corresponding to modules ``yyy/<name>{.py,.so,/}``.
+
+* Private modules should be prefixed with an underscore ``_``,
+  for instance ``yyy/_somemodule.py``.
+
+* User-visible functions should have good documentation following
+  the `NumPy documentation style`_.
+
+* The ``__init__.py`` of the module should contain the main reference
+  documentation in its docstring. This is connected to the Sphinx
+  documentation under ``doc/`` via Sphinx's automodule directive.
+
+  The reference documentation should first give a categorized list of
+  the contents of the module using ``autosummary::`` directives, and
+  after that explain points essential for understanding the use of the
+  module.
+
+  Tutorial-style documentation with extensive examples should be
+  separate, and put under ``doc/source/tutorial/``
+
+See the existing SciPy submodules for guidance.
+
+For further details on NumPy distutils, see `NumPy Distutils - User's Guide <https://github.com/numpy/numpy/blob/master/doc/DISTUTILS.rst.txt>`_.
+
+.. _NumPy documentation style: https://numpydoc.readthedocs.io/en/latest/format.html

doc/source/building/faq.rst

@@ -0,0 +1,63 @@
+.. _build-install-faq:
+
+=================
+Build/Install FAQ
+=================
+
+*How do I set up a development version of SciPy in parallel to a released
+version that I use to do my job/research?*
+
+If you use the ``conda`` package manager, this is covered in the
+:ref:`quickstart-mac`, specifically in :ref:`quickstart-mac-install`.
+
+Another simple way to achieve this is to install the released version in
+site-packages, by using a binary installer or pip for example, and set
+up the development version in a virtualenv.  First install
+`virtualenv`_ (optionally use `virtualenvwrapper`_), then create your
+virtualenv (named scipy-dev here) with::
+
+    $ virtualenv scipy-dev
+
+Now, whenever you want to switch to the virtual environment, you can use the
+command ``source scipy-dev/bin/activate``, and ``deactivate`` to exit from the
+virtual environment and back to your previous shell.  With scipy-dev
+activated, install first Scipy's dependencies::
+
+    $ pip install numpy pytest cython pybind11
+
+After that, you can install a development version of Scipy, for example via::
+
+    $ python setup.py install
+
+The installation goes to the virtual environment.
+
+
+*How do I set up an in-place build for development*
+
+For development, you can set up an in-place build so that changes made to
+``.py`` files have effect without rebuild. First, run::
+
+    $ python setup.py build_ext -i
+
+Then you need to point your PYTHONPATH environment variable to this directory.
+Some IDEs (`Spyder`_ for example) have utilities to manage PYTHONPATH.  On Linux
+and OSX, you can run the command::
+
+    $ export PYTHONPATH=$PWD
+
+and on Windows::
+
+    $ set PYTHONPATH=/path/to/scipy
+
+Now editing a Python source file in SciPy allows you to immediately
+test and use your changes (in ``.py`` files), by simply restarting the
+interpreter.
+
+If you use the macOS, please find more information in the
+:ref:`quickstart-mac`.
+
+.. _virtualenv: https://virtualenv.pypa.io/
+
+.. _virtualenvwrapper: https://bitbucket.org/dhellmann/virtualenvwrapper/
+
+.. _Spyder: https://www.spyder-ide.org/

doc/source/building/index.rst

@@ -1,3 +1,5 @@
+:orphan:
+
 .. _building:
 
 Building from sources
@@ -8,12 +10,12 @@ Building from sources
    If you are only trying to install SciPy, see
    :doc:`../install_upgrade`.
 
-Build instructions for different operating systems:
+Build instructions for different operating systems and an FAQ:
 
 .. toctree::
    :maxdepth: 2
 
    linux
    windows
    macosx
-
+   faq

doc/source/building/macosx.rst

@@ -2,6 +2,11 @@
 Building From Source on Mac OSX
 ===============================
 
+.. note::
+
+   This document has not been maintained and is retained for reference only.
+   For building on macOS, please see :ref:`quickstart-mac`.
+
 These instructions describe how to build NumPy and SciPy libraries from
 source.
 

doc/source/conf.py

@@ -104,19 +104,10 @@
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = 'sphinx'
 
-# Enusre all our internal links work
+# Ensure all our internal links work
 nitpicky = True
 exclude_patterns = [  # glob-style
-    # these are all included directly in dev/index.rst:
-    'dev/decisions.rst',
-    'dev/deprecations.rst',
-    'dev/distributing.rst',
-    'dev/github.rst',
-    'dev/licensing.rst',
-    'dev/modules.rst',
-    'dev/newfeatures.rst',
-    'dev/releasing.rst',
-    'dev/versioning.rst',
+
 ]
 
 # be strict about warnings in our examples, we should write clean code
@@ -325,6 +316,7 @@
     'python': ('https://docs.python.org/dev', None),
     'numpy': ('https://docs.scipy.org/doc/numpy', None),
     'matplotlib': ('https://matplotlib.org', None),
+    'asv': ('https://asv.readthedocs.io/en/stable/', None),
 }