Docs feature branch: set up Sphinx, update pydocs, implement `streaml…

…it hello`, change `streamlit help` (streamlit#341)

* Make st.title/header/subheader use Markdown under the hood. This make sure there's no difference in how we render those vs. Markdown titles.

* Improve st.help(). Add support for function signatures, better text rendering, and st.help(obj)

* Forgot to commit doc_string.py

* Add tests

* Make tests compatible with python2

* Fix tests.

* If no funcsigs module and not python3, don't provide function signatures.

* Require funcsigs

* Fix copyright year.

* Add some docs at site/reports

* Fix bug with st.help(st.line_chart)

* Use ImportError instead of ModuleNotFoundError for compatibility

* Fix typo in reference.py

* Add a few chart examples to charts.py

* Move docs/ to contribute/docs/, to make room for our user-facing docs.

* Remove site/reports

* Add support for ?embed=true for Streamlit reports (streamlit#324)

* Sphinx docs, part 3: Finally add Sphinx-based docs. (streamlit#325)

* Add support for ?embed=true for Streamlit reports

* Add Sphinx-based docs.

* [docs] Fix make publish-docs

* [docs] Make sphinx not create _build/_sources folder

* [docs] Add sphinx to pipfile

* [docs] Clean up index and fix typos

* [docs] Rename `/_/` to `/secret/`.

* [docs] Make changes from Armando's comments.

* [docs] Remove make.bat

* [docs] Move conf.py exclude_patterns into docs/excludes file.

* [docs] Fix docs/README.md typos

* Sphinx docs, part 4: Improve docstrings and add report examples. (streamlit#326)

* Add support for ?embed=true for Streamlit reports

* Add Sphinx-based docs.

* [docs] Fix make publish-docs

* [docs] Make sphinx not create _build/_sources folder

* [docs] Add sphinx to pipfile

* [docs] Clean up index and fix typos

* Improve docstrings and add report examples.

* Don't test doctests. Those are just for documentation purposes right now.

* [docs] Fix Python 2 breakage due to partials() and wraps(). Streamlit wasn't even running.

* [docs] Fix tests

* [docs] Fix typo.

* [docs] Clean up index and fix typos

* [docs] Rename `/_/` to `/secret/`.

* [docs] Make changes from Armando's comments.

* [docs] Remove make.bat

* [docs] Move conf.py exclude_patterns into docs/excludes file.

* [docs] Fix docs/README.md typos

* Sphinx docs, part 5: Make 'streamlit help' show docs, and implement 'streamlit hello'. (streamlit#339)

* Add support for ?embed=true for Streamlit reports

* Add Sphinx-based docs.

* [docs] Fix make publish-docs

* [docs] Make sphinx not create _build/_sources folder

* [docs] Add sphinx to pipfile

* [docs] Clean up index and fix typos

* Improve docstrings and add report examples.

* Don't test doctests. Those are just for documentation purposes right now.

* [docs] Fix Python 2 breakage due to partials() and wraps(). Streamlit wasn't even running.

* [docs] Fix tests

* [docs] Fix typo.

* [docs] Clean up index and fix typos

* [docs] Rename `/_/` to `/secret/`.

* Make 'streamlit help' show docs (same in menus), and implements 'streamlit hello'.

* Replace `streamlit help` with running `reference.py` in CONTRIBUTING

* [docs] Update docs link at util.py

* [docs] Make changes from Armando's comments.

* [docs] Remove make.bat

* [docs] Move conf.py exclude_patterns into docs/excludes file.

* [docs] Fix docs/README.md typos

* Make 'streamlit help' show docs (same in menus), and implements 'streamlit hello'.

* Replace `streamlit help` with running `reference.py` in CONTRIBUTING

* [docs] Update docs link at util.py

* [docs] Fix path for reference.py in CONTRIBUTING.md

* Remove newline before copyright.

* Remove spare merge conflict marker

* Improve hello.py

* [site] Remove /site/content/secret/docs since it should get populated via `make publish-docs`

.gitignore

@@ -60,6 +60,7 @@ frontend/src/protobuf.js
 .streamlit/
 lib/streamlit/static
 streamlit-storage
+docs/_build
 
 # Data Files
 **/uber-raw-data-sep14.csv

CONTRIBUTING.md

@@ -169,7 +169,8 @@ the following in the latest Python 2.x and 3.x:
 
 ## Coding conventions
 
-Streamlit's coding conventions can be found [here](docs/conventions.md).
+Streamlit's coding conventions can be found
+[here](contrib/docs/conventions.md).
 
 
 ## Versioning convention
@@ -189,7 +190,7 @@ Given a version number MAJOR.MINOR.PATCH, increment the:
 
 #### Write The Release notes
 
-Place them [here](docs/release-notes.md). Then:
+Place them [here](contrib/docs/release-notes.md). Then:
 
 #### Make Sure You Don't Have the Proxy Running
 
@@ -250,11 +251,12 @@ Test in in a **fresh 2.7 install**:
 ```
 cd ../streamlit-staging
 pip install ../streamlit/lib/dist/streamlit-0.20.0-py3-none-any.whl
-streamlit help
+streamlit hello
+python ../streamlit/examples/reference.py
 python -m streamlit clear_cache
 python -m streamlit clear_cache
-streamlit help
-streamlit help
+python ../streamlit/examples/reference.py
+python ../streamlit/examples/reference.py
 python -m streamlit clear_cache
 python ../streamlit/examples/mnist-cnn.py
 python

Makefile

@@ -10,6 +10,9 @@ help:
 	@echo " install      - Install streamlit pointing to PYTHONPATH."
 	@echo " wheel        - Create a wheel file in dist/."
 	@echo " loc          - Count lines of code."
+	@echo " clean-docs   - Deletes the autogenerated HTML documentation."
+	@echo " docs         - Generates HTML documentation at /docs/_build."
+	@echo " publish-docs - Builds and pushes the documentation to prod."
 	@echo " site         - Builds the site at /site/public."
 	@echo " devel-site   - Starts the dev server for the site."
 	@echo " publish-site - Builds and pushes the site to prod."
@@ -53,11 +56,11 @@ pylint:
 
 pytest:
 	# Just testing. No code coverage.
-	cd lib; PYTHONPATH=. pytest -v -l --doctest-modules tests/ $(modules)
+	cd lib; PYTHONPATH=. pytest -v -l tests/ $(modules)
 
 pycoverage:
 	# testing + code coverage
-	cd lib; PYTHONPATH=. pytest -v -l --doctest-modules $(foreach dir,$(modules),--cov=$(dir)) --cov-report=term-missing tests/ $(modules)
+	cd lib; PYTHONPATH=. pytest -v -l $(foreach dir,$(modules),--cov=$(dir)) --cov-report=term-missing tests/ $(modules)
 
 install:
 	cd lib ; python setup.py install
@@ -90,24 +93,50 @@ clean:
 	find . -name .streamlit -type d -exec rm -rfv {} \; || true
 	cd lib; rm -rf .coverage .coverage\.*
 
+.PHONY: clean-docs
+clean-docs:
+	cd docs; \
+		make distclean
+
+.PHONY: docs
+docs: clean-docs
+	cd docs; \
+		make html
+
+.PHONY: devel-docs
+devel-docs: docs
+	cd docs/_build/html; \
+		python -m SimpleHTTPServer 8000 || python -m http.server 8000
+
+.PHONY: publish-docs
+publish-docs: docs
+	cd docs/_build; \
+		aws s3 sync \
+				--acl public-read html s3://streamlit.io/secret/docs/ \
+				--profile streamlit
+
 .PHONY: site
 site:
 	cd site; hugo
 
-.PHONY: site
+.PHONY: devel-site
 devel-site:
 	cd site; hugo server -D
 
 .PHONY: publish-site
-publish-site:
+publish-site: site
 	cd site; \
 		hugo; \
 		rm public/secret/index.*; \
-		aws s3 sync --acl public-read public s3://streamlit.io/ --profile streamlit
+		aws s3 sync \
+				--acl public-read public s3://streamlit.io/ \
+				--profile streamlit
 
 .PHONY: protobuf
 protobuf:
-	protoc --proto_path=protobuf protobuf/*.proto --python_out=lib/streamlit/protobuf
+	protoc \
+				--proto_path=protobuf protobuf/*.proto \
+				--python_out=lib/streamlit/protobuf
 	cd frontend/ ; ( \
 		echo "/* eslint-disable */" ; \
 		echo ; \

admin/test_streamlit.py

@@ -76,7 +76,6 @@ def main():
     # First run the 'streamlit commands'
     run_commands('Basic Commands', [
         'streamlit version',
-        'streamlit help'
     ])
 
     run_commands('Examples', [

docs/Makefile

@@ -0,0 +1,24 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Sometimes things get stuck in a mode where editing .md files doesn't cause
+# StOutput to change. If that happens, call "make distclean".
+distclean: clean
+	rm -rf _build/html/_sources/*
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/README.md

@@ -0,0 +1,130 @@
+# Streamlit documentation
+
+We use Sphinx to build our documentation site. This allows us autogenerate some
+of the documentation, using Sphinx's autodoc directives.
+
+## Basics
+
+The documentation is written using Markdown (`.md`), ReStructuredText (`.rst`),
+and pure text (`.txt`) files in this folder and its subfolders. The default
+format for Sphinx is rst, but md syntax is simpler so that's what we're
+using for Streamlit.
+
+However, since md syntax is less powerful than rst, sometimes we insert
+ReStructuredText code into our Markdown files using a special code block (more
+on this below).
+
+## File and folder structure
+
+* `conf.py` This is Sphinx's configuration file.
+* `_build/` This is where the generated documentation goes.
+* `_ext/` This is where custom extensions to ReStructuredText are placed.
+* `_templates/` Not used right now, but if we ever decide to use custom
+  templates, this is where they should go.
+* `_static/` Not used right now, but if we ever decide to use custom
+  templates, this is where their static files (CSS, etc) should go.
+
+## Building
+
+To build the docs just **go to the folder above this one** and run:
+
+```bash
+$ make docs
+```
+
+Or you can build _and start a web server_ by running:
+
+```bash
+$ make devel-docs
+```
+
+The docs will be viewable at http://localhost:8000. **Note that any time you
+modify the source files you'll need to manually rebuild the docs.**
+
+
+## Publishing
+
+To publish the docs into our public site, **go to the folder above this one**
+and run:
+
+```bash
+$ make publish-docs
+```
+
+## Embedding ReStructuredText into Markdown
+
+Sometimes we need our Markdown files to go beyond what its allowed in the md
+spec. In those cases, we can embed ReStructuredText code in a special
+block that tells Sphinx to parse it with the rst parser:
+
+    This code is Markdown
+
+    ```eval_rst
+    This code is ReStructuredText
+    ```
+
+    An this is Markdown again.
+
+This is used often in our documentation in order to do things like generating
+our table of contents and automatically generating documentation for our Python
+modules.
+
+
+## Embedding Streamlit reports
+
+It's a good idea to make sure our docs have plenty of example code and their
+results.  Rather than showing the results using a boring static image, you can
+embed a Streamlit report right in the documentation. To do this, just use our
+custom directive, called `output`:
+
+```rst
+.. output::
+   [URL to Streamlit report]
+   [inline styles to pass to the <iframe> tag]
+```
+
+For example:
+
+```rst
+.. output::
+   http://share.streamlit.io/0.25.0-22EmZ/index.html?id=YHHvgDnAdo5JKQivWhK6tE
+   height: 200px
+```
+
+or simply:
+
+```rst
+.. output::
+   http://share.streamlit.io/0.25.0-22EmZ/index.html?id=YHHvgDnAdo5JKQivWhK6tE
+```
+
+You can use this directive "as is" inside pydoc strings and ReStructuredText
+files. To use it in Markdown files, though, you need to do the following:
+
+    This is my Markdown file
+
+    ```eval_rst
+    .. output::
+       http://share.streamlit.io/0.25.0-22EmZ/index.html?id=YHHvgDnAdo5JKQivWhK6tE
+    ```
+
+    An this is Markdown again.
+
+### Location of embedded report source files
+
+Before we can use `.. output::` to embed a Streamlit report in some doc, we
+first need to create a Streamlit-powered Python script, run it, share it, and
+take note of its shared URL. To make these scripts easier to find, we place
+them in the same folder as the file that references them, and name them using
+the following pattern:
+
+```
+[referrer file name].[example name].py
+```
+
+For example, if `docs/api/charts.py` includes an embedded example of a Vega Lite chart,
+then the Python script to generate that should be:
+
+```
+docs/api/charts.vega_lite_chart.py
+```

docs/_ext/stoutput.py

@@ -0,0 +1,57 @@
+from docutils import nodes
+from docutils.parsers.rst import Directive
+
+class StOutput(Directive):
+    """Insert Streamlit report into HTML doc.
+
+    The first argument is a URL to be iframed, and the second argument
+    (optional) is a string of inline styles to assign to the iframe.
+
+    Examples
+    --------
+
+        .. output::
+        http://share.streamlit.io/0.25.0-2EdmD/index.html?id=jD8gaXYmw8WZeSNQbko9p
+
+        .. output::
+        http://share.streamlit.io/0.25.0-2EdmD/index.html?id=jD8gaXYmw8WZeSNQbko9p
+        height: 5rem; border: 1px solid red;
+
+    """
+
+    has_content = True
+    required_arguments = 1
+    optional_arguments = 1
+    final_argument_whitespace = True
+
+    def run(self):
+
+        src = self.arguments[0]
+
+        if len(self.arguments) > 1:
+            additional_styles = self.arguments[1]
+        else:
+            additional_styles = 'height: 10rem;'
+
+
+        node = nodes.raw(
+            rawsource='',
+            format='html',
+            text='''
+                <iframe
+                    src="%(src)s&embed=true"
+                    style="
+                        width: 100%%;
+                        border: none;
+                        %(additional_styles)s
+                    "
+                ></iframe>
+                <sup><a href="%(src)s" target="_blank">
+                    (view standalone Streamlit report)
+                </a></sup>
+            ''' % {'src': src, 'additional_styles': additional_styles})
+        return [node]
+
+
+def setup(app):
+    app.add_directive('output', StOutput)

docs/api/charts.audio.py

@@ -0,0 +1,26 @@
+import streamlit as st
+import urllib
+
+@st.cache
+def read_file_from_url(url):
+    return urllib.request.urlopen(url).read()
+
+file_bytes = read_file_from_url(
+    'http://streamlit.io/media/Muriel-Nguyen-Xuan-Chopin-valse-opus64-1.ogg')
+
+st.audio(file_bytes, format='audio/ogg')
+
+st.write('''
+    #### Audio credit:
+
+    Performer: _Muriel Nguyen Xuan_ and _Stéphane Magnenat_
+
+    Composer: Frédéric Chopin
+
+    License: Creative Commons Attribution-Share Alike 4.0 International, 3.0 Unported, 2.5 Generic, 2.0 Generic and 1.0 Generic license.
+    https://creativecommons.org/licenses/by-sa/4.0/
+
+    URL:
+    https://upload.wikimedia.org/wikipedia/commons/c/c4/Muriel-Nguyen-Xuan-Chopin-valse-opus64-1.ogg
+
+''')

docs/api/charts.deck_gl_charts1.py

@@ -0,0 +1,23 @@
+import streamlit as st
+import pandas as pd
+import numpy as np
+
+df = pd.DataFrame(
+   np.random.randn(1000, 2) / [50, 50] + [37.76, -122.4],
+   columns=['lat', 'lon'])
+
+# TODO: Use this instead of the example below. Need to autodetect viewport
+# first, thought.
+#st.deck_gl_chart(df)
+
+st.deck_gl_chart(
+    viewport={
+        'latitude': 37.76,
+        'longitude': -122.4,
+        'zoom': 11,
+        'pitch': 50,
+    },
+    layers=[{
+        'type': 'ScatterplotLayer',
+        'data': df,
+    }])