docs: use custom roles to generate GitHub links

This change replaces direct links to GitHub master branch with auto-generated links using docutils custom roles. These auto-generated links point to the tree or blob for the git commit ID (or tag) of the repository. This is needed to ensure that links don’t become broken when files in master branch are moved around or deleted. The following roles are introduced: - :idf:`path` - points to directory inside ESP-IDF - :idf_blob:`path` - points to file inside ESP-IDF - :idf_raw:`path` - points to raw view of the file inside ESP-IDF - :component:`path` - points to directory inside ESP-IDF components dir - :component_blob:`path` - points to file inside ESP-IDF components dir - :component_raw:`path` - points to raw view of the file inside ESP-IDF components dir - :example:`path` - points to directory inside ESP-IDF examples dir - :example_blob:`path` - points to file inside ESP-IDF examples dir - :example_raw:`path` - points to raw view of the file inside ESP-IDF examples dir A check is added to the CI build script, which searches RST files for presence of hard-coded links (identified by tree/master, blob/master, or raw/master part of the URL). This check can be run manually: cd docs && make gh-linkcheck Additionally, Sphinx linkcheck build type is used to create new CI test, which check for broken links. This test has to be triggered explicitly, because including it in normal build process (when the commit is not yet deployed to Github) will not work. It can be triggered in a regular fashion using a combination of cron and Curl, similar to stress tests.
TrackliDev · Jan 20, 2017 · 1507202 · 1507202
1 parent 7eb570b
commit 1507202
Show file tree

Hide file tree

Showing 52 changed files with 187 additions and 116 deletions.
diff --git a/.gitignore b/.gitignore
@@ -27,6 +27,7 @@ examples/*/*/build
 docs/_build/
 docs/doxygen-warning-log.txt
 docs/xml/
+docs/man/
 
 # Unit test app files
 tools/unit-test-app/sdkconfig

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
@@ -125,6 +125,7 @@ build_docs:
     - doxygen
     # If not building master branch, and there are Doxygen warnings, print them and bail out
     - test "${CI_BUILD_REF_NAME}" = "master" || test $(cat doxygen-warning-log.txt | wc -l) -eq 0 || ( echo "Doxygen pass had some warnings:" && cat doxygen-warning-log.txt && false )
+    - make gh-linkcheck
     - make html
   artifacts:
     paths:
@@ -246,6 +247,26 @@ deploy_docs:
     - scp $GIT_VER.tar.gz $DOCS_SERVER:$DOCS_PATH
     - ssh $DOCS_SERVER -x "cd $DOCS_PATH && tar xzvf $GIT_VER.tar.gz && rm -f latest && ln -s $GIT_VER latest"
 
+check_doc_links:
+  stage: test
+  image: espressif/esp32-ci-env
+  tags:
+    - check_doc_links
+  only:
+    # can only be triggered
+    - triggers
+  script:
+    # must be triggered with CHECK_LINKS=Yes, otherwise exit without test
+    - test $CHECK_LINKS = "Yes" || exit 0
+    # can only run on master branch (otherwise the commit is not on Github yet)
+    - test "${CI_BUILD_REF_NAME}" = "master" || exit 0
+    - cd docs
+    - make linkcheck
+  artifacts:
+    paths:
+      - docs/_build/linkcheck
+    expire_in: 1 mos
+
 
 # AUTO GENERATED PART START, DO NOT MODIFY CONTENT BELOW
 # template for test jobs

diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
@@ -21,7 +21,7 @@ Before sending us a Pull Request, please consider this list of points:
 
 * Is the code adequately commented for people to understand how it is structured?
 
-* Is there documentation or examples that go with code contributions? `There are additional suggestions for writing good examples in the examples README <https://github.com/espressif/esp-idf/tree/master/examples>`_.
+* Is there documentation or examples that go with code contributions? There are additional suggestions for writing good examples in :idf:`examples` readme.
 
 * Are comments and documentation written in clear English, with no spelling or grammar errors?
 

diff --git a/components/vfs/README.rst b/components/vfs/README.rst
@@ -140,7 +140,7 @@ actually is translated to to this (by the preprocessor):
 
     fprintf(__getreent()->_stderr, "42\n");
 
-where the ``__getreent()`` function returns a per-task pointer to ``struct _reent`` (`source <https://github.com/espressif/esp-idf/blob/master/components/newlib/include/sys/reent.h#L370-L417>`_). This structure is allocated on the TCB of each task. When a task is initialized, ``_stdin``, ``_stdout`` and ``_stderr`` members of ``struct _reent`` are set to the values of ``_stdin``, ``_stdout`` and ``_stderr`` of ``_GLOBAL_REENT`` (i.e. the structure which is used before FreeRTOS is started).
+where the ``__getreent()`` function returns a per-task pointer to ``struct _reent`` (:component_file:`newlib/include/sys/reent.h#L370-L417>`). This structure is allocated on the TCB of each task. When a task is initialized, ``_stdin``, ``_stdout`` and ``_stderr`` members of ``struct _reent`` are set to the values of ``_stdin``, ``_stdout`` and ``_stderr`` of ``_GLOBAL_REENT`` (i.e. the structure which is used before FreeRTOS is started).
 
 Such a design has the following consequences:
 

diff --git a/docs/COPYRIGHT.rst b/docs/COPYRIGHT.rst
@@ -14,7 +14,7 @@ Additional third party copyrighted code is included under the following licenses
 
 * Xtensa header files (components/esp32/include/xtensa) are Copyright (C) 2013 Tensilica Inc and are licensed under the MIT License as reproduce in the individual header files.
 
-* `esptool.py`_ (bin/esptool.py) is Copyright (C) 2014-2016 Fredrik Ahlberg, Angus Gratton and is licensed under the GNU General Public License v2, as described in the file components/esptool_py/LICENSE.
+* `esptool.py`_ (components/esptool_py/esptool) is Copyright (C) 2014-2016 Fredrik Ahlberg, Angus Gratton and is licensed under the GNU General Public License v2, as described in the file components/esptool_py/LICENSE.
 
 * Original parts of FreeRTOS_ (components/freertos) are Copyright (C) 2015 Real Time Engineers Ltd and is licensed under the GNU General Public License V2 with the FreeRTOS Linking Exception, as described in the file components/freertos/license.txt.
 
@@ -96,7 +96,7 @@ Copyright (C) 2011, ChaN, all right reserved.
 
 .. _Newlib: https://sourceware.org/newlib/
 .. _FreeRTOS: http://freertos.org/
-.. _esptool.py: https://github.com/themadinventor/esptool
+.. _esptool.py: https://github.com/espressif/esptool
 .. _LWIP: http://savannah.nongnu.org/projects/lwip/
 .. _TinyBasic: https://github.com/BleuLlama/TinyBasicPlus
 .. _miniz: https://code.google.com/archive/p/miniz/

diff --git a/docs/Makefile b/docs/Makefile
@@ -161,6 +161,30 @@ linkcheck:
 	@echo "Link check complete; look for any errors in the above output " \
 	      "or in $(BUILDDIR)/linkcheck/output.txt."
 
+gh-linkcheck:
+	@echo "Checking for hardcoded GitHub links"
+	@if (find ../ -name '*.rst' | xargs grep \
+		'https://github.com/espressif/esp-idf/tree\|https://github.com/espressif/esp-idf/blob\|https://github.com/espressif/esp-idf/raw'\
+		); \
+	then \
+		echo "WARNINIG: Some .rst files contain hardcoded Github links."; \
+		echo "Please check above output and replace links with one of the following:"; \
+		echo "- :idf:\`dir\` - points to directory inside ESP-IDF"; \
+		echo "- :idf_blob:\`file\` - points to file inside ESP-IDF"; \
+		echo "- :idf_raw:\`file\` - points to raw view of the file inside ESP-IDF"; \
+		echo "- :component:\`dir\` - points to directory inside ESP-IDF components dir"; \
+		echo "- :component_blob:\`file\` - points to file inside ESP-IDF components dir"; \
+		echo "- :component_raw:\`file\` - points to raw view of the file inside ESP-IDF"; \
+		echo "  components dir"; \
+		echo "- :example:\`dir\` - points to directory inside ESP-IDF examples dir"; \
+		echo "- :example_blob:\`file\` - points to file inside ESP-IDF examples dir"; \
+		echo "- :example_raw:\`file\` - points to raw view of the file inside ESP-IDF"; \
+		echo "  examples dir"; \
+		echo "These link types will point to the correct GitHub version automatically"; \
+		exit 1; \
+	fi
+	@echo "No hardcoded links found"
+
 doctest:
 	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
 	@echo "Testing of doctests in the sources finished, look at the " \

diff --git a/docs/api/bluetooth/controller_vhci.rst b/docs/api/bluetooth/controller_vhci.rst
@@ -11,9 +11,9 @@ Overview
 Application Example
 -------------------
 
-Check `/examples/bluetooth <https://github.com/espressif/esp-idf/tree/master/examples/bluetooth>`_ folder of `espressif/esp-idf <https://github.com/espressif/esp-idf/>`_ repository, that contains the following example:
+Check :example:`bluetooth` folder in ESP-IDF examples, which contains the following example:
 
-`ble_adv <https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/ble_adv>`_ 
+:example:`bluetooth/ble_adv`
 
   This is a BLE advertising demo with virtual HCI interface. Send Reset/ADV_PARAM/ADV_DATA/ADV_ENABLE HCI command for BLE advertising.
 
@@ -23,7 +23,7 @@ API Reference
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/include/bt.h <https://github.com/espressif/esp-idf/blob/master/components/bt/include/bt.h>`_
+  * :component_file:`bt/include/bt.h`
 
 Type Definitions
 ^^^^^^^^^^^^^^^^

diff --git a/docs/api/bluetooth/esp_blufi.rst b/docs/api/bluetooth/esp_blufi.rst
@@ -11,9 +11,9 @@ Use should concern these things:
 Application Example
 -------------------
 
-Check `/examples/bluetooth <https://github.com/espressif/esp-idf/tree/master/examples/bluetooth>`_ folder of `espressif/esp-idf <https://github.com/espressif/esp-idf>`_ repository, that contains the following example:
+Check :example:`bluetooth` folder in ESP-IDF examples, which contains the following example:
 
-`blufi <https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/blufi>`_ 
+:example:`bluetooth/blufi` 
 
   This is a BLUFI demo. This demo can set ESP32's wifi to softap/station/softap&station mode and config wifi connections.
 
@@ -24,7 +24,7 @@ API Reference
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_blufi_api.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_blufi_api.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_blufi_api.h`
 
 Macros
 ^^^^^^

diff --git a/docs/api/bluetooth/esp_bt_defs.rst b/docs/api/bluetooth/esp_bt_defs.rst
@@ -20,7 +20,7 @@ API Reference
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_bt_defs.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_bt_defs.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_bt_defs.h`
 
 
 Macros

diff --git a/docs/api/bluetooth/esp_bt_device.rst b/docs/api/bluetooth/esp_bt_device.rst
@@ -22,7 +22,7 @@ API Reference
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_bt_device.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_bt_device.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_bt_device.h`
 
 
 Macros

diff --git a/docs/api/bluetooth/esp_bt_main.rst b/docs/api/bluetooth/esp_bt_main.rst
@@ -20,7 +20,7 @@ API Reference
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_bt_main.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_bt_main.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_bt_main.h`
 
 
 Macros

diff --git a/docs/api/bluetooth/esp_gap_ble.rst b/docs/api/bluetooth/esp_gap_ble.rst
@@ -11,19 +11,19 @@ Overview
 Application Example
 -------------------
 
-Check `/examples/bluetooth <https://github.com/espressif/esp-idf/tree/master/examples/bluetooth>`_ folder of `espressif/esp-idf <https://github.com/espressif/esp-idf>`_ repository, that contains the following examples:
+Check :example:`bluetooth` folder in ESP-IDF examples, which contains the following examples:
 
-`gatt_server <https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/gatt_server>`_, `gatt_client <https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/gatt_client>`_ 
+:example:`bluetooth/gatt_server`, :example:`bluetooth/gatt_client` 
 
-  The two demos use different gap api, such like advertising, scan, set device name and others.
+  The two demos use different GAP APIs, such like advertising, scan, set device name and others.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_gap_ble_api.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_gap_ble_api.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_gap_ble_api.h`
 
 
 Macros

diff --git a/docs/api/bluetooth/esp_gatt_defs.rst b/docs/api/bluetooth/esp_gatt_defs.rst
@@ -20,7 +20,7 @@ API Reference
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_gatt_defs.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_gatt_defs.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_gatt_defs.h`
 
 
 Macros

diff --git a/docs/api/bluetooth/esp_gattc.rst b/docs/api/bluetooth/esp_gattc.rst
@@ -11,11 +11,11 @@ Overview
 Application Example
 -------------------
 
-Check `/examples/bluetooth <https://github.com/espressif/esp-idf/tree/master/examples/bluetooth>`_ folder of `espressif/esp-idf <https://github.com/espressif/esp-idf>`_ repository, that contains the following example:
+Check :example:`bluetooth` folder in ESP-IDF examples, which contains the following examples:
 
-`gatt_client <https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/gatt_client>`_ 
+:example:`bluetooth/gatt_client`
 
-  This is a gatt client demo. This demo can scan devices, connect to the gatt server and discover the service.
+  This is a GATT client demo. This demo can scan devices, connect to the GATT server and discover the service.
 
 
 API Reference
@@ -24,7 +24,7 @@ API Reference
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_gattc_api.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_gattc_api.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_gattc_api.h`
 
 Macros
 ^^^^^^

diff --git a/docs/api/bluetooth/esp_gatts.rst b/docs/api/bluetooth/esp_gatts.rst
@@ -11,19 +11,19 @@ Overview
 Application Example
 -------------------
 
-Check `/examples/bluetooth <https://github.com/espressif/esp-idf/tree/master/examples/bluetooth>`_ folder of `espressif/esp-idf <https://github.com/espressif/esp-idf>`_ repository, that contains the following example:
+Check :example:`bluetooth` folder in ESP-IDF examples, which contains the following example:
 
-`gatt_server <https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/gatt_server>`_ 
+:example:`bluetooth/gatt_server` 
 
-  This is a gatt server demo. Use gatt api to create a gatt server with send advertising. This gatt server can be connected and the service can be discovery.
+  This is a GATT server demo. Use GATT API to create a GATT server with send advertising. This GATT server can be connected and the service can be discovery.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `bt/bluedroid/api/include/esp_gatts_api.h <https://github.com/espressif/esp-idf/blob/master/components/bt/bluedroid/api/include/esp_gatts_api.h>`_
+  * :component_file:`bt/bluedroid/api/include/esp_gatts_api.h`
 
 Macros
 ^^^^^^

diff --git a/docs/api/bluetooth/index.rst b/docs/api/bluetooth/index.rst
@@ -9,4 +9,4 @@ Bluetooth API
    Bluetooth LE <bt_le>
 
 
-Example code for this API section is provided in `examples/bluetooth <https://github.com/espressif/esp-idf/tree/master/examples/bluetooth>`_ directory of ESP-IDF repository.
+Example code for this API section is provided in :example:`bluetooth` directory of ESP-IDF examples.
diff --git a/docs/api/ethernet/esp_eth.rst b/docs/api/ethernet/esp_eth.rst
@@ -4,15 +4,15 @@ ETHERNET
 Application Example
 -------------------
 
-ethernet example: `examples/ethernet/ethernet <https://github.com/espressif/esp-idf/tree/master/examples/ethernet/ethernet>`_.
+Ethernet example: :example:`ethernet/ethernet`.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `components/ethernet/include/esp_eth.h <https://github.com/espressif/esp-idf/blob/master/components/ethernet/include/esp_eth.h>`_
+  * :component_file:`ethernet/include/esp_eth.h`
 
 Macros
 ^^^^^^

diff --git a/docs/api/ethernet/index.rst b/docs/api/ethernet/index.rst
@@ -7,4 +7,4 @@ Ethernet API
    Ethernet <esp_eth>
 
 
-Example code for this API section is provided in `examples/ethernet <https://github.com/espressif/esp-idf/tree/master/examples/ethernet>`_ directory of ESP-IDF repository.
+Example code for this API section is provided in :example:`ethernet` directory of ESP-IDF examples.
diff --git a/docs/api/peripherals/gpio.rst b/docs/api/peripherals/gpio.rst
@@ -10,15 +10,15 @@ Note that GPIO6-11 are usually used for SPI flash. GPIO34-39 can only be set as
 Application Example
 -------------------
 
-GPIO output and input interrupt example: `examples/peripherals/gpio <https://github.com/espressif/esp-idf/tree/master/examples/peripherals/gpio>`_.
+GPIO output and input interrupt example: :example:`peripherals/gpio`.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `driver/include/driver/driver/gpio.h <https://github.com/espressif/esp-idf/blob/master/components/driver/include/driver/gpio.h>`_
+  * :component_file:`driver/include/driver/gpio.h`
 
 Macros
 ^^^^^^

diff --git a/docs/api/peripherals/i2c.rst b/docs/api/peripherals/i2c.rst
@@ -9,15 +9,15 @@ ESP32 has two I2C controllers which can be set as master mode or slave mode.
 Application Example
 -------------------
 
-I2C master and slave example: `examples/peripherals/i2c <https://github.com/espressif/esp-idf/tree/master/examples/peripherals/i2c>`_.
+I2C master and slave example: :example:`peripherals/i2c`.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `driver/include/driver/i2c.h <https://github.com/espressif/esp-idf/blob/master/components/driver/include/driver/i2c.h>`_
+  * :component_file:`driver/include/driver/i2c.h`
 
 Macros
 ^^^^^^

diff --git a/docs/api/peripherals/index.rst b/docs/api/peripherals/index.rst
@@ -15,4 +15,4 @@ Peripherals API
    Remote Control <rmt>
 
 
-Example code for this API section is provided in `examples/peripherals <https://github.com/espressif/esp-idf/tree/master/examples/peripherals>`_ directory of ESP-IDF repository.
+Example code for this API section is provided in :example:`peripherals` directory of ESP-IDF examples.
diff --git a/docs/api/peripherals/ledc.rst b/docs/api/peripherals/ledc.rst
@@ -12,15 +12,15 @@ decrease the duty cycle gradually, allowing for fades without any processor inte
 Application Example
 -------------------
 
-LEDC change duty cycle and fading control example: `examples/peripherals/ledc <https://github.com/espressif/esp-idf/tree/master/examples/peripherals/ledc>`_.
+LEDC change duty cycle and fading control example: :example:`peripherals/ledc`.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `driver/include/driver/ledc.h <https://github.com/espressif/esp-idf/blob/master/components/driver/include/driver/ledc.h>`_
+  * :component_file:`driver/include/driver/ledc.h`
 
 Macros
 ^^^^^^

diff --git a/docs/api/peripherals/pcnt.rst b/docs/api/peripherals/pcnt.rst
@@ -9,15 +9,15 @@ The PCNT (Pulse Counter) module is designed to count the number of rising and/or
 Application Example
 -------------------
 
-Pulse counter with control signal and event interrupt example: `examples/peripherals/pcnt <https://github.com/espressif/esp-idf/tree/master/examples/peripherals/pcnt>`_.
+Pulse counter with control signal and event interrupt example: :example:`peripherals/pcnt`.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `driver/pcnt.h <https://github.com/espressif/esp-idf/blob/master/components/driver/include/driver/pcnt.h>`_
+  * :component_file:`driver/include/driver/pcnt.h`
 
 
 Macros

diff --git a/docs/api/peripherals/rmt.rst b/docs/api/peripherals/rmt.rst
@@ -9,15 +9,15 @@ The RMT (Remote Control) module driver can be used to send and receive infrared
 Application Example
 -------------------
 
-NEC remote control TX and RX example: `examples/peripherals/rmt_nec_tx_rx <https://github.com/espressif/esp-idf/tree/master/examples/peripherals/rmt_nec_tx_rx>`_.
+NEC remote control TX and RX example: :example:`peripherals/rmt_nec_tx_rx`.
 
 API Reference
 -------------
 
 Header Files
 ^^^^^^^^^^^^
 
-  * `driver/rmt.h <https://github.com/espressif/esp-idf/blob/master/components/driver/include/driver/rmt.h>`_
+  * :component_file:`driver/include/driver/rmt.h`
 
 Macros
 ^^^^^^
Original file line number	Diff line number	Diff line change
Expand Up		@@ -9,4 +9,4 @@ Bluetooth API
		Bluetooth LE <bt_le>


		Example code for this API section is provided in `examples/bluetooth <https://github.com/espressif/esp-idf/tree/master/examples/bluetooth>`_ directory of ESP-IDF repository.
		Example code for this API section is provided in :example:`bluetooth` directory of ESP-IDF examples.
Original file line number	Diff line number	Diff line change
Expand Up		@@ -7,4 +7,4 @@ Ethernet API
		Ethernet <esp_eth>


		Example code for this API section is provided in `examples/ethernet <https://github.com/espressif/esp-idf/tree/master/examples/ethernet>`_ directory of ESP-IDF repository.
		Example code for this API section is provided in :example:`ethernet` directory of ESP-IDF examples.
Original file line number	Diff line number	Diff line change
Expand Up		@@ -15,4 +15,4 @@ Peripherals API
		Remote Control <rmt>


		Example code for this API section is provided in `examples/peripherals <https://github.com/espressif/esp-idf/tree/master/examples/peripherals>`_ directory of ESP-IDF repository.
		Example code for this API section is provided in :example:`peripherals` directory of ESP-IDF examples.