[air/docs] Refactor AIR documentation -- update user guides (2/n) (ra…

…y-project#37742)
lmco · Jul 25, 2023 · feca9b4 · feca9b4
1 parent 5f1e274
commit feca9b4
Show file tree

Hide file tree

Showing 43 changed files with 376 additions and 755 deletions.
diff --git a/doc/source/_toc.yml b/doc/source/_toc.yml
@@ -45,15 +45,6 @@ parts:
         title: "Ray AI Runtime (AIR)"
         sections:
           - file: ray-air/key-concepts
-          - file: ray-air/user-guides
-            sections:
-              - file: ray-air/preprocessors
-              - file: ray-air/trainers
-              - file: ray-air/check-ingest
-              - file: ray-air/tuner
-              - file: ray-air/predictors
-              - file: ray-air/computer-vision
-              - file: ray-air/deployment
           - file: ray-air/api/api
           - file: ray-air/benchmarks
 
@@ -82,10 +73,17 @@ parts:
                 title: "Configuring Ray Train"
               - file: train/dl_guide
                 title: "Deep Learning Guide"
+              - file: train/hf_trainers
+                title: "Hugging Face Trainers"
               - file: train/gbdt
                 title: "XGBoost/LightGBM guide"
               - file: train/architecture
                 title: "Ray Train Architecture"
+              - file: train/train-with-tune
+                title: "Using Ray Train with Ray Tune"
+              - file: train/check-ingest
+                title: "Configuring Training Datasets"
+              - file: train/predictors
           - file: train/examples
             title: "Examples"
             sections:

diff --git a/doc/source/ray-air/doc_code/preprocessors.py → doc/source/data/doc_code/preprocessors.py b/doc/source/ray-air/doc_code/preprocessors.py → doc/source/data/doc_code/preprocessors.py
diff --git a/doc/source/data/images/air-preprocessor.svg b/doc/source/data/images/air-preprocessor.svg
diff --git a/doc/source/data/images/preprocessors.svg b/doc/source/data/images/preprocessors.svg
diff --git a/doc/source/ray-air/preprocessors.rst → doc/source/data/preprocessors.rst b/doc/source/ray-air/preprocessors.rst → doc/source/data/preprocessors.rst
@@ -15,7 +15,7 @@ Ray AIR provides several common preprocessors out of the box and interfaces to d
 Overview
 --------
 
-The most common way of using a preprocessor is by passing it as an argument to the constructor of a :ref:`Trainer <air-trainers>` in conjunction with a :ref:`Ray Data <data>`.
+The most common way of using a preprocessor is by passing it as an argument to the constructor of a Ray Train :ref:`Trainer <train-getting-started>` in conjunction with a :ref:`Ray Data dataset <data>`.
 For example, the following code trains a model with a preprocessor that normalizes the data.
 
 .. literalinclude:: doc_code/preprocessors.py
@@ -119,7 +119,7 @@ Predictor
 A ``Predictor`` can be constructed from a saved ``Checkpoint``. If the ``Checkpoint`` contains a ``Preprocessor``,
 then the ``Preprocessor`` calls ``transform_batch`` on input batches prior to performing inference.
 
-In the following example, we show the Batch Predictor flow. The same logic applies to the :ref:`Online Inference flow <air-key-concepts-online-inference>`.
+In the following example, we show the Batch Predictor flow.
 
 .. literalinclude:: doc_code/preprocessors.py
     :language: python

diff --git a/doc/source/data/user-guide.rst b/doc/source/data/user-guide.rst
@@ -23,3 +23,4 @@ show you how achieve several tasks.
     working-with-pytorch
     batch_inference
     performance-tips
+    preprocessors
diff --git a/doc/source/ray-air/api/checkpoint.rst b/doc/source/ray-air/api/checkpoint.rst
@@ -1,6 +1,7 @@
-.. _air-checkpoint-ref:
 .. _checkpoint-api-ref:
 
+.. _air-checkpoint-ref:
+
 Ray AIR Checkpoint
 ==================
 

diff --git a/doc/source/ray-air/computer-vision.rst b/doc/source/ray-air/computer-vision.rst
@@ -1,3 +1,5 @@
+:orphan:
+
 .. _computer-vision:
 
 Computer Vision
@@ -183,8 +185,7 @@ Training vision models
             :end-before: __torch_trainer_stop__
             :dedent:
 
-        For more in-depth examples, see
-        :ref:`Using Trainers <air-trainers>`.
+        For more in-depth examples, see :ref:`Using Trainers <train-getting-started>`.
 
     .. tab-item:: TensorFlow
 
@@ -203,7 +204,7 @@ Training vision models
             :end-before: __tensorflow_trainer_stop__
             :dedent:
 
-        For more information, read :ref:`Using Trainers <air-trainers>`.
+        For more information, read :ref:`Using Trainers <train-getting-started>`.
 
 Creating checkpoints
 --------------------

diff --git a/doc/source/ray-air/deployment.rst b/doc/source/ray-air/deployment.rst
@@ -1,3 +1,5 @@
+:orphan:
+
 .. _air-deployment:
 
 How to Deploy AIR

diff --git a/doc/source/ray-air/key-concepts.rst b/doc/source/ray-air/key-concepts.rst
@@ -35,7 +35,7 @@ Trainers
 
 Trainers are wrapper classes around third-party training frameworks such as XGBoost and Pytorch. They are built to help integrate with core Ray actors (for distribution), Ray Tune, and Ray Data.
 
-See the documentation on :ref:`Trainers <air-trainers>`.
+See the documentation on :ref:`Trainers <train-getting-started>`.
 
 .. literalinclude:: doc_code/air_key_concepts.py
     :language: python
@@ -54,7 +54,7 @@ These objects contain training metrics as well as checkpoints to retrieve the be
 Tuner
 -----
 
-:ref:`Tuners <air-tuner-ref>` offer scalable hyperparameter tuning as part of :ref:`Ray Tune <tune-main>`.
+:ref:`Tuners <tune-tutorial>` offer scalable hyperparameter tuning as part of :ref:`Ray Tune <tune-main>`.
 
 Tuners can work seamlessly with any Trainer but also can support arbitrary training functions.
 

diff --git a/doc/source/ray-air/trainers.rst b/doc/source/ray-air/trainers.rst