Rllg branch (huawei-noah#68)

* Add dosctrings, types and docker

* Add dosctrings, types and docker

.gitignore

@@ -1 +1,2 @@
 /.idea/
+*.DS_Store

RLLG/.dockerignore

@@ -0,0 +1,9 @@
+__pycache__/
+ray_results
+.idea/
+logs/
+.pytype/
+.vscode/
+# ignore for docker builds
+.git/
+.mypy_cache/

RLLG/LICENSE

@@ -1,21 +1,21 @@
-MIT License
-
-Copyright (c) 2023 Paul Daoudi
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+MIT License
+
+Copyright (c) 2023 Paul Daoudi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.

RLLG/README.md

@@ -1,49 +1,57 @@
-# Enhancing Reinforcement Learning agents with Local Guides 
-
-This is the official implementation of the techniques discussed in the paper [Enhancing Reinforcement Learning agents with Local Guides](https://hal.science/hal-04052358/file/Final_Reinforcement_Learning_with_Local_Guides.pdf).
-
-## Steps to launch it for a new environment
-
-- In the folder `envs`, create a folder with the name of the environment with 3 files:
-  - `create_env_name` to create the environment
-  - `local_expert_policy` for the local expert
-  - `confidence` for the confidence function $\lambda$
-- Add the environment in the global files `creation` and `confidence` in `envs`
-- Add a config file in `ray_config`
-- Modify the `main` file to include the new environment
-- Enjoy :)
-
-## Notes regarding the Point-Reach environment
-
-PointReach is based on [Bullet-Safety-Gym](https://github.com/SvenGronauer/Bullet-Safety-Gym), and has been modified internally (directly in their source code) to make it more difficult.
-
-All the details can be found in Appendix B of the paper.
-
-## Visualization
-
-All the results are saved in a ray tune `Experimentanalysis`. You can plot them in the `Visualization.ipynb` notebook.
-
-## License
-
-We follow MIT License. Please see the [License](./LICENSE) file for more information.
-
-**Disclaimer:** This is not an officially supported Huawei Product.
-
-
-## Credits
-
-This code is built upon the [SimpleSAC Github](https://github.com/young-geng/SimpleSAC), and some wrappers of [gym](https://github.com/openai/gym/tree/master).
-
-
-## Cite us
-
-If you find this technique useful and you use it in a project, please cite it:
-```
-@inproceedings{daoudi2023enhancing,
-  title={Enhancing Reinforcement Learning Agents with Local Guides},
-  author={Daoudi, Paul and Robu, Bogdan and Prieur, Christophe and Dos Santos, Ludovic and Barlier, Merwan},
-  booktitle={Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems},
-  pages={829--838},
-  year={2023}
-}
-```
+# Enhancing Reinforcement Learning agents with Local Guides 
+
+This is the official implementation of the techniques discussed in the paper [Enhancing Reinforcement Learning agents with Local Guides](https://hal.science/hal-04052358/file/Final_Reinforcement_Learning_with_Local_Guides.pdf).
+
+## Create the conda virtual environment
+
+```
+conda create --name rllg python=3.8
+conda activate rllg
+pip install -e .
+```
+
+## Steps to launch it for a new environment
+
+- In the folder `envs`, create a folder with the name of the environment with 3 files:
+  - `create_env_name` to create the environment
+  - `local_expert_policy` for the local expert
+  - `confidence` for the confidence function $\lambda$
+- Add the environment in the global files `creation` and `confidence` in `envs`
+- Add a config file in `ray_config`
+- Modify the `main` file to include the new environment
+- Enjoy :)
+
+## Notes regarding the Point-Reach environment
+
+PointReach is based on [Bullet-Safety-Gym](https://github.com/SvenGronauer/Bullet-Safety-Gym), and has been modified internally (directly in their source code) to make it more difficult.
+
+All the details can be found in Appendix B of the paper.
+
+## Visualization
+
+All the results are saved in a ray tune `Experimentanalysis`. You can plot them in the `Visualization.ipynb` notebook.
+
+## License
+
+We follow MIT License. Please see the [License](./LICENSE) file for more information.
+
+**Disclaimer:** This is not an officially supported Huawei Product.
+
+
+## Credits
+
+This code is built upon the [SimpleSAC Github](https://github.com/young-geng/SimpleSAC), and some wrappers of [gym](https://github.com/openai/gym/tree/master).
+
+
+## Cite us
+
+If you find this technique useful and you use it in a project, please cite it:
+```
+@inproceedings{daoudi2023enhancing,
+  title={Enhancing Reinforcement Learning Agents with Local Guides},
+  author={Daoudi, Paul and Robu, Bogdan and Prieur, Christophe and Dos Santos, Ludovic and Barlier, Merwan},
+  booktitle={Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems},
+  pages={829--838},
+  year={2023}
+}
+```