Replace OpenAI Gym with Gymnasium and fix box-2d installation bug

18_reinforcement_learning.ipynb

@@ -170,9 +170,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Let's install the gym library, which provides many environments for Reinforcement Learning. We'll also install the extra libraries needed for classic control environments (including CartPole, which we will use shortly), as well as for Box2D and Atari environments, which are needed for the exercises.\n",
+    "Let's install the Gymnasium library, which provides many environments for Reinforcement Learning. We'll also install the extra libraries needed for classic control environments (including CartPole, which we will use shortly), as well as for Box2D and Atari environments, which are needed for the exercises.\n",
     "\n",
-    "**Note:** by running the following cell, you accept the Atari ROM license."
+    "**Important notes:**\n",
+    "* OpenAI has handed over the maintenance and developmnent of the Gym library to the Farama foundation (see the [announcement](https://farama.org/Announcing-The-Farama-Foundation)), and the library was renamed to Gymnasium. It's a drop-in replacement for OpenAI Gym: you can just install `gymnasium` instead of `gym`, and `import gymnasium as gym`, and everything should work fine.\n",
+    "* by running the following cell, you accept the Atari ROM license."
    ]
   },
   {
@@ -182,22 +184,22 @@
    "outputs": [],
    "source": [
     "if \"google.colab\" in sys.modules or \"kaggle_secrets\" in sys.modules:\n",
-    "    %pip install -q -U gym\n",
-    "    %pip install -q -U gym[classic_control,box2d,atari,accept-rom-license]"
+    "    %pip install -q -U gymnasium swig\n",
+    "    %pip install -q -U gymnasium[classic_control,box2d,atari,accept-rom-license]"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Introduction to OpenAI gym"
+    "# Introduction to ~OpenAI gym~ Gymnasium"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this notebook we will be using [OpenAI gym](https://gym.openai.com/), a great toolkit for developing and comparing Reinforcement Learning algorithms. It provides many environments for your learning *agents* to interact with. Let's import Gym and make a new CartPole environment:"
+    "In this notebook we will be using [gymnasium](https://github.com/Farama-Foundation/Gymnasium), a great toolkit for developing and comparing Reinforcement Learning algorithms. It provides many environments for your learning *agents* to interact with. Let's import Gym and make a new CartPole environment:"
    ]
   },
   {
@@ -206,7 +208,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import gym\n",
+    "import gymnasium as gym\n",
     "\n",
     "env = gym.make(\"CartPole-v1\", render_mode=\"rgb_array\")"
    ]
@@ -2195,7 +2197,7 @@
    "metadata": {},
    "source": [
     "## 8.\n",
-    "_Exercise: Use policy gradients to solve OpenAI Gym's LunarLander-v2 environment._"
+    "_Exercise: Use policy gradients to solve ~OpenAI Gym~ Gymnasium's LunarLander-v2 environment._"
    ]
   },
   {
@@ -2269,7 +2271,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "From the [source code](https://github.com/openai/gym/blob/master/gym/envs/box2d/lunar_lander.py), we can see that these each 8D observation (x, y, h, v, a, w, l, r) correspond to:\n",
+    "From the [source code](https://github.com/Farama-Foundation/Gymnasium/blob/main/gymnasium/envs/box2d/lunar_lander.py), we can see that these each 8D observation (x, y, h, v, a, w, l, r) correspond to:\n",
     "* x,y: the coordinates of the spaceship. It starts at a random location near (0, 1.4) and must land near the target at (0, 0).\n",
     "* h,v: the horizontal and vertical speed of the spaceship. It starts with a small random speed.\n",
     "* a,w: the spaceship's angle and angular velocity.\n",
@@ -2307,7 +2309,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Looking at the [LunarLander-v2's description](https://gym.openai.com/envs/LunarLander-v2/), these actions are:\n",
+    "Looking at the [LunarLander-v2's description](https://gymnasium.farama.org/environments/box2d/lunar_lander/), these actions are:\n",
     "* do nothing\n",
     "* fire left orientation engine\n",
     "* fire main engine\n",
@@ -2624,7 +2626,7 @@
    "hash": "95c485e91159f3a8b550e08492cb4ed2557284663e79130c96242e7ff9e65ae1"
   },
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -2638,7 +2640,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.10"
+   "version": "3.10.13"
   }
  },
  "nbformat": 4,