01_get_piles.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1 - Downloading The Pile - Deduplicated and MiniPile\n",
    "\n",
    "**Objectives:**\n",
    "- [x] Download \"The Pile - Deduplicated\" to a specified directory\n",
    "- [x] Download \"MiniPile\" to a specified directory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to download the datasets [EleutherAI/the_pile_deduplicated](https://huggingface.co/datasets/EleutherAI/the_pile_deduplicated) and [JeanKaddour/minipile](https://huggingface.co/datasets/JeanKaddour/minipile) through HuggingFace to a specific directory.<br>\n",
    "HuggingFace can be a bit stubborn in that it loads the datasets and their caches to some default directory.<br>\n",
    "We can't do that here, so we need to implement a custom download function based on the HuggingFace API.<br>\n",
    "Thankfully, the HuggingFace-Hub's `snapshot_download` function performs much of the heavy lifting for us in that regard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from pathlib import Path\n",
    "from huggingface_hub import snapshot_download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "down_dir = \"/vol/tmp/koppelmm\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def download_dataset(down_dir: str, target_folder: str, cache_folder: str, repo_id: str) -> None:\n",
    "    # Download a dataset without affecting local cache. \n",
    "    # Download genuinely only to target_folder.\n",
    "    down_dir = Path(down_dir)\n",
    "    target_dir = down_dir / target_folder\n",
    "    cache_dir = down_dir / cache_folder\n",
    "\n",
    "    os.makedirs(target_dir, exist_ok=True)\n",
    "    os.makedirs(cache_dir, exist_ok=True)\n",
    "\n",
    "    print(f\"Downloading {repo_id}...\")\n",
    "\n",
    "    # I tried fiddling with os.environs, I wanted to use the load_dataset function\n",
    "    # but that's really not needed, snapshot_download suffices\n",
    "    while True:\n",
    "        try:\n",
    "            snapshot_download(repo_id, repo_type=\"dataset\", cache_dir=str(cache_dir), local_dir=str(target_dir))\n",
    "            break\n",
    "        except Exception as _:\n",
    "            continue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the download logic in place, we can go right ahead and download the datasets to their intended directories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "download_dataset(down_dir=down_dir, target_folder=\"Pile_Deduplicated\", \n",
    "                 cache_folder=\"Pile_Deduplicated_Cache\", repo_id=\"EleutherAI/the_pile_deduplicated\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[The Pile Deduplicated](https://huggingface.co/datasets/EleutherAI/the_pile_deduplicated) is stated to measure $451$ GB at $134,318,121$ rows.<br>\n",
    "The dataset only consists of a train split. Each row only consists of the value to the single attribute `text`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "download_dataset(down_dir=down_dir, target_folder=\"MiniPile\", \n",
    "                 cache_folder=\"MiniPile_Cache\", repo_id=\"JeanKaddour/minipile\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[MiniPile](https://huggingface.co/datasets/JeanKaddour/minipile) is stated to measure $3.18$ GB at $1,000,000:500:10,000$ train:val:test rows.<br>\n",
    "Again, each row only consists of the value to the single attribute `text`.\n",
    "\n",
    "MiniPile's training set is therefore $0.745\\%$ the size of The Pile Deduplicated, while retaining a memory footprint of $0.705\\%$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## (Some) Exploratory Data Analysis\n",
    "\n",
    "For comparison, and mostly for fun.<br>\n",
    "I stumbled across [PyGWalker](https://github.com/Kanaries/pygwalker) and decided to give it a shot for looking a bit further into training data from MiniPile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install matplotlib seaborn wordcloud"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from datasets import load_dataset\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from collections import Counter\n",
    "from wordcloud import WordCloud\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAIjCAYAAAA0vUuxAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAVdBJREFUeJzt3Xl0VPX9//HXTEJWCIshCWAgyCo7hophEZEgIqViW0VECRRRFASJuKAFVNQgCoVfRRGUxRWkRVoVQQwEBRFkCYogOwaRBPgCCQmQQOb+/tAMTBNIBjLcT8jzcc6cdu7ce+ed5PaU1/l8Pu+Pw7IsSwAAAACA83LaXQAAAAAAmI7gBAAAAADFIDgBAAAAQDEITgAAAABQDIITAAAAABSD4AQAAAAAxSA4AQAAAEAxCE4AAAAAUAyCEwAAAAAUg+AEAJAkpaSkyOFwKCUlxe5Syoy9e/fK4XBo9uzZPv+u2bNny+FwaO/eve5jMTEx+uMf/+jz75Z4PgCA4AQAl9FHH30kh8Ohjz/+uNBnLVu2lMPh0PLlywt9Vrt2bbVr1+5ylFhiu3bt0oMPPqhrrrlGQUFBCgsLU/v27TVlyhSdPHnS7vIkSa+//rpXocbhcLhf/v7+qlatmmJjYzV8+HBt2bLFtrouJ5NrAwA7+dtdAACUJx06dJAkrVy5UnfccYf7eFZWljZv3ix/f3+tWrVKnTt3dn+2b98+7du3T3ffffdlr/d8PvvsM915550KDAxUv3791KxZM+Xl5WnlypV6/PHH9eOPP2r69Ol2l6nXX39d4eHh6t+/f4mv6dq1q/r16yfLspSZmalNmzZpzpw5ev311/Xyyy8rMTHRfW6dOnV08uRJVahQwed13Xfffbr77rsVGBjo1Xd563y13XjjjTp58qQCAgJ8+v0AYCqCEwBcRjVr1lTdunW1cuVKj+OrV6+WZVm68847C31W8L4gdF0sy7J06tQpBQcHX9J99uzZo7vvvlt16tTRsmXLVKNGDfdnQ4YM0c6dO/XZZ59d0nfYqWHDhrr33ns9jo0fP149e/bUY489psaNG+u2226T9NsIVVBQkE/rycnJUWhoqPz8/OTn5+fT77oQp9Pp858VAEzGVD0AuMw6dOigjRs3ekxnW7VqlZo2baru3bvr22+/lcvl8vjM4XCoffv2kqQzZ85o3LhxqlevngIDAxUTE6Onn35aubm5Ht9TsP5lyZIlatOmjYKDg/Xmm29Kkn755Rf16tVLoaGhioiI0IgRIwpdfz4TJkxQdna23n77bY/QVKB+/foaPny4+31J63U4HHr22WcL3S8mJsZj9KNgrc+qVauUmJio6tWrKzQ0VHfccYcOHTrkcd2PP/6oFStWuKff3XTTTSX6Gf/XVVddpblz58rf318vvvii+3hRa5zS09M1YMAAXX311QoMDFSNGjV0++23u9cmXaiugp9txYoVevjhhxUREaGrr77a47Nz1zgV+OKLL9SqVSsFBQWpSZMmWrBggcfnzz77rBwOR6Hr/veeF6rtfGuc5s+fr9jYWAUHBys8PFz33nuv9u/f73FO//79VbFiRe3fv1+9evVSxYoVVb16dY0cOVL5+fnF/PYBwAyMOAHAZdahQwe9++67WrNmjfsfpatWrVK7du3Url07ZWZmavPmzWrRooX7s8aNG+uqq66SJN1///2aM2eO/vrXv+qxxx7TmjVrlJSUpK1btxZaO7Vt2zb16dNHDz74oAYNGqRGjRrp5MmT6tKli9LS0jRs2DDVrFlT7777rpYtW1ai+j/55BNdc801JV5z5U293njkkUdUtWpVjR07Vnv37tXkyZM1dOhQzZs3T5I0efJkPfLII6pYsaKeeeYZSVJkZORFf1/t2rXVqVMnLV++XFlZWQoLCyvyvL/85S/68ccf9cgjjygmJkYHDx7U0qVLlZaWppiYmBLV9fDDD6t69eoaM2aMcnJyLljXjh071Lt3bw0ePFgJCQmaNWuW7rzzTi1evFhdu3b16mf09nc2e/ZsDRgwQH/4wx+UlJSkjIwMTZkyRatWrdLGjRtVpUoV97n5+fnq1q2b2rZtq1dffVVffvmlJk6cqHr16umhhx7yqk4AsIUFALisfvzxR0uSNW7cOMuyLOv06dNWaGioNWfOHMuyLCsyMtKaOnWqZVmWlZWVZfn5+VmDBg2yLMuyUlNTLUnW/fff73HPkSNHWpKsZcuWuY/VqVPHkmQtXrzY49zJkydbkqyPPvrIfSwnJ8eqX7++Jclavnz5eWvPzMy0JFm33357iX5Wb+qVZI0dO7bQPerUqWMlJCS438+aNcuSZMXHx1sul8t9fMSIEZafn5917Ngx97GmTZtanTp1KlGtBTUMGTLkvJ8PHz7ckmRt2rTJsizL2rNnjyXJmjVrlmVZlnX06FFLkvXKK69c8HvOV1fBz9ahQwfrzJkzRX62Z88e97GCv/G///1v97HMzEyrRo0aVuvWrd3Hxo4daxX1f/lF3fN8tS1fvtzj+cjLy7MiIiKsZs2aWSdPnnSf9+mnn1qSrDFjxriPJSQkWJKs559/3uOerVu3tmJjYwt9FwCYiKl6AHCZXXvttbrqqqvca5c2bdqknJwc9whOu3bttGrVKkm/rX3Kz893r29atGiRJHk0KJCkxx57TJIKrS2qW7euunXr5nFs0aJFqlGjhv7617+6j4WEhOiBBx4otvasrCxJUqVKlUr0s3pbrzceeOABj+lnHTt2VH5+vn7++eeLvmdxKlasKEk6fvx4kZ8HBwcrICBAKSkpOnr06EV/z6BBg0q8nqlmzZoejUbCwsLUr18/bdy4Uenp6RddQ3HWrVungwcP6uGHH/ZY+9SjRw81bty4yL/t4MGDPd537NhRu3fv9lmNAFCaynVw+uqrr9SzZ0/VrFlTDodDCxcu9Or6gjnj//sKDQ31TcEArggOh0Pt2rVzr2VatWqVIiIiVL9+fUmewangPwuC088//yyn0+k+t0BUVJSqVKlSKDTUrVu30Pf//PPPql+/fqE1L40aNSq29oLpaecLDkV9lzf1eqN27doe76tWrSpJlxRYipOdnS3p/MExMDBQL7/8sj7//HNFRkbqxhtv1IQJE7wOMEX93c6nqL9lw4YNJanI9VClpeBvV9Rz07hx40J/26CgIFWvXt3jWNWqVX369wKA0lSug1NOTo5atmypqVOnXtT1I0eO1IEDBzxeTZo00Z133lnKlQK40nTo0EGZmZn64Ycf3OubCrRr104///yz9u/fr5UrV6pmzZq65pprPK4vaqF/US61g97/CgsLU82aNbV582avritpvUU5X/OA843IWJZ10d9VnM2bN8vPz++CwebRRx/V9u3blZSUpKCgII0ePVrXXnutNm7cWOLvKe2/2/l+/5ezMYOdHQEBoDSU6+DUvXt3vfDCCx5THM6Vm5urkSNHqlatWgoNDVXbtm09uglVrFhRUVFR7ldGRoa2bNmigQMHXqafAEBZde5+TqtWrXJ3zJOk2NhYBQYGKiUlRWvWrPH4rE6dOnK5XNqxY4fH/TIyMnTs2DHVqVOn2O+uU6eOdu3aVShgbNu2rUS1//GPf9SuXbu0evXqEn1XSeutWrWqjh075nFeXl6eDhw4UKK6inIpge1/paWlacWKFYqLiyt2qmK9evX02GOP6YsvvtDmzZuVl5eniRMn+qSunTt3Fvpbbt++XdJvXfKks6Nx//v7LWrEr6S1Ffztinputm3bVqJnEQDKknIdnIozdOhQrV69WnPnztX333+vO++8U7feemuhfwAUeOutt9SwYUN17NjxMlcKoKxp06aNgoKC9P7772v//v0eI06BgYG67rrrNHXqVOXk5Hjs31Swf9DkyZM97jdp0iRJv60vKc5tt92mX3/9Vf/617/cx06cOFHiDWufeOIJhYaG6v7771dGRkahz3ft2qUpU6Z4XW+9evX01VdfeZw3ffr0SxoVCQ0NLRQWLsaRI0fUp08f5efnu7vNFeXEiRM6deqUx7F69eqpUqVKHu3XS6suSfr11189uhNmZWXpnXfeUatWrRQVFeWuQZLH7zcnJ0dz5swpdL+S1tamTRtFRERo2rRpHj/b559/rq1bt5boWQSAsoR25OeRlpamWbNmKS0tTTVr1pT029S8xYsXa9asWXrppZc8zj916pTef/99PfXUU3aUC6CMCQgI0B/+8Ad9/fXXCgwMVGxsrMfn7dq1c49QnBucWrZsqYSEBE2fPl3Hjh1Tp06dtHbtWs2ZM0e9evVS586di/3uQYMG6bXXXlO/fv20fv161ahRQ++++65CQkJKVHu9evX0wQcfqHfv3rr22mvVr18/NWvWTHl5efrmm280f/58975L3tR7//33a/DgwfrLX/6irl27atOmTVqyZInCw8NLVFdRYmNj9cYbb+iFF15Q/fr1FRERoZtvvvmC12zfvl3vvfeeLMtSVlaWNm3apPnz5ys7O1uTJk3SrbfeesFru3TporvuuktNmjSRv7+/Pv74Y2VkZOjuu+++pLrOp2HDhho4cKC+++47RUZGaubMmcrIyNCsWbPc59xyyy2qXbu2Bg4cqMcff1x+fn6aOXOmqlevrrS0tIv6nVWoUEEvv/yyBgwYoE6dOqlPnz7uduQxMTEaMWLERf08AGAse5v6mUOS9fHHH7vfF7RTDQ0N9Xj5+/tbd911V6HrP/jgA8vf399KT0+/jFUDKMtGjRplSbLatWtX6LMFCxZYkqxKlSoVakt9+vRp67nnnrPq1q1rVahQwYqOjrZGjRplnTp1yuO8OnXqWD169Cjyu3/++WfrT3/6kxUSEmKFh4dbw4cPtxYvXlxsO/Jzbd++3Ro0aJAVExNjBQQEWJUqVbLat29v/fOf//SopaT15ufnW08++aQVHh5uhYSEWN26dbN27tx53nbk3333ncf1/9su27IsKz093erRo4dVqVIlS1KxrckluV9Op9OqUqWK1bp1a2v48OHWjz/+WOj8/21HfvjwYWvIkCFW48aNrdDQUKty5cpW27ZtPVq/X6iu8/1s5372v+3Ie/ToYS1ZssRq0aKFFRgYaDVu3NiaP39+oevXr19vtW3b1goICLBq165tTZo0qch7nq+2on6/lmVZ8+bNs1q3bm0FBgZa1apVs/r27Wv98ssvHuckJCRYoaGhhWo6X5t0ADCRw7J8uIq2DHE4HPr444/Vq1cvSdK8efPUt29f/fjjj4UWtBasbTpXly5dFBYWdkmbOQIAAAAwE1P1zqN169bKz8/XwYMHi12ztGfPHi1fvlz//e9/L1N1AAAAAC6nch2csrOztXPnTvf7PXv2KDU1VdWqVVPDhg3Vt29f9evXTxMnTlTr1q116NAhJScnq0WLFh6LXmfOnKkaNWqoe/fudvwYAAAAAHysXE/VS0lJKXIhdUJCgmbPnq3Tp0/rhRde0DvvvKP9+/crPDxcN9xwg5577jk1b95ckuRyuVSnTh3169dPL7744uX+EQAAAABcBuU6OAEAAABASbCPEwAAAAAUg+AEAAAAAMUod80hXC6Xfv31V1WqVEkOh8PucgAAAADYxLIsHT9+XDVr1pTTeeExpXIXnH799VdFR0fbXQYAAAAAQ+zbt09XX331Bc8pd8GpUqVKkn775YSFhdlcDQAAAAC7ZGVlKTo62p0RLqTcBaeC6XlhYWEEJwAAAAAlWsJDcwgAAAAAKAbBCQAAAACKQXACAAAAgGIQnAAAAACgGAQnAAAAACgGwQkAAAAAikFwAgAAAIBiEJwAAAAAoBgEJwAAAAAoBsEJAAAAAIpBcAIAAACAYtganL766iv17NlTNWvWlMPh0MKFC0t87apVq+Tv769WrVr5rD4AAAAAkGwOTjk5OWrZsqWmTp3q1XXHjh1Tv3791KVLFx9VBgAAAABn+dv55d27d1f37t29vm7w4MG655575Ofn59UoFQAAAABcjDK3xmnWrFnavXu3xo4dW6Lzc3NzlZWV5fECAAAAAG+UqeC0Y8cOPfXUU3rvvffk71+ywbKkpCRVrlzZ/YqOjvZxlQAAAACuNGUmOOXn5+uee+7Rc889p4YNG5b4ulGjRikzM9P92rdvnw+rBAAAAHAlsnWNkzeOHz+udevWaePGjRo6dKgkyeVyybIs+fv764svvtDNN99c6LrAwEAFBgZe7nJL5Kf0LO05lKOY8FBdWyPM7nIAAAAAnEeZCU5hYWH64YcfPI69/vrrWrZsmf71r3+pbt26NlV28RZs2K/pX+3WAzdeQ3ACAAAADGZrcMrOztbOnTvd7/fs2aPU1FRVq1ZNtWvX1qhRo7R//3698847cjqdatasmcf1ERERCgoKKnS8rHA4fvtPl8uytxAAAAAAF2RrcFq3bp06d+7sfp+YmChJSkhI0OzZs3XgwAGlpaXZVZ7POX9PTuQmAAAAwGwOy7LK1T/bs7KyVLlyZWVmZioszN7pca8s+UlTl+9S/3YxevZPTW2tBQAAAChvvMkGZaar3pXIzz3iVK6yKwAAAFDmEJxs5CA4AQAAAGUCwclGrHECAAAAygaCk42cv3fVK2fLzAAAAIAyh+BkI+fvycnlsrkQAAAAABdEcLKRex8nRpwAAAAAoxGcbMQaJwAAAKBsIDjZiDVOAAAAQNlAcLKRk3bkAAAAQJlAcLKRg6l6AAAAQJlAcLJRwVS9fEacAAAAAKMRnGzk93tyYo0TAAAAYDaCk43cU/XYxwkAAAAwGsHJRk72cQIAAADKBIKTjdjHCQAAACgbCE42Yh8nAAAAoGwgONnIwT5OAAAAQJlAcLIRU/UAAACAsoHgZCOaQwAAAABlA8HJRgUjTuQmAAAAwGwEJxs5GHECAAAAygSCk40KRpzyWeQEAAAAGI3gZCM/J1P1AAAAgLKA4GQjmkMAAAAAZQPByUbs4wQAAACUDQQnG7GPEwAAAFA2EJxsVDBVz2LECQAAADAawclGjDgBAAAAZQPByUbs4wQAAACUDQQnGzHiBAAAAJQNBCcbFQQn1jgBAAAAZiM42Yh9nAAAAICygeBko4J9nPKZqwcAAAAYjeBkIz9nwVQ9mwsBAAAAcEEEJxsxVQ8AAAAoGwhONnLQVQ8AAAAoEwhONmLECQAAACgbCE42OtuO3OZCAAAAAFwQwclGZzfAJTkBAAAAJiM42cjBVD0AAACgTCA42chJcwgAAACgTCA42cj5+2/fYsQJAAAAMBrByUaMOAEAAABlA8HJRgXtyPNJTgAAAIDRCE42oqseAAAAUDYQnGzEPk4AAABA2UBwshEjTgAAAEDZQHCyEfs4AQAAAGUDwclGTidd9QAAAICygOBko4KueuzjBAAAAJiN4GQj9nECAAAAygaCk41Y4wQAAACUDQQnG53bjpzpegAAAIC5bA1OX331lXr27KmaNWvK4XBo4cKFFzx/wYIF6tq1q6pXr66wsDDFxcVpyZIll6dYHygIThJ7OQEAAAAmszU45eTkqGXLlpo6dWqJzv/qq6/UtWtXLVq0SOvXr1fnzp3Vs2dPbdy40ceV+obzbG5SPskJAAAAMJa/nV/evXt3de/evcTnT5482eP9Sy+9pP/85z/65JNP1Lp16yKvyc3NVW5urvt9VlbWRdXqC85zkhPrnAAAAABzlek1Ti6XS8ePH1e1atXOe05SUpIqV67sfkVHR1/GCi+MqXoAAABA2VCmg9Orr76q7Oxs3XXXXec9Z9SoUcrMzHS/9u3bdxkrvLBzp+ox4gQAAACYy9apepfigw8+0HPPPaf//Oc/ioiIOO95gYGBCgwMvIyVldy5I07s5QQAAACYq0wGp7lz5+r+++/X/PnzFR8fb3c5F83BiBMAAABQJpS5qXoffvihBgwYoA8//FA9evSwu5xL4rHGyWVjIQAAAAAuyNYRp+zsbO3cudP9fs+ePUpNTVW1atVUu3ZtjRo1Svv379c777wj6bfpeQkJCZoyZYratm2r9PR0SVJwcLAqV65sy89wKTyn6jHiBAAAAJjK1hGndevWqXXr1u5W4omJiWrdurXGjBkjSTpw4IDS0tLc50+fPl1nzpzRkCFDVKNGDfdr+PDhttR/qWgOAQAAAJQNto443XTTTbIuEBhmz57t8T4lJcW3BV1mDppDAAAAAGVCmVvjdKUpGHW6UIAEAAAAYC+Ck838fk9O+QQnAAAAwFgEJ5sVTNdjqh4AAABgLoKTzQqm6rlITgAAAICxCE42K2hJzkw9AAAAwFwEJ5s53VP1SE4AAACAqQhONivoSE5wAgAAAMxFcLKZk+YQAAAAgPEITjZjHycAAADAfAQnmzHiBAAAAJiP4GQzB80hAAAAAOMRnGzmpDkEAAAAYDyCk838fk9OLpfNhQAAAAA4L4KTzdjHCQAAADAfwclm7OMEAAAAmI/gZDO66gEAAADmIzjZjH2cAAAAAPMRnGzGiBMAAABgPoKTzVjjBAAAAJiP4GQzuuoBAAAA5iM42awgOJGbAAAAAHMRnGzGVD0AAADAfAQnm9EcAgAAADAfwclmfr/3I3eRnAAAAABjEZxs5mSqHgAAAGA8gpPNHEzVAwAAAIxHcLIZI04AAACA+QhONjvbjpzgBAAAAJiK4GQzuuoBAAAA5iM42Yx9nAAAAADzEZxsxogTAAAAYD6Ck82cv/8FWOMEAAAAmIvgZLOzI04EJwAAAMBUBCebFezjlO+yuRAAAAAA50VwspkfzSEAAAAA4xGcbMY+TgAAAID5CE42c9BVDwAAADAewclmTqbqAQAAAMYjONmMfZwAAAAA8xGcbMY+TgAAAID5CE42c69xYsgJAAAAMBbByWZM1QMAAADMR3CyGc0hAAAAAPMRnGx2dh8nmwsBAAAAcF4EJ5v9npuUT3ICAAAAjEVwspmfe40TwQkAAAAwFcHJZkzVAwAAAMxHcLJZwT5OtCMHAAAAzEVwspmDduQAAACA8QhONqMdOQAAAGA+gpPNzq5xIjgBAAAApiI42czJVD0AAADAeLYGp6+++ko9e/ZUzZo15XA4tHDhwmKvSUlJ0XXXXafAwEDVr19fs2fP9nmdvuRgqh4AAABgPFuDU05Ojlq2bKmpU6eW6Pw9e/aoR48e6ty5s1JTU/Xoo4/q/vvv15IlS3xcqe8w4gQAAACYz9/OL+/evbu6d+9e4vOnTZumunXrauLEiZKka6+9VitXrtQ//vEPdevWzVdl+lRBcwjWOAEAAADmKlNrnFavXq34+HiPY926ddPq1avPe01ubq6ysrI8XiYpGHHKZ8gJAAAAMFaZCk7p6emKjIz0OBYZGamsrCydPHmyyGuSkpJUuXJl9ys6OvpylFpiTidT9QAAAADTlangdDFGjRqlzMxM92vfvn12l+SBfZwAAAAA89m6xslbUVFRysjI8DiWkZGhsLAwBQcHF3lNYGCgAgMDL0d5F4V9nAAAAADzXdKI06lTp0qrjhKJi4tTcnKyx7GlS5cqLi7ustZRmhx01QMAAACM53VwcrlcGjdunGrVqqWKFStq9+7dkqTRo0fr7bff9upe2dnZSk1NVWpqqqTf2o2npqYqLS1N0m/T7Pr16+c+f/Dgwdq9e7eeeOIJ/fTTT3r99df10UcfacSIEd7+GMZgqh4AAABgPq+D0wsvvKDZs2drwoQJCggIcB9v1qyZ3nrrLa/utW7dOrVu3VqtW7eWJCUmJqp169YaM2aMJOnAgQPuECVJdevW1WeffaalS5eqZcuWmjhxot56660y24pcYh8nAAAAoCzweo3TO++8o+nTp6tLly4aPHiw+3jLli31008/eXWvm2666YJre2bPnl3kNRs3bvTqe0zGPk4AAACA+bwecdq/f7/q169f6LjL5dLp06dLpajy5OwaJ4ITAAAAYCqvg1OTJk309ddfFzr+r3/9yz3lDiXHVD0AAADAfF5P1RszZowSEhK0f/9+uVwuLViwQNu2bdM777yjTz/91Bc1XtFoDgEAAACYz+sRp9tvv12ffPKJvvzyS4WGhmrMmDHaunWrPvnkE3Xt2tUXNV7R/H5PTi6GnAAAAABjXdQGuB07dtTSpUtLu5ZyiX2cAAAAAPN5PeL03Xffac2aNYWOr1mzRuvWrSuVosoTpuoBAAAA5vM6OA0ZMkT79u0rdHz//v0aMmRIqRRVnhQ0hyA3AQAAAObyOjht2bJF1113XaHjrVu31pYtW0qlqPKEEScAAADAfF4Hp8DAQGVkZBQ6fuDAAfn7X9SSqXKNfZwAAAAA83kdnG655RaNGjVKmZmZ7mPHjh3T008/TVe9i8A+TgAAAID5vB4ievXVV3XjjTeqTp067g1vU1NTFRkZqXfffbfUC7zSFUzVsxhxAgAAAIzldXCqVauWvv/+e73//vvatGmTgoODNWDAAPXp00cVKlTwRY1XNEacAAAAAPNd1KKk0NBQPfDAA6VdS7nkoDkEAAAAYLyLCk47duzQ8uXLdfDgQblcLo/PxowZUyqFlReMOAEAAADm8zo4zZgxQw899JDCw8MVFRXl7gon/dYhjuDkHb/fFzm5SE4AAACAsbwOTi+88IJefPFFPfnkk76op9xhqh4AAABgPq/bkR89elR33nmnL2opl5zs4wQAAAAYz+vgdOedd+qLL77wRS3lEmucAAAAAPN5PVWvfv36Gj16tL799ls1b968UAvyYcOGlVpx5QH7OAEAAADm8zo4TZ8+XRUrVtSKFSu0YsUKj88cDgfByUsORpwAAAAA43kdnPbs2eOLOsotJ80hAAAAAON5vcapQF5enrZt26YzZ86UZj3lDmucAAAAAPN5HZxOnDihgQMHKiQkRE2bNlVaWpok6ZFHHtH48eNLvcArnfP3vwBrnAAAAABzeR2cRo0apU2bNiklJUVBQUHu4/Hx8Zo3b16pFlceOGhHDgAAABjP6zVOCxcu1Lx583TDDTe4/9EvSU2bNtWuXbtKtbjywD1Vz2VzIQAAAADOy+sRp0OHDikiIqLQ8ZycHI8ghZLxY8QJAAAAMJ7XwalNmzb67LPP3O8LwtJbb72luLi40qusnKCrHgAAAGA+r6fqvfTSS+revbu2bNmiM2fOaMqUKdqyZYu++eabQvs6oXjs4wQAAACYz+sRpw4dOmjTpk06c+aMmjdvri+++EIRERFavXq1YmNjfVHjFY0RJwAAAMB8Xo04nT59Wg8++KBGjx6tGTNm+KqmcsX5e3IiNwEAAADm8mrEqUKFCvr3v//tq1rKJUacAAAAAPN5PVWvV69eWrhwoQ9KKZ/YxwkAAAAwn9fNIRo0aKDnn39eq1atUmxsrEJDQz0+HzZsWKkVVx6wjxMAAABgPq+D09tvv60qVapo/fr1Wr9+vcdnDoeD4OSlgql6FiNOAAAAgLG8Dk579uzxRR3llpN25AAAAIDxvF7jhNLloDkEAAAAYDyvR5z+9re/XfDzmTNnXnQx5ZGfk+YQAAAAgOm8Dk5Hjx71eH/69Glt3rxZx44d080331xqhZUXTNUDAAAAzOd1cPr4448LHXO5XHrooYdUr169UimqPGEfJwAAAMB8pbLGyel0KjExUf/4xz9K43blinsfJ4acAAAAAGOVWnOIXbt26cyZM6V1u3KjYKoeA04AAACAubyeqpeYmOjx3rIsHThwQJ999pkSEhJKrbDygql6AAAAgPm8Dk4bN270eO90OlW9enVNnDix2I57KIzmEAAAAID5vA5Oy5cv90Ud5Rb7OAEAAADm83qN0549e7Rjx45Cx3fs2KG9e/eWRk3lCmucAAAAAPN5HZz69++vb775ptDxNWvWqH///qVRU7lydqoeyQkAAAAwldfBaePGjWrfvn2h4zfccINSU1NLo6ZyheYQAAAAgPm8Dk4Oh0PHjx8vdDwzM1P5+fmlUlR54nSebQ5hEZ4AAAAAI3kdnG688UYlJSV5hKT8/HwlJSWpQ4cOpVpceVAwVU9inRMAAABgKq+76r388su68cYb1ahRI3Xs2FGS9PXXXysrK0vLli0r9QKvdM6zuUkuy5JTjvOfDAAAAMAWXo84NWnSRN9//73uuusuHTx4UMePH1e/fv30008/qVmzZr6o8YrmOGfEib2cAAAAADN5PeIkSTVr1tRLL71U2rWUS/874gQAAADAPF6POM2aNUvz588vdHz+/PmaM2eO1wVMnTpVMTExCgoKUtu2bbV27doLnj958mQ1atRIwcHBio6O1ogRI3Tq1Cmvv9cUrHECAAAAzOd1cEpKSlJ4eHih4xEREV6PQs2bN0+JiYkaO3asNmzYoJYtW6pbt246ePBgked/8MEHeuqppzR27Fht3bpVb7/9tubNm6enn37a2x/DGE6PqXokJwAAAMBEXgentLQ01a1bt9DxOnXqKC0tzat7TZo0SYMGDdKAAQPUpEkTTZs2TSEhIZo5c2aR53/zzTdq37697rnnHsXExOiWW25Rnz59ih2lMpmDqXoAAACA8bwOThEREfr+++8LHd+0aZOuuuqqEt8nLy9P69evV3x8/NlinE7Fx8dr9erVRV7Trl07rV+/3h2Udu/erUWLFum222477/fk5uYqKyvL42USJ80hAAAAAON53RyiT58+GjZsmCpVqqQbb7xRkrRixQoNHz5cd999d4nvc/jwYeXn5ysyMtLjeGRkpH766acir7nnnnt0+PBhdejQQZZl6cyZMxo8ePAFp+olJSXpueeeK3Fdl9u5zSHYABcAAAAwk9cjTuPGjVPbtm3VpUsXBQcHKzg4WLfccotuvvlmn3faS0lJ0UsvvaTXX39dGzZs0IIFC/TZZ59p3Lhx571m1KhRyszMdL/27dvn0xq9xYgTAAAAYD6vR5wCAgI0b948jRs3Tps2bVJwcLCaN2+uOnXqeHWf8PBw+fn5KSMjw+N4RkaGoqKiirxm9OjRuu+++3T//fdLkpo3b66cnBw98MADeuaZZ+R0Fs6BgYGBCgwM9Kq2y8l5zpBTPskJAAAAMJLXI04FqlWrps6dO+uPf/yj16FJ+i2AxcbGKjk52X3M5XIpOTlZcXFxRV5z4sSJQuHIz89PUtme5laQncryzwAAAABcybwKTseOHdOQIUMUHh6uyMhIRUZGKjw8XEOHDtWxY8e8/vLExETNmDFDc+bM0datW/XQQw8pJydHAwYMkCT169dPo0aNcp/fs2dPvfHGG5o7d6727NmjpUuXavTo0erZs6c7QJVFBdP1GHACAAAAzFTiqXpHjhxRXFyc9u/fr759++raa6+VJG3ZskWzZ89WcnKyvvnmG1WtWrXEX967d28dOnRIY8aMUXp6ulq1aqXFixe7G0akpaV5jDD9/e9/l8Ph0N///nft379f1atXV8+ePfXiiy+W+DtN9FtwsmhHDgAAABjKYZVwftijjz6q5ORkffnll4U64aWnp+uWW25Rly5d9I9//MMnhZaWrKwsVa5cWZmZmQoLC7O7HElSo79/rtwzLq18srOurhpidzkAAABAueBNNijxVL2FCxfq1VdfLRSaJCkqKkoTJkzQxx9/7H21cE/VY8AJAAAAMFOJg9OBAwfUtGnT837erFkzpaenl0pR5U1Bcwim6gEAAABmKnFwCg8P1969e8/7+Z49e1StWrXSqKncoTkEAAAAYLYSB6du3brpmWeeUV5eXqHPcnNzNXr0aN16662lWlx54WDECQAAADBaibvqPf/882rTpo0aNGigIUOGqHHjxrIsS1u3btXrr7+u3Nxcvfvuu76s9YpVsAku+zgBAAAAZipxcLr66qu1evVqPfzwwxo1apT7H/kOh0Ndu3bVa6+9pujoaJ8VeiVjqh4AAABgthIHJ0mqW7euPv/8cx09elQ7duyQJNWvX5+1TZeoIDjlk5wAAAAAI3kVnApUrVpV119/fWnXUm7RVQ8AAAAwW4mbQ8B32McJAAAAMBvByQCMOAEAAABmIzgZwEFzCAAAAMBoJQpO1113nY4ePSrpt7bkJ06c8GlR5Y3z978CI04AAACAmUoUnLZu3aqcnBxJ0nPPPafs7GyfFlXenF3jRHACAAAATFSirnqtWrXSgAED1KFDB1mWpVdffVUVK1Ys8twxY8aUaoHlAfs4AQAAAGYrUXCaPXu2xo4dq08//VQOh0Off/65/P0LX+pwOAhOF8FR0ByC5AQAAAAYqUTBqVGjRpo7d64kyel0Kjk5WRERET4trDxhxAkAAAAwm9cb4LpcLl/UUa75scYJAAAAMJrXwUmSdu3apcmTJ2vr1q2SpCZNmmj48OGqV69eqRZXXhRM1csnOAEAAABG8nofpyVLlqhJkyZau3atWrRooRYtWmjNmjVq2rSpli5d6osar3hM1QMAAADM5vWI01NPPaURI0Zo/PjxhY4/+eST6tq1a6kVV16wjxMAAABgNq9HnLZu3aqBAwcWOv63v/1NW7ZsKZWiyhv2cQIAAADM5nVwql69ulJTUwsdT01NpdPeRXIUTNWj7wYAAABgJK+n6g0aNEgPPPCAdu/erXbt2kmSVq1apZdfflmJiYmlXmB54CzYx4kRJwAAAMBIXgen0aNHq1KlSpo4caJGjRolSapZs6aeffZZDRs2rNQLLA9oDgEAAACYzevg5HA4NGLECI0YMULHjx+XJFWqVKnUCytPCkacWOMEAAAAmOmi9nEqQGAqHQ5GnAAAAACjed0cAqWPNU4AAACA2QhOBvBzFow4EZwAAAAAExGcDHC2OQTBCQAAADCR18Fp9+7dvqijXGMfJwAAAMBsXgen+vXrq3Pnznrvvfd06tQpX9RU7rDGCQAAADCb18Fpw4YNatGihRITExUVFaUHH3xQa9eu9UVt5UbBVD1yEwAAAGAmr4NTq1atNGXKFP3666+aOXOmDhw4oA4dOqhZs2aaNGmSDh065Is6r2iMOAEAAABmu+jmEP7+/vrzn/+s+fPn6+WXX9bOnTs1cuRIRUdHq1+/fjpw4EBp1nlFYx8nAAAAwGwXHZzWrVunhx9+WDVq1NCkSZM0cuRI7dq1S0uXLtWvv/6q22+/vTTrvKIx4gQAAACYzd/bCyZNmqRZs2Zp27Ztuu222/TOO+/otttuk9P5WwarW7euZs+erZiYmNKu9Yp1do0TwQkAAAAwkdfB6Y033tDf/vY39e/fXzVq1CjynIiICL399tuXXFx54WSqHgAAAGA0r4PTjh07ij0nICBACQkJF1VQeeRgqh4AAABgNK/XOM2aNUvz588vdHz+/PmaM2dOqRRV3vg5GXECAAAATOZ1cEpKSlJ4eHih4xEREXrppZdKpajyxj1Vj+QEAAAAGMnr4JSWlqa6desWOl6nTh2lpaWVSlHlDVP1AAAAALN5HZwiIiL0/fffFzq+adMmXXXVVaVSVHlDcwgAAADAbF4Hpz59+mjYsGFavny58vPzlZ+fr2XLlmn48OG6++67fVHjFY99nAAAAACzed1Vb9y4cdq7d6+6dOkif//fLne5XOrXrx9rnC4S+zgBAAAAZvM6OAUEBGjevHkaN26cNm3apODgYDVv3lx16tTxRX3lgoOpegAAAIDRvA5OBRo2bKiGDRuWZi3lFlP1AAAAALN5HZzy8/M1e/ZsJScn6+DBg3K5XB6fL1u2rNSKKy9oDgEAAACYzevgNHz4cM2ePVs9evRQs2bN3NPMcPEKRpxY4wQAAACYyevgNHfuXH300Ue67bbbfFFPuXR2jRPBCQAAADCR1+3IAwICVL9+fV/UUm75OZmqBwAAAJjM6+D02GOPacqUKUwrK0Xu5hAkJwAAAMBIXk/VW7lypZYvX67PP/9cTZs2VYUKFTw+X7BgQakVV144maoHAAAAGM3r4FSlShXdcccdvqil3GIfJwAAAMBsXgenWbNmlWoBU6dO1SuvvKL09HS1bNlS//znP3X99def9/xjx47pmWee0YIFC3TkyBHVqVNHkydPLtPNKtjHCQAAADCb12ucJOnMmTP68ssv9eabb+r48eOSpF9//VXZ2dle3WfevHlKTEzU2LFjtWHDBrVs2VLdunXTwYMHizw/Ly9PXbt21d69e/Wvf/1L27Zt04wZM1SrVq2L+TGMUTBVj9wEAAAAmMnrEaeff/5Zt956q9LS0pSbm6uuXbuqUqVKevnll5Wbm6tp06aV+F6TJk3SoEGDNGDAAEnStGnT9Nlnn2nmzJl66qmnCp0/c+ZMHTlyRN988417bVVMTIy3P4JxGHECAAAAzOb1iNPw4cPVpk0bHT16VMHBwe7jd9xxh5KTk0t8n7y8PK1fv17x8fFni3E6FR8fr9WrVxd5zX//+1/FxcVpyJAhioyMVLNmzfTSSy8pPz//vN+Tm5urrKwsj5dp2McJAAAAMJvXI05ff/21vvnmGwUEBHgcj4mJ0f79+0t8n8OHDys/P1+RkZEexyMjI/XTTz8Vec3u3bu1bNky9e3bV4sWLdLOnTv18MMP6/Tp0xo7dmyR1yQlJem5554rcV12cNIcAgAAADCa1yNOLperyBGeX375RZUqVSqVoi703REREZo+fbpiY2PVu3dvPfPMMxecHjhq1ChlZma6X/v27fNpjRejYKoee2MBAAAAZvI6ON1yyy2aPHmy+73D4VB2drbGjh3rVWe78PBw+fn5KSMjw+N4RkaGoqKiirymRo0aatiwofz8/NzHrr32WqWnpysvL6/IawIDAxUWFubxMo3z9+TkctlcCAAAAIAieR2cJk6cqFWrVqlJkyY6deqU7rnnHvc0vZdffrnE9wkICFBsbKzHuiiXy6Xk5GTFxcUVeU379u21c+dOuc5JGNu3b1eNGjUKTR0sS9gAFwAAADCb12ucrr76am3atElz587V999/r+zsbA0cOFB9+/b1aBZREomJiUpISFCbNm10/fXXa/LkycrJyXF32evXr59q1aqlpKQkSdJDDz2k1157TcOHD9cjjzyiHTt26KWXXtKwYcO8/TGMUjBVL5/gBAAAABjJ6+AkSf7+/rr33nsv+ct79+6tQ4cOacyYMUpPT1erVq20ePFid8OItLQ0OZ1nB8Wio6O1ZMkSjRgxQi1atFCtWrU0fPhwPfnkk5dci53YxwkAAAAwm9fB6Z133rng5/369fPqfkOHDtXQoUOL/CwlJaXQsbi4OH377bdefYfpHOzjBAAAABjN6+A0fPhwj/enT5/WiRMnFBAQoJCQEK+DE2hHDgAAAJjO6+YQR48e9XhlZ2dr27Zt6tChgz788ENf1HjFczLiBAAAABjN6+BUlAYNGmj8+PGFRqNQMgXtyNnHCQAAADBTqQQn6beGEb/++mtp3a5ccTjYxwkAAAAwmddrnP773/96vLcsSwcOHNBrr72m9u3bl1ph5QlT9QAAAACzeR2cevXq5fHe4XCoevXquvnmmzVx4sTSqqtcoTkEAAAAYDavg5OL+WSlrmDEiTVOAAAAgJlKbY0TLt7ZESeCEwAAAGAir0ecEhMTS3zupEmTvL19uVQQnPLJTQAAAICRvA5OGzdu1MaNG3X69Gk1atRIkrR9+3b5+fnpuuuuc59X0CkOxXP+Pu7HVD0AAADATF4Hp549e6pSpUqaM2eOqlatKum3TXEHDBigjh076rHHHiv1Iq90TNUDAAAAzOb1GqeJEycqKSnJHZokqWrVqnrhhRfoqneR2McJAAAAMJvXwSkrK0uHDh0qdPzQoUM6fvx4qRRV3rCPEwAAAGA2r4PTHXfcoQEDBmjBggX65Zdf9Msvv+jf//63Bg4cqD//+c++qPGKVzBVj9wEAAAAmMnrNU7Tpk3TyJEjdc899+j06dO/3cTfXwMHDtQrr7xS6gWWB4w4AQAAAGbzOjiFhITo9ddf1yuvvKJdu3ZJkurVq6fQ0NBSL668cNAcAgAAADDaRW+Ae+DAAR04cEANGjRQaGgorbQvwdmuejYXAgAAAKBIXgen//u//1OXLl3UsGFD3XbbbTpw4IAkaeDAgbQiv0gFU/UInwAAAICZvA5OI0aMUIUKFZSWlqaQkBD38d69e2vx4sWlWlx54XQy4gQAAACYzOs1Tl988YWWLFmiq6++2uN4gwYN9PPPP5daYeVJwVS9fJITAAAAYCSvR5xycnI8RpoKHDlyRIGBgaVSVHlDVz0AAADAbF4Hp44dO+qdd95xv3c4HHK5XJowYYI6d+5cqsWVF+zjBAAAAJjN66l6EyZMUJcuXbRu3Trl5eXpiSee0I8//qgjR45o1apVvqjxiudgxAkAAAAwmtcjTs2aNdP27dvVoUMH3X777crJydGf//xnbdy4UfXq1fNFjVc8J/s4AQAAAEbzasTp9OnTuvXWWzVt2jQ988wzvqqp3GGqHgAAAGA2r0acKlSooO+//95XtZRbNIcAAAAAzOb1VL17771Xb7/9ti9qKbccDvZxAgAAAEzmdXOIM2fOaObMmfryyy8VGxur0NBQj88nTZpUasWVF4w4AQAAAGbzOjht3rxZ1113nSRp+/btHp8VjJzAO35O1jgBAAAAJitxcNq9e7fq1q2r5cuX+7KecomuegAAAIDZSrzGqUGDBjp06JD7fe/evZWRkeGTosqbgoG6fBY5AQAAAEYqcXCy/mc0ZNGiRcrJySn1gsojJ80hAAAAAKN53VUPpe/sPk4kJwAAAMBEJQ5ODoejUPMHmkGUDrrqAQAAAGYrcXMIy7LUv39/BQYGSpJOnTqlwYMHF2pHvmDBgtKtsBxgHycAAADAbCUOTgkJCR7v77333lIvprxixAkAAAAwW4mD06xZs3xZR7l2do2TzYUAAAAAKBLNIQzAPk4AAACA2QhOBnAwVQ8AAAAwGsHJAH5OmkMAAAAAJiM4GYB9nAAAAACzEZwMUNBVL58hJwAAAMBIBCcDsI8TAAAAYDaCkwEKRpwkpusBAAAAJiI4GaBgjZPEqBMAAABgIoKTATyDE8kJAAAAMA3ByQCOc/4KBCcAAADAPAQnA5w74kRuAgAAAMxDcDLAuc0hGHECAAAAzENwMgDNIQAAAACzEZwMQHMIAAAAwGwEJwN4TNVjyAkAAAAwjhHBaerUqYqJiVFQUJDatm2rtWvXlui6uXPnyuFwqFevXr4t0MeYqgcAAACYzfbgNG/ePCUmJmrs2LHasGGDWrZsqW7duungwYMXvG7v3r0aOXKkOnbseJkq9R0HzSEAAAAAo9kenCZNmqRBgwZpwIABatKkiaZNm6aQkBDNnDnzvNfk5+erb9++eu6553TNNddcxmp9w+FwuMMTwQkAAAAwj63BKS8vT+vXr1d8fLz7mNPpVHx8vFavXn3e655//nlFRERo4MCBxX5Hbm6usrKyPF4mKpiuR24CAAAAzGNrcDp8+LDy8/MVGRnpcTwyMlLp6elFXrNy5Uq9/fbbmjFjRom+IykpSZUrV3a/oqOjL7luX3Ay4gQAAAAYy/apet44fvy47rvvPs2YMUPh4eElumbUqFHKzMx0v/bt2+fjKi+O4/cRJ5pDAAAAAObxt/PLw8PD5efnp4yMDI/jGRkZioqKKnT+rl27tHfvXvXs2dN9zOVySZL8/f21bds21atXz+OawMBABQYG+qD60uUecSI5AQAAAMaxdcQpICBAsbGxSk5Odh9zuVxKTk5WXFxcofMbN26sH374Qampqe7Xn/70J3Xu3FmpqanGTsMrCdY4AQAAAOaydcRJkhITE5WQkKA2bdro+uuv1+TJk5WTk6MBAwZIkvr166datWopKSlJQUFBatasmcf1VapUkaRCx8sap3uqHskJAAAAMI3twal37946dOiQxowZo/T0dLVq1UqLFy92N4xIS0uT01mmlmJdFJpDAAAAAOZyWFb5+pd6VlaWKleurMzMTIWFhdldjlur57/QsROn9WXijaofUcnucgAAAIArnjfZ4MofyikjnHTVAwAAAIxFcDIEU/UAAAAAcxGcDOHex8llcyEAAAAACiE4GYIRJwAAAMBcBCdDsI8TAAAAYC6CkyHYxwkAAAAwF8HJEA6m6gEAAADGIjgZgnbkAAAAgLkIToYoaA5RzvYjBgAAAMoEgpMhnE5GnAAAAABTEZwMUTBVL5/kBAAAABiH4GQIpuoBAAAA5iI4GYLmEAAAAIC5CE6GcLCPEwAAAGAsgpMhnOzjBAAAABiL4GSIgql65CYAAADAPAQnQzDiBAAAAJiL4GQIB80hAAAAAGMRnAzBiBMAAABgLoKTIc6ucSI4AQAAAKYhOBnC6WSqHgAAAGAqgpMhCqbq5ZOcAAAAAOMQnAzhZANcAAAAwFgEJ0OwjxMAAABgLoKTIRx01QMAAACMRXAyhJN9nAAAAABjEZwMwT5OAAAAgLkIToZgHycAAADAXAQnQziYqgcAAAAYi+BkCKbqAQAAAOYiOBmC5hAAAACAuQhOhvBzssYJAAAAMBXByRAF+zjlM+QEAAAAGIfgZAim6gEAAADmIjgZoqA5BFP1AAAAAPMQnAxxdsSJ4AQAAACYhuBkCPZxAgAAAMxFcDIE+zgBAAAA5iI4GaJgqh65CQAAADAPwckQzt//Ei7m6gEAAADGITgZgjVOAAAAgLkITobwo6seAAAAYCyCkyHYxwkAAAAwF8HJEAVT9fIJTgAAAIBxCE6GcLLGCQAAADAWwckQ7OMEAAAAmIvgZAink32cAAAAAFMRnAzhKBhxYq4eAAAAYByCkyFY4wQAAACYi+BkCNY4AQAAAOYiOBmiYMSJfZwAAAAA8xCcDOFgqh4AAABgLCOC09SpUxUTE6OgoCC1bdtWa9euPe+5M2bMUMeOHVW1alVVrVpV8fHxFzy/rPBzByeSEwAAAGAa24PTvHnzlJiYqLFjx2rDhg1q2bKlunXrpoMHDxZ5fkpKivr06aPly5dr9erVio6O1i233KL9+/df5spL19k1TvbWAQAAAKAw24PTpEmTNGjQIA0YMEBNmjTRtGnTFBISopkzZxZ5/vvvv6+HH35YrVq1UuPGjfXWW2/J5XIpOTn5Mldeugr2caIdOQAAAGAeW4NTXl6e1q9fr/j4ePcxp9Op+Ph4rV69ukT3OHHihE6fPq1q1aoV+Xlubq6ysrI8XiZy0FUPAAAAMJatwenw4cPKz89XZGSkx/HIyEilp6eX6B5PPvmkatas6RG+zpWUlKTKlSu7X9HR0Zdcty+wjxMAAABgLtun6l2K8ePHa+7cufr4448VFBRU5DmjRo1SZmam+7Vv377LXGXJFKxxoh05AAAAYB5/O788PDxcfn5+ysjI8DiekZGhqKioC1776quvavz48fryyy/VokWL854XGBiowMDAUqnXl5x01QMAAACMZeuIU0BAgGJjYz0aOxQ0eoiLizvvdRMmTNC4ceO0ePFitWnT5nKU6nPs4wQAAACYy9YRJ0lKTExUQkKC2rRpo+uvv16TJ09WTk6OBgwYIEnq16+fatWqpaSkJEnSyy+/rDFjxuiDDz5QTEyMey1UxYoVVbFiRdt+jkvlpDkEAAAAYCzbg1Pv3r116NAhjRkzRunp6WrVqpUWL17sbhiRlpYmp/PswNgbb7yhvLw8/fWvf/W4z9ixY/Xss89eztJLVcFUPXITAAAAYB7bg5MkDR06VEOHDi3ys5SUFI/3e/fu9X1BNmDECQAAADBXme6qdyVxb4BLcAIAAACMQ3AyBPs4AQAAAOYiOBnCPVWP5AQAAAAYh+BkCAf7OAEAAADGIjgZgql6AAAAgLkIToagqx4AAABgLoKTIdjHCQAAADAXwckQDkacAAAAAGMRnAzhpDkEAAAAYCyCkyFoDgEAAACYi+BkiILmEBYjTgAAAIBxCE6GcDoZcQIAAABMRXAyBGucAAAAAHMRnAzh3seJIScAAADAOAQnQ9AcAgAAADAXwckQ7OMEAAAAmIvgZAhGnAAAAABzEZwMURCcaEcOAAAAmIfgZAgnU/UAAAAAYxGcDOFgqh4AAABgLIKTIRhxAgAAAMxFcDKE01mwxsnmQgAAAAAUQnAyxNmueiQnAAAAwDQEJ0MwVQ8AAAAwl7/dBUBKS0vT9u2/SpJOnsrThg0bSnRdeHi4ateu7cvSAAAAAIjgZLu0tDQ1vvZa5VeqoRr9pyg9PV2xsd1LdG1wSIh+2rqV8AQAAAD4GMHJZocPH9bJEyd0+yPDlSqpYtVwJU5dUOx1GWm79P7Lj+vw4cMEJwAAAMDHCE6GuCoqWkqXnH7+urpBQ7vLAQAAAHAOmkMY4vfeELQjBwAAAAxEcDKEo/hTAAAAANiE4GQKx29DTRZDTgAAAIBxCE6GcE/Vs7UKAAAAAEUhOBmGAScAAADAPAQnQzh+H3KyGHMCAAAAjENwMgRd9QAAAABzEZwMwRonAAAAwFwEJ8PQVQ8AAAAwD8HJEIw4AQAAAOYiOBnC3RyC5AQAAAAYh+BkIKbrAQAAAGYhOBnCUfwpAAAAAGxCcDLEucGJAScAAADALAQnU5yTnMhNAAAAgFkITobwHHEiOgEAAAAmITgZwiM42VYFAAAAgKIQnAzBGicAAADAXAQnU3iscSI5AQAAACYhOBmCEScAAADAXAQnQ7DGCQAAADAXwclALhfRCQAAADAJwckQDodUKchfkrTpl2P2FgMAAADAA8HJIB0bhEuS1v18VAePn7K5GgAAAAAFCE4GaRBRSfUjKsqypC+3HFQ+U/YAAAAAIxgRnKZOnaqYmBgFBQWpbdu2Wrt27QXPnz9/vho3bqygoCA1b95cixYtukyV+t5NDasryN+pQ9m5Wv/zUbvLAQAAACADgtO8efOUmJiosWPHasOGDWrZsqW6deumgwcPFnn+N998oz59+mjgwIHauHGjevXqpV69emnz5s2XuXLfCA30V6dG1SVJa/cc0c6D2co9k29zVQAAAED55rAse3cNatu2rf7whz/otddekyS5XC5FR0frkUce0VNPPVXo/N69eysnJ0effvqp+9gNN9ygVq1aadq0acV+X1ZWlipXrqzMzEyFhYWV3g9ykTZs2KDY2FglTl2gqxs0lSRZlqX/bvpVe//vhKTfWpVXrxSoiEqBCqzgpwA/p3KOZGj5R9P19FNPqU6dOnI6JKfDIafT4f7vjnP+0+HR8PzK4bgyf6wr9K/1myv1bwYAALxzU6MIBVXws7UGb7KB/2WqqUh5eXlav369Ro0a5T7mdDoVHx+v1atXF3nN6tWrlZiY6HGsW7duWrhwYZHn5+bmKjc31/0+MzNT0m+/JBNkZ2dLkn7Z8aNyT55wH29oSfn+fjqU69SJMw6l555Q+mHPa6t26q83VqdLq9MvZ8kAAADAJVs+spOqVwqytYaCTFCSsSRbg9Phw4eVn5+vyMhIj+ORkZH66aefirwmPT29yPPT04sOD0lJSXruuecKHY+Ojr7Iqn3jo8mj7S4BAAAAuGzqT7a7grOOHz+uypUrX/AcW4PT5TBq1CiPESqXy6UjR47oqquuksOAOUNZWVmKjo7Wvn37jJg6iCsHzxZ8iecLvsKzBV/i+cL/sixLx48fV82aNYs919bgFB4eLj8/P2VkZHgcz8jIUFRUVJHXREVFeXV+YGCgAgMDPY5VqVLl4ov2kbCwMP4HDJ/g2YIv8XzBV3i24Es8XzhXcSNNBWztqhcQEKDY2FglJye7j7lcLiUnJysuLq7Ia+Li4jzOl6SlS5ee93wAAAAAuFS2T9VLTExUQkKC2rRpo+uvv16TJ09WTk6OBgwYIEnq16+fatWqpaSkJEnS8OHD1alTJ02cOFE9evTQ3LlztW7dOk2fPt3OHwMAAADAFcz24NS7d28dOnRIY8aMUXp6ulq1aqXFixe7G0CkpaXJ6Tw7MNauXTt98MEH+vvf/66nn35aDRo00MKFC9WsWTO7foRLEhgYqLFjxxaaTghcKp4t+BLPF3yFZwu+xPOFS2H7Pk4AAAAAYDpb1zgBAAAAQFlAcAIAAACAYhCcAAAAAKAYBCcAAAAAKAbByUZTp05VTEyMgoKC1LZtW61du9buknAZJSUl6Q9/+IMqVaqkiIgI9erVS9u2bfM459SpUxoyZIiuuuoqVaxYUX/5y18KbQCdlpamHj16KCQkRBEREXr88cd15swZj3NSUlJ03XXXKTAwUPXr19fs2bML1VPc81iSWmCm8ePHy+Fw6NFHH3Uf49nCpdi/f7/uvfdeXXXVVQoODlbz5s21bt069+eWZWnMmDGqUaOGgoODFR8frx07dnjc48iRI+rbt6/CwsJUpUoVDRw4UNnZ2R7nfP/99+rYsaOCgoIUHR2tCRMmFKpl/vz5aty4sYKCgtS8eXMtWrTI4/OS1AIz5Ofna/To0apbt66Cg4NVr149jRs3Tuf2MePZgq0s2GLu3LlWQECANXPmTOvHH3+0Bg0aZFWpUsXKyMiwuzRcJt26dbNmzZplbd682UpNTbVuu+02q3bt2lZ2drb7nMGDB1vR0dFWcnKytW7dOuuGG26w2rVr5/78zJkzVrNmzaz4+Hhr48aN1qJFi6zw8HBr1KhR7nN2795thYSEWImJidaWLVusf/7zn5afn5+1ePFi9zkleR6LqwVmWrt2rRUTE2O1aNHCGj58uPs4zxYu1pEjR6w6depY/fv3t9asWWPt3r3bWrJkibVz5073OePHj7cqV65sLVy40Nq0aZP1pz/9yapbt6518uRJ9zm33nqr1bJlS+vbb7+1vv76a6t+/fpWnz593J9nZmZakZGRVt++fa3NmzdbH374oRUcHGy9+eab7nNWrVpl+fn5WRMmTLC2bNli/f3vf7cqVKhg/fDDD17VAjO8+OKL1lVXXWV9+umn1p49e6z58+dbFStWtKZMmeI+h2cLdiI42eT666+3hgwZ4n6fn59v1axZ00pKSrKxKtjp4MGDliRrxYoVlmVZ1rFjx6wKFSpY8+fPd5+zdetWS5K1evVqy7Isa9GiRZbT6bTS09Pd57zxxhtWWFiYlZuba1mWZT3xxBNW06ZNPb6rd+/eVrdu3dzvi3seS1ILzHP8+HGrQYMG1tKlS61OnTq5gxPPFi7Fk08+aXXo0OG8n7tcLisqKsp65ZVX3MeOHTtmBQYGWh9++KFlWZa1ZcsWS5L13Xffuc/5/PPPLYfDYe3fv9+yLMt6/fXXrapVq7qft4LvbtSokfv9XXfdZfXo0cPj+9u2bWs9+OCDJa4F5ujRo4f1t7/9zePYn//8Z6tv376WZfFswX5M1bNBXl6e1q9fr/j4ePcxp9Op+Ph4rV692sbKYKfMzExJUrVq1SRJ69ev1+nTpz2ek8aNG6t27dru52T16tVq3ry5e8NoSerWrZuysrL0448/us859x4F5xTcoyTPY0lqgXmGDBmiHj16FPr782zhUvz3v/9VmzZtdOeddyoiIkKtW7fWjBkz3J/v2bNH6enpHn/TypUrq23bth7PV5UqVdSmTRv3OfHx8XI6nVqzZo37nBtvvFEBAQHuc7p166Zt27bp6NGj7nMu9AyWpBaYo127dkpOTtb27dslSZs2bdLKlSvVvXt3STxbsJ+/3QWUR4cPH1Z+fr7HP0gkKTIyUj/99JNNVcFOLpdLjz76qNq3b69mzZpJktLT0xUQEKAqVap4nBsZGan09HT3OUU9RwWfXeicrKwsnTx5UkePHi32eSxJLTDL3LlztWHDBn333XeFPuPZwqXYvXu33njjDSUmJurpp5/Wd999p2HDhikgIEAJCQnuv1tRf/dzn52IiAiPz/39/VWtWjWPc+rWrVvoHgWfVa1a9bzP4Ln3KK4WmOOpp55SVlaWGjduLD8/P+Xn5+vFF19U3759JZXs78mzBV8iOAEGGDJkiDZv3qyVK1faXQquAPv27dPw4cO1dOlSBQUF2V0OrjAul0tt2rTRSy+9JElq3bq1Nm/erGnTpikhIcHm6lCWffTRR3r//ff1wQcfqGnTpkpNTdWjjz6qmjVr8mzBCEzVs0F4eLj8/PwKdY3KyMhQVFSUTVXBLkOHDtWnn36q5cuX6+qrr3Yfj4qKUl5eno4dO+Zx/rnPSVRUVJHPUcFnFzonLCxMwcHBJXoeS1ILzLF+/XodPHhQ1113nfz9/eXv768VK1bo//2//yd/f39FRkbybOGi1ahRQ02aNPE4du211yotLU3S2eejuL/7wYMHPT4/c+aMjhw5UirP4LmfF1cLzPH444/rqaee0t13363mzZvrvvvu04gRI5SUlCSJZwv2IzjZICAgQLGxsUpOTnYfc7lcSk5OVlxcnI2V4XKyLEtDhw7Vxx9/rGXLlhWaNhAbG6sKFSp4PCfbtm1TWlqa+zmJi4vTDz/84PF/EkuXLlVYWJj7HzZxcXEe9yg4p+AeJXkeS1ILzNGlSxf98MMPSk1Ndb/atGmjvn37uv87zxYuVvv27QttnbB9+3bVqVNHklS3bl1FRUV5/E2zsrK0Zs0aj+fr2LFjWr9+vfucZcuWyeVyqW3btu5zvvrqK50+fdp9ztKlS9WoUSNVrVrVfc6FnsGS1AJznDhxQk6n5z9N/fz85HK5JPFswQB2d6cor+bOnWsFBgZas2fPtrZs2WI98MADVpUqVTw6WOHK9tBDD1mVK1e2UlJSrAMHDrhfJ06ccJ8zePBgq3bt2tayZcusdevWWXFxcVZcXJz784KW0bfccouVmppqLV682KpevXqRLaMff/xxa+vWrdbUqVOLbBld3PNYXC0w27ld9SyLZwsXb+3atZa/v7/14osvWjt27LDef/99KyQkxHrvvffc54wfP96qUqWK9Z///Mf6/vvvrdtvv73IltGtW7e21qxZY61cudJq0KCBR8voY8eOWZGRkdZ9991nbd682Zo7d64VEhJSqGW0v7+/9eqrr1pbt261xo4dW2TL6OJqgRkSEhKsWrVquduRL1iwwAoPD7eeeOIJ9zk8W7ATwclG//znP63atWtbAQEB1vXXX299++23dpeEy0hSka9Zs2a5zzl58qT18MMPW1WrVrVCQkKsO+64wzpw4IDHffbu3Wt1797dCg4OtsLDw63HHnvMOn36tMc5y5cvt1q1amUFBARY11xzjcd3FCjueSxJLTDX/wYnni1cik8++cRq1qyZFRgYaDVu3NiaPn26x+cul8saPXq0FRkZaQUGBlpdunSxtm3b5nHO//3f/1l9+vSxKlasaIWFhVkDBgywjh8/7nHOpk2brA4dOliBgYFWrVq1rPHjxxeq5aOPPrIaNmxoBQQEWE2bNrU+++wzr2uBGbKysqzhw4dbtWvXtoKCgqxrrrnGeuaZZzzahvNswU4OyzpnO2YAAAAAQCGscQIAAACAYhCcAAAAAKAYBCcAAAAAKAbBCQAAAACKQXACAAAAgGIQnAAAAACgGAQnAAAAACgGwQkAAAAAikFwAgBcsW666SY9+uijdpcBALgCEJwAAD4xbdo0VapUSWfOnHEfy87OVoUKFXTTTTd5nJuSkiKHw6Fdu3Zd5iqlvLw8TZgwQS1btlRISIjCw8PVvn17zZo1S6dPn76stRD0AMBc/nYXAAC4MnXu3FnZ2dlat26dbrjhBknS119/raioKK1Zs0anTp1SUFCQJGn58uWqXbu26tWr5/X3WJal/Px8+ft7/39peXl56tatmzZt2qRx48apffv2CgsL07fffqtXX31VrVu3VqtWrby+LwDgysOIEwDAJxo1aqQaNWooJSXFfSwlJUW333676tatq2+//dbjeOfOnSVJubm5GjZsmCIiIhQUFKQOHTrou+++8zjX4XDo888/V2xsrAIDA7Vy5Url5OSoX79+qlixomrUqKGJEycWW+PkyZP11VdfKTk5WUOGDFGrVq10zTXX6J577tGaNWvUoEGDEtU0e/ZsValSxePeCxculMPhcL9/9tln1apVK7377ruKiYlR5cqVdffdd+v48eOSpP79+2vFihWaMmWKHA6HHA6H9u7dW+LfNwDAtwhOAACf6dy5s5YvX+5+v3z5ct10003q1KmT+/jJkye1Zs0ad3B64okn9O9//1tz5szRhg0bVL9+fXXr1k1HjhzxuPdTTz2l8ePHa+vWrWrRooUef/xxrVixQv/5z3/0xRdfKCUlRRs2bLhgfe+//77i4+PVunXrQp9VqFBBoaGhXtVUnF27dmnhwoX69NNP9emnn2rFihUaP368JGnKlCmKi4vToEGDdODAAR04cEDR0dFe3R8A4DsEJwCAz3Tu3FmrVq3SmTNndPz4cW3cuFGdOnXSjTfe6B6JWr16tXJzc9W5c2fl5OTojTfe0CuvvKLu3burSZMmmjFjhoKDg/X222973Pv5559X165dVa9ePQUEBOjtt9/Wq6++qi5duqh58+aaM2eOx/qqouzYsUONGze+4Dne1FQcl8ul2bNnq1mzZurYsaPuu+8+JScnS5IqV66sgIAAhYSEKCoqSlFRUfLz8/Pq/gAA3yE4AQB85qabblJOTo6+++47ff3112rYsKGqV6+uTp06udc5paSk6JprrlHt2rW1a9cunT59Wu3bt3ffo0KFCrr++uu1detWj3u3adPG/d937dqlvLw8tW3b1n2sWrVqatSo0QXrsyyr2J/Bm5qKExMTo0qVKrnf16hRQwcPHvTqHgAAe9AcAgDgM/Xr19fVV1+t5cuX6+jRo+rUqZMkqWbNmoqOjtY333yj5cuX6+abb/b63gXT6C5Fw4YN9dNPP13yfZxOZ6EQVlRHvgoVKni8dzgccrlcl/z9AADfY8QJAOBTnTt3VkpKilJSUjzakN944436/PPPtXbtWvf6poJpd6tWrXKfd/r0aX333Xdq0qTJeb+jXr16qlChgtasWeM+dvToUW3fvv2Ctd1zzz368ssvtXHjxkKfnT59Wjk5OSWqqXr16jp+/LhycnLc56Smpl7wu4sSEBCg/Px8r68DAPgewQkA4FOdO3fWypUrlZqa6h5xkqROnTrpzTffVF5enjs4hYaG6qGHHtLjjz+uxYsXa8uWLRo0aJBOnDihgQMHnvc7KlasqIEDB+rxxx/XsmXLtHnzZvXv319O54X/b+7RRx9V+/bt1aVLF02dOlWbNm3S7t279dFHH+mGG27Qjh07SlRT27ZtFRISoqefflq7du3SBx98oNmzZ3v9u4qJidGaNWu0d+9eHT58mNEoADAIU/UAAD7VuXNnnTx5Uo0bN1ZkZKT7eKdOnXT8+HF32/IC48ePl8vl0n333afjx4+rTZs2WrJkiapWrXrB73nllVeUnZ2tnj17qlKlSnrssceUmZl5wWsCAwO1dOlS/eMf/9Cbb76pkSNHKiQkRNdee62GDRumZs2alaimatWq6b333tPjjz+uGTNmqEuXLnr22Wf1wAMPePW7GjlypBISEtSkSROdPHlSe/bsUUxMjFf3AAD4hsMqycpYAAAAACjHmKoHAAAAAMUgOAEAAABAMQhOAAAAAFAMghMAAAAAFIPgBAAAAADFIDgBAAAAQDEITgAAAABQDIITAAAAABSD4AQAAAAAxSA4AQAAAEAxCE4AAAAAUIz/D/Dv/pA5lv1/AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('the', 46273675), ('of', 23645711), ('and', 20278174), ('to', 19549375), ('a', 16105849), ('in', 15588963), ('is', 9679999), ('that', 8111288), ('for', 8014297), ('with', 5896119), ('on', 5010363), ('as', 4912927), ('was', 4342515), ('be', 4315514), ('this', 4178044), ('it', 4153150), ('by', 4075041), ('i', 3997621), ('=', 3966025), ('are', 3891138), ('at', 3458970), ('from', 3430069), ('or', 3279427), ('not', 3182725), ('you', 3124774), ('we', 2998420), ('have', 2956292), ('an', 2917529), ('if', 2299889), ('but', 2214059), ('-', 2096678), ('were', 2080686), ('he', 2074007), ('which', 2016593), ('can', 1954685), ('has', 1874547), ('+', 1780226), ('{', 1774643), ('will', 1714208), ('all', 1713836), ('they', 1691760), ('his', 1675734), ('one', 1566157), ('}', 1539036), ('their', 1452464), ('when', 1451952), ('more', 1411543), ('what', 1392283), ('your', 1308072), ('had', 1274550)]\n"
     ]
    }
   ],
   "source": [
    "down_path = Path(down_dir)\n",
    "\n",
    "# Loading minipile train + val splits from the local directory \n",
    "# https://stackoverflow.com/questions/77020278/how-to-load-a-huggingface-dataset-from-local-path\n",
    "# https://github.com/MK2112/mobileYOLOv3/blob/main/mobileyolov3-cocotext.ipynb\n",
    "# Split is named exactly like with the original dataset https://huggingface.co/datasets/JeanKaddour/minipile\n",
    "minipile_train = load_dataset(\"parquet\",\n",
    "                              data_files={\n",
    "                                  \"train\": str(down_path / \"MiniPile\" / \"data\" / \"train-*.parquet\"),\n",
    "                                  \"validation\": str(down_path / \"MiniPile\" / \"data\" / \"validation-*.parquet\"),\n",
    "                                  \"test\": str(down_path / \"MiniPile\" / \"data\" / \"test-*.parquet\")\n",
    "                              },\n",
    "                              cache_dir=str(down_path / \"MiniPile_Cache\"),\n",
    "                              split=\"train\")\n",
    "\n",
    "df = pd.DataFrame(minipile_train)[['text']]\n",
    "\n",
    "# Word Counts\n",
    "df['word_count'] = df['text'].str.split().str.len()\n",
    "plt.figure(figsize=(10, 6))\n",
    "sns.histplot(df['word_count'], bins=50, kde=True)\n",
    "plt.title('Word Count Distribution')\n",
    "plt.xlabel('Word Count')\n",
    "plt.ylabel('Frequency of Occurence')\n",
    "plt.show()\n",
    "\n",
    "print(Counter(' '.join(df['text']).lower().split()).most_common(50))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I find that conducting EDA like this, while informative, takes hilarious amounts of time.<br>\n",
    "I will elect not to apply this further, but instead utilize HuggingFace's SQL-based Interaction options for aggregations and informations about the dataset. For example, after seeing the above graphs, I wanted to know the largest and the smallest example in the dataset, as the distribution is shown to be pretty extreme.\n",
    "\n",
    "On HuggingFace, I ran this to get the answers within about a minute:<br>\n",
    "`SELECT MIN(LENGTH(text)), MAX(LENGTH(text)) FROM train;`\n",
    "\n",
    "`JeanKaddour/MiniPile` contains a smallest example of char size $8$, and a largest one with size $5,766,478$."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "minipile",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}