criticism_detection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# criticism_detection.ipynb\n",
    "\n",
    "### 1 ###\n",
    "•\tIt analysis papers citing another one that has been retracted.\n",
    "•\tIt takes papers that triggered the retraction and classify them as highly_critical.\n",
    "•\tPapers published a year before the first highly_critical paper are classified as non_critical.\n",
    "•\tPapers published after the highly_critical papers are classified as critical (they are not used in the analysis later on because they may contain mixed criticism).\n",
    "•\tDate of publication of the papers are taken from the “PMC-ids.csv.gz” file, available through the PMC FTP service:\n",
    "https://www.ncbi.nlm.nih.gov/pmc/pmctopmid/\n",
    "(Downloaded in data/PMC_ids folder)\n",
    "\n",
    "### 2 ###\n",
    "•\tIt takes all the sentences citing a particular DOI in all the papers disregarding its section. \n",
    "•\tIt shows that clustering algorithms such as PCA and TSNE fail if trying to cluster the sentences from non_critical and high_critical papers just using vader lexicon.\n",
    "\n",
    "### 3 ###\n",
    "•\tIt gets all the sentences in the high_critical papers and a similar number of sentences from the non_critical papers. It finds the words and bigrams used in the high_critical set and are not used in the non_critical set.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import re\n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import nltk\n",
    "from sklearn.decomposition import PCA\n",
    "import seaborn as sns\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.manifold import TSNE\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "sys.path.insert(0, \"./modules\")\n",
    "import words_frec_analysis_get_sentence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data_path = 'data'\n",
    "\n",
    "ds_name = 'DOI_cited_science_1179052_retracted'\n",
    "\n",
    "# The paper was retracted based on:\n",
    "DOI_high_critical = [\n",
    "    '10.1186/1742-4690-7-63',\n",
    "    '10.1371/journal.pone.0008519',\n",
    "    '10.1136/bmj.c1018',\n",
    "    '10.1186/1742-4690-7-10',\n",
    "    '10.1371/journal.pone.0008519',\n",
    "    '10.1186/1743-422X-7-224'\n",
    "]\n",
    "\n",
    "analysis_path = os.path.join(data_path, 'analysis')\n",
    "\n",
    "# In\n",
    "citing_sections_tsv = os.path.join(analysis_path, '%s_sections.tsv' % ds_name)\n",
    "pmc_ids_csv = os.path.join(data_path, 'PMC_ids/PMC-ids.csv.gz')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package vader_lexicon to\n",
      "[nltk_data]     /home/sherlock/nltk_data...\n",
      "[nltk_data]   Package vader_lexicon is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get Vader data for sentiment analysis\n",
    "nltk.download('vader_lexicon') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1 ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The code analysis papers citing another one that has been retracted. It takes papers that triggered the retraction and classify them as highly_critical. Papers published a year before the first highly_critical paper are classified as non_critical. Papers published after the highly_critical papers are classified as critical (they are not used in the analysis later on because they may contain mixed criticism). Date of publication of the papers are taken from the “PMC-ids.csv.gz” file, available through the PMC FTP service: https://www.ncbi.nlm.nih.gov/pmc/pmctopmid/ (Downloaded in data/PMC_ids folder)\n"
     ]
    }
   ],
   "source": [
    "print(\"The code analysis papers citing another one that has been retracted. It takes papers that triggered the retraction and classify them as highly_critical. Papers published a year before the first highly_critical paper are classified as non_critical. Papers published after the highly_critical papers are classified as critical (they are not used in the analysis later on because they may contain mixed criticism). Date of publication of the papers are taken from the “PMC-ids.csv.gz” file, available through the PMC FTP service: https://www.ncbi.nlm.nih.gov/pmc/pmctopmid/ (Downloaded in data/PMC_ids folder)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    citing_sections_tsv,\n",
    "    sep='\\t',\n",
    "    encoding='utf-8'\n",
    ")\n",
    "\n",
    "#  df['cited_DOI', 'cited_in_conclusions','cited_in_discussion',\n",
    "#    'cited_in_introduction', 'cited_in_maintext', 'citing_DOI',\n",
    "#    'conclusions_found', 'discussion_found', 'introduction_found',\n",
    "#    'maintext_found', 'reference_id', 'sentence_citing_conclusions',\n",
    "#    'sentence_citing_discussion', 'sentence_citing_intro', 'sentence_citing_maintext']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df_ids = pd.read_csv(pmc_ids_csv, sep=',', encoding='utf-8', low_memory=False)\n",
    "\n",
    "\n",
    "df_ids_select = df_ids[df_ids['DOI'].isin(df.citing_DOI)]\n",
    "\"\"\"\n",
    "for element in df.citing_DOI:\n",
    "    df_ids['DOI'].isin(df.citingDOI)\n",
    "    df_ids_selection2.append(df_ids[[df_ids[\"DOI\"] == element]])\n",
    "    #print(element)\n",
    "\"\"\"\n",
    "\n",
    "df_ids_select.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# The paper was retracted based on:\n",
    "#DOI_high_critical = ['10.1186/1742-4690-7-63', '10.1371/journal.pone.0008519', '10.1136/bmj.c1018', '10.1186/1742-4690-7-10', '10.1371/journal.pone.0008519', '10.1186/1743-422X-7-224']\n",
    "\n",
    "df_ids_select_info_highCritical = df_ids[df_ids[\"DOI\"].isin(DOI_high_critical)]\n",
    "\n",
    "df_ids_select_nonCritical = df_ids[((df_ids[\"Year\"] < 2011)) & (df_ids['DOI'].isin(DOI_high_critical) == False)]\n",
    "\n",
    "df_ids_select_Critical = df_ids[(df_ids[\"Year\"] >= 2011) | (df_ids['DOI'].isin(DOI_high_critical))]\n",
    "\n",
    "\n",
    "df_highCritical =df[df['citing_DOI'].isin(df_ids_select_info_highCritical['DOI'])]\n",
    " \n",
    "df_nonCritical = df[df['citing_DOI'].isin(df_ids_select_nonCritical['DOI'])]\n",
    "df_Critical = df[df['citing_DOI'].isin(df_ids_select_Critical['DOI'])]\n",
    "#df_critical\n",
    "\n",
    "df_highCritical_text_part1 = df_highCritical[['citing_DOI','sentence_citing_conclusions']].rename(columns = {'sentence_citing_conclusions': 'text'})\n",
    "df_highCritical_text_part2 = df_highCritical[['citing_DOI','sentence_citing_discussion']].rename(columns = {'sentence_citing_discussion': 'text'})\n",
    "df_highCritical_text_part3 = df_highCritical[['citing_DOI','sentence_citing_intro']].rename(columns = {'sentence_citing_intro': 'text'})\n",
    "df_full_highCritical = pd.concat([df_highCritical_text_part1, df_highCritical_text_part2, df_highCritical_text_part3]).dropna().reset_index(drop=True)\n",
    "df_full_highCritical['label'] = 'high_critical'\n",
    "\n",
    "df_part1 = df_nonCritical[['citing_DOI','sentence_citing_conclusions']].rename(columns = {'sentence_citing_conclusions': 'text'})\n",
    "df_part2 = df_nonCritical[['citing_DOI','sentence_citing_discussion']].rename(columns = {'sentence_citing_discussion': 'text'})\n",
    "df_part3 = df_nonCritical[['citing_DOI','sentence_citing_intro']].rename(columns = {'sentence_citing_intro': 'text'})\n",
    "df_full_nonCritical = pd.concat([df_part1, df_part2, df_part3]).dropna().reset_index(drop=True)\n",
    "df_full_nonCritical['label'] = 'non_critizising'\n",
    "\n",
    "df_part1 = df_Critical[['citing_DOI','sentence_citing_conclusions']].rename(columns = {'sentence_citing_conclusions': 'text'})\n",
    "df_part2 = df_Critical[['citing_DOI','sentence_citing_discussion']].rename(columns = {'sentence_citing_discussion': 'text'})\n",
    "df_part3 = df_Critical[['citing_DOI','sentence_citing_intro']].rename(columns = {'sentence_citing_intro': 'text'})\n",
    "df_full_Critical = pd.concat([df_part1, df_part2, df_part3]).dropna().reset_index(drop=True)\n",
    "df_full_Critical['label'] = 'critizising'\n",
    "\n",
    "df_full = pd.concat([df_full_nonCritical, df_full_Critical]).reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "df_highCritical.shape;  (3, 15)\n",
      "df_Critical.shape;  (21, 15)\n",
      "df_nonCritical.shape:  (19, 15)\n",
      "df_full.shape:  (38, 3)\n"
     ]
    }
   ],
   "source": [
    "#print(\"df_highCritical.shape; \", df_highCritical.shape)\n",
    "#print(\"df_Critical.shape; \", df_Critical.shape)\n",
    "#print(\"df_nonCritical.shape: \", df_nonCritical.shape)\n",
    "#print(\"df_full.shape: \", df_full.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cited_DOI</th>\n",
       "      <th>cited_in_conclusions</th>\n",
       "      <th>cited_in_discussion</th>\n",
       "      <th>cited_in_introduction</th>\n",
       "      <th>cited_in_maintext</th>\n",
       "      <th>citing_DOI</th>\n",
       "      <th>conclusions_found</th>\n",
       "      <th>discussion_found</th>\n",
       "      <th>introduction_found</th>\n",
       "      <th>maintext_found</th>\n",
       "      <th>reference_number</th>\n",
       "      <th>sentence_citing_conclusions</th>\n",
       "      <th>sentence_citing_discussion</th>\n",
       "      <th>sentence_citing_intro</th>\n",
       "      <th>sentence_citing_maintext</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10.1126/science.1179052</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>10.1186/1742-4690-7-10</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>B8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>['[&lt;xref ref-type=\"bibr\" rid=\"B8\"&gt;8&lt;/xref&gt;] co...</td>\n",
       "      <td>['More recently, additional groups of samples ...</td>\n",
       "      <td>['[&lt;xref ref-type=\"bibr\" rid=\"B8\"&gt;8&lt;/xref&gt;] (T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>10.1126/science.1179052</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>10.1186/1742-4690-7-63</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>B15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>['More recently, XMRV has also been found, wit...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>10.1126/science.1179052</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>10.1186/1743-422X-7-224</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>B3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>['The virus was not detected in any of our stu...</td>\n",
       "      <td>['In a recent US study, Lombardi et al [&lt;xref ...</td>\n",
       "      <td>['This sequence was obtained by overlapping ex...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  cited_DOI  cited_in_conclusions  cited_in_discussion  \\\n",
       "11  10.1126/science.1179052                 False                 True   \n",
       "12  10.1126/science.1179052                 False                False   \n",
       "31  10.1126/science.1179052                 False                 True   \n",
       "\n",
       "    cited_in_introduction  cited_in_maintext               citing_DOI  \\\n",
       "11                   True               True   10.1186/1742-4690-7-10   \n",
       "12                  False               True   10.1186/1742-4690-7-63   \n",
       "31                   True               True  10.1186/1743-422X-7-224   \n",
       "\n",
       "    conclusions_found  discussion_found  introduction_found  maintext_found  \\\n",
       "11               True              True                True            True   \n",
       "12              False             False               False            True   \n",
       "31               True              True                True            True   \n",
       "\n",
       "   reference_number sentence_citing_conclusions  \\\n",
       "11               B8                         NaN   \n",
       "12              B15                         NaN   \n",
       "31               B3                         NaN   \n",
       "\n",
       "                           sentence_citing_discussion  \\\n",
       "11  ['[<xref ref-type=\"bibr\" rid=\"B8\">8</xref>] co...   \n",
       "12                                                NaN   \n",
       "31  ['The virus was not detected in any of our stu...   \n",
       "\n",
       "                                sentence_citing_intro  \\\n",
       "11  ['More recently, additional groups of samples ...   \n",
       "12                                                NaN   \n",
       "31  ['In a recent US study, Lombardi et al [<xref ...   \n",
       "\n",
       "                             sentence_citing_maintext  \n",
       "11  ['[<xref ref-type=\"bibr\" rid=\"B8\">8</xref>] (T...  \n",
       "12  ['More recently, XMRV has also been found, wit...  \n",
       "31  ['This sequence was obtained by overlapping ex...  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_highCritical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2 ###\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " We now take all the sentences citing a particular DOI in all the papers disregarding its section. It shows that clustering algorithms such as PCA and TSNE fail if trying to cluster the sentences from non_critical and high_critical papers just using vader lexicon. \n"
     ]
    }
   ],
   "source": [
    "print(\"\\n We now take all the sentences citing a particular DOI in all the papers disregarding its section. It shows that clustering algorithms such as PCA and TSNE fail if trying to cluster the sentences from non_critical and high_critical papers just using vader lexicon. \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try to use vader_lexicon and pca and tsne to cluster papers criticising and not criticising. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'TfidfVectorizer' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-3-8fe4a4547bae>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     22\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     23\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 24\u001b[0;31m \u001b[0mtfidf_vectorizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTfidfVectorizer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstop_words\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'english'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_features\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m10000\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     25\u001b[0m \u001b[0mtfidf_vectors\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtfidf_vectorizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit_transform\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdf_full\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'text'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     26\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'TfidfVectorizer' is not defined"
     ]
    }
   ],
   "source": [
    "# dimension reduction algorithms can be pretty slow, so let's work with a sample\n",
    "# try on the whole data set if you want!\n",
    "\n",
    "# list of colours for making nice plots later\n",
    "COLOURS = ['#E91D0E', '#00A6EF']\n",
    "\n",
    "\n",
    "def scatter(x, label, selected_labels, selected_colors, title):\n",
    "    f = plt.figure(figsize=(8, 8))\n",
    "    ax = plt.subplot(aspect='equal')\n",
    "    for selected_label, selected_color in zip(selected_labels, selected_colors):\n",
    "        x_selected = x[(label == selected_label), :]\n",
    "        ax.scatter(\n",
    "            x_selected[:, 0],\n",
    "            x_selected[:, 1],\n",
    "            c=selected_color,\n",
    "            label=selected_label,\n",
    "            alpha=0.5\n",
    "        )\n",
    "    plt.legend(selected_labels)\n",
    "    plt.title(title)\n",
    "    return f, ax\n",
    "\n",
    "tfidf_vectorizer = TfidfVectorizer(stop_words='english', max_features=10000)\n",
    "tfidf_vectors = tfidf_vectorizer.fit_transform(df_full['text'])\n",
    "\n",
    "pca = PCA(n_components=2)\n",
    "pca_comp = pca.fit_transform(tfidf_vectors.toarray())\n",
    "scatter(pca_comp, df_full['label'], ['critizising', 'non_critizising'], COLOURS[:2], \"PCA\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<matplotlib.figure.Figure at 0x7f5fc3fbf128>,\n",
       " <matplotlib.axes._subplots.AxesSubplot at 0x7f5f877a6a20>)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAHVCAYAAAAkfVjMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUXmV99//3dw5JIEAyOcghA02oIAQICCPkqY8KghAQ\nibKgjS0aniqUR7A8i4JA6a8gbdeSllUpWqQpKqA+DQdRUn8qBbWt/qjAJE1ISMCEkxlOxpyABJJM\n5vv7Y3biZDI53jOZa8b3a617zd7Xvva+v1fuJJ/Z+77ue0dmIkmSylDX3wVIkqTfMJglSSqIwSxJ\nUkEMZkmSCmIwS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBWno7wJ605gxY3L8+PH9XYYkSVuZPXv2\nrzNz7I76DapgHj9+PK2trf1dhiRJW4mIF3emn5eyJUkqiMEsSVJBDGZJkgoyqN5jlqTBasOGDbS1\ntfH222/3dynagWHDhtHc3ExjY+Nu7W8wS9IA0NbWxr777sv48eOJiP4uR9uQmSxfvpy2tjYmTJiw\nW8fwUrYkDQBvv/02o0ePNpQLFxGMHj26pisbBrMkDRCG8sBQ6+tkMEuSVBCDWZLUK2bNmsUXvvAF\nAL773e+ycOHCzdv+8i//kkceeWSb++5o++23387dd9/de8UWLDKzv2voNS0tLek3f0kajBYtWsSR\nRx650/03LFrAuh/8Kx0vL6XuoIMZeuZHaDzy6D6rr729nYaG38wnvvDCCzn77LM577zz+uw5S9bT\n6xURszOzZUf7esYsSYPMhkULWDvjS3SsXkUcMI6O1atYO+NLbFi0oKbj3n333UyaNIljjz2WT3zi\nE1x44YVccsklnHTSSXzuc5/jzjvv5LLLLuPRRx9l1qxZXHXVVRx33HE8++yzXHjhhdx///20trZy\n3HHHcdxxx3HMMcdsfj9203aAa665hokTJzJp0iSuvPJKAG644QZuvvlmAE4++WSuvvpqTjzxRA4/\n/HB++tOfArB27Vp+//d/n4kTJ/Kxj32Mk046aUB+TbMfl5KkQWbdD/6V2G8kdSNGAhAjRtJRte/u\nWfNTTz3FX//1X/Poo48yZswYVqxYwRVXXEFbWxuPPvoo9fX13HnnnQD83u/9Huecc06PZ8wtLS3M\nnTsXgKuuuoopU6ZssX358uV85zvf4emnnyYiWLVqVY/1tLe38/jjj/P973+fz3/+8zzyyCPcdttt\nNDU1sXDhQhYsWMBxxx23W2Ptb54xS9Ig0/HyUmLf/bZoi333o+Plpbt9zB//+Mecf/75jBkzBoBR\no0YBcP7551NfX7/Lx7vnnnuYM2fO5vekNxkxYgTDhg3jU5/6FA888AB77713j/ufe+65AJxwwgm8\n8MILAPzsZz9j2rRpABx99NFMmjRpl+sqgcEsSYNM3UEHk2+8vkVbvvE6dQcd3OvPNXz48F3eZ8GC\nBdxwww3MnDlzq1BvaGjg8ccf57zzzuN73/veVmfUmwwdOhSA+vp62tvbd73wghnMkjTIDD3zI+Tr\nq+hYvYrs6Oj8+foqhp75kd0+5gc/+EHuu+8+li9fDsCKFSu223/ffffljTfe2Kp91apVfPzjH+fu\nu+9m7Nitb0385ptvsnr1as466yy++MUvMm/evJ2u8b3vfS/33nsvAAsXLmT+/Pk7vW9JfI9ZkgaZ\nxiOPZu+LP7vlrOxpn6hpVvZRRx3Fddddxwc+8AHq6+t597vfvd3+06ZN46KLLuLWW2/dPKkL4MEH\nH+TFF1/koosu2ty26T1ngDfeeIOpU6fy9ttvk5n8/d///U7X+JnPfIbp06czceJEjjjiCI466ihG\njBixC6Msgx+XkqQBYFc/LvXbaOPGjWzYsIFhw4bx7LPPctppp/HMM88wZMiQPV5LLR+X8oxZkjQo\nrF27llNOOYUNGzaQmdx22239Esq1MpglSYPCvvvuOyA/t9ydk78kSSqIwSxJUkEMZkmSCmIwS5JU\nEINZkqSCGMySNAg9ubKdG+a/xR//fC03zH+LJ1cOjq+t7Hrf5ltuuYW1a9du3nbWWWdt86YXO7P9\n05/+9Bb3kO4vfsGIJA0Au/IFI0+ubOfmp9fRNCQY0QirN8DK9cmVRwxlUtPA/ZTsxo0bt/hu7fHj\nx9Pa2rr5xhol8X7MkqTNHmjbQNOQoGlIUBexefmBtg01HfeFF17gyCOP5KKLLuKoo47i9NNP5623\n3mLu3LlMnjyZSZMm8bGPfYyVK1cC275vck82btzIlVdeufmuUF/60peAzvC9+uqrOf7447nvvvs2\n37f51ltv5eWXX+aUU07hlFNO2dz317/+Nbfffvvmez5PmDBhq+1r1qzhwx/+MMceeyxHH30099xz\nz+Z6N53c7bPPPlx33XUce+yxTJ48mddeew2AZ599lsmTJ3PMMcfwF3/xF+yzzz41/Zn2xGCWpEHm\nl2uSEY1bto1o7Gyv1eLFi7n00kt56qmnGDlyJN/+9rf55Cc/yU033cSTTz7JMcccw+c///nN/Tfd\nN/mWW27Zor27GTNm8MILLzB37lyefPJJ/uiP/mjzttGjRzNnzpzNt3QE+NM//VMOOuggfvKTn/CT\nn/xki2NdcsklzJ07lyeeeILm5mauuOKKLbb/8Ic/5KCDDmLevHksWLCgxztYrVmzhsmTJzNv3jze\n//7388///M8AXH755Vx++eXMnz+f5ubmXfvD20kGsyQNMocMD1Z3OzlevaGzvVYTJkzguOOOAzrv\nhfzss8+yatUqPvCBDwAwffp0/vM//3Nz/57um9yTRx55hD/5kz+hoaHzUvum+z0D/MEf/MFu1Xr5\n5ZfzwQ9+kI98ZMu7ah1zzDE8/PDDXH311fz0pz/t8UYXQ4YM4eyzz96q9v/6r//i/PPPB+AP//AP\nd6uuHTGYJWmQObe5kZXrk5Xrk47MzcvnNjfueOcd2HQfZOi8F/L2JlN17V/LfZN3557Pd955Jy++\n+CLXX3/9VtsOP/xw5syZs/ly9I033rhVn8bGRiI6f5HZ0/d8NpglaZCZ1NTAlUcMpWlI0LYWmoZE\nn038GjFiBE1NTZvfP/7GN76x+ex5V3zoQx/in/7pnzYH4I7u9wzbvufz7Nmzufnmm/nmN79JXd3W\nMffyyy+z9957c8EFF3DVVVcxZ86cna5z8uTJfPvb3wZg5syZO73frhi40/MkSds0qalhj83Avuuu\nu7jkkktYu3Ythx56KF//+td3+Rif/vSn+cUvfsGkSZNobGzkoosu4rLLLtvuPhdffDFTpkzZ/F7z\nJl/+8pdZsWLF5klfLS0t3HHHHZu3z58/n6uuuoq6ujoaGxv5yle+stN13nLLLVxwwQX8zd/8DVOm\nTOmT+z37cSlJGgC8H3MZ1q5dy1577UVEMHPmTP7lX/6FBx98cKt+3o9ZkqQ9YPbs2Vx22WVkJiNH\njuRrX/tarz+HwSxJ2mMeeughrr766i3aJkyYwHe+851+qmjXvO9972PevHl9+hwGsyQNEJm5eabw\nQHXGGWdwxhln9HcZfarWt4idlS1JA8CwYcNYvnx5zf/pq29lJsuXL2fYsGG7fQzPmCVpAGhubqat\nrY1ly5b1dynagWHDhtX0rWAGsyQNAI2NjUyYMKG/y9Ae4KVsSZIKYjBLklQQg1mSpIIYzJIkFcRg\nliSpIL0SzBExJSKeiYglEXFND9uHRsQ91fbHImJ8l23XVu3PRMQZXdpHRsT9EfF0RCyKiP/RG7VK\nklSymoM5IuqBfwTOBCYCH4+Iid26fQpYmZnvBL4I3FTtOxGYBhwFTAFuq44H8A/ADzPzCOBYYFGt\ntUqSVLreOGM+EViSmc9l5npgJjC1W5+pwF3V8v3AqdH5vXJTgZmZuS4znweWACdGxAjg/cBXATJz\nfWZu/27ckiQNAr0RzOOApV3W26q2HvtkZjuwGhi9nX0nAMuAr0fEf0fEHRExvKcnj4iLI6I1Ilr9\nRhxJ0kBX6uSvBuB44CuZ+W5gDbDVe9cAmTkjM1sys2Xs2LF7skZJknpdbwTzS8DBXdabq7Ye+0RE\nAzACWL6dfduAtsx8rGq/n86gliRpUOuNYH4COCwiJkTEEDonc83q1mcWML1aPg/4cXbeImUWMK2a\ntT0BOAx4PDNfBZZGxLuqfU4FFvZCrZIkFa3mm1hkZntEXAY8BNQDX8vMpyLiRqA1M2fROYnrGxGx\nBFhBZ3hT9buXztBtBy7NzI3VoT8LfKsK++eA/1VrrZIklS4G0709W1pasrW1tb/LkCRpKxExOzNb\ndtSv1MlfkiT9VjKYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJ\nKojBLElSQQxmSZIKYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIw\nS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQV\nxGCWJKkgBrMkSQUxmCVJKojBLElSQQxmSZIKYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkFMZgl\nSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMySJBWkV4I5IqZExDMRsSQirulh+9CIuKfa/lhEjO+y\n7dqq/ZmIOKPbfvUR8d8R8b3eqFOSpNLVHMwRUQ/8I3AmMBH4eERM7NbtU8DKzHwn8EXgpmrficA0\n4ChgCnBbdbxNLgcW1VqjJEkDRW+cMZ8ILMnM5zJzPTATmNqtz1Tgrmr5fuDUiIiqfWZmrsvM54El\n1fGIiGbgw8AdvVCjJEkDQm8E8zhgaZf1tqqtxz6Z2Q6sBkbvYN9bgM8BHdt78oi4OCJaI6J12bJl\nuzsGSZKKUOTkr4g4G/hVZs7eUd/MnJGZLZnZMnbs2D1QnSRJfac3gvkl4OAu681VW499IqIBGAEs\n386+7wXOiYgX6Lw0/sGI+GYv1CpJUtF6I5ifAA6LiAkRMYTOyVyzuvWZBUyvls8DfpyZWbVPq2Zt\nTwAOAx7PzGszszkzx1fH+3FmXtALtUqSVLSGWg+Qme0RcRnwEFAPfC0zn4qIG4HWzJwFfBX4RkQs\nAVbQGbZU/e4FFgLtwKWZubHWmiRJGqii88R1cGhpacnW1tb+LkOSpK1ExOzMbNlRvyInf0mS9NvK\nYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJ\nKojBLElSQQxmSZIKYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIw\nS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQV\nxGCWJKkgBrMkSQUxmCVJKojBLElSQQxmSZIKYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkFMZgl\nSSqIwSxJUkEMZkmSCmIwS5JUkF4J5oiYEhHPRMSSiLimh+1DI+KeavtjETG+y7Zrq/ZnIuKMqu3g\niPhJRCyMiKci4vLeqFOSpNLVHMwRUQ/8I3AmMBH4eERM7NbtU8DKzHwn8EXgpmrficA04ChgCnBb\ndbx24M8ycyIwGbi0h2NKkjTo9MYZ84nAksx8LjPXAzOBqd36TAXuqpbvB06NiKjaZ2bmusx8HlgC\nnJiZr2TmHIDMfANYBIzrhVolSSpabwTzOGBpl/U2tg7RzX0ysx1YDYzemX2ry97vBh7r6ckj4uKI\naI2I1mXLlu32ICRJKkHRk78iYh/g28D/yczXe+qTmTMysyUzW8aOHbtnC5QkqZf1RjC/BBzcZb25\nauuxT0Q0ACOA5dvbNyIa6Qzlb2XmA71QpyRJxeuNYH4COCwiJkTEEDonc83q1mcWML1aPg/4cWZm\n1T6tmrU9ATgMeLx6//mrwKLM/PteqFGSpAGhodYDZGZ7RFwGPATUA1/LzKci4kagNTNn0Rmy34iI\nJcAKOsObqt+9wEI6Z2JfmpkbI+J/Ap8A5kfE3Oqp/jwzv19rvZIklSw6T1wHh5aWlmxtbe3vMiRJ\n2kpEzM7Mlh31K3rylyRJv20MZkmSCmIwS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBTGYJUkqiMEs\nSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJKojBLElSQQxmSZIKYjBLklQQ\ng1mSpIIYzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMySJBXEYJYk\nqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJKojB\nLElSQQxmSZIK0tDfBUgqz5Mr23mgbQO/XJMcMjw4t7mRSU3+dyHtCf5Lk7SFJ1e2c/PT62gaEhy4\n5tcsnPM8920cziErlnL06l/y0RFrOfH3P0LjkUf3d6nSoOSlbElbeKBtA/u9uYJh//YgL33/+zz1\n9hBYt45VDXuxan0H/7DmAP6/a6/nrf/3u/1dqjQoecYs/RbbsGgB637wr3S8vJS6gw5m6Jkf4YWX\nRzLmZw/R8dor/OKQFoauf4uh7et4Y9i+rK9rYMmo3+GyD1zJmf8xm48f8DTHn3BEfw9DGlQMZum3\n1IZFC1g740vEfiOJA8bRsXoVj3/jPuYe8jGen/Ah2n+3gQ119TStXcWoNSsZsnEdrb9zAkPa15MJ\nqxuGc/PcVVxzaLvvP0u9qFcuZUfElIh4JiKWRMQ1PWwfGhH3VNsfi4jxXbZdW7U/ExFn7OwxJdVm\n3Q/+ldhvJHUjRhJ1dSwcfSh/Ne50XmUYaxr3Yl19IxvqG/nVPmNY/I5DeW7UeFbsPZK3G4cy4q3X\naarfyIg3lvNA24b+Hoo0qNT8a25E1AP/CHwIaAOeiIhZmbmwS7dPASsz850RMQ24CfiDiJgITAOO\nAg4CHomIw6t9dnRMSTXoeHkpccC4zesPduzPr7KO9sZ6MqGjvh4IqKtjYwZvNw6lLjeycvR41ry5\njJ/lCN41rJ1frsn+G4Q0CPXGGfOJwJLMfC4z1wMzgand+kwF7qqW7wdOjYio2mdm5rrMfB5YUh1v\nZ44pqQZ1Bx1MvvH65vV564axdO8xLN+7iY76Bog6iKi2Bu0NQ3h7yHD2Wr+WjvoG3t6YPDrqcIbU\ndfTPAKRBqjeCeRywtMt6W9XWY5/MbAdWA6O3s+/OHBOAiLg4IlojonXZsmU1DEP67TL0zI+Qr6+i\nY/Uq5r3ewYtDRkIH3QIZICEAgg31DWxoGMKbjXvB0KHEkKEE0fMTSNotA/7jUpk5IzNbMrNl7Nix\n/V2ONGA0Hnk0e1/8WepGjOTBt0byrhXPs9/6N4Hul6aj8xFBRrChroG3G/eiY6/hvH9sPes6DGap\nN/VGML8EHNxlvblq67FPRDQAI4Dl29l3Z44pqUaNRx7NPldcy8vN7+Kdo4Yx8dVF1HVs79J0EAET\nVrzIfk0jGNZQxyHDDWapN/VGMD8BHBYREyJiCJ2TuWZ16zMLmF4tnwf8ODOzap9WzdqeABwGPL6T\nx5TUSw4ZmrwxZDjLRxzAISuXUr9xA1ufOXde0u6gjpENybL6vVm5Pjm3ubE/SpYGrZqDuXrP+DLg\nIWARcG9mPhURN0bEOVW3rwKjI2IJcAVwTbXvU8C9wELgh8ClmblxW8estVZJPTvvhINZtbGe5fuO\nYf/2Nzl49ctVLv8mnBs2tlPX0cHGqGPl8FG8YyhcecRQP8Ms9bLoPHEdHFpaWrK1tbW/y5AGpDmz\nn+ZPZ7/FsvUwpn0NrWMOp506IMmooz476IigITs4d+UCrr3g/YaytAsiYnZmtuyo34Cf/CWpdxx/\nwhHcdv4xvOed7+DY4RsZ3rGeOjqICBpJIoI6YJ+ODVyeiw1lqY/4L0vSZpOaGvjcSfvzwLhRvOPF\n9Wx8s50h69YSETRE0rixnVNXPc0JZ7y3v0uVBi2DWdIWJjU1MKmpgXObG7nuybd4bWU7b73+OkPf\nfouxDRu57H0TaDzSG1dIfcVgltSjSU0N/M2kvXigrYFfrtmXQ4YH5zY3eglb6mP+C5O0TZvOniXt\nOU7+kiSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBTGY\nJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJKojBLElSQQxmSZIK\nYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMyS\nJBXEYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQWp\nKZgjYlREPBwRi6ufTdvoN73qszgipndpPyEi5kfEkoi4NSKiav+7iHg6Ip6MiO9ExMha6pQkaaCo\n9Yz5GuBHmXkY8KNqfQsRMQq4HjgJOBG4vkuAfwW4CDisekyp2h8Gjs7MScAvgGtrrFOSpAGh1mCe\nCtxVLd8FfLSHPmcAD2fmisxcSWfoTomIA4H9MvPnmZnA3Zv2z8x/y8z2av+fA8011ilJ0oBQazDv\nn5mvVMuvAvv30GccsLTLelvVNq5a7t7e3R8DP9hWARFxcUS0RkTrsmXLdqV2SZKK07CjDhHxCHBA\nD5uu67qSmRkR2VuFVc99HdAOfGtbfTJzBjADoKWlpVefX5KkPW2HwZyZp21rW0S8FhEHZuYr1aXp\nX/XQ7SXg5C7rzcC/V+3N3dpf6nLsC4GzgVOrS92SJA16tV7KngVsmmU9HXiwhz4PAadHRFM16et0\n4KHqEvjrETG5mo39yU37R8QU4HPAOZm5tsYaJUkaMGoN5i8AH4qIxcBp1ToR0RIRdwBk5grgr4An\nqseNVRvAZ4A7gCXAs/zmveQvA/sCD0fE3Ii4vcY6JUkaEGIwXSVuaWnJ1tbW/i5DkqStRMTszGzZ\nUT+/+UuSpIIYzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMySJBXE\nYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJ\nKojBLElSQQxmSZIKYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIw\nS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQV\nxGCWJKkgBrMkSQUxmCVJKojBLElSQQxmSZIKUlMwR8SoiHg4IhZXP5u20W961WdxREzv0n5CRMyP\niCURcWtERLf9/iwiMiLG1FKnJEkDRa1nzNcAP8rMw4AfVetbiIhRwPXAScCJwPVdAvwrwEXAYdVj\nSpf9DgZOB35ZY42SJA0YtQbzVOCuavku4KM99DkDeDgzV2TmSuBhYEpEHAjsl5k/z8wE7u62/xeB\nzwFZY42SJA0YtQbz/pn5SrX8KrB/D33GAUu7rLdVbeOq5e7tRMRU4KXMnLejAiLi4ohojYjWZcuW\n7cYQJEkqR8OOOkTEI8ABPWy6rutKZmZE1Hx2GxF7A39O52XsHcrMGcAMgJaWFs+uJUkD2g6DOTNP\n29a2iHgtIg7MzFeqS9O/6qHbS8DJXdabgX+v2pu7tb8E/C4wAZhXzQVrBuZExImZ+eqO6pUkaSCr\n9VL2LGDTLOvpwIM99HkIOD0imqpJX6cDD1WXwF+PiMnVbOxPAg9m5vzMfEdmjs/M8XRe4j7eUJYk\n/TaoNZi/AHwoIhYDp1XrRERLRNwBkJkrgL8CnqgeN1ZtAJ8B7gCWAM8CP6ixHkmSBrTonBA9OLS0\ntGRra2t/lyFJ0lYiYnZmtuyon9/8JUlSQQxmSZIKYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkF\nMZglSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJ\nkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJKojBLElSQQxmSZIKYjBLklQQg1mSpIIY\nzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJ\nBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQWpKZgjYlREPBwRi6uf\nTdvoN73qszgipndpPyEi5kfEkoi4NSKiy7bPRsTTEfFURPxtLXVKkjRQ1HrGfA3wo8w8DPhRtb6F\niBgFXA+cBJwIXN8lwL8CXAQcVj2mVPucAkwFjs3Mo4Cba6xTkqQBodZgngrcVS3fBXy0hz5nAA9n\n5orMXAk8DEyJiAOB/TLz55mZwN1d9v/fwBcycx1AZv6qxjolSRoQag3m/TPzlWr5VWD/HvqMA5Z2\nWW+r2sZVy93bAQ4H3hcRj0XEf0TEe7ZVQERcHBGtEdG6bNmy3R2HJElFaNhRh4h4BDigh03XdV3J\nzIyI7MW6RgGTgfcA90bEodWZ9RYycwYwA6ClpaW3nl+SpH6xw2DOzNO2tS0iXouIAzPzlerSdE+X\nnF8CTu6y3gz8e9Xe3K39pWq5DXigCuLHI6IDGAN4SixJGtRqvZQ9C9g0y3o68GAPfR4CTo+IpmrS\n1+nAQ9Ul8NcjYnI1G/uTXfb/LnAKQEQcDgwBfl1jrZIkFa/WYP4C8KGIWAycVq0TES0RcQdAZq4A\n/gp4onrcWLUBfAa4A1gCPAv8oGr/GnBoRCwAZgLTe7qMLUnSYBODKe9aWlqytbW1v8uQJGkrETE7\nM1t21M9v/pIkqSAGsyRJBTGYJUkqiMEsSVJBDGZJkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMk\nSQUxmCXzdhm7AAAGLUlEQVRJKojBLElSQQxmSZIKYjBLklQQg1mSpIIYzJIkFcRgliSpIAazJEkF\nMZglSSqIwSxJUkEMZkmSCmIwS5JUEINZkqSCGMySJBXEYJYkqSAGsyRJBTGYJUkqiMEsSVJBDGZJ\nkgpiMEuSVBCDWZKkghjMkiQVxGCWJKkgBrMkSQUxmCVJKojBLElSQQxmSZIKYjBLklQQg1mSpIIY\nzJIkFcRgliSpIAazJEkFMZglSSqIwSxJUkEiM/u7hl4TEcuAF/u7jt00Bvh1fxfRhxzfwOb4BjbH\nV4bfycyxO+o0qIJ5IIuI1sxs6e86+orjG9gc38Dm+AYWL2VLklQQg1mSpIIYzOWY0d8F9DHHN7A5\nvoHN8Q0gvscsSVJBPGOWJKkgBrMkSQUxmPtQRIyKiIcjYnH1s2kb/aZXfRZHxPQu7SdExPyIWBIR\nt0ZEdNn22Yh4OiKeioi/3RPj6aHuPhtftf3PIiIjYkxfj6UnfTW+iPi76rV7MiK+ExEj99SYquef\nEhHPVHVd08P2oRFxT7X9sYgY32XbtVX7MxFxxs4ec0/q7fFFxMER8ZOIWFj9e7t8z41ma33x+lXb\n6iPivyPie30/im3ro7+fIyPi/urf3aKI+B97ZjS7KTN99NED+Fvgmmr5GuCmHvqMAp6rfjZVy03V\ntseByUAAPwDOrNpPAR4Bhlbr7xhM46u2HQw8ROcXxowZTOMDTgcaquWbejpuH46pHngWOBQYAswD\nJnbr8xng9mp5GnBPtTyx6j8UmFAdp35njjnAx3cgcHzVZ1/gF4NpfF32uwL4v8D3+mNsfTk+4C7g\n09XyEGBkf41xZx6eMfetqXT+haD6+dEe+pwBPJyZKzJzJfAwMCUiDgT2y8yfZ+ffpru77P+/gS9k\n5jqAzPxVXw5iO/pqfABfBD4H9OfsxD4ZX2b+W2a2V/v/HGjuy0F0cyKwJDOfy8z1wEw6x9lV13Hf\nD5xane1PBWZm5rrMfB5YUh1vZ465p/T6+DLzlcycA5CZbwCLgHF7YCw96YvXj4hoBj4M3LEHxrA9\nvT6+iBgBvB/4KkBmrs/MVXtgLLvNYO5b+2fmK9Xyq8D+PfQZByztst5WtY2rlru3AxwOvK+6jPMf\nEfGe3i17p/XJ+CJiKvBSZs7r9Yp3TV+9fl39MZ1n03vKturtsU/1C8RqYPR29t2ZY+4pfTG+zarL\npu8GHuvFmndFX43vFjp/Ee7o/ZJ3SV+MbwKwDPh6dan+jogY3jfl946G/i5goIuIR4ADeth0XdeV\nzMyI6K2zvwY6L51OBt4D3BsRh1ZnZr1qT48vIvYG/pzOy719rp9ev03PfR3QDnyrN4+rvhER+wDf\nBv5PZr7e3/X0log4G/hVZs6OiJP7u54+0AAcD3w2Mx+LiH+g862p/6d/y9o2g7lGmXnatrZFxGsR\ncWBmvlJd2uzpkvNLwMld1puBf6/am7u1v1QttwEPVEH8eER00Pkl7st2dxzb0g/j+106f8OdV82V\nagbmRMSJmflqDUPpUT+9fkTEhcDZwKl98QvVdrxE5/v3PdbVrU9bRDQAI4DlO9h3R8fcU/pkfBHR\nSGcofyszH+ib0ndKX4zvHOCciDgLGAbsFxHfzMwL+mYI29UX42sD2jJz01WO++kM5nL195vcg/kB\n/B1bTh762x76jAKep3PiUFO1PKra1n3y0FlV+yXAjdXy4XRevonBMr5u+79A/03+6qvXbwqwEBjb\nD2NqoHOC2gR+M7nmqG59LmXLyTX3VstHseXkmufonKyzw2MO8PEFnXMEbumPMfX1+LrtezL9O/mr\nT8YH/BR4V7V8A/B3/f1abvfPob8LGMwPOt/3+BGwmM5Z1Jv+w24B7ujS74/pnKiwBPhfXdpbgAV0\nzi78Mr/5prYhwDerbXOADw6m8XV7jhfov2Duq9dvCZ2/TM2tHrfv4XGdRefM4meB66q2G4FzquVh\nwH1VnY8Dh3bZ97pqv2fYchb9Vsfsr0dvjw/4n3ROQnyyy2u21S+RA3V83Y59Mv0YzH349/M4oLV6\nDb9L9cmJUh9+JackSQVxVrYkSQUxmCVJKojBLElSQQxmSZIKYjBLklQQg1mSpIIYzJIkFeT/B43U\n5shS7dFcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5fc3fbf128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# we have suggested some parameters below, feel free to experiment\n",
    "tsne = TSNE(perplexity = 800, random_state = 42)\n",
    "\n",
    "tsne_comp = tsne.fit_transform(tfidf_vectors.toarray())\n",
    "\n",
    "scatter(tsne_comp, df_full['label'], ['critizising', 'non_critizising'], COLOURS[:2], \"TSNE\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected it fails PCA and TSNE using vader lexicon fails."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3 ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We now get all the sentences in the high_critical papers and a similar number of sentences from the non_critical papers. We get the words and bigrams used in the high_critical set and are not used in the non_critical set.\n"
     ]
    }
   ],
   "source": [
    "print(\"We now get all the sentences in the high_critical papers and a similar number of sentences from the non_critical papers. We get the words and bigrams used in the high_critical set and are not used in the non_critical set.\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's analyse the sentences where the paper is cited."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['appeared', 'association', 'carried', 'cells', 'chronic', 'concerning', 'fatigue', 'findings', 'found', 'inconsistent', 'isolation', 'levels', 'negative', 'paper', 'patients', 'previous', 'published', 'report', 'reporting', 'syndrome']\n",
      "2,\"['More recently, additional groups of samples from patients with prostate cancer have been examined for the presence of XMRV with both positive [<xref ref-type=\"\"bibr\"\" rid=\"\"B5\"\">5</xref>] and negative [<xref ref-type=\"\"bibr\"\" rid=\"\"B6\"\">6</xref>,<xref ref-type=\"\"bibr\"\" rid=\"\"B7\"\">7</xref>] results.</p><p>Very recently, a paper reporting the PCR detection of XMRV in PBMC from 68/101 patients with chronic fatigue syndrome (CFS) has been published [<xref ref-type=\"\"bibr\"\" rid=\"\"B8\"\">8</xref>].']\"\n",
      "['agreement', 'among', 'antibody', 'cell', 'compared', 'could', 'data', 'failed', 'finding', 'large', 'leukemia', 'murine', 'one', 'patients', 'persons', 'recent', 'reported', 'responses', 'sequences', 'specific', 'studies', 'study', 'testing', 'two', 'using', 'virus']\n",
      "6,\"['Our results contrast with the high rate of XMRV detection reported by Lombardi<italic>et al.</italic>among both CFS patients and controls, but are in agreement with recent data reported in two large studies in the UK and a smaller study in the Netherlands that could not detect XMRV sequences in CFS patients and one UK study that also failed to detect specific XMRV neutralizing antibody responses in CFS [<xref ref-type=\"\"bibr\"\" rid=\"\"B11\"\">11</xref>,<xref ref-type=\"\"bibr\"\" rid=\"\"B16\"\">16</xref>-<xref ref-type=\"\"bibr\"\" rid=\"\"B18\"\">18</xref>].']\"\n",
      "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n",
      "Set of words only found in the highCritical paragraphs\n",
      "{'carried', 'paper', 'negative', 'fatigue', 'syndrome', 'previous', 'concerning', 'association', 'appeared', 'found', 'cells', 'findings', 'inconsistent', 'published', 'reporting', 'report', 'levels', 'chronic', 'isolation'}\n",
      "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n",
      "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n",
      "Set of words only found in the nonCritical paragraphs\n",
      "{'recent', 'data', 'among', 'specific', 'compared', 'testing', 'murine', 'agreement', 'sequences', 'study', 'failed', 'persons', 'cell', 'two', 'antibody', 'finding', 'reported', 'leukemia', 'studies', 'large', 'using', 'could', 'virus', 'responses', 'one'}\n",
      "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n",
      "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n",
      "Set of bi-grams only found in the highCritical paragraphs\n",
      "{(1, 'reporting patients'), (1, 'syndrome published'), (1, 'negative paper'), (1, 'inconsistent previous'), (1, 'report isolation'), (1, 'patients carried'), (1, 'carried levels'), (1, 'isolation patients'), (1, 'concerning association'), (1, 'appeared inconsistent'), (1, 'patients chronic'), (1, 'paper reporting'), (1, 'findings appeared'), (1, 'found patients'), (1, 'fatigue syndrome'), (1, 'chronic fatigue'), (1, 'levels cells'), (1, 'previous report')}\n",
      "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n"
     ]
    }
   ],
   "source": [
    "#print(\"df_highCritical.shape; \", df_highCritical.shape)\n",
    "#print(\"df_Critical.shape; \", df_Critical.shape)\n",
    "#print(\"df_nonCritical.shape: \", df_nonCritical.shape)\n",
    "#print(\"df_full.shape: \", df_full.shape)\n",
    "\n",
    "#file_df = '/project/elife/data/analysis/df_science1179052_retracted.csv'\n",
    "\n",
    "#file_df = '/project/elife/data/analysis/df_1000_1000v2_prep_.csv'\n",
    "#df = pd.read_csv(file_df, sep='\\t', header = None, encoding='utf-8', names = ['cited_DOI','cited_in_conclusions','cited_in_discussion','cited_in_introduction', 'cited_in_maintext', 'citing_DOI','conclusions_found', 'discussion_found', 'introduction_found','maintext_found', 'reference_id', 'sentence_citing_conclusions','sentence_citing_discussion', 'sentence_citing_intro', 'sentence_citing_maintext'])\n",
    "#  df['cited_DOI', 'cited_in_conclusions','cited_in_discussion',\n",
    "#    'cited_in_introduction', 'cited_in_maintext', 'citing_DOI',\n",
    "#    'conclusions_found', 'discussion_found', 'introduction_found',\n",
    "#    'maintext_found', 'reference_id', 'sentence_citing_conclusions',\n",
    "#    'sentence_citing_discussion', 'sentence_citing_intro', 'sentence_citing_maintext']\n",
    "\n",
    "df_full_highCritical_sample = df_full_highCritical # All the papers the retraction is based on\n",
    "#df_full_highCritical_sample = df_full_Critical.sample(4, random_state = 1)\n",
    "\n",
    "sentences_highCritical = df_full_highCritical_sample.text.dropna()\n",
    "frequent_words_highCritical = words_frec_analysis_get_sentence.analysis_nolimit(sentences_highCritical)\n",
    "\n",
    "# I get a similar number of nonCritical text paragraphs:\n",
    "df_full_nonCritical_sample = df_full_nonCritical.sample(2*sentences_highCritical.shape[0], random_state = 10)\n",
    "\n",
    "sentences_nonCritical = df_full_nonCritical_sample.text.dropna()\n",
    "frequent_words_nonCritical = words_frec_analysis_get_sentence.analysis_nolimit(sentences_nonCritical)\n",
    "\n",
    "\n",
    "set_words_nonCritical = set(frequent_words_nonCritical)\n",
    "set_words_highCritical = set(frequent_words_highCritical)\n",
    "\n",
    "words_only_highCritical = set_words_highCritical - set_words_nonCritical\n",
    "\n",
    "print(\"\\n-------------------------------------------------------------\")\n",
    "print(\"Set of words only found in the highCritical paragraphs\")\n",
    "print(words_only_highCritical)\n",
    "print(\"-------------------------------------------------------------\")\n",
    "\n",
    "words_only_nonCritical = set_words_nonCritical - set_words_highCritical\n",
    "\n",
    "print(\"\\n-------------------------------------------------------------\")\n",
    "print(\"Set of words only found in the nonCritical paragraphs\")\n",
    "print(words_only_nonCritical)\n",
    "print(\"-------------------------------------------------------------\")\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "bigrams_highCritical = words_frec_analysis_get_sentence.analyse_bigrams(sentences_highCritical)\n",
    "bigrams_nonCritical =  words_frec_analysis_get_sentence.analyse_bigrams(sentences_nonCritical)    \n",
    "\n",
    "set_bigrams_highCritical = set(bigrams_highCritical)\n",
    "set_bigrams_nonCritical = set(bigrams_nonCritical)\n",
    "\n",
    "bigrams_only_highCritical = set_bigrams_highCritical-set_bigrams_nonCritical\n",
    "\n",
    "\n",
    "print(\"\\n-------------------------------------------------------------\")\n",
    "print(\"Set of bi-grams only found in the highCritical paragraphs\")\n",
    "print(bigrams_only_highCritical)\n",
    "print(\"-------------------------------------------------------------\")\n",
    "\n",
    "\n",
    "### Let's check the frequency of apereance of the words and bigrams highCritical in all the papers###\n",
    "#df_full_Critical['mark'] =\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SOME CHECKS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    ['The virus was not detected in any of our stu...\n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#sentences_highCritical[1:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "yes\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "with open('./english_words/wordsEn.txt', 'r') as word_file:\n",
    "    english_words = list(word.strip().lower() for word in word_file)\n",
    "#english_words[1000:1005]\n",
    "\n",
    "if 'workload' in english_words:\n",
    "    print(\"yes\")\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0,\"['[<xref ref-type=\"\"bibr\"\" rid=\"\"B8\"\">8</xref>] concerning the association of XMRV with CFS.']\"\n",
      "1,\"['The virus was not detected in any of our study subjects; these results do not support an association between XMRV and CFS in Chinese.</p><p>Our findings appeared to be inconsistent with a previous report of XMRV DNA isolation from PBMCs of CFS patients in the US [<xref ref-type=\"\"bibr\"\" rid=\"\"B3\"\">3</xref>].']\"\n",
      "2,\"['More recently, additional groups of samples from patients with prostate cancer have been examined for the presence of XMRV with both positive [<xref ref-type=\"\"bibr\"\" rid=\"\"B5\"\">5</xref>] and negative [<xref ref-type=\"\"bibr\"\" rid=\"\"B6\"\">6</xref>,<xref ref-type=\"\"bibr\"\" rid=\"\"B7\"\">7</xref>] results.</p><p>Very recently, a paper reporting the PCR detection of XMRV in PBMC from 68/101 patients with chronic fatigue syndrome (CFS) has been published [<xref ref-type=\"\"bibr\"\" rid=\"\"B8\"\">8</xref>].']\"\n",
      "3,\"['In a recent US study, Lombardi et al [<xref ref-type=\"\"bibr\"\" rid=\"\"B3\"\">3</xref>] found that about 67% (68/101) of patients with CFS carried detectable levels of XMRV DNA in their peripheral blood mononuclear cells (PBMCs).']\"\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "sentence = sentences_highCritical[0:4]\n",
    "sentence.to_csv(sys.stdout)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'agreement'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frequent_words_nonCritical[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['appeared',\n",
       " 'association',\n",
       " 'carried',\n",
       " 'cells',\n",
       " 'chronic',\n",
       " 'concerning',\n",
       " 'fatigue',\n",
       " 'findings',\n",
       " 'found',\n",
       " 'inconsistent',\n",
       " 'isolation',\n",
       " 'levels',\n",
       " 'negative',\n",
       " 'paper',\n",
       " 'patients',\n",
       " 'previous',\n",
       " 'published',\n",
       " 'report',\n",
       " 'reporting',\n",
       " 'syndrome']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frequent_words_highCritical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>citing_DOI</th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.3201/eid1606.100066</td>\n",
       "      <td>['&lt;sec sec-type=\"other2\"&gt;&lt;title&gt;Conclusions&lt;/t...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.1186/1742-4690-7-112</td>\n",
       "      <td>['Additional findings from the Hué study sugge...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.1186/1479-5876-8-1</td>\n",
       "      <td>['These deficiencies in NK activity may increa...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10.1186/1479-5876-7-96</td>\n",
       "      <td>['Both cell associated and cell-free transmiss...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10.1186/1742-4690-7-111</td>\n",
       "      <td>['&lt;sec&gt;&lt;title&gt;Results and Discussion&lt;/title&gt;&lt;s...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10.1186/1742-4690-7-109</td>\n",
       "      <td>['For this purpose, we accumulated a library o...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10.1186/1742-4690-7-57</td>\n",
       "      <td>['Our results contrast with the high rate of X...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10.1186/1742-4690-7-68</td>\n",
       "      <td>['Several studies have reported detection of r...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10.1186/1471-2377-10-73</td>\n",
       "      <td>['Recently, a link between the xenotropic muri...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10.1186/1744-9081-6-76</td>\n",
       "      <td>['Evidence of xenotropic murine leukemia virus...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10.1186/1479-5876-7-96</td>\n",
       "      <td>['These observations have led to reports of as...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>10.1186/1742-4690-7-70</td>\n",
       "      <td>['Importantly, the chromosomal locations of XM...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>10.1186/1742-4690-7-112</td>\n",
       "      <td>['&lt;sec&gt;&lt;title&gt;Introduction&lt;/title&gt;&lt;p&gt;Reports o...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10.1186/1742-4690-7-111</td>\n",
       "      <td>['XMRV has also been detected in 1-6% of healt...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>10.1186/1742-4690-7-109</td>\n",
       "      <td>['In 2009, it was reported that 67% of chronic...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>10.1186/1742-4690-7-108</td>\n",
       "      <td>['Several papers have since demonstrated a lin...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>10.1186/1742-4690-7-113</td>\n",
       "      <td>['[&lt;xref ref-type=\"bibr\" rid=\"B3\"&gt;3&lt;/xref&gt;], w...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>10.1186/1742-4690-7-57</td>\n",
       "      <td>['In October 2009, Lombardi&lt;italic&gt;et al.&lt;/ita...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>10.1186/1742-4690-7-68</td>\n",
       "      <td>['Recently, XMRV has been also identified in 6...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 citing_DOI  \\\n",
       "0    10.3201/eid1606.100066   \n",
       "1   10.1186/1742-4690-7-112   \n",
       "2     10.1186/1479-5876-8-1   \n",
       "3    10.1186/1479-5876-7-96   \n",
       "4   10.1186/1742-4690-7-111   \n",
       "5   10.1186/1742-4690-7-109   \n",
       "6    10.1186/1742-4690-7-57   \n",
       "7    10.1186/1742-4690-7-68   \n",
       "8   10.1186/1471-2377-10-73   \n",
       "9    10.1186/1744-9081-6-76   \n",
       "10   10.1186/1479-5876-7-96   \n",
       "11   10.1186/1742-4690-7-70   \n",
       "12  10.1186/1742-4690-7-112   \n",
       "13  10.1186/1742-4690-7-111   \n",
       "14  10.1186/1742-4690-7-109   \n",
       "15  10.1186/1742-4690-7-108   \n",
       "16  10.1186/1742-4690-7-113   \n",
       "17   10.1186/1742-4690-7-57   \n",
       "18   10.1186/1742-4690-7-68   \n",
       "\n",
       "                                                 text            label  \n",
       "0   ['<sec sec-type=\"other2\"><title>Conclusions</t...  non_critizising  \n",
       "1   ['Additional findings from the Hué study sugge...  non_critizising  \n",
       "2   ['These deficiencies in NK activity may increa...  non_critizising  \n",
       "3   ['Both cell associated and cell-free transmiss...  non_critizising  \n",
       "4   ['<sec><title>Results and Discussion</title><s...  non_critizising  \n",
       "5   ['For this purpose, we accumulated a library o...  non_critizising  \n",
       "6   ['Our results contrast with the high rate of X...  non_critizising  \n",
       "7   ['Several studies have reported detection of r...  non_critizising  \n",
       "8   ['Recently, a link between the xenotropic muri...  non_critizising  \n",
       "9   ['Evidence of xenotropic murine leukemia virus...  non_critizising  \n",
       "10  ['These observations have led to reports of as...  non_critizising  \n",
       "11  ['Importantly, the chromosomal locations of XM...  non_critizising  \n",
       "12  ['<sec><title>Introduction</title><p>Reports o...  non_critizising  \n",
       "13  ['XMRV has also been detected in 1-6% of healt...  non_critizising  \n",
       "14  ['In 2009, it was reported that 67% of chronic...  non_critizising  \n",
       "15  ['Several papers have since demonstrated a lin...  non_critizising  \n",
       "16  ['[<xref ref-type=\"bibr\" rid=\"B3\">3</xref>], w...  non_critizising  \n",
       "17  ['In October 2009, Lombardi<italic>et al.</ita...  non_critizising  \n",
       "18  ['Recently, XMRV has been also identified in 6...  non_critizising  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_full_nonCritical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>citing_DOI</th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10.1186/1479-5876-7-96</td>\n",
       "      <td>['Both cell associated and cell-free transmiss...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>10.1186/1742-4690-7-108</td>\n",
       "      <td>['Several papers have since demonstrated a lin...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10.1186/1742-4690-7-57</td>\n",
       "      <td>['Our results contrast with the high rate of X...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10.1186/1479-5876-7-96</td>\n",
       "      <td>['These observations have led to reports of as...</td>\n",
       "      <td>non_critizising</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 citing_DOI  \\\n",
       "3    10.1186/1479-5876-7-96   \n",
       "15  10.1186/1742-4690-7-108   \n",
       "6    10.1186/1742-4690-7-57   \n",
       "10   10.1186/1479-5876-7-96   \n",
       "\n",
       "                                                 text            label  \n",
       "3   ['Both cell associated and cell-free transmiss...  non_critizising  \n",
       "15  ['Several papers have since demonstrated a lin...  non_critizising  \n",
       "6   ['Our results contrast with the high rate of X...  non_critizising  \n",
       "10  ['These observations have led to reports of as...  non_critizising  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_full_nonCritical.sample(4, random_state = 1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}