notebooks/s2qa_sources_langchain.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from utils import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "query1 = \"what are some transformer based approaches for information retrieval?\"\n",
    "\n",
    "# search and re-rank for the query\n",
    "df = get_results(query1)\n",
    "df, query1 = rerank(df, query1)\n",
    "docs = get_doc_objects_from_df(df)\n",
    "chain_output = get_langchain_response(docs, query1, k=5) #  k=df.shape[0] to get all the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### what are some transformer based approaches for information retrieval?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Answer\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Transformer based approaches for information retrieval include a hybrid approach combining classical (PRF) and transformer (AraBERT), a system based on a pretrained BERT for document and snippet retrieval, question answering and summarization, and a hierarchical architecture such as a transformer to build a document-level representation on top of the representations of each passage.\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Sources: \n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [Lecture Notes on Neural Information Retrieval](https://www.semanticscholar.org/paper/4e042409c4dc5210787fcd149485a59be3eef0f6) - ArXiv, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [Improving Efficiency and Robustness of Transformer-based Information Retrieval Systems](https://www.semanticscholar.org/paper/80d7b66d4d6a0d47bf4e7e9ef1277e72b9526cfb) - Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [A Hybrid Semantic Statistical Query Expansion for Arabic Information Retrieval Systems](https://www.semanticscholar.org/paper/9abbb9abf2d512f210e33e3e923bb726323ed874) - International Symposium on Information and Automation, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [Transformer-Based Open Domain Biomedical Question Answering at BioASQ8 Challenge](https://www.semanticscholar.org/paper/ce1956497b7d8e40860d18f4484fccfa00276884) - Conference and Labs of the Evaluation Forum, 2020"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [KeyBLD: Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval](https://www.semanticscholar.org/paper/9583d94fe6297acb3f287329dff6fe09b1c5c4c2) - Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "return_answer_markdown(chain_output, df, query1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "query2 = \"How does iron supplementation affect anemia?\"\n",
    "\n",
    "# search and re-rank for the query\n",
    "df = get_results(query2)\n",
    "df, query1 = rerank(df, query2)\n",
    "docs = get_doc_objects_from_df(df)\n",
    "chain_output = get_langchain_response(docs, query2, k=5) #  k=df.shape[0] to get all the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "## Question\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### How does iron supplementation affect anemia?\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Answer\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Iron supplementation can be effective at improving iron status and preventing iron deficiency anemia. Studies have shown that iron supplementation during pregnancy can have beneficial effects on mother and her child, such as maintaining hemoglobin and serum iron at adequate levels. Additionally, iron and vitamin C supplementation has a greater impact on the changes of hemoglobin levels compared to iron supplementation alone. Iron supplementation in mothers with high hemoglobin did not increase the risk of maternal anemia, however, prenatal iron and multiple micronutrient supplements did not affect perinatal anemia.\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Sources: \n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [Efficacy of daily versus intermittent administration of iron supplementation in anemia or blood indices during pregnancy](https://www.semanticscholar.org/paper/a6d2264066edf46f03f1c7df46b0e8079f8e0b63) - , 2013"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [Effectivity of Iron Supplementation With and Without Vitamin C for Increasing Hemoglobin Levels Among Women Aged 16-21 Years Old](https://www.semanticscholar.org/paper/e056f91c0ba1bc3214e0f89aaf375cafc95f3da3) - SRIWIJAYA JOURNAL OF MEDICINE, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [The effect of iron supplementation on birth outcome in mothers with high hemoglobin: a randomized double-blind clinical trial study](https://www.semanticscholar.org/paper/f312120e9e0fa93842dadf5cc55ee367e4d22615) - , 2019"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [Benefits and Risks of Early Life Iron Supplementation](https://www.semanticscholar.org/paper/459828144e59ace72493adae1068d460d0989c46) - Nutrients, 2022"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "* #### [Iron-containing micronutrient supplementation of Chinese women with no or mild anemia during pregnancy improved iron status but did not affect perinatal anemia.](https://www.semanticscholar.org/paper/c910b35eb94711fb0f17075f88e132f8cf0bf85a) - Journal of NutriLife, 2014"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "return_answer_markdown(chain_output, df, query2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "misc",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "4fbfa9044b95e6f6717ef013aac1b71f8b1f998a3df349d4a874bbb0804681f7"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}