bin/02_intro-dataset_solutions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tutorial Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this tutorial, we will be using a subset of a publicly available dataset, ds000030, from [openneuro.org](https://openneuro.org/datasets/ds000030). The dataset is structured according to the Brain Imaging Data Structure ([BIDS](https://bids-specification.readthedocs.io/en/stable/)). BIDS is a simple and intuitive way to organize and describe your neuroimaging and behavioural data. Neuroimaging experiments result in complicated data that can be arranged in several different ways. BIDS tackles this problem by suggesting a new standard (based on consensus from multiple researchers across the world) for the arrangement of neuroimaging datasets. Using the same organizational standard for *all* of your studies will also allow you to easily reuse your scripts and share data and code with other researchers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a tree diagram showing the folder structure of single MR session within ds000030. This was obtained by using the bash command `tree`.  \n",
    "`!tree ../data/ds000030`\n",
    "\n",
    "```\n",
    "ds000030\n",
    "├── CHANGES\n",
    "├── dataset_description.json\n",
    "├── derivatives\n",
    "│   └── fmriprep\n",
    "├── participants.tsv\n",
    "├── README\n",
    "├── sub-50083\n",
    "│   ├── anat\n",
    "│   │   ├── sub-50083_T1w.json\n",
    "│   │   └── sub-50083_T1w.nii.gz\n",
    "│   └── func\n",
    "│       ├── sub-50083_task-rest_bold.json\n",
    "│       └── sub-50083_task-rest_bold.nii.gz\n",
    "└── task-rest_bold.json\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `participants.tsv` file is meant to describe some demographic information on each participant within your study (eg. age, handedness, sex, etc.) Let's take a look at the `participants.tsv` file to see what's been included in this dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to load the data into Python, we'll need to import the `pandas` package. The `pandas` **dataframe** is Python's equivalent to an Excel spreadsheet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use the `read_csv()` function. It requires us to specify the name of the file we want to import and the separator that is used to distinguish each column in our file (`\\t` since we're working with a `.tsv` file)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "participant_metadata = pd.read_csv('../data/ds000030/participants.tsv', sep='\\t')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to get a glimpse of our data, we'll use the `head()` function. By default, `head` prints the first 5 rows of our dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>participant_id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>bart</th>\n",
       "      <th>bht</th>\n",
       "      <th>dwi</th>\n",
       "      <th>pamenc</th>\n",
       "      <th>pamret</th>\n",
       "      <th>rest</th>\n",
       "      <th>scap</th>\n",
       "      <th>stopsignal</th>\n",
       "      <th>T1w</th>\n",
       "      <th>taskswitch</th>\n",
       "      <th>ScannerSerialNumber</th>\n",
       "      <th>ghost_NoGhost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>sub-10159</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>30</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>sub-10171</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>24</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>sub-10189</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>49</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>sub-10193</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>40</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sub-10206</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>21</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  participant_id diagnosis  age gender  bart  bht  dwi  pamenc  pamret  rest  \\\n",
       "0      sub-10159   CONTROL   30      F   1.0  NaN  1.0     NaN     NaN   1.0   \n",
       "1      sub-10171   CONTROL   24      M   1.0  1.0  1.0     NaN     NaN   1.0   \n",
       "2      sub-10189   CONTROL   49      M   1.0  NaN  1.0     NaN     NaN   1.0   \n",
       "3      sub-10193   CONTROL   40      M   1.0  NaN  1.0     NaN     NaN   NaN   \n",
       "4      sub-10206   CONTROL   21      M   1.0  NaN  1.0     NaN     NaN   1.0   \n",
       "\n",
       "   scap  stopsignal  T1w  taskswitch  ScannerSerialNumber ghost_NoGhost  \n",
       "0   1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "1   1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "2   1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "3   NaN         NaN  1.0         NaN              35343.0      No_ghost  \n",
       "4   1.0         1.0  1.0         1.0              35343.0      No_ghost  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can view any number of rows by specifying `n=?` as an argument within `head()`.  \n",
    "If we want to select particular rows within the dataframe, we can use the `loc[]` function and identify the rows we want based on their index label (the numbers in the left-most column)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>participant_id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>bart</th>\n",
       "      <th>bht</th>\n",
       "      <th>dwi</th>\n",
       "      <th>pamenc</th>\n",
       "      <th>pamret</th>\n",
       "      <th>rest</th>\n",
       "      <th>scap</th>\n",
       "      <th>stopsignal</th>\n",
       "      <th>T1w</th>\n",
       "      <th>taskswitch</th>\n",
       "      <th>ScannerSerialNumber</th>\n",
       "      <th>ghost_NoGhost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>sub-10225</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>35</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>sub-10249</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>28</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>sub-10271</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>41</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   participant_id diagnosis  age gender  bart  bht  dwi  pamenc  pamret  rest  \\\n",
       "6       sub-10225   CONTROL   35      M   1.0  1.0  1.0     NaN     NaN   1.0   \n",
       "10      sub-10249   CONTROL   28      M   1.0  1.0  1.0     NaN     NaN   1.0   \n",
       "12      sub-10271   CONTROL   41      F   1.0  1.0  1.0     NaN     NaN   1.0   \n",
       "\n",
       "    scap  stopsignal  T1w  taskswitch  ScannerSerialNumber ghost_NoGhost  \n",
       "6    1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "10   1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "12   1.0         1.0  1.0         1.0              35343.0      No_ghost  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata.loc[[6, 10, 12]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE**: Select the first 5 rows of the dataframe using `loc[]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>participant_id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>bart</th>\n",
       "      <th>bht</th>\n",
       "      <th>dwi</th>\n",
       "      <th>pamenc</th>\n",
       "      <th>pamret</th>\n",
       "      <th>rest</th>\n",
       "      <th>scap</th>\n",
       "      <th>stopsignal</th>\n",
       "      <th>T1w</th>\n",
       "      <th>taskswitch</th>\n",
       "      <th>ScannerSerialNumber</th>\n",
       "      <th>ghost_NoGhost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>sub-10159</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>30</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>sub-10171</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>24</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>sub-10189</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>49</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>sub-10193</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>40</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sub-10206</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>21</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  participant_id diagnosis  age gender  bart  bht  dwi  pamenc  pamret  rest  \\\n",
       "0      sub-10159   CONTROL   30      F   1.0  NaN  1.0     NaN     NaN   1.0   \n",
       "1      sub-10171   CONTROL   24      M   1.0  1.0  1.0     NaN     NaN   1.0   \n",
       "2      sub-10189   CONTROL   49      M   1.0  NaN  1.0     NaN     NaN   1.0   \n",
       "3      sub-10193   CONTROL   40      M   1.0  NaN  1.0     NaN     NaN   NaN   \n",
       "4      sub-10206   CONTROL   21      M   1.0  NaN  1.0     NaN     NaN   1.0   \n",
       "\n",
       "   scap  stopsignal  T1w  taskswitch  ScannerSerialNumber ghost_NoGhost  \n",
       "0   1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "1   1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "2   1.0         1.0  1.0         1.0              35343.0      No_ghost  \n",
       "3   NaN         NaN  1.0         NaN              35343.0      No_ghost  \n",
       "4   1.0         1.0  1.0         1.0              35343.0      No_ghost  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata.loc[:4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE:** How many participants do we have in total?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(272, 16)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 2 different methods of selecting columns in a dataframe:  \n",
    "*  participant_metadata[`'<column_name>'`] (this is similar to selecting a key in a Python dictionary)  \n",
    "*  participant_metadata.`<column_name>`  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to see how many participants are in the study is to select the `participant_id` column and use the `count()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "272"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata['participant_id'].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE:** Which diagnosis groups are part of the study?  \n",
    "*Hint: use the* `unique()` *function.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['CONTROL', 'SCHZ', 'BIPOLAR', 'ADHD'], dtype=object)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata['diagnosis'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to count the number of participants in each diagnosis group, we can use the `value_counts()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CONTROL    130\n",
       "SCHZ        50\n",
       "BIPOLAR     49\n",
       "ADHD        43\n",
       "Name: diagnosis, dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata['diagnosis'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE:** How many males and females are in the study? How many are in each diagnosis group?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "M    155\n",
       "F    117\n",
       "Name: gender, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata['gender'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "diagnosis  gender\n",
       "ADHD       F         22\n",
       "           M         21\n",
       "BIPOLAR    F         21\n",
       "           M         28\n",
       "CONTROL    F         62\n",
       "           M         68\n",
       "SCHZ       F         12\n",
       "           M         38\n",
       "dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata.groupby(['diagnosis', 'gender']).size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When looking at the participant dataframe, we noticed that there is a column called `ghost_NoGhost`. We should look at the README file that comes with the dataset to find out more about this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## UCLA Consortium for Neuropsychiatric Phenomics LA5c Study\r\n",
      "\r\n",
      "Preprocessed data described in\r\n",
      "\r\n",
      "\r\n",
      "Gorgolewski KJ, Durnez J and Poldrack RA. Preprocessed Consortium for Neuropsychiatric Phenomics dataset. F1000Research 2017, 6:1262\r\n",
      "https://doi.org/10.12688/f1000research.11964.2\r\n",
      "\r\n",
      "\r\n",
      "are available at https://legacy.openfmri.org/dataset/ds000030/ and via Amazon Web Services S3 protocol at: s3://openneuro/ds000030/ds000030_R1.0.5/uncompressed/derivatives/\r\n",
      "\r\n",
      "## Subjects / Participants\r\n",
      "The participants.tsv file contains subject IDs with demographic informations as well as an inventory of the scans that are included for each subject.\r\n",
      "\r\n",
      "## Dataset Derivatives (/derivatives)\r\n",
      "The /derivaties folder contains summary information that reflects the data and its contents:\r\n",
      "\r\n",
      "1. Final_Scan_Count.pdf - Plot showing the over all scan inclusion, for quick reference.\r\n",
      "2. parameter_plots/ - Folder contains many scan parameters plotted over time. Plot symbols are color coded by imaging site. Intended to provide a general sense of protocol adherence throughout the study. Individual parameters scan be found in the scan .json sidecar file. A single file containing the combined data from all of the imaging .json sidecars if provided in parameter_plots/MR_Scan_Parameters.tsv file.\r\n",
      "3. physio_plots/ - Folder contains a plot of the physiological recording trace for the Breath Hold and Resting State functional scans. For the BHT, the instructional cue timings are represented by shaded background.\r\n",
      "4. event_plots/ - Simple plots of the function task events files. The x-axis is always time (onset), and the y-axis can be task-specific. Also intended as a quick reference or summary.\r\n",
      "5. mriqcp/ - Output of the current version (as of 27 Jan 2016) of MRIQCP (MRI Quality Control Protocol: https://github.com/poldracklab/mriqc). Included are numeric results of anatomical and functional protocols as well as single subject results plotted against group distribution.\r\n",
      "6. data_browser/ - a rudimentary data visualization for MRIQP (see: http://wtriplett.github.io/ds030/)\r\n",
      "\r\n",
      "## Scan-specific Notes\r\n",
      "\r\n",
      "All scan files were converted from scanner DICOM files using dcm2niix (0c9e5c8 from https://github.com/neurolabusc/dcm2niix.git). Extra DICOM metadata elements were extracted using GDCM (http://gdcm.sourceforge.net/wiki/index.php/Main_Page) and combined to form each scan's .json sidecar.\r\n",
      "\r\n",
      "**Note regarding scan and task timing**: In most cases, the trigger time was provided in the task data file and has been transferred into the TaskParameter section of each scans *_bold.json file. If the trigger time is available, a correction was performed to the onset times to account for trigger delay. The uncompensated onset times are included in the onset_NoTriggerAdjust column. There will be an 8 second discrepancy between the compensated and uncompensated that accounts for pre-scans (4 TRs) performed by the scanner. In the cases where the trigger time is not available, the output of (TotalScanTime - nVols*RepetitionTime) may provide an estimate of pre-scan time.\r\n",
      "\r\n",
      "### T1w Anatomical\r\n",
      "Defacing was performed using freesurfer mri_deface (https://surfer.nmr.mgh.harvard.edu/fswiki/mri_deface)\r\n",
      "\r\n",
      "    Bischoff-Grethe, Amanda et al. \"A Technique for the Deidentification of Structural Brain MR Images.\" Human brain mapping 28.9 (2007): 892–903. PMC. Web. 27 Jan. 2016.\r\n",
      "\r\n",
      "### PAMenc / PAMret\r\n",
      "The larger amount of missing PAM scans is due to a task design change early in the study. It was decided that data collected before the design change would be excluded.\r\n",
      "\r\n",
      "### Stop Signal\r\n",
      "The Stop Signal task consisted of both a training task (no MRI) and the in-scanner fMRI task. The data from the training run is included in each subject's beh folder with the task name \"stopsignaltraining\".\r\n",
      "\r\n",
      "\r\n",
      "## Known Issues:\r\n",
      "Some of the T1-weighted images included within this dataset  (around 20%) show an aliasing artifact potentially generated by a headset. The artifact renders as a ghost that may overlap the cortex through one or both temporal lobes. A list of participants showing the artifact has been added to the dataset.\r\n"
     ]
    }
   ],
   "source": [
    "!cat ../data/ds000030/README"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this tutorial, we're just going to work with participants that are either CONTROL or SCHZ (`diagnosis`) and have both a T1w (`T1w == 1`) and rest (`rest == 1`) scan. Also, we'll only use data without a ghosting artifact in the T1w scan (`ghost_NoGhost == 'No_ghost'`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>EXERCISE:</b> Filter <code>participant_metadata</code> so that only the above conditions are present."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>participant_id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>bart</th>\n",
       "      <th>bht</th>\n",
       "      <th>dwi</th>\n",
       "      <th>pamenc</th>\n",
       "      <th>pamret</th>\n",
       "      <th>rest</th>\n",
       "      <th>scap</th>\n",
       "      <th>stopsignal</th>\n",
       "      <th>T1w</th>\n",
       "      <th>taskswitch</th>\n",
       "      <th>ScannerSerialNumber</th>\n",
       "      <th>ghost_NoGhost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>sub-10159</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>30</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>sub-10171</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>24</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>sub-10189</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>49</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sub-10206</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>21</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>sub-10217</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>33</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>sub-10225</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>35</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>sub-10227</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>31</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>sub-10228</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>40</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>sub-10235</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>22</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>sub-10249</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>28</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>sub-10269</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>38</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>sub-10271</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>41</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>sub-10273</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>30</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>sub-10274</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>43</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>sub-10280</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>27</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>sub-10290</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>48</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>sub-10292</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>49</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>sub-10304</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>23</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>sub-10321</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>34</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>sub-10325</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>29</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>sub-10329</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>22</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>sub-10339</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>45</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>sub-10340</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>32</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>sub-10345</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>25</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>sub-10347</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>25</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>sub-10356</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>43</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>sub-10361</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>42</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>sub-10365</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>24</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>sub-10376</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>42</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>sub-10377</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>49</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>sub-50006</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>44</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>sub-50007</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>47</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>sub-50010</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>41</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>sub-50013</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>30</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>sub-50014</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>22</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>138</th>\n",
       "      <td>sub-50015</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>23</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139</th>\n",
       "      <td>sub-50016</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>49</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>141</th>\n",
       "      <td>sub-50021</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>41</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>sub-50023</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>49</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>sub-50025</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>48</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>sub-50035</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>29</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>sub-50047</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>44</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>sub-50048</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>43</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>sub-50050</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>23</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>sub-50052</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>45</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>sub-50056</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>49</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>sub-50058</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>38</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165</th>\n",
       "      <td>sub-50059</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>39</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>sub-50061</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>48</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>sub-50064</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>31</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>sub-50066</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>33</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>sub-50067</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>25</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>sub-50069</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>23</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>sub-50073</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>47</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>sub-50075</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>33</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>174</th>\n",
       "      <td>sub-50076</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>42</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>175</th>\n",
       "      <td>sub-50077</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>29</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>sub-50080</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>29</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>sub-50081</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>32</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>178</th>\n",
       "      <td>sub-50083</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>40</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>138 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    participant_id diagnosis  age gender  bart  bht  dwi  pamenc  pamret  \\\n",
       "0        sub-10159   CONTROL   30      F   1.0  NaN  1.0     NaN     NaN   \n",
       "1        sub-10171   CONTROL   24      M   1.0  1.0  1.0     NaN     NaN   \n",
       "2        sub-10189   CONTROL   49      M   1.0  NaN  1.0     NaN     NaN   \n",
       "4        sub-10206   CONTROL   21      M   1.0  NaN  1.0     NaN     NaN   \n",
       "5        sub-10217   CONTROL   33      F   1.0  NaN  1.0     NaN     NaN   \n",
       "6        sub-10225   CONTROL   35      M   1.0  1.0  1.0     NaN     NaN   \n",
       "7        sub-10227   CONTROL   31      F   1.0  NaN  1.0     NaN     NaN   \n",
       "8        sub-10228   CONTROL   40      F   1.0  1.0  1.0     NaN     NaN   \n",
       "9        sub-10235   CONTROL   22      M   1.0  NaN  1.0     NaN     NaN   \n",
       "10       sub-10249   CONTROL   28      M   1.0  1.0  1.0     NaN     NaN   \n",
       "11       sub-10269   CONTROL   38      M   1.0  NaN  1.0     NaN     NaN   \n",
       "12       sub-10271   CONTROL   41      F   1.0  1.0  1.0     NaN     NaN   \n",
       "13       sub-10273   CONTROL   30      F   1.0  1.0  1.0     NaN     NaN   \n",
       "14       sub-10274   CONTROL   43      M   1.0  NaN  1.0     NaN     NaN   \n",
       "15       sub-10280   CONTROL   27      M   1.0  1.0  1.0     NaN     NaN   \n",
       "16       sub-10290   CONTROL   48      M   1.0  1.0  1.0     NaN     NaN   \n",
       "17       sub-10292   CONTROL   49      M   1.0  1.0  1.0     NaN     NaN   \n",
       "19       sub-10304   CONTROL   23      F   1.0  1.0  1.0     NaN     NaN   \n",
       "21       sub-10321   CONTROL   34      M   1.0  1.0  1.0     NaN     NaN   \n",
       "22       sub-10325   CONTROL   29      F   1.0  1.0  1.0     NaN     NaN   \n",
       "23       sub-10329   CONTROL   22      M   1.0  1.0  1.0     NaN     NaN   \n",
       "24       sub-10339   CONTROL   45      F   1.0  1.0  1.0     NaN     NaN   \n",
       "25       sub-10340   CONTROL   32      M   1.0  1.0  1.0     NaN     NaN   \n",
       "26       sub-10345   CONTROL   25      M   NaN  1.0  1.0     NaN     NaN   \n",
       "27       sub-10347   CONTROL   25      F   1.0  1.0  1.0     NaN     NaN   \n",
       "28       sub-10356   CONTROL   43      F   1.0  1.0  1.0     NaN     NaN   \n",
       "29       sub-10361   CONTROL   42      F   1.0  1.0  1.0     NaN     NaN   \n",
       "30       sub-10365   CONTROL   24      F   1.0  1.0  1.0     NaN     NaN   \n",
       "31       sub-10376   CONTROL   42      F   1.0  1.0  1.0     NaN     NaN   \n",
       "32       sub-10377   CONTROL   49      M   1.0  1.0  1.0     NaN     NaN   \n",
       "..             ...       ...  ...    ...   ...  ...  ...     ...     ...   \n",
       "132      sub-50006      SCHZ   44      F   1.0  1.0  1.0     NaN     NaN   \n",
       "133      sub-50007      SCHZ   47      F   1.0  1.0  1.0     NaN     NaN   \n",
       "135      sub-50010      SCHZ   41      M   1.0  1.0  1.0     NaN     NaN   \n",
       "136      sub-50013      SCHZ   30      M   1.0  1.0  1.0     1.0     1.0   \n",
       "137      sub-50014      SCHZ   22      M   1.0  1.0  1.0     1.0     1.0   \n",
       "138      sub-50015      SCHZ   23      M   1.0  1.0  1.0     1.0     1.0   \n",
       "139      sub-50016      SCHZ   49      M   1.0  1.0  1.0     1.0     1.0   \n",
       "141      sub-50021      SCHZ   41      F   1.0  1.0  1.0     1.0     1.0   \n",
       "143      sub-50023      SCHZ   49      F   1.0  1.0  1.0     1.0     1.0   \n",
       "144      sub-50025      SCHZ   48      M   1.0  1.0  1.0     1.0     1.0   \n",
       "150      sub-50035      SCHZ   29      F   1.0  1.0  1.0     1.0     1.0   \n",
       "154      sub-50047      SCHZ   44      M   1.0  1.0  1.0     1.0     1.0   \n",
       "155      sub-50048      SCHZ   43      M   1.0  1.0  1.0     1.0     1.0   \n",
       "157      sub-50050      SCHZ   23      M   1.0  1.0  1.0     1.0     1.0   \n",
       "159      sub-50052      SCHZ   45      M   1.0  1.0  1.0     1.0     1.0   \n",
       "163      sub-50056      SCHZ   49      F   1.0  1.0  1.0     1.0     1.0   \n",
       "164      sub-50058      SCHZ   38      F   1.0  1.0  1.0     1.0     1.0   \n",
       "165      sub-50059      SCHZ   39      M   1.0  1.0  1.0     1.0     1.0   \n",
       "167      sub-50061      SCHZ   48      M   1.0  1.0  1.0     1.0     1.0   \n",
       "168      sub-50064      SCHZ   31      F   1.0  1.0  1.0     1.0     1.0   \n",
       "169      sub-50066      SCHZ   33      M   1.0  1.0  1.0     1.0     1.0   \n",
       "170      sub-50067      SCHZ   25      M   1.0  1.0  1.0     1.0     1.0   \n",
       "171      sub-50069      SCHZ   23      M   1.0  1.0  1.0     1.0     1.0   \n",
       "172      sub-50073      SCHZ   47      M   1.0  1.0  NaN     1.0     1.0   \n",
       "173      sub-50075      SCHZ   33      M   1.0  1.0  1.0     1.0     1.0   \n",
       "174      sub-50076      SCHZ   42      M   1.0  1.0  1.0     1.0     1.0   \n",
       "175      sub-50077      SCHZ   29      M   1.0  1.0  1.0     1.0     1.0   \n",
       "176      sub-50080      SCHZ   29      M   1.0  1.0  1.0     1.0     1.0   \n",
       "177      sub-50081      SCHZ   32      M   1.0  1.0  1.0     1.0     1.0   \n",
       "178      sub-50083      SCHZ   40      M   1.0  1.0  1.0     1.0     1.0   \n",
       "\n",
       "     rest  scap  stopsignal  T1w  taskswitch  ScannerSerialNumber  \\\n",
       "0     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "1     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "2     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "4     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "5     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "6     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "7     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "8     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "9     1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "10    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "11    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "12    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "13    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "14    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "15    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "16    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "17    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "19    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "21    1.0   1.0         1.0  1.0         NaN              35343.0   \n",
       "22    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "23    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "24    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "25    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "26    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "27    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "28    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "29    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "30    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "31    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "32    1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "..    ...   ...         ...  ...         ...                  ...   \n",
       "132   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "133   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "135   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "136   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "137   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "138   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "139   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "141   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "143   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "144   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "150   1.0   1.0         1.0  1.0         1.0              35343.0   \n",
       "154   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "155   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "157   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "159   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "163   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "164   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "165   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "167   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "168   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "169   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "170   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "171   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "172   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "173   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "174   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "175   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "176   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "177   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "178   1.0   1.0         1.0  1.0         1.0              35426.0   \n",
       "\n",
       "    ghost_NoGhost  \n",
       "0        No_ghost  \n",
       "1        No_ghost  \n",
       "2        No_ghost  \n",
       "4        No_ghost  \n",
       "5        No_ghost  \n",
       "6        No_ghost  \n",
       "7        No_ghost  \n",
       "8        No_ghost  \n",
       "9        No_ghost  \n",
       "10       No_ghost  \n",
       "11       No_ghost  \n",
       "12       No_ghost  \n",
       "13       No_ghost  \n",
       "14       No_ghost  \n",
       "15       No_ghost  \n",
       "16       No_ghost  \n",
       "17       No_ghost  \n",
       "19       No_ghost  \n",
       "21       No_ghost  \n",
       "22       No_ghost  \n",
       "23       No_ghost  \n",
       "24       No_ghost  \n",
       "25       No_ghost  \n",
       "26       No_ghost  \n",
       "27       No_ghost  \n",
       "28       No_ghost  \n",
       "29       No_ghost  \n",
       "30       No_ghost  \n",
       "31       No_ghost  \n",
       "32       No_ghost  \n",
       "..            ...  \n",
       "132      No_ghost  \n",
       "133      No_ghost  \n",
       "135      No_ghost  \n",
       "136      No_ghost  \n",
       "137      No_ghost  \n",
       "138      No_ghost  \n",
       "139      No_ghost  \n",
       "141      No_ghost  \n",
       "143      No_ghost  \n",
       "144      No_ghost  \n",
       "150      No_ghost  \n",
       "154      No_ghost  \n",
       "155      No_ghost  \n",
       "157      No_ghost  \n",
       "159      No_ghost  \n",
       "163      No_ghost  \n",
       "164      No_ghost  \n",
       "165      No_ghost  \n",
       "167      No_ghost  \n",
       "168      No_ghost  \n",
       "169      No_ghost  \n",
       "170      No_ghost  \n",
       "171      No_ghost  \n",
       "172      No_ghost  \n",
       "173      No_ghost  \n",
       "174      No_ghost  \n",
       "175      No_ghost  \n",
       "176      No_ghost  \n",
       "177      No_ghost  \n",
       "178      No_ghost  \n",
       "\n",
       "[138 rows x 16 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_metadata = participant_metadata[(participant_metadata.diagnosis.isin(['CONTROL', 'SCHZ'])) & \n",
    "                                            (participant_metadata.T1w == 1) & \n",
    "                                            (participant_metadata.rest == 1) & \n",
    "                                            (participant_metadata.ghost_NoGhost == 'No_ghost')]\n",
    "participant_metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To ease the analysis and quicken the amount of time required to download the data, we're just going to use scans from 10 randomly sampled CONTROL and 10 SCHZ participants."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>participant_id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>bart</th>\n",
       "      <th>bht</th>\n",
       "      <th>dwi</th>\n",
       "      <th>pamenc</th>\n",
       "      <th>pamret</th>\n",
       "      <th>rest</th>\n",
       "      <th>scap</th>\n",
       "      <th>stopsignal</th>\n",
       "      <th>T1w</th>\n",
       "      <th>taskswitch</th>\n",
       "      <th>ScannerSerialNumber</th>\n",
       "      <th>ghost_NoGhost</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>diagnosis</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"10\" valign=\"top\">CONTROL</th>\n",
       "      <th>25</th>\n",
       "      <td>sub-10340</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>32</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>sub-10228</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>40</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125</th>\n",
       "      <td>sub-11131</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>38</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>sub-10678</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>24</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126</th>\n",
       "      <td>sub-11142</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>24</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>sub-10963</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>45</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>sub-10527</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>24</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>sub-10304</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>23</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>sub-10376</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>42</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>sub-10871</td>\n",
       "      <td>CONTROL</td>\n",
       "      <td>41</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"10\" valign=\"top\">SCHZ</th>\n",
       "      <th>159</th>\n",
       "      <td>sub-50052</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>45</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>141</th>\n",
       "      <td>sub-50021</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>41</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>sub-50013</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>30</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>sub-50056</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>49</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>175</th>\n",
       "      <td>sub-50077</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>29</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>sub-50047</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>44</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>sub-50069</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>23</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>sub-50081</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>32</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>sub-50080</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>29</td>\n",
       "      <td>M</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35426.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>sub-50006</td>\n",
       "      <td>SCHZ</td>\n",
       "      <td>44</td>\n",
       "      <td>F</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>35343.0</td>\n",
       "      <td>No_ghost</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              participant_id diagnosis  age gender  bart  bht  dwi  pamenc  \\\n",
       "diagnosis                                                                    \n",
       "CONTROL   25       sub-10340   CONTROL   32      M   1.0  1.0  1.0     NaN   \n",
       "          8        sub-10228   CONTROL   40      F   1.0  1.0  1.0     NaN   \n",
       "          125      sub-11131   CONTROL   38      F   1.0  1.0  1.0     1.0   \n",
       "          64       sub-10678   CONTROL   24      F   1.0  1.0  1.0     1.0   \n",
       "          126      sub-11142   CONTROL   24      M   1.0  1.0  1.0     1.0   \n",
       "          93       sub-10963   CONTROL   45      F   1.0  1.0  1.0     1.0   \n",
       "          51       sub-10527   CONTROL   24      M   1.0  1.0  1.0     1.0   \n",
       "          19       sub-10304   CONTROL   23      F   1.0  1.0  1.0     NaN   \n",
       "          31       sub-10376   CONTROL   42      F   1.0  1.0  1.0     NaN   \n",
       "          82       sub-10871   CONTROL   41      M   1.0  1.0  1.0     1.0   \n",
       "SCHZ      159      sub-50052      SCHZ   45      M   1.0  1.0  1.0     1.0   \n",
       "          141      sub-50021      SCHZ   41      F   1.0  1.0  1.0     1.0   \n",
       "          136      sub-50013      SCHZ   30      M   1.0  1.0  1.0     1.0   \n",
       "          163      sub-50056      SCHZ   49      F   1.0  1.0  1.0     1.0   \n",
       "          175      sub-50077      SCHZ   29      M   1.0  1.0  1.0     1.0   \n",
       "          154      sub-50047      SCHZ   44      M   1.0  1.0  1.0     1.0   \n",
       "          171      sub-50069      SCHZ   23      M   1.0  1.0  1.0     1.0   \n",
       "          177      sub-50081      SCHZ   32      M   1.0  1.0  1.0     1.0   \n",
       "          176      sub-50080      SCHZ   29      M   1.0  1.0  1.0     1.0   \n",
       "          132      sub-50006      SCHZ   44      F   1.0  1.0  1.0     NaN   \n",
       "\n",
       "               pamret  rest  scap  stopsignal  T1w  taskswitch  \\\n",
       "diagnosis                                                        \n",
       "CONTROL   25      NaN   1.0   1.0         1.0  1.0         1.0   \n",
       "          8       NaN   1.0   1.0         1.0  1.0         1.0   \n",
       "          125     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          64      1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          126     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          93      1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          51      1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          19      NaN   1.0   1.0         1.0  1.0         1.0   \n",
       "          31      NaN   1.0   1.0         1.0  1.0         1.0   \n",
       "          82      1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "SCHZ      159     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          141     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          136     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          163     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          175     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          154     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          171     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          177     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          176     1.0   1.0   1.0         1.0  1.0         1.0   \n",
       "          132     NaN   1.0   1.0         1.0  1.0         1.0   \n",
       "\n",
       "               ScannerSerialNumber ghost_NoGhost  \n",
       "diagnosis                                         \n",
       "CONTROL   25               35343.0      No_ghost  \n",
       "          8                35343.0      No_ghost  \n",
       "          125              35426.0      No_ghost  \n",
       "          64               35343.0      No_ghost  \n",
       "          126              35426.0      No_ghost  \n",
       "          93               35343.0      No_ghost  \n",
       "          51               35343.0      No_ghost  \n",
       "          19               35343.0      No_ghost  \n",
       "          31               35343.0      No_ghost  \n",
       "          82               35343.0      No_ghost  \n",
       "SCHZ      159              35426.0      No_ghost  \n",
       "          141              35343.0      No_ghost  \n",
       "          136              35343.0      No_ghost  \n",
       "          163              35426.0      No_ghost  \n",
       "          175              35426.0      No_ghost  \n",
       "          154              35426.0      No_ghost  \n",
       "          171              35426.0      No_ghost  \n",
       "          177              35426.0      No_ghost  \n",
       "          176              35426.0      No_ghost  \n",
       "          132              35343.0      No_ghost  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diagnosis_groups = participant_metadata.groupby('diagnosis')\n",
    "filtered_participant_metadata = diagnosis_groups.apply(lambda x: x.sample(n = 10))\n",
    "filtered_participant_metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['sub-10340',\n",
       " 'sub-10228',\n",
       " 'sub-11131',\n",
       " 'sub-10678',\n",
       " 'sub-11142',\n",
       " 'sub-10963',\n",
       " 'sub-10527',\n",
       " 'sub-10304',\n",
       " 'sub-10376',\n",
       " 'sub-10871',\n",
       " 'sub-50052',\n",
       " 'sub-50021',\n",
       " 'sub-50013',\n",
       " 'sub-50056',\n",
       " 'sub-50077',\n",
       " 'sub-50047',\n",
       " 'sub-50069',\n",
       " 'sub-50081',\n",
       " 'sub-50080',\n",
       " 'sub-50006']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "participant_list = filtered_participant_metadata.participant_id.tolist()\n",
    "participant_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Downloading Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've already randomly sampled 10 CONTROL and 10 SCHZ participants and placed the participant list in the `../download_list` text file. Let's download that data now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# # download T1w scans\n",
    "# !cat ../download_list | \\\n",
    "#   xargs -I '{}' aws s3 sync --no-sign-request \\\n",
    "#   s3://openneuro/ds000030/ds000030_R1.0.5/uncompressed/{}/anat \\\n",
    "#   ../data/ds000030/{}/anat\n",
    "\n",
    "# # download resting state fMRI scans\n",
    "# !cat ../download_list | \\\n",
    "#   xargs -I '{}' aws s3 sync --no-sign-request \\\n",
    "#   s3://openneuro/ds000030/ds000030_R1.0.5/uncompressed/{}/func \\\n",
    "#   ../data/ds000030/{}/func \\\n",
    "#   --exclude '*' \\\n",
    "#   --include '*task-rest_bold*'\n",
    "\n",
    "# # download fmriprep preprocessed anat data\n",
    "# !cat ../download_list | \\\n",
    "#   xargs -I '{}' aws s3 sync --no-sign-request \\\n",
    "#   s3://openneuro/ds000030/ds000030_R1.0.5/uncompressed/derivatives/fmriprep/{}/anat \\\n",
    "#   ../data/ds000030/derivatives/fmriprep/{}/anat\n",
    "\n",
    "# # download fmriprep preprocessed func data\n",
    "# !cat ../download_list | \\\n",
    "#   xargs -I '{}' aws s3 sync --no-sign-request \\\n",
    "#   s3://openneuro/ds000030/ds000030_R1.0.5/uncompressed/derivatives/fmriprep/{}/func \\\n",
    "#   ../data/ds000030/derivatives/fmriprep/{}/func \\\n",
    "#   --exclude '*' \\\n",
    "#   --include '*task-rest_bold*'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Querying a BIDS Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[pybids](https://bids-standard.github.io/pybids/) is a Python API for querying, summarizing and manipulating the BIDS folder structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bids.layout import BIDSLayout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout = BIDSLayout('../data/ds000030')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pybids layout object lets you query your BIDS dataset according to a number of parameters by using a `get_*()` method.  \n",
    "We can get a list of the subjects we've downloaded from the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['10171',\n",
       " '10292',\n",
       " '10365',\n",
       " '10438',\n",
       " '10565',\n",
       " '10788',\n",
       " '11106',\n",
       " '11108',\n",
       " '11122',\n",
       " '11131',\n",
       " '50010',\n",
       " '50035',\n",
       " '50047',\n",
       " '50048',\n",
       " '50052',\n",
       " '50067',\n",
       " '50075',\n",
       " '50077',\n",
       " '50081',\n",
       " '50083']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.get_subjects()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get a list of all of the files, just use `get()`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/CHANGES'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/dataset_description.json'>,\n",
       " <BIDSDataFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/participants.tsv'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/README'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10171/anat/sub-10171_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10171/anat/sub-10171_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10171/func/sub-10171_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10171/func/sub-10171_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10292/anat/sub-10292_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10292/anat/sub-10292_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10292/func/sub-10292_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10292/func/sub-10292_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10365/anat/sub-10365_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10365/anat/sub-10365_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10365/func/sub-10365_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10365/func/sub-10365_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10438/anat/sub-10438_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10438/anat/sub-10438_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10438/func/sub-10438_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10438/func/sub-10438_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10565/anat/sub-10565_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10565/anat/sub-10565_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10565/func/sub-10565_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10565/func/sub-10565_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10788/anat/sub-10788_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10788/anat/sub-10788_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10788/func/sub-10788_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10788/func/sub-10788_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11106/anat/sub-11106_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11106/anat/sub-11106_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11106/func/sub-11106_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11106/func/sub-11106_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11108/anat/sub-11108_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11108/anat/sub-11108_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11108/func/sub-11108_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11108/func/sub-11108_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11122/anat/sub-11122_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11122/anat/sub-11122_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11122/func/sub-11122_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11122/func/sub-11122_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11131/anat/sub-11131_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11131/anat/sub-11131_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11131/func/sub-11131_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11131/func/sub-11131_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50010/anat/sub-50010_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50010/anat/sub-50010_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50010/func/sub-50010_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50010/func/sub-50010_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50035/anat/sub-50035_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50035/anat/sub-50035_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50035/func/sub-50035_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50035/func/sub-50035_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50047/anat/sub-50047_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50047/anat/sub-50047_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50047/func/sub-50047_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50047/func/sub-50047_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50048/anat/sub-50048_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50048/anat/sub-50048_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50048/func/sub-50048_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50048/func/sub-50048_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50052/anat/sub-50052_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50052/anat/sub-50052_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50052/func/sub-50052_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50052/func/sub-50052_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50067/anat/sub-50067_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50067/anat/sub-50067_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50067/func/sub-50067_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50067/func/sub-50067_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50075/anat/sub-50075_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50075/anat/sub-50075_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50075/func/sub-50075_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50075/func/sub-50075_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50077/anat/sub-50077_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50077/anat/sub-50077_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50077/func/sub-50077_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50077/func/sub-50077_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50081/anat/sub-50081_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50081/anat/sub-50081_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50081/func/sub-50081_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50081/func/sub-50081_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50083/anat/sub-50083_T1w.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50083/anat/sub-50083_T1w.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50083/func/sub-50083_task-rest_bold.json'>,\n",
       " <BIDSImageFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50083/func/sub-50083_task-rest_bold.nii.gz'>,\n",
       " <BIDSFile filename='/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/task-rest_bold.json'>]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.get()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many arguments we can use to filter down this list. Any BIDS-defined keyword can be passed on as a constraint. In `pybids`, these keywords are known as **entities**. For a complete list of possibilities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'subject': <bids.layout.models.Entity at 0x7fc31104cfd0>,\n",
       " 'session': <bids.layout.models.Entity at 0x7fc3110ccdd8>,\n",
       " 'task': <bids.layout.models.Entity at 0x7fc31104cf98>,\n",
       " 'acquisition': <bids.layout.models.Entity at 0x7fc31104cc88>,\n",
       " 'ce': <bids.layout.models.Entity at 0x7fc31104ceb8>,\n",
       " 'reconstruction': <bids.layout.models.Entity at 0x7fc310fe33c8>,\n",
       " 'dir': <bids.layout.models.Entity at 0x7fc310fe3eb8>,\n",
       " 'run': <bids.layout.models.Entity at 0x7fc310fe3358>,\n",
       " 'proc': <bids.layout.models.Entity at 0x7fc310fe3ac8>,\n",
       " 'modality': <bids.layout.models.Entity at 0x7fc311005898>,\n",
       " 'echo': <bids.layout.models.Entity at 0x7fc311005ba8>,\n",
       " 'recording': <bids.layout.models.Entity at 0x7fc311005e10>,\n",
       " 'suffix': <bids.layout.models.Entity at 0x7fc311005cf8>,\n",
       " 'scans': <bids.layout.models.Entity at 0x7fc2b82b49e8>,\n",
       " 'fmap': <bids.layout.models.Entity at 0x7fc3110056d8>,\n",
       " 'datatype': <bids.layout.models.Entity at 0x7fc3110053c8>,\n",
       " 'extension': <bids.layout.models.Entity at 0x7fc311005630>,\n",
       " 'ImageType': <bids.layout.models.Entity at 0x7fc310e70550>,\n",
       " 'ManufacturerModelName': <bids.layout.models.Entity at 0x7fc310e700f0>,\n",
       " 'ScanningSequence': <bids.layout.models.Entity at 0x7fc310e70080>,\n",
       " 'SequenceVariant': <bids.layout.models.Entity at 0x7fc310e701d0>,\n",
       " 'ScanOptions': <bids.layout.models.Entity at 0x7fc310e703c8>,\n",
       " 'MRAcquisitionType': <bids.layout.models.Entity at 0x7fc310e700b8>,\n",
       " 'SequenceName': <bids.layout.models.Entity at 0x7fc310e90b38>,\n",
       " 'RepetitionTime': <bids.layout.models.Entity at 0x7fc310e90668>,\n",
       " 'EchoTime': <bids.layout.models.Entity at 0x7fc310e907b8>,\n",
       " 'InversionTime': <bids.layout.models.Entity at 0x7fc310e902b0>,\n",
       " 'NumberOfAverages': <bids.layout.models.Entity at 0x7fc310e90278>,\n",
       " 'ImagingFrequency': <bids.layout.models.Entity at 0x7fc310e90828>,\n",
       " 'MagneticFieldStrength': <bids.layout.models.Entity at 0x7fc310f43198>,\n",
       " 'NumberOfPhaseEncodingSteps': <bids.layout.models.Entity at 0x7fc310f43cf8>,\n",
       " 'EchoTrainLength': <bids.layout.models.Entity at 0x7fc310f43f28>,\n",
       " 'PercentSampling': <bids.layout.models.Entity at 0x7fc310f43128>,\n",
       " 'PercentPhaseFieldOfView': <bids.layout.models.Entity at 0x7fc310f43908>,\n",
       " 'PixelBandwidth': <bids.layout.models.Entity at 0x7fc310f437f0>,\n",
       " 'AccelerationFactorPE': <bids.layout.models.Entity at 0x7fc310f43ac8>,\n",
       " 'AccelNumReferenceLines': <bids.layout.models.Entity at 0x7fc310f433c8>,\n",
       " 'TotalScanTimeSec': <bids.layout.models.Entity at 0x7fc310f435f8>,\n",
       " 'ReceiveCoilName': <bids.layout.models.Entity at 0x7fc310f43898>,\n",
       " 'DeviceSerialNumber': <bids.layout.models.Entity at 0x7fc310f43a58>,\n",
       " 'SoftwareVersions': <bids.layout.models.Entity at 0x7fc310f43b70>,\n",
       " 'ProtocolName': <bids.layout.models.Entity at 0x7fc310f43cc0>,\n",
       " 'TransmitCoilName': <bids.layout.models.Entity at 0x7fc310f43080>,\n",
       " 'AcquisitionMatrix': <bids.layout.models.Entity at 0x7fc310f432b0>,\n",
       " 'InPlanePhaseEncodingDirection': <bids.layout.models.Entity at 0x7fc310f439e8>,\n",
       " 'FlipAngle': <bids.layout.models.Entity at 0x7fc310f43208>,\n",
       " 'VariableFlipAngleFlag': <bids.layout.models.Entity at 0x7fc310f43a20>,\n",
       " 'PatientPosition': <bids.layout.models.Entity at 0x7fc310f43978>,\n",
       " 'PhaseEncodingDirection': <bids.layout.models.Entity at 0x7fc310f434a8>,\n",
       " 'TaskName': <bids.layout.models.Entity at 0x7fc310f43400>,\n",
       " 'TaskFullName': <bids.layout.models.Entity at 0x7fc310f43320>,\n",
       " 'Instructions': <bids.layout.models.Entity at 0x7fc310f438d0>,\n",
       " 'TaskDescription': <bids.layout.models.Entity at 0x7fc310f43748>,\n",
       " 'CogAtlasID': <bids.layout.models.Entity at 0x7fc310f43f60>,\n",
       " 'CogPOID': <bids.layout.models.Entity at 0x7fc310f43780>,\n",
       " 'EPIFactor': <bids.layout.models.Entity at 0x7fc310f36128>,\n",
       " 'EffectiveEchoSpacing': <bids.layout.models.Entity at 0x7fc310f36da0>,\n",
       " 'SliceTiming': <bids.layout.models.Entity at 0x7fc310f36b70>}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.entities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, if we only want the file paths of all of our resting state fMRI scans,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/scratch/mjoseph/.pyenv/versions/3.7.3/envs/fmri_venv/lib/python3.7/site-packages/bids/layout/layout.py:659: UserWarning: In pybids 0.9.0, the 'extensions' filter was deprecated in favor of 'extension'. The former will stop working in 0.11.0.\n",
      "  warnings.warn(\"In pybids 0.9.0, the 'extensions' filter was \"\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10171/func/sub-10171_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10292/func/sub-10292_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10365/func/sub-10365_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10438/func/sub-10438_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10565/func/sub-10565_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10788/func/sub-10788_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11106/func/sub-11106_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11108/func/sub-11108_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11122/func/sub-11122_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-11131/func/sub-11131_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50010/func/sub-50010_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50035/func/sub-50035_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50047/func/sub-50047_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50048/func/sub-50048_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50052/func/sub-50052_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50067/func/sub-50067_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50075/func/sub-50075_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50077/func/sub-50077_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50081/func/sub-50081_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50083/func/sub-50083_task-rest_bold.nii.gz']"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.get(datatype='func', suffix='bold', task='rest', extensions=['.nii.gz'], return_type='file')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE**: Retrieve the file paths of any scan where the subject is '10292' or '50081' and the `RepetitionTime` is 2 seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-10292/func/sub-10292_task-rest_bold.nii.gz',\n",
       " '/mnt/tigrlab/projects/mjoseph/tutorials/func/scwg2018_python_neuroimaging/data/ds000030/sub-50081/func/sub-50081_task-rest_bold.nii.gz']"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.get(subject=['10292', '50081'], RepetitionTime=2, return_type='file')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's save the first file from our list of file paths to a variable and pull the metadata from its associated JSON file using the `get_metadata()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'AccelNumReferenceLines': 24,\n",
       " 'AccelerationFactorPE': 2,\n",
       " 'AcquisitionMatrix': '64/0/0/64',\n",
       " 'CogAtlasID': 'trm_4c8a834779883',\n",
       " 'CogPOID': 'COGPO_00086',\n",
       " 'DeviceSerialNumber': '35343',\n",
       " 'EPIFactor': 128,\n",
       " 'EchoTime': 0.03,\n",
       " 'EchoTrainLength': 1,\n",
       " 'EffectiveEchoSpacing': 0.000395,\n",
       " 'FlipAngle': 90,\n",
       " 'ImageType': 'ORIGINAL/PRIMARY/M/ND/MOSAIC',\n",
       " 'ImagingFrequency': 123249635,\n",
       " 'InPlanePhaseEncodingDirection': 'COL',\n",
       " 'Instructions': \"For the following part, you do not need to do anything other than keep your eyes open and remain relaxed. It is still important that you remain still and do not fall asleep, but you don't need to respond to anything. Any questions?\",\n",
       " 'MRAcquisitionType': '2D',\n",
       " 'MagneticFieldStrength': 3,\n",
       " 'ManufacturerModelName': 'TrioTim',\n",
       " 'NumberOfAverages': 1,\n",
       " 'NumberOfPhaseEncodingSteps': 63,\n",
       " 'PatientPosition': 'HFS',\n",
       " 'PercentPhaseFieldOfView': 100,\n",
       " 'PercentSampling': 100,\n",
       " 'PhaseEncodingDirection': 'j-',\n",
       " 'PixelBandwidth': 1420,\n",
       " 'ProtocolName': 'BOLD - RESTING',\n",
       " 'ReceiveCoilName': 'HeadMatrix',\n",
       " 'RepetitionTime': 2,\n",
       " 'ScanOptions': 'FS',\n",
       " 'ScanningSequence': 'EP',\n",
       " 'SequenceName': '*epfid2d1_64',\n",
       " 'SequenceVariant': 'SK',\n",
       " 'SliceTiming': [1.0025,\n",
       "  0,\n",
       "  1.0625,\n",
       "  0.06,\n",
       "  1.1225,\n",
       "  0.1175,\n",
       "  1.18,\n",
       "  0.1775,\n",
       "  1.24,\n",
       "  0.235,\n",
       "  1.3,\n",
       "  0.295,\n",
       "  1.3575,\n",
       "  0.355,\n",
       "  1.4175,\n",
       "  0.4125,\n",
       "  1.475,\n",
       "  0.4725,\n",
       "  1.535,\n",
       "  0.5325,\n",
       "  1.595,\n",
       "  0.59,\n",
       "  1.6525,\n",
       "  0.65,\n",
       "  1.7125,\n",
       "  0.7075,\n",
       "  1.7725,\n",
       "  0.7675,\n",
       "  1.83,\n",
       "  0.8275,\n",
       "  1.89,\n",
       "  0.885,\n",
       "  1.9475,\n",
       "  0.945],\n",
       " 'SoftwareVersions': 'syngo MR B15',\n",
       " 'TaskDescription': 'In the Resting scan, participants were asked to remain relaxed and keep their eyes open for five minutes. They were not presented any stimuli or asked to respond during the scan.',\n",
       " 'TaskFullName': 'Resting State',\n",
       " 'TaskName': 'rest',\n",
       " 'TotalScanTimeSec': 312,\n",
       " 'TransmitCoilName': 'Body',\n",
       " 'VariableFlipAngleFlag': 'N'}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fmri_file = layout.get(subject=['10292', '50081'], RepetitionTime=2, return_type='file')[0]\n",
    "layout.get_metadata(fmri_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can even collect the metadata for all of our fmri scans into a list and convert this into a dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AccelNumReferenceLines</th>\n",
       "      <th>AccelerationFactorPE</th>\n",
       "      <th>AcquisitionMatrix</th>\n",
       "      <th>CogAtlasID</th>\n",
       "      <th>CogPOID</th>\n",
       "      <th>DeviceSerialNumber</th>\n",
       "      <th>EPIFactor</th>\n",
       "      <th>EchoTime</th>\n",
       "      <th>EchoTrainLength</th>\n",
       "      <th>EffectiveEchoSpacing</th>\n",
       "      <th>...</th>\n",
       "      <th>SequenceName</th>\n",
       "      <th>SequenceVariant</th>\n",
       "      <th>SliceTiming</th>\n",
       "      <th>SoftwareVersions</th>\n",
       "      <th>TaskDescription</th>\n",
       "      <th>TaskFullName</th>\n",
       "      <th>TaskName</th>\n",
       "      <th>TotalScanTimeSec</th>\n",
       "      <th>TransmitCoilName</th>\n",
       "      <th>VariableFlipAngleFlag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.065, 0.06, 1.1225, 0.12, 1.1825, ...</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.18...</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.182...</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.065, 0.06, 1.1225, 0.12, 1.1825, ...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.0625, 0.06, 1.1225, 0.12, 1.1825,...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.18...</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35343</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....</td>\n",
       "      <td>syngo MR B15</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.0625, 0.06, 1.1225, 0.12, 1.1825,...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.18,...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.065, 0.06, 1.1225, 0.12, 1.1825, ...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.0625, 0.06, 1.1225, 0.12, 1.1825,...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.005, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.182...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>64/0/0/64</td>\n",
       "      <td>trm_4c8a834779883</td>\n",
       "      <td>COGPO_00086</td>\n",
       "      <td>35426</td>\n",
       "      <td>128</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000395</td>\n",
       "      <td>...</td>\n",
       "      <td>*epfid2d1_64</td>\n",
       "      <td>SK</td>\n",
       "      <td>[1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...</td>\n",
       "      <td>syngo MR B17</td>\n",
       "      <td>In the Resting scan, participants were asked t...</td>\n",
       "      <td>Resting State</td>\n",
       "      <td>rest</td>\n",
       "      <td>312</td>\n",
       "      <td>Body</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20 rows × 40 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    AccelNumReferenceLines  AccelerationFactorPE AcquisitionMatrix  \\\n",
       "0                       24                     2         64/0/0/64   \n",
       "1                       24                     2         64/0/0/64   \n",
       "2                       24                     2         64/0/0/64   \n",
       "3                       24                     2         64/0/0/64   \n",
       "4                       24                     2         64/0/0/64   \n",
       "5                       24                     2         64/0/0/64   \n",
       "6                       24                     2         64/0/0/64   \n",
       "7                       24                     2         64/0/0/64   \n",
       "8                       24                     2         64/0/0/64   \n",
       "9                       24                     2         64/0/0/64   \n",
       "10                      24                     2         64/0/0/64   \n",
       "11                      24                     2         64/0/0/64   \n",
       "12                      24                     2         64/0/0/64   \n",
       "13                      24                     2         64/0/0/64   \n",
       "14                      24                     2         64/0/0/64   \n",
       "15                      24                     2         64/0/0/64   \n",
       "16                      24                     2         64/0/0/64   \n",
       "17                      24                     2         64/0/0/64   \n",
       "18                      24                     2         64/0/0/64   \n",
       "19                      24                     2         64/0/0/64   \n",
       "\n",
       "           CogAtlasID      CogPOID DeviceSerialNumber  EPIFactor  EchoTime  \\\n",
       "0   trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "1   trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "2   trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "3   trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "4   trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "5   trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "6   trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "7   trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "8   trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "9   trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "10  trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "11  trm_4c8a834779883  COGPO_00086              35343        128      0.03   \n",
       "12  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "13  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "14  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "15  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "16  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "17  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "18  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "19  trm_4c8a834779883  COGPO_00086              35426        128      0.03   \n",
       "\n",
       "    EchoTrainLength  EffectiveEchoSpacing  ...  SequenceName SequenceVariant  \\\n",
       "0                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "1                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "2                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "3                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "4                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "5                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "6                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "7                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "8                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "9                 1              0.000395  ...  *epfid2d1_64              SK   \n",
       "10                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "11                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "12                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "13                1              0.000395  ...   epfid2d1_64              SK   \n",
       "14                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "15                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "16                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "17                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "18                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "19                1              0.000395  ...  *epfid2d1_64              SK   \n",
       "\n",
       "                                          SliceTiming SoftwareVersions  \\\n",
       "0   [1.005, 0, 1.065, 0.06, 1.1225, 0.12, 1.1825, ...     syngo MR B15   \n",
       "1   [1.0025, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.18...     syngo MR B15   \n",
       "2   [1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....     syngo MR B15   \n",
       "3   [1.005, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.182...     syngo MR B15   \n",
       "4   [1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...     syngo MR B15   \n",
       "5   [1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...     syngo MR B15   \n",
       "6   [1.005, 0, 1.065, 0.06, 1.1225, 0.12, 1.1825, ...     syngo MR B17   \n",
       "7   [1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...     syngo MR B17   \n",
       "8   [1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....     syngo MR B17   \n",
       "9   [1.005, 0, 1.0625, 0.06, 1.1225, 0.12, 1.1825,...     syngo MR B17   \n",
       "10  [1.0025, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.18...     syngo MR B15   \n",
       "11  [1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....     syngo MR B15   \n",
       "12  [1.005, 0, 1.0625, 0.06, 1.1225, 0.12, 1.1825,...     syngo MR B17   \n",
       "13  [1.005, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.18,...     syngo MR B17   \n",
       "14  [1.005, 0, 1.065, 0.06, 1.1225, 0.12, 1.1825, ...     syngo MR B17   \n",
       "15  [1.0025, 0, 1.0625, 0.0575, 1.1225, 0.1175, 1....     syngo MR B17   \n",
       "16  [1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...     syngo MR B17   \n",
       "17  [1.005, 0, 1.0625, 0.06, 1.1225, 0.12, 1.1825,...     syngo MR B17   \n",
       "18  [1.005, 0, 1.0625, 0.06, 1.1225, 0.1175, 1.182...     syngo MR B17   \n",
       "19  [1.0025, 0, 1.0625, 0.0575, 1.12, 0.1175, 1.18...     syngo MR B17   \n",
       "\n",
       "                                      TaskDescription   TaskFullName  \\\n",
       "0   In the Resting scan, participants were asked t...  Resting State   \n",
       "1   In the Resting scan, participants were asked t...  Resting State   \n",
       "2   In the Resting scan, participants were asked t...  Resting State   \n",
       "3   In the Resting scan, participants were asked t...  Resting State   \n",
       "4   In the Resting scan, participants were asked t...  Resting State   \n",
       "5   In the Resting scan, participants were asked t...  Resting State   \n",
       "6   In the Resting scan, participants were asked t...  Resting State   \n",
       "7   In the Resting scan, participants were asked t...  Resting State   \n",
       "8   In the Resting scan, participants were asked t...  Resting State   \n",
       "9   In the Resting scan, participants were asked t...  Resting State   \n",
       "10  In the Resting scan, participants were asked t...  Resting State   \n",
       "11  In the Resting scan, participants were asked t...  Resting State   \n",
       "12  In the Resting scan, participants were asked t...  Resting State   \n",
       "13  In the Resting scan, participants were asked t...  Resting State   \n",
       "14  In the Resting scan, participants were asked t...  Resting State   \n",
       "15  In the Resting scan, participants were asked t...  Resting State   \n",
       "16  In the Resting scan, participants were asked t...  Resting State   \n",
       "17  In the Resting scan, participants were asked t...  Resting State   \n",
       "18  In the Resting scan, participants were asked t...  Resting State   \n",
       "19  In the Resting scan, participants were asked t...  Resting State   \n",
       "\n",
       "    TaskName TotalScanTimeSec  TransmitCoilName  VariableFlipAngleFlag  \n",
       "0       rest              312              Body                      N  \n",
       "1       rest              312              Body                      N  \n",
       "2       rest              312              Body                      N  \n",
       "3       rest              312              Body                      N  \n",
       "4       rest              312              Body                      N  \n",
       "5       rest              312              Body                      N  \n",
       "6       rest              312              Body                      N  \n",
       "7       rest              312              Body                      N  \n",
       "8       rest              312              Body                      N  \n",
       "9       rest              312              Body                      N  \n",
       "10      rest              312              Body                      N  \n",
       "11      rest              312              Body                      N  \n",
       "12      rest              312              Body                      N  \n",
       "13      rest              312              Body                      N  \n",
       "14      rest              312              Body                      N  \n",
       "15      rest              312              Body                      N  \n",
       "16      rest              312              Body                      N  \n",
       "17      rest              312              Body                      N  \n",
       "18      rest              312              Body                      N  \n",
       "19      rest              312              Body                      N  \n",
       "\n",
       "[20 rows x 40 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metadata_list = []\n",
    "all_fmri_files = layout.get(datatype='func', suffix='bold', return_type='file', extensions='.nii.gz')\n",
    "for fmri_file in all_fmri_files:\n",
    "    fmri_metadata = layout.get_metadata(fmri_file)\n",
    "    metadata_list.append(fmri_metadata)\n",
    "df = pd.DataFrame.from_records(metadata_list)\n",
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}