FaceOff

Using deep learning to optimize stimuli to target brain states

Live Demo at : https://faceoff.eegedu.com/

• https://science.sciencemag.org/content/sci/364/6439/eaav9436.full.pdf
• https://www.biorxiv.org/content/10.1101/516484v1.full.pdf

Stack

• EEG

  • https://choosemuse.com/
  • https://github.com/urish/muse-js
  • https://github.com/neurosity/eeg-pipes
  • https://github.com/kylemath/eegedu

• UX

  • https://github.com/facebook/react
  • https://github.com/Shopify/polaris-react
  • https://www.npmjs.com/package/react-full-screen

• Webcam

  • https://www.npmjs.com/package/react-webcam

• Machine Learning

  • https://github.com/tensorflow/tfjs

• Deploy

  • https://github.com/firebase/

Plan and Inspiration

https://colab.research.google.com/drive/17HvdPhwhe1cmkTWnA50vyOmMlWHhg6_P?usp=sharing

https://alantian.github.io/ganshowcase/

latence space brain surfer

start as webapp

tensorflow.js? load a pretrained model, input vectors to it, output images

then we can connect with bluetoothweb and use muse-js and neurosity-pipes

plus this video at this moment Trash | Off the Air | adult swimTrash | Off the Air | adult swim

https://youtu.be/BGSUYAKquBU?t=114

Amazing... then we connect with talkgenerator.com and watch our thoughts

ok hardest part first how does EEG stack connect to tensorflow.js output vector from eeg stack input to tensorflow

pulling out the five elements alpha beta gamma delta kapa and then pipe that right in

yeah specifically the "pipe in" how does a live tensorflow compute work

we could use peoples webcams to seed it isntead you can project an image into the latent space and find nearest approximation but it takes a few seconds

Installation for Development

If you are interested in developing, here are some instructions to get the software running on your system. Note: Currently development requires a Mac OSX operating system.

To start, you will need to install Homebrew and yarn. These are easy to install with the following Terminal / bash commands:

## Install homebrew
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

## Install yarn
# NOTE: this will also install Node.js if it is not already installed.
brew install yarn 

# NOTE: Node.js must be version 10.x for Muse interaction

# Thus, if you are getting version issues, install n, with the following command:
# sudo npm install -g n

# Then, you can switch to version 10.x with the following command:
# sudo n 10.16.0

Then, in Terminal/bash, clone this Git repository and change directory into the newly cloned folder:

git clone https://github.com/kylemath/lags-bs
cd EEGEdu

Then, you can install the required yarn packages:

yarn install

Local Development Environment

Then, you can run the Local Development Environment:

yarn start dev

If it is working correctly, the application will automatically open in a browser window at http://localhost:3000.

Local Production Environment

To start the Local Production Environment, you can use the following commands:

yarn cache clean
yarn run build
serve -s build

Local Testing of Changes

1. Install any new packages yarn install
2. Start the Local Development Environment yarn start dev
3. Look for errors in terminal log
4. Open's browser to http://localhost:3000
5. Open Javascript console
6. Look for errors in console
7. Connect Mock data stream by clicking Connect button
8. step through website features while reading console

Deployment

FaceOff is running on Firebase and deployment happens automagically using GitHub post-commit hooks, or Actions, as they are commonly called. You can see how the application is build and deployed by inspecting the workflow.

Currently this automagic deployment is not working, so we can deploy to firebase manually:

First, install the Firebase deployment tools:

sudo brew install firebase
sudo yarn global add firebase-tools
sudo yarn global add firebase

The first deployment requires login and initialization once:

firebase login

Browser opens, and login to Google account authorized for Firebase deployment:

firebase init

• options: Hosting Sites only
• public directory: build
• single-page app: No
• Overwrite - No
• Overwrite - No

Then, deployment to Firebase happens with the following commands:

# clean the local cache to ensure most recent version is served
yarn cache clean

# build the latest version of the site
yarn run build

# deploy the latest version to firebase
firebase deploy