Stock-Exchange-PyTorch

Originally, I started this project to learn some basics of reinforcement learning and have taken a detour. The code-base has become a mess. git stashing for now.

Before you say you want reinforcement learning, please make sure you really do.

Examples

Some of the ideas used for this project are illustrated in the examples folder. Things do get interesting once you start digging into some of the log file analysis.

Original

stock-exchange-pytorch applies deep learning algorithms to financial market data in pytorch. Namely, reinforcement learning and supervised learning are applied to a portfolio.

Reinforcement Learning

• Deep Q Learning (DQN) [1], [2]
• Double DQN [3]
• Dueling network DQN (Dueling DQN) [4]
• Deep Deterministic Policy Gradient (DDPG) [5]

Supervised Learning

• Gated Recurrent Unit (GRU) approach to fit distributions [7]
• Try as few parameters as possible to actually make some viable products and it works better than some work by pros

Data

• market data from Investor's Exchange (IEX)
• Use Investor's Exchange to automate downloading and saving You can download data from IEXtrading directly. If you want something serious (by millisecond level), look here.

Requirements

By default, it assumes you have installed PyTorch-0.4+ as the name suggests. It also assumes you have a decent NVIDIA GPU with Python 3.5+, and pip install gym if necessary.

You can run the following to train a demo ddpg with the provided data.

python3 train_reinforce_ddpg.py

You can run the following to train a demo dueling DQN with the provided data.

python3 train_reinforce_dqn.py

If you want to test the result, simply run

python3 test_reinforce.py

If all went well, you might see something like this:

Supervised learning is done with GRU network, and can be found in train_supervised.py

gym_stock_exchange link

This is an environment which depends on open-ai's gym. It supports single stock or a portfolio. For action spaces , both continuous and discrete are supported. Obviously, DDPG uses continuous action-space, and DQN uses discrete settings. Code for continuous and discrete are found here.

Reminder

Things to think about before trying any models:

• Data Size
• Param Size for a model
• Are you sure reinforcement learning adds much value?

Future work

• Provide constraints in holdings for gym_stock_exchange
• Provide portfolio by default in the gym_stock_exchange
• Provide examples
• Provide more thorough support for supervised learnings
• Implement PPO and other policy gradient methods
• Implement options and other derivatives valuations