energy-py

energypy is a framework for running reinforcement learning experiments on energy environments.

energypy is built and maintained by Adam Green - [email protected].

Installation

$ git clone https://github.com/ADGEfficiency/energy-py

$ pip install --ignore-installed -r requirements.txt

$ python setup.py install

Running experiments

energy-py has a high level API to run a specific run of an experiment from a yaml config file:

$ energypy-experiment energypy/examples/example_config.yaml battery

An example config file (energypy/examples/example_config.yaml):

expt:
    name: example

battery: &defaults
    total_steps: 10000

    env:
        env_id: battery
        dataset: example

    agent:
        agent_id: random

Results (log files for each episode & experiment summaries) are placed into a folder in the users $HOME. The progress of an experiment can be watched with TensorBoard by running a server looking at this results folder:

$ tensorboard --logdir='~/energy-py-results'

Low level API

energypy provides the familiar gym style low-level API for agent and environment initialization and interactions:

import energypy

env = energypy.make_env(env_id='battery')

agent = energypy.make_agent(
    agent_id='dqn',
    env=env,
    total_steps=10000
	)

observation = env.reset()

while not done:
    action = agent.act(observation)
    next_observation, reward, done, info = env.step(action)
    training_info = agent.learn()
    observation = next_observation

Library

energy-py environments follow the design of OpenAI gym. energy-py also wraps some classic gym environments such as CartPole, Pendulum and MountainCar.

energy-py currently implements:

• naive agents
• DQN agent
• Battery storage environment
• Demand side flexibility environment
• Wrappers around the OpenAI gym CartPole, Pendulum and MountainCar environments

Further reading

• Introductory blog post
• DQN debugging
• DDQN hyperparameter tuning
• Jupyter notebook example of low level API - DQN and battery environment
• talk covering two years of lessons working on energypy