Distributed-MADDPG

Distributed Multi-Agent Cooperation Algorithm based on MADDPG with prioritized batch data.

Distributed Multi-Agent Architecture

Introduction

This work focus on Multi-Agent Cooperation Problem. We proposed a method which consists 3 components:

1. Related research - MADDPG This algorithm comes from Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
2. Prioritized Batch Data To optimize one-step update without losing diversity, we divide batch data into several parts and prioritize these batches. Using the batch data with maximal loss to do one-step update.
3. Distributed Multi-Agent Architecture Similar to A3C algorithm, we adopt this Master and Multi-Worker architecture in our work.

Experiment

Implementation

• Keras 2.1.2 （tensorflow 1.4 as backend）
• mpi4py
• Python 3.6
• CUDA 8.0 + cuDNN 6.0

Environment

• Modified original environment (you can find in my repo) from OpenAI
  • Fixed landmark
  • Border

Neural Network

Result

Learning Progress

• DDPG & MADDPG & PROPOSED

How to run this program

For program using MPI:

• mpiexec -np [worker_number] python mpi-xxx.py

mpiexec -np 4 python mpirun_main.py

For others:

python xxx.py

Future Work (4 vs 2)

Thanks to

• agakshat's MADDPG implementation repo
• OpenAI baselines
• OpenAI envs