Skip to content

Commit

Permalink
Merge branch 'network-grid' of https://github.com/cauemello/mesa into…
Browse files Browse the repository at this point in the history
… network-grid
cauemello committed Jul 12, 2017

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
2 parents d56fc95 + ad75d6f commit 4196c38
Showing 10 changed files with 515 additions and 403 deletions.
42 changes: 31 additions & 11 deletions HISTORY.rst
Original file line number Diff line number Diff line change
@@ -3,28 +3,48 @@
Release History
---------------

Next release - 0.8.2 (2017-07-?) Gila Bend
+++++++++++++++++++++++++++++++++++++++++++

Next release - 0.8.1 (2017-05-?) Flagstaff (PyCon Sprints)
++++++++++++++++++
**Improvements**

* None

**Fixes**

* None


0.8.1 (2017-07-03) Flagstaff (PyCon Sprints & then some)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++

**Improvements**

* Bootstrap UI starter #383
* Add Sugarscape Constant Growback example #385
* Add best-practices document and describe models. #371
* Added function to use local images as shapes in GridDraw
* Added drawImage function to GridDraw
* Enables to draw a local imagefile as shape for agents
* Refactored & model standards related:
* Prisoner's Dilemma refactor to meet new model standard format. #377
* refactored boltzmann wealth model to new layout #376
* Update tutorial to follow new model standards #370
* Moving wolf sheep pngs to sub-folder for better organization #372
* Add best-practices document and describe models. #371
* Modified loop over agents in schedule step method #356
* Added function to use local images as shapes in GridDraw #355

**Fixes**

* Modified loop over agents in schedule step method
* If agents are deleted while looping over agents list in the model schedule,
looping becomes unpredictable and agents step methods for some agents may not be called for a given model step
* Fix math problems in flockers; use numpy in space #378
* Seed both global random number generators #373, #368
* Dictionary parameters fix #309
* Downgrade setuptools to fix #353
* Fix tutorial and example readme for port change
* Minor forest fire fix #338, #346
* Allow fixed seed for replication #107
* Fix tutorial and example readme for port change 8b57aa


0.8.0 (2017-01-29) - Edgar
++++++++++++++++++
+++++++++++++++++++++++++++

**Improvements**

@@ -40,7 +60,7 @@ Next release - 0.8.1 (2017-05-?) Flagstaff (PyCon Sprints)


0.7.8.1 (2016-11-02) Duncan
++++++++++++++++++
++++++++++++++++++++++++++++

**Improvements**

2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Copyright 2016 Core Mesa Team
Copyright 2017 Core Mesa Team

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
138 changes: 68 additions & 70 deletions docs/tutorials/intro_tutorial.ipynb

Large diffs are not rendered by default.

28 changes: 14 additions & 14 deletions docs/tutorials/intro_tutorial.rst
Original file line number Diff line number Diff line change
@@ -804,16 +804,15 @@ indefinitely in ``__init__``.
self.running = True
# ...
We instantiate a BatchRunner with a model class to run, and a dictionary
mapping parameters to values for them to take. If any of these
parameters are assigned more than one value, as a list or an iterator,
the BatchRunner will know to run all the combinations of these values
and the other ones. The BatchRunner also takes an argument for how many
model instantiations to create and run at each combination of parameter
values, and how many steps to run each instantiation for. Finally, like
the DataCollector, it takes dictionaries of model- and agent-level
reporters to collect. Unlike the DataCollector, it won't collect the
data every step of the model, but only at the end of each run.
We instantiate a BatchRunner with a model class to run, and two dictionaries:
one of the fixed parameters (mapping model arguments to values) and one of
varying parameters (mapping each parameter name to a sequence of values for it
to take). The BatchRunner also takes an argument for how many model
instantiations to create and run at each combination of parameter values, and
how many steps to run each instantiation for. Finally, like the DataCollector,
it takes dictionaries of model- and agent-level reporters to collect. Unlike
the DataCollector, it won't collect the data every step of the model, but only
at the end of each run.

In the following example, we hold the height and width fixed, and vary
the number of agents. We tell the BatchRunner to run 5 instantiations of
@@ -827,12 +826,13 @@ Now, we can set up and run the BatchRunner:
# run.py
from mesa.batchrunner import BatchRunner
parameters = {"width": 10,
"height": 10,
"N": range(10, 500, 10)}
fixed_params = {"width": 10,
"height": 10}
variable_params = {"N": range(10, 500, 10)}
batch_run = BatchRunner(MoneyModel,
parameters,
fixed_parameters=fixed_params,
variable_parameters=variable_params,
iterations=5,
max_steps=100,
model_reporters={"Gini": compute_gini})
196 changes: 110 additions & 86 deletions examples/Schelling/analysis.ipynb

Large diffs are not rendered by default.

195 changes: 82 additions & 113 deletions examples/forest_fire/Forest Fire Model.ipynb

Large diffs are not rendered by default.

155 changes: 94 additions & 61 deletions mesa/batchrunner.py
Original file line number Diff line number Diff line change
@@ -6,11 +6,33 @@
A single class to manage a batch run or parameter sweep of a given model.
"""
from itertools import product
import collections
import copy
from itertools import product, count
import pandas as pd
from tqdm import tqdm


def combinations(*items):
"""
A small fix to handle dictionary type parameters in cartesian product.
"""
prepared = [(item,) if isinstance(item, collections.Mapping) else item
for item in items]
yield from (param for param in product(*prepared))


class VariableParameterError(TypeError):
MESSAGE = ('variable_parameters must map a name to a sequence of values. '
'These parameters were given with non-sequence values: {}')

def __init__(self, bad_names):
self.bad_names = bad_names

def __str__(self):
return self.MESSAGE.format(self.bad_names)


class BatchRunner:
""" This class is instantiated with a model class, and model parameters
associated with one or more values. It is also instantiated with model and
@@ -23,19 +45,26 @@ class BatchRunner:
entire DataCollector object.
"""
def __init__(self, model_cls, parameter_values, iterations=1,
max_steps=1000, model_reporters=None, agent_reporters=None,
display_progress=True):
def __init__(self, model_cls, variable_parameters=None,
fixed_parameters=None, iterations=1, max_steps=1000,
model_reporters=None, agent_reporters=None, display_progress=True):
""" Create a new BatchRunner for a given model with the given
parameters.
Args:
model_cls: The class of model to batch-run.
parameter_values: Dictionary of parameters to their values or
ranges of values. For example:
variable_parameters: Dictionary of parameters to lists of values.
The model will be run with every combination of these paramters.
For example, given variable_parameters of
{"param_1": range(5),
"param_2": [1, 5, 10],
"const_param": 100}
"param_2": [1, 5, 10]}
models will be run with {param_1=1, param_2=1},
{param_1=2, param_2=1}, ..., {param_1=4, param_2=10}.
fixed_parameters: Dictionary of parameters that stay same through
all batch runs. For example, given fixed_parameters of
{"constant_parameter": 3},
every instantiated model will be passed constant_parameter=3
as a kwarg.
iterations: The total number of times to run the model for each
combination of parameters.
max_steps: The upper limit of steps above which each run will be halted
@@ -51,8 +80,8 @@ def __init__(self, model_cls, parameter_values, iterations=1,
"""
self.model_cls = model_cls
self.parameter_values = {param: self.make_iterable(vals)
for param, vals in parameter_values.items()}
self.variable_parameters = self._process_parameters(variable_parameters)
self.fixed_parameters = fixed_parameters or {}
self.iterations = iterations
self.max_steps = max_steps

@@ -67,36 +96,42 @@ def __init__(self, model_cls, parameter_values, iterations=1,

self.display_progress = display_progress

def _process_parameters(self, params):
params = copy.deepcopy(params)
bad_names = []
for name, values in params.items():
if (isinstance(values, str) or
not hasattr(values, "__iter__")):
bad_names.append(name)
if bad_names:
raise VariableParameterError(bad_names)
return params

def run_all(self):
""" Run the model at all parameter combinations and store results. """
params = self.parameter_values.keys()
param_ranges = self.parameter_values.values()
run_count = 0

if self.display_progress:
pbar = tqdm(total=len(list(product(*param_ranges))) * self.iterations)

for param_values in list(product(*param_ranges)):
kwargs = dict(zip(params, param_values))
for _ in range(self.iterations):
param_names, param_ranges = zip(*self.variable_parameters.items())
run_count = count()
total_iterations = self.iterations
for param_range in param_ranges:
total_iterations *= len(param_range)
with tqdm(total_iterations, disable=not self.display_progress) as pbar:
for param_values in product(*param_ranges):
kwargs = dict(zip(param_names, param_values))
kwargs.update(self.fixed_parameters)
model = self.model_cls(**kwargs)
self.run_model(model)
# Collect and store results:
if self.model_reporters:
key = tuple(list(param_values) + [run_count])
self.model_vars[key] = self.collect_model_vars(model)
if self.agent_reporters:
agent_vars = self.collect_agent_vars(model)
for agent_id, reports in agent_vars.items():
key = tuple(list(param_values) + [run_count, agent_id])
self.agent_vars[key] = reports
if self.display_progress:
pbar.update()

run_count += 1

if self.display_progress:
pbar.close()
for _ in range(self.iterations):
self.run_model(model)
# Collect and store results:
model_key = param_values + (next(run_count),)
if self.model_reporters:
self.model_vars[model_key] = self.collect_model_vars(model)
if self.agent_reporters:
agent_vars = self.collect_agent_vars(model)
for agent_id, reports in agent_vars.items():
agent_key = model_key + (agent_id,)
self.agent_vars[agent_key] = reports
pbar.update()

def run_model(self, model):
""" Run a model object to completion, or until reaching max steps.
@@ -126,38 +161,36 @@ def collect_agent_vars(self, model):
return agent_vars

def get_model_vars_dataframe(self):
""" Generate a pandas DataFrame from the model-level variables collected.
""" Generate a pandas DataFrame from the model-level variables
collected.
"""
index_col_names = list(self.parameter_values.keys())
index_col_names.append("Run")
records = []
for key, val in self.model_vars.items():
record = dict(zip(index_col_names, key))
for k, v in val.items():
record[k] = v
records.append(record)
return pd.DataFrame(records)
return self._prepare_report_table(self.model_vars)

def get_agent_vars_dataframe(self):
""" Generate a pandas DataFrame from the agent-level variables
collected.
"""
index_col_names = list(self.parameter_values.keys())
index_col_names += ["Run", "AgentID"]
return self._prepare_report_table(self.agent_vars,
extra_cols=['AgentId'])

def _prepare_report_table(self, vars_dict, extra_cols=None):
"""
Creates a dataframe from collected records and sorts it using 'Run'
column as a key.
"""
extra_cols = ['Run'] + (extra_cols or [])
index_cols = list(self.variable_parameters.keys()) + extra_cols

records = []
for key, val in self.agent_vars.items():
record = dict(zip(index_col_names, key))
for k, v in val.items():
record[k] = v
for param_key, values in vars_dict.items():
record = dict(zip(index_cols, param_key))
record.update(values)
records.append(record)
return pd.DataFrame(records)

@staticmethod
def make_iterable(val):
""" Helper method to ensure a value is a non-string iterable. """
if hasattr(val, "__iter__") and not isinstance(val, str):
return val
else:
return [val]

df = pd.DataFrame(records)
rest_cols = set(df.columns) - set(index_cols)
ordered = df[index_cols + list(sorted(rest_cols))]
ordered.sort_values(by='Run', inplace=True)
return ordered
4 changes: 2 additions & 2 deletions mesa/time.py
Original file line number Diff line number Diff line change
@@ -139,7 +139,7 @@ class StagedActivation(BaseScheduler):
shuffle_between_stages = False
stage_time = 1

def __init__(self, model, stage_list=["step"], shuffle=False,
def __init__(self, model, stage_list=None, shuffle=False,
shuffle_between_stages=False):
""" Create an empty Staged Activation schedule.
@@ -154,7 +154,7 @@ def __init__(self, model, stage_list=["step"], shuffle=False,
"""
super().__init__(model)
self.stage_list = stage_list
self.stage_list = stage_list or ["step"]
self.shuffle = shuffle
self.shuffle_between_stages = shuffle_between_stages
self.stage_time = 1 / len(self.stage_list)
156 changes: 112 additions & 44 deletions tests/test_batchrunner.py
Original file line number Diff line number Diff line change
@@ -1,46 +1,63 @@
"""
Test the BatchRunner
"""
from functools import reduce
from operator import mul
import unittest

from mesa import Agent, Model
from mesa.batchrunner import BatchRunner
from mesa.time import BaseScheduler
from mesa.batchrunner import BatchRunner


NUM_AGENTS = 7


class MockAgent(Agent):
"""
Minimalistic model for testing purposes
Minimalistic agent implementation for testing purposes
"""
def __init__(self, unique_id, val):
def __init__(self, unique_id, model, val):
super().__init__(unique_id, model)
self.unique_id = unique_id
self.val = val

def step(self):
"""
increment val by 1
"""
self.val += 1


class MockModel(Model):
"""
Minimalistic model for testing purposes
"""
def __init__(self, model_param, agent_param):
"""
Args:
model_param (any): parameter specific to the model
agent_param (int): parameter specific to the agent
"""
self.schedule = BaseScheduler(None)
self.model_param = model_param
def __init__(self, variable_model_param, variable_agent_param,
fixed_model_param=None, schedule=None, **kwargs):
super().__init__()
self.schedule = BaseScheduler(None) if schedule is None else schedule
self.variable_model_param = variable_model_param
self.variable_agent_param = variable_agent_param
self.fixed_model_param = fixed_model_param
self.n_agents = kwargs.get('n_agents', NUM_AGENTS)
self.running = True
self.init_agents()

def init_agents(self):
for i in range(self.n_agents):
self.schedule.add(MockAgent(i, self, self.variable_agent_param))

def step(self):
self.schedule.step()


class MockMixedModel(Model):

def __init__(self, **other_params):
super().__init__()
self.variable_name = other_params.get('variable_name', 42)
self.fixed_name = other_params.get('fixed_name')
self.running = True
for i in range(NUM_AGENTS):
a = MockAgent(i, agent_param)
self.schedule.add(a)
self.schedule = BaseScheduler(None)
self.schedule.add(MockAgent(1, self, 0))

def step(self):
self.schedule.step()
@@ -51,44 +68,95 @@ class TestBatchRunner(unittest.TestCase):
Test that BatchRunner is running batches
"""
def setUp(self):
"""
Create the model and run it for some steps
"""
self.model_reporter = {"model": lambda m: m.model_param}
self.agent_reporter = {
self.mock_model = MockModel
self.model_reporters = {
"reported_variable_value": lambda m: m.variable_model_param,
"reported_fixed_value": lambda m: m.fixed_model_param
}
self.agent_reporters = {
"agent_id": lambda a: a.unique_id,
"agent_val": lambda a: a.val}
self.params = {
'model_param': range(3),
'agent_param': [1, 8],
"agent_val": lambda a: a.val
}
self.variable_params = {
"variable_model_param": range(3),
"variable_agent_param": [1, 8]
}
self.fixed_params = None
self.iterations = 17
self.batch = BatchRunner(
MockModel,
self.params,
self.max_steps = 3

def launch_batch_processing(self):
batch = BatchRunner(
self.mock_model,
variable_parameters=self.variable_params,
fixed_parameters=self.fixed_params,
iterations=self.iterations,
max_steps=3,
model_reporters=self.model_reporter,
agent_reporters=self.agent_reporter)
self.batch.run_all()
max_steps=self.max_steps,
model_reporters=self.model_reporters,
agent_reporters=self.agent_reporters)
batch.run_all()
return batch

@property
def model_runs(self):
"""
Returns total number of batch runner's iterations.
"""
return (reduce(mul, map(len, self.variable_params.values())) *
self.iterations)

def test_model_level_vars(self):
"""
Test that model-level variable collection is of the correct size
"""
model_vars = self.batch.get_model_vars_dataframe()
rows = len(self.params['model_param']) * \
len(self.params['agent_param']) * \
self.iterations
assert model_vars.shape == (rows, 4)
batch = self.launch_batch_processing()
model_vars = batch.get_model_vars_dataframe()
expected_cols = (len(self.variable_params) +
len(self.model_reporters) +
1) # extra column with run index

self.assertEqual(model_vars.shape, (self.model_runs, expected_cols))

def test_agent_level_vars(self):
"""
Test that agent-level variable collection is of the correct size
"""
agent_vars = self.batch.get_agent_vars_dataframe()
rows = NUM_AGENTS * \
len(self.params['agent_param']) * \
len(self.params['model_param']) * \
self.iterations
assert agent_vars.shape == (rows, 6)
batch = self.launch_batch_processing()
agent_vars = batch.get_agent_vars_dataframe()
expected_cols = (len(self.variable_params) +
len(self.agent_reporters) +
2) # extra columns with run index and agentId

self.assertEqual(agent_vars.shape,
(self.model_runs * NUM_AGENTS, expected_cols))

def test_model_with_fixed_parameters_as_kwargs(self):
"""
Test that model with fixed parameters passed like kwargs is
properly handled
"""
self.fixed_params = {'fixed_model_param': 'Fixed', 'n_agents': 1}
batch = self.launch_batch_processing()
model_vars = batch.get_model_vars_dataframe()
agent_vars = batch.get_agent_vars_dataframe()

self.assertEqual(len(model_vars), len(agent_vars))
self.assertEqual(len(model_vars), self.model_runs)
self.assertEqual(model_vars['reported_fixed_value'].unique(), ['Fixed'])

def test_model_with_variable_and_fixed_kwargs(self):
self.mock_model = MockMixedModel
self.model_reporters = {
'reported_fixed_param': lambda m: m.fixed_name,
'reported_variable_param': lambda m: m.variable_name
}
self.fixed_params = {'fixed_name': 'Fixed'}
self.variable_params = {'variable_name': [1, 2, 3]}
batch = self.launch_batch_processing()
model_vars = batch.get_model_vars_dataframe()
expected_cols = (len(self.variable_params) +
len(self.model_reporters) +
1)
self.assertEqual(model_vars.shape, (self.model_runs, expected_cols))
self.assertEqual(model_vars['reported_fixed_param'].iloc[0],
self.fixed_params['fixed_name'])
2 changes: 1 addition & 1 deletion tests/test_visualization.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@ def __init__(self, width, height, key1=103, key2=104):
self.grid = Grid(width, height, torus=True)

for (c, x, y) in self.grid.coord_iter():
a = MockAgent(x + y * 100, x * y * 3)
a = MockAgent(x + y * 100, self, x * y * 3)
self.grid.place_agent(a, (x, y))
self.schedule.add(a)

0 comments on commit 4196c38

Please sign in to comment.