Merge branch 'expr_dev' of https://github.com/Pyomo/pyomo into expr_dev

.appveyor.yml

@@ -14,10 +14,10 @@ environment:
 
     # Removing testing of 3.4, as it inexplicably runs >10x slower than
     # other Python versions and regularly times out (60 minutes)
-    #- PYTHON_VERSION: 3.4
-    #  #PYTHON: "C:\\Python34"
-    #  PYTHON: "C:\\Miniconda3-x64"
-    #  CATEGORY: "nightly"
+    - PYTHON_VERSION: 3.4
+      #PYTHON: "C:\\Python34"
+      PYTHON: "C:\\Miniconda34-x64"
+      CATEGORY: "nightly"
 
     - PYTHON_VERSION: 3.5
       #PYTHON: "C:\\Python35"
@@ -35,22 +35,25 @@ environment:
       CATEGORY: "nightly"
       EXTRAS: YES
 
+    - PYTHON_VERSION: 3.4
+      #PYTHON: "C:\\Python34"
+      PYTHON: "C:\\Miniconda34-x64"
+      CATEGORY: "nightly"
+      EXTRAS: YES
+
+    - PYTHON_VERSION: 3.5
+      #PYTHON: "C:\\Python35"
+      PYTHON: "C:\\Miniconda35-x64"
+      CATEGORY: "nightly"
+      EXTRAS: YES
+
     - PYTHON_VERSION: 3.6
       #PYTHON: "C:\\Python36"
       PYTHON: "C:\\Miniconda36-x64"
       CATEGORY: "nightly"
       EXTRAS: YES
 
 
-    ##- PYTHON: "C:\\Python33"
-    #- PYTHON: "C:\\Python34"
-    #- PYTHON: "C:\\Python27-x64"
-    ##- PYTHON: "C:\\Python33-x64"
-    ##  DISTUTILS_USE_SDK: "1"
-    #- PYTHON: "C:\\Python34-x64"
-    #  DISTUTILS_USE_SDK: "1"
-    #- PYTHON: "C:\\Python35-x64"
-
 install:
   # We need wheel installed to build wheels
   - "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PYTHON%\\Library\\bin;%PATH%"
@@ -65,12 +68,13 @@ install:
   - SET CONDA_INSTALL=conda install -q -y
   - "SET ANACONDA=%CONDA_INSTALL% -c anaconda"
   - "SET CONDAFORGE=%CONDA_INSTALL% -c conda-forge --no-update-dependencies"
-  - "SET CACHEMEORG=%CONDA_INSTALL% -c cachemeorg --no-update-dependencies"
   #
   # Update conda, then force it to NOT update itself again
   #
+  # Somehow, the update from anaconda stalls for Python 3.4. So we're not specifying the channel here.
+  #
   - conda config --set always_yes yes
-  - conda update -q -y -c anaconda conda
+  - conda update -q -y conda
   - conda config --set auto_update_conda false
   #
   - "%CONDAFORGE% setuptools pip"
@@ -93,17 +97,17 @@ install:
   #
   # Install pyomo.extras
   #
-  - "IF DEFINED EXTRAS (%CONDAFORGE% pyomo.extras)"
+  - "IF DEFINED EXTRAS (%CONDAFORGE% numpy scipy ipython openpyxl sympy pymysql pyodbc pyro4 pyyaml networkx xlrd pandas matplotlib dill)"
   #
   - "%CONDAFORGE% glpk"
   - "glpsol -v"
   # NOTE: conda-forge doesn't build ipopt for Windows....
-  - "%CACHEMEORG% ipopt_bin"
+  - "%CONDAFORGE% ipopt"
   - "ipopt -v"
   #
   # Clone but don't install pyomo-model-libraries
   #
-  - "git clone https://github.com/Pyomo/pyomo-model-libraries.git"
+  - "git clone -b expr_dev https://github.com/Pyomo/pyomo-model-libraries.git"
   - "python -m pip install git+https://github.com/PyUtilib/pyutilib"
   - "python setup.py develop"
   #

.travis.yml

@@ -111,8 +111,11 @@ install:
   #
   # Install Pyomo extras
   #
+  # NOTE: We explicitly install all of the packages in pyomo.extras, since these this
+  # package is not kept up-to-date with Python 3.4
+  #
   - if [ -n "${EXTRAS}" ]; then
-       conda install -q -y -c conda-forge --no-update-dependencies pyomo.extras;
+        conda install -q -y -c conda-forge --no-update-dependencies numpy scipy ipython openpyxl sympy pymysql pyodbc pyro4 pyyaml networkx xlrd pandas matplotlib dill;
     fi
 
   #

CHANGELOG.txt

@@ -6,7 +6,10 @@ Pyomo CHANGELOG
 Current Development
 -------------------------------------------------------------------------------
 
-- None
+- Resolved Python 3.4 build errors on Appveyor.
+- Wrap Vars in value() for assignment to numpy vectors (#436)
+- Adding TerminationCondition and SolverStatus to pyomo.environ (#429)
+- Adding the Pyomo5 expression system, which supports PyPy (#272)
 
 -------------------------------------------------------------------------------
 Pyomo 5.5

README.md

@@ -22,6 +22,10 @@ Pyomo is a Python-based open-source software package that supports a diverse set
 
 Pyomo is available under the BSD License.
 
+Pyomo is currently tested with the following Python implementations:
+
+* CPython: 2.6, 2.7, 3.4, 3.5, 3.6
+
 ### Installation
 
 #### PyPI [![PyPI](https://img.shields.io/pypi/v/pyomo.svg?maxAge=2592000)]()

doc/OnlineDocs/developer_reference/expressions/design.rst

@@ -6,16 +6,16 @@ Design Details
 ==============
 
 .. warning::
-    Unfortunately, Pyomo expression trees are not composed of Python
-    objects from a single class hierarchy.  There are fundamental
-    design constraints (referenced below) that impact the form of
-    Pyomo expression trees.
+    Pyomo expression trees are not composed of Python
+    objects from a single class hierarchy.  Consequently, Pyomo
+    relies on duck typing to ensure that valid expression trees are 
+    created.
 
-Most Pyomo expression trees have the following form:
+Most Pyomo expression trees have the following form
 
-#. Interior nodes are objects that inherit from the :class:`ExpressionBase <pyomo.core.expr.current.ExpressionBase>` class.
+1. Interior nodes are objects that inherit from the :class:`ExpressionBase <pyomo.core.expr.current.ExpressionBase>` class.  These objects typically have one or more child nodes.  Linear expression nodes do not have child nodes, but they are treated as interior nodes in the expression tree because they references other leaf nodes.
 
-#. Leaf nodes are numeric values, parameter components and variable components, which represent the *inputs* to the expresion.
+2. Leaf nodes are numeric values, parameter components and variable components, which represent the *inputs* to the expresion.
 
 Expression Classes
 ------------------
@@ -30,7 +30,7 @@ sum        ``x + y``     :class:`SumExpression <pyomo.core.expr.current.SumExpre
 product    ``x * y``     :class:`ProductExpression <pyomo.core.expr.current.ProductExpression>`
 negation   ``- x``       :class:`NegationExpression <pyomo.core.expr.current.NegationExpression>`
 reciprocal ``1 / x``     :class:`ReciprocalExpression <pyomo.core.expr.current.ReciprocalExpression>`
-power      ``x ** y``    :class:`PowerExpression <pyomo.core.expr.current.PowerExpression>`
+power      ``x ** y``    :class:`PowExpression <pyomo.core.expr.current.PowExpression>`
 inequality ``x <= y``    :class:`InequalityExpression <pyomo.core.expr.current.InequalityExpression>`
 equality   ``x == y``    :class:`EqualityExpression <pyomo.core.expr.current.EqualityExpression>`
 ========== ============= =============================================================================
@@ -42,7 +42,7 @@ logical relationships, which are summarized in the following table:
 Operation            Example                                Pyomo Class
 ==================== ====================================   ========================================================================================
 exernal function     ``myfunc(x,y,z)``                      :class:`ExternalFunctionExpression <pyomo.core.expr.current.ExternalFunctionExpression>`
-logical if-then-else ``Expr_if(IF_=x, THEN_=y, ELSE_=z)``   :class:`Expr_if <pyomo.core.expr.current.Expr_if>`
+logical if-then-else ``Expr_if(IF=x, THEN=y, ELSE=z)``      :class:`Expr_ifExpression <pyomo.core.expr.current.Expr_ifExpression>`
 intrinsic function   ``sin(x)``                             :class:`UnaryFunctionExpression <pyomo.core.expr.current.UnaryFunctionExpression>`
 absolute function    ``abs(x)``                             :class:`AbsExpression <pyomo.core.expr.current.AbsExpression>`
 ==================== ====================================   ========================================================================================
@@ -71,17 +71,12 @@ Expression trees can be categorized in four different ways:
 * potentially variable expressions - expressions that contain variables, which may be fixed.
 * fixed expressions - expressions that contain variables, all of which are fixed.
 
-These three categories are illustrated with the following example::
+These three categories are illustrated with the following example:
 
-    m = ConcreteModel()
-    m.p = Param(default=10, mutable=False)
-    m.q = Param(default=10, mutable=True)
-    m.x = Var()
-    m.y = Var(initialize=1)
-    m.y.fixed = True
+.. literalinclude:: ../../tests/expr/design_categories.spy
 
-The following table describes four diffrent simple expressions,
-which consist of a single model component, and it shows how they
+The following table describes four different simple expressions
+that consist of a single model component, and it shows how they
 are categorized:
 
 ======================== ===== ===== ===== =====
@@ -103,22 +98,16 @@ potentially variability.
 Special Expression Classes
 --------------------------
 
-The following classes are *exceptions* to some of the design principles describe above.
+The following classes are *exceptions* to the design principles describe above.
 
 Named Expressions
 ~~~~~~~~~~~~~~~~~
 
-Pyomo includes several classes that *named expressions*, which allow for flexible changes to 
-an expression after it has been constructed.  For example, consider the expression ``f`` defined
-with the :class:`Expression <pyomo.core.base.Expression>` component::
+Named expressions allow for changes to an expression after it has
+been constructed.  For example, consider the expression ``f`` defined
+with the :class:`Expression <pyomo.core.base.Expression>` component:
 
-    M = ConcreteModel()
-    M.v = Var()
-    M.w = Var()
-
-    M.e = Expression(expr=2*M.v)
-    f = M.e + 3                     # f == 2*v + 3
-    M.e += M.w                      # f == 2*v + 3 + w
+.. literalinclude:: ../../tests/expr/design_named_expression.spy
 
 Although ``f`` is an immutable expression, whose definition is
 fixed, a sub-expressions is the named expression ``M.e``.  Named
@@ -129,12 +118,11 @@ the named expression.
 
 .. note::
 
-    The named expression classes are not implemented as
-    sub-classes of :class:`ExpressionBase
-    <pyomo.core.expr.current.ExpressionBase>`.  This reflects design
-    constraints related to the fact that these are modeling components
-    that belong to class hierarchies other than the expression class
-    hierarchy, and Pyomo uses code optimizations that prohibit
+    The named expression classes are not implemented as sub-classes
+    of :class:`ExpressionBase <pyomo.core.expr.current.ExpressionBase>`.
+    This reflects design constraints related to the fact that these
+    are modeling components that belong to class hierarchies other
+    than the expression class hierarchy, and Pyomo's design prohibits
     the use of multiple inheritance for these classes.
 
 Linear Expressions
@@ -169,7 +157,7 @@ object owns the first ``n`` arguments in the shared list, but
 different objects may have different values of ``n``.
 
 This class acts like a normal immutable expression class, and the
-API described above works fine.  But direct access to the shared
+API described above works normally.  But direct access to the shared
 list could have unexpected results.
 
 Mutable Expressions
@@ -227,3 +215,54 @@ The following classes are valid leaf nodes:
     not represent a leaf node in the tree.
 
 
+
+Context Managers
+----------------
+
+Pyomo defines several context managers that can be used to declare
+the form of expressions, and to define a mutable expression object that
+efficiently manages sums.
+
+The :data:`linear_expression <pyomo.core.expr.current.linear_expression>` 
+object is a context manager that can be used to declare a linear sum.  For
+example, consider the following two loops:
+
+.. literalinclude:: ../../tests/expr/design_cm1.spy
+
+The first apparent difference in these loops is that the value of
+``s`` is explicitly initialized while ``e`` is initialized when the
+context manager is entered.  However, a more fundamental difference
+is that the expression representation for ``s`` differs from ``e``.
+Each term added to ``s`` results in a new, immutable expression.
+By contrast, the context manager creates a mutable expression
+representation for ``e``.  This difference allows for both (a) a
+more efficient processing of each sum, and (b) a more compact
+representation for the expression.
+
+The difference between :data:`linear_expression
+<pyomo.core.expr.current.linear_expression>` and
+:data:`nonlinear_expression <pyomo.core.expr.current.nonlinear_expression>`
+is the underlying representation that each supports.  Note that
+both of these are instances of context manager classes.  In
+singled-threaded applications, these objects can be safely used to
+construct different expressions with different context declarations.
+
+Finally, note that these context managers can be passed into the :attr:`start`
+method for the :func:`quicksum <pyomo.core.util.quicksum>` function.  For example:
+
+.. literalinclude:: ../../tests/expr/design_cm2.spy
+
+This sum contains terms for ``M.x[i]`` and ``M.y[i]``.  The syntax
+in this example is not intuitive because the sum is being stored
+in ``e``.
+
+.. note::
+
+    We do not generally expect users or developers to use these
+    context managers.  They are used by the :func:`quicksum
+    <pyomo.core.util.quicksum>` and :func:`sum_product
+    <pyomo.core.util.sum_product>` functions to accelerate expression
+    generation, and there are few cases where the direct use of
+    these context managers would provide additional utility to users
+    and developers.
+

doc/OnlineDocs/developer_reference/expressions/index.rst

@@ -5,49 +5,28 @@
 Pyomo Expressions
 =================
 
-.. warning::
-
-    This is draft documentation for Pyomo expressions.  In particular,
-    this documentation describes Pyomo5 expressions in detail.  The 
-    API for the functions and classes being described here is being
-    finalized.  Please submit a ticket to suggest a change in the
-    API or to request clarification in the design below.
-
-.. warning::
-
-    The examples in this draft documentation are not well-tested.
-    The examples look fine to the author, but I have not confirmed
-    that these examples are tested with Pymo.
-
 .. warning::
 
     This documentation does not explicitly reference objects in
     pyomo.core.kernel.  While the Pyomo5 expression system works
     with pyomo.core.kernel objects, the documentation of these
-    documents wasn't sufficient to appropriately descibe the use
+    documents was not sufficient to appropriately descibe the use
     of kernel objects in expressions.
 
-Pyomo supports the declaration of abstract expressions that represent
+Pyomo supports the declaration of symbolic expressions that represent
 objectives, constraints and other optimization modeling components.
 Pyomo expressions are represented in an expression tree, where the
 leaves are operands, such as constants or variables, and the internal
 nodes contain operators.  Pyomo relies on so-called magic methods
-to automate the construction of abstract expressions.  For example,
+to automate the construction of symbolic expressions.  For example,
 consider an expression ``e`` declared as follows:
 
-.. doctest::
-
-   >>> from pyomo.environ import *
-
-   >>> M = ConcreteModel()
-   >>> M.v = Var()
-
-   >>> e = M.v*2
+.. literalinclude:: ../../tests/expr/index_simple.spy
 
 Python determines that the magic method ``__mul__`` is called on
 the ``M.v`` object, with the argument ``2``.  This method returns
-a Pyomo expression object ``_ProductExpression`` that has arguments
-``M.v`` and ``2``.  This represents the following abstract expression
+a Pyomo expression object ``ProductExpression`` that has arguments
+``M.v`` and ``2``.  This represents the following symbolic expression
 tree:
 
 .. graphviz::
@@ -60,7 +39,7 @@ tree:
 .. note::
 
     End-users will not likely need to know details related to how
-    abstract expressions are generated and managed in Pyomo.  Thus,
+    symbolic expressions are generated and managed in Pyomo.  Thus,
     most of the following documentation of expressions in Pyomo is most
     useful for Pyomo developers.  However, the discussion of runtime
     performance in the first section will help end-users write large-scale