Skip to content

Latest commit

 

History

History
279 lines (204 loc) · 8.39 KB

operations.rst

File metadata and controls

279 lines (204 loc) · 8.39 KB
.. currentmodule:: sparse

Operations on :obj:`COO` arrays

Operators

:obj:`COO` objects support a number of operations. They interact with scalars, :doc:`Numpy arrays <reference/generated/numpy.ndarray>`, other :obj:`COO` objects, and :obj:`scipy.sparse.spmatrix` objects, all following standard Python and Numpy conventions.

For example, the following Numpy expression produces equivalent results for both Numpy arrays, COO arrays, or a mix of the two:

np.log(X.dot(beta.T) + 1)

However some operations are not supported, like operations that implicitly cause dense structures, or numpy functions that are not yet implemented for sparse arrays.

x + 1      # operations that produce dense results not supported
np.svd(x)  # sparse svd not implemented

This page describes those valid operations, and their limitations.

:obj:`elemwise`

This function allows you to apply any arbitrary broadcasting function to any number of arguments where the arguments can be :obj:`SparseArray` objects or :obj:`scipy.sparse.spmatrix` objects. For example, the following will add two arrays:

sparse.elemwise(np.add, x, y)

Warning

Previously, :obj:`elemwise` was a method of the :obj:`COO` class. Now, it has been moved to the :obj:`sparse` module.

Auto-Densification

Operations that would result in dense matrices, such as binary operations with :doc:`Numpy arrays <reference/generated/numpy.ndarray>` objects or certain operations with scalars are not allowed and will raise a :obj:`ValueError`. For example, all of the following will raise a :obj:`ValueError`. Here, x and y are :obj:`COO` objects.

x == y
x + 5
x == 0
x != 5
x / y

However, all of the following are valid operations.

x + 0
x != y
x + y
x == 5
5 * x
x / 7.3
x != 0

If densification is needed, it must be explicit. In other words, you must call :obj:`COO.todense` on the :obj:`COO` object. If both operands are :obj:`COO`, both must be densified.

Warning

Previously, operations with Numpy arrays were sometimes supported. Now, it is necessary to convert Numpy arrays to :obj:`COO` objects.

Operations with :obj:`scipy.sparse.spmatrix`

Certain operations with :obj:`scipy.sparse.spmatrix` are also supported. For example, the following are all allowed if y is a :obj:`scipy.sparse.spmatrix`:

x + y
x - y
x * y
x > y
x < y

In general, if operating on a scipy.sparse.spmatrix is the same as operating on :obj:`COO`, as long as it is to the right of the operator.

Note

Results are not guaranteed if x is a :obj:`scipy.sparse.spmatrix`. For this reason, we recommend that all Scipy sparse matrices should be explicitly converted to :obj:`COO` before any operations.

Broadcasting

All binary operators support :doc:`broadcasting <user/basics.broadcasting>`. This means that (under certain conditions) you can perform binary operations on arrays with unequal shape. Namely, when the shape is missing a dimension, or when a dimension is 1. For example, performing a binary operation on two :obj:`COO` arrays with shapes (4,) and (5, 1) yields an object of shape (5, 4). The same happens with arrays of shape (1, 4) and (5, 1). However, (4, 1) and (5, 1) will raise a :obj:`ValueError`.

Full List of Operators

Here, x and y can be :obj:`COO` arrays, :obj:`numpy.ndarray` objects or scalars, keeping in mind :ref:`auto densification rules <operations-auto-densification>`. In addition, y can also be a :obj:`scipy.sparse.spmatrix` The following operators are supported:

Warning

While in-place operations are supported for compatibility with Numpy, they are not truly in-place, and will effectively calculate the result separately.

Element-wise Operations

:obj:`COO` arrays support a variety of element-wise operations. However, as with operators, operations that map zero to a nonzero value are not supported.

To illustrate, the following are all possible, and will produce another :obj:`COO` array:

np.abs(x)
np.sin(x)
np.sqrt(x)
np.conj(x)
np.expm1(x)
np.log1p(x)

However, the following are all unsupported and will raise a :obj:`ValueError`:

np.exp(x)
np.cos(x)
np.log(x)

Notice that you can apply any unary or binary :doc:`numpy.ufunc <reference/ufuncs>` to :obj:`COO` arrays, and :obj:`numpy.ndarray` objects and scalars and it will work so long as the result is not dense. When applying to :obj:`numpy.ndarray` objects, we check that operating on the array with zero would always produce a zero.

Reductions

:obj:`COO` objects support a number of reductions. However, not all important reductions are currently implemented (help welcome!) All of the following currently work:

x.sum(axis=1)
np.max(x)
np.min(x, axis=(0, 2))
x.prod()

Note

If you are performing multiple reductions along the same axes, it may be beneficial to call :obj:`COO.enable_caching`.

:obj:`COO.reduce`

This method can take an arbitrary :doc:`numpy.ufunc <reference/ufuncs>` and performs a reduction using that method. For example, the following will perform a sum:

x.reduce(np.add, axis=1)

Note

This library currently performs reductions by grouping together all coordinates along the supplied axes and reducing those. Then, if the number in a group is deficient, it reduces an extra time with zero. As a result, if reductions can change by adding multiple zeros to it, this method won't be accurate. However, it works in most cases.

Partial List of Supported Reductions

Although any binary :doc:`numpy.ufunc <reference/ufuncs>` should work for reductions, when calling in the form x.reduction(), the following reductions are supported:

Indexing

:obj:`COO` arrays can be :obj:`indexed <numpy.doc.indexing>` just like regular :obj:`numpy.ndarray` objects. They support integer, slice and boolean indexing. However, currently, numpy advanced indexing is not properly supported. This means that all of the following work like in Numpy, except that they will produce :obj:`COO` arrays rather than :obj:`numpy.ndarray` objects, and will produce scalars where expected. Assume that z.shape is (5, 6, 7)

z[0]
z[1, 3]
z[1, 4, 3]
z[:3, :2, 3]
z[::-1, 1, 3]
z[-1]
z[[True, False, True, False, True], 3, 4]

All of the following will raise an :obj:`IndexError`, like in Numpy 1.13 and later.

z[6]
z[3, 6]
z[1, 4, 8]
z[-6]
z[[True, True, False, True], 3, 4]

Note

Numpy advanced indexing is currently not supported.

Other Operations

:obj:`COO` arrays support a number of other common operations. Among them are :obj:`dot`, :obj:`tensordot`, :obj:`concatenate` and :obj:`stack`, :obj:`transpose <COO.transpose>` and :obj:`reshape <COO.reshape>`. You can view the full list on the :doc:`API reference page <generated/sparse>`.