Skip to content

Commit

Permalink
Merge branch 'pr/2448' into master.
Browse files Browse the repository at this point in the history
  • Loading branch information
rgommers committed May 5, 2013
2 parents 2c49b31 + cb0f89a commit 009be70
Showing 1 changed file with 42 additions and 42 deletions.
84 changes: 42 additions & 42 deletions doc/source/tutorial/weave.rst
Original file line number Diff line number Diff line change
Expand Up @@ -306,8 +306,8 @@ inline, it is still less efficient for simple code snippets than using
equivalent Python code. The simple ``printf`` example is actually slower by
30% or so than using Python ``print`` statement. And, it is not difficult to
create code fragments that are 8-10 times slower using inline than equivalent
Python. However, for more complicated algorithms, the speed up can be worth
while -- anywhwere from 1.5- 30 times faster. Algorithms that have to
Python. However, for more complicated algorithms, the speedup can be
worthwhile -- anywhere from 1.5-30 times faster. Algorithms that have to
manipulate Python objects (sorting a list) usually only see a factor of 2 or
so improvement. Algorithms that are highly computational or manipulate NumPy
arrays can see much larger improvements. The examples/vq.py file shows a
Expand All @@ -319,7 +319,7 @@ More with printf
================

MSVC users will actually see a bit of compiler output that distutils does not
supress the first time the code executes::
suppress the first time the code executes::

>>> weave.inline(r'printf("%d\n",a);',['a'])
sc_e013937dbc8c647ac62438874e5795131.cpp
Expand Down Expand Up @@ -483,7 +483,7 @@ CXX library can be found at its `website`_. The basics are that the CXX
provides C++ class equivalents for Python objects that simplify, or at least
object orientify, working with Python objects in C/C++. For example,
``seq.length()`` returns the length of the list. A little more about CXX and
its class methods, etc. is in the ** type conversions ** section.
its class methods, etc. is in the `Type Conversions` section.

.. note::
CXX uses templates and therefore may be a little less portable than
Expand Down Expand Up @@ -883,7 +883,7 @@ Microsoft VC++ installed.


When ``inline`` is first run, you'll notice that pause and some trash printed
to the screen. The "trash" is acutually part of the compilers output that
to the screen. The "trash" is actually part of the compiler's output that
distutils does not supress. The name of the extension file,
``sc_bighonkingnumber.cpp``, is generated from the md5 check sum of the C/C++
code fragment. On Unix or windows machines with only gcc installed, the trash
Expand Down Expand Up @@ -1269,7 +1269,7 @@ this, either uses exceptions or set ``return_val`` to NULL and use
Technical Details
=================

There are several main steps to using C/C++ code withing Python:
There are several main steps to using C/C++ code within Python:

1. Type conversion
2. Generating C/C++ code
Expand Down Expand Up @@ -1343,10 +1343,10 @@ This is actually the exact code that is generated by calling


The ``/* inline code */`` section is filled with the code passed to the
``inline()`` function call. The ``/*argument convserion code*/`` and ``/*
``inline()`` function call. The ``/*argument conversion code*/`` and ``/*
cleanup code */`` sections are filled with code that handles conversion from
Python to C++ types and code that deallocates memory or manipulates reference
counts before the function returns. The following sections demostrate how
counts before the function returns. The following sections demonstrate how
these two areas are filled in by the default conversion methods. * Note: I'm
not sure I have reference counting correct on a few of these. The only thing
I increase/decrease the ref count on is NumPy arrays. If you see an issue,
Expand Down Expand Up @@ -1417,7 +1417,7 @@ The argument conversion code inserted for ``a`` is::


``get_variable()`` reads the variable ``a`` from the local and global
namespaces. ``py_to_list()`` and its friends has the following form::
namespaces. ``py_to_list()`` and its friends have the following form::

static Py::List py_to_list(PyObject* py_obj,char* name)
{
Expand Down Expand Up @@ -1543,8 +1543,8 @@ really is::

PythonWin has reassigned ``sys.stdout`` to a special object that implements
the Python file interface. This works great in Python, but since the special
object doesn't have a FILE* pointer underlying it, fprintf doesn't know what
to do with it (well this will be the problem when instance conversion is
object doesn't have a FILE* pointer underlying it, ``fprintf`` doesn't know
what to do with it (well this will be the problem when instance conversion is
implemented...).

Callable, Instance, and Module Conversion
Expand All @@ -1563,7 +1563,7 @@ Callable, Instance, and Module Conversion


Callable and instance variables are converted to PyObject*. Nothing is done
to there reference counts.
to their reference counts.

::

Expand Down Expand Up @@ -1596,12 +1596,12 @@ There is no cleanup code for callables, modules, or instances.
Customizing Conversions
-----------------------

Converting from Python to C++ types is handled by xxx_specification classes.
A type specification class actually serve in two related but different roles.
The first is in determining whether a Python variable that needs to be
converted should be represented by the given class. The second is as a code
generator that generate C++ code needed to convert from Python to C++ types
for a specific variable.
Converting from Python to C++ types is handled by ``xxx_specification``
classes. A type specification class actually serve in two related but
different roles. The first is in determining whether a Python variable that
needs to be converted should be represented by the given class. The second is
as a code generator that generates C++ code needed to convert from Python to
C++ types for a specific variable.

When

Expand Down Expand Up @@ -1690,7 +1690,7 @@ code string and the value is either a single function (as in the "level 1"
cache) or a list of functions (as in the main catalog cache). On disk
catalogs are stored in the same manor using standard Python shelves.

Early on, there was a question as to whether md5 check sums of the C++ code
Early on, there was a question as to whether md5 checksums of the C++ code
strings should be used instead of the actual code strings. I think this is
the route inline Perl took. Some (admittedly quick) tests of the md5 vs. the
entire string showed that using the entire string was at least a factor of 3
Expand All @@ -1712,7 +1712,7 @@ you're out of luck. I don't think either of these should ever fail though. On
Windows, a directory called pythonXX_compiled is created in the user's
temporary directory.

The actual catalog file that lives in this directory is a Python shelve with
The actual catalog file that lives in this directory is a Python shelf with
a platform specific name such as "nt21compiled_catalog" so that multiple OSes
can share the same file systems without trampling on each other. Along with
the catalog file, the .cpp and .so or .pyd files created by inline will live
Expand Down Expand Up @@ -1850,7 +1850,7 @@ get the most use.
Limitations
===========

1. Currently, ``weave.blitz`` handles all standard mathematic operators
1. Currently, ``weave.blitz`` handles all standard mathematical operators
except for the ** power operator. The built-in trigonmetric, log,
floor/ceil, and fabs functions might work (but haven't been tested). It
also handles all types of array indexing supported by the NumPy module.
Expand Down Expand Up @@ -1878,7 +1878,7 @@ Limitations

3. ``weave.blitz`` works best when algorithms can be expressed in a
"vectorized" form. Algorithms that have a large number of if/thens and
other conditions are better hand written in C or Fortran. Further, the
other conditions are better hand-written in C or Fortran. Further, the
restrictions imposed by requiring vectorized expressions sometimes
preclude the use of more efficient data structures or algorithms. For
maximum speed in these cases, hand-coded C or Fortran code is the only
Expand Down Expand Up @@ -1976,13 +1976,13 @@ need for a temporary array. This provides a significant speed improvement
over the above example (write me and tell me what you get).

So, converting NumPy expressions into C/C++ loops that fuse the loops and
eliminate temporary arrays can provide big gains. The goal then,is to convert
NumPy expression to C/C++ loops, compile them in an extension module, and
then call the compiled extension function. The good news is that there is an
obvious correspondence between the NumPy expression above and the C loop. The
bad news is that NumPy is generally much more powerful than this simple
eliminate temporary arrays can provide big gains. The goal, then, is to
convert NumPy expression to C/C++ loops, compile them in an extension module,
and then call the compiled extension function. The good news is that there is
an obvious correspondence between the NumPy expression above and the C loop.
The bad news is that NumPy is generally much more powerful than this simple
example illustrates and handling all possible indexing possibilities results
in loops that are less than straight forward to write. (take a peak in NumPy
in loops that are less than straightforward to write. (Take a peek at NumPy
for confirmation). Luckily, there are several available tools that simplify
the process.

Expand All @@ -1996,7 +1996,7 @@ Oliphant's NumPy module. On the compiled language side, Todd Veldhuizen's
blitz++ array library, written in C++ (shhhh. don't tell David Beazley), does
the heavy lifting. Don't assume that, because it's C++, it's much slower than
C or Fortran. Blitz++ uses a jaw dropping array of template techniques
(metaprogramming, template expression, etc) to convert innocent looking and
(metaprogramming, template expression, etc) to convert innocent-looking and
readable C++ expressions into to code that usually executes within a few
percentage points of Fortran code for the same problem. This is good.
Unfortunately all the template raz-ma-taz is very expensive to compile, so
Expand Down Expand Up @@ -2064,7 +2064,7 @@ list representation of the tree.
['ENDMARKER', '']]


Despite its looks, with some tools developed by Jermey H., its possible to
Despite its looks, with some tools developed by Jermey H., it's possible to
search these trees for specific patterns (sub-trees), extract the sub-tree,
manipulate them converting python specific code fragments to blitz code
fragments, and then re-insert it in the parse tree. The parser module
Expand Down Expand Up @@ -2100,12 +2100,12 @@ In Blitz it is as follows::

Here the range object works exactly like Python slice objects with the
exception that the top index (3) is inclusive where as Python's (4) is
exclusive. Other differences include the type declaraions in C++ and
exclusive. Other differences include the type declarations in C++ and
parentheses instead of brackets for indexing arrays. Currently,
``weave.blitz`` handles the inclusive/exclusive issue by subtracting one from
upper indices during the translation. An alternative that is likely more
robust/maintainable in the long run, is to write a PyRange class that behaves
like Python's range. This is likely very easy.
robust/maintainable in the long run is to write a ``PyRange`` class that
behaves like Python's ``range``. This is likely very easy.

The stock blitz also doesn't handle negative indices in ranges. The current
implementation of the ``blitz()`` has a partial solution to this problem. It
Expand All @@ -2119,7 +2119,7 @@ maximum index in the array so that::

This approach fails, however, when the top index is calculated from other
values. In the following scenario, if ``i+j`` evaluates to a negative value,
the compiled code will produce incorrect results and could even core- dump.
the compiled code will produce incorrect results and could even core-dump.
Right now, all calculated indices are assumed to be positive.

::
Expand Down Expand Up @@ -2162,8 +2162,8 @@ expression with the following translations::

``_all`` is defined in the compiled function as ``blitz::Range.all()``. These
translations could of course happen directly in the syntax tree. But the
string replacement is slightly easier. Note that name spaces are maintained
in the C++ code to lessen the likelyhood of name clashes. Currently no effort
string replacement is slightly easier. Note that namespaces are maintained
in the C++ code to lessen the likelihood of name clashes. Currently no effort
is made to detect name clashes. A good rule of thumb is don't use values that
start with '_' or 'py\_' in compiled expressions and you'll be fine.

Expand Down Expand Up @@ -2249,7 +2249,7 @@ However, expressions like::


are not so trivial. Since slicing is involved, the size of the slices, not
the input arrays must be checked. Broadcasting complicates things further
the input arrays, must be checked. Broadcasting complicates things further
because arrays and slices with different dimensions and shapes may be
compatible for math operations (broadcasting isn't yet supported by
``weave.blitz``). Reductions have a similar effect as their results are
Expand All @@ -2270,7 +2270,7 @@ traces the changing array sizes through all operations and fails if
incompatible array sizes are ever found.

The machinery for this is housed in ``weave.size_check``. It basically
involves writing a new class (dummy array) and overloading it math operators
involves writing a new class (dummy array) and overloading its math operators
to calculate the new sizes correctly. All the code is in Python and there is
a fair amount of logic (mainly to handle indexing and slicing) so the
operation does impose some overhead. For large arrays (ie. 50x50x50), the
Expand Down Expand Up @@ -2357,7 +2357,7 @@ creates an ``ext_module`` instance that is ready to have ``ext_function``
instances added to it. ``ext_function`` instances are created much with a
calling convention similar to ``weave.inline()``. The most common call
includes a C/C++ code snippet and a list of the arguments for the function.
The following
The following::

ext_code = "return_val = Py::new_reference_to(Py::Int(a+1));"
func = ext_tools.ext_function('increment',ext_code,['a'])
Expand All @@ -2383,9 +2383,9 @@ the current working directory. This example is available in the
the bottom of the file in the module's "main" program, an attempt to import
``increment_ext`` without building it is made. If this fails (the module
doesn't exist in the PYTHONPATH), the module is built by calling
``build_increment_ext()``. This approach only takes the time consuming ( a
few seconds for this example) process of building the module if it hasn't
been built before.
``build_increment_ext()``. This approach only takes the time-consuming (a few
seconds for this example) process of building the module if it hasn't been
built before.

::

Expand Down

0 comments on commit 009be70

Please sign in to comment.