require and use the new pytest-caching plugin, implement mtime-checking

CHANGELOG

@@ -1,6 +1,10 @@
 0.9.2.dev
 ----------------------------------------------
 
+- extend pep8ignore to allow lines of form
+  "glob CODE1 CODE2", for example: "test/*.py W293 E203"
+- speed up tests by preventing pep8 checking if 
+  a file was unmodified after its last change.
 - simplified pep8 checker usage (thanks flox)
 
 0.9.1

README.txt

@@ -1,70 +1,171 @@
-py.test plugin for checking PEP8 source code compliance using the `pep8 module <http://pypi.python.org/pypi/pep8>`_.
+py.test plugin for efficiently checking PEP8 compliance 
+========================================================================
 
 Usage
 ---------
 
 install via::
 
-    easy_install pytest-pep8 # or
     pip install pytest-pep8
 
-and then type::
+if you then type::
 
     py.test --pep8
 
-to activate source code checking. Every file ending in ``.py`` will be
-discovered and checked, starting from the command line arguments.
-For example, if you have a file like this::
+every file ending in ``.py`` will be discovered and pep8-checked, 
+starting from the command line arguments. 
+
+.. warning::
+
+    Running pep8 tests on your project is likely to cause a lot of 
+    issues.  This plugin allows to configure on a per-project and
+    per-file basis which errors or warnings to care about, see
+    pep8ignore_.  As a preliminary advise, if you have 
+    projects where you don't want to care at all about pep8 checks, 
+    you can put configure it like this::
+
+        # content of setup.cfg (or pytest.ini)
+        [pytest]
+        pep8ignore = * ALL
+
+
+A little example 
+-----------------------
+
+If you have a pep8-violating file like this::
 
     # content of myfile.py
 
     somefunc( 123,456)
 
-you can run it with::
+you can run it with the plugin installed::
 
     $ py.test --pep8
     =========================== test session starts ============================
-    platform linux2 -- Python 2.6.5 -- pytest-2.0.1.dev1
-    pep8 ignore opts: (performing all available checks)
+    platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev2
+    pep8: performing checks
     collecting ... collected 1 items
 
     myfile.py F
 
     ================================= FAILURES =================================
     ________________________________ PEP8-check ________________________________
-    /tmp/doc-exec-20/myfile.py:2:10: E201 whitespace after '('
+    /home/hpk/tmp/doc-exec-259/myfile.py:2:10: E201 whitespace after '('
     somefunc( 123,456)
              ^
-    /tmp/doc-exec-20/myfile.py:2:14: E231 missing whitespace after ','
+    /home/hpk/tmp/doc-exec-259/myfile.py:2:14: E231 missing whitespace after ','
     somefunc( 123,456)
                  ^
 
     ========================= 1 failed in 0.01 seconds =========================
 
-In the testing header you will always see the list of pep8 checks that are ignored
-(non by default).  For the meaning of these error and warning codes, see the error
+For the meaning of (E)rror and (W)arning codes, see the error
 output when running against your files or checkout `pep8.py
 <https://github.com/jcrocholl/pep8/blob/master/pep8.py>`_.
 
-Configuring PEP8 options per-project
+Let's not now fix the PEP8 errors::
+
+    # content of myfile.py
+    somefunc(123, 456)
+
+and run again::
+
+    $ py.test --pep8
+    =========================== test session starts ============================
+    platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev2
+    pep8: performing checks
+    collecting ... collected 1 items
+
+    myfile.py .
+
+    ========================= 1 passed in 0.01 seconds =========================
+
+the pep8 check now is passing. Moreover, if
+you run it once again (and report skip reasons)::
+
+    $ py.test --pep8 -rs 
+    =========================== test session starts ============================
+    platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev2
+    pep8: performing checks
+    collecting ... collected 1 items
+
+    myfile.py s
+    ========================= short test summary info ==========================
+    SKIP [1] /home/hpk/p/pytest-pep8/pytest_pep8.py:63: file(s) previously passed PEP8 checks
+
+    ======================== 1 skipped in 0.01 seconds =========================
+
+you can see that the pep8 check was skipped because
+the file has not been modified since it was last checked.
+As the pep8 plugin uses the 
+`pytest-cache plugin <http://pypi.python.org/pypi/pytest-cache>`_
+to implement its caching, you can use its ``--clearcache`` option to 
+remove all pytest caches, among them the pep8 related one, which 
+will trigger the pep8 checking code to run once again::
+
+    $ py.test --pep8 --clearcache
+    =========================== test session starts ============================
+    platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev2
+    pep8: performing checks
+    collecting ... collected 1 items
+
+    myfile.py .
+
+    ========================= 1 passed in 0.01 seconds =========================
+
+.. _pep8ignore:
+
+Configuring PEP8 options per project and file
 ---------------------------------------------
 
 You may configure PEP8-checking options for your project
-by adding an ``pep8ignore`` entry to your ``pytest.ini``
+by adding an ``pep8ignore`` entry to your ``setup.cfg``
 or ``setup.cfg`` file like this::
 
-    # content of pytest.ini
+    # content of setup.cfg
     [pytest]
     pep8ignore = E201 E231
 
-This would prevent complaints about some whitespace issues (see above).
-Rerunning it with the above example will now look better::
+This would globally prevent complaints about two whitespace issues.
+Rerunning with the above example will now look better::
 
-    $ py.test -q --pep8
+    $ py.test -q  --pep8
     collecting ... collected 1 items
     .
     1 passed in 0.01 seconds
 
+If you have some files where you want to specifically ignore 
+some errors or warnings you can start a pep8ignore line with 
+a glob-pattern and a space-separated list of codes::
+
+    # content of setup.cfg
+    [pytest]
+    pep8ignore = 
+        *.py E201
+        doc/conf.py ALL
+
+So if you have a conf.py like this::
+
+    # content of doc/conf.py
+
+    func (  [1,2,3]) #this line lots pep8 errors :)
+
+then running again with the previous example will show a single
+failure and it will ignore doc/conf.py alltogether::
+
+    $ py.test --pep8 -v # verbose shows what is ignored
+    =========================== test session starts ============================
+    platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev2 -- /home/hpk/venv/1/bin/python
+    pep8: performing checks
+    cachedir: /home/hpk/tmp/doc-exec-259/.cache
+    collecting ... collected 1 items
+
+    myfile.py:0: PEP8-check(ignoring E201) PASSED
+
+    ========================= 1 passed in 0.01 seconds =========================
+
+Note that doc/conf.py was not considered or imported.
+
 Running PEP8 checks and no other tests
 ---------------------------------------------
 

pytest_pep8.py

@@ -1,61 +1,109 @@
-import pep8
+import re
 import py
 import pytest
+import pep8
+
+__version__ = '0.9.2.dev2'
 
-__version__ = '0.9.2.dev1'
+HISTKEY = "pep8/mtimes"
 
 
 def pytest_addoption(parser):
     group = parser.getgroup("general")
     group.addoption('--pep8', action='store_true',
         help="perform some pep8 sanity checks on .py files")
-    parser.addini("pep8ignore", type="args",
-        help="warning/errors to ignore, example value: W293 W292")
+    parser.addini("pep8ignore", type="linelist",
+        help="each line specifies a glob pattern and whitespace "
+             "separated PEP8 errors or warnings which will be ignored, "
+             "example: *.py W293")
 
 
-def pytest_collect_file(path, parent):
-    if parent.config.option.pep8 and path.ext == '.py':
-        return Pep8Item(path, parent)
+def pytest_sessionstart(session):
+    config = session.config
+    if config.option.pep8:
+        config._pep8ignore = Ignorer(config.getini("pep8ignore"))
+        config._pep8mtimes = config.cache.get(HISTKEY, {})
 
 
-def pytest_configure(config):
-    if config.option.pep8:
-        config._pep8ignore = config.getini("pep8ignore")
+def pytest_collect_file(path, parent):
+    config = parent.config
+    if config.option.pep8 and path.ext == '.py':
+        pep8ignore = config._pep8ignore(path)
+        if pep8ignore is not None:
+            return Pep8Item(path, parent, pep8ignore)
 
 
-def pytest_report_header(config):
-    pep8ignore = getattr(config, '_pep8ignore', None)
-    if pep8ignore is not None:
-        if pep8ignore:
-            pep8ignore = " ".join(pep8ignore)
-        else:
-            pep8ignore = "(performing all available checks)"
-        return "pep8 ignore opts: " + pep8ignore
+def pytest_sessionfinish(session):
+    config = session.config
+    if hasattr(config, "_pep8mtimes"):
+        config.cache.set(HISTKEY, config._pep8mtimes)
 
 
 class Pep8Error(Exception):
     """ indicates an error during pep8 checks. """
 
 
 class Pep8Item(pytest.Item, pytest.File):
-    def __init__(self, path, parent):
+
+    def __init__(self, path, parent, pep8ignore):
         super(Pep8Item, self).__init__(path, parent)
         self.keywords['pep8'] = True
+        self.pep8ignore = pep8ignore
+
+    def setup(self):
+        pep8mtimes = self.config._pep8mtimes
+        self._pep8mtime = self.fspath.mtime()
+        old = pep8mtimes.get(str(self.fspath), (0, []))
+        if old == (self._pep8mtime, self.pep8ignore):
+            pytest.skip("file(s) previously passed PEP8 checks")
 
     def runtest(self):
         call = py.io.StdCapture.call
-        pep8ignore = self.config._pep8ignore
-        found_errors, out, err = call(check_file, self.fspath, pep8ignore)
+        found_errors, out, err = call(check_file, self.fspath, self.pep8ignore)
         if found_errors:
             raise Pep8Error(out, err)
+        # update mtime only if test passed
+        # otherwise failures would not be re-run next time
+        self.config._pep8mtimes[str(self.fspath)] = (self._pep8mtime,
+                                                     self.pep8ignore)
 
     def repr_failure(self, excinfo):
         if excinfo.errisinstance(Pep8Error):
             return excinfo.value.args[0]
         return super(Pep8Item, self).repr_failure(excinfo)
 
     def reportinfo(self):
-        return (self.fspath, -1, "PEP8-check")
+        if self.pep8ignore:
+            ignores = "(ignoring %s)" % " ".join(self.pep8ignore)
+        else:
+            ignores = ""
+        return (self.fspath, -1, "PEP8-check%s" % ignores)
+
+
+class Ignorer:
+    def __init__(self, ignorelines, coderex=re.compile("[EW]\d\d\d")):
+        self.ignores = ignores = []
+        for line in ignorelines:
+            try:
+                glob, ign = line.split(None, 1)
+            except ValueError:
+                glob, ign = None, line
+            if glob and coderex.match(glob):
+                glob, ign = None, line
+            if ign == "ALL":
+                ign = None
+            else:
+                ign = ign.split()
+            ignores.append((glob, ign))
+
+    def __call__(self, path):
+        l = []
+        for (glob, ignlist) in self.ignores:
+            if not glob or path.fnmatch(glob):
+                if ignlist is None:
+                    return None
+                l.extend(ignlist)
+        return l
 
 
 def check_file(path, pep8ignore):

setup.py

@@ -1,13 +1,15 @@
 from setuptools import setup
-setup(
-    name='pytest-pep8',
-    description='pytest plugin to check source code against PEP8 requirements',
-    long_description=open("README.txt").read(),
-    version='0.9.2.dev1',
-    author='Holger Krekel and Ronny Pfannschmidt',
-    author_email='[email protected]',
-    url='http://bitbucket.org/hpk42/pytest-pep8/',
-    py_modules=['pytest_pep8'],
-    entry_points={'pytest11': ['pep8 = pytest_pep8']},
-    install_requires=['pytest>=2.0', 'pep8>=1.3', ],
-)
+
+if __name__ == "__main__":
+    setup(
+        name='pytest-pep8',
+        description='pytest plugin to check PEP8 requirements',
+        long_description=open("README.txt").read(),
+        version='0.9.2.dev2',
+        author='Holger Krekel and Ronny Pfannschmidt',
+        author_email='[email protected]',
+        url='http://bitbucket.org/hpk42/pytest-pep8/',
+        py_modules=['pytest_pep8'],
+        entry_points={'pytest11': ['pep8 = pytest_pep8']},
+        install_requires=['pytest>=2.0', 'pep8>=1.3', ],
+    )