Header is not used with -t TestRunner option

[bsipocz]

While it works with -P. This might either be a header issue, or a TestRunner one, feel free to move the issue back to astropy core accordingly.

$ ~/munka/devel/astropy [:c354794b1|✚ 1…3⚑ 26] $ pt -t docs/whatsnew/4.0.rst 
running test
installing to temporary directory: /private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-yk1n1o_h
warning: no files found matching '*' under directory 'scripts'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'build'
warning: no files found matching '*.pyx' under directory 'astropy_helpers/astropy_helpers'
warning: no files found matching '*.h' under directory 'astropy_helpers/astropy_helpers'
warning: no files found matching '*.rst' under directory 'astropy_helpers/astropy_helpers'
no previously-included directories found matching 'astropy_helpers/build'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.o' found anywhere in distribution
================================================= test session starts ==================================================
platform darwin -- Python 3.7.4, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-yk1n1o_h/lib/python3.7/site-packages, inifile: setup.cfg
plugins: astropy-header-0.1.1, forked-1.0.2, arraydiff-0.3, dependency-0.4.0, xonsh-0.9.13.dev1, remotedata-0.3.2, openfiles-0.4.0, asdf-2.4.2, doctestplus-0.5.0, cov-2.8.1
collected 1 item                                                                                                       

docs/whatsnew/4.0.rst F                                                                                          [100%]

======================================================= FAILURES =======================================================

[astrofrog]

Is pt == pytest or python setup.py test?

[bsipocz]

oh, sorry. yes, it's python setup.py test

[bsipocz]

Hmm, in fact it seems not working with pytest either if the source subdir is not specified (this aiming the test from docs being picked up, too)

This is with astroML:

$ ~/munka/devel/worktrees/astroml/hacking [pytest_astropy_header L|✚ 1] $ pytest .
================================================= test session starts ==================================================
platform darwin -- Python 3.7.4, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /Users/bsipocz/munka/devel/worktrees/astroml/hacking, inifile: pytest.ini
plugins: astropy-header-0.1.1, forked-1.0.2, arraydiff-0.3, dependency-0.4.0, xonsh-0.9.13.dev1, remotedata-0.3.2, openfiles-0.4.0, asdf-2.4.2, doctestplus-0.5.0, cov-2.8.1
collected 247 items                                                                                                    

CHANGES.rst .            

And this is on astropy master:

$ ~/munka/devel/astropy [hacking ↓·5|✚ 1…3⚑ 26] $ pytest .
================================================= test session starts ==================================================
platform darwin -- Python 3.7.4, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /Users/bsipocz/munka/devel/astropy, inifile: setup.cfg
plugins: astropy-header-0.1.1, forked-1.0.2, arraydiff-0.3, dependency-0.4.0, xonsh-0.9.13.dev1, remotedata-0.3.2, openfiles-0.4.0, asdf-2.4.2, doctestplus-0.5.0, cov-2.8.1
collected 0 items / 1 errors                                                                                           

======================================================== ERRORS ========================================================
____________________________________________ ERROR collecting test session _____________________________________________
/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py:456: in _importconftest
    return self._conftestpath2mod[key]
E   KeyError: PosixPath('/Users/bsipocz/munka/devel/astropy/astropy/conftest.py')

During handling of the above exception, another exception occurred:
/usr/local/lib/python3.7/site-packages/py/_path/common.py:377: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
/usr/local/lib/python3.7/site-packages/py/_path/common.py:418: in gen
    dirs = self.optsort([p for p in entries
/usr/local/lib/python3.7/site-packages/py/_path/common.py:419: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
/usr/local/lib/python3.7/site-packages/_pytest/main.py:606: in _recurse
    ihook = self.gethookproxy(dirpath)
/usr/local/lib/python3.7/site-packages/_pytest/main.py:424: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py:434: in _getconftestmodules
    mod = self._importconftest(conftestpath)
/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py:481: in _importconftest
    self.consider_conftest(mod)
/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py:534: in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py:340: in register
    ret = super().register(plugin, name)
/usr/local/lib/python3.7/site-packages/pluggy/manager.py:126: in register
    hook._maybe_apply_history(hookimpl)
/usr/local/lib/python3.7/site-packages/pluggy/hooks.py:333: in _maybe_apply_history
    res = self._hookexec(self, [method], kwargs)
/usr/local/lib/python3.7/site-packages/pluggy/manager.py:92: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/usr/local/lib/python3.7/site-packages/pluggy/manager.py:86: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
astropy/conftest.py:42: in pytest_configure
    matplotlibrc_cache.update(matplotlib.rcParams)
/usr/local/lib/python3.7/site-packages/matplotlib/__init__.py:826: in __getitem__
    "3.0", name=key, obj_type="rcparam", addendum="In the future, "
/usr/local/lib/python3.7/site-packages/matplotlib/cbook/deprecation.py:114: in warn_deprecated
    _warn_external(warning)
/usr/local/lib/python3.7/site-packages/matplotlib/cbook/__init__.py:2055: in _warn_external
    warnings.warn(message, category, stacklevel)
E   matplotlib.cbook.deprecation.MatplotlibDeprecationWarning: 
E   The examples.directory rcparam was deprecated in Matplotlib 3.0 and will be removed in 3.2. In the future, examples will be found relative to the 'datapath' directory.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================================== 1 error in 0.35s ===================================================

I suppose this whole business will distil down to the fact that pytest doesn't pick up the config from packagename/conftest.py in either of these cases.

[astrofrog]

The easiest fix is to add the following option in the setup.cfg:

[tool:pytest]
astropy_header = true

and you can also use the astropy_header_packages option to specify the packages to show. I can update the docs for this plugin to mention this.

[pllim]

I added the proposed fix to README in 289e158