yapy-cookiecutter

Yet Another Python Cookiecutter template for a Python modules and applications focussing on data science, processing and analysis natural sciences stuff.

Notes:

It is based on and derived from the excellent cookiecutter-pylibrary cookiecutter by Ionel Cristian Mărieș.

Table of Contents

• Features
• Requirements
• Usage and options
  • Developing the project
  • Releasing the project
    • Version management
      • Commit Message Format
        • Type
        • Scope
        • Subject
        • Body
        • Footer
    • Building and uploading
• Changelog
• Questions & answers
• Not Exactly What You Want?

Features

Related to the original cookiecutter-pylibrary some flexibility was removed and some choices are implemented:

• Python packaging without the now obsolete Setuptools:
  • based on pyproject.toml
  • no setup.py or setup.cfg
• Employs automatic semantic versioning based on commit messages, please see version-management
• Generates GitHub workflow files for CI/CD, which
  1. run tests matrix of multiple Python versions and OS platforms
  2. set release version number
  3. generate documentation
  4. build packages (binary and source)
  5. upload the packages to PyPI (test.pypi by default) and create a GitHub release page for regular releases
• Tox for managing test environments for Python 3.7+
• Pytest for testing Python 3.7+
• Code coverage report of all tests compiled by pytest-cov for coverage tracking, added to the online documentation
• Documentation with Sphinx to be hosted on GitHub Pages
• Choice of various licenses.
• Configurations for (also via pre-commit):
  • isort
  • black
  • flake8
• EditorConfig - for being picked up by VS-Code, it seems a plugin is needed
• Packaging and code quality checks. This template comes with a tox environment (check) that will:
  • Check if your README.rst is valid.
  • Check if the MANIFEST.in has any issues.
  • Run flake8 (a combo of PEP8, pyflakes and McCabe checks)

Requirements

Projects using this template have these minimal dependencies:

• Cookiecutter - just for creating the project
• Tox - for running the tests

To get quickly started on a new system, just install pip. And install Tox and Cookiecutter by running this in your shell or command prompt:

pip install tox cookiecutter

Usage and options

Please see also the common cookiecutter use guide. First generate your project:

cookiecutter gh:BAMresearch/yapy-cookiecutter

You will be asked for these fields:

Note

Fields that work together usually use the same prefix. If you answer "no" on the first one then the rest won't have any effect so just ignore them. Maybe in the future cookiecutter will allow option hiding or something like a wizard.

Field	Default	Description
full_name	"Ingo Breßler"	Main author of this library or application (used in AUTHORS.rst and pyproject.toml). Can be set in your ~/.cookiecutterrc config file.
email	"[email protected]"	Contact email of the author (used in AUTHORS.rst and pyproject.toml). Can be set in your ~/.cookiecutterrc config file.
website	"https://github.com/ibressler"	Website of the author (used in AUTHORS.rst). Can be set in your ~/.cookiecutterrc config file.
repo_userorg	"ibressler"	GitHub user name or organization name of this project (used for GitHub link, as in https://github.com/ibressler). Can be set in your ~/.cookiecutterrc config file.
project_name	"My Test Project"	Verbose project name, used in headings (docs, readme, etc).
repo_hosting_domain	"github.com"	Use "no" for no hosting (various links will disappear). You can also use "gitlab.com".
repo_name	"my-test-project"	Repository name on GitHub (and project's root directory name). Derived from the project_name.
package_name	"my_test_project"	Python package name (whatever you would import), with underscores for spaces. Derived from the project_name.
distribution_name	"my-test-project"	PyPI distribution name (what you would pip install), with dashes for spaces. Derived from the project_name.
project_short_description	"An example package [...]"	One line description of the project (used in README.rst and pyproject.toml).
release_date	"today"	Release date of the project (ISO 8601 format) default to today (used in CHANGELOG.md).
year	"now"	Copyright year (used in Sphinx conf.py).
version	"0.1.0"	Release version (see pyproject.toml, src/my_test_project/__init__.py and in Sphinx docs/conf.py).
license	"BSD license"	License to use. Available options: BSD license MIT license Apache License 2.0 and more … What license to pick? https://choosealicense.com/
sphinx_theme	"sphinx-rtd-theme"	What Sphinx theme to use. Suggested alternative: sphinx-py3doc-enhanced-theme for a responsive theme based on the Python 3 documentation.
pypi_host	"test.pypi.org"	Choose between the PyPI Test repo (which is the default) and the real PyPI repository. Please remember, files uploaded to PyPI once can not be replaced with files of the same name: files can be deleted (removed from being visible) but not replaced. Therefore, package deployment should be tested on PyPI Testing first.

The testing (tox.ini) configuration is generated from templates. For your convenience there's an initial bootstrap tox.ini, to get the initial generation going just run:

tox

You can later regenerate files in .github/workflows by running:

tox -e bootstrap

After this you can create the initial repository (make sure you create an empty Github project):

git init .
git add .
git commit -m "Initial skel."
git remote add origin [email protected]:ibressler/my-test-project.git
git push -u origin master

Developing the project

To run all the tests, just run:

tox

To see all the tox environments:

tox -l

To only build the docs:

tox -e docs

To build and verify that the built package is proper and other code QA checks:

tox -e check

Releasing the project

Before releasing your package on PyPI you should have all the tox environments passing.

Version management

This template provides a semantic-release configuration. It raises the version number by parsing the GIT commit history:

• Commits starting with fix(…): … will increase the patch version number (1.0.0 to 1.0.1) and
• Commit messages starting with feat(…): … will increase the minor version number (1.0.0 to 1.1.0).
• Commits with BREAKING CHANGE: in the message footer will increase the major version number 1.0.0 to 2.0.0.
• All other changes will generate a new pre-release version number (1.0.1 to 1.0.2-dev.1).

This behavior is implemented by the GitHub action workflow files and templates in ci/templates.

For the basic convention of semantic version numbering, please see Semantic Versioning 2.0.0 and the commit message format expected here is documented in the AngularJS project:

Commit Message Format

Each commit message consists of a header, a body and a footer. The header has a special format that includes a type, a scope and a subject:

<type>(<scope>): <subject>
<BLANK LINE>
<body>
<BLANK LINE>
<footer>

The header is mandatory and the scope of the header is optional.

Any line of the commit message cannot be longer than 100 characters! This allows the message to be easier to read on GitHub as well as in various git tools.

Type

Must be one of the following:

• feat: A new feature
• fix: A bug fix
• docs: Documentation only changes
• style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
• refactor: A code change that neither fixes a bug nor adds a feature
• perf: A code change that improves performance
• test: Adding missing or correcting existing tests
• chore: Changes to the build process or auxiliary tools and libraries such as documentation generation

Scope

The scope could be anything specifying place of the commit change. For example $location, UI, calculus, $rootScope, etc...

You can use * when the change affects more than a single scope.

Subject

The subject contains succinct description of the change:

• use the imperative, present tense: "change" not "changed" nor "changes"
• don't capitalize first letter
• no dot (.) at the end

Body

Just as in the subject, use the imperative, present tense: "change" not "changed" nor "changes". The body should include the motivation for the change and contrast this with previous behavior.

Footer

The footer should contain any information about Breaking Changes and is also the place to reference GitHub issues that this commit closes.

Breaking Changes should start with the word BREAKING CHANGE: with a space or two newlines. The rest of the commit message is then used for this.

Building and uploading

Before building dists make sure you got a clean build area:

rm -rf build
rm -rf src/*.egg-info

Note:

Dirty build or egg-info dirs can cause problems: missing or stale files in the resulting dist or strange and confusing errors. Avoid having them around.

Then you should check that you got no packaging issues:

tox -e check

And then you can build the sdist, and if possible, the bdist_wheel too:

tox -e build

To make a release of the project on PyPI, assuming you got some distributions in dist/, the most simple usage is:

twine upload --skip-existing dist/*.whl dist/*.gz dist/*.zip

Note:

twine is a tool that you can use to securely upload your releases to PyPI. You can still use the old python setup.py register sdist bdist_wheel upload but it's not very secure - your PyPI password will be sent over plaintext.

Changelog

Please see the GIT commit history ;)

Questions & answers

Why does tox.ini have a passenv = *?

Tox 2.0 changes the way it runs subprocesses - it no longer passes all the environment variables by default. This causes all sorts of problems if you want to run/use any of these with Tox: SSH Agents, Browsers (for Selenium), Appengine SDK, VC Compiler and so on.

By default this is not needed. You can always add passenv = * if you like the convenience.

Why is the version stored in several files (pkg/__init__.py, docs/conf.py)?

We cannot use a metadata/version file [1] because this template is to be used with both distributions of packages (dirs with __init__.py) and modules (simple .py files that go straight in site-packages). There's no good place for that extra file if you're distributing modules.

But this isn't so bad - semantic-release manages the version string quite neatly.

[1]	Example, an __about__.py file.

Not Exactly What You Want?

No way, this is the best. 😜

If you have criticism or suggestions please open up an Issue or Pull Request.