forked from Python-Markdown/markdown
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
5 changed files
with
178 additions
and
140 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,174 @@ | ||
title: Test Tools | ||
|
||
# Test Tools | ||
|
||
Python-Markdown provides some testing tools which simplify testing actual | ||
Markdown output again expected output. The tools are built on the Python | ||
standard library [`unittest`][unittest]. Therefore, no additional libraries are | ||
required. While Python-Markdown uses the tools for its own tests, they were | ||
designed and built so that third party extensions could use them as well. | ||
Therefore, the tools are importable from `markdown.test_tools`. | ||
|
||
The test tools include two different `unittest.TestCase` subclasses: | ||
`markdown.test_tools.TestCase` and `markdown.test_tools.LegacyTestCase`. | ||
|
||
## markdown.test_tools.TestCase | ||
|
||
The `markdown.test_tools.TestCase` class is a `unittest.TestCase` subclass with | ||
a few additional helpers to make testing Markdown output easier. | ||
|
||
Properties | ||
: `default_kwargs`: A `dict` of keywords to pass to Markdown for each | ||
test. The defaults can be overridden on individual tests. | ||
|
||
Methods | ||
: `assertMarkdownRenders`: accepts the source text, the expected output, | ||
and any keywords to pass to Markdown. The `default_kwargs` defined on the | ||
class are used except where overridden by keyword arguments. The output and | ||
expected output are passed to `TestCase.assertMultiLineEqual`. An | ||
`AssertionError` is raised with a diff if the actual output does not equal the | ||
expected output. | ||
|
||
: `dedent`: Dedent triple-quoted strings. | ||
|
||
In all other respects, `markdown.test_tools.TestCase` behaves as | ||
`unittest.TestCase`. In fact, `assertMarkdownRenders` tests could be mixed with | ||
other `unittest` style tests within the same test class. | ||
|
||
An example Markdown test might look like this: | ||
|
||
```python | ||
from markdown.test_tools import TestCase | ||
|
||
class TestHr(TestCase): | ||
def test_hr_before_paragraph(self): | ||
self.assertMarkdownRenders( | ||
# The Markdown source text used as input | ||
self.dedent( | ||
""" | ||
*** | ||
An HR followed by a paragraph with no blank line. | ||
""" | ||
), | ||
# The expected HTML output | ||
self.dedent( | ||
""" | ||
<hr> | ||
<p>An HR followed by a paragraph with no blank line.</p> | ||
""" | ||
), | ||
# Other keyword arguments to pass to `markdown.markdown` | ||
output_format='html' | ||
) | ||
``` | ||
|
||
## markdown.test_tools.LegacyTestCase | ||
|
||
In the past Python-Markdown exclusively used file-based tests. Many of those | ||
tests still exist in Python-Markdown's test suite, including the test files from | ||
the [reference implementation][perl] (`markdown.pl`) and [PHP Markdown][PHP]. | ||
Each test consists of a matching pair of text and HTML files. The text file | ||
contains a snippet of Markdown source text formatted for a specific syntax | ||
feature and the HTML file contains the expected HTML output of that snippet. | ||
When the test suite is run, each text file is run through Markdown and the | ||
output is compared with the HTML file as a separate unit test. When a test | ||
fails, the error report includes a diff of the expected output compared to the | ||
actual output to easily identify any problems. | ||
|
||
A separate `markdown.test_tools.LegacyTestCase` subclass must be created for | ||
each directory of test files. Various properties can be defined within the | ||
subclass to point to a directory of text-based test files and define various | ||
behaviors/defaults for those tests. The following properties are supported: | ||
|
||
* `location`: A path to the directory of test files. An absolute path is | ||
preferred. | ||
* `exclude`: A list of tests to skip. Each test name should comprise of a | ||
file name without an extension. | ||
* `normalize`: A boolean value indicating if the HTML should be normalized. | ||
Default: `False`. Note: Normalization of HTML requires that [PyTidyLib] be | ||
installed on the system. If PyTidyLib is not installed and `normalize` is set | ||
to `True`, then the test will be skipped, regardless of any other settings. | ||
* `input_ext`: A string containing the file extension of input files. | ||
Default: `.txt`. | ||
* `output_ext`: A string containing the file extension of expected output files. | ||
Default: `html`. | ||
* `default_kwargs`: A `markdown.test_tools.Kwargs` instance which stores the | ||
default set of keyword arguments for all test files in the directory. | ||
|
||
In addition, properties can be defined for each individual set of test files | ||
within the directory. The property should be given the name of the file without | ||
the file extension. Any spaces and dashes in the file name should be replaced | ||
with underscores. The value of the property should be a | ||
`markdown.test_tools.Kwargs` instance which contains the keyword arguments that | ||
should be passed to `markdown.markdown` for that test file. The keyword | ||
arguments will "update" the `default_kwargs`. | ||
|
||
When the class instance is created during a test run, it will walk the given | ||
directory and create a separate unit test for each set of test files using the | ||
naming scheme: `test_filename`. One unit test will be run for each set of input | ||
and output files. | ||
|
||
The definition of an example set of tests might look like this: | ||
|
||
```python | ||
from markdown.test_tools import LegacyTestCase, Kwargs | ||
import os | ||
|
||
# Get location of this file and use to find text file dirs. | ||
parent_test_dir = os.path.abspath(os.path.dirname(__file__)) | ||
|
||
|
||
class TestFoo(LegacyTestCase): | ||
# Define location of text file directory. In this case, the directory is | ||
# named "foo" and is in the same parent directory as this file. | ||
location = os.path.join(parent_test_dir, 'foo') | ||
# Define default keyword arguments. In this case, unless specified | ||
# differently, all tests should use the output format "html". | ||
default_kwargs = Kwargs(output_format='html') | ||
|
||
# The "xhtml" test should override the output format and use "xhtml". | ||
xhtml = Kwargs(output_format='xhtml') | ||
|
||
# The "toc" test should use the "toc" extension with a custom permalink | ||
# setting. | ||
toc = Kwargs( | ||
extensions=['markdown.extensions.toc'], | ||
extension_configs={'markdown.extensions.toc': {'permalink': "[link]"}} | ||
) | ||
``` | ||
|
||
Note that in the above example, the text file directory may contain many more | ||
text-based test files than `xhtml` (`xhtml.txt` and `xhtml.html`) and `toc` | ||
(`toc.txt` and `toc.html`). As long as each set of files exists as a pair, a | ||
test will be created and run for each of them. Only the `xhtml` and `toc` tests | ||
needed to be specifically identified as they had specific, non-default settings | ||
which needed to be defined. | ||
|
||
## Running Python-Markdown's Tests | ||
|
||
As all of the tests for the `markdown` library are unit tests, standard | ||
`unittest` methods of calling tests can be used. For example, to run all of | ||
Python-Markdown's tests, from the root of the git repository, run the following | ||
command: | ||
|
||
```sh | ||
python -m unittest discover tests | ||
``` | ||
|
||
That simple command will search everything in the `tests` directory and it's | ||
sub-directories and run all `unittest` tests that it finds, including | ||
`unittest.TestCase`, `markdown.test_tools.TestCase`, and | ||
`markdown.test_tools.LegacyTestCase` subclasses. Normal `unittest` discovery | ||
rules apply. | ||
|
||
Python-Markdown's git repository also includes a `tox.ini` file, so [tox] can be | ||
used to automate the creation of virtual environments, installation of all | ||
testing dependencies and running of the tests on each supported Python version. | ||
See the wiki for instructions on [setting up a testing environment] to use tox. | ||
|
||
[unittest]: https://docs.python.org/3/library/unittest.html | ||
[Perl]: http://daringfireball.net/projects/markdown/ | ||
[PHP]: http://michelf.com/projects/php-markdown/ | ||
[PyTidyLib]: http://countergram.com/open-source/pytidylib/ | ||
[tox]: http://testrun.org/tox/latest/ | ||
[setting up a testing environment]: https://github.com/Python-Markdown/markdown/wiki/Test-Environment-Setup |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters