Always wrap CodeHilite code in <code> tags (Python-Markdown#862)

Pygments added the `wrapcode` option in version 2.4. Users need to have 2.4+ installed to see the change. However, as earlier versions accepted arbitrary keywords passed to the HTMLFormatter, no error will be raised if the user has an older version of Pygments installed.

docs/change_log/release-3.2.md

@@ -7,7 +7,7 @@ PyPy and PyPy3.
 
 ## Backwards-incompatible changes
 
-## `em` and `strong` inline processor changes
+### `em` and `strong` inline processor changes
 
 In order to fix issue #792, `em`/`strong` inline processors were refactored. This
 translated into removing many of the existing inline processors that handled this
@@ -27,6 +27,23 @@ These processors were replaced with two new ones:
 The [`legacy_em`](../extensions/legacy_em.md) extension was also modified with new,
 refactored logic and simply overrides the `em_strong2` inline processor.
 
+### CodeHilite now always wraps with `<code>` tags
+
+Before, the HTML generated by CodeHilite looked like:
+- `<pre><code>foo = 'bar'</code></pre>` if you **were not** using Pygments.
+- `<pre>foo = 'bar'</pre>`  if you **were** using Pygments.
+
+To make the cases more consistent (and adhere to many Markdown specifications and 
+HTML code block markup suggestions), CodeHilite will now always additionally wrap
+code with `<code>` tags. See #862 for more details.
+
+This change does not alter the Python-Markdown API, but users relying on the old 
+markup will find their output now changed.
+
+Internally, this change relies on the Pygments 2.4, so you must be using at least
+that version to see this effect. Users with earlier Pygments versions will
+continue to see the old behavior.
+
 ## New features
 
 The following new features have been included in the release:
@@ -39,3 +56,4 @@ The following new features have been included in the release:
 The following bug fixes are included in the 3.2 release:
 
 * Refactor bold and italic logic in order to solve complex nesting issues (#792).
+* Always wrap CodeHilite code in <code> tags (#862).

docs/extensions/code_hilite.md

@@ -216,11 +216,18 @@ The following options are provided to configure the output:
     Use inline styles instead of CSS classes. Defaults to `False`.
 
 * **`use_pygments`**:
-    Defaults to `True`. Set to `False` to disable the use of Pygments.
-    If a language is defined for a code block, it will be assigned to the
-    `<code>` tag as a class in the manner suggested by the [HTML5 spec][spec]
-    (alternate output will not be entertained) and might be used by a JavaScript
-    library in the browser to highlight the code block.
+    Specifies the use of Pygments in generating the output.
+
+    If `True` (the default) and Pygments is available, CodeHilite will use
+    Pygments to analyze and format the output. Additionally, if using Pygments
+    &gt;= 2.4, the output will be wrapped in `<code>` tags, whereas earlier
+    versions will not.
+
+    Otherwise, Pygments will not be used. If a language is defined for a code
+    block, it will be assigned to the `<code>` tag as a class in the manner
+    suggested by the [HTML5 spec][spec] (alternate output will not be
+    entertained) and may be used by a JavaScript library in the browser to
+    highlight the code block.
 
 A trivial example:
 

markdown/extensions/codehilite.py

@@ -119,7 +119,8 @@ def hilite(self):
                                               cssclass=self.css_class,
                                               style=self.style,
                                               noclasses=self.noclasses,
-                                              hl_lines=self.hl_lines)
+                                              hl_lines=self.hl_lines,
+                                              wrapcode=True)
             return highlight(self.src, lexer, formatter)
         else:
             # just escape and build markup usable by JS highlighting libs

tests/test_extensions.py

@@ -227,7 +227,7 @@ def testHighlightLinesWithColon(self):
             md = markdown.Markdown(extensions=['codehilite'])
             if self.has_pygments:
                 self.assertStartsWith(
-                    '<div class="codehilite"><pre><span class="hll"',
+                    '<div class="codehilite"><pre><code><span class="hll"',
                     md.convert(text).replace('<span></span>', '')
                 )
             else:
@@ -262,10 +262,10 @@ def testDoubleEscape(self):
                 md.convert(text),
                 '<div class="codehilite"><pre>'
                 '<span></span>'
-                '<span class="p">&lt;</span><span class="nt">span</span><span class="p">&gt;</span>'
+                '<code><span class="p">&lt;</span><span class="nt">span</span><span class="p">&gt;</span>'
                 'This<span class="ni">&amp;amp;</span>That'
                 '<span class="p">&lt;/</span><span class="nt">span</span><span class="p">&gt;</span>'
-                '\n</pre></div>'
+                '\n</code></pre></div>'
             )
         else:
             self.assertEqual(
@@ -285,7 +285,7 @@ def testHighlightAmps(self):
         if self.has_pygments:
             self.assertEqual(
                 md.convert(text),
-                '<div class="codehilite"><pre><span></span>&amp;\n&amp;amp;\n&amp;amp;amp;\n</pre></div>'
+                '<div class="codehilite"><pre><span></span><code>&amp;\n&amp;amp;\n&amp;amp;amp;\n</code></pre></div>'
             )
         else:
             self.assertEqual(
@@ -391,7 +391,7 @@ def testFencedCodeWithHighlightLines(self):
 
         if self.has_pygments:
             self.assertStartsWith(
-                '<div class="codehilite"><pre><span class="hll"',
+                '<div class="codehilite"><pre><code><span class="hll"',
                 md.convert(text).replace('<span></span>', '')
             )
         else:
@@ -426,7 +426,7 @@ def testFencedLanguageAndHighlightLines(self):
             )
             if self.has_pygments:
                 self.assertStartsWith(
-                    '<div class="codehilite"><pre><span class="hll"',
+                    '<div class="codehilite"><pre><code><span class="hll"',
                     md.convert(text).replace('<span></span>', '')
                 )
             else:
@@ -463,11 +463,11 @@ def testFencedLanguageDoubleEscape(self):
             self.assertEqual(
                 md.convert(text),
                 '<div class="codehilite"><pre>'
-                '<span></span>'
+                '<span></span><code>'
                 '<span class="p">&lt;</span><span class="nt">span</span><span class="p">&gt;</span>'
                 'This<span class="ni">&amp;amp;</span>That'
                 '<span class="p">&lt;/</span><span class="nt">span</span><span class="p">&gt;</span>'
-                '\n</pre></div>'
+                '\n</code></pre></div>'
             )
         else:
             self.assertEqual(
@@ -490,7 +490,7 @@ def testFencedAmps(self):
         if self.has_pygments:
             self.assertEqual(
                 md.convert(text),
-                '<div class="codehilite"><pre><span></span>&amp;\n&amp;amp;\n&amp;amp;amp;\n</pre></div>'
+                '<div class="codehilite"><pre><span></span><code>&amp;\n&amp;amp;\n&amp;amp;amp;\n</code></pre></div>'
             )
         else:
             self.assertEqual(