Support custom labels in TOC. (Python-Markdown#700)

New `toc_tokens` attribute on Markdown class.
Contains the raw tokens used to build the Table of Contents. Users can 
use this to build their own custom Table of Contents rather than needing 
to parse the HTML available on the `toc` attribute of the Markdown 
class.

docs/change_log/release-3.0.md

@@ -202,5 +202,17 @@ The following new features have been included in the release:
 * A new `toc_depth` parameter has been added to the
   [Table of Contents Extension](../extensions/toc.md).
 
+* A new `toc_tokens` attribute has been added to the Markdown class by the
+  [Table of Contents Extension](../extensions/toc.md), which contains the raw
+  tokens used to build the Table of Contents. Users can use this to build their
+  own custom Table of Contents rather than needing to parse the HTML available
+  on the `toc` attribute of the Markdown class.
+
+* When the [Table of Contents Extension](../extensions/toc.md) is used in
+  conjunction with the [Attribute Lists Extension](../extensions/attr_list.md)
+  and a `data-toc-label` attribute is defined on a header, the content of the
+  `data-toc-label` attribute is now used as the content of the Table of Contents
+  item for that header.
+
 * Additional CSS class names can be appended to
   [Admonitions](../extensions/admonition.md).

docs/extensions/toc.md

@@ -56,7 +56,7 @@ would generate the following output:
   </ul>
 </div>
 <h1 id="header-1">Header 1</h1>
-<h1 id="header-2">Header 2</h1>
+<h2 id="header-2">Header 2</h2>
 ```
 
 Regardless of whether a `marker` is found in the document (or disabled), the
@@ -70,6 +70,67 @@ template. For example:
 >>> page = render_some_template(context={'body': html, 'toc': md.toc})
 ```
 
+The `toc_tokens` attribute is also available on the Markdown class and contains
+a nested list of dict objects. For example, the above document would result in
+the following object at `md.toc_tokens`:
+
+```python
+[
+    {
+        'level': 1,
+        'id': 'header-1',
+        'name': 'Header 1',
+        'children': [
+            {'level': 2, 'id': 'header-2', 'name': 'Header 2', 'children':[]}
+        ]
+    }
+]
+```
+
+Note that the `level` refers to the `hn` level. In other words, `<h1>` is level
+`1` and `<h2>` is level `2`, etc. Be aware that improperly nested levels in the
+input may result in odd nesting of the output.
+
+### Custom Labels
+
+In most cases, the text label in the Table of Contents should match the text of
+the header. However, occasionally that is not desirable. In that case, if this
+extension is used in conjunction with the [Attribute Lists Extension] and a
+`data-toc-label` attribute is defined on the header, then the contents of that
+attribute will be used as the text label for the item in the Table of Contents.
+For example, the following Markdown:
+
+[Attribute Lists Extension]: attr_list.md
+
+```md
+[TOC]
+
+# Functions
+
+## `markdown.markdown(text [, **kwargs])` { #markdown data-toc-label='markdown.markdown' }
+```
+would generate the following output:
+
+```html
+<div class="toc">
+  <ul>
+    <li><a href="#functions">Functions</a></li>
+      <ul>
+        <li><a href="#markdown">markdown.markdown</a></li>
+      </ul>
+  </ul>
+</div>
+<h1 id="functions">Functions</h1>
+<h2 id="markdown"><code>markdown.markdown(text [, **kwargs])</code></h2>
+```
+
+Notice that the text in the Table of Contents is much cleaner and easier to read
+in the context of a Table of Contents. The `data-toc-label` is not included in
+the HTML header element. Also note that the ID was manually defined in the
+attribute list to provide a cleaner URL when linking to the header. If the ID is
+not manually defined, it is always derived from the text of the header, never
+from the `data-toc-label` attribute.
+
 Usage
 -----
 

markdown/extensions/toc.py

@@ -247,22 +247,28 @@ def run(self, doc):
                 toc_tokens.append({
                     'level': int(el.tag[-1]),
                     'id': el.attrib["id"],
-                    'name': text
+                    'name': el.attrib.get('data-toc-label', text)
                 })
 
+                # Remove the data-toc-label attribute as it is no longer needed
+                if 'data-toc-label' in el.attrib:
+                    del el.attrib['data-toc-label']
+
                 if self.use_anchors:
                     self.add_anchor(el, el.attrib["id"])
                 if self.use_permalinks:
                     self.add_permalink(el, el.attrib["id"])
 
-        div = self.build_toc_div(nest_toc_tokens(toc_tokens))
+        toc_tokens = nest_toc_tokens(toc_tokens)
+        div = self.build_toc_div(toc_tokens)
         if self.marker:
             self.replace_marker(doc, div)
 
         # serialize and attach to markdown instance.
         toc = self.md.serializer(div)
         for pp in self.md.postprocessors:
             toc = pp.run(toc)
+        self.md.toc_tokens = toc_tokens
         self.md.toc = toc
 
 
@@ -310,6 +316,7 @@ def extendMarkdown(self, md):
 
     def reset(self):
         self.md.toc = ''
+        self.md.toc_tokens = []
 
 
 def makeExtension(**kwargs):  # pragma: no cover

tests/test_extensions.py

@@ -852,24 +852,40 @@ def testTitle(self):
     def testWithAttrList(self):
         """ Test TOC with attr_list Extension. """
         md = markdown.Markdown(extensions=['toc', 'attr_list'])
-        text = '# Header 1\n\n## Header 2 { #foo }'
+        text = '# Header 1\n\n## Header 2 { #foo }\n\n## Header 3 { data-toc-label="Foo Bar"}'
         self.assertEqual(
             md.convert(text),
             '<h1 id="header-1">Header 1</h1>\n'
-            '<h2 id="foo">Header 2</h2>'
+            '<h2 id="foo">Header 2</h2>\n'
+            '<h2 id="header-3">Header 3</h2>'
         )
         self.assertEqual(
             md.toc,
             '<div class="toc">\n'
-              '<ul>\n'                                        # noqa
-                '<li><a href="#header-1">Header 1</a>'        # noqa
-                  '<ul>\n'                                    # noqa
-                    '<li><a href="#foo">Header 2</a></li>\n'  # noqa
-                  '</ul>\n'                                   # noqa
-                '</li>\n'                                     # noqa
-              '</ul>\n'                                       # noqa
+              '<ul>\n'                                           # noqa
+                '<li><a href="#header-1">Header 1</a>'           # noqa
+                  '<ul>\n'                                       # noqa
+                    '<li><a href="#foo">Header 2</a></li>\n'     # noqa
+                    '<li><a href="#header-3">Foo Bar</a></li>\n' # noqa
+                  '</ul>\n'                                      # noqa
+                '</li>\n'                                        # noqa
+              '</ul>\n'                                          # noqa
             '</div>\n'
         )
+        self.assertEqual(
+            md.toc_tokens,
+            [
+                {
+                    'level': 1,
+                    'id': 'header-1',
+                    'name': 'Header 1',
+                    'children': [
+                        {'level': 2, 'id': 'foo', 'name': 'Header 2', 'children': []},
+                        {'level': 2, 'id': 'header-3', 'name': 'Foo Bar', 'children': []}
+                    ]
+                }
+            ]
+        )
 
     def testUniqueFunc(self):
         """ Test 'unique' function. """