Further modifications to the module formatter to adjust to the new th…

…eme, and some misc docs text corrections.

.gitignore

@@ -24,7 +24,9 @@ docs/man/man3/*
 *.sublime-project
 *.sublime-workspace
 # docsite stuff...
-docsite/rst/modules
+docsite/rst/modules_by_category.rst
+docsite/rst/list_of_*.rst
+docsite/rst/*_module.rst
 docsite/*.html
 docsite/_static/*.gif
 docsite/_static/*.png

docsite/Makefile

@@ -27,6 +27,6 @@ clean:
 .PHONEY: docs clean
 
 modules: $(FORMATTER) ../hacking/templates/rst.j2
-	PYTHONPATH=../lib $(FORMATTER) -t rst --template-dir=../hacking/templates --module-dir=../library -o rst/modules/ --includes-file=rst/modules/_categories.rst
+	PYTHONPATH=../lib $(FORMATTER) -t rst --template-dir=../hacking/templates --module-dir=../library -o rst/ 
 
 

docsite/rst/developing_modules.rst

@@ -226,8 +226,7 @@ The 'group' and 'user' modules are reasonably non-trivial and showcase what this
 
 Key parts include always ending the module file with::
 
-    # include magic from lib/ansible/module_common.py
-    #<<INCLUDE_ANSIBLE_MODULE_COMMON>>
+    from ansible.module_utils.basic import *
     main()
 
 And instantiating the module class like::
@@ -372,10 +371,7 @@ syntax highlighting before you include it in your Python file.
 Example
 +++++++
 
-To print a basic documentation string, run ``./hacking/module_formatter.py -G``. 
-
-You can copy it into your module and use it as a starting point 
-when writing your own docs.
+See an example documentation string in the checkout under examples/DOCUMENTATION.yml
 
 Include it in your module file like this::
 
@@ -389,8 +385,9 @@ Include it in your module file like this::
     # ... snip ...
     '''
 
-The ``description``, and ``notes`` 
-support formatting in some of the output formats (e.g. ``rst``, ``man``).
+The ``description``, and ``notes`` fields 
+support formatting with some special macros.  
+
 These formatting functions are ``U()``, ``M()``, ``I()``, and ``C()``
 for URL, module, italic, and constant-width respectively. It is suggested
 to use ``C()`` for file and option names, and ``I()`` when referencing
@@ -405,9 +402,8 @@ like this::
     - action: modulename opt1=arg1 opt2=arg2
     '''
 
-The ``module_formatter.py`` script and ``ansible-doc(1)`` append the
-``EXAMPLES`` blob after any existing (deprecated) ``examples`` you may have in the
-YAML ``DOCUMENTATION`` string.
+The EXAMPLES section, just like the documentation section, is required in
+all module pull requests for new modules.
 
 .. _module_dev_testing:
 

docsite/rst/index.rst

@@ -23,12 +23,13 @@ This documentation covers the current released version of Ansible (1.4.3) and al
 .. _an_introduction:
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
    intro
    playbooks
    playbooks_special_topics
    modules
+   modules_by_category
    guides
    developing
    awx

docsite/rst/modules.rst

@@ -1,5 +1,5 @@
-Ansible Modules
-===============
+About Modules
+=============
 
 .. toctree::
    :maxdepth: 4
@@ -9,11 +9,10 @@ Ansible Modules
 Introduction
 ````````````
 
-
 Ansible ships with a number of modules (called the 'module library')
 that can be executed directly on remote hosts or through :doc:`Playbooks <playbooks>`.
-Users can also write their own modules.   These modules can control system
-resources, like services, packages, or files (anything really), or
+
+Users can also write their own modules.   These modules can control system resources, like services, packages, or files (anything really), or
 handle executing system commands.
 
 Let's review how we execute three different modules from the command line::
@@ -23,64 +22,28 @@ Let's review how we execute three different modules from the command line::
     ansible webservers -m command -a "/sbin/reboot -t now"
 
 Each module supports taking arguments.  Nearly all modules take ``key=value``
-arguments, space delimited.  Some modules take no arguments, and the
-command/shell modules simply take the string of the command you want to run.
+arguments, space delimited.  Some modules take no arguments, and the command/shell modules simply 
+take the string of the command you want to run.
 
 From playbooks, Ansible modules are executed in a very similar way::
 
     - name: reboot the servers
       action: command /sbin/reboot -t now
 
-Version 0.8 and higher support the following shorter syntax::
+Which can be abbreviated to:
 
     - name: reboot the servers
       command: /sbin/reboot -t now
 
-All modules technically return JSON format data, though if you are using the
-command line or playbooks, you don't really need to know much about
-that.  If you're writing your own module, you care, and this means you do
-not have to write modules in any particular language -- you get to choose.
+All modules technically return JSON format data, though if you are using the command line or playbooks, you don't really need to know much about
+that.  If you're writing your own module, you care, and this means you do not have to write modules in any particular language -- you get to choose.
 
 Modules are `idempotent`, meaning they will seek to avoid changes to the system unless a change needs to be made.  When using Ansible
-playbooks, these modules can trigger 'change events' in the form of notifying 'handlers'
-to run additional tasks.
-
-Documentation for each module can be accessed from the command line with the
-ansible-doc as well as the man command::
-
-    ansible-doc command
-
-    man ansible.template
-
-Let's see what's available in the Ansible module library, out of the box:
-
-.. include:: modules/_categories.rst
-
-.. _ansible_doc:
-
-Reading Module Documentation Locally
-````````````````````````````````````
-
-ansible-doc is a friendly command line tool that allows you to access module documentation locally.
-It comes with Ansible.
-
-To list documentation for a particular module::
-
-   ansible-doc yum | less
-
-To list all modules available::
-
-   ansible-doc --list | less
-
-To access modules outside of the stock module path (such as custom modules that live in your playbook directory), 
-use the '--module-path' option to specify the directory where the module lives.
-
-.. _writing_modules:
+playbooks, these modules can trigger 'change events' in the form of notifying 'handlers' to run additional tasks.
 
-Writing your own modules
-````````````````````````
+Documentation for each module can be accessed from the command line with the ansible-doc tool::
 
-See :doc:`developing_modules`.
+    ansible-doc yum
 
 .. seealso::
 

hacking/module_formatter.py

@@ -102,7 +102,7 @@ def rst_xline(width, char="="):
 
 #####################################################################################
 
-def return_data(text, options, outputname, module):
+def write_data(text, options, outputname, module):
     ''' dumps module output to a file or the screen, as requested '''
 
     if options.output_dir is not None:
@@ -188,7 +188,7 @@ def jinja2_environment(template_dir, typ):
         env.filters['fmt'] = rst_fmt
         env.filters['xline'] = rst_xline
         template = env.get_template('rst.j2')
-        outputname = "%s.rst"
+        outputname = "%s_module.rst"
     else:
         raise Exception("unknown module format type: %s" % typ)
 
@@ -214,7 +214,7 @@ def process_module(module, options, env, template, outputname, module_map):
         sys.stderr.write("*** ERROR: CORE MODULE MISSING DOCUMENTATION: %s, %s ***\n" % (fname, module))
         sys.exit(1)
     if doc is None:
-        return
+        return "SKIPPED"
 
     all_keys = []
 
@@ -250,14 +250,18 @@ def process_module(module, options, env, template, outputname, module_map):
     # here is where we build the table of contents...
 
     text = template.render(doc)
-    return_data(text, options, outputname, module)
+    write_data(text, options, outputname, module)
 
 #####################################################################################
 
 def process_category(category, categories, options, env, template, outputname):
 
     module_map = categories[category]
 
+    category_file_path = os.path.join(options.output_dir, "list_of_%s_modules.rst" % category)
+    category_file = open(category_file_path, "w")
+    print "*** recording category %s in %s ***" % (category, category_file_path) 
+
     # TODO: start a new category file
 
     category = category.replace("_"," ")
@@ -266,8 +270,22 @@ def process_category(category, categories, options, env, template, outputname):
     modules = module_map.keys()
     modules.sort()
 
+    category_header = "%s Modules" % (category.title())
+    underscores = "`" * len(category_header)
+
+    category_file.write(category_header)
+    category_file.write("\n")
+    category_file.write(underscores)
+    category_file.write("\n")
+    category_file.write(".. toctree::\n")
+
     for module in modules:
-        process_module(module, options, env, template, outputname, module_map)
+        result = process_module(module, options, env, template, outputname, module_map)
+        if result != "SKIPPED":
+            category_file.write("    %s_module\n" % module)
+
+
+    category_file.close()
 
     # TODO: end a new category file
 
@@ -305,9 +323,19 @@ def main():
     last_category = None
     category_names = categories.keys()
     category_names.sort()
+
+    category_list_path = os.path.join(options.output_dir, "modules_by_category.rst")
+    category_list_file = open(category_list_path, "w")
+    category_list_file.write("Module Index\n")
+    category_list_file.write("============\n")
+    category_list_file.write("\n\n")
+    category_list_file.write(".. toctree::\n")
 
     for category in category_names:
+        category_list_file.write("    list_of_%s_modules\n" % category)
         process_category(category, categories, options, env, template, outputname)
 
+    category_list_file.close()
+
 if __name__ == '__main__':
     main()