Role arg spec validation implementation (ansible#73152)

* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning

changelogs/fragments/73152-role-arg-spec.yaml

@@ -0,0 +1,4 @@
+major_changes:
+  - Support for role argument specification validation at role execution time.
+    When a role contains an argument spec, an implicit validation task is inserted
+    at the start of role execution.

docs/docsite/rst/user_guide/playbooks_reuse_roles.rst

@@ -256,6 +256,118 @@ You can pass other keywords, including variables and tags, when importing roles:
 
 When you add a tag to an ``import_role`` statement, Ansible applies the tag to `all` tasks within the role. See :ref:`tag_inheritance` for details.
 
+Role Argument Validation
+========================
+
+Beginning with version 2.11, you may choose to enable role argument validation based on an argument
+specification defined in the role ``meta/main.yml`` file. When this argument specification is defined,
+a new task is inserted at the beginning of role execution that will validate the parameters supplied
+for the role against the specification. If the parameters fail validation, the role will fail execution.
+
+Specification Format
+--------------------
+
+The role argument specification must be defined in a top-level ``argument_specs`` block within the
+role ``meta/main.yml`` file. All fields are lower-case.
+
+:entry-point-name:
+
+    * The name of the role entry point.
+    * This should be ``main`` in the case of an unspecified entry point.
+    * This will be the base name of the tasks file to execute, with no ``.yml`` or ``.yaml`` file extension.
+
+    :short_description:
+
+        * A short, one-line description of the entry point.
+        * The ``short_description`` is displayed by ``ansible-doc -t role -l``.
+
+    :description:
+
+        * A longer description that may contain multiple lines.
+
+    :author:
+
+        * Name of the entry point authors.
+        * Use a multi-line list if there is more than one author.
+
+    :options:
+
+        * Options are often called "parameters" or "arguments". This section defines those options.
+        * For each role option (argument), you may include:
+
+        :option-name:
+
+           * The name of the option/argument.
+
+        :description:
+
+           * Detailed explanation of what this option does. It should be written in full sentences.
+
+        :type:
+
+           * The data type of the option. Default is ``str``.
+           * If an option is of type ``list``, ``elements`` should be specified.
+
+        :required:
+
+           * Only needed if ``true``.
+           * If missing, the option is not required.
+
+        :default:
+
+           * If ``required`` is false/missing, ``default`` may be specified (assumed 'null' if missing).
+           * Ensure that the default value in the docs matches the default value in the code. The actual
+             default for the role variable will always come from ``defaults/main.yml``.
+           * The default field must not be listed as part of the description, unless it requires additional information or conditions.
+           * If the option is a boolean value, you can use any of the boolean values recognized by Ansible:
+             (such as true/false or yes/no).  Choose the one that reads better in the context of the option.
+
+        :choices:
+
+           * List of option values.
+           * Should be absent if empty.
+
+        :elements:
+
+           * Specifies the data type for list elements when type is ``list``.
+
+        :suboptions:
+
+           * If this option takes a dict or list of dicts, you can define the structure here.
+
+Sample Specification
+--------------------
+
+.. code-block:: yaml
+
+ # roles/myapp/meta/main.yml
+ ---
+ argument_specs:
+   # roles/myapp/tasks/main.yml entry point
+   main:
+     short_description: The main entry point for the myapp role.
+     options:
+       myapp_int:
+         type: "int"
+         required: false
+         default: 42
+         description: "The integer value, defaulting to 42."
+
+       myapp_str:
+         type: "str"
+         required: true
+         description: "The string value"
+
+   # roles/maypp/tasks/alternate.yml entry point
+   alternate:
+     short_description: The alternate entry point for the myapp role.
+     options:
+       myapp_int:
+         type: "int"
+         required: false
+         default: 1024
+         description: "The integer value, defaulting to 1024."
+
 .. _run_role_twice:
 
 Running a role multiple times in one playbook

lib/ansible/modules/validate_argument_spec.py

@@ -0,0 +1,63 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+
+# Copyright 2021 Red Hat
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+
+DOCUMENTATION = r'''
+---
+module: validate_argument_spec
+short_description: Validate role argument specs.
+description:
+     - This module validates role arguments with a defined argument specification.
+version_added: "2.11"
+options:
+  argument_spec:
+    description:
+        - A dictionary like AnsibleModule argument_spec
+    required: true
+  provided_arguments:
+    description:
+        - A dictionary of the arguments that will be validated according to argument_spec
+author:
+    - Ansible Core Team
+'''
+
+EXAMPLES = r'''
+'''
+
+RETURN = r'''
+argument_errors:
+  description: A list of arg validation errors.
+  returned: failure
+  type: list
+  elements: str
+  sample:
+    - "error message 1"
+    - "error message 2"
+
+argument_spec_data:
+  description: A dict of the data from the 'argument_spec' arg.
+  returned: failure
+  type: dict
+  sample:
+    some_arg:
+      type: "str"
+    some_other_arg:
+      type: "int"
+      required: true
+
+validate_args_context:
+  description: A dict of info about where validate_args_spec was used
+  type: dict
+  returned: always
+  sample:
+    name: my_role
+    type: role
+    path: /home/user/roles/my_role/
+    argument_spec_name: main
+'''

lib/ansible/playbook/role/__init__.py

@@ -255,6 +255,9 @@ def _load_role_data(self, role_include, parent_role=None):
                 self.collections.append(default_append_collection)
 
         task_data = self._load_role_yaml('tasks', main=self._from_files.get('tasks'))
+
+        task_data = self._prepend_validation_task(task_data)
+
         if task_data:
             try:
                 self._task_blocks = load_list_of_blocks(task_data, play=self._play, role=self, loader=self._loader, variable_manager=self._variable_manager)
@@ -271,6 +274,67 @@ def _load_role_data(self, role_include, parent_role=None):
                 raise AnsibleParserError("The handlers/main.yml file for role '%s' must contain a list of tasks" % self._role_name,
                                          obj=handler_data, orig_exc=e)
 
+    def _prepend_validation_task(self, task_data):
+        '''Insert a role validation task if we have a role argument spec.
+
+        This method will prepend a validation task to the front of the role task
+        list to perform argument spec validation before any other tasks, if an arg spec
+        exists for the entry point. Entry point defaults to `main`.
+
+        :param task_data: List of tasks loaded from the role.
+
+        :returns: The (possibly modified) task list.
+        '''
+        if self._metadata.argument_specs:
+            if self._dependencies:
+                display.warning("Dependent roles will run before roles with argument specs even if validation fails.")
+
+            # Determine the role entry point so we can retrieve the correct argument spec.
+            # This comes from the `tasks_from` value to include_role or import_role.
+            entrypoint = self._from_files.get('tasks', 'main')
+            entrypoint_arg_spec = self._metadata.argument_specs.get(entrypoint)
+
+            if entrypoint_arg_spec:
+                validation_task = self._create_validation_task(entrypoint_arg_spec, entrypoint)
+
+                # Prepend our validate_argument_spec action to happen before any tasks provided by the role.
+                # 'any tasks' can and does include 0 or None tasks, in which cases we create a list of tasks and add our
+                # validate_argument_spec task
+                if not task_data:
+                    task_data = []
+                task_data.insert(0, validation_task)
+        return task_data
+
+    def _create_validation_task(self, argument_spec, entrypoint_name):
+        '''Create a new task data structure that uses the validate_argument_spec action plugin.
+
+        :param argument_spec: The arg spec definition for a particular role entry point.
+            This will be the entire arg spec for the entry point as read from the input file.
+        :param entrypoint_name: The name of the role entry point associated with the
+            supplied `argument_spec`.
+        '''
+
+        # If the arg spec provides a short description, use it to flesh out the validation task name
+        task_name = "Validating arguments against arg spec '%s'" % entrypoint_name
+        if 'short_description' in argument_spec:
+            task_name = task_name + ' - ' + argument_spec['short_description']
+
+        return {
+            'action': {
+                'module': 'ansible.builtin.validate_argument_spec',
+                # Pass only the 'options' portion of the arg spec to the module.
+                'argument_spec': argument_spec.get('options', {}),
+                'provided_arguments': self._role_params,
+                'validate_args_context': {
+                    'type': 'role',
+                    'name': self._role_name,
+                    'argument_spec_name': entrypoint_name,
+                    'path': self._role_path
+                },
+            },
+            'name': task_name,
+        }
+
     def _load_role_yaml(self, subdir, main=None, allow_dir=False):
         '''
         Find and load role YAML files and return data found.

lib/ansible/plugins/action/validate_argument_spec.py

@@ -0,0 +1,97 @@
+# Copyright 2021 Red Hat
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+from ansible.errors import AnsibleError
+from ansible.plugins.action import ActionBase
+from ansible.module_utils.six import iteritems, string_types
+from ansible.module_utils.common.arg_spec import ArgumentSpecValidator
+
+
+class ActionModule(ActionBase):
+    ''' Validate an arg spec'''
+
+    TRANSFERS_FILES = False
+
+    def get_args_from_task_vars(self, argument_spec, task_vars):
+        '''
+        Get any arguments that may come from `task_vars`.
+
+        Expand templated variables so we can validate the actual values.
+
+        :param argument_spec: A dict of the argument spec.
+        :param task_vars: A dict of task variables.
+
+        :returns: A dict of values that can be validated against the arg spec.
+        '''
+        args = {}
+
+        for argument_name, argument_attrs in iteritems(argument_spec):
+            if argument_name in task_vars:
+                if isinstance(task_vars[argument_name], string_types):
+                    value = self._templar.do_template(task_vars[argument_name])
+                    if value:
+                        args[argument_name] = value
+                else:
+                    args[argument_name] = task_vars[argument_name]
+        return args
+
+    def run(self, tmp=None, task_vars=None):
+        '''
+        Validate an argument specification against a provided set of data.
+
+        The `validate_argument_spec` module expects to receive the arguments:
+            - argument_spec: A dict whose keys are the valid argument names, and
+                  whose values are dicts of the argument attributes (type, etc).
+            - provided_arguments: A dict whose keys are the argument names, and
+                  whose values are the argument value.
+
+        :param tmp: Deprecated. Do not use.
+        :param task_vars: A dict of task variables.
+        :return: An action result dict, including a 'argument_errors' key with a
+            list of validation errors found.
+        '''
+        if task_vars is None:
+            task_vars = dict()
+
+        result = super(ActionModule, self).run(tmp, task_vars)
+        del tmp  # tmp no longer has any effect
+
+        # This action can be called from anywhere, so pass in some info about what it is
+        # validating args for so the error results make some sense
+        result['validate_args_context'] = self._task.args.get('validate_args_context', {})
+
+        if 'argument_spec' not in self._task.args:
+            raise AnsibleError('"argument_spec" arg is required in args: %s' % self._task.args)
+
+        # Get the task var called argument_spec. This will contain the arg spec
+        # data dict (for the proper entry point for a role).
+        argument_spec_data = self._task.args.get('argument_spec')
+
+        # the values that were passed in and will be checked against argument_spec
+        provided_arguments = self._task.args.get('provided_arguments', {})
+
+        if not isinstance(argument_spec_data, dict):
+            raise AnsibleError('Incorrect type for argument_spec, expected dict and got %s' % type(argument_spec_data))
+
+        if not isinstance(provided_arguments, dict):
+            raise AnsibleError('Incorrect type for provided_arguments, expected dict and got %s' % type(provided_arguments))
+
+        args_from_vars = self.get_args_from_task_vars(argument_spec_data, task_vars)
+        provided_arguments.update(args_from_vars)
+
+        validator = ArgumentSpecValidator(argument_spec_data, provided_arguments)
+
+        if not validator.validate():
+            result['failed'] = True
+            result['msg'] = 'Validation of arguments failed:\n%s' % '\n'.join(validator.error_messages)
+            result['argument_spec_data'] = argument_spec_data
+            result['argument_errors'] = validator.error_messages
+            return result
+
+        result['changed'] = False
+        result['msg'] = 'The arg spec validation passed'
+
+        return result

test/integration/targets/roles_arg_spec/aliases

@@ -0,0 +1 @@
+shippable/posix/group5

test/integration/targets/roles_arg_spec/roles/a/meta/main.yml

@@ -0,0 +1,17 @@
+argument_specs:
+    main:
+        short_description: Main entry point for role A.
+        options:
+            a_str:
+                type: "str"
+                required: true
+
+    alternate:
+        short_description: Alternate entry point for role A.
+        options:
+            a_int:
+                type: "int"
+                required: true
+
+    no_spec_entrypoint:
+        short_description: An entry point with no spec

test/integration/targets/roles_arg_spec/roles/a/tasks/alternate.yml

@@ -0,0 +1,3 @@
+---
+- debug:
+    msg: "Role A (alternate) with {{ a_int }}"

test/integration/targets/roles_arg_spec/roles/a/tasks/main.yml

@@ -0,0 +1,3 @@
+---
+- debug:
+    msg: "Role A with {{ a_str }}"

test/integration/targets/roles_arg_spec/roles/a/tasks/no_spec_entrypoint.yml

@@ -0,0 +1,3 @@
+---
+- debug:
+    msg: "Role A no_spec_entrypoint"