Release 1.1

Conflicts:
	docs/conf.py

.gitignore

@@ -13,3 +13,5 @@ proxy.log
 MANIFEST
 coverage_report
 test.log
+examples/*/local
+examples/*/metadata

CHANGELOG.md

@@ -1,4 +1,24 @@
-# Splunk Python SDK Changelog
+# Splunk SDK for Python Changelog
+
+## Version 1.1
+
+### New features and APIs
+
+* Added support for building modular input scripts in Python using the Splunk
+  SDK for Python.
+
+### Minor additions
+
+* Added 2 modular input examples: `Github forks` and `random numbers`.
+
+* Added a `dist` command to `setup.py`. Running `setup.py dist` will generate
+  2 `.spl` files for the new modular input example apps.
+
+* `client.py` in  the `splunklib` module will now restart Splunk via an HTTP
+  post request instead of an HTTP get request.
+
+* `.gitignore` has been updated to ignore `local` and `metadata` subdirectories
+for any examples.
 
 ## Version 1.0
 

README.md

@@ -1,8 +1,8 @@
-# The Splunk Python Software Development Kit
+# The Splunk Software Development Kit for Python
 
-#### Version 1.0
+#### Version 1.1
 
-The Splunk Python Software Development Kit (SDK)  contains library code and 
+The Splunk Software Development Kit (SDK) for Python contains library code and 
 examples designed to enable developers to build applications using Splunk.
 
 Splunk is a search engine and analytic environment that uses a distributed
@@ -19,9 +19,9 @@ technology used by the Splunk product to build exciting new applications that
 are enabled by Splunk's unique capabilities.
 
 
-## Getting started with the Splunk Python SDK
+## Getting started with the Splunk SDK for Python
 
-The Splunk Python SDK contains library code and examples that show how to 
+The Splunk SDK for Python contains library code and examples that show how to 
 programmatically interact with Splunk for a variety of scenarios including 
 searching, saved searches, data inputs, and many more, along with building 
 complete applications. 
@@ -31,11 +31,11 @@ in-depth information be sure to visit the
 [Splunk Developer Portal](http://dev.splunk.com/view/SP-CAAAEBB). 
 ### Requirements
 
-Here's what you need to get going with the Splunk Python SDK.
+Here's what you need to get going with the Splunk SDK for Python.
 
 #### Python
 
-The Splunk Python SDK requires Python 2.6+. 
+The Splunk SDK for Python requires Python 2.6+. 
 
 #### Splunk
 
@@ -44,15 +44,15 @@ If you haven't already installed Splunk, download it
 Splunk and system requirements, see 
 [Installing & Running Splunk](http://dev.splunk.com/view/SP-CAAADRV). 
 
-#### Splunk Python SDK
-Get the Splunk Python SDK from [GitHub](https://github.com/) and clone the 
-resources to your computer. For example, use the following command: 
+#### Splunk SDK for Python
+Get the Splunk SDK for Python; [download the SDK as a ZIP](http://dev.splunk.com/view/SP-CAAAEBB) 
+and extract the files. Or, if you want to contribute to the SDK, clone the 
+repository from [GitHub](https://github.com/splunk/splunk-sdk-python).
 
-    git clone https://github.com/splunk/splunk-sdk-python.git
 
 ### Installing the SDK
 
-You can install the Splunk Python SDK libraries by using `easy_install` or `pip`:
+You can install the Splunk SDK for Python libraries by using `easy_install` or `pip`:
 
     [sudo] easy_install splunk-sdk
 
@@ -151,7 +151,7 @@ To get help for an example, use the `--help` argument with an example:
 
 #### Unit tests
 
-The Splunk Python SDK contains a collection of unit tests. To run them, open a 
+The Splunk SDK for Python contains a collection of unit tests. To run them, open a 
 command prompt in the **/splunk-sdk-python** directory and enter:
 
     python setup.py test
@@ -220,10 +220,10 @@ If you need to know more:
   [Splunk Developer Portal](http://dev.splunk.com).
 
 * For conceptual and how-to documentation, see the 
-  [Overview of the Splunk Python SDK](http://dev.splunk.com/view/SP-CAAAEBB).
+  [Overview of the Splunk SDK for Python](http://dev.splunk.com/view/SP-CAAAEBB).
 
 * For API reference documentation, see the 
-  [Splunk Python SDK Reference](http://docs.splunk.com/Documentation/PythonSDK).
+  [Splunk SDK for Python Reference](http://docs.splunk.com/Documentation/PythonSDK).
 
 * For more about the Splunk REST API, see the 
   [REST API Reference](http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI).
@@ -281,7 +281,7 @@ If you would like to contribute to the SDK, go here for more information:
 
 1. You will be granted support if you or your company are already covered 
    under an existing maintenance/support agreement. Send an email to 
-   _[email protected]_ and include "Splunk Python SDK" in the subject line. 
+   _[email protected]_ and include "Splunk SDK for Python" in the subject line. 
 
 2. If you are not covered under an existing maintenance/support agreement, you 
    can find help through the broader community at:

docs/conf.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Splunk Python SDK documentation build configuration file, created by
+# Splunk SDK for Python documentation build configuration file, created by
 # sphinx-quickstart on Fri Apr 13 12:28:15 2012.
 #
 # This file is execfile()d with the current directory set to its containing dir.
@@ -110,7 +110,7 @@
 html_title = "Splunk SDK for Python API Reference"
 
 # A shorter title for the navigation bar.  Default is the same as html_title.
-#html_short_title = "Splunk Python SDK Reference"
+#html_short_title = "Splunk SDK for Python Reference"
 
 # The name of an image file (relative to this directory) to place at the top
 # of the sidebar.

docs/index.rst

@@ -1,5 +1,5 @@
-Welcome to the API reference for the Splunk Python SDK, which describes the modules that are included in the Splunk Python SDK.
-For more information about the SDK, see the `Splunk Developer Portal <http://dev.splunk.com/view/python-sdk/SP-CAAAEBB>`_.
+Welcome to the API reference for the Splunk SDK for Python, which describes the modules that are included in the SDK.
+For more information, see the `Splunk Developer Portal <http://dev.splunk.com/view/python-sdk/SP-CAAAEBB>`_.
 
 .. toctree::
    :maxdepth: 2
@@ -122,3 +122,20 @@ For more information about the SDK, see the `Splunk Developer Portal <http://dev
     :class:`~splunklib.results.ResultsReader` class 
 
     :class:`~splunklib.results.Message` class 
+
+:doc:`modularinput`
+--------------
+
+    :class:`~splunklib.modularinput.Argument` class 
+
+    :class:`~splunklib.modularinput.Event` class 
+
+    :class:`~splunklib.modularinput.EventWriter` class 
+
+    :class:`~splunklib.modularinput.InputDefinition` class 
+
+    :class:`~splunklib.modularinput.Scheme` class 
+
+    :class:`~splunklib.modularinput.Script` class 
+
+    :class:`~splunklib.modularinput.ValidationDefinition` class 

docs/modularinput.rst

@@ -0,0 +1,25 @@
+splunklib.modularinput
+--------------
+
+.. automodule:: splunklib.modularinput
+
+.. autoclass:: Argument
+    :members: 
+
+.. autoclass:: Event
+    :members: 
+
+.. autoclass:: EventWriter
+    :members: 
+
+.. autoclass:: InputDefinition
+    :members: 
+
+.. autoclass:: Scheme
+    :members: 
+
+.. autoclass:: Script
+    :members: 
+
+.. autoclass:: ValidationDefinition
+    :members: 

examples/async/async.py

@@ -15,7 +15,7 @@
 # under the License.
 
 # A sample that demonstrates a custom HTTP handler for the Splunk service,
-# as well as showing how you could use the Splunk Python SDK with coroutine
+# as well as showing how you could use the Splunk SDK for Python with coroutine
 # based systems like Eventlet.
 
 #### Main Code

examples/export/export.py

@@ -28,7 +28,7 @@
 # splunk support files
 from splunklib.binding import connect
 try:
-    from utils import error, parse
+    from utils import parse
 except ImportError:
     raise Exception("Add the SDK repository to your PYTHONPATH to run the examples "
                     "(e.g., export PYTHONPATH=~/splunk-sdk-python.")
@@ -329,7 +329,9 @@ def main():
 
     if path.exists(options.kwargs['output']):
         if options.kwargs['recover'] == False:
-            error("Export file %s exists, and recover option nor specified" % options.kwargs['output'], exitcode=1)
+            print "Export file %s exists, and recover option nor specified" % \
+                  options.kwargs['output']
+            sys.exit(1)
         else:
             options.kwargs['end'] = recover(options)
             options.kwargs['fixtail'] = True

examples/github_forks/README/inputs.conf.spec

@@ -0,0 +1,5 @@
+[github_forks://<name>]
+*Streams events giving the number of forks of a GitHub repository
+
+owner = <value>
+repo_name = <value>

examples/github_forks/default/app.conf

@@ -0,0 +1,11 @@
+[install]
+is_configured = 0
+
+[ui]
+is_visible = 1
+label = Github Repository Forks
+
+[launcher]
+author=Splunk
+description=Streams events giving the number of forks of a GitHub repository
+version = 1.0

examples/github_forks/github_forks.py

@@ -0,0 +1,144 @@
+#!/usr/bin/env python
+#
+# Copyright 2013 Splunk, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"): you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+import sys, urllib2, json
+
+from splunklib.modularinput import *
+
+try:
+    import xml.etree.cElementTree as ET
+except ImportError:
+    import xml.etree.ElementTree as ET
+
+class MyScript(Script):
+    """All modular inputs should inherit from the abstract base class Script
+    from splunklib.modularinput.script.
+    They must override the get_scheme and stream_events functions, and,
+    if the scheme returned by get_scheme has Scheme.use_external_validation
+    set to True, the validate_input function.
+    """
+    def get_scheme(self):
+        """When Splunk starts, it looks for all the modular inputs defined by
+        its configuration, and tries to run them with the argument --scheme.
+        Splunkd expects the modular inputs to print a description of the
+        input in XML on stdout. The modular input framework takes care of all
+        the details of formatting XML and printing it. The user need only
+        override get_scheme and return a new Scheme object.
+
+        :return: scheme, a Scheme object
+        """
+        # Splunk will display "Github Repository Forks" to users for this input
+        scheme = Scheme("Github Repository Forks")
+
+        scheme.description = "Streams events giving the number of forks of a GitHub repository."
+        # If you set external validation to True, without overriding validate_input,
+        # the script will accept anything as valid. Generally you only need external
+        # validation if there are relationships you must maintain among the
+        # parameters, such as requiring min to be less than max in this example,
+        # or you need to check that some resource is reachable or valid.
+        # Otherwise, Splunk lets you specify a validation string for each argument
+        # and will run validation internally using that string.
+        scheme.use_external_validation = True
+        scheme.use_single_instance = True
+
+        owner_argument = Argument("owner")
+        owner_argument.data_type = Argument.data_type_string
+        owner_argument.description = "Github user or organization that created the repository."
+        owner_argument.required_on_create = True
+        # If you are not using external validation, you would add something like:
+        #
+        # scheme.validation = "owner==splunk"
+        scheme.add_argument(owner_argument)
+
+        repo_name_argument = Argument("repo_name")
+        repo_name_argument.data_type = Argument.data_type_string
+        repo_name_argument.description = "Name of the Github repository."
+        repo_name_argument.required_on_create = True
+        scheme.add_argument(repo_name_argument)
+
+        return scheme
+
+    def validate_input(self, validation_definition):
+        """In this example we are using external validation to verify that the Github
+        repository exists. If validate_input does not raise an Exception, the input
+        is assumed to be valid. Otherwise it prints the exception as an error message
+        when telling splunkd that the configuration is invalid.
+
+        When using external validation, after splunkd calls the modular input with
+        --scheme to get a scheme, it calls it again with --validate-arguments for
+        each instance of the modular input in its configuration files, feeding XML
+        on stdin to the modular input to do validation. It is called the same way
+        whenever a modular input's configuration is edited.
+
+        :param validation_definition: a ValidationDefinition object
+        """
+        # Get the values of the parameters, and construct a URL for the Github API
+        owner = validation_definition.parameters["owner"]
+        repo_name = validation_definition.parameters["repo_name"]
+        repo_url = "https://api.github.com/repos/%s/%s" % (owner, repo_name)
+
+        # Read the response from the Github API, then parse the JSON data into an object
+        response = urllib2.urlopen(repo_url).read()
+        jsondata = json.loads(response)
+
+        # If there is only 1 field in the jsondata object,some kind or error occurred
+        # with the Github API.
+        # Typically, this will happen with an invalid repository.
+        if len(jsondata) == 1:
+            raise ValueError("The Github repository was not found.")
+
+        # If the API response seems normal, validate the fork count
+        # If there's something wrong with getting fork_count, raise a ValueError
+        try:
+            fork_count = int(jsondata["forks_count"])
+        except ValueError as ve:
+            raise ValueError("Invalid fork count: %s", ve.message)
+
+    def stream_events(self, inputs, ew):
+        """This function handles all the action: splunk calls this modular input
+        without arguments, streams XML describing the inputs to stdin, and waits
+        for XML on stout describing events.
+
+        If you set use_single_instance to True on the scheme in get_scheme, it
+        will pass all the instances of this input to a single instance of this
+        script.
+
+        :param inputs: an InputDefinition object
+        :param ew: an EventWriter object
+        """
+        # Go through each input for this modular input
+        for input_name, input_item in inputs.inputs.iteritems():
+            # Get fields from the InputDefinition object
+            owner = input_item["owner"]
+            repo_name = input_item["repo_name"]
+
+            # Get the fork count from the Github API
+            repo_url = "https://api.github.com/repos/%s/%s" % (owner, repo_name)
+            response = urllib2.urlopen(repo_url).read()
+            jsondata = json.loads(response)
+            fork_count = jsondata["forks_count"]
+
+            # Create an Event object, and set its fields
+            event = Event()
+            event.stanza = input_name
+            event.data = 'owner="%s" repository="%s" fork_count=%s' % \
+                         (owner.replace('"', '\\"'), repo_name.replace('"', '\\"'), fork_count)
+
+            # Tell the EventWriter to write this event
+            ew.write_event(event)
+
+if __name__ == "__main__":
+    sys.exit(MyScript().run(sys.argv))

examples/random_numbers/README/inputs.conf.spec

@@ -0,0 +1,5 @@
+[random_numbers://<name>]
+*Generates events containing a random floating point number.
+
+min = <value>
+max = <value>