Merge pull request elastic#4230 from palecur/offline-plugins

Adds new material to Managing Plugins

docs/asciidoc/static/offline-plugins.asciidoc

@@ -0,0 +1,58 @@
+[[offline-plugins]]
+=== Offline Plugin Management
+
+The Logstash <<working-with-plugins,plugin manager>> was introduced in the 1.5 release. This section discusses setting up
+local repositories of plugins for use on systems without access to the Internet.
+
+The procedures in this section require a staging machine running Logstash that has access to a public or private Rubygems 
+server. This staging machine downloads and packages the files used for offline installation.
+
+See the <<private-rubygem,Private Gem Repositories>> section for information on setting up your own private 
+Rubygems server.
+
+[float]
+=== Building the Offline Package
+
+Working with offline plugins requires you to create an _offline package_, which is a compressed file that contains all of 
+the plugins your offline Logstash installation requires, along with the dependencies for those plugins.
+
+. Create the offline package with the `bin/plugin pack` subcommand.
++
+When you run the `bin/plugin pack` subcommand, Logstash creates a compressed bundle that contains all of the currently
+installed plugins and the dependencies for those plugins. By default, the compressed bundle is a GZipped TAR file when you 
+run the `bin/plugin pack` subcommand on a UNIX machine. By default, when you run the `bin/plugin pack` subcommand on a 
+Windows machine, the compressed bundle is a ZIP file. See <<managing-packs,Managing Plugin Packs>> for details on changing 
+these default behaviors.
++
+NOTE: Downloading all dependencies for the specified plugins may take some time, depending on the plugins listed.
+
+. Move the compressed bundle to the offline machines that are the source for offline plugin installation, then use the
+`bin/plugin unpack` subcommand to make the packaged plugins available.
+
+[float]
+=== Install or Update a local plugin
+
+To install or update a local plugin, use the `--local` option with the install and update commands, as in the following 
+examples:
+
+.Installing a local plugin
+============
+`bin/plugin install --local logstash-input-jdbc`
+============
+
+.Updating a local plugin
+============
+`bin/plugin update --local logstash-input-jdbc`
+============
+
+[float]
+[[managing-packs]]
+=== Managing Plugin Packs
+
+The `pack` and `unpack` subcommands for `bin/plugin` take the following options:
+
+[horizontal]
+`--tgz`:: Generate the offline package as a GZipped TAR file. The default behavior on UNIX systems.
+`--zip`:: Generate the offline package as a ZIP file. The default behavior on Windows systems.
+`[packname] --override`:: Generates a new offline package that overwrites an existing offline with the specified name.
+`[packname] --[no-]clean`: Deletes offline packages matching the specified name.

docs/asciidoc/static/plugin-manager.asciidoc

@@ -1,15 +1,17 @@
 [[working-with-plugins]]
 == Working with plugins
 
-Logstash has a rich collection of input, filter, codec and output plugins. Plugins are available as self-contained packages called gems and hosted on RubyGems.org. The plugin manager accesed via `bin/plugin` script is used to manage the lifecycle of plugins in your Logstash deployment. You can install, uninstall and upgrade plugins using these Command Line Interface (CLI) described below.
-
-NOTE: Some sections here are for advanced users
+Logstash has a rich collection of input, filter, codec and output plugins. Plugins are available as self-contained 
+packages called gems and hosted on RubyGems.org. The plugin manager accesed via `bin/plugin` script is used to manage the 
+lifecycle of plugins in your Logstash deployment. You can install, uninstall and upgrade plugins using these Command Line 
+Interface (CLI) described below.
 
 [float]
 [[listing-plugins]]
 === Listing plugins
 
-Logstash release packages bundle common plugins so you can use them out of the box. To list the plugins currently available in your deployment:
+Logstash release packages bundle common plugins so you can use them out of the box. To list the plugins currently 
+available in your deployment:
 
 [source,shell]
 ----------------------------------
@@ -30,7 +32,9 @@ bin/plugin list --group output <4>
 [[installing-plugins]]
 === Adding plugins to your deployment
 
-The most common situation when dealing with plugin installation is when you have access to internet. Using this method, you will be able to retrieve plugins hosted on the public repository (RubyGems.org) and install on top of your Logstash installation.
+The most common situation when dealing with plugin installation is when you have access to internet. Using this method, 
+you will be able to retrieve plugins hosted on the public repository (RubyGems.org) and install on top of your Logstash 
+installation.
 
 [source,shell]
 ----------------------------------
@@ -43,7 +47,8 @@ Once the plugin is successfully installed, you can start using it in your config
 [float]
 ==== Advanced: Adding a locally built plugin
 
-In some cases, you want to install plugins which have not yet been released and not hosted on RubyGems.org. Logstash provides you the option to install a locally built plugin which is packaged as a ruby gem. Using a file location:
+In some cases, you want to install plugins which have not yet been released and not hosted on RubyGems.org. Logstash 
+provides you the option to install a locally built plugin which is packaged as a ruby gem. Using a file location:
 
 [source,shell]
 ----------------------------------
@@ -54,7 +59,8 @@ bin/plugin install /path/to/logstash-output-kafka-1.0.0.gem
 [float]
 ==== Advanced: Using `--pluginpath`
 
-Using the `--pluginpath` flag, you can load a plugin source code located on your file system. Typically this is used by developers who are iterating on a custom plugin and want to test it before creating a ruby gem.
+Using the `--pluginpath` flag, you can load a plugin source code located on your file system. Typically this is used by 
+developers who are iterating on a custom plugin and want to test it before creating a ruby gem.
 
 [source,shell]
 ----------------------------------
@@ -65,7 +71,8 @@ bin/logstash --pluginpath /opt/shared/lib/logstash/input/my-custom-plugin-code.r
 [float]
 === Updating plugins
 
-Plugins have their own release cycle and are often released independent of Logstash’s core release cycle. Using the update sub-command you can get the latest or update to a particular version of the plugin.
+Plugins have their own release cycle and are often released independent of Logstash’s core release cycle. Using the update 
+subcommand you can get the latest or update to a particular version of the plugin.
 
 [source,shell]
 ----------------------------------
@@ -91,7 +98,9 @@ bin/plugin uninstall logstash-output-kafka
 [float]
 === Proxy Support
 
-The previous sections relied on Logstash being able to communicate with RubyGems.org. In certain environments, Forwarding Proxy is used to handle HTTP requests. Logstash Plugins can be installed and updated through a Proxy by setting the `HTTP_PROXY` environment variable:
+The previous sections relied on Logstash being able to communicate with RubyGems.org. In certain environments, Forwarding 
+Proxy is used to handle HTTP requests. Logstash Plugins can be installed and updated through a Proxy by setting the 
+`HTTP_PROXY` environment variable:
 
 [source,shell]
 ----------------------------------
@@ -101,3 +110,7 @@ bin/plugin install logstash-output-kafka
 ----------------------------------
 
 Once set, plugin commands install, update can be used through this proxy.
+
+include::offline-plugins.asciidoc[]
+
+include::private-gem-repo.asciidoc[]

docs/asciidoc/static/private-gem-repo.asciidoc

@@ -0,0 +1,53 @@
+[[private-rubygem]]
+=== Private Gem Repositories
+
+The Logstash plugin manager connects to a Ruby gems repository to install and update Logstash plugins. By default, this
+repository is http://rubygems.org.
+
+Some use cases are unable to use the default repository, as in the following examples:
+
+* A firewall blocks access to the default repository.
+* You are developing your own plugins locally.
+* Airgap requirements on the local system.
+
+When you use a custom gem repository, be sure to make plugin dependencies available.
+
+Several open source projects enable you to run your own plugin server, among them:
+
+* https://github.com/geminabox/geminabox[Geminabox]
+* https://github.com/PierreRambaud/gemirro[Gemirro]
+* https://gemfury.com/[Gemfury]
+* http://www.jfrog.com/open-source/[Artifactory]
+
+==== Editing the Gemfile
+
+The gemfile is a configuration file that specifies information required for plugin management. Each gem file has a
+`source` line that specifies a location for plugin content.
+
+By default, the gemfile's `source` line reads:
+
+[source,shell]
+----------
+# This is a Logstash generated Gemfile.
+# If you modify this file manually all comments and formatting will be lost.
+
+source "https://rubygems.org"
+----------
+
+To change the source, edit the `source` line to contain your preferred source, as in the following example:
+
+[source,shell]
+----------
+# This is a Logstash generated Gemfile.
+# If you modify this file manually all comments and formatting will be lost.
+
+source "https://my.private.repository"
+----------
+
+After saving the new version of the gemfile, use <<working-with-plugins,plugin management commands>> normally.
+
+The following links contain further material on setting up some commonly used repositories:
+
+* https://github.com/geminabox/geminabox/blob/master/README.markdown[Geminabox]
+* https://www.jfrog.com/confluence/display/RTF/RubyGems+Repositories[Artifactory]
+* Running a http://guides.rubygems.org/run-your-own-gem-server/[rubygems mirror]