Skip to content

Commit

Permalink
Convert instructions for Java plugins to asciidoc
Browse files Browse the repository at this point in the history
  • Loading branch information
karenzone committed Mar 11, 2019
1 parent 80c966b commit 31f7314
Show file tree
Hide file tree
Showing 7 changed files with 1,056 additions and 1 deletion.
5 changes: 4 additions & 1 deletion docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -247,12 +247,15 @@ include::static/maintainer-guide.asciidoc[]

// A space is necessary here ^^^


// Submitting a Plugin

:edit_url: https://github.com/elastic/logstash/edit/{branch}/docs/static/submitting-a-plugin.asciidoc
include::static/submitting-a-plugin.asciidoc[]

// Contributing to Logstash - JAVA EDITION

:edit_url: https://github.com/elastic/logstash/edit/{branch}/docs/static/contributing-java-plugin.asciidoc
include::static/contributing-java-plugin.asciidoc[]

// Glossary of Terms

Expand Down
54 changes: 54 additions & 0 deletions docs/static/contributing-java-plugin.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
[[contributing-java-plugin]]
== Contributing a Java Plugin

experimental[]

Now you can write your own Java plugin for use with {ls}.
We have provided instructions and GitHub examples to give
you a head start.

Native support for Java plugins in {ls} consists of several components
including:

* Extensions to the Java execution engine to support running Java plugins in
Logstash pipelines
* APIs for developing Java plugins.
The APIs are in the `co.elastic.logstash.api.v0` package.
A Java plugin might break if it references classes or specific concrete
implementations of API interfaces outside that package. The implementation of
classes outside of the API package may change at any time.
* Coming in a future release: Tooling to automate the packaging and deployment of
Java plugins in Logstash. (Currently, this process is manual.)

[float]
=== Process overview
Here are the steps:

. Choose the type of plugin you want to create: input, filter, or output.
. Set up your environment.
. Code the plugin.
. Package and deploy the plugin.
. Run Logstash with your new plugin.

[float]
==== Let's get started
Here are the example repos:

* https://github.com/logstash-plugins/logstash-input-java_input_example[Input plugin example]
//* https://github.com/logstash-plugins/logstash-codec-java_codec_example[Codec plugin example]
* https://github.com/logstash-plugins/logstash-filter-java_filter_example[Filter plugin example]
* https://github.com/logstash-plugins/logstash-output-java_output_example[Output plugin example]

Here are the instructions:

* <<java-input-plugin>>
//* <<java-codec-plugin>>
* <<java-filter-plugin>>
* <<java-output-plugin>>


include::java-input.asciidoc[]
//include::java-codec.asciidoc[]
include::java-filter.asciidoc[]
include::java-output.asciidoc[]

133 changes: 133 additions & 0 deletions docs/static/include/javapluginpkg.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
[float]
=== Package and deploy

Java plugins are packaged as Ruby gems for dependency management and
interoperability with Ruby plugins.

NOTE: One of the goals for Java plugin support is to eliminate the need for any
knowledge of Ruby or its toolchain for Java plugin development. Future phases of
the Java plugin project will automate the packaging of Java plugins as Ruby gems
so no direct knowledge of or interaction with Ruby will be required. In the
current phase, Java plugins must still be manually packaged as Ruby gems
and installed with the `logstash-plugin` utility.

[float]
==== Compile to JAR file

The Java plugin should be compiled and assembled into a fat jar with the
`vendor` task in the Gradle build file. This will package all Java dependencies
into a single jar and write it to the correct folder for later packaging into a
Ruby gem.

[float]
==== Manually package as Ruby gem

Several Ruby source files are required to package the jar file as a
Ruby gem. These Ruby files are used only at Logstash startup time to identify
the Java plugin and are not used during runtime event processing.

NOTE: These Ruby source files will be automatically generated in a future release.

**+logstash-{plugintype}-<{plugintype}-name>.gemspec+**

[source,txt]
[subs="attributes"]
-----
Gem::Specification.new do |s|
s.name = 'logstash-{plugintype}-java_{plugintype}_example'
s.version = PLUGIN_VERSION
s.licenses = ['Apache-2.0']
s.summary = "Example {plugintype} using Java plugin API"
s.description = ""
s.authors = ['Elasticsearch']
s.email = '[email protected]'
s.homepage = "http://www.elastic.co/guide/en/logstash/current/index.html"
s.require_paths = ['lib', 'vendor/jar-dependencies']
# Files
s.files = Dir["lib/**/*","spec/**/*","*.gemspec","*.md","CONTRIBUTORS","Gemfile","LICENSE","NOTICE.TXT", "vendor/jar-dependencies/**/*.jar", "vendor/jar-dependencies/**/*.rb", "VERSION", "docs/**/*"]
# Special flag to let us know this is actually a logstash plugin
s.metadata = { 'logstash_plugin' => 'true', 'logstash_group' => '{plugintype}'}
# Gem dependencies
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
s.add_runtime_dependency 'jar-dependencies'
s.add_development_dependency 'logstash-devutils'
end
-----

You can use this file with the following modifications:

* `s.name` must follow the +logstash-pass:attributes[{plugintype}]-<{plugintype}-name>+ pattern
* `s.version` must match the `project.version` specified in the `build.gradle` file.

**+lib/logstash/{plugintype}s/<{plugintype}-name>.rb+**

[source,ruby]
[subs="attributes"]
-----
# encoding: utf-8
require "logstash/{plugintype}s/base"
require "logstash/namespace"
require "logstash-{plugintype}-java_{plugintype}_example_jars"
require "java"
class LogStash::{plugintype}s::Java{plugintypecap}Example < LogStash::{pluginclass}::Base
config_name "java_{plugintype}_example"
def self.javaClass() org.logstash.javaapi.Java{plugintypecap}Example.java_class; end
end
-----

Modify these items in the file above:

* Change the name to correspond with the {plugintype} name.
* Change +require "logstash-{plugintype}-java_{plugintype}_example_jars"+ to reference the appropriate "jars" file
as described below.
* Change +class LogStash::{pluginclass}::Java{plugintypecap}Example < LogStash::{pluginclass}::Base+ to provide a unique and
descriptive Ruby class name.
* Change +config_name "java_{plugintype}_example"+ to match the name of the plugin as specified in the `name` property of
the `@LogstashPlugin` annotation.
* Change +def self.javaClass() org.logstash.javaapi.Java{plugintypecap}Example.java_class; end+ to return the
class of the Java {plugintype}.

**+lib/logstash-{plugintype}-<{plugintype}-name>_jars.rb+**

[source,txt]
[subs="attributes"]
-----
require 'jar_dependencies'
require_jar('org.logstash.javaapi', 'logstash-{plugintype}-java_{plugintype}_example', {sversion})
-----

In the file above:

* Rename the file to correspond to the {plugintype} name.
* Change the `require_jar` directive to correspond to the `group` specified in the
Gradle build file, the name of the {plugintype} JAR file, and the version as
specified in both the gemspec and Gradle build file.

After you have created the previous files and the plugin JAR file, build the gem using the
following command:

[source,shell]
[subs="attributes"]
-----
gem build logstash-{plugintype}-<{plugintype}-name>.gemspec
-----

[float]
==== Installing the Java plugin in Logstash

After you have packaged your Java plugin as a Ruby gem, you can install it in
Logstash with this command:

[source,shell]
-----
bin/logstash-plugin install --no-verify --local /path/to/javaPlugin.gem
-----

For Windows platforms: Substitute backslashes for forward slashes as appropriate in the command.

52 changes: 52 additions & 0 deletions docs/static/include/javapluginsetup.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
To develop a new Java {plugintype} for Logstash, you write a new Java class that
conforms to the Logstash Java {pluginclass} API, package it, and install it with the
logstash-plugin utility. We'll go through each of those steps.

[float]
=== Set up your environment

[float]
==== Copy the example repo

Start by copying the {pluginrepo}. The plugin API is currently part of the
Logstash codebase so you must have a local copy of that available. You can
obtain a copy of the Logstash codebase with the following `git` command:

[source,shell]
-----
git clone --branch <branch_name> --single-branch https://github.com/elastic/logstash.git <target_folder>
-----

The `branch_name` should correspond to the version of Logstash containing the
preferred revision of the Java plugin API.

NOTE: The experimental version of the Java plugin API is available in the `6.6`
branch of the Logstash codebase.

Specify the `target_folder` for your local copy of the Logstash codebase. If you
do not specify `target_folder`, it defaults to a new folder called `logstash`
under your current folder.

[float]
==== Generate the .jar file

After you have obtained a copy of the appropriate revision of the Logstash
codebase, you need to compile it to generate the .jar file containing the Java
plugin API. From the root directory of your Logstash codebase ($LS_HOME), you
can compile it with `./gradlew assemble` (or `gradlew.bat assemble` if you're
running on Windows). This should produce the
`$LS_HOME/logstash-core/build/libs/logstash-core-x.y.z.jar` where `x`, `y`, and
`z` refer to the version of Logstash.

After you have successfully compiled Logstash, you need to tell your Java plugin
where to find the `logstash-core-x.y.z.jar` file. Create a new file named
`gradle.properties` in the root folder of your plugin project. That file should
have a single line:

[source,txt]
-----
LOGSTASH_CORE_PATH=<target_folder>/logstash-core
-----

where `target_folder` is the root folder of your local copy of the Logstash codebase.

Loading

0 comments on commit 31f7314

Please sign in to comment.