Skip to content

Commit

Permalink
- Add '%{+%s}' to event sprintf so you can get the unix epoch.
Browse files Browse the repository at this point in the history
  (Joda's DateTime doesn't support it otherwise from date formatting)
- Add graphite output. Still needs fault tolerance added, though.
  https://logstash.jira.com/browse/LOGSTASH-126
  • Loading branch information
jordansissel committed Jul 21, 2011
1 parent 9c07293 commit 197b1c6
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 2 deletions.
9 changes: 7 additions & 2 deletions lib/logstash/event.rb
Original file line number Diff line number Diff line change
Expand Up @@ -147,11 +147,16 @@ def sprintf(format)
# Take the inside of the %{ ... }
key = tok[2 ... -1]

if key[0,1] == "+"
# Parse event.timestamp with
if key == "+%s"
# Got %{+%s}, support for unix epoch time
datetime = @@date_parser.parseDateTime(self.timestamp)
(datetime.getMillis / 1000).to_i
elsif key[0,1] == "+"
# We got a %{+TIMEFORMAT} so use joda to format it.
datetime = @@date_parser.parseDateTime(self.timestamp)
format = key[1 .. -1]
datetime.toString(format) # return requested time format
p :datetime => datetime.class
else
# Use an event field.
value = self[key]
Expand Down
53 changes: 53 additions & 0 deletions lib/logstash/outputs/graphite.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
require "logstash/outputs/base"
require "logstash/namespace"
require "socket"

# This output allows you to pull metrics from your logs and ship them to
# graphite. Graphite is an open source tool for storing and graphing metrics.
#
# An example use case: At loggly, some of our applications emit aggregated
# stats in the logs every 10 seconds. Using the grok filter and this output,
# I can capture the metric values from the logs and emit them to graphite.
#
# TODO(sissel): Figure out how to manage multiple metrics coming from the same
# event.
class LogStash::Outputs::Graphite < LogStash::Outputs::Base
config_name "graphite"

# The address of the graphite server.
config :host, :validate => :string, :default => "localhost"

# The port to connect on your graphite server.
config :port, :validate => :number, :default => 2003

# The metric to use. This supports dynamic strings like %{@source_host}
# TODO(sissel): This should probably be an array.
config :metric, :validate => :string, :required => true

# The value to use. This supports dynamic strings like %{bytes}
# It will be coerced to a floating point value. Values which cannot be
# coerced will zero (0)
config :value, :validate => :string, :required => true

def register
# TODO(sissel): Retry on failure.
@socket = connect
end # def register

def connect
# TODO(sissel): Test error cases. Catch exceptions. Find fortune and glory.
socket = TCPSocket.new(@host, @port)
end # def connect

public
def receive(event)
# Graphite message format: metric value timestamp\n
message = [event.sprintf(@metric), event.sprintf(@value).to_f,
event.sprintf("%{+%s}")].join(" ")
# TODO(sissel): Test error cases. Catch exceptions. Find fortune and glory.
@socket.puts(message)

# TODO(sissel): retry on failure TODO(sissel): Make 'retry on failure'
# tunable; sometimes it's OK to drop metrics.
end # def receive
end # class LogStash::Outputs::Statsd

0 comments on commit 197b1c6

Please sign in to comment.