Skip to content

malesch/telemere

 
 

Repository files navigation

Taoensso open source
API | Wiki | Latest releases | Slack channel

Telemere logo

Structured telemetry library for Clojure/Script

Telemere is a next-generation replacement for Timbre that offers a simple unified API for structured and traditional logging, tracing, and basic performance monitoring.

It helps enable Clojure/Script systems that are observable, robust, and debuggable - and it represents the refinement and culmination of ideas brewing over 12+ years in Timbre, Tufte, Truss, etc.

[Terminology] Telemetry derives from the Greek tele (remote) and metron (measure). It refers to the collection of in situ (in position) data, for transmission to other systems for monitoring/analysis. Logs are the most common form of software telemetry. So think of telemetry as the superset of logging-like activities that help monitor and understand (software) systems.

Latest release/s

  • 2024-08-08 v1.0.0-beta16: release info (for early adopters/feedback)

Main tests Graal tests

See here for earlier releases.

Why Telemere?

Ergonomics

  • Elegant, lightweight API that's easy to use, easy to configure, and deeply flexible.
  • Sensible defaults to make getting started fast and easy.
  • Extensive beginner-oriented documentation, docstrings, and error messages.

Interop

Scaling

  • Hyper-optimized and blazing fast, see benchmarks.
  • An API that scales comfortably from the smallest disposable code, to the most massive and complex real-world production environments.
  • Auto handler stats for debugging performance and other issues at scale.

Flexibility

  • Config via plain Clojure vals and fns for easy customization, composition, and REPL debugging.
  • Unmatched environmental config support: JVM properties, environment variables, or classpath resources. Per platform, or cross-platform.
  • Unmatched filtering support: by namespace, id pattern, level, level by namespace pattern, etc. At runtime and compile-time.
  • Fully configurable a/sync dispatch support: blocking, dropping, sliding, etc.
  • Turn-key sampling, rate-limiting, and back-pressure monitoring with sensible defaults.

Comparisons

Video demo

See for intro and basic usage:

Telemere demo video

Quick examples

(require '[taoensso.telemere :as t])

(t/log! {:id ::my-id, :data {:x1 :x2}} "My message") %>

;; 2024-04-11T10:54:57.202869Z INFO LOG Schrebermann.local examples(56,1) ::my-id - My message
;;    data: {:x1 :x2}

(t/log!       "This will send a `:log` signal to the Clj/s console")
(t/log! :info "This will do the same, but only when the current level is >= `:info`")

;; Easily construct messages from parts
(t/log! :info ["Here's a" "joined" "message!"])

;; Attach an id
(t/log! {:level :info, :id ::my-id} "This signal has an id")

;; Attach arb user data
(t/log! {:level :info, :data {:x :y}} "This signal has structured data")

;; Capture for debug/testing
(t/with-signal (t/log! "This will be captured"))
;; => {:keys [location level id data msg_ ...]}

;; `:let` bindings are available to `:data` and message, but only paid
;; for when allowed by minimum level and other filtering criteria
(t/log!
  {:level :info
   :let [expensive-metric1 (last (for [x (range 100), y (range 100)] (* x y)))]
   :data {:metric1 expensive-metric1}}
  ["Message with metric value:" expensive-metric1])

;; With sampling 50% and 1/sec rate limiting
(t/log!
  {:sample-rate 0.5
   :rate-limit  {"1 per sec" [1 1000]}}
  "This signal will be sampled and rate limited")

;; There are several signal creators available for convenience.
;; All support the same options but each offer's a calling API
;; optimized for a particular use case. Compare:

;; `log!` - [msg] or [level-or-opts msg]
(t/with-signal (t/log! {:level :info, :id ::my-id} "Hi!"))

;; `event!` - [id] or [id level-or-opts]
(t/with-signal (t/event! ::my-id {:level :info, :msg "Hi!"}))

;; `signal!` - [opts]
(t/with-signal (t/signal! {:level :info, :id ::my-id, :msg "Hi!"}))

;; See `t/help:signal-creators` docstring for more

;;; A quick taste of filtering

(t/set-ns-filter! {:disallow "taoensso.*" :allow "taoensso.sente.*"})
(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}})

(t/set-min-level!       :warn) ; Set minimum level for all    signals
(t/set-min-level! :log :debug) ; Set minimul level for `log!` signals

;; Set minimum level for `event!` signals originating in
;; the "taoensso.sente.*" ns
(t/set-min-level! :event "taoensso.sente.*" :warn)

;; See `t/help:filters` docstring for more

;;; Use middleware to transform signals and/or filter signals
;;; by signal data/content/etc.

(t/set-middleware!
  (fn [signal]
    (if (get-in signal [:data :hide-me?])
      nil ; Suppress signal (don't handle)
      (assoc signal :passed-through-middleware? true))))

(t/with-signal (t/event! ::my-id {:data {:hide-me? true}}))  ; => nil
(t/with-signal (t/event! ::my-id {:data {:hide-me? false}})) ; => {...}

See examples.cljc for more REPL-ready snippets!

API overview

See relevant docstrings (links below) for usage info-

Creating signals

Name Signal kind Main arg Optional arg Returns
log! :log msg opts/level Signal allowed?
event! :event id opts/level Signal allowed?
error! :error error opts/id Given error
trace! :trace form opts/id Form result
spy! :spy form opts/level Form result
catch->error! :error form opts/id Form value or given fallback
signal! <arb> opts - Depends on opts

Internal help

Detailed help is available without leaving your IDE:

Var Help with
help:signal-creators Creating signals
help:signal-options Options when creating signals
help:signal-content Signal content (map given to middleware/handlers)
help:filters Signal filtering and transformation
help:handlers Signal handler management
help:handler-dispatch-options Signal handler dispatch options
help:environmental-config Config via JVM properties, environment variables, or classpath resources.

Included handlers

See linked docstrings below for features and usage:

Name Platform Output target Output format
handler:console Clj *out* or *err* edn/JSON or human-readable
handler:console Cljs Browser console edn/JSON or human-readable
handler:console-raw Cljs Browser console Raw signals for cljs-devtools, etc.
handler:file Clj File/s on disk edn/JSON or human-readable
handler:open-telemetry-logger Clj OpenTelemetry Java client LogRecord
handler:postal Clj Email (via postal) edn/JSON or human-readable
handler:slack Clj Slack (via clj-slack) edn/JSON or human-readable
handler:tcp-socket Clj TCP socket edn/JSON or human-readable
handler:udp-socket Clj UDP socket edn/JSON or human-readable

See here for more/upcoming handlers, community handlers, info on writing your own handlers, etc.

Documentation

Benchmarks

Telemere is highly optimized and offers terrific performance at any scale:

Compile-time filtering? Runtime filtering? Time? Trace? nsecs
✓ (elide) - - - 0
- - - 200
- - 280
- 650

Measurements:

  • Are ~nanoseconds per signal call (= milliseconds per 1e6 calls)
  • Exclude handler runtime (which depends on handler/s, is usually async)
  • Taken on a 2020 Macbook Pro M1, running OpenJDK 21

Tip: Telemere offers extensive filtering support - including dead-easy per-signal and per-handler sampling and rate-limiting. Use these to ensure that you're not capturing useless/low-value information in production. See here for more tips!

Funding

You can help support continued work on this project, thank you!! 🙏

License

Copyright © 2023-2024 Peter Taoussanis.
Licensed under EPL 1.0 (same as Clojure).

About

Structured telemetry library for Clojure/Script

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Clojure 96.2%
  • Java 3.7%
  • Other 0.1%