GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
Inspired by Delayed::Job and Que, GoodJob is designed for maximum compatibility with Ruby on Rails, ActiveJob, and Postgres to be simple and performant for most workloads.
- Designed for ActiveJob. Complete support for async, queues, delays, priorities, timeouts, and retries with near-zero configuration.
- Built for Rails. Fully adopts Ruby on Rails threading and code execution guidelines with Concurrent::Ruby.
- Backed by Postgres. Relies upon Postgres integrity and session-level Advisory Locks to provide run-once safety and stay within the limits of
schema.rb. - For most workloads. Targets full-stack teams, economy-minded solo developers, and applications that enqueue less than 1-million jobs/day.
For more of the story of GoodJob, read the introductory blog post.
Add this line to your application's Gemfile:
gem 'good_job'And then execute:
$ bundle install-
Create a database migration:
$ bin/rails g good_job:install
Run the migration:
$ bin/rails db:migrate
-
Configure the ActiveJob adapter:
# config/application.rb config.active_job.queue_adapter = :good_job
By default, using
:good_jobis equivalent to manually configuring the adapter:# config/environments/development.rb config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :inline) # config/environments/test.rb config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :inline) # config/environments/production.rb config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :external)
-
Queue your job 🎉:
YourJob.set(queue: :some_queue, wait: 5.minutes, priority: 10).perform_later
-
In production, the scheduler is designed to run in its own process:
$ bundle exec good_jobConfiguration options available with
help:$ bundle exec good_job help start # Usage: # good_job start # # Options: # [--max-threads=N] # Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size) # [--queues=queue1,queue2] # Queues to work from. Separate multiple queues with commas (default: *) # [--poll-interval=N] # Interval between polls for available jobs in seconds (default: 1)
ActiveJob has a rich set of built-in functionality for timeouts, error handling, and retrying. For example:
class ApplicationJob < ActiveJob::Base
# Retry errors an infinite number of times with exponential back-off
retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
# Timeout jobs after 10 minutes
JobTimeoutError = Class.new(StandardError)
around_perform do |_job, block|
Timeout.timeout(10.minutes, JobTimeoutError) do
block.call
end
end
endGoodJob executes enqueued jobs using threads. There is a lot than can be said about multithreaded behavior in Ruby on Rails, but briefly:
- Each GoodJob execution thread requires its own database connection, which are automatically checked out from Rails’s connection pool. Allowing GoodJob to schedule more threads than are available in the database connection pool can lead to timeouts and is not recommended.
- The maximum number of GoodJob threads can be configured, in decreasing precedence:
$ bundle exec good_job --max_threads 4$ GOOD_JOB_MAX_THREADS=4 bundle exec good_job$ RAILS_MAX_THREADS=4 bundle exec good_job- Implicitly via Rails's database connection pool size (
ActiveRecord::Base.connection_pool.size)
If your application is already using an ActiveJob backend, you will need to install GoodJob to enqueue and perform newly created jobs and finish performing pre-existing jobs on the previous backend.
-
Enqueue newly created jobs on GoodJob either entirely by setting
ActiveJob::Base.queue_adapter = :good_jobor progressively via individual job classes:# jobs/specific_job.rb class SpecificJob < ApplicationJob self.queue_adapter = :good_job # ... end
-
Continue running executors for both backends. For example, on Heroku it's possible to run two processes within the same dyno:
# Procfile # ... worker: bundle exec que ./config/environment.rb & bundle exec good_job & wait -n
-
Once you are confident that no unperformed jobs remain in the previous ActiveJob backend, code and configuration for that backend can be completely removed.
GoodJob is fully instrumented with ActiveSupport::Notifications.
By default, GoodJob will delete job records after they are run, regardless of whether they succeed or not (raising a kind of StandardError), unless they are interrupted (raising a kind of Exception).
To preserve job records for later inspection, set an initializer:
# config/initializers/good_job.rb
GoodJob.preserve_job_records = trueIt is also necessary to delete these preserved jobs from the database after a certain time period:
-
For example, in a Rake task:
# GoodJob::Job.finished(1.day.ago).delete_all -
For example, using the
good_jobcommand-line utility:$ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
To run tests:
# Clone the repository locally
$ git clone [email protected]:bensheldon/good_job.git
# Set up the local environment
$ bin/setup_test
# Run the tests
$ bin/rspecThis gem uses Appraisal to run tests against multiple versions of Rails:
# Install Appraisal(s) gemfiles
$ bundle exec appraisal
# Run tests
$ bundle exec appraisal bin/rspec
For developing locally within another Ruby on Rails project:
# Within Ruby on Rails directory...
$ bundle config local.good_job /path/to/local/git/repository
# Confirm that the local copy is used
$ bundle install
# => Using good_job 0.1.0 from https://github.com/bensheldon/good_job.git (at /Users/You/Projects/good_job@dc57fb0)Package maintainers can release this gem with the following gem-release command:
# Sign into rubygems
$ gem signin
# Update version number, changelog, and create git commit:
$ bundle exec rake commit_version[minor] # major,minor,patch
# ..and follow subsequent directions. Contribution directions go here.
The gem is available as open source under the terms of the MIT License.