Running Sidekiq on Heroku w/ Unicorn

Alex Egg,

This article describes how to run a sidekiq worker on heroku w/o paying for an additional heroku worker dyno.

First of all, I think heroku is doing a great service for the community by serving sites for free w/ one dyno. I know a lot of people who start on this free-tier and then migrate up to a paid tier. However, if you’re not ready to make that leap yet, there is still a way to have a sidekiq working by using Unicorn.

Unicorn is a webserver written in ruby which gives us the flexibility to hook into it’s event callbacks. A unicorn instance takes a config file:

worker_processes (ENV["WEB_CONCURRENCY"] || 3).to_i
timeout (ENV["WEB_TIMEOUT"] || 5).to_i
preload_app true

run_sidekiq_in_this_thread = ENV["RACK_ENV"]=="development" ? false : true
@sidekiq_pid = nil

before_fork do |server, worker|

  Signal.trap 'TERM' do
    puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
    Process.kill 'QUIT',

  defined?(ActiveRecord::Base) and

  if run_sidekiq_in_this_thread
    p "starting Sidekiq w/ unicorn"
    @sidekiq_pid ||= spawn("bundle exec sidekiq")'Spawned sidekiq #{@sidekiq_pid}')

after_fork do |server, worker|

  Signal.trap 'TERM' do
    puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT'

  if defined? ActiveRecord::Base
    config = ActiveRecord::Base.configurations[Rails.env] ||
    config["reaping_frequency"] = (ENV["DB_REAPING_FREQUENCY"] || 10).to_i
    config["pool"] = (ENV["DB_POOL"] || 2).to_i

Focus on the if run_sidekiq_in_this_thread block – because Unicorn lets us run arbitrary code in it’s fork callback, we can launch our sidekiq rake task along with our rails app.

So in this example we are running 3 rails processes and 1 sidekiq process in the same heroku dyno. The alternative is to pay ~$35/month for your delayed job that runs < %1 of the time.

Another alternative is to use a proper heroku worker dyno but adaptively scale up and down (turn on turn off) when you task needs to run. I’ll have an app that does this with good success, which I’ll document later.

Permalink: running-sidekiq-on-heroku-w-unicorn


Last edited by Alex Egg, 2016-10-05 19:09:33
View Revision History