This article describes how to run a sidekiq worker on heroku w/o paying for an additional heroku worker dyno.
First of all, I think heroku is doing a great service for the community by serving sites for free w/ one dyno. I know a lot of people who start on this free-tier and then migrate up to a paid tier. However, if you’re not ready to make that leap yet, there is still a way to have a sidekiq working by using Unicorn.
Unicorn is a webserver written in ruby which gives us the flexibility to hook into it’s event callbacks. A unicorn instance takes a config file:
worker_processes (ENV["WEB_CONCURRENCY"] || 3).to_i timeout (ENV["WEB_TIMEOUT"] || 5).to_i preload_app true run_sidekiq_in_this_thread = ENV["RACK_ENV"]=="development" ? false : true @sidekiq_pid = nil before_fork do |server, worker| Signal.trap 'TERM' do puts 'Unicorn master intercepting TERM and sending myself QUIT instead' Process.kill 'QUIT', Process.pid end defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect! if run_sidekiq_in_this_thread p "starting Sidekiq w/ unicorn" @sidekiq_pid ||= spawn("bundle exec sidekiq") Rails.logger.info('Spawned sidekiq [email protected]_pid}') end end after_fork do |server, worker| Signal.trap 'TERM' do puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT' end if defined? ActiveRecord::Base config = ActiveRecord::Base.configurations[Rails.env] || Rails.application.config.database_configuration[Rails.env] config["reaping_frequency"] = (ENV["DB_REAPING_FREQUENCY"] || 10).to_i config["pool"] = (ENV["DB_POOL"] || 2).to_i ActiveRecord::Base.establish_connection(config) end end
Focus on the
if run_sidekiq_in_this_thread block – because Unicorn lets us run arbitrary code in it’s fork callback, we can launch our sidekiq rake task along with our rails app.
So in this example we are running 3 rails processes and 1 sidekiq process in the same heroku dyno. The alternative is to pay ~$35/month for your delayed job that runs < %1 of the time.
Another alternative is to use a proper heroku worker dyno but adaptively scale up and down (turn on turn off) when you task needs to run. I’ll have an app that does this with good success, which I’ll document later.