Long-Running Tasks In Rails Without Much Effort

Andy Stewart
April 2008

Not me

Still not me

Why?

User starts something lengthy

Delayed task

Regular task

Example

Synchronous link to Campaign Monitor

10

BackgroundJob (BJ) Sparrow AM4R BackgrounDRb Workling

Spawn Delayed Job (DJ) Conveyor Beanstalkd & async-observer

BackgroundFu

Factors

Durability Progress reporting Scheduling Serial vs. Parallel Process management Execution environment Error handling Learning curve Installation burden Constraints on task code

Durability

Durable
Background Job BackgroundFu Delayed Job Workling

Hopeful
BackgrounDRb Beanstalk Spawn Sparrow Workling

0%

50%

100%

Tool
Background Job BackgroundFu BackgrounDRb Workling

Method
Process’s stats Incremental Ask workers DIY “return store”

Scheduling

BackgrounDRb cron + {rake, script/runner, whatever}

Delaying
(on purpose)

Delayed Job Beanstalk

Process Management

No Problem
Background Job Spawn

Hassle
Everything else

Examples

BackgrounDRb
# Lots of configuration ... # Your worker code class BillingWorker < BackgrounDRb::MetaWorker set_worker_name :billing_worker # Called when worker is loaded for the first time def create(args = nil) end # The lengthy task. def charge_customer(customer_id = nil) # ... do stuff ... end end # Your invocation code class CustomersController < ApplicationController def upgrade_account # ... MiddleMan.worker(:billing_worker).charge_customer(@customer.id) end end

Beanstalk
# Some configuration ... # Comments controller def create @comment = Comment.new(params[:comment]) if @comment.save BEANSTALK.yput({:type => "comment", :id => @comment.id}) rescue nil # Then redirect and return end end # Worker - Rake task loop do job = BEANSTALK.reserve job_hash = job.ybody # ybody deserializes the job case job_hash[:type] when "comment" if Comment.check_for_spam(job_hash[:id]) job.delete else job.bury end else puts "Don't know what type of job this is: #{job_hash.inspect}" end end

Source: nubyonrails.com

Async Observer
# Some configuration ... # Start some workers $ ./vendor/plugins/async_observer/bin/worker # Your code class Person < ActiveRecord::Base async_after_create do |person| SiteStats.increment_members() end def befriend(other_person) Friendship.async_send(:create, self, other_person) end end

Delayed Job
# Small config - db migration # A job is a Ruby object with #perform class NewsletterJob < Struct.new(:text, :emails) def perform emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) } end end # Stick in queue Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email)) # Delayed job BatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv) # Running tasks $ rake jobs:work

<CTRL-C to cancel>

Background Job
# Command line $ Bj.submit "./jobs/background_job_to_run" $ Bj.submit "./script/runner ./jobs/background_job_to_run"

# Within Rails def upload_to_s3 Bj.submit "./script/runner ./jobs/s3_uploader.rb #{self.id}" end

Source: slackworks.com

1

Background Job
Two-line installation Zero configuration No job-code constraints Automatic* process management Durable, prioritised, taggable jobs Comprehensive job results Clustering
* Manual if you want

Feed my spaniel

Buy my PeepCode PDF

Photo Credits:
1. BBC 2. Forever in Song (CD cover) 3. Me 4. Me

My Photos’ Licence:
Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales Licence

Text’s Licence:
Creative Commons Attribution-Share Alike 2.0 UK: England & Wales Licence