Tap --[not] a talk about

replacing rake
Simon Chiang
Biomolecular Structure
 Data Analysis

Submit to search engine

...
Pre-process Harvest results
 Design Goals

• Each task is independent, configurable

• Tasks must have an immediate command
line interface
• Ability to link tasks together into
workflows
 Originally, Rake

[Rakefile]
desc “a simple goodnight task”
task :goodnight, :obj do |task, args|
puts “#{ENV[‘msg’] || ‘goodnight’} #{args.obj}”
end

% rake -T
% rake goodnight[moon]
% rake goodnight[world] msg=hello
Dependencies

x1 y

x2 y1 z
 The nature of Rake

• Insufficient command line support

• No imperative workflows
• Hard to test/reuse/distribute

• Not a knock on rake!

• Domain specific solution
 Tap (Task Application)
• A command line framework
• Imperative and dependency workflows
• Distribution of tasks as gems
x c

x1 y b c1

x2 y1 z a ? b1 c2
Rake, Rap, Tap
 Rake Task
[Rakefile, rakefile, rakefile.rb]
desc “a simple goodnight task”
task :goodnight, :obj do |task, args|
puts “#{ENV[‘msg’] || ‘goodnight’} #{args.obj}”
end

% rake -T
% rake goodnight[moon]
% rake goodnight[world] msg=hello
 Rap Task
[Rapfile, rapfile, rapfile.rb]
# ::desc a simple goodnight task
# Extended documentation for the goodnight task...
task :goodnight, :obj, :msg => ‘goodnight’ do |task,args|
puts “#{task.msg} #{args.obj}”
end

% rap -T
% rap goodnight moon
% rap goodnight world --msg hello
% rap goodnight --help
Command Line
 Quite Compatible
require ‘tap’
include Tap::Declartions

namespace :name do
desc ‘one line description’
task({:name => [:deps]}, :inputs, {:key => ‘value’}) do
sh “...”
end
end

• Declarations must be included

• No FileList, Rake::TestTask, etc.
 Rap runs Rake
[Rakefile] [Rapfile]
require ‘rake’ require ‘tap’
include Tap::Declarations
desc “says hello”
task :say_hello do desc “says goodbye”
puts “hello” task :say_goodbye do
end puts “goodbye”
end

% rap say_hello
% rap say_goodbye
% rap -T
 Rap is like a
supercharged Rake
 Class vs Instance
• Rake makes instances
t = task :name
t.class # => Rake::Task

• Rap makes singleton subclasses

t = task :name

t.class # => Name

Name.superclass # => Tap::Task
Name.instance # => t
 Beyond Rap
# ::desc a simple goodnight task
task :goodnight, :obj, :msg => ‘goodnight’ do |task,args|
puts “#{task.msg} #{args.obj}”
end

# ::manifest a simple goodnight task

class Goodnight < Tap::Task
config :msg, ‘goodnight’

def process(obj)
puts “#{msg} #{obj}”
end
end
Anatomy of a Task
Class
 Executable
Configurable
(Lazydoc)
 Executable

• Wraps a method
• Adds workflowability c

b c1

a ? b1 c2
 Process

class Goodnight < Tap::Task

def process(obj=‘opus’, *objects)
objects.unshift(obj)
puts “goodnight #{objects.join(‘,’)}”
end
end

% rap goodnight
% rap goodnight moon lamp “little toy boat”
% rap goodnight --help
Command Line
 Configurable
class Goodnight < Tap::Task
config :key, ‘value’ do |input|
input.downcase
end
end

t = Goodnight.new
t.key # => ‘value’
t.key = ‘New Value’
t.config[:key] # => ‘new value’
 Standard Blocks
class Goodnight < Tap::Task
config :message, ‘goodnight’
config :reverse, false, &c.switch
config :repeat, 1, &c.integer

def process
str = “#{message} moon”
str = str.reverse if reverse == true
repeat.times { puts str }
end
end

% rap goodnight --reverse --repeat 3

% rap goodnight --help
Command Line
 Lazydoc
# SomeClass::key this is the value
# Extended value...

class SomeClass
extend Tap::Support::LazyAttributes
lazy_attr :key
end

SomeClass.source_file = __FILE__
SomeClass.key.value # => “this is the value”
SomeClass.key.to_s # => “Extended value...”
 Lazydoc
[lib/goodnight.rb]
# ::manifest a fancy goodnight task
# Says goodnight to lots of objects.
class Goodnight < Tap::Task
config :message, ‘goodnight’ # a goodnight message
config :reverse, false, &c.switch # reverses output
config :repeat, 1, &c.integer # repeats output
end

Goodnight.manifest.value
# => “a fancy goodnight task”
Goodnight.manifest.to_s
# => “Says goodnight to lots of objects.”
Command Line
Off the command line
class Goodnight < Tap::Task
config :msg, ‘goodnight’

def process(obj)
“#{msg} #{obj}”
end
end

g = Goodnight.new
g.process(‘moon’) # => ‘goodnight moon’

g.msg = ‘hello’
g.process(‘world’) # => ‘hello world’
 Testing

class GoodnightTest < Test::Unit::Testcase

def test_goodnight_says_goodnight
g = Goodnight.new
assert_equal ‘goodnight moon’, g.process(‘moon’)
end
end
 Subclasses
class ReverseGoodnight < Goodnight
config :reverse, true, &c.switch

def process(obj)
super(reverse ? obj.reverse : obj)
end
end

g = ReverseGoodnight.new(:message => ‘hello’)

g.process(‘world’) # => ‘dlrow olleh’
g.reverse = false
g.process(‘world’) # => ‘hello world’
 Pass Results

g1 = Goodnight.new(:message => ‘yo’)

g2 = Goodnight.new(:message => ‘yo’)

result = g1.process(‘moon’)
g2.process(result) # => ‘yo yo moon’
Workflows
 Executable

• Audits execution of the method

g = Goodnight.new
_result = g._execute(‘moon’)
_result.class # => Tap::Support::Audit
_result._current # => ‘goodnight moon’
_result._source # => g
 Executable

• Adds an on_complete block

current = nil
g = Goodnight.new
g.on_complete do |_result|
current = _result._current
end

g._execute(‘moon’)
current # => ‘goodnight moon’
 Executable

• Allows specification of dependencies

dep_ran = false
dep = Tap::Task.intern {|task| dep_ran = true }

g = Goodnight.new
g.depends_on(dep)
g._execute

dep_ran # => true

 Build, Enq, Run

Aggregator

ExecutableQueue

Tap::App
 Example
• Simple Trace task, adds name to str
class Trace < Tap::Task
def process(str)
str + name
end
end

t = Trace.new({}, ‘a’)
t.name # => ‘a’
t.process(‘’) # => ‘a’
t.process(‘xyz’) # => ‘xyza’
 Standard Joins
a, b, b1, c, c1, c2 = %w{a b b. c c. c..}.map do |name|
Tracer.new({}, name)
end

a.switch(b, b1) do |_result| c

_result._current == ‘a’ ? 0 : 1
end
b.fork(c, c1)
c.sequence(a)
b c1
c2.merge(b1, c1)

a ? b1 c2
 Enque and Run
# (could just do this)
# a.execute(‘’)

app = Tap::App.instance c
app.enq(a, ‘’)
app.run

app.results(c2) b c1
# => ["abcab.c..", "abc.c.."]

a ? b1 c2
 Audit Trail
puts app._results(c2)[0]._to_s

# o-[] ""
# o-[a] "a" c
# o-[b] "ab"
# o-[c] "abc"
# o-[a] "abca"
# o-[b.] "abcab."
b c1
# o-[c..] "abcab.c.."

a ? b1 c2
 Workflow Syntax

task a b c --key value --: task x y z --key value

---
- [task, a, b, c, --key, value]
- [task, x, y, z, --key, value]
- 0:1
 Delimiters
break meaning
-- enque next to round 0
--+ enque next to round 1
--++ enque next to round 2
--+3 enque next to round 3

--0:1 sequence 0 -> 1

--0[1,2] fork 0 -> [1,2]
--2{0,1} merge [0,1] -> 2
--2(0,1) sync_merge [0,1] -> 2

--* enque a ‘global’ dependency instance

Distribution
 Env, Manifests
tap: (/Users/simonchiang/Documents/Gems/tap)
commands
console (cmd/console.rb)
destroy (cmd/destroy.rb)
generate (cmd/generate.rb)
manifest (cmd/manifest.rb)
run (cmd/run.rb)
generators
command (lib/tap/generator/generators/command/command_generator.rb)
config (lib/tap/generator/generators/config/config_generator.rb)
file_task (lib/tap/generator/generators/file_task/file_task_generator.rb)
generator (lib/tap/generator/generators/generator/generator_generator.rb)
root (lib/tap/generator/generators/root/root_generator.rb)
task (lib/tap/generator/generators/task/task_generator.rb)
tasks
dump (lib/tap/tasks/dump.rb)
load (lib/tap/tasks/load.rb)
rake (lib/tap/tasks/rake.rb)
 Manifest via Lazydoc
[lib/nested/sample.rb]
module Nested
# ::manifest
class Sample < Tap::Task
end
implicit, by path
end

# Another::manifest
class Another < Tap::Task explicitly named
end

class Hidden < Tap::Task overlooked

end
Nested Envs
Desktop
|- molecules
|- sample_tasks
`- tap
 Mini-paths
/path/to/molecules-0.1.0:/path/to/molecules/lib/calc.rb

calc
molecules:calc
molecules-0.1.0:calc.rb
 RubyGems

• Package normally (Rapfile doesn’t work)

• Add a tap.yml file (can be empty)
• Tasks will be discovered automatically
Context
Server
Ubiquity
 Scripts
#!/usr/bin/env ruby

require ‘rubygems’
require ‘tap’

# Goodnight::manifest a fancy goodnight task

# Says goodnight to lots of objects.
class Goodnight < Tap::Task
config :message, ‘goodnight’

def process(*objects)
puts “#{message} #{objects.join(‘,’)}”
end
end

Goodnight.execute(ARGV)
 Wrapup

• A supercharged rake
• A framework for configurable,
distributable tasks and workflows
 Dr. Kirk Hansen
Lauren Kiemele
Ashley Zurawel

Acknowledgements
Mom, Dad, Brother, Girlfriend

Thanks
tap.rubyforge.org