Transforming Ruby Code

Ben Hughes @rubiety http://benhugh.es

Tuesday, October 11, 11

for element in collection do_something(element) end

collection.each do |element| do_something(element) end

Tuesday, October 11, 11

Factory.define(:product) do |f| f.association :category f.sequence(:name) {|n| "Product #{n}" } f.price 19.95 end

FactoryGirl.define do factory :product do category sequence(:name) {|n| "Product #{n}" } price 19.95 end end
Tuesday, October 11, 11

Why?
Automated Refactoring Enforcing Coding Style or Best Practices DSL Conversion (e.g. factory_girl 1 => 2) Reduce Friction of changing technical paths
Tuesday, October 11, 11

String#gsub!
Tuesday, October 11, 11

for element in collection do_something(element) end gsub!( /for (\S+) in (\S+)/, '\2.each do |\1|' )

collection.each do |element| do_something(element) end

Tuesday, October 11, 11

for element in find(1, 2) do_something(element) end gsub!( /for (\S+) in (\S+)/, '\2.each do |\1|' )

collection.each do |element| do_something(element) end

Tuesday, October 11, 11

AST Transformations
Parse into AST Modify AST De-parse AST
Tuesday, October 11, 11

What is an AST?
2 + 3 2.+(3)
Invoke Method (call) 2 3

:+

Tuesday, October 11, 11

S-Expressions
class Sexp < Array def kind self[0] end def body self[1..-1] end end def s(*args) Sexp.new(*args) end

kind

body

[:lasgn, :foo, [:lit, 1]]

Tuesday, October 11, 11

2 + 3

=>

[:call, [:lit, 2], :+, [:arglist, [:lit, 3]] ]

foo = 1

=>

[:lasgn, :foo, [:lit, 1] ]

Tuesday, October 11, 11

if a 1 else 2 end

=>

[:if, [:call, nil, :a, [:arglist] ], [:lit, 1], [:lit, 2] ]

Tuesday, October 11, 11

class Project def name "Rails" end end

[:class, :Project, nil, [:scope, [:defn, :name, [:args], => [:scope, [:block, [:str, "Rails"] ] ] ] ] ]

Tuesday, October 11, 11

Parsing
ruby_parser: Pure Ruby (uses racc),
Parses 1.8 only

ParseTree: Uses C Extension, same AST

as ruby_parser, Parses 1.8 only

Ripper: Internal Ruby 1.9 Parser exposes

via standard library
Tuesday, October 11, 11

Tree Walker Pattern

module RubyTransform class Transformer include TransformerHelpers def transform(sexp) if sexp.is_a?(Sexp) Sexp.new(*([sexp.kind] + sexp.body.map {|c| transform(c) })) else sexp end end end end Recursion on Children AST Nodes

Tuesday, October 11, 11

Eachier (for => .each)

for element in collection do_something(element) end

collection.each do |element| do_something(element) end

Tuesday, October 11, 11

class Eachifier < RubyTransform::Transformer def transform(e) super(transform_fors_to_eaches(e)) end def transform_fors_to_eaches(e) if sexp?(e) && e.kind == :for transform_for_to_each(e) else e end end def transform_for_to_each(e) s(:iter, s(:call, e.body.first, :each, s(:arglist)), e.body.second, e.body.third ) end end
Tuesday, October 11, 11

Clear Explicit Returns

def my_method a = 1 call_something(a) return a end def my_method a = 1 call_something(a) a end

Tuesday, October 11, 11

class ClearExplicitReturns < RubyTransform::Transformer def transform(e) super(transform_explicit_returns(e)) end def transform_explicit_returns(e) if matches_explicit_return_method?(e) transform_explicit_return_method(e) else e end end def matches_explicit_return_method?(e) e.is_a?(Sexp) && e.method? && e.block && e.block.body.last && e.block.body.last.kind == :return end def transform_explicit_return_method(e) e.clone.tap do |exp| exp.block[-1] = exp.block[-1].body[0] end end end
Tuesday, October 11, 11

Block Method To Proc-ier

collection.map {|d| d.name }

collect.map(&:name)

Tuesday, October 11, 11

Tapier
def my_method temp = "" temp << "Something" call_something(temp) temp end def my_method "".tap do |temp| temp << "Something" call_something(temp) end end

Tuesday, October 11, 11

Custom Transforms
# Reverse all string literals! RubyTransform::Transformers::Custom.new do |expression| if sexp?(expression) && expression.kind == :str s(:str, expression.body[0].reverse) else super end end

Tuesday, October 11, 11

AST De-Parsing Challenges

Lots of Lost Information! All Whitespace Comments (Depending on Parser) Line Numbers (Depending on Parser) String Literal Quote Style Block Style: { .. } vs do .. end
Tuesday, October 11, 11

AST De-Parsers
ruby2ruby:
Ruby 1.8 only, minimal code formatting considerations Ruby 1.9 (operates on a ripper AST), OO sexpression abstractions Ruby 1.8 only, intelligent code formatting (emitter congurable)

ripper2ruby: ruby_scribe:
Tuesday, October 11, 11

Intelligent Code Formatting

Compositing the parsing and de-parsing operation turns it into an intelligent (and potentially congurable) code formatter.

Tuesday, October 11, 11

ruby_scribe Example
require( "pp" ) class Project; attr_accessor(:name) def title if active? then "Active Project: #{name}" else "Disabled Project: #{name}" end end end

Tuesday, October 11, 11

[:block, [:call, nil, :require, [:arglist, [:str, "pp"]]], [:class, :Project, nil, [:scope, [:block, [:call, nil, :attr_accessor, [:arglist, [:lit, :name]]], [:defn, :title, [:args], [:scope, [:block, [:if, [:call, nil, :active?, [:arglist]], [:dstr, "Active Project: ", [:evstr, [:call, nil, :name, [:arglist]]]], [:dstr, "Disabled Project: ", [:evstr, [:call, nil, :name, [:arglist]]]] ] ]] ] ]] ] ]

Tuesday, October 11, 11

sexp = RubyParser.new.parse(File.read(path)) RubyScribe::Emitter.new.emit(sexp)

require "pp" class Project attr_accessor :name def title if active? "Active Project: #{name}" else "Disabled Project: #{name}" end end end
Tuesday, October 11, 11

Impediments
Standard & Solid AST Format: Ripper? Even Better De-Parsing S-expression Matchers (Tree Expressions) Better OO Tools for Transformations
(Abstract Underlying AST Verbosity)

Tuesday, October 11, 11

rspecify
http://github.com/rubiety/rspecify
$ rspecify cat my_class_test.rb class MyClass < ActiveSupport::TestCase def test_should_be_one assert_equal something, 1 end def test_should_be_one assert_not_equal something, 1 end end

describe MyClass do it "should be one" do something.should == 1 end it "should not be one" do something.should_not == 1 end end

Tuesday, October 11, 11

factory_girl_upgrader
http://github.com/rubiety/factory_girl_upgrader
$ factory_girl_upgrader cat factories.rb Factory.define(:product) do |f| f.association :category f.sequence(:name) {|n| "Product #{n}" } f.price 19.95 end

FactoryGirl.define do factory :product do category sequence(:name) {|n| "Product #{n}" } price 19.95 end end
Tuesday, October 11, 11

Future: AST Translation

Parse Java AST Translate Java AST => Ruby AST Apply Idiomizing Transformations De-parse AST => Working, fairly idiomatic, JRuby!
Tuesday, October 11, 11

Questions?
http://github.com/rubiety/ruby_scribe http://github.com/rubiety/ruby_transform http://github.com/rubiety/rspecify http://github.com/rubiety/factory_girl_upgrader

Ben Hughes @rubiety http://benhugh.es

Tuesday, October 11, 11