Writing a DSL in Lua

PostedAugust08,2015byleafo(@moonscript)Tags:lua
22points

Tweet

DSLs,ordomainspecificlanguages,areprogramming
languagesthataredesignedtoimplementasetoffeatures
specifictoaparticularproblemorfield.Anexamplecould
beMake,thebuildtool,whichisaspeciallydesigned
languageforcombiningcommandsandfileswhilemanaging
dependencies.
Contents

Droppingtheparenthesis
Chaining
Usingfunctionenvironments
ImplementingtheHTM Lbuilder
Closing

Alotofmodernprogramminglanguageshavesomuch

flexibilityintheirsyntaxthatitspossibletobuildlibraries
thatexposetheirownmini-languageswithinthehost
language.ThedefinitionofDSLhasbroadenedtoinclude
thesekindsoflibraries.
Inthisguidewe'llbuildaDSLforgeneratingHTML.Itlooks
likethis:

html {
body {
h1 "Welcome to my Lua site",
a{
href = "http://leafo.net",
"Go home"
}
}
}

Beforejumpingin,herearesomeDSLbuildingtechniques:

Dropping the parenthesis

OneofthecasesforLuaasdescribedinitsinitialpublic
release(1996)isthatitmakesagoodconfigurationlanguage.
Thatsstilltruetothisday,andLuaisfriendlytobuilding
DSLs.
AuniquepartaboutLuassyntaxisparenthesisareoptional

insomescenarioswhencallingfunctions.Tersenessis
importantwhenbuildingaDSL,andremovingsuperfluous
charactersisagoodwaytodothat.
Whencallingafunctionthathasasingleargumentofeither
atableliteralorastringliteral,theparenthesisareoptional.

print "hello" --> print("hello")

my_function { 1,2,3 } --> my_function({1,2,3})
-- whitespace isn't needed, these also work:
print"hello" --> print("hello")
my_function{ 1,2,3 } --> my_function({1,2,3})

Thissyntaxhasveryhighprecedence,thesameasifyou
wereusingparenthesis:

tonumber "1234" + 5 -- > tonumber("1234") + 5

Chaining
Parenthesis-lessinvocationcanbechainedaslongaseach
expressionfromtheleftevaluatestoafunction(oracallable
table).Heressomeexamplesyntaxforahypotheticalweb
routingframework:

match "/post-comment" {
GET = function ()
-- render the form
end,

POST = function ()
-- save to database
end

Ifitsnotimmediatelyobviouswhatsgoingon,writingthe
parenthesisinwillclearthingsup.Theprecedenceofthe
parenthesis-lessinvocationgoesfromlefttoright,sothe
aboveisequivalentto:

match("/post-comment")({ ... })

Thepatternwewouldusetoimplementthissyntaxwould
looksomethinglikethis:

local function match(path)

print("match:", path)
return function(params)
print("params:", params)
-- both path and params are now availble for use here
end
end

Usingarecursivefunctionconstructoritspossibletomake

chainingworkforanylength.

Using function environments

WheninteractingwithaLuamoduleyouregularlyhaveto
bringanyfunctionsorvaluesintoscopeusing

require .When

workingwithaDSL,itsnicetohaveallthefunctionality
availablewithouthavingtomanuallyloadanything.
Oneoptionwouldbetomakeallthefunctionsandvalues
globalvariables,butitsnotrecommendedasitmight
interferewithotherlibraries.
Afunctionenvironmentcanbeusedtochangehowa
functionresolvesglobalvariablereferenceswithinitsscope.
ThiscanbeusedtoautomaticallyexposeaDSLs
functionalitywithoutpollutingtheregularglobalscope.
ForthesakeofthisguideI'llassumethat

setfenv existsin

theversionofLuawe'reusing.Ifyou'reusing5.2orabove
you'llneedtoprovideyouownimplementation:
ImplementingsetfenvinLua5.2,5.3,andabove
Heresafunction

run_with_env thatrunsanotherfunction

withaparticularenvironment.

local function run_with_env(env, fn, ...)

setfenv(fn, env)
fn(...)
end

TheenvironmentpassedwillrepresenttheDSL:

local dsl_env = {
move = function(x,y)
print("I moved to", x, y)
end,

speak = function(message)
print("I said", message)
end

run_with_env(dsl_env, function()
move(10, 10)
speak("I am hungry!")
end)

Inthistrivialexamplethebenefitsmightnotbeobvious,but
typicallyyourDSLwouldbeimplementedinanother
module,andeachplaceyouinvokeitisnotnecessaryto
bringeachfunctionintoscopemanually,butratheractivate
thewholesscopewith

run_with_env .

Functionenvironmentsalsoletyoudynamicallygenerate
methodsonthefly.Usingthe

__index metamethod

implementedasafunction,anyvaluecanbe

programmaticallycreated.ThisishowtheHTMLbuilder
DSLwillbecreated.

Implementing the HTML builder

Ourgoalistomakethefollowingsyntaxwork:

html {
body {
h1 "Welcome to my Lua site",
a{
href = "http://leafo.net",
"Go home"
}
}
}

EachHTMLtagisrepresentedbyaLuafunctionthatwill
returntheHTMLstringrepresentingthattagwiththecorrect
attributeandcontentifnecessary.
Althoughitwouldbepossibletowritecodetogenerateall
theHTMLtagbuilderfunctionsaheadoftime,afunction
__index metamethodwillbeusedtogeneratethemonthefly.

InordertoruncodeinthecontextofourDSL,itmustbe
packagedintoafunction.The

render_html functionwilltake

thatfunctionandconvertittoaHTMLstring:

render_html(function()
return div {
img { src = "http://leafo.net/hi" }
}
end) -- > <div><img src="http://leafo.net/hi" /></div>

Note: The

img tagisself-closing,ithasnoseparateclosetag.

HTM Lcallsthesevoidelements.Thesew illbetreated

differentlyintheimplementation.

render_html mightbeimplementedlikethis:

local function render_html(fn)

setfenv(fn, setmetatable({}, {
__index = function(self, tag_name)
return function(opts)
return build_tag(tag_name, opts)
end
end
}))
return fn()
end

The

build_tag functioniswhereallactualworkisdone.It

takesthenameofthetag,andtheattributesandcontentasa
singletable.

Note: Thisfunctioncouldbeoptimizedbycachingthe

generatedfunctionsintheenvironmenttable.

Thevoidelements,asmentionedabove,aredefinedasa
simpleset:

local void_tags = {
img = true,
-- etc...
}

ThemostefficientwaytoconcatenatestringsinregularLua
istoaccumulatethemintoatablethencall
Manycallsto

table.concat .

table.insert couldbeusedtoappendtothis

buffertable,butIpreferthefollowingfunctiontoallow
multiplevaluestobeappendedatonce:

local function append_all(buffer, ...)

for i=1,select("#", ...) do
table.insert(buffer, (select(i, ...)))
end
end
-- example:
-- local buffer = {}
-- append_all(buffer, "a", "b", c)
-- buffer now is {"a", "b", "c"}

Note:

append_all usesLuasbuiltinfunction select toavoid

anyextraallocationsbyqueryingthevarargsobjectinsteadof

creatinganew table.

Nowtheimplementationof

build_tag :

local function build_tag(tag_name, opts)

local buffer = {"<", tag_name}
if type(opts) == "table" then
for k,v in pairs(opts) do
if type(k) ~= "number" then
append_all(buffer, " ", k, '="', v, '"')
end
end
end
if void_tags[tag_name] then
append_all(buffer, " />")
else
append_all(buffer, ">")
if type(opts) == "table" then
append_all(buffer, unpack(opts))
else
append_all(buffer, opts)
end
append_all(buffer, "</", tag_name, ">")
end
return table.concat(buffer)
end

Thereareacoupleinterestingthingshere:
The

opts argumentcaneitherbeastringliteraloratable.

WhenitsatableittakesadvantageofthefactthatLuatables
arebothhashtablesandarraysatthesametime.Thehash
tableportionholdstheattributesoftheHTMLelement,and

thearrayportionholdsthecontentsoftheelement.
Checkingifthekeyina

pairs iterationisnumericisaquick

waytoapproximateisolatingarraylikeelements.Itsnot
perfect,butwillworkforthiscase.

for k,v in pairs(opts) do

if type(k) ~= "number" then
-- access hash table key and values
end
end

Whenthecontentofthetagisinsertedintothebufferforthe
tablebased

opts ,thefollowinglineisused:

append_all(buffer, unpack(opts))

Luasbuiltinfunction

unpack convertsthearrayvaluesina

tabletovarargs.Thisfitsperfectlyintothe

append_all

functiondefinedabove.

Note:

unpack is table.unpack inLua5.2andabove.

Closing

ThissimpleimplementationofanHTMLbuilderthatshould
giveyouagoodintroductiontobuildingyourownDSLsin
Lua.
TheHTMLbuilderprovidedperformsnoHTMLescaping.Its
notsuitableforrenderinguntrustedinput.Ifyou'relooking
forawaytoenhancethebuilderthentryaddinghtml
escaping.Forexample:

local unsafe_text = [[<script type="text/javascript">alert('hack

render_html(function()
return div(unsafe_text)
end)

-- should not return a functional script tag:

-- <div>&lt;script type=&quot;text/javascript&quot;&gt;alert('ha

Herearesomemoreguidestagged'lua'
Howitch.iousesCoroutinesfornon-blockingIO
PostedJune09,2016

UsingLuaRockstoinstallpackagesinthecurrentdirectory
PostedJanuary28,2016

DynamicscopinginLua
PostedJanuary24,2016

CloningafunctioninLua
PostedJuly08,2015

ImplementingsetfenvinLua5.2,5.3,andabove
PostedJuly08,2015

Anin-depthlookintotheM oonScriptclassimplementation
PostedJuly05,2015

AnintroductiontoParsingExpressionGrammarswithLPeg
PostedJuly04,2015

UsingPostgreSQLwithOpenResty
PostedJuly04,2015

4 Comments

leaf o.net

Recommend

Share

Login

Sort byBest

Jointhediscussion
mrbngle 6daysago

Thanksforthisarticle.Itwasagreatinspiration,andwellachallenge,to
me.Ireallyhadtotrythisout.Ispendacoupleofhoursandendedupwith
this:https://github.com/bungle/lua...(itisimplementedinLua).

Reply Share

DamilareAkinlaja 2monthsago

IamtryingtoimplementthistutorialusingMoonScriptbutIamgetting
error
https://gist.github.com/darmie...

Reply Share

abcdef >DamilareAkinlaja 24daysago

setfenvdoesn'texistinLua5.2+,seethis:
http://leafo.net/guides/setfen...

Reply Share

daurnimat or 10 monthsago

Nicewriteup.Ididasimilarblogpostalongtimeback:
http://daurnimator.com/post/60...

Reply Share

ALSOONLEAFO.NET

Int roducingSt reakClubanddraw

wit hmef orayear

Howit ch.iousesCorout inesf or

nonblockingIO

1comment ayearago

6comments 5daysago

Wref Awesomefortworeasons.1)I

leaf o Itispossibletotriggertwo

couldreallyseemyselfusingthissite
formotivation.2)I'vebeenfollowing
moonscriptandlapis

thingsatonceinsideofopenresty,or
buildingsomethinginpureluathat
schedulestwo

MoonScript v0.2.3

2comments 3yearsago

leaf o Shouldbegoodnow,

accidentallyuploadedabuildofthe
manualusingoldbetaversionon
MoonScript.Thanks.(Tryahard

Anint roduct iont oParsing

ExpressionGrammarswit hLPeg
4comments ayearago

HallisonBat ist a Thanks!You

presentedtomePEGandPegineasy
way.:)

leafo.net2015GeneratedThuJun9 13:44:582016bySitegen