Creating a Domain Specific Language (DSL) with Xtext

Arjan Mooij, Jozef Hooman

v2.0, 2013-10-11

1 Introduction
This report explains the use of Eclipse and the Xtext plug-ins for the definition and use of a Domain
Specific Language (DSL). Section 2 describes the steps needed to install Eclipse and all required plugins. The creation of a new DSL is explained in Section 3. Section 4 demonstrates how an existing DSL
can be loaded and adapted. Section 5 contains a small example illustrating the extension of scoping
rules. Background information and various small topics can be found in Section 6. The creation and
use of plug-ins is described in Section 7. Finally, Section 8 contains a number of useful commands
and shortcuts.
Note that a number of screenshots are based on earlier Eclipse versions and might be slightly
different from the current version.

2 Install Eclipse and the required plug-ins

1. Install Java Runtime Environment (JRE), see
http://www.oracle.com/technetwork/java/javase/downloads
2. Install Eclipse IDE for Java EE Developers (http://www.eclipse.org/downloads/). Currently,
this is called Eclipse Kepler. For Windows, 32-bits, file name: eclipse-jee-kepler--win32.zip;
for 64-bits, file name: eclipse-jee-kepler--win32-x86_64.zip
3. Unzip the Eclipse file1 and locate the eclipse.exe file in directory eclipse; send it to
desktop (create a shortcut). If there are multiple JREs on the machine, one can specify the
JRE to be used as follows: right-click on Eclipse shortcut, select Properties, tab Shortcut; edit
the Target field to specify the location of the Java Virtual Machine. For instance:
C:\Eclipse\eclipse-jee-kepler-RC3-win32\eclipse\eclipse.exe -vm "C:\Program
Files\Java\jre6\bin\javaw.exe"
Note the use of javaw to avoid an additional window.
4. Create a Workspaces directory, with a [temporal] directory ExampleWorkspace, with an
empty subdirectory ExampleMeta (for the meta-level). Open Eclipse and select workspace
ExampleMeta. (These directories can also be created when opening Eclipse and pushing the
Browse button.)
[Close Welcome screen; use Help > Welcome if you would like to get it back]
1

The Windows 7 zipper might create problems, use 7zip instead.

2
5.

Install Ecore Tools SDK (Help > Install new software > Work with (pull-down menu): select
Kepler - http://download.eclipse.org/releases/kepler
Wait till everything is loaded under Name ,
open Modelling, select Diagram Editor for Ecore (SDK) and Xtext SDK
open Programming Languages, select Xtend SDK
Click Next (2 times), accept license agreement, and Finish; restart Eclipse when asked.

3 Create and use a new DSL

This section illustrates how a new DSL can be defined and used.

3.1 DSL definition at the meta-level

Create a new directory XtextBasicExample with an empty subdirectory

XtextExampleMeta and open Eclipse with workspace XtextExampleMeta.
[Close Welcome screen (use Help > Welcome to get it back)]

3.1.1 Create Xtext project

Create new Xtext project:

o File > New > Other > select: Xtext, choose: Xtext Project
o Give it a name, e.g. nl.xtext.demo, choose Language name, e.g.
nl.xtext.demo.MyDsl (note that last word should start with a capital following java
conventions) and Extensions: e.g. dsl
o Finish (use default location and generator configuration) [it takes some time to
create a new project]
Leads to three projects:
o nl.xtext.demo: Contains the grammar definition and related all components such as
parser, lexer, linker, validation, etc.
o nl.xtext.demo.tests: Unit tests go here.
o nl.xtext.demo.ui: The Eclipse editor and other workbench-related functionality. The
generated project does not contain any functionality, but in this project advanced
functionality can be implemented, such as content assist, outline tree, and quick fix
functionality.

Figure 1 Initial Xtext file

See nl.xtext.demo, folder src , package nl.xtext.demo:

GenerateMyDsl.mwe2: some kind of build file
MyDsl.xtext: defines the grammar, contains some default example initially, as shown in
Figure 1.

3.1.2 Version management and export

The project structure of the previous step is very suitable for version management. There is no need
to archive the large .metadata folder.
To use subversion management, install the next plug-in:
6. Install SubClipse (Help > Install new software > Work with:
http://subclipse.tigris.org/update_1.8.x
Click on Add, give name, for instance, SubClipse.
Wait till everything is loaded under Name , select Subclipse .
Click Next (2 times), accept license agreement, and Finish; restart Eclipse when asked.
To check out an existing project from an SVN:
File > Import > SVN > Checkout Projects from SVN
If the SVN location is already listed: Use existing repository location, select required part,
etc.; otherwise, Create a new repository location and select URL
To update project from existing SVN location:
right-click on project, select Team > Update to HEAD
when needed, regenerate files as described in later sections
To create an SVN repository for an existing project:
Right-click on project, select Team > Share Project
To upload project updates to SVN:
right-click on project, select Team > Commit

Eclipse also offers an export mechanism via File > Export > General > Archive file, which includes the
metadata. Smaller archive files can be obtained by compressing all folders except the .metadata
folder. Section 4 explains how to import archive files.

3.1.3 Construct grammar of DSL

Change MyDsl.xtext in nl.xtext.demo/src /nl.xtext.demo to the grammar depicted in Figure 2.

Figure 2 Definition of grammar in Xtext

This defines a grammar to describe robot applications by means of a number of production rules:

The production rule for Robot allows a repeated (indicated by *) choice (indicated by |)
between messages and tasks. The notation += indicates a list of elements. Note that Robot
may be empty, if we prefer at least one message or type then we should replace * by +.
A message starts with the terminal string Display and then defines attribute msg.
A task starts with terminal string Task name, then a name attribute is defined, followed by
an optional (indicated by ?) attribute frequency, and next a positive number (indicated by
+) of moves can be added.
Similarly, the production rule for a move is defined.
Finally, a condition is defined as an enumeration.

Generate parser, serializer, and other infrastructure code:

Generate the language infrastructure by Run > Run Configurations ;
click on + before Mwe Launch and double-click on Generate Language Infrastructure as
depicted in Figure 3.

Figure 3 Generate Language Infrastructure from Run Configurations

Answer y to question about download of ANTLR 3 parser generator, see Figure 4.

If downloading the file gives problems with the proxy server:
o download the file http://download.itemis.com/antlr-generator-3.2.0.jar
o store it as . antlr-generator-3.2.0.jar in directory nl.xtext.demo of workspace
XtextExampleMeta (note the dot before the name).

Figure 4 Generate infrastructure

[An alternative is to right-click on src/nl.text.demo/GenerateMySD.mwe2 -> Run as -> MWE2

Workflow]

6
Note that after the first generation of the infrastructure, the application is available in the pull-down
menu right of the green play button:

Ecore model and diagram:

The ecore model MyDsl.ecore, which has been generated from the Xtext grammar, can be
found in folder model/generated. Often the meta model is called the abstract syntax,
whereas the grammar is the concrete syntax.
A diagram can be generated by right-clicking on the ecore model and selecting "Initialize
Ecore Diagram File". This leads to MyDsl.ecorediag in the same folder. A little bit of editing
leads to the diagram of Figure 5.

Figure 5 Ecore diagram

Observe that the ecore model is more liberal than the concrete syntax by allowing a Task with zero
Moves.
View diagram of Xtext grammar:
Open the .xtext file and select: Window > Show View > Other... > Xtext > Xtext Syntax Graph. This
leads to a graphical representation of the syntax, as depicted in Figure 6.

Figure 6 Graphical representation of the grammar

Double-click on an element in the diagram to jump to the corresponding part in the Xtext syntax.

3.1.4 Runtime workspace

Restart Eclipse on workspace XtextExampleMeta. Next open the runtime workspace, which can be
launched from the meta-level workspace by clicking Launch Runtime Eclipse on the down arrow
right of the green button; see the next figure.

3.2 DSL usage in the runtime workspace

To construct concrete models, work in the runtime workspace.
Create a new project (File > New > Project, select General > Project)
o Give name, e.g.: nl.xtext.demo.spec (use default location); Finish
Create a new folder: New > Folder, parent: nl.xtext.demo.spec
o Give it a name, e.g., name: Robot1
Create a new file in folder for concrete model right click on folder:
o File > New > File

Parent: nl.xtext.demo.spec/Robot1

8

File name: spec1.dsl (choose extension .dsl specified before)

Finish, after question: Do you want to add the Xtext nature to the project
nl.xtext.demo.spec? answer: Yes

Edit spec1.dsl
o During editing, Ctrl-Space gives possible completions or inserts a keyword if there
is no choice.

Figure 7 gives an example of a concrete model instance spec1.dsl of the DSL.

Figure 7 Model instance

To view this model as an instance of the ecore model, right click on spec1.dsl, select Open With /
Other , choose (from Internal editors): Sample Ecore Model Editor. Right click on an element and
select Show Properties View to inspect details, such as the value of Speed attribute.
To return to the DSL view, right click on spec1.dsl and select Open With / MyDsl editor.

3.3 Model transformation (e.g., to generate code or text) with Xtend

As an example, the models expressed in the DSL are transformed to plain text.

3.3.1 Define the model transformations at the meta level

Close runtime workspace, start working in meta-level workspace XtextExampleMeta.

Open the file MyDslGenerator.xtend in nl.text.demo/src/nl.xtext.demo.generator. The
generated file is shown in Figure 8.`

Figure 8 Generated part of model transformation

Change this file into the text depicted in Figure 9 and Figure 10, taking the following notes account:
The three imports of nl.text. have been added automatically using hints on errors; when
an error occurs, hover over the erroneous text and select a suggestion to add an import
from nl.xtext.demo.myDsl.
The brackets << >> can be obtained by typing ctrl-space (pressing both keys
simultaneously).
The first editing step is to implement the doGenerate method. We start with the definition of the
path where the generated file should be placed. Note that resource.getURI().lastSegment yields
the last part of the name of the URI of the resource, which is in this case the name of the source file.
Next we define the name of the generated file and use a function toText to transform the resource
to sequence of characters. In Eclipse, a resource is a project, a folder, or a file.
Function toText is defined using a so-called rich string (the part between the triple quotes . ).
Note that the text on the blue background occurs in the generated file, including all spaces, together
with the values of the attributes specified between the angle brackets <<>>.
Since the function toText for resources refers to a similar function for Robots, we continue with
the definition of this function and a few other auxiliary definitions, as show in Figure 10.
Note that non-existing optional attributes of type INT have the default value 0, whereas non-existing
references have the null value.

10

Figure 9 First part of model transformation

11

Figure 10 Second part of model transformation

File MyDslGenerator.xtend is translated into Java, see file MyDslGenerator.Java in project

nl.text.demo, folder xtend-gen, package nl.xtext.demo.generator.

3.3.2 Use the model transformations in the runtime workspace

Open the runtime workspace (via the menu right of the green button). Since we already have a
concrete model (spec1.dsl), we can generate a text file from it by running from the menu:
Project > Clean (Clean all projects)
Note that the first time folder src-gen is generated. Part of the output is shown in Figure 11.

12

Figure 11 Result of model transformation

3.4 Changing files at the meta-level

Typically, a DSL is defined in an iterative way.
After changes in .xtext file, regenerate the language infrastructure and restart the runtime
workspace:
o Click the down arrow right of the green "Run" button
o Select Generate Language Infrastructure (nl.xtext.demo)
o Restart the runtime workspace and perform Project > Clean
After changes in .xtend file (or any file it uses), restart the runtime workspace:
o Click the down arrow right of the green "Run" button
o Launch Runtime Eclipse

3.5 Using debug mode

While developing a DSL, debugging facilities can be obtained by opening the runtime workspace in
debug mode, using the down arrow of the green bug icon, left of the green button used before;
see the top left of Figure 12.

13

Figure 12 Launch runtime in debug mode

In this mode it is possible to place breakpoints in files of the meta-level workspace. For instance, in
the MyDslGenerator.xtend file which defines the model transformation. When, after a Project >
Clean or a change and save in the runtime workspace, a breakpoint is hit, there will be a question at
the meta-level workspace whether the Debug perspective has to be opened; answer yes. In this
perspective, for instance, tracing information and values of variables can be inspected.
In the debug mode, changes in the body of methods are immediately available at the runtime level.
So avoids time-consuming restarts of the runtime workspace. This is still needed for larger changes,
e.g., when adding methods, but then the user is warned to restart.
To return to the usual view, select the JavaEE perspective.

3.6 Discussion on alternatives

3.6.1 Use of enumeration type
Observe that in the grammar of Section 3.1.3 the optional attributes start and end are defined as
a Condition, which refers to a ConditionValue, which is an enumeration type. It might be tempting
to remove this indirection and try the syntax of Figure 13.

Figure 13 Use of enum type for optional attributes

Disadvantage of this grammar is that if the optional attributes start and end are no longer null
when they are not specified. Unspecified attributes obtain value border which corresponds to 0,
the value of the first field of the enum type Condition. Hence it is impossible to distinguish an
unspecified attribute from one with value border.
A possible solution could be to add a new first default value to the enum type and test on this value
in the MyDslGenerator.xtend file. A more general approach is to ensure that optional fields always
refer to a non-enum type. Then the xtend file for the generation can always test on null for
optional attributes. This approach has been used in Section 3.1.3. An alternative is to remove the
enumeration type completely, as shown in Section 3.7.

14

3.6.2 Parsing rules have to assign a result

Figure 14 shows an attempt to define the choice between Messages and Tasks.

Figure 14 Alternative grammar for the choice between Messages and Tasks

The grammar leads to error: An unassigned rule call is not allowed, when the 'current' was already
created." The problem is that in parser rules it is not allowed to call other parser rules without
assigning the outcome to something. Note that each rule leads to class in the meta model, and
classes have to be linked somehow; in the grammar above no link is specified between Robot and
the classes Messages and Tasks.
A correct alternative with the choice between Messages and Tasks is shown in Figure 15. It
introduces an intermediate type Element.

Figure 15 Alternative grammar

15
This leads to the ecore diagram of the meta model of Figure 16 (after some minimal editing). Note
that the fact that an Elements is a Message of a Task is represented by inheritance.

Figure 16 Meta model corresponding to the alternative grammar of Figure 15

3.7 Object instantiation

Observe that adding a key word to Robot grammar, as follows:

leads to a warning: The entry rule 'Robot' may consume non empty input without object
instantiation. Add an action to ensure object creation, e.g. '{Robot}'.
The warning disappears indeed by adding the action (or removing the key word):

These actions can also be used to create subtypes. For instance, consider the alternative definition
of Condition of Figure 17.

16

Figure 17 Alternatives with subtypes

The corresponding meta model is shown in Figure 18.

Figure 18 Meta model using condition subtypes

Note that the same meta model is obtained by the following grammar:

The subtypes of Condition can be used to obtain polymorphic behavior using the ``dispatch feature
of xtend. For instance, in MyDslGenerator.xtend the conditions can be transformed to text as
follows:

17

4 Consider an existing DSL

In this section we consider an existing DSL. The steps to import an existing meta-level workspace and
runtime workspace are listed in Section 4.1 and Section 4.2, respectively. The content of the metalevel workspace is explained in Section 4.3. The runtime workspace is described in Section 4.4.
Section 4.5 summarizes what to do after changes.
Create a directory ExprExample with an empty subdirectory ExprExampleMeta (for the metalevel). Open Eclipse and select workspace ExprExampleMeta.

4.1 Import archive and generate language infrastructure

1. Get file expr.zip, e.g. from http://www.cs.ru.nl/~hooman/DES/expr.zip
2. File > Import > General > Existing Projects into Workspace ; Next
3. Select archive file > expr.zip
4. Below Projects: select all projects
5. Finish; wait till "Building workspace" has finished (see indicator at bottom right corner).
Note that the created project may contain various warnings
6. Generate the language infrastructure by Run > Run Configurations ;
click on + before Mwe Launch and double-click on Generate Language Infrastructure, as
shown in Figure 3. If a pop-up complaining about errors appears: Proceed
[An alternative is to right-click on src/statediag.example/GenerateMySD.mwe2 Run as ->
MWE2 Workflow]
Note that after the first generation of the infrastructure, the application is available in the
pull-down menu right of the green play button (see Figure 19 in Section 4.2).
7. Restart Eclipse with workspace ExprExampleMeta
8. Project > Clean... ; Clean all projects
9. Wait till "Building workspace" has finished (see indicator at bottom right corner)
All errors should be removed
Note: if not all errors are removed (e.g., errors in the ui project are reported), remove the .antlrgenerator-3.2.0.jar file in the directory of the main project (here statediag.example.jh), and
generate the language infrastructure again. Answer y as in Figure 4.

4.2 Start and load the runtime workspace

1. Get file exprruntime.zip, e.g. from http://www.cs.ru.nl/~hooman/DES/exprruntime.zip
2. Click the downarrow right of the green "Run" button, see Figure 19.

18

Figure 19 Launch runtime workspace using green Run button

3. Click on Launch Runtime Eclipse

4. Close Welcome screen and set up the new workspace, which is called "runtimeEclipseXtext", from the zip file as follows:
a. File > Import > General > Existing Projects into Workspace; Next
b. Select archive file > exprruntime.zip
c. Select all projects
d. Finish
5. Note that spec1 contains an error, which will be explained in Section 4.4.

4.3 Browse meta-level workspace

Consider the workspace of the meta model, see the next figure.

19

4.3.1 Syntax definition and model transformation

Consider the .xtext and .xtend files used to define the DSL and model transformation.
Grammar in Xtext
In statediag.example.jh /src /statediag.example the file MySD.xtext contains the grammar which
defines the syntax of the DSL. First a predefined set of terminals is imported, see Figure 20.

20

Figure 20 Predefined xtext terminals

Note that this example uses the notation [State | ID] to define a reference to an instance of type
State, using predefined terminal ID. This terminal is the name of the state, which is used to identify
the State instance. It is recommended to make sure that the type to which the reference is made has
an attribute name. Note the difference with attributes of type State, such as in the following
fragment:

This leads to an ecore diagram with a containment relation, as shown in the next figure:

If we use references to type State, as in the next fragment:

21
then we obtain the following ecore diagram:

The example also contains a definition of expressions; more explanation can be found in Section
6.2.1. Finally, the Xtext file shows the definition of a terminal type for booleans. See Section 6.2 for
more information about Xtext.
Model transformation using Xtend
The code generation is specified as model transformations in the file MySDGenerator.xtend in
statediag.example.jh/src/statediag.example.generator. This example shows the generation of
multiple files, a few auxiliary functions which show alternative ways to obtain instances of attributes,
and the translation of expressions.
Section 6.3 contains some background information about Xtend.
The MySDGenerator.xtend file is translated to Java file MySDGenerator.java in folder xtend-gen,
package statediag.example.generator. Observe, for instance, how the dispatch mechanism of Xtend
(applied to expressions) is translated to Java.

4.3.2 Generated files

In project statediag.example.jh the folders src-gen and xtend-gen have been generated; this also
holds for the projects statediag.example.sdk, statediag.example.tests, and statediag.example.ui.
Consider a few of the generated files:

The ecore model MySD.ecore can be found in statediag.example.jh /src-gen

/statediag.example; background information about Ecore models can be found in Section
6.1. Since this DSL was made with an earlier version of Xtext, the ecore model is at a
different location compared to new DSLs in the current version (see Section 3.1.3).

Note that a diagram can be generated by right-clicking on ecore model and selecting
"Initialize ecore_diagram diagram file"; Finish. Using the defaults this leads to a model
MySD.ecorediag in src-gen. This option might help to understand the DSL definition; the
diagram is not used in the remainder of this section.

In the file src/statediag.example/GenerateMySD.mwe2 the line

composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
leads to a check to validate that no two objects of same type can have the same name.
More info about options in the .mwe2 file can be found in section 6.5.1.

22

4.3.3 Validation
Starting point of validation is the file <LanguageName>JavaValidator.java in subpackage validation.
This file is generated automatically. To distribute the validation over a number of files or to be able
to use the Xtend language to define validation checks, proceed as follows:

Create several Java or Xtend classses with validation checks.

Link these classes by an inheritance chain. One class inherits from

Abstract<LanguageName>JavaValidator, the others inherit from each other. The last one in
the chain is <LanguageName>JavaValidator.java.

Note that Xtend and Java files can be mixed, but the starting point should be the generated Java file.
In our current example, this leads to the following files:

MySDJavaValidator.java in package statediag.example.jh/src/statediag.example.validation is

the starting point; this generated Java file is modified such that it extends the Xtend file
Validate3TypeChecking. (The default text is commented out.)

Validate3TypeChecking.xtend checks that expressions have the right type. It uses

Validate3TypeCheckingHelper.xtend and extends Validate1Cycles.xtend.

Validate3TypeCheckingHelper.xtend defines the type check rules for expressions. It extends

TypeChecker.xtend.

TypeChecker.xtend defines a number of general type checking patterns.

Validate1Cycles.xtend checks whether there is a cycle in the definitions of abbreviations; it

uses Validate1CyclesHelper.xtend.

Validate1CyclesHelper.xtend contains a number of definitions to detect cycles in

abbreviations.

A validation class consists of a number of methods, which in Java have the form:
@Check
public void anyName(SomeClassFromYourMetaModel instance) {

}
In Xtend these methods have the following form:
@Check
def anyName(SomeClassFromYourMetaModel instance) {

}
The method name anyName is completely free; the @Check takes care that the method is called
on every instance of type SomeClassFromYourMetaModel.
Since validation takes place while the user is typing a concrete model instance, it is important to take
into account that most types may have the null value (this does not hold for lists). For these types,
always test on null values in the validation checks to avoid exceptions on the console. See, for
instance, method checkTypeTransitionGuard in Validate3TypeChecking.xtend. Also cyclic relations

23
(e.g., a parent relation) may give problems (call stack overflow), so it is good practice to be robust
w.r.t. cyclic relations.
To report [potential] faults to the user, in the body of a validation check three pre-defined methods
can be called (using the standard inheritance relation with Abstract<LanguageName>JavaValidator):

info(): leads to an i in the margin of the instance editor;

warning(...): leads to a yellow exclamation mark in the margin and text in the instance is
underlined yellow;

error(...): leads to a red cross in the margin and text in the instance editor is underlined red;
this also ensures that no code can be generated.

Concerning the parameters of these three methods, there are a number of patterns to indicate the
location in the instance editor. We list a number of convenient parameters:

error("Error message", null)

: the error is reported on the full instance of the type
mentioned in the corresponding @Check method;

error("Error message", Literals.SOME_CLASS_FROM_YOUR_META_MODEL__SOME_FIELD) :

the error is reported for the field someField of the instance of the type mentioned in the
corresponding @Check method;

error("Error message", Literals.SOME_CLASS_FROM_YOUR_META_MODEL__SOME_FIELD,

index) : the error is reported for occurrence index of the field someField of the
instance of the type mentioned in the corresponding @Check method (useful if someField
has a list type);

error("Error message", instanceOfSomeEObject, null, -1)

full instanceOfSomeEObject;

: the error is reported on the

Combinations of the above patterns and many other possibilities are also available.
To obtain fragments such as Literals.SOME_CLASS_FROM_YOUR_META_MODEL__SOME_FIELD:
Add an import, depending on the language used:
o In an Xtend file: add an import of the form: <LanguageName>Package$Literals;
in our example: import statediag.example.mySD.MySDPackage$Literals
o In a Java file: add an import of the form: <LanguageName>Package.Literals; in
our example: import statediag.example.mySD.MySDPackage.Literals
On the place where the literal is needed, type Literals. and use content assist (<CTRL><SPACE>) to select one of the options of type EReference. (For instance, in our example:
Literals.ACTION__LAMP)
In our example, Literals can be found in src-gen/statediag.example.mySD/MySDPackage.java/
Literals.

24

4.4 Browse runtime workspace

Next consider the runtime workspace, launched from the meta-level workspace by clicking Launch
Runtime Eclipse on the down arrow right of the green button; see Figure 19. This opens the runtime
workspace (also called runtime workspace), as shown in Figure 21.

Figure 21 Runtime workspace state diagram example

The .sd files in statediag.example.spec/RobotSpecs contain a few instances of the DSL:

o spec1.sd contains a number of errors detected by the validation rules explained
above.
o spec2.sd contains a correct instance. When this file is selected, command
Project/Clean applies validation and generates the files as specified by the
MySDGenerator.xtend file. During editing, code generation is invoked automatically
when saving.
Experiment with these files to test validation and text generation. Folder src-gen contains
the generated code.
Create a new specification in a file spec3.sd; when pop-up Add Xtext Nature appears,
answers Yes to question to add the Xtext nature to the project.
WARNING: If there are multiple correct source files (*.sd), Project > Clean invokes code
generation on each source file (and may overwrite output).
To view a model as an instance of the ecore model, right click on a .sd file, select Open
With / Other , choose (from Internal editors): Sample Ecore Model Editor.
To return to the DSL view, right click on the .sd file and select MySD editor.

4.5 Changing files at the meta-level

After changes in .xtext file or the .mwe2 file, regenerate the language infrastructure and
restart the runtime workspace:
o Click the down arrow right of the green "Run" button
o Select Generate Language Infrastructure (statediag.example.jh)
o Restart the runtime workspace
After changes in .xtend file, restart the runtime workspace

4.6 Create archive file

We list the steps to create an archive (zip) file at the meta level:

25
File > Export
Select General > Archive File; Next
Select all projects
Browse to specify archive file at appropriate location
Finish
Similarly, an archive of the runtime workspace can be generated.

5 Example Scoping
This example is based on three files: MyDsl.xtext, MyDslScopeProvider.xtend, and test.mydsl. The
text of these files can be found in file ScopingFiles.txt and can be copied from this file.

5.1 Meta-level workspace

Create a new directory XtextScopingDemo with an empty subdirectory
XtextScopingDemoMeta and open it with Eclipse.
Create new Xtext project:
o File > New > Other > select: Xtext, choose: Xtext Project

Project name: nl.xtext.scoping.mydsl

Language name: nl.xtext.scoping.mydsl.MyDsl

Extensions: mydsl; Finish
Edit file MyDsl.xtext (in nl.xtext.scoping.mydsl/src/nl.xtext.scoping.mydsl), e.g. by copying
from ScopingFiles.txt, as in Figure 22.

Figure 22 DSL to illustrate scoping

Generate the language infrastructure by Run > Run Configurations ; click on + before Mwe
Launch and click on Generate Language Infrastructure (see Figure 3 in section 4.1).
Answer y to question about download of ANTLR 3 parser generator.
Note that this grammar contains references to earlier defined rules: the extends attribute of a
Model may be a reference to a Model and the variable attribute of VariableUse may refer to a

26
Variable. This can also be seen in the Ecore diagram that can be generated by right-clicking on the
MyDsl.ecore file in nl.xtext.scoping.mydsl/model/generated and selecting Initialize Ecore Diagram
File. See Figure 23, which has be edited slightly:

Figure 23 Ecore diagram of DSL to illustrate scoping

5.2 Runtime workspace

Restart Eclipse on workspace XtextScopingDemoMeta.
Clicking Launch Runtime Eclipse on the down arrow right of the green button.
Construct a concrete model:
Create a new project (File > New > Project, select General > Project)
Give name: nl.xtext.scope.spec (use default location); Finish
Create a new folder: New > Folder, parent: nl.xtext.scope.spec
Give it a name: Demo
Create a new file in folder for concrete model right click on folder: File > New > File
Parent: nl.xtext.scope.spec/Demo
File name: test.mydsl
Finish, after question: Do you want to add the Xtext nature to the project
nl.xtext.scope.spec? answer: Yes
Edit test.mydsl as in Figure 24 (or copy from ScopingFiles.txt).

27

Figure 24 Test model for scoping

Note there is an error Couldnt resolve reference to Variable x. Use ctrl-space after
occurrences of use in the file to observe which variables are in scope.

5.3 Scoping in meta-level workspace

To resolve the error, we extend the scoping rules (which now only resolve local references) in the
meta-level workspace. Close the runtime workspace.
Open file MyDslScopeProvider.xtend in nl.xtext.scoping.mydsl/src/nl.xtext.scoping.mydsl.scoping
and edit (e.g., by copying from ScopingFiles.txt), is as in Figure 25.
The added implementation of MyDslScopeProvider computes the scope of variables using the
extends relation in the DSL. The name scope_VariableUse_variable indicates that it concerns the
scope of variable in the VariableUse rule. The model parameter is the first node higher in the
parse tree than VariableUse that may contain useful declarations. Variable visited ensures
termination if the extends relation contains a cycle.

28

Figure 25 Scoping definition

5.4 Scoping in runtime workspace

Open the runtime workspace by Launch Runtime Eclipse in the pull-down menu right of
the green button.
Execute Project > Clean
Observe that the error is no longer available in test.mydsl and by Ctrl-space one can observe
which variables are in scope.

6 Background information
6.1 Ecore information
The Eclipse Modeling Framework (EMF) is a modeling framework and code generation facility for
building tools and other applications based on a structured data model. The core EMF framework
includes a meta model (Ecore) for describing models. A meta model is a model of models, that is,
instances of the meta model are models.

29
The Ecore model allows the definition of a number of elements such as:

EClass : represents a class, with zero or more attributes and zero or more references.

EAttribute : represents an attribute which has a name and a type.

EReference : represents one end of an association between two classes. It has flag to
indicate if it represents a containment and a reference class to which it points.

EDataType : represents the type of an attribute, e.g. int, float or java.util.Date

Note that EMF also contains a meta model called Genmodel which contains additional information
for the code generation, e.g. the path and file information.
Info:

Tutorial from Lars Vogel: http://www.vogella.com/articles/EclipseEMF/article.html

6.2 Xtext information

Xtext defines the grammar of the concrete syntax of the DSL language, i.e., the legal sequences of
terminal symbols (alphanumeric characters, punctuation marks, and white space characters). These
sequences are defined by means of a set of production rules for non-terminal symbols, using
Extended Backus-Naur Form-like (EBNF) expressions (e.g., using | for the choice between
alternatives). There are four different possible cardinalities in these expressions:

exactly one (the default, no operator)

one or none (operator ?)

any (zero or more, operator * )

one or more (operator +)

Assignments are used to assign the consumed information to a feature of the currently produced
object. The type of the current object, its EClass, is specified by the return type of the parser rule. If
it is not explicitly stated it is implied that the type's name equals the rule's name. The type of the
assigned feature is inferred from the right hand side of the assignment.
There are three different assignment operators for features, each with different semantics.

The simple equal sign = is the straight forward assignment, and used for features which take
only one element.

The += sign (the add operator) is used for a multi-valued feature (i.e., a list of values) and
adds the value on the right hand to that feature. Since attribute_name += ... is translated
in the meta model to a list, it is convenient to use a plural term (e.g., values) for
attribute_name. Similarly, it is a good convention to use a singular term (e.g., name) for
the attribute_name in grammar fragment attribute_name = ....

The ?= sign (boolean assignment operator) is used for a feature of type Boolean.

30

6.2.1 Expressions in Xtext

We list three patterns to define expressions, depending on the desired associativity.
Left Associativity (left-to-right evaluation):
ExpressionLevel1 returns Expression:
ExpressionLevel2
( ({ExpressionAnd.left=current}
| ({ExpressionOr.left=current}
)*
;
Right Associativity (right-to-left evaluation):
ExpressionLevel1 returns Expression:
ExpressionLevel2
( ({ExpressionAnd.left=current}
| ({ExpressionOr.left=current}
)?
;
No Associativity:
ExpressionLevel1 returns Expression:
ExpressionLevel2
( ({ExpressionAnd.left=current}
| ({ExpressionOr.left=current}
)?
;
Info:

a && b && c = (a && b) && c

"&&" right=ExpressionLevel2)
"||" right=ExpressionLevel2)

a && b && c = a && (b && c)

"&&" right=ExpressionLevel1)
"||" right=ExpressionLevel1)

a && b && c is not allowed

"&&" right=ExpressionLevel2)
"||" right=ExpressionLevel2)

Eclipse: Help/Contents/Xtext 2.3 Documentation, see also

http://www.eclipse.org/Xtext/documentation.html

6.3 Parsing
Xtext uses ANTLR (http://www.antlr.org/) which is a LL(*) recursive-descent parser generator. See,
e.g., http://en.wikipedia.org/wiki/LL_parser . LL parsers do not allow left recursive grammars, see
the Xtext 2.3 documentation: p 67 on Assigned Actions and p 72, section 6.2.8.
Syntactic analysis takes place in three phases:
1. Lexing: a sequence of characters (the text input) is transformed into a sequence of so called
tokens (such as ID, INTEGER, REAL, etc.);
2. Parsing: a sequence of tokens is transformed into a parse tree;
3. Linking: the parse tree is converted into a graph based on the cross-references.

31
These phases are executed sequentially and, hence, earlier phases cannot use knowledge of later
phases:
lexing has no information about the tokes that are expected by the parser;
parsing cannot use knowledge about possible cross references.
An example of an ambiguity in lexing, consider a grammar contains the following:
terminal INTEGER
returns ecore::EInt:
('0'..'9')+;
terminal REAL
returns ecore::EFloat:
('0'..'9')+ '.' ('0'..'9')+
the grammar uses . in another rule (which means that this also becomes a token)
This is ambiguous, because input 3.4 can be tokenized in two ways (even if the grammar rules out
certain combinations): as one token (REAL) or as three tokens (INTEGER; .; INTEGER). Xtext does
not give a warning in this case; it will be tokenized as a REAL. More problematic is the input 3.,
which will be parsed as a REAL (no backtracking) and hence yields an error.
As an example of an ambiguity in the parsing, consider:
Expression:
ExpressionConstant | ExpressionVariable;
ExpressionConstant:

variable = [Constant|ID] ;

ExpressionVariable:

method = [Variable|ID] ;

This is ambiguous because token input ID can be parsed in two ways, even if the declaration of the
ID clearly indicates, for instance, that it is a constant (so cross-referencing would excludes one of the
possibilities). In general, ambiguity of a grammar is undecidable. If backtracking is not enabled (see
section 6.5.1), there are limitations on the grammars and often Xtext reports ambiguities after
performing Generate Language Infrastructure. Since backtracking can have a negative impact on
performance and the clarity of error messages, it is advisable to avoid backtracking as much as
possible.

6.4 Xtend information

Xtend can be considered a simplified version of the Java programming language. For instance, many
type information can be omitted. It also has some additional features such as rich strings which
allow for readable string concatenation. They are denoted between triple quotes and may
contain expressions between angle brackets << >>. Inside these angle brackets, one can access
attributes of objects and use expressions like FOR and IF; see, for instance, Figure 9 and Figure 10.
The return type of a rich string is CharSequence.
Info:

Chapter 13 of Xtext 2.1 Documentation mentioned in previous section

Eclipse: Help/Contents/Xtend User Guide

http://www.eclipse.org/xtend/

32

6.5 Miscellaneous
6.5.1 Options in the .mwe2 file
A number of options can be enabled/disables in the .mwe2 file which can be found in the src
folder, in the package where also the .xtext file is located. We list a few options which are disabled
by default; they can be enabled by uncommenting the corresponding lines.

Default check on uniqueness of names, which is often convenient.

// composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"

Backtracking (enable only if really needed, see also section 6.3), below the line
fragment = parser.antlr.XtextAntlrGeneratorFragment {
// options = {
// backtrack = true
// }
It is also needed to add these lines to
parser.antlr.XtextAntlrUiGeneratorFragment{}

After a change, Generate Language Infrastructure has to be executed.

6.5.2 Compare Ecore models

To compare Ecore metamodels (diff, merge, etc.), there is a convenient tool, see:
http://www.eclipse.org/emf/compare/
It can be installed in Eclipse using:
Help > Install New Software
Work with: Juno - http://download.eclipse.org/releases/juno > EMF Compare
Select Modelling: EMF Compare
Next; Next; Accept license agreement; restart Eclipse
To use the tool, select two Ecore models (select first file, press and hold Ctrl key, select second file),
right-click on them and choose Compare With > Each Other.
(There is also a possibility to compare to other versions from a version management system .)

6.5.3 Java JRE version

Note that Eclipse only requires a Java Runtime Environment (JRE) because it comes with its own Java
compiler. So there is no need for a Java Development Kit (JDK).
If the JRE uses an older Java version than Eclipse, this might lead to an error message about
unsupported versions of the UI when opening a file in the runtime workspace. In this case , adapt
the following in the meta-level workspace:
Select one of the projects
Project -> Properties -> Java Compiler

33

Configure Workspace Settings

Java -> Compiler
Compler compliance level: 1.6
Then perform a Clean on all projects in the runtime workspace [although this may happen
automatically]

6.5.4 Create runtime workspace manually

Instead of using Launch Runtime Eclipse, one could create the runtime workspace as follows:
Eclipse menu: Run > Run configurations
Select Eclipse Application, click New button (may take some time)
Name: e.g., XtextExampleModels
Tab main, Location should be: ${workspace_loc}/../runtime- XtextExampleModels
Note: default is OK; this creates directory on same level as workspace of meta level, so it is
important to use different names.
Tab Arguments VM Arguments add -XX:MaxPermSize=512m
Click on Run  this opens new Eclipse environment for working on runtime level
Start XtextExampleModels from meta-level workspace using green play button.

7 Create and use plug-in

According to http://wiki.eclipse.org/FAQ_What_is_a_plug-in%3F, a plug-in is a component that
builds up an Eclipse application. It is an encapsulation of behavior and/or data that interacts with
other plug-ins to form a running program. Minimally, a plug-in consists of a bundle manifest file,
MANIFEST.MF. This manifest provides important details about the plug-in, such as its name, ID, and
version number. The manifest may also tell the platform what Java code it supplies and what other
plug-ins it requires, if any. Note that everything except the basic plug-in description is optional. A
plug-in may provide code, or it may provide only documentation, resource bundles, or other data to
be used by other plug-ins. A plug-in also typically provides a plug-in manifest file, plug-in.xml, that
describes how it extends other plug-ins, or what capabilities it exposes to be extended by others
(extensions and extension points).
More information about the plug-in.xml file can be found at:
http://wiki.eclipse.org/FAQ_What_is_the_plug-in_manifest_file_%28plugin.xml%29%3F
We describe the steps to create and use a plug-in for a DSL, based on:
http://charandeepmatta.com/2012/09/01/how-to-generate-eclipse-plug-in-for-your-dsl-written-inxtext/

34
Generating the target plug-in for distribution to the users of the DSL:
1. Start with a completed Xtext project. Note that it contains a project ending with .sdk. In
this project, there is a file feature.xml. Open this file and inspect the properties. Note that
tab Information allows entering description, license and copyright information.
2. Click on Update Site project under Publishing in the Overview tab of feature.xml
mentioned in the previous step. [Or create an update site project using: File > New > Other
> Plug-in Development > Update Site Project Next ]
Give project a name, for instance, MyDsl-Update-Site Finish
3. This will open site.xml (of map My-Update-Site) in the Site Map editor.
Choose New Category and give it a name, e.g. My Dsl with ID: my_dsl
[This step is optional, without it, the feature will be Uncategorized.]
Select category my_dsl. Choose Add Feature, scroll through the list and add
your dsl project package ending with sdk ; select Ok.
Select Build All - as an example, see Figure 26.

Figure 26 Create update site

This will package your dsl project plug-ins inside this update site project MyDsl-Update-Site. This
folder MyDsl-Update-Site can now be used directly as a local update site for your eclipse users.
Alternatively, the folder can be put on a web-site.
Using the plug-in:
1. Create a new folder for the runtime workspace (e.g. ExprDevelopment)
2. Open Eclipse with this workspace
3. In Eclipse, go to Help > Install New Software
In Work with: , click the Add button. The Add Repository dialog box appears:

35

Two options:
1. Click Local and select the directory of the update site, and then click OK.
The path of the directory appears in the Location field. Leave the Name
field empty.
2. Type in Location the URL of the update site and click OK.
From the display select My Dsl (or Uncategorized if no category was made).
Click Next and then Finish to complete the installation.
Click OK on security warning
When asked, restart Eclipse.
4. Generate a new project, folder, and start creating new Dsl models

8 Commands and shortcuts

8.1 Useful basic Eclipse shortcuts

Switch between tabs:

o <CTRL>-<Page Up>
o <CTRL>-<Page Down>
Maximize current editor (tab):
o <CTRL>-<M>
o Double click on tab
Jump to position of last edit
o <CTRL>-<Q>
Save file
o <CTRL>-S

8.2 Navigation

Outline view:
o Window -> Show view -> Outline
o <CTRL>-O
opens a quick outline that is searchable (including wildcards)
Hyper-linking to referenced elements:
o <F3> while the cursor position is at a cross-reference
o <CTRL>-click on a cross-reference
Find references:
o <CTRL>-<SHIFT>-<G>
Label (with extra information):
o Point at a cross-reference

36

8.3 Editing

Content assist:
(including templates for system and data/process/cluster classes)
o <CTRL>-<SPACE>
Rename element:
(including any cross-references)
o <ALT>-<SHIFT>-R
Automatic formatting: (pretty printing)
o <CTRL>-<SHIFT>-F

8.4 Fault detection and correction

Validation:
Automatically upon changes and upon saving
Revalidate all files:
o Project -> Clean...
Validation results:
o error:
icon: cross in a red box
o warning:
icon: exclamation mark in a yellow triangle
Quick fix
(available if a light bulb is displayed next to the validation icon)
o <CTRL>-<1>

Click on the validation icon