Noname manuscript No.

(will be inserted by the editor)

Reflection in Scala

Íñigo Mediavilla

15th November 2013

Abstract This article will explain the two main reflection APIs available for
Scala: runtime and compile-time reflection. It will describe the limitations of the
previously existent solutions and the goals that motivated them, their main fea-
tures as well as some of the challenges faced during their implementation. It will
end up with a review of their limitations.
Keywords Scala · Reflection · Macro

1 Introduction

Reflection has many definitions. According to [6]:

“ A reflective system is a causally connected meta-system that has as object-
system itself. [. . . ] When a system is reasoning or acting upon itself, we speak of
reflective computation. ”
This point of view is similar of that of metaprogramming that sees reflection as
”The ability of a programming language to be its own metalanguage” [1]. However
when talking about auto-adaptability the most appropiated sense of reflection
sees it as ”ability to manipulate as data something that represents the state of a
program during its execution”.
In the broader sense given by metaprogramming, any language that provides
a reified version of a program that can be manipulated by other programs written
in the same language provides reflection. In the more concrete sense given by the
second definition, reflection is restricted to the reflective manipulations that can
be done at runtime. Since reflection serves mainly as a mean for a program to
adapt its behaviour to the circumstances of its environment (auto-adaptability),
runtime reflection is preferred because at runtime programs have access to more
information about their environment. However, according to the definition given
by metaprogramming, reflection can also be done at compile time. One example
of compile time reflection being macros in statically typed languages like [9].

UPMC Master STL

E-mail: imediava@gmail.com
 2 Íñigo Mediavilla

Due to the more powerful nature of runtime reflection this article will focus
on the runtime reflection API provided by Scala. However in the end, this article
provides an introduction to Scala’s macro system, a facility for doing compile-time
reflection that although less powerful than its runtime counterpart, allows to solve
some of the problems that have been traditionally solved with runtime reflection,
but preserving type-safety.

2 Runtime reflection

2.1 Initial situation

Initially Scala didn’t have an specialized reflection API, however as a language that
was executed inside the JVM, Scala provided the possibility to use Java’s reflection
API. This possibility came with some serious deficiences. Java’s reflection API
had no understanding of Scala’s model what meant that Java’s reflection didn’t
provide means to manipulate scala-specific metaobjects. On top of that Java’s API
has limitations such as erasure of generic types at runtime or a design that lacks
encapsulation making it impossible to redefine its default implementation.

2.1.1 Limitations

Scala-specific objects As an evolution of Java, Scala comes with a richer ontology

model. Concepts like type-aliases, higher kinded types, modules and singleton
objects are scala-specific constructions that don’t exist in Java. A Scala developer
can manipulate them freely but at compile time these concepts are translated by
the Scala compiler to intermmediate representations. Java’s API has no notion of
them, therefore it provides no way for the developer to manipulate them.
As an example Figure 1 shows how it’s impossible for Java to find the type
Human even though from the perspective of Scala both types Person and Human
should be interchangeable since the latter is only a renaming of the former.

Fig. 1 Example of trying to instantiate a type alias with Java Reflection API.

1 package com.test
2 /* Example on trying to do a lookup for a type alias */
3 class Person(name:String)
4 type Human = Person
5 // The following throws ClassNotFoundException
6 val human = Class.forName("com.test.Human");

Inconsistencies Apart from having problems to understand scala-specific objects,

Java reflection presents an inconsistent view of a concept like inheritance in Scala.
Take the example shown in figure 2 that has been extracted from [7]. From
Scala’s point of view both since the class D extends C, any instance of D should
be seen as an instance of C. However, again due to Scala’s internal translations,
Java reflection returns false when asked if D can be seen as C.
 Reflection in Scala 3

Fig. 2 Example of trying to verify if two clases extend the same superclass.

1 class E {
2 type T
3 val x: Option[T] = None
4 }
5 class C extends E
6 class D extends C
7 val c = new C { type T = String }
8 val d = new D { type T = String }
9 val shouldBeTrue = c.getClass.isAssignableFrom(d.getClass)
10 assert(shouldBeTrue) // Throws AssertionError

Scala’s syntax In Scala’s unified model every operation is a call to a method.

Even primitive operators like the arithmetic operators + - are methods that can
be redefined for a class. This gives library constructs that cannot be differentiated
from the primitives provided by the language.
A method in Scala can have the name +, what is not allowed by Java since
this symbol represents a call to the addition primitive. The consequence is that
methods that have characters that are not allowed for Java need to be translated
by the Scala compiler. Therefore when manipulating this methods reflectively with
Java’s API, the Scala developer has to be aware of this name rewriting.

Erasure of generic types Finally if Scala wanted to provide a meaningful library

for doing reflection, it needed to overcome some intrinsic limitations of Java’s API.
The most significant of those limitations is the erasure of generic types at runtime.
In the case of Scala this problem is extremely relevant, since generic or parametric
types are used extensively in Scala code, not having this information available
when doing reflection represented a problem.
All this limitations required the definition of a reflection API particular to
Scala. The following sections will describe the principles that drove the implemen-
tation of this new API that is available for Scala 2.10.

2.2 Principles

The principles required for a good reflection library have been defined by [2] and
can be summarized in three: encapsulation, ontology correspondance and strat-
ification. This section will describe these principles, relating them to the initial
situation of Scala.

2.2.1 Encapsulation

This principle applies to reflection what is a good general programming practice:

the facilities to manipulate metaobjects must provide an interface that encapsulate
their implementation.
Reflection can be useful in a different number of scenarios. For every scenario
the requirements can be completely different. Most programming languages offer
an implementation for the basic scenario of programs that manipulate themselves
4 Íñigo Mediavilla

locally. However there might be other scenarios where reflection can be useful that
don’t fit exactly with the same requirements as the basic scenario. As an example,
reflection can be used for debugging or for doing instrospection in code executed
in a remote environment.
According to this principle the API that allows reflection should be defined in
a way that its implementation can be replaced transparently for its clients.
Java violates this principle by providing the meta-class Class as a class in-
stead of an interface making it difficult to redefine the implementation of the core
reflection package.

2.2.2 Stratification

Meta-level facilities must be separated from base-level functionality [2].

It is often useful to be able to disable the reflection from a language in certain
circumstances either for performance or security reasons. However as demonstrated
in [2], in general if reflective capabilities are part of a class that has other uses
than reflection, it is hard to safely remove those reflective capabilities from the
system.
An example of the lack of stratification appears in Java where any object can
call its getClass method to obtain its reflective representation.

2.2.3 Ontological correspondance

Structural correspondence: the structure of the metaobjects should correspond to

that of the objects of the language they represent, this is what makes what is
known as a mirrored system.
This problem of lack of ontological correspondance wasn’t of course a problem
in the design of Java’s reflection API, but in the fact that the reflection API for
Java was used for Scala a language with a richer ontology model.

2.3 Scala’s API

The designers of Scala’s API having these principles in mind designed an API with
the following properties:

– All the concepts available in the language have a representation on the meta-
language that can be analyzed through reflection.
– Access to the metaobjects is done through mirrors and not through the objects
themselves, thus preserving the principle of stratification.
– The metaobjects are represented by interfaces and not by classes what allows
for code that works for different implementations of reflection as long as they
respect these interfaces (encapsulation).

Along the section we will see examples of how to use the API and how the
principles described above are respected. We will also see how some of the limita-
tions like erasure of generic types are solved. However, it will start by describing
quickly the concepts of universe and mirror to then see how to use them to perform
introspection on the code.
Reflection in Scala 5

Universe is the entry point to the reflection API. It allows the developer to access
any element of the API. The concept of universe serves mainly to differentiate
between runtime and compile-time reflection. According to the required temporal
correspondance [2], reflection on computation and reflection on code needs to be
done separately. Universes enforce this restriction that is further emphasized by
the different operations available through their corresponding mirrors.

Mirrors are the objects that provide any information or operation that is defined
in the API. Access to mirrors is facilitated either by their universe in the case
of high level mirrors or by their parent mirror in the case of specialized mirrors.
When looking at the runtime reflection API the root mirror is the runtime mir-
ror, the ClassLoader mirror is accessed through the root and Invoker mirrors are
respectively accessed through the ClassLoader.

Scala’s reflection API uses mirrors as the principle for performing introspec-
tion. As we’ve seen previously an API based on mirrors provides like any other
reflection API access to representations of the entities of a programming language,
this representation allows to manipulate and extract information about the objects
represented. What makes a mirror API special is that the access to the representa-
tions is not done through the object themselves but through an external function
that given the object returns its representation. This kind of API has the ben-
efits that the meta-level operations are encapsulated from their implementations
and that base-level functionality has no knowledge of the reflection infraestructure.

In terms of implementation, Scala provides a uniform way to access the rep-

resentation of any object of the language. Under this uniform layer, Scala hides
two different implementations of the mirrors that are required due to the different
requirements for pure Java or Scala objects.

In the case of Java objects, Scala has no mean to extract any additional in-
formation to the one provided by the JVM and Java’s reflection API. In this case
Scala just provides a wrapper over Java’s reflection API in the form of a function
that takes a Java object and returns its mirror. This means that when a Scala
developer introspects an object which class belongs to a Java library, it will ob-
tain a mirror that is just a wrapper over the methods provided by java.lang.reflect.

In the case of Scala objects, the mechanism is a bit more complex. Since the
information provided by JVM’s bytecode is not complete enough, Scala’s manages
to insert additional information about the objects in the .class file by using the
fact that according to the Java bytecode specification [5] the compiler can generate
additional attribute in class files, that are ignored by the JVM. The Scala compiler
uses this right to insert an attribute ”ScalaSig” into the .class files generated
that contains many of the information needed by the reflection API, including
information about generic types.

2.3.1 Basic operations

Accessing a type A basic reflection operation is accessing the metaclass for a given
class. In most reflective languages metaclasses contain the methods that allow
 6 Íñigo Mediavilla

developers to do the interesting manipulations such as create instances or list the

members of the class. The equivalent to Java classes in Scala are types. Types in
Scala include not only classes but also traits that are similar to Java interfaces
but that allow the definition of implementations for its method. Obtaining a type
representation is done a bit differently depending on wether we want to do it by
specifying the type explicitly, if we want to obtain the type of an instance or if
we want to do it for a parametric type for which the generic type has not been
defined yet.
The first case is the simplest since it requires only a call to the method typeOf.

Fig. 3 Accessing the representation of a type

1 import scala.reflect.runtime.universe._
2 val mytype : reflect.runtime.universe.Type = typeOf[Set[Boolean]]

The second case is a bit different, let’s say that we want to define a method that
given an instance of any instance object it returns the representation of its type.
We’ve already seen that the method typeOf obtains the representation for a type
defined explicitly. However, how do we pass to typeOf the type of our instance?
Well, doing this in Scala requires just the definition of a polymorphic method
whose polymorphic parameter is the type of the object passed. The problem is
that this doesn’t work since in the JVM the information about generics is erased
at runtime. Fortunately Scala has found a way to overcome this limitation by
a technique that includes additional metainformation about generic types in the
.class file when requested by the code. The way to request that is to add the
implicit parameter TypeTag for the parametric type that we want to inspect at
runtime. When the compiler sees this implicit parameter in the signature of a
method it adds the needed information so it’s accessible at runtime.

Fig. 4 Accessing the type of an instance

1 import scala.reflect.runtime.universe._
2
3 object ReflectionUtils {
4 def getType[T: TypeTag](obj: T) = typeOf[T]
5 }
6
7 val mytype = ReflectionUtils.getType(List(1,2,3)) // contains type List[Int]

The third is case when we want to get the representation for a polymorphic
type that has a non-defined parameter (e.g List[A]) is a bit more complex, we will
not explain it here but it requires a different kind of tag called WeakTypeTag.

Create an instance of the type Once the representation for a type has been recov-
ered, we can do things like instantiating an object of a type as we can see in figure
5
 Reflection in Scala 7

Fig. 5 Instantiating an object

1 scala> val ctor = typeOf[Person].declaration(ru.nme.CONSTRUCTOR).asMethod

2 ctor: reflect.runtime.universe.MethodSymbol = constructor Person
3
4 scala> val m = ru.runtimeMirror(getClass.getClassLoader)
5 .reflectClass(typeOf[Person].typeSymbol.asClass)
6 m: reflect.runtime.universe.ClassMirror = class mirror for
7 tutorial.Person (bound to null)
8
9 scala> m.reflectConstructor(ctor)("tal", 2).asInstanceOf[Person]
10 res16: tutorial.Person = Person(tal,2)

Listing the members of the type The members of a type are accessed simply
through its members field. The additional benefit is that it is a field of type Scala
collection what means that they can be manipulated with the common functions
for collections like map, filter, .. . For example the code in figure 6 allows to list
the fields of the type list that are not private.

Fig. 6 Listing the non-private members of a type

1 ReflectionUtils.getType(List(1,2,3)).members.filter(!_.isPrivate)

Modifying the value of a field Fields are a type of member. To extract the specific
field from the list of members and then we modify it calling ”set”. It’s important
to note that in the same way that final fields are not modifiable on Java, fields
declared with val are not mutable in Scala so calling set is only allowed for var
fields.

Fig. 7 Modifying the value of a field with reflection

1 class Counter { var count = 3 }

2 typeOf[Counter].members.filter(!_.isPrivate)
3 val fieldX = ru.typeOf[Counter]
4 .declaration(ru.newTermName("count"))
5 .asTerm.accessed.asTerm
6 val instanceMirror = loader.reflect(new Counter)
7 val fieldMirror = instanceMirror.reflectField(fieldX)
8 fieldMirror.set(6)

2.3.2 Advantages and disadvantages

Advantages The resulting API has interesting properties. As described in the be-
ginning of the chapter the main principles of reflective APIs [2] are respected,
 8 Íñigo Mediavilla

meaning that the API provides meta-level representations of all the concepts avail-
able in the language, that the implementation is encapsulated making it replace-
able and that it is stratified in a way that it can be disabled easily.
On top of these properties the new API is capable of overcoming Java’s limita-
tion to access generic types at runtime. Due to Scala’s functional nature, paramet-
ric polymorphism is widely used in Scala: to give an example collections, functions
and asynchrounous primitives like Futures are all implemented as polymorphic
types. The solution to the previous limitation gives an API on which developers
know the generic types at runtime and can do specific manipulations based on
them. This achievement have been made possible thanks to the addition of some
specific metainformation to the .class files generated by the Scala compiler that
includes the parametric types.
Another additional benefit of the new reflection in Scala is that it has forced
the designers of Scala to focus on exposing some of the internals of the compiler.
The main classes in the “scala.reflect” package provide a safe way to do intro-
spection for non expert developers. However this effor has also given advanced
developers the possibility to do more complex manipulations thanks to classes
like scala.tools.reflect.Toolbox that allow to compile AST trees, typecheck them
and evaluate expressions at runtime. For example figure 8 shows how to build an
expression and execute at runtime.

Fig. 8 Example of building and executing an expresion at runtime.

1 import scala.reflect.runtime.universe._
2 import scala.tools.reflect.ToolBox
3 val cm = runtimeMirror(getClass.getClassLoader)
4 val toolbox = cm.mkToolBox()
5 val expr = reify { println ("Hello world") }
6 toolbox.eval(expr.tree) // This prints Hello world on stdout

Limitations Even though it represents an important improvement when compared

with Java’s API, Scala’s reflection API comes however with some limitations,
specially when compared with the reflection APIs available in other languages.
First of all Scala’s reflection API is not thread safe. Neither the code that
executes the initialization of reflection nor the code that does symbol initialization
can be called from multiple threads. Compile time reflection is less concerned by
this problem since by the time the macros are called, all the reflection platform
has already been initialized. Nonetheless, in runtime reflection this represents a
relevant limitation. Specially when one of the more attractives features of Scala is
its support for concurrent programming.
The flexibility of the API is remarkable but the same cannot be said about
its simplicity. Doing behavioural reflection requires a deep understanding of many
concepts that only those who are used to working with compilers manipulate on a
daily basis. In comparison with reflection APIs for pure object oriented languages
like Smalltalk where the metalevel constructions are in most cases just represen-
tations of the concepts all developers of the language use every day, Scala’s API is
complex and this complexity represents a barrier for a more generalized adoption.
Reflection in Scala 9

Finally, even though Scala allows to do runtime code manipulation, this kind
of manipulation is done in a really low level and it implies loosing the warranties
provided by the type system a really high price to pay. Macros provide a type safe
way to do code manipulations, but they can only be applied at compile time and
Scala doesn’t provide any other mecanism to do type safe runtime manipulations
in a way similar to languages like MetaOcaml. The low level of the API for code
manipulations contrasts with the syntax provided by nemerle [9] and MetaOcaml,
this problem will be mitigated with the arrival of quasiquotes [3] but for the
moment they are not available in the language.

3 Macros

The runtime reflection API allowed mainly to do introspection. Some kind of

behavioural reflection is allowed at runtime but at the expense of loosing any
kind of safety. Macros are the mechanism provided by Scala to allow manipulating
code at compile time. This code manipulation has multiple applications including
performance optimization, materialization, type providers, language virtualization.
Macros are only an experimental feature on Scala 2.10 and 2.11 however it has
already seen applications on the most popular Scala frameworks.

3.1 Scala Macros vs Scheme and the reflective tower

Even though Scala compile time metaprogramming capabilities’ come under the
name of macros, due to the completely difference nature of Scala and Scheme both
macro systems have many things that separate them. In fact, Scala macros are
inspired by the macro system available at [9]. The following section will explain
the basic concepts behind Scheme’s macros to then compare them to Scala’s,
explaining the reasons that justify their differences and how Scala has taken a
different approach to solve some of problems in the implementation of macros.

3.2 Hygiene

”Macro’s are reflective tools that operate on the structure of a program” [8] they
were born in the context of the languages of the lisp family and they represent a
powerful technique to modify the semantics of a language. However its implemen-
tation in a language conveys some serious challenges. Probably one of the most
important challenges is the preservation of what is called ”hygiene”. A macro is
considered hygienic if its expansion doesn’t generate the capture of any identifier.
In languages that don’t provide macro hygiene, developers need to use tricks such
as obfuscation or temporary symbol creation to avoid variable capture.

3.3 The reflective tower

Languages like Scheme already come with mechanisms that ensure the hygiene
of macros in the language. An example of an interesting algorithm for enforcing
hygiene that has been implemented in Scheme is described in [8].
10 Íñigo Mediavilla

The main idea behind this mechanism is that ”macroexpansion expansion is

a disguished evaluation”. Since macros can depend on other macros this forms a
reflective hierarchy or ”tower” where the variables of every layer of the hierarchy
need to remain independent from those of the other layers. This effectively repre-
sents a hierarchy of environments that are used on the macroexpansion process.
The algorithm goes into deeper detail by dividing the expansion process into the
steps: expansion, evaluation and denotation and providing an initial implementa-
tion for the process.

3.4 Scala’s solution

On the implementation of macros Scala also had to face the problem on how to
ensure hygiene. However, in the case of Scala the number of levels of the reflective
tower is limited to two: the execution level and the macro expansion level that is
executed as a part of the phase of compilation. Due to Scala’s statically typed and
non-homoiconic nature a solution with multiple level wasn’t appropiate. Under
these circumstances the solution proposed by [8] wasn’t appropiate so Scala opted
for another solution to the problem of hygiene, a macro called reify.
Reify is just a macro that given an expression, converts it into a reified ex-
pression that can be manipulated. At runtime this makes a more involved kind
of reflection but also a more powerful one that allows the generation of arbitrary
expression. When reify is executed at macroexpansion time, it not only gives a
convenient way to build expressions but it also ensures the hygiene of the macros
it generates. The reason why reify ensures hygiene is that the variables used in
the expression passed to reify are bound at definition site, so since all identifiers
and types are resolved at runtime (for the macros context macroexpansion time),
nothing that can be done at runtime can affect their binding.

3.5 An example

The compile time reflection API will in the future provide multiple types of macros
but in Scala 2.10 it comes with only one form, Def macros. Def macros are normal
Scala functions with the particularity that they are called by the compiler at
compile time. The responsability of a Def macro is to take an abstract syntax tree
for every argument passed to the macro and generate another abstract syntax tree
as an output or signal an error. Notice that this possibility to signal an error with
an appropiate message serves as a really useful way to verify incorrect programs
that extends the type system verification.
This article only intended to give an introduction into the principles behind
Scala’s macros so it won’t try to explain all the details on how to implement macros
(for those interested [4] is a good reference). The following example will only help
show the basic definition of a macro.
In 9 the macro definition hello is a function without implementation that only
has the keywords macro followed by the name of its implementation method impl.
The macro function already defines the signature of the macro seen from the
perspective of the developer who will call the macro. The implementation treats
the input arguments as Scala expressions (AST representations of the inputs) and
 Reflection in Scala 11

returns another expression that contains the rewriting that happens as a result of
the execution of the macro. As we can see the input and output expressions of the
implementation are parameterized by the types of the input arguments and the
result of the macro definition. The context object that appears in the signature
of impl provides information about the environment of the call to the macro and
gives the utility methods needed to build the resulting expression.

Fig. 9 Simple hello world macro

1 object Macros {
2 def hello(input:String) : Unit = macro impl
3 def impl(c: Context)(input: c.Expr[String]) : c.Expr[Unit] =
4 c.universe.reify(println("Hello " +
5 c.Expr[String](input.tree).splice + " world!" ) )
6 }
7 Macros.hello(" reflective ") // prints Hello reflective world!

3.6 Macro Limitations

Macros are a fairly new addition to the set of tools provided by Scala. They
are a really powerful technique that has already make its way into some really
popular Scala frameworks and has demonstrated its many applications. However
as a feature it has many rough edges.
In terms of code manipulation it uses the same low level API than runtime
reflection, what means that writing macros is difficult, implies a lot of boiler plate
and requires a great deal of knowledge of the compiler internals. Quasiquotes [3]
will alleviate the problem with boilerplate code but knowledge of the compiler will
still be required. To add more difficulty debugging macros is a daunting task.
On top of that, Def macros only have access to the local context of the method,
therefore they cannot have into account information related to the class from where
they are called, even less from the whole structure of the program. For example,
sometimes it would be useful to define transformations that affect all the methods
in a program according to certain criteria but due to the local nature of Def macros
they don’t allow this kind of transformation.
Finally, the current macro implementation in the compiler requires that macro
implementations be compiled before they are used. Practically this means that
when developing a project with macros using sbt, the standard build tool for
scala, developers need to go into the trouble of creating two subprojects one for
the macros and another for the code.

4 Conclusion

This article has introduced the initial situation for doing reflection on Scala as
well as the result of the development of the new reflection API. It has explained
some of the basic principles for creating reflection APIs and how the developers
12 Íñigo Mediavilla

of Scala used this principles embodied by the concept of mirrors to develope the
new reflection system of their language. The resulting API has been explained
with some examples of their use. Finally that last section has explained how the
fact of adding the reflection API, opened the door to a kind of compile time
metaprogramming known as Scala macros. The section has provided an overview
of some of the implications of adding macros to the language including the problem
of macro hygiene and how Scala macros provide a solution to this problem.

References

1. Metaprogramming. URL:http://en.wikipedia.org/wiki/Metaprogramming, 10 2013.

2. Gilad Bracha and David Ungar. Mirrors: design principles for meta-level facilities of object-
oriented programming languages. ACM SIGPLAN Notices, 39(10):331–344, 2004.
3. E Burmako. Quasiquotes. URL:http://docs.scala-
lang.org/overviews/macros/quasiquotes.html, 2013 10.
4. Eugene Burmako and Martin Odersky. Scala macros, a technical report. In Third Interna-
tional Valentin Turchin Workshop on Metacomputation, page 23, 2012.
5. Yohann Coppel, Martin Odersky, and Gilles Dubochet. Reflecting scala. URL: http://lamp.
epfl. ch/teaching/projects/archive/coppel report. pdf, 2008.
6. Pattie Maes. Concepts and experiments in computational reflection. In ACM Sigplan
Notices, volume 22, pages 147–155. ACM, 1987.
7. H. Miller, E. Burmako, and P. Haller. Scala reflection overview. URL:http://docs.scala-
lang.org/overviews/reflection/overview.html, 10 2013.
8. Christian Queinnecy. Macroexpansion reflective tower. 1996.
9. Kamil Skalski, Michal Moskal, and Pawel Olszta. Meta-programming in nemerle. In Proc.
Generative Programming and Component Engineering. Citeseer, 2004.