Introduction to MSIL – Part 1 – Hello World

When describing C++/CLI and how it relates to C#, I am often tempted to discuss the
Microsoft intermediate language (MSIL) that the Visual C++ and Visual C# compilers
generate. The trouble is that most programmers are not familiar with MSIL and programs
like ILDASM aren’t that helpful to the newbie because the MSIL they display, although
painfully correct, is not very readable. For this reason I decided to post a few entries
introducing MSIL at a basic level. Since I doubt anybody reading this will actually go
and start writing code in MSIL, I will skim over some of the uninteresting details and
focus on things like defining types, writing methods, calling instructions and handling
exceptions.

Please let me know if you find this helpful.

Let’s start with a simple main (entry point) function that displays a really unique
message. Unlike C#, CLI does not have any requirement that a method must belong to a
class. The entry point function also does not have to be called main; however I will use
this name in this series for simplicity.

.method static void main()

{
.entrypoint
.maxstack 1

ldstr "Hello world!"

call void [mscorlib]System.Console::WriteLine(string)

ret
}

The main method above is called a method definition since both the signature as well as
the body of the method is provided. In contrast, when a method signature is provided
without a body it is referred to as a method declaration. Method declarations are typically
used as call targets (when a method is being called) whereas a method definition provides
the actual implementation for a method.

A method definition begins with the .method directive and can be defined at global scope
or within a class. The application entry point must be static, meaning an instance is not
required to call the method, and that is indicated by the static keyword. Declaring a
global method static seems redundant but the ILASM compiler complains if you omit the
static keyword in some cases. Think of ‘void main()’ as the signature of the method
which, as you would expect, indicates that it does not return a value and takes zero
arguments.

The .entrypoint directive signals to the runtime that this method is the entry point for the
application. Only one method in the application can have this directive.
The .maxstack directive indicates how many stack slots the method expects to use. For
example, adding two numbers together involves pushing both numbers onto the stack and
then calling the add instruction which pops both numbers off the stack and pushes the
result onto the stack. In that example you will need two stack slots.

The ldstr instruction pushes the string that is passed to the WriteLine method onto the
stack. The call instruction invokes the static WriteLine method on the System.Console
class from the mscorlib assembly. This is an example of a method declaration. It provides
the full signature of the WriteLine method (including the string argument) so that the
runtime can determine which overload of the WriteLine method to call.

And finally the ret instruction returns execution to the caller. In the case of the entry
point method, this would bring your application to an end.

NOTE:

Although CLI doesn't require the method to be contained in a class a .assembly section
_is_ required. So the example would be like this:

.assembly hello
{
}
.method static void main()
{
.entrypoint
.maxstack 1
ldstr "Hello world!"
call void [mscorlib]System.Console::WriteLine(string)
ret
}

I'm unsure if this is MSIL specific though.

Introduction to MSIL – Part 2 – Using Local Variables
In part 2 of the Introduction to MSIL series, I will be exploring the use of local variables.
Without variables, programs would not be very interesting. To illustrate the use of
variables, let’s write a simple program to add numbers together.

In an MSIL method, variables are declared using the .locals directive.

.locals init (int32 first,

int32 second,
int32 result)

This statement declares three local variables for the current method. In this case they all
happen to be of type int32, which is a synonym for the System.Int32 type. init specifies
that the variables must be initialized to the default values for their respective types. It is
also possible to omit the variable names. In that case you would refer to the variables by
their zero-based index in the declaration. Of course, using variable names improves
readability.

Before we continue, I want to make sure that it is clear how the stack is used explicitly in
MSIL. When you want to pass values to an instruction, those values need to be pushed
onto the stack. To read those values, the instruction must pop them off the stack.
Similarly when calling a method, you need to push the object reference (if any) onto the
stack followed by each argument that you wish to pass to the method. In the process of
invoking the method, all the arguments as well as the object reference will be popped off
the stack. To push a value onto the stack, use the ldloc instruction indicating the variable
that holds the value. To pop a value off the stack, use the stloc instruction specifying the
variable you wish to store the value in. Also keep in mind that values (based on value
types) are stored directly on the stack, but objects (instances of reference types) are not
since the CLI does not allow reference types to be allocated on the stack. Rather
references to objects are stored on the stack. This is analogous to a native C++ object
allocated on the heap with a pointer to it stored on the stack. Keep the stack in your mind
as you read this series on MSIL. It should help you understand why values are
continually pushed and popped on and off the stack.

The next step is to get the numbers from the user.

ldstr "First number: "

call void [mscorlib]System.Console::Write(string)
call string [mscorlib]System.Console::ReadLine()
call int32 [mscorlib]System.Int32::Parse(string)
stloc first

As I mentioned in Part 1, the ldstr instruction pushes the string onto the stack and the call
instruction invokes the Write method, popping its argument off the stack. The next call
instruction invokes the ReadLine method which returns a string. The returned string is
pushed onto the stack and since it is already there, we simply call the Int32::Parse method
which pops the string off the stack and pushes the int32 equivalent on. Note that I am
omitting any error handling for the sake of clarity. The stloc instruction then pops the
value off the stack and stores it in the local variable named 'first'. Getting the next number
from the user works the same way except that the value is stored in the local variable
named 'second'.

Now that we have read the two numbers from standard input, it is time to add them up.
The add instruction can be used for this purpose.

ldloc first
ldloc second
add
stloc result

The add instruction pops two values off the stack and calculates the sum. To push the
values of the local variables onto the stack, we use the ldloc instruction. When the add
instruction completes, it pushes the result onto the stack and the program pops the value
off the stack and stores it in a variable named 'result' using the stloc instruction.

The final step is to display the result to the user.

ldstr "{0} + {1} = {2}"

ldloc first
box int32

ldloc second
box int32

ldloc result
box int32

call void [mscorlib]System.Console::WriteLine(string, object, object,

object)

We use the WriteLine overload that takes a format string followed by three object
arguments. Each argument to the WriteLine method must be pushed onto the stack one
by one. Since the numbers are stored as int32 value types, we need to box each value;
otherwise the method signature won’t match.

The ldloc instruction pushes each argument onto the stack. The box instruction is then
used for each int32 argument. Boxing involves popping the value off the stack,
constructing a new object containing a copy of the value and then pushing a reference to
the object onto the stack.
Here is the complete program.

.assembly extern mscorlib {}

.assembly Example {}

.method static void main()

{
.entrypoint
.maxstack 4

.locals init (int32 first,

int32 second,
int32 result)

ldstr "First number: "

call void [mscorlib]System.Console::Write(string)
call string [mscorlib]System.Console::ReadLine()
call int32 [mscorlib]System.Int32::Parse(string)
stloc first

ldstr "Second number: "

call void [mscorlib]System.Console::Write(string)
call string [mscorlib]System.Console::ReadLine()
call int32 [mscorlib]System.Int32::Parse(string)
stloc second

ldloc first
ldloc second
add
stloc result

ldstr "{0} + {1} = {2}"

ldloc first
box int32

ldloc second
box int32

ldloc result
box int32

call void [mscorlib]System.Console::WriteLine(string, object,

object, object)

ret
}

The last thing to notice about this example is that I indicated the method will use at most
four stack slots. This is to accommodate the four arguments passed to the WriteLine
method at the end of the main method.
Introduction to MSIL – Part 3 – Defining Types
In this installment of the MSIL series, I describe how types are defined.

Here is a minimal reference type called House. As I write this, we are looking for a house
in British Columbia, so this was the first thing that came to mind.

.class Kerr.RealEstate.House
{
.method public void .ctor()
{
.maxstack 1

ldarg.0 // push "this" instance onto the stack

call instance void [mscorlib]System.Object::.ctor()

ret
}
}

This is a very simple type. Note that you must declare a constructor for a concrete
reference type. Unlike languages like C# and C++, the IL assembler will not generate a
constructor for you automatically.

Types are defined using the .class directive followed by a type header. The class keyword
is used instead of the more intuitive type for historical reasons. When you read class in
MSIL source code, just think type. The type header consists of a number of type
attributes followed by the name of the type you are defining. To define the equivalent of
a C# static class in MSIL you can write the following.

.class abstract sealed Kerr.RealEstate.MortgageCalculator

{
/* members */
}

abstract and sealed are the type attributes. An abstract type cannot be instantiated and a
sealed type cannot have sub-types. There are attributes to control visibility, such as
public and private. There are attributes to control field layout, such as auto and
sequential. For a complete list of attributes please consult the CLI specification. Many
attributes are applied automatically, which can save you a lot of typing. Fortunately these
defaults are quite intuitive so you should become familiar with them quickly. As an
example, extending the System.ValueType from the mscorlib assembly defines a value
type. Since the CLI requires that value types be sealed, the IL assembler will
automatically add this attribute for you.

The name of the type in the example above is Kerr.RealEstate.MortgageCalculator. The

CLI does not recognize namespaces as a distinct concept. Rather the full type name is
always used. The syntax shown above is only supported by the IL Assembler that ships
with version 2.0 of the .NET Framework. If you are working with version 1.x then you
need to group your namespace members with a .namespace directive, as shown below.
Note that this syntax is also supported in version 2.0.

.namespace Kerr.RealEstate
{
.class abstract sealed MortgageCalculator
{
/* members */
}
}

Following the type name you have the option of specifying the base type. The extend
keyword is used for this purpose. If no base type is specified, the IL assembler will add
the extend clause to make the type inherit from the System.Object type from the mscorlib
assembly, resulting in a reference type. And finally, the type header can provide a list of
interfaces that the type and its descendants will implement and satisfy, becoming the
interfaces of the type.

.class Kerr.RealEstate.RoomList
extends [System.Windows.Forms]System.Windows.Forms.ListView
implements Kerr.IView
{
/* members */
}

In this example, the Kerr.RealEstate.RoomList type has

System.Windows.Forms.ListView, defined in the System.Windows.Forms assembly, as
its base type. Keep in mind that the CLI requires that every user-defined type extend
exactly one other type. The RoomList type also implements the Kerr.IView interface
type.

With this basic introduction to type definitions, you should now be able to start defining
more interesting types. To define an interface, simply use the interface attribute in your
type header. If you want a value type, known as a struct in C#, simply extend the
System.ValueType type from the mscorlib assembly. Now you should be able to see why
the .class directive is perhaps not the best name for it.

.class interface Kerr.IView

{
/* members */
}

.class Kerr.RealEstate.HouseData
extends [mscorlib]System.ValueType
{
/* members */
}
Introduction to MSIL – Part 4 – Defining Type
Members
In Part 3 of the MSIL series, I introduced the basic syntax for defining types. Using the
.class directive, you can define reference types and value types. Choosing the type
attributes correctly, you can exercise complete control over the definition of your type.

.class abstract Kerr.Sample.Object

{
}

In this installment, we are going to explore how to declare members of a type. To

simplify the samples, I may omit the class definitions at times and simply focus on
defining the members.

Constructors

Let’s begin with initializers, known as constructors in languages like C++ and C#. CLI
supports both type initializers and instance initializers. Type initializers are a very handy
thing and address many of the multithreading challenges you encounter when
implementing a singleton in native C++. A type initializer is run exactly once for a given
type and the runtime guarantees that no methods of the type will be accessible before the
type initializer completes.

.method static void .cctor()

{
.maxstack 1

ldstr ".cctor"
call void [mscorlib]System.Console::WriteLine(string)

ret
}

.method public void .ctor()

{
.maxstack 1

ldarg.0
call instance void [mscorlib]System.Object::.ctor()

ldstr ".ctor"
call void [mscorlib]System.Console::WriteLine(string)

ret
}
.cctor and .ctor are known as special method names. A type initializer, which is optional,
is defined as a static method named .cctor with no return value. Type initializers are
called static constructors in the C++/CLI and C# languages.

An instance initializer, more commonly referred to simply as a constructor, initializes an

instance of a type. A constructor is defined as an instance method named .ctor also with
not return value. A constructor is called when an instance is created using the newobj
instruction. Here is an example.

.locals (class TypeName obj)

newobj void TypeName::.ctor()
stloc obj

When executed, it will result in the following written to the console, assuming the
constructors defined above.

.cctor
.ctor

The newobj instruction allocates a new instance of the type and initializes all its fields to
the type-equivalent of zero. Then it calls the particular constructor, disambiguated by the
signature, ensuring that the first (zero-based) argument refers to the newly created
instance. Once the constructor completes, an object reference is pushed onto the stack for
access by the caller.

Methods

Although the CLI defines constructors and properties in terms of methods (with some
extra metadata for properties), in this section we are going to look at methods in the sense
of C++ member functions or C# methods. Of course most of what is said of methods,
also applies to instance constructors and property getters and setters. Virtually all of the
interesting things that you can do with methods are controlled by method attributes.
There are some common scenarios programmers expect so lets cover a few of them.

Static methods are defined using the static attribute. Static methods, as you would expect,
are associated with a type but not an instance.

.method static void StaticMethod() { /* impl */ }

Instance methods simply use the instance attribute in place of the static attribute. The IL
Assembler assumes instance as the default so you rarely need to specify it explicitly for
method declarations.

.method void InstanceMethod() { /* impl */ }

The opposite is true when calling methods. The call instruction assumes a static method
unless you specify otherwise. Here is an example of calling both methods.
call void TypeName::StaticMethod()

ldloc obj
call instance void TypeName::InstanceMethod()

Remember to push the object reference pointing to your instance onto the stack before
calling the instance method.

Virtual function calls are an important part of object-oriented design and the CLI
provides a great deal of flexibility in controlling whether the static or dynamic type of the
object will be used to service the call, as well as how this behavior can be overridden in
subclasses. When I refer to the static and dynamic type in this context I am referring to it
in the C++ sense of the static type known at compile time and the dynamic type
determined at runtime. This is generally referred to as polymorphism. There are two
aspects to the virtual function support that you need to keep in mind when programming
in MSIL. The first is how you declare your instance methods to support virtual function
invocation and the second is how you call the method. It should also go without saying
that static methods are by definition not virtual.

A method is marked virtual by adding the virtual attribute to the type header. Consider
the following example.

.class House
{
.method public virtual void Buy()
{
.maxstack 1

ldstr "House::Buy"
call void [mscorlib]System.Console::WriteLine(string)

ret
}

/* etc */
}

.class TownHouse
extends House
{
.method public virtual void Buy()
{
.maxstack 1

ldstr "TownHouse::Buy"
call void [mscorlib]System.Console::WriteLine(string)

ret
}

/* etc */
}
The House type has a virtual method called Buy. The TownHouse type extends House
and also has a virtual method with the same name. Because of this, TownHouse::Buy is
said to override House::Buy. So how do we tell the runtime which method to pick?
Obviously if I have a House instance I would like House::Buy to be called, but if I have a
TownHouse instance I would like TownHouse::Buy to be called and, being a virtual
method, I want this decision to be made at runtime when the actual type is known. So far
I have used the call instruction in a number of examples in this series. The call instruction
invokes the specified method and will always call the same method regardless of the
dynamic type of the object. The callvirt instruction, on the other hand, allows the runtime
to determine the specific virtual method implementation to invoke based on the actual
type of the object. Consider the following example.

newobj instance void House::.ctor()

stloc house
newobj instance void TownHouse::.ctor()
stloc townHouse

ldloc house
call instance void House::Buy()

ldloc townHouse
call instance void TownHouse::Buy()

ldloc townHouse
call instance void House::Buy()

ldloc townHouse
callvirt instance void House::Buy()

When executed, it will result in the following written to the console.

House::Buy
TownHouse::Buy
House::Buy
TownHouse::Buy

The first call to the Buy method with the house reference invokes the House::Buy
implementation since call is only interested in the static, or compile-time, type. The
second call to Buy with the townhouse reference invokes the TownHouse::Buy
implementation for the same reason. The third call will once again invoke House::Buy
despite the fact that the object reference points to a TownHouse. It should now be clear
that using the call instruction implies making a compile-time decision on which method
to execute. The final method call uses the callvirt instruction to invoke the virtual
method House::Buy and since the object reference actually points to a TownHouse, the
TownHouse::Buy method will be executed. To be clear, the runtime is not looking at the
type of the local variable you declared but rather the type of the object being referenced.
We could have stored a reference to a TownHouse in a House local variable and the
TownHouse::Buy method would still have been called.
If you want to declare a virtual method but do not want to override an inherited virtual
method with the same name, you can use the newslot attribute on the new virtual method
in the subclass. If you consider that virtual method invocation by the runtime is not
concerned about method names then you should see how this is possible. Just think of
newslot as adding a new virtual function pointer to the vtbl for the given type.

CLI virtual methods are very interesting, especially when you consider how and to what
extent they are exposed by C++/CLI and C#. This entry is getting long enough so I’ll
save that discussion for another day.
Introduction to MSIL – Part 5 – Exception Handling
In this part of the Introduction to MSIL series I will introduce the constructs that the CLI
provides for exception handling.

A try block is used to protect a range of instructions. If an exception is thrown by one of

the instructions in the protected block, or by any method called directly or indirectly
within the protected block, control is transferred to an appropriate exception handler. A
try block is declared with the .try directive.

.try
{
/* protected code */

leave.s _CONTINUE
}
<exception handler>

_CONTINUE:

The last instruction in the try block is the leave.s instruction which transfers control to the
_CONTINUE label. The leave.s instruction ensures that the evaluation stack is emptied
and that the appropriate finally blocks are executed. It follows that if control leaves the
protected block by some other means, that an exception was thrown. The exception
handler must immediately follow the try block.

The CLI offers four different kinds of handlers that you can employ to construct the error
handling semantics that you require for your programming language or application. Let’s
examine each one individually.

Catch

The catch exception handler, or catch block, has to be the most well-known form of
exception handling, since it is provided directly by C++ as well as C#. A catch block is
declared with the catch keyword. The catch clause includes the type of exception that the
handler is willing to catch as well as the block of code to transfer control to, given a
matching exception object. Catch blocks can also be chained together after a single try
block for convenience. The first catch clause with a matching exception type will be
chosen as the handler. Consider this example.
.try
{
ldstr "I'm not a number"
// ldnull
// ldstr "123"
call int32 [mscorlib]System.Int32::Parse(string)

leave.s _CONTINUE
}
catch [mscorlib]System.ArgumentNullException
{
callvirt instance string [mscorlib]System.Exception::get_Message()
call void [mscorlib]System.Console::WriteLine(string)

leave.s _CONTINUE
}
catch [mscorlib]System.FormatException
{
callvirt instance string [mscorlib]System.Exception::get_Message()
call void [mscorlib]System.Console::WriteLine(string)

leave.s _CONTINUE
}

Here we ask the In32::Parse method to parse "I'm not a number", which predictably
throws a FormatException. The first catch handler is never given an opportunity to
execute. The FormatException handler dutifully writes a message describing the
exception to the console and then calls the leave.s instruction to transfer control to the
_CONTINUE label. Where was the exception object? The runtime ensure that a reference
to the exception is pushed onto the stack before the handler is invoked. You can
experiment by commenting out the ldstr instruction in the try block and un-commenting
the ldnull instruction. This will push a null reference onto the stack and result in an
ArgumentNullException instance being thrown by the Parse method.

Filter

The filter exception handler is a strange construct for a C++ programmer like me. Instead
of matching on an exception type, the filter handler provides a scope block where it can
evaluate whether it wants to handle the exception. It indicates to the runtime that it wants
to handle the exception by pushing a value of 1 onto the stack. If it decides not to handle
the exception it pushes a value of 0 onto the stack. Following the “evaluation” block is
the handler block that contains the code that will handle the exception.
.try
{
// ldstr "I'm not a number"
ldnull
// ldstr "123"
call int32 [mscorlib]System.Int32::Parse(string)

leave.s _CONTINUE
}
filter
{
ldstr "filter evaluation\n\t"
call void [mscorlib]System.Console::Write(string)

callvirt instance string [mscorlib]System.Exception::get_Message()

call void [mscorlib]System.Console::WriteLine(string)

ldc.i4.1
endfilter
}
{
ldstr "filter handler\n\t"
call void [mscorlib]System.Console::Write(string)

callvirt instance string [mscorlib]System.Exception::get_Message()

call void [mscorlib]System.Console::WriteLine(string)

leave.s _CONTINUE
}

The try block will result in an ArgumentNullException exception because a null

reference is pushed onto the stack as an argument to the Int32::Parse method.

The filter is then given a chance to determine whether it wants to handle the exception. In
this case it simply writes the error message to the console and then pushes the value 1
onto the stack, using the ldc.i4.1 instruction, to indicate that it wants to handle the
exception. The endfilter instruction is called to return from the filter clause.

The filter handler block is then called to handle the exception. Notice that both the
evaluation and handler blocks can access the exception in flight by popping it off of the
stack. Keep in mind that you can throw any reference type so don’t assume that the object
reference you pop off of the stack in the filter handler is an instance of System.Exception.
This is not a problem for the catch handler since you will know the type of the exception
object.

Finally

The finally exception handler should be recognized by C# programmers and C++

programmers familiar with Structured Exception Handling (SEH). A finally block
associated with a try block is always executed, regardless of whether control leaves the
protected try block through normal means, using the leave.s instruction, or as a result of
an exception, using the throw instruction. It provides a reliable mechanism of ensuring a
certain block of code is always run for those languages that do not provide destructors,
such as C and C#. Although the C programming language does not target the CLI, SEH is
used often enough that the Microsoft C/C++ compiler provides the __finally keyword for
the same reason.

.try
{
/* protected code */

leave.s _CONTINUE
}
finally
{
/* cleanup code */

endfinally
}

Fault

The fault exception handler is similar to the finally block except that it is invoked only if
it’s associated try block is left as a result of an exception. After the fault handler has been
given an opportunity to execute, the exception continues on its way in search or a handler
that is willing to catch it.

.try
{
/* protected code */

leave.s _CONTINUE
}
fault
{
/* cleanup code */

endfault
}

With that brief introduction to exception handling out of the way, it is time to change
gears a bit and talk about how this all relates to popular programming languages like C#
and C++/CLI.
Introduction to MSIL – Part 6 – Common Language
Constructs
In parts 1 through 5 of the Introduction to MSIL series we focused on MSIL and the
constructs it provides for writing managed code. In the next few sections we are going to
examine some common language features that are not intrinsic to instruction-based
languages like MSIL. Understanding the intermediate language instructions that are
generated for common, programming language statements is critical to acquiring an
instinct for the performance characteristics of your code as well as to more easily track
down subtle bugs.

In this part we are going to look at some common constructs that are present in many
languages. I will be expressing simple examples in C# although the explanations are
common to many programming languages that offer the same general constructs. As I am
trying to explain a general principle, I may not always present the exact code that a given
compiler may produce. The point is to learn more about what compilers generate in
general. I encourage you to compare the instructions generated by different compilers
using tools like ILDASM.

Programming languages like C and C++ enabled much more rapid development of
software compared with classic assembly language programming. A number of rich
constructs played a significant part in providing this productivity. Concepts like
expressions, selection statements (e.g. if and switch), and iteration statements (e.g. while
and for) are what makes these portable languages so powerful. Even more productivity
was gained by introducing concepts like objects, but before objects were even considered,
expressions and statements provided enormous benefit to the assembly language
programmer struggling to build large applications and operating systems. Let’s look at
some of these statements.

The if-else Statement

Consider the following simple method that checks that its argument is not null before
proceeding.

void Send(string message)

{
if (null == message)
{
throw new ArgumentNullException("message");
}

/* impl */
}

The if statement looks innocent enough. If the expression is true, execution enters the
scope block and an exception is thrown. If the expression is false, execution jumps to the
first statement following the if statement. There’s enough going on here that I will leave
the else clause as an exercise for the reader.

Even if you have never used C# before, this code should be pretty clear (assuming you’re
a programmer). So how does the compiler turn this innocent little if statement into
something the runtime will understand? As with most programming tasks, there are a
number of ways to solve the problem.

In many cases it is necessary for a compiler to create temporary objects. Expressions are
a common source of temporary objects. In practice, compilers avoid creating temporaries
as much as possible. Assuming a non-optimizing compiler you could imagine the
compiler turning the code above into something like this.

bool isNull = null == message;

if (isNull)
{
throw new ArgumentNullException("message");
}

This is not to say that an optimizing compiler would not use temporary objects in this
manner. Temporary objects can often help to generate more efficient code depending on
the circumstance. Here the result of the expression is first evaluated to a boolean value.
This value is then passed to the if statement. This is not all that different from the
previous example, but after reading the last 5 parts of this series it should be clear why
this is much more appealing to an instruction-based language. It’s all about breaking
down statements and expressions into simple instructions that can be executed one by
one. Let’s consider one implementation of the Send method.

.method void Send(string message)

{
.maxstack 2

ldnull
ldarg message
ceq

ldc.i4.0
ceq

brtrue.s _CONTINUE

ldstr "message"
newobj instance void
[mscorlib]System.ArgumentNullException::.ctor(string)
throw

_CONTINUE:

/* impl */
 ret
}

If you’ve been following along with this series you should be able to understand much of
this method declaration. Let’s step through it quickly. The temporary object that the
compiler might generate silently is made explicit in MSIL, although it is not named and
lives very briefly on the stack. (I guess whether it is actually explicit is debatable.) Can
you spot it? First we compare message to null. The ceq instruction pops two values off
the stack, compares them, and then pushes 1 onto the stack if they are equal or 0 if they
are not (this is the temporary). The code may seem overly complicated. The reason is that
MSIL does not have a cneq, or compare-not-equal-to, instruction. So we first compare
message to null then compare the result to zero, effectively negating the first comparison.
More on this in a moment.

Now that we’ve got a handle on the expression result, the brtrue.s instruction provides
the conditional branching you would expect from an if statement. It transfers control to
the given target if the value on the stack is non-zero, thereby skipping over the if clause’s
logical scope block.

Of course there is more than one way to skin a cat. The example above is very similar to
what the C# compiler that I am using generates, although it uses an explicit temporary
local variable to store the result of the expression. This implementation seems a bit
awkward. Ultimately it does not matter too much as the JIT compiler can probably
optimize away any differences. Nevertheless it is a useful exercise to see how we can
simplify this implementation. The first thing we can attempt is to reduce the number of
comparisons. I mentioned that there is no matching not-equal-to version of the ceq
instruction. On the other hand there is a branch-on-false version of the brtrue.s
instruction. Quite predictably, it is called brfalse.s. Using the brfalse.s instruction
completely removes the need for the second comparison.

Finally, as a C++ programmer you would expect the compiler to use the equality
operator, since one of the operands is a System.String type which has an equality operator
defined for it, and this is exactly what the C++ compiler does.

.method void Send(string message)

{
.maxstack 2

ldnull
ldarg message
call bool string::op_Equality(string, string)

brfalse.s _CONTINUE

ldstr "message"
newobj instance void
[mscorlib]System.ArgumentNullException::.ctor(string)
throw

_CONTINUE:
 /* impl */

ret
}

Ultimately the JIT compiler optimizes the code as best it can, including inlining methods
where appropriate. You should not be surprised if these two implementations result in the
same machine code at the end of the day.

The for Statement

Before we delve into the implementation of the for statement, commonly referred to as
for loops, lets quickly review how the for statement works. If your background is in
Visual Basic or even C# you may not be all that familiar with this extremely useful
statement. Even C++ programmers are encouraged to avoid the for statement for
common iterations in favor of the safer std::for_each algorithm from the Standard C++
Library. There are however, many interesting applications of the for loop that have
nothing to do with iterating over a container from beginning to end.

The following simple pseudo code illustrates the construction of a for statement.

for ( initialization expression ; condition expression ; loop

expression )
{
statements
}

The for statement consists of three semi-colon delimited expressions followed by a scope
block. (Don’t get me started on for statements without scope blocks.) You can use any
expressions you wish as long as the condition expression results in a value that can be
interpreted to mean either true or false. The initialization expression is executed first and
exactly once. It is typically used to initialize loop indices for iteration or any other
variables required by the for statement. The condition expression is executed next and is
executed before each subsequent loop. If the expression evaluates to true, the scope block
will be entered. If the expression evaluates to false, control is passed to the first statement
following the for statement. The loop expression is executed after every iteration. This
can be used to increment loop indices or move cursors or anything else that might be
appropriate. Following the loop expression, the condition expression is evaluated again
and so on and so forth. For a more complete description, please consult your
programming language documentation. Here is a simple example that will write the
numbers zero through nine to any trace listeners you have set up.

for (int index = 0; 10 != index; ++index)

{
Debug.WriteLine(index);
}
We have already spoken about how an if statement might be implemented in MSIL. Lets
now make use of that knowledge to deconstruct the for statement into something simpler
to interpret by a computer while still using C#.

int index = 0;
goto _CONDITION;

_LOOP:

++index;

_CONDITION:

if (10 != index)
{
// for statements
Debug.WriteLine(index);

goto _LOOP;
}

That looks nothing like the for statement! Here I use the infamous goto, more generally
referred to as a branch instruction. Branching is inevitable in languages that don’t support
selection and iteration statements. It serves no purpose in languages like C++ and C#
other than to obfuscate the code. (If you’re a major supporter of the goto statement,
please don’t feel the need to share that with me. Feel free to talk about its merits on your
own blog.) Code generators however often use goto statements as it is a lot simpler for a
code generator to use than to construct the expressions for a for statement for example.

By following the branching in the code example above you should be able to see how we
construct the for statement semantics by executing the condition and then jumping up to
the loop expression to conditionally loop again. Now let’s consider how we can
implement this in MSIL.

.locals init (int32 index)

br.s _CONDITION

_LOOP:

ldc.i4.1
ldloc index
add
stloc index

_CONDITION:

ldc.i4.s 10
ldloc index
beq _CONTINUE

// for statements
ldloc index
box int32
 call void [System]System.Diagnostics.Debug::WriteLine(object)

br.s _LOOP

_CONTINUE:

After initializing the index local variable we branch to the _CONDITION label. To
evaluate the condition I pushed the value 10 onto the stack followed by the index value.
The beq, or branch on equal, instruction pops two values off the stack and compares
them. If they are equal it transfers control to the _CONTINUE label thereby ending the
loop; otherwise control continues through the ‘statements’ in the for loop. To write the
index to the debug trace listener I push the index onto the stack, box it and call the static
WriteLine method on the System.Diagnostics.Debug reference type from the System
assembly. Following this, the br.s instruction is used to transfer control to the loop
expression for the next iteration.
Introduction to MSIL – Part 7 – Casts and Conversions
After a brief hiatus, I decided to post a few more parts of the Introduction to MSIL series.

Casting and type conversion are often raised as topics of concern among programmers. It
might be concerns over performance and safety or simply that the implications of casts
and conversions are not well understood. In this part of the series I explore these
concepts. I will primarily use C++ for illustrations as it is more descriptive in its language
constructs related to casting. Of course we will also consider the CLI instructions that are
ultimately generated.

Static Casts

When converting from a type with a small value space to a larger value space, an implicit
cast can be employed as there is no danger of losing data. This is safe as long as the
resulting value space is a superset of the smaller value space. Converting from a 32-bit
integer to a 64-bit integer is guaranteed to be accurate, so compilers don’t require any
explicit confirmation from the programmer. On the other hand, when converting from a
larger type to a smaller type, it is possible for the data to be truncated; therefore
compilers typically require confirmation in the form of a cast. This is referred to as a
static cast, since only static type information is used in determining the conversion
behavior. A static cast is often considered dangerous, as the compiler places the
responsibility of ensuring the safety of the cast squarely in the hands of the programmer.
Consider the following C++ code:

Int32 small = 123;

Int64 big = small;

small = static_cast<Int32>(big);

Conversion from the small variable to the big variable is implicit but conversion from big
to small requires the static_cast operator to avoid a compiler warning regarding possible
loss of data. Although static casts can and should be considered dangerous, the compiler
goes to great lengths to ensure that it is at least in the realm of possibility that the cast
will actually be correct at runtime. It does this by considering the static type information,
and in the case of user-defined types, whether there are any conversion operators defined.
All this is determined at compile-time. Consider the following representation in MSIL:

.locals init (int32 small,

int64 big)

// Int32 small = 123;

ldc.i4.s 123
stloc small

// Int64 big = small;

ldloc small
conv.i8
stloc big
// small = static_cast<Int32>(big);
ldloc big
conv.i4
stloc small

The ldc.i4.s instruction pushes a 4 byte (32-bit) integer with a value of 123 onto the
stack. This is then stored in the small local variable using the stloc instruction. To assign
the value of the small variable to the big variable, the value is first pushed onto the stack
using the ldloc instruction. The conv.i8 instruction then converts the value to an 8 byte
(64-bit) integer. This is then popped off the stack and stored in the big local variable
using the stloc instruction. Finally, to convert the value of the big variable to the small
variable again, the value stored in the big local variable is pushed onto the stack and the 8
byte value on the stack is converted to a 4 byte value using the conv.i4 instruction. This
is then stored in the small local variable using the stloc instruction.

As you can see, MSIL does not make a distinction between implicit and explicit
conversions. Everything is explicit. Overflow protection, however, needs to be requested
directly, as by default no checking is provided. In the code above, no provision is made
for overflow. This code happens to be safe because we know it to be so, but if the value
stored in big was too large to be stored in a 4 byte integer, it would simply overflow with
no warning at runtime.

Checked Conversions

To detect overflow errors, you can simply replace the conv.i4 instruction in the previous
example with the conv.ovf.i4 instruction. If the value on the stack is too large to be
represented by the specified type, an OverflowException object is thrown. The C++/CLI
language design, introduced in Visual C++ 2005, does not yet provide a language feature
to request conversion with overflow checking. A feature is being considered to add a
checked keyword, similar to the one provided by C#:

checked
{
int small = 123;
long big = small;

small = (int) big;

}

Any arithmetic operations and conversions in a checked block or expression will include
overflow-checking. As mentioned above, the conv.<type> set of instructions are replaced
by conv.ovf.<type>. Other instructions that could result in overflow also have
corresponding checked versions. For example the add instruction, described in part 2, has
corresponding add.ovf and add.ovf.un instructions for checked addition of signed and
unsigned values respectively.

Dynamic Casts
Static casts can also be used to cast up a class hierarchy, in effect performing a cast of a
polymorphic type but making an assumption that the cast will always succeed at runtime.
This is useful to avoid the cost of a run-time type check, assuming it is safe. Of course,
this is generally a dangerous assumption to make.

Dynamic casts are used to allow the determination of whether the cast is valid to be
deferred to run-time. Of course this only makes sense with polymorphic types. As with
static casts, the compiler tries to ensure that it is at least plausible that the cast may
succeed. Trying to dynamically cast from an integer to a House object will be caught by
the compiler and flagged as an error. Consider the following C++ code. Assume
BeachHouse and Townhouse are subclasses of House.

House^ house = gcnew BeachHouse;

if (nullptr != dynamic_cast<BeachHouse^>(house))
{
Console::WriteLine("It's a beach house!");
}

if (nullptr != dynamic_cast<Townhouse^>(house))
{
Console::WriteLine("It's a townhouse!");
}

This code will compile perfectly without the compiler knowing whether the casts will
succeed. Both casts require a run-time type check to determine whether the house is in
fact a BeachHouse or a Townhouse. dynamic_cast returns a nullptr, the C++
representation for a null pointer or handle, if the cast cannot be performed. Let’s see how
we can represent this in MSIL:

.locals init (class House house)

newobj instance void BeachHouse::.ctor()

stloc house

ldloc house
isinst BeachHouse
ldnull

beq.s _TOWN_HOUSE

ldstr "It's a beach house!"

call void [mscorlib]System.Console::WriteLine(string)

_TOWN_HOUSE:

ldloc house
isinst TownHouse
ldnull

beq.s _CONTINUE
ldstr "It's a town house!"
call void [mscorlib]System.Console::WriteLine(string)

_CONTINUE:

Having read through this series thus far, this code should be pretty clear. A BeachHouse
object is created and a reference to it is stored as a local variable. The dynamic cast is
performed by pushing the reference onto the stack and using the isinst instruction to do
the type check. The isinst instruction evaluates whether the reference on the stack is a
BeachHouse or a subclass of the BeachHouse type. If it is, then the reference is cast to
the type defined in the isinst instruction and pushed onto the stack. Otherwise the value
null is pushed onto the stack. The expression for the if statement is constructed by
pushing a null reference onto the stack and using the beq.s instruction to transfer control
to the target if isinst also pushed null onto the stack. The same type check and
conditional branching is done for the Townhouse type, with execution ultimately
continuing on.

If your application assumes that the dynamic cast must always succeed for the program to
be correct, but you don’t want to resort to a static cast to ensure correctness in the
exceptional case, you can employ the castclass instruction in place of the isinst
instruction. The castclass instruction performs the same type check, but instead of
pushing null onto the stack if the type check fails, it throws an InvalidCastException
object. If this is the behavior you are looking for, you can use the safe_cast operator in
C++ or a simple (C-style) cast in C#.
Introduction to MSIL – Part 8 – The for each Statement
The for each statement; popularized by Visual Basic, humbly accepted by C++, and
forever immortalized by a creation lovingly known as ECMA-334 (OK so some people
simply call it C#).

In this installment of my open-ended series entitled Introduction to MSIL, I am talking

about one of the most popular statements in commercially successful programming
languages. Whether your focus has been on Visual Basic, COM, Standard C++, or .NET
(or all of the above) you’ve inevitably come across some form of the for each statement.

You may have noticed a trend in this series. As we dig deeper into more interesting
constructs, the focus is less and less on MSIL instructions and more on language design
and compiler implementations. There is really nothing especially new in the instructions
you might use to implement a for each statement. Nevertheless it is useful to understand
what makes the statement work and how it is interpreted and converted to MSIL
instructions depending on the context in which it is used. This is where you thank your
lucky stars that your boss lets you write code in a language with statements that abstract
away the dirty secrets hidden behind their elegant exteriors. for each is just such a
statement.

Variations of the for each statement, as it relates to .NET, can be found in C#, Visual
Basic .NET and C++/CLI. There may also be other languages with a similar statement
that I’m not aware of. The examples in this part of the series use C++/CLI only because
C# and VB get enough press as it is. As I’ve said before, I am not focusing on a particular
compiler’s implementation, but rather demonstrating the techniques that may be used by
compilers in general.

Let’s start with a simple example:

array<int>^ numbers = gcnew array<int> { 1, 2, 3 };

for each (int element in numbers)

{
Console::WriteLine(element);
}

numbers is a synthesized reference type extending the built-in System.Array type. It is

initialized with three elements and the for each statement executes the statements in the
scope block for each of the elements. Pretty straight forward; I’m sure you can imagine
the equivalent code in your language of choice. Quite predictably this can be
implemented in terms of the following set of instructions:
.locals init (int32[] numbers,
int32 index)

// Create the array

ldc.i4.3
newarr int32
stloc numbers

// Populate the array

ldloc numbers
ldc.i4.0 // index
ldc.i4.1 // value
stelem.i4

ldloc numbers
ldc.i4.1 // index
ldc.i4.2 // value
stelem.i4

ldloc numbers
ldc.i4.2 // index
ldc.i4.3 // value
stelem.i4

br.s _CONDITION

_LOOP:

ldc.i4.1
ldloc index
add
stloc index

_CONDITION:

ldloc numbers
ldlen
ldloc index
beq _CONTINUE

// for each statements

ldloc numbers
ldloc index
ldelem.i4
call void [mscorlib]System.Console::WriteLine(int32)

br.s _LOOP

_CONTINUE:

The only instructions not already discussed in this series are those related to arrays. The
newarr instruction pops an integer off the stack indicating the number of elements in the
array. It then creates the array of the given type and pushes a reference to it onto the
stack. The stelem instruction replaces an element in an array with a given value. The
stack needs to be prepopulated with a reference to the array, the index of the element in
the array, and the value to store at the given position. Finally, the ldelem instruction loads
an element from an array and pushes it onto the stack by first popping off a reference to
an array as well as the index into the array.

As you can see, this is not unlike the code that might be generated for a for statement
looping over the same elements, as discussed in part 6. Of course the for each statement
can be used with more than just types deriving from System.Array. In fact it is capable of
enumerating the elements of any type that either implements the IEnumerable interface or
even a type providing the same functionality without actually implementing that
interface. The degree of support naturally depends on the scope of your language specific
implementation. Some languages could even, nay, do provide support for completely
unrelated containers in the for each statement. Consider the following example using the
Standard C++ Library vector container class:

std::vector<int> numbers;
numbers.reserve(3);
numbers.push_back(1);
numbers.push_back(2);
numbers.push_back(3);

for each (int element in numbers)

{
std::cout << element << std::endl;
}

In this example, the compiler calls the vector’s begin method to get an iterator pointing to
the first element in the container. This iterator is then used to enumerate the elements,
incrementing the iterator until it compares equal to the iterator returned by vector’s end
method. For each iteration, the iterator is dereferenced with the resulting value assigned
to the element variable. The beauty here is that using the for each statement avoids many
common errors related to buffer overruns and the like.

Let’s get back to managed code. If the for each statement has to support collections other
than arrays, surely this leads to different implementations depending on the use of the
statement. Consider the following example:

Collections::ArrayList numbers(3);
numbers.Add(1);
numbers.Add(2);
numbers.Add(3);

for each (int element in %numbers)

{
Console::WriteLine(element);
}
As an aside, keep in mind that most of the Standard C++ Library containers that include a
constructor taking a single integer typically set the initial size of the container, whereas
the .NET collection types use this constructor to set the initial capacity.

So what does for each do in this case? Well the ArrayList type implements the
IEnumerable interface with its single GetEnumerator method. The for each statement
calls the GetEnumerator method to get an implementation of the IEnumerator interface
that is then used to enumerate the collection. Let’s examine a simple implementation for
this code:

.locals init (class [mscorlib]System.Collections.ArrayList numbers,

class [mscorlib]System.Collections.IEnumerator
enumerator)

// Create the array

ldc.i4.3
newobj instance void
[mscorlib]System.Collections.ArrayList::.ctor(int32)
stloc numbers

// Populate the array

ldloc numbers
ldc.i4.1
box int32
callvirt instance int32
[mscorlib]System.Collections.ArrayList::Add(object)
pop

ldloc numbers
ldc.i4.2
box int32
callvirt instance int32
[mscorlib]System.Collections.ArrayList::Add(object)
pop

ldloc numbers
ldc.i4.2
box int32
callvirt instance int32
[mscorlib]System.Collections.ArrayList::Add(object)
pop

// Get the enumerator

ldloc numbers

callvirt instance class [mscorlib]System.Collections.IEnumerator

[mscorlib]System.Collections.IEnumerable::GetEnumerator()

stloc enumerator

br.s _CONDITION
_CONDITION:

ldloc enumerator
callvirt instance bool
[mscorlib]System.Collections.IEnumerator::MoveNext()
brfalse.s _CONTINUE

// for each statements

ldloc enumerator
callvirt instance object
[mscorlib]System.Collections.IEnumerator::get_Current()
call void [mscorlib]System.Console::WriteLine(object)

br.s _CONDITION

_CONTINUE:

Well that looks good, but there’s something missing. Can you tell what it is? Since the for
each statement is resulting in a new object being created by virtue of calling the
GetEnumerator method, it is conceivable that this object requires deterministic cleanup to
avoid resource leaks. The enumerator expresses this by implementing the IDisposable
interface. Unfortunately, the compiler is not always capable of discerning this at compile
time, though if the compiler has enough static type information there is certainly value in
taking advantage of it to optimize the generated instructions. As it turns out, the
enumerator provided by the ArrayList type does not implement the IDisposable interface.
Of course this could change in future so “optimizing” your code for this type of thing is
not a wise move. Compilers can use the isinst instruction to determine whether the
enumerator implements the IDisposable interface. Arguably this is an oversight in the
design of the IEnumerable and IEnumerator interfaces.

To address this quirk, the generic IEnumerator interface introduced in version 2.0 of the
.NET Framework makes it explicit by deriving from IDisposable so implementations
have to provide a Dispose method even if it does nothing. This certainly simplifies
matters for the caller. Collections.Generic.IEnumerator<T> is the return type for the
GetEnumerator method from the Collections.Generic.IEnumerable<T> interface
implemented by the new generic collection types. For compatibility with existing code,
the enumerator implementations also implement the older, non-generic IEnumerator
interface. Consider the following example:

Collections::Generic::List<int> numbers(3);
numbers.Add(1);
numbers.Add(2);
numbers.Add(3);

for each (int element in %numbers)

{
Console::WriteLine(element);
}
In this case there is no question of whether the enumerator implements IDisposable. The
only issue remaining is coming up with a reliable way of calling the Dispose method in
the face of exceptions. Here’s a reasonable solution using the exception handling
constructs discussed in part 5.

.locals init (class

[mscorlib]System.Collections.Generic.'List`1'<int32> numbers,
class
[mscorlib]System.Collections.Generic.'IEnumerator`1'<int32> enumerator)

// Create the array

ldc.i4.3
newobj instance void class
[mscorlib]System.Collections.Generic.'List`1'<int32>::.ctor(int32)
stloc numbers

// Populate the array

ldloc numbers
ldc.i4.1 // value
callvirt instance void class
[mscorlib]System.Collections.Generic.'List`1'<int32>::Add(!0)

ldloc numbers
ldc.i4.2 // value
callvirt instance void class
[mscorlib]System.Collections.Generic.'List`1'<int32>::Add(!0)

ldloc numbers
ldc.i4.3 // value
callvirt instance void class
[mscorlib]System.Collections.Generic.'List`1'<int32>::Add(!0)

// Get the enumerator

ldloc numbers

callvirt instance class

[mscorlib]System.Collections.Generic.'IEnumerator`1'<!0>
class
[mscorlib]System.Collections.Generic.'IEnumerable`1'<int32>::GetEnumera
tor()

stloc enumerator

.try
{

_CONDITION:

ldloc enumerator
callvirt instance bool class
[mscorlib]System.Collections.Generic.'IEnumerator`1'<int32>::MoveNext()
brfalse.s _LEAVE
 _STATEMENTS:

ldloc enumerator
callvirt instance !0 class
[mscorlib]System.Collections.Generic.'IEnumerator`1'<int32>::get_Curren
t()
call void [mscorlib]System.Console::WriteLine(int32)
br.s _CONDITION

_LEAVE:

leave.s _CONTINUE
}
finally
{
ldloc enumerator
callvirt instance void [mscorlib]System.IDisposable::Dispose()

endfinally
}

_CONTINUE:

With the comments and labels in place, it should be pretty easy to follow this code. Let’s
walk through it quickly. Two local variables are required, one for the list and one for the
enumerator. The strange-looking type names are to support generics in MSIL. 'List`1'
indicates that the List type with a single type parameter is to be used. This distinguishes
between generic types with the same name but with a different number of type
parameters. The type list between the angle brackets denotes the actual types to use in
runtime specialization. With the local variables defined, the next step is to create the list
using the newobj instruction and populate it by calling its Add method. Generics are
primarily intended for creating type-safe collections. A good example of this is the code
for adding the numbers to the list. The previous example used an ArrayList so the
numbers first had to be boxed before adding them to the collection. In this case we can
simply push the number on the stack and call the Add method. We just need to make sure
to indicate the particular specialization we require. The Add method’s single parameter is
!0 which indicates the zero-based type parameter. Now that the collection is ready, we
can implement the actual for each loop. To begin enumerating we first get the
IEnumerator<T> implementation from the collection and store it in out enumerator local
variable. At this point we enter the protected range of instructions, calling the
enumerator’s MoveNext method until it returns false. The finally exception handler is
then used to call the enumerator’s Dispose method.