Professional Documents
Culture Documents
Abstract templates.
Inspired by Pizza, Eiffel and C++, Arkadios1 tries This paper will introduce the key considerations
to extend the language of Borland Delphi in a way, we made when designing Arkadios and will also
that it adds parametric classes. Arkadios is a pre- provide an overview of features currently imple-
processor that translates a superset of the language mented and planned for future releases.
Borland Delphi into “normal” Delphi Pascal.
In order to use Arkadios, a running version of The first part introduces the extensions Arka-
Delphi is still needed. Arkadios is licensed under dios adds to the Delphi syntax, discusses how con-
the GPL and developed by Willibald Krenn, student strained genericity works and why we need it and -
at the technical university Graz (Austria). The initial finally - describes the steps taken to generate “nor-
version of this paper (and Arkadios) is the result of mal” Delphi source code.
a project at the Institute of Software Technology at In the second part, future concepts of Arkadios
TU Graz under supervision of Dr. Roderick Bloem. are being described and the third part introduces
some internal structures that are being used in Arka-
dios.
Introduction The fourth part provides a short - personal - sum-
mary of the project and the final fifth part provides
A lot of computer programming languages from references to used papers and a last example that
today consider generics as an important part of Arkadios is able to preprocess.
their feature set. This is clearly demonstrated by
the efforts to include generics in Java, C# and by
languages that have support for generics out of the Part I
box, such as Eiffel and C++.
The most commonly known incarnation of generics Features
are C++ style templates. Due to some limitations
regarding type safety, Arkadios will not implement This part will first introduce the implemented
another version of C++ templates! Instead Arkadios syntax extensions and will then explain in section
tries to pick up the Eiffel approach of constrained 1 what generic classes are for. This will be done
genericity. by looking at a simple generic container class.
The main idea behind the Eiffel approach is to limit Subsection 1.1 explains how constrained genericity
the class parameter type to a special class. If we take works in Arkadios and shows what limitations are
the polymorphism of every object oriented language currently imposed. In subsection 1.2 we discuss dif-
into consideration, this gives us the freedom to not ferent methods of translating the generic language
only use the “special class” as parameter, but also extensions to Delphi and we will show in several
all descendant classes. The reason for taking this sub sections how Arkadios solves this problem.
approach is that Arkadios is now able to apply type
checking on all methods that are using generic class As Arkadios is based on the freely available
parameters as function arguments. This would not MWSimplePasPar class, that was written by Mar-
have been possible when implementing C++ like tin Waldenburg, current support for Delphi syntax
is not complete. Arkadios is based on a version of
1 east-roman emperor; also known as “Arcadius” MWSimplePasPar that supports almost all features
1
up to Delphi 4 - however, when porting the parser TString = class
to C++, we noticed that several things for complete Value : S t r i n g ;
Delphi 4 support are missing (e.g. export of over- end ;
loaded procedures is not available). All additions to T O b j e c t L i s t = c l a s s { . . . } end ;
the Delphi language in versions 5, 6 and especially GStack <T > = c l a s s
the upcoming .NET version are not supported this AList : TObjectList ;
time. public
As Arkadios started out as a one semester project constructor Create ;
for university, this initial is more a proof of concept destructor Destroy ; override ;
implementation, than a release-ready, feature com- p r o c e d u r e Push ( I t e m : T ) ;
plete and bug free Delphi parser. Arkadios - in it’s f u n c t i o n Pop : T ;
current version - is not able to parse the Delphi sys- end ;
tem units2 or the Visual Class Library that ships with
Delphi! However, the feature set will be extended c o n s t r u c t o r GStack . C r e a t e ;
over the next few years. begin
Now, we will provide a short glimpse on what AList : = TObjectList . Create ;
Arkadios adds to ’normal’ Delphi syntax: end ;
2
Delphi itself already brings several flavors of list the methods Pop and Push of GStack have changed
classes. The most commonly used ones are called to:
TList and TObjectList. The TList class takes point-
ers as arguments where TObjectList takes objects as Listing 2: new signature of Pop and Push
arguments. It is easy to see that these classes have
the same basic functionality, but differ mainly in one p r o c e d u r e Push ( I t e m : T S t r i n g ) ;
place: The type that may be stored. f u n c t i o n Pop : T S t r i n g ;
Let’s consider the case we want to store only TCus-
tomer objects in a list. As Delphi brings no TCus- This is exactly what we need.
tomerList class with it’s class library, we have two
options to circumvent this restriction: Now, let us look at the implementation part of
Push. This method has one parameter called “Item”,
• Solution number one is to create a new de- that is of the type “T”. (See the first example). Here
scendant class of TObjectList, that takes TCus- we have a problem, because “T” is - at best guess -
tomer objects instead of plain objects. only compatible with “Pointer”, but actually we do
not even know if “T” is some type of pointer! In fact
• Solution number two is to take a normal TOb- we do not know anything about “T”.
jectList for storing our TCustomer objects and As we need to store “Item” somewhere, but we
remember every time we take an item out of (and Delphi) do not know the type of “Item”, we are
the list that it is actually a TCustomer object stuck. In order to solve our problem, we need to
and not a “normal” object. In other words: We know more about “T”.
have to insert a downcast. The easiest way to accomplish this, is to assume
“T” is at least some basic type, say TObject. Now
Both ways are somewhat inconvenient to use and at that we know that “Item” is at least some TObject,
least the latter option is a likely source of errors, as we know that we can store “Item” in a TObjectList.
there is no guarantee that the list only contains the Thus, our problem is solved.
type we expect. It turns out, that it is a useful practice, to assume
In any case, we still have quite a bunch of lines to that all generic class parameters are of type TObject.
type in order to get the task done: When using al- Naturally, this design decision narrows the set of
ternative one, we have to copy and paste complete types we can use as generic class parameter. How-
class implementations, which we have to keep in ever, we know that only the polymorphism of class
synchronization in case of bug fixes. Alternative two types gives us the freedom to use the same source
demands that we insert lots of down-casts, that are code for different types. There simply is no way to
either not type safe, or may lead to runtime errors handle Integers, Strings, Booleans, Floats etc. with
and are pretty slow. Additionally we have to know all the same source code. (Overloading comes to
exactly what types of objects our list is storing. mind!)
As neither option is in the rapid application devel- Consequently, our design decision is no big deal,
opment philosophy that Delphi has followed since as we would have to write different implementation
version one, we find it important to introduce gener- parts for a Push method that can store all the basic
ics in Delphi. types listed above and this is exactly what we want
to avoid.
1.1 How Constrained Genericity In summing up, let me stress again that a generic
Works class parameter can only be a class.
In the given example, the methods Push and Pop do
not have a “normal” Delphi type assigned that was
defined somewhere before the GStack class. In fact In the previous example, we were quite satis-
it seems that “T” might be some sort of parameter fied with “Item” as TObject. There are cases how-
for the class. ever, where we need to do some operations with the
The idea behind this approach is that the program- “Item”. As a simple example, let us assume that we
mer is able to select the type GStack will store. As want to write out the contents of the stack to disk
one can see in the example’s “var” section (look at on application termination. As experienced Delphi
variable “MyStringStack”), we selected “T” to be of developers might know, all classes that inherit from
type “TString” . This means that the signatures of TPersistent have the ability to be saved into a stream.
3
As a consequence it would be desirable to choose case, this is TObject. But when looking from out-
the base type of the generic class parameter “T” as side at the class parameter, we see the type we’ve
TPersistent. Arkadios makes this constraint avail- put in.
able by following syntax extensions: This is somewhat similar to the behavior of an ob-
ject - type function parameter, as found in “proce-
Listing 3: generic List class dure D.Demo( a: TObject);”.
There is, however, one main difference: A class
GStreamableStack < T – > parameter always is a class (a type), whereas an ob-
T P e r s i s t e n t > = c l a s s . . . end ; ject - type function parameter always is an object
(an instance of a type). In order to reflect this differ-
Note, that this example is currently not supported ent semantical meaning and to enhance readability,
by Arkadios, because TPersistent is defined in Arkadios uses the new ’->’ operator.
some of the VCL units, that can not be parsed by
Arkadios.
1.2 Translation to Delphi
Future versions of Arkadios will also support There are two possible ways to translate Arkadios
other constraints (“like current”, Interfaces) that are into Delphi: A homogenous translation or a het-
beyond the scope of the current implementation and erogenous one [Od1].
therefore not discussed in this paper. In section 1 two possible ways of overcoming
the limits of a programming language without sup-
Readers familiar with Delphi might already have port for generics were shown. Alternative number
noticed the new ’->’ operator, Arkadios uses for one demanded a re-implementation of the list class
defining constraints. The more Delphi like opera- whenever a new type should be exclusively stored in
tor would have been a colon, as it is used in variable a list. Consider this as a heterogenous implementa-
declarations. tion of generics.
The reason for the new operator is mainly for bet- Alternative two demanded to remember which type
ter readability, but also because of semantical differ- is stored in the list and to insert a downcast when
ences. needed. Consider this as the homogenous approach
to implement generics.
Unlike Java, Delphi supports two different tech-
niques for casting objects. One of them does not
lead to a runtime penalty, but is basically unsafe.
The other one slows down the running program, but
is safe - that means Delphi actually checks if the type
may be casted.
Having said that, it is probably easy to under-
stand, why we chose a homogenous approach for
translating the Arkadios language features to Del-
phi. To put it in a nutshell: The program gets shorter
and we can decide if we want to have safe or unsafe
casts. It should probably be added, that Delphi has
support for assert statements too. So that gives us a
third way to check for runtime errors in debugging
mode and does not necessarily lead to performance
losses in the release version of a program.
The outcome of the translation process is very
similar to that one described in the “Pizza to Java”
paper, which is mainly because Java and Delphi
Figure 1: Semantic of class parameters have very much in common.
Figure 1 shows the way class parameters work se- 1.2.1 Parametric Class Declaration
mantically in Arkadios. When looking from inside a
generic class at the class parameter, the parameter is To show what Arkadios has to do when translating
mapped to the given constraint. In the most general the generics extensions to Delphi, we’ll break the
4
example from Listing 1 into parts and look at the 1.2.2 Parametric Classes as Types
translation process in detail.
Let us now look at the variable declaration sec-
First, let us look at the generic GStack class. As
tion in front of the main program. As Arkadios
Arkadios parses the type name, it’ll find the Gener-
comes across the “GStack<TString>” tokens, it has
icClassParameterList3 . As the generic class param-
to create a new GStack type, with a TString class as
eter “T” does not have a constraint, Arkadios will
class parameter “T”. Arkadios checks at that time,
assume - out of the reasons mentioned in section 1.1
if TString matches the constraint we applied to the
- that “T” is a TObject. Due to the homogenous
generic class parameter. If TString would not be a
translation Arkadios creates, it now simply deletes
valid TObject, Arkadios would issue a syntax error.
the class parameters to get a valid Delphi type name.
In our case the Delphi type name will be “GStack”. Of course it is also possible to de-
Note, that this procedure allows us later on to intro- clare a generic GStack of GStacks, like
duce raw types [Bra1] more easily. “GStack<GStack<TString> >”. Arkadios will then
create a new GStack type with GStack<TString> as
The next step is to parse the class heritage section, generic class parameter “T”. Needless to say, that
that may follow the class keyword in round brackets. Arkadios will also check if we met the constraint in
If a class does not have a heritage section - as GStack this case.
- Arkadios assumes that TObject is the base class.
In the final Delphi source code, Arkadios replaces
What follows, is the translation of the class mem- the generic type by it’s homogenous class type. That
bers. Here Arkadios has to replace all generic would be “GStack” in our case (and for both exam-
class parameters by their constraint type. In other ples mentioned above!):
words: “T” has to be replaced by “TObject” in all
class members. We face a limitation here again, Listing 5: translated generic variable type
because Arkadios currently only supports methods
with generic class parameters as types. Properties
and Fields will follow in future. var
M y S t r i n g S t a c k : GStack { gen . : ’ / /
By parsing the “end” keyword, Arkadios has now GENT# G S t a c k # 1 3 9 9 0 4 6 3 2 ; # ’ } ;
translated the generic class into a Delphi class with-
out class parameters. Below one can find the com-
The text Arkadios adds as comment after the vari-
plete Delphi source of the - translated - GStack
able type is the internal name of the generic type
class:
Arkadios created.
5
Arkadios now matches the class parameter “T” 2 Generic Interfaces, Arrays
with the type we used to create MyStringStack. In
our case, Arkadios will find out that T is a TString. As already stated several times, Arkadios currently
In Delphi source code Arkadios has to add a cast has neither support for generic interfaces nor for Ar-
here, as GStack.Pop has TObject as return type, but rays of generic types.
we expect the method to give a TString object back.
As Delphi offers two different kinds of cast oper-
ators, Arkadios lets the user choose, which type
should be used. As a default, Arkadios uses the
Part II
slower - but type-safe - “as” operator.
In the end the Delphi code looks like: Future Plans
Listing 6: translated generic method call (1) This part, gives a short view of the things to come.
Most notably is probably the full support for the
S t r : = ( M y S t r i n g S t a c k . Pop a s Delphi language. That means that Arkadios will be-
TString ) come a feature complete Delphi parser that parses
VCL and CLX without issuing syntax errors. With
Let us look now at methods with class parameters this goal reached, it will be far more convenient to
as argument types. extend the support for generics. Perhaps it should be
If Arkadios encounters a call to such a method, it mentioned, that we heard of plans at Borland to add
checks that the types of the arguments match the generics to Delphi. If Borland really implements
type the class parameters stand for. In our case generics, Arkadios should of course be able to parse
Arkadios will check, that “MyStringStack.Push” Borland’s syntax too. Arkadios could then serve as
gets some sort of TString as parameter. If this is a link for developers that use a Delphi version that
not the case, Arkadios will create a syntax - error doesn’t support generics, if Borland does not pro-
message. vide the same service somehow.
6
3.3 Parametric Interfaces obj : TObject ;
begin
If Arkadios works well with generic classes, the next
myCustomerList : = GList <TObject
step is to add generic interfaces in order to get max-
>. Create ; / / Compiletime
imum use of generics. In the end Arkadios should
error
support the code below, in which the following is
o b j : = G L i s t < TCustomer2 > . C r e a t e
special:
; / / Compiletime error
• “like current” as generic parameter (see TCus- myCustomerList : = GList <
tomer class) TCustomer > . C r e a t e ; / / ok
myCustomerList : = GList <
• generic interface as constraint (see GList class) TCustomer3 > . C r e a t e ; / / ok
l i s t 2 : = myCustomerList . Clone ;
• generic interfaces implemented by generic / / Compiletime error
class (see GList class) l i s t 2 : = G L i s t < TCustomer3 > .
• generic type as generic parameter (see GList C r e a t e ; / / ok
class) DoSomething ( m y C u s t o m e r L i s t ) ;
end .
7
4 Symbol Tables 7 Parser
Symbols are stored in Arkadios in several 2-4 trees The C++ class “CakParser” is the heart of Arkadios.
[Ten1, WWW1]. As unique key serves the symbol As reference for this parser, the MWSimplePas-
name, that is stored in an AnsiString object. In fu- Par [Wald1] parser written by Martin Waldenburg
ture versions of Arkadios the usage of this class will [Wald2] was used. Martin’s parser was translated to
be removed as far as possible to gain speed. A hash C++ and the symbol generation part was added, be-
function will then serve as a helper for the unique cause MWSimplePasPar does nothing, but basically
key creation so that the tree is sorted by 32 bit inte- “NextToken”. During the work on Arkadios, sev-
ger values. eral fixes were introduced into the CakParser, that
Currently every tree node has a pointer to an are not available in Martin’s parser.
Arkadios symbol struct, where all symbol informa- CakParser is a recursive descend parser that is
tion is stored. A procedure symbol - for example able to parse all Delphi syntax up to about Delphi
- contains a pointer to a statement tree where the 4 plus some additions that may be found in Delphi
code of the procedure is stored. 6 up. (However, these additions are not saved for
code-gen!)
Additionally to the 2-4 tree there exists some-
times a linked list, that contains pointers to the tree
leafs. This list is used for generating the Delphi Part IV
source, as the tree is sorted, but Arkadios has to re-
construct the original order of the tokens. It is, of
course, not used for search operations!
Summary
This part of the paper is “reserved” for a more
5 Code Emitter personal reflection of the work that went into the
project. It should give a rough idea of how much
At first we did not intend to write a code emitter that time was invested for implementing Arkadios 0.1b
does a pretty print of the Delphi source, but as Arka- and it will also explain why the name “Arkadios”
dios began generating statements, it became obvi- was chosen.
ous that the code Arkadios emitted, was everything
- but readable. Additionally we used a recursive An- The work on Arkadios lasted until today (Feb.
siString code generation approach. As a result Arka- 2003) for about 3.5 months where only the last two
dios was not able to generate a 2 MB file in accept- months involved most of the programming work. At
able time. the beginning of the project, we first had to dig into
To put it shortly: We completely reworked the generics and evaluated different ways to implement
code-gen part and introduced a code emitter class them for Delphi.
that features indention and is much faster than the After the decision was made to implement the ho-
previous AnsiString approach. Basically the class mogenous version of generics, we began searching
takes some kind of string and copies it into a mem- for a Delphi grammar or some parser that we could
ory buffer. possibly use. Unfortunately Martin’s slightly out-
This class is not accessible through the interface! dated parser was the best we could find as there is
The only way one may influence the code-gen part no advanced Delphi parser freely available. At least
is to modify a symbol in one of the 2-4 trees. this will hopefully change in future.
Given the little material that was ready to use, it’s
probably not surprising that the work on generics
6 Interface started about two weeks before the project’s dead-
line - so this might help explain why Arkadios only
All necessary types for manipulating symbols, ex- supports a very basic version of generics.
pressions and statements are defined in the “AK-
treetypes.h” header file. All functions that are ex- Despite all difficulties, Arkadios proved to be a
ported by Arkadios are defined in the “AKinter- very interesting and challenging project. Due to the
face.h” header file. These functions are also being fact that Arkadios is licensed under GPL (LGPL is
used by Arkadios itself - they should therefore work being considered), we hope to find a few people that
as expected. are interested in generics for Delphi and that give
8
Arkadios a try.
Thanks to ...
Arkadios would not have been possible, without the
help of several people. So a very special “thank you”
flys out to
9
Part V
Appendix
References
[Bra1] Gilad Bracha, Norman Cohen, Christian Kemper, Steve Marx, Martin Odersky, Sven-Eric
Panitz, David Stoutamire, Kresten Thorup, Philip Wadler: Adding Generics to the Java Pro-
gramming Language: Participant Draft Specification , April 2001.
[Od1] Martin Odersky and Philip Wadler: Pizza into Java: Translating theory into practice. Proc. 24th
ACM Symposium on Principles of Programming Languages, Paris, France, January 1997.
[Cah1] Randall Cahoon: The Oracle at Delphi.
http://www.geocities.com/WestHollywood/Stonewall/3018/delphi.html
[Wald1] Martin Waldenburg: Sources of MWSimplePasPar used in Arkadios and other MwTools;
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/codelens/codelens/
[Wald2] Martin Waldenburg: Personal homepage; http://home.t-online.de/home/Martin.Waldenburg/
[Ten1] Allan Tengg: Implementation of a 2-4 tree in C++;
http://www.cis.tugraz.at/igi/lehre/D&A/WS99/24BaumCPP.zip, 1999
[WWW1] Short introduction of 2-4 trees: http://www.datastructures.net/presentations/24Trees.pdf
Example
program G e n e r i c L i s t ;
type
TString = class
Value : S t r i n g ;
end ;
TObjectList = class
function Last : TObject ;
end ;
10
var
e r r o r l i s t : G L i s t <GStack < T S t r i n g > >;
c o n s t r u c t o r GStack . C r e a t e ;
begin
AList : = TObjectList . Create ;
end ;
d e s t r u c t o r GStack . D e s t r o y ; o v e r r i d e ;
begin
A L i s t . Free ;
end ;
p r o c e d u r e GStack . Push ( I t e m : T ) ;
begin
A L i s t . Add ( I t e m ) ;
end ;
f u n c t i o n GStack . Pop : T ;
begin
r e s u l t : = AList . Last ( ) ;
A L i s t . D e l e t e ( A L i s t . Count – 1 ) ;
end ;
var
M y S t r i n g S t a c k : GStack <GStack < T S t r i n g > >;
Str : TString ;
AString : String ;
MyStringList : TMyStringList ;
A S t r i n g S t a c k : GStack ; / / n o t r e a l l y s u p p o r t e d . .
begin
Str : = TString . Create ;
M y S t r i n g S t a c k : = GStack < T S t r i n g > . C r e a t e ;
AStringStack : = MyStringStack ; / / not r e a l l y supported . .
M y S t r i n g S t a c k . Push ( S t r ) ;
A S t r i n g : = M y S t r i n g S t a c k . Pop . Pop . V a l u e ;
M y S t r i n g S t a c k . Free ;
S t r . Free ;
M y S t r i n g L i s t : = GList < TString >. C r e a t e ;
A S t r i n g : = M y S t r i n g L i s t . DoSomething . V a l u e ;
M y S t r i n g L i s t . Free ;
MyStringList : = TMyStringList . Create ;
S t r : = M y S t r i n g L i s t . DoSomething ;
A S t r i n g : = M y S t r i n g L i s t . DoSomething ;
M y S t r i n g L i s t . Free ;
end .
11
Contents
I Features 1
1 Generic Classes 2
1.1 How Constrained Genericity Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Translation to Delphi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 Parametric Class Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.2 Parametric Classes as Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.3 Calling Generic Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.4 Constructor Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
II Future Plans 6
3 Features to Come 6
3.1 Raw Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2 Inherit from Generic Classes, Generic Classes as Constraints . . . . . . . . . . . . . . . . 6
3.3 Parametric Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5 Code Emitter 8
6 Interface 8
7 Parser 8
IV Summary 8
V Appendix 10
12