Professional Documents
Culture Documents
1. Introduction 2. Lexical analysis 3. LL parsing 4. LR parsing 5. JavaCC and JTB 6. Semantic analysis 7. Translation and simplication 8. Liveness analysis and register allocation 9. Activation Records
Chapter 1: Introduction
Things to do
make sure you have a working mentor account start brushing up on Java review Java development tools nd http://www.cs.purdue.edu/homes/palsberg/cs352/F00/index.html add yourself to the course mailing list by writing (on a CS computer) mailer add me to cs352
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 3
Compilers
What is a compiler?
a program that translates an executable program in one language into an executable program in another language we expect the program produced by the compiler to be better, in some way, than the original
What is an interpreter?
a program that reads an executable program and produces the results of running that program usually, this involves executing the source program in some fashion
This course deals mainly with compilers Many of the same issues arise in interpreters
4
Motivation
Interest
Compiler construction is a microcosm of computer science articial intelligence algorithms greedy algorithms learning algorithms graph algorithms union-nd dynamic programming DFAs for scanning parser generators lattice theory for analysis allocation and naming locality synchronization pipeline management hierarchy management instruction set use
theory
systems
architecture
Intrinsic Merit
Compiler construction is challenging and fun
interesting problems primary responsibility for performance new architectures new challenges (blame)
real results
extremely complex interactions
Experience
You have used several compilers What qualities are important in a compiler?
1. Correct code 2. Output runs fast 3. Compiler runs fast 4. Compile time proportional to program size 5. Support for separate compilation 6. Good diagnostics for syntax errors 7. Works well with the debugger 8. Good diagnostics for ow anomalies 9. Cross language calls 10. Consistent, predictable optimization
Each of these shapes your feelings about the correct contents of this course
9
Abstract view
source code machine code
compiler
errors
Implications:
recognize legal (and illegal) programs generate correct code manage storage of all variables and code agreement on format for object (or assembly) code
errors
Implications:
intermediate representation (IR) front end maps legal code into IR back end maps IR onto target machine simplify retargeting allows multiple front ends multiple passes better code
11
A fallacy
FORTRAN code C++ code CLU code Smalltalk code front end front end front end front end back end back end back end target1 target2 target3
must encode all the knowledge in each front end must represent all the features in one IR must handle all the features in each back end
Front end
source code tokens
scanner
parser
IR
errors
Responsibilities:
recognize legal procedure report errors produce IR preliminary storage map shape the code for the back end
Front end
source code tokens
scanner
parser
IR
errors
Scanner:
id,
becomes id,
id,
character string value for a token is a lexeme typical tokens: number, id, , , , , , eliminates white space (tabs, blanks, comments ) a key issue is speed use specialized recognizer (as opposed to )
14
Front end
source code scanner tokens parser IR
errors
Parser:
recognize context-free syntax guide context-sensitive analysis construct IR(s) produce meaningful error messages attempt error correction
Front end
Context-free syntax is specied with a grammar
sheep noise ::= sheep noise This grammar denes the set of noises that a sheep makes under normal circumstances The format is called Backus-Naur form (BNF) Formally, a grammar G S is the start symbol N is a set of non-terminal symbols T is a set of terminal symbols P is a set of productions or rewrite rules (P : N N T)
16
N T P
Front end
Context free syntax can be put to better use
1 2 3 4 5 6 7 goal expr term op ::= ::= ::= ::= expr expr term op term
This grammar denes simple expressions with addition and subtraction over the tokens and S=
goal
T = N=
, ,
goal ,
expr ,
term ,
op
P = 1, 2, 3, 4, 5, 6, 7
17
Front end
Given a grammar, valid sentences can be derived by repeated substitution. Prodn. Result goal 1 expr 2 expr op term 5 expr op 7 expr 2 expr op term 4 expr op 6 expr 3 term 5
To recognize a valid sentence in some CFG, we reverse this process and build up a parse
18
Front end
A parse can be represented by a tree called a parse or syntax tree
goal
expr
expr
op
term
expr
op
term
<id:y>
term
<num:2>
<id:x>
Front end
So, compilers often use an abstract syntax tree
+ <id:x>
This is much more concise
<id:y> <num:2>
Abstract syntax trees (ASTs) are often used as an IR between front end and back end
20
Back end
instruction selection errors register allocation machine code
IR
Responsibilities
translate IR into target machine code choose instructions for each IR operation decide what to keep in registers at each point ensure conformance with system interfaces
Back end
IR instruction selection errors register allocation machine code
Instruction selection:
produce compact, fast code use available addressing modes pattern matching problem ad hoc techniques tree pattern matching string pattern matching dynamic programming
22
Back end
IR instruction selection errors register allocation machine code
Register Allocation:
have value in a register when used limited resources changes instruction choices can move loads and stores optimal allocation is difcult
Code Improvement
24
IR
opt1
IR
...
IR
opt n
IR
errors
constant propagation and folding code motion reduction of operator strength common subexpression elimination redundant store elimination dead code elimination
25
Pass 2 Environments
Source Program
Abstract Syntax
Pass 1
Reductions
Pass 3
Pass 4
IR Trees
Tokens
Lex
Parse
Parsing Actions
Canoncalize
Instruction Selection
Frame
Frame Layout
Register Assignment
Assembly Language
Assembler
Linker
Pass 9
Pass 10
Machine Language
Interference Graph
Flow Graph
Assem
26
Assem
27
Stm Stm Stm Exp Exp Exp Exp ExpList ExpList Binop Binop Binop Binop
e.g., prints: : 5 3; :
Stm ; Stm CompoundStm : Exp AssignStm ExpList PrintStm IdExp NumExp Exp Binop Exp OpExp Stm Exp EseqExp Exp ExpList PairExpList Exp LastExpList Plus Minus Times Div
1 10
28
Tree representation
: 5 3; :
1 10
CompoundStm
CompoundStm
EseqExp
IdExp a
30
31
Scanner
source code scanner tokens parser IR
errors
id,
becomes id,
id,
character string value for a token is a lexeme typical tokens: number, id, , , , , , eliminates white space (tabs, blanks, comments ) a key issue is speed use specialized recognizer (as opposed to )
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 32
Specifying patterns
A scanner must recognize various parts of the languages syntax Some parts are easy:
ws ws
33
Specifying patterns
A scanner must recognize various parts of the languages syntax
Other parts are much harder:
Operations on languages
Operation union of L and M written L M concatenation of L and M written LM Kleene closure of L written L positive closure of L written L Denition s s L or s M st s L and t M L L
L M LM
i i 0 L
Li i 1
35
Regular expressions
Patterns are often specied as regular languages Notations used to describe a regular language (or a regular set) include both regular expressions and regular grammars Regular expressions (over an alphabet ): 1. is a RE denoting the set 2. if a , then a is a RE denoting a 3. if r and s are REs, denoting Lr and Ls, then:
r r
is a RE denoting Lr
Ls s is a RE denoting Lr
is a RE denoting LrLs
rs r
is a RE denoting Lr
If we adopt a precedence for operators, the extra parentheses can go away. We assume closure, then concatenation, then alternation as the order of precedence.
36
Examples
identier letter
a 0
b c
z A B C
digit id
1 2 3 4 5 6 7 8 9
letter
letter digit
numbers integer
2 3
decimal real
9 digit
digit
complex
real real
38
Examples
Let ab
1. a b denotes a b
2. a ba b denotes aa ab ba bb i.e., a ba b aa ab ba bb 3. a denotes a aa aaa 4. a b denotes the set of all strings of as and bs (including ) i.e., a b ab 5. a ab denotes a b ab aab aaab aaaab
39
Recognizers
From a regular expression we can construct a
accept
error
a 0
b c
z A B C
1 2 3 4 5 6 7 8 9
letter
letter digit
40
41
value
az letter
AZ letter
09 digit
other other
0 1 3 3
1 1 1 2
Automatic construction
Scanner generators automatically construct code from regular expressionlike descriptions
construct a dfa use state minimization techniques emit code for the scanner (table driven or direct code )
emits C code for scanner provides macro denitions for each token (used in the parser)
43
The grammars that generate regular sets are called regular grammars Denition: In a regular grammar, all productions have one of two forms: 1. A 2. A aA a
1 s0 1 0 0 1 s2 1
The RE is 00 1101 1000 1101 1000 11
s1
s3
45
s0
s1
s2
s3
46
Finite automata
A non-deterministic nite automaton (NFA) consists of: 1. a set of states S s0 sn
2. a set of input symbols (the alphabet) 3. a transition function move mapping state-symbol pairs to sets of states 4. a distinguished start state s0 5. a set of distinguished accepting or nal states F A Deterministic Finite Automaton (DFA) is a special case of an NFA: 1. no state has a -transition, and 2. for each state s and input symbol a, there is at most one edge labelled a leaving s. A DFA accepts x iff. there exists a unique path through the transition graph from the s0 to an accepting state such that the labels along the edges spell x.
47
2. Any NFA can be converted into a DFA, by simulating sets of simultaneous states:
each DFA state corresponds to a set of NFA states possible exponential blowup
48
s0
s1
s2
s3
s0 s0 s1 s0 s2 s0 s3
a s0 s1 s0 s1 s0 s1 s0 s1
b
b s0 s0 s2 s0 s3 s0
a a
fs0 g
fs0 ; s1 g
fs0 ; s2 g
fs0 ; s3 g
a a
49
RE
DFA
NFA moves
RE
NFA w/ moves build NFA for each term connect them with moves
NFA w/ moves to DFA construct the simulation the subset construction DFA minimized DFA merge compatible states RE DFA construct Rkj i Rk1Rk1 Rk1 ik kk kj
R k 1
ij
50
RE to NFA
N
a
N a
N(A) A
N(B)
N A B
N(A) A N(A) A N(B) B
N AB
N A
51
RE to NFA: example
a
babb
2 1 4 b 5 a 3 6
ab
2 0 1 4
3 6 7
b
7 a 8
b 9 b 10
abb
52
moveT a
add state T -closures0 unmarked to Dstates while unmarked state T in Dstates mark T for each input symbol a U -closuremoveT a if U Dstates then add U to Dstates unmarked Dtrans T a U endfor endwhile -closures0 is the start state of D A state of D is accepting if it contains at least one accepting state in N
53
2 0 1 4
3 6 7 a 8 b 9 b 10
A B C
D E
1245679 1 2 4 5 6 7 10
A B C D E
a B B B B B
b C D C E C
54
L L
wcwr w
pk qk
55
So what is hard?
Language features that can cause problems:
signicant blanks FORTRAN and Algol68 ignore blanks string constants special characters in strings , , ,
nite closures some languages limit identier lengths adds states to count length FORTRAN 66 6 characters These can be swept under the rug in the language design
56
Chapter 3: LL Parsing
58
errors
Parser
performs context-free syntax analysis guides context-sensitive analysis constructs an intermediate representation produces meaningful error messages attempts error correction
Syntax analysis
Context-free syntax is specied with a context-free grammar.
Formally, a CFG G is a 4-tuple Vt Vn S P, where: Vt is the set of terminal symbols in the grammar. For our purposes, Vt is the set of tokens returned by the scanner. Vn, the nonterminals, is a set of syntactic variables that denote sets of (sub)strings occurring in the language. These are used to impose a structure on the grammar. S is a distinguished nonterminal S Vn denoting the entire set of strings in LG. This is sometimes called a goal symbol. P is a nite set of productions specifying how terminals and non-terminals can be combined to form strings in the language. Each production must have a single non-terminal on its left hand side.
The set V
Vt Vn V V Vt
1 steps
Note, LG
Syntax analysis
Grammars are often written in Backus-Naur form (BNF). Example: 1 2 3 4 5 6 7 8 goal expr :: :: expr expr op expr
op
::
This describes simple expressions over numbers and identiers. In a BNF for a grammar, we represent 1. non-terminals with angle brackets or capital letters 2. terminals with font or underline 3. productions as in the example
62
0
term
0 9
opterm
identiers, numbers, keywords REs are more concise and simpler for tokens than a grammar more efcient scanners can be built from REs (DFAs) than grammars
. . . ,
...
. . .
Syntactic analysis is complicated enough: grammar for C has around 200 productions. Factoring out lexical analysis as a separate phase makes compiler more manageable.
63
Derivations
We can view the productions of a CFG as rewriting rules. Using our example CFG: goal
expr expr op expr expr op expr op expr id, op expr op expr id, expr op expr id, num, op expr id, num, expr id, num, id,
We have derived the sentence . We denote this goal . Such a sequence of rewrites is a derivation or a parse. The process of discovering a derivation is called parsing.
64
Derivations
At each step, we chose a non-terminal to replace. This choice can lead to different derivations.
Two are of particular interest:
leftmost derivation the leftmost non-terminal is replaced at each step rightmost derivation the rightmost non-terminal is replaced at each step
Rightmost derivation
For the string : goal
op expr op id, id, op expr id, op num, id, num, id, num, id,
Again, goal
66
Precedence
goal
expr
expr
op
expr
expr
op
expr
<id,y>
<id,x>
<num,2>
Precedence
These two derivations point out a problem with the grammar. It has no notion of precedence, or implied order of evaluation.
To add precedence takes additional machinery: 1 2 3 4 5 6 7 8 9 goal expr :: :: expr expr term expr term term term factor term factor factor
term
::
factor
::
Precedence
Now, for the string : goal
Again, goal
expr expr term expr term factor expr term id, expr factor id, expr num, id, term num, id, factor num, id, id, num, id, , but this time, we build the desired tree.
69
Precedence
goal
expr
expr
term
term
term
factor
factor
factor
<id,y>
<id,x>
<num,2>
Ambiguity
If a grammar has more than one derivation for a single sentential form, then it is ambiguous Example: stmt ::=
stmt stmt
expr expr
stmt
E2
S1 S2
71
Ambiguity
May be able to eliminate ambiguities by rearranging the grammar: stmt ::= matched unmatched matched ::= expr matched matched unmatched
unmatched
This generates the same language as the ambiguous grammar, but applies the common sense rule:
Ambiguity
Ambiguity is often due to confusion in the context-free specication.
Context-sensitive confusions can arise from overloading. Example:
In many Algol-like languages, could be a function or subscripted variable. Disambiguating this statement requires context:
tokens
grammar
parser generator
parser
code
IR
start at the root of derivation tree and ll in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)
Bottom-up parsers
start at the leaves and ll in start in a state valid for legal rst tokens as input is consumed, change state to encode possibilities (recognize valid prexes ) use a stack to store both state and sentential forms
75
Top-down parsing
A top-down parser starts with the root of the parse tree, labelled with the start or goal symbol of the grammar.
To build a parse, it repeats the following steps until the fringe of the parse tree matches the input string 1. At a node labelled A, select a production A appropriate child for each symbol of and construct the
2. When a terminal is added to the fringe that doesnt match the input string, backtrack 3. Find the next node to be expanded (must have a label in Vn)
term
::
factor
::
77
Example
Prodn 1 2 4 7 9 3 4 7 9 7 8 5 7 8 9 Sentential form goal expr expr term term term factor term term term expr expr term term term factor term term term term factor Input
78
Example
Another possible parse for Prodn Sentential form goal 1 expr 2 expr term 2 expr term term 2 expr term 2 expr term 2 Input
If the parser makes the wrong choices, expansion doesnt terminate. This isnt a good property for a parser to have. (Parsers should terminate!)
79
Left-recursion
Top-down parsers cannot handle left-recursion in a grammar
Formally, a grammar is left-recursive if A Vn such that A A for some string
Eliminating left-recursion
To remove left-recursion, we can transform the grammar
Consider the grammar fragment: foo :: foo
where and do not start with foo We can rewrite this as: foo bar where bar is a new non-terminal :: :: bar bar
Example
Our expression grammar contains two cases of left-recursion expr :: expr term expr term term term factor term factor factor term expr term expr term expr factor term factor term factor term
term
::
term term
:: ::
Example
This cleaner grammar denes the same language 1 2 3 4 5 6 7 8 9 It is goal expr :: :: expr term expr term expr term factor term factor term factor
term
::
factor
::
Example
Our long-suffering expression grammar: 1 2 3 4 5 6 7 8 9 10 11 goal expr expr :: :: :: expr term expr term expr term expr factor term factor term factor term
term term
:: ::
factor
::
Fortunately
large subclasses of CFGs can be parsed with limited lookahead most programming language constructs can be expressed in a grammar that falls in these subclasses
Among the interesting subclasses are: LL(1): left to right scan, left-most derivation, 1-token lookahead; and LR(1): left to right scan, right-most derivation, 1-token lookahead
85
Predictive parsing
Basic idea:
For any two productions A , we would like a distinct way of choosing the correct production to expand. For some RHS G, dene FIRST as the set of tokens that appear rst in some string derived from That is, for some w Vt, w FIRST iff. w.
and A
FIRST
FIRST
This would allow the parser to make a correct choice with a lookahead of only one symbol!
Left factoring
What if a grammar does not have this property?
Sometimes, we can transform a grammar to have this property. For each non-terminal A nd the longest prex common to two or more of its alternatives. if then replace all of the A productions A 1 2 n with A A A 1 2 n where A is a new non-terminal. Repeat until no two alternatives for a single non-terminal have a common prex.
87
Example
Consider a right-recursive version of the expression grammar: 1 2 3 4 5 6 7 8 9 goal expr :: :: expr term expr term expr term factor term factor term factor
term
::
factor
::
To choose between productions 2, 3, & 4, the parser must see past the or and look at the , , , or .
FIRST 2 FIRST 3 FIRST 4
Example
There are two nonterminals that must be left factored: expr :: term term term factor factor factor
term
::
Applying the transformation gives us: expr expr :: :: term expr expr expr
term term
:: ::
Example
Substituting back into the grammar yields 1 2 3 4 5 6 7 8 9 10 11 goal expr expr :: :: :: expr term expr expr expr factor term term term
term term
:: ::
factor
::
Example
1 2 6 11 9 4 2 6 10 7 6 11 9 5 Sentential form goal expr term expr factor term expr term expr term expr expr expr expr term expr factor term expr term expr term expr term expr term expr factor term expr term expr term expr expr
Input
92
Generality
Question: By left factoring and eliminating left-recursion, can we transform an arbitrary context-free grammar to a form where it can be predictively parsed with a single token lookahead? Answer: Given a context-free grammar that doesnt meet our conditions, it is undecidable whether an equivalent grammar exists that does meet our conditions. Many context-free languages do not have such a grammar: an0bn n 1 an1b2n n 1
Must look past an arbitrary number of as to discover the 0 or the 1 and so determine the derivation.
93
94
can pop 3, build and push subtree can stack nodes , can pop and return tree can pop 3, build and push subtree
96
Our recursive descent parser encodes state information in its runtime stack, or call stack.
Using recursive procedure calls to implement a stack abstraction may not be particularly efcient. This suggests other implementation methods:
97
source code
scanner
tokens
table-driven parser
IR
parsing tables
Table-driven parsers
A parser generator system often looks like:
stack
source code
scanner
tokens
table-driven parser
IR
grammar
parser generator
parsing tables
This is true for both top-down (LL) and bottom-up (LR) parsers
99
100
4 9 7 8
term term
:: ::
3 9
$ 5 9
factor
::
we use $ to represent
101
FIRST
For a string of grammar symbols , dene FIRST as:
the set of terminal symbols that begin strings derived from : a Vt a If then FIRST contains the set of tokens valid in the initial position in
FIRST
Vt then FIRSTX is
Y1Y2 Yk :
FIRST Y1
(a) Put
(c) If FIRST Y1
in FIRST X
FIRST Yk
FOLLOW
For a non-terminal A, dene FOLLOWA as the set of terminals that can appear immediately to the right of A in some sentential form Thus, a non-terminals FOLLOW set species the tokens that can legally appear after it. A terminal symbol has no FOLLOW set. To build FOLLOWA: 1. Put $ in 2. If A (a) Put
FOLLOW
goal
B:
FIRST
(b) If (i.e., A B) or FIRST (i.e., ) then put FOLLOWA in FOLLOWB Repeat until no more additions can be made
in FOLLOWB
103
LL(1) grammars
Previous denition A grammar G is LL(1) iff. for all non-terminals A, each distinct pair of pro FIRST . ductions A and A satisfy the condition FIRST
What if A ?
1 2
n:
FIRST 1 FIRST 2
FIRST n
104
LL(1) grammars
Provable facts about LL(1) grammars: 1. No left-recursive grammar is LL(1) 2. No ambiguous grammar is LL(1) 3. Some languages have no LL(1) grammar 4. A free grammar where each alternative expansion for A begins with a distinct terminal is a simple LL(1) grammar. Example S aS a is not LL(1) because FIRSTaS FIRSTa S aS S aS accepts the same language and is LL(1)
105
to M A a to M A b to M A $
b FOLLOWA, add A
2. Set each undened entry of M to If M A a with multiple entries then grammar is not LL(1).
Note: recall a b Vt , so a b
106
Example
Our long-suffering expression grammar: S E E
FIRST FOLLOW
E T E E
T T E F
FT T
S E E T T F
$ $ $
$ $ $
S S E E E T T T F F
E S TE E
FT T F
E TE
FT
E
T
T T
E TT
107
stmt stmt
stmt
stmt
Now, FIRST stmt Also, FOLLOW stmt $ FOLLOW stmt But, FIRST stmt stmt ::
stmt
and
stmt ::
Error recovery
Key notion:
For each non-terminal, construct a set of terminals on which the parser can synchronize When an error occurs looking for A, scan until an element of SYNCH A is found
Building SYNCH: 1. a FOLLOWA a SYNCH A 2. place keywords that start statements in SYNCH A 3. add symbols in FIRST A to
SYNCH A
If we cant match a terminal on top of stack: 1. pop the terminal 2. print a message saying the terminal was inserted 3. continue the parse (i.e., SYNCH a Vt a )
109
Chapter 4: LR Parsing
110
Some denitions
Recall
For a grammar G, with start symbol S, any string such that S is called a sentential form
If Vt, then is called a sentence in LG Otherwise it is just a sentential form (not a sentence in LG)
A left-sentential form is a sentential form that occurs in the leftmost derivation of some sentence. A right-sentential form is a sentential form that occurs in the rightmost derivation of some sentence.
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 111
Bottom-up parsing
Goal:
Given an input string w and a grammar G, construct a parse tree by starting at the leaves and working to the root.
The parser repeatedly matches a right-sentential form from the language against the trees upper frontier. At each match, it applies a reduction to build on the frontier:
each reduction matches an upper frontier of the partially built tree to the RHS of some production each reduction adds a node on top of the frontier
Example
Consider the grammar 1 S 2 A 3 4 B and the input string Prodn. Sentential Form 3 2 A 4 A 1 aABe S The trick appears to be scanning the input and nding valid sentential forms. AB A
113
Handles
What are we trying to nd?
A substring of the trees upper frontier that matches some production A where reducing to A is one step in the reverse of a rightmost derivation We call such a string a handle. Formally: a handle of a right-sentential form is a production A and a position in where may be found and replaced by A to produce the previous right-sentential form in a rightmost derivation of
Because is a right-sentential form, the substring to the right of a handle contains only terminal symbols.
114
Handles
The handle A
w
115
Handles
Theorem:
If G is unambiguous then every right-sentential form has a unique handle.
a unique production A
116
Example
The left-recursive expression grammar (original form) Prodn. 1 3 5 9 7 8 4 7 9 Sentential Form goal expr expr term expr term factor expr term expr factor expr term factor
1 2 3 4 5 6 7 8 9
goal expr
:: ::
term
::
factor
::
expr expr term expr term term term factor term factor factor
117
Handle-pruning
The process to construct a bottom-up parse is called handle-pruning. To construct a rightmost derivation S 0 1 2 n1 n w
1.
Ai
i Ai i
i i1
2.
118
Stack implementation
One scheme to implement a handle-pruning, bottom-up parser is called a shift-reduce parser. Shift-reduce parsers use a stack and an input buffer 1. initialize stack with $ 2. Repeat until the top of the stack is the goal symbol and the input token is $ a) nd the handle if we dont have a handle on top of the stack, shift an input symbol onto the stack b) prune the handle if we have a handle A on the stack, reduce
i) pop symbols off the stack ii) push A onto the stack
119
Example: back to
Stack $ $ $ factor $ term $ expr $ expr $ expr $ expr factor $ expr term $ expr term $ expr term $ expr term factor $ expr term $ expr $ goal
1 2 3 4 5 6 7 8 9
goal expr
:: ::
term
::
factor ::
expr expr term expr term term term factor term factor factor
Input
Action shift reduce 9 reduce 7 reduce 4 shift shift reduce 8 reduce 7 shift shift reduce 9 reduce 5 reduce 3 reduce 1 accept
1. Shift until top of stack is the right end of a handle 2. Find the left end of the handle and reduce 5 shifts + 9 reduces + 1 accept
120
Shift-reduce parsing
Shift-reduce parsers are simple to understand
A shift-reduce parser has just four canonical actions: 1. shift next input symbol is shifted onto the top of the stack 2. reduce right end of handle is on top of stack; locate left end of handle within the stack; pop handle off stack and push appropriate non-terminal LHS 3. accept terminate parsing and signal success 4. error call an error recovery routine The key problem: to recognize handles (not covered in this course).
121
LRk grammars
Informally, we say that a grammar G is LRk if, given a rightmost derivation S 0 1 2 n w
we can, for each right-sentential form in the derivation, 1. isolate the handle of each right-sentential form, and 2. determine the production by which to reduce by scanning i from left to right, going at most k symbols beyond the right end of the handle of i.
122
LRk grammars
Formally, a grammar G is LRk iff.: 1. S Aw rm w, and rm 2. S Bx rm y, and rm 3. FIRST k w FIRSTk y
Ay
Bx
i.e., Assume sentential forms w and y, with common prex and common k-symbol lookahead FIRSTk y FIRSTk w, such that w reduces to Aw and y reduces to Bx. But, the common prex means y also reduces to Ay, for the same result. Thus Ay Bx.
123
everyones favorite parser virtually all context-free programming language constructs can be expressed in an LR(1) form LR grammars are the most general grammars parsable by a deterministic, bottom-up parser efcient parsers can be implemented for LR(1) grammars LR parsers detect an error as soon as possible in a left-to-right scan of the input LR grammars describe a proper superset of the languages recognized by predictive (i.e., LL) parsers LLk: recognize use of a production A seeing rst k symbols of LRk: recognize occurrence of (the handle) having seen all of what is derived from plus k symbols of lookahead
124
needed for termination in predictive parsers requires more stack space right associative operators
Left Recursion: works ne in bottom-up parsers limits required stack space left associative operators
Rule of thumb: right recursion for top-down parsers left recursion for bottom-up parsers
125
Parsing review
Recursive descent
A hand coded recursive descent parser directly encodes a grammar (typically an LL(1) grammar) into a series of mutually recursive procedures. It has most of the linguistic limitations of LL(1). LLk An LLk parser must be able to recognize the use of a production after seeing only the rst k symbols of its right hand side. LRk An LRk parser must be able to recognize the occurrence of the right hand side of a production after having seen all that is derived from that right hand side with k symbols of lookahead.
126
127
Can be thought of as Lex and Yacc for Java. It is based on LL(k) rather than LALR(1). Grammars are written in EBNF. The Java Compiler Compiler transforms an EBNF grammar into an LL(k) parser. The JavaCC grammar can have embedded action code written in Java, just like a Yacc grammar can have embedded action code written in C. The lookahead can be changed by writing The whole input is given in just one le (not two).
128
129
Example of a production:
130
131
Sneak Preview
When using the Visitor pattern,
133
134
Advantage: The code is written without touching the classes and . Drawback: The code constantly uses type casts and determine what class of object it is considering.
to
135
We can now compute the sum of all components of a given -object by writing .
136
Advantage: The type casts and operations have disappeared, and the code can be written in a systematic way. Disadvantage: For each new operation on -objects, new dedicated methods have to be written, and all classes must be recompiled.
137
Divide the code into an object structure and a Visitor (akin to Functional Programming!) Insert an method in each class. Each accept method takes a Visitor as argument. A Visitor contains a method for each class (overloading!) A method for a class C takes an argument of type C.
138
The purpose of the methods is to invoke the method in the Visitor which can handle the current object.
139
The control ow goes back and forth between the methods in the Visitor and the methods in the object structure.
Notice: The methods describe both 1) actions, and 2) access of subobjects.
140
Comparison
The Visitor pattern combines the advantages of the two other approaches. Frequent type casts? Yes No No Frequent recompilation? No Yes No
The advantage of Visitors: New methods without recompilation! Requirement for using Visitors: All classes must have an accept method.
JJTree (from Sun Microsystems) and the Java Tree Builder (from Purdue University), both frontends for The Java Compiler Compiler from Sun Microsystems.
141
Visitors: Summary
Visitor makes adding new operations easy. Simply write a new visitor. A visitor gathers related operations. It also separates unrelated ones. Adding new classes to the object structure is hard. Key consideration: are you most likely to change the algorithm applied over an object structure, or are you most like to change the classes of objects that make up the structure. Visitors can accumulate state. Visitor can break encapsulation. Visitors approach assumes that the interface of the data structure classes is powerful enough to let visitors do their job. As a result, the pattern often forces you to provide public operations that access internal state, which may compromise its encapsulation.
142
a JavaCC grammar with embedded Java code for building a syntax tree; one class for every form of syntax tree node; and a default visitor which can do a depth-rst traversal of a syntax tree.
143
JavaCC grammar
JTB
Parser
Syntax-tree-node
classes
Default visitor
144
Using JTB
145
Example (simplied)
For example, consider the Java 1.1 production
JTB produces:
Notice that the production returns a syntax tree represented as an object.
146
Example (simplied)
JTB produces a syntax-tree-node class for
Notice the method; it invokes the method for the default visitor.
in
147
Example (simplied)
The default visitor looks like this:
Notice the body of the method which visits each of the three subtrees of the node.
148
Example (simplied)
Here is an example of a program which operates on syntax trees for Java 1.1 programs. The program prints the right-hand side of every assignment. The entire program is six lines:
When this visitor is passed to the root of the syntax tree, the depth-rst traversal will begin, and when nodes are reached, the method in is executed. Notice the use of . It is a visitor which pretty prints Java 1.1 programs. JTB is bootstrapped.
149
150
Semantic Analysis
The compilation process is driven by the syntactic structure of the program as discovered by the parser
Semantic routines:
interpret meaning of the program based on its syntactic structure two purposes: nish analysis by deriving context-sensitive information begin synthesis by generating the IR or target code associated with individual productions of a context free grammar or subtrees of a syntax tree
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 151
Context-sensitive analysis
What context-sensitive questions might the compiler ask? 1. Is scalar, an array, or a function? 2. Is declared before it is used? 3. Are any names declared but not used? 4. Which declaration of does this reference? 5. Is an expression type-consistent ? 6. Does the dimension of a reference match the declaration? 7. Where can be stored? (heap, stack, 9. Is dened before it is used? 10. Is an array reference in bounds ? 11. Does function produce a constant value? 12. Can be implemented as a memo-function? ) 8. Does reference the result of a malloc()?
Context-sensitive analysis
Why is context-sensitive analysis hard?
answers depend on values, not syntax questions and answers involve non-local information answers may involve computation
Several alternatives:
abstract syntax tree specify non-local computations (attribute grammars) automatic evaluators symbol tables central store for facts express checking code language design simplify language avoid problems
153
Symbol tables
For compile-time efciency, compilers often use a symbol table:
What items should be entered? variable names dened constants procedure and function names literal constants and strings source text labels compiler-generated temporaries (well get there) (eld offsets and lengths )
textual name data type dimension information declaring procedure lexical level of declaration storage class offset in storage if record, pointer to structure table if parameter, by-reference or by-value? can it be aliased? to what other names? number and type of arguments to functions
155
(for aggregates)
(base address )
when we ask about a name, we want the most recent declaration the declaration may be from the current scope or some enclosing scope innermost scope overrides declarations from outer scopes
Key point: new declarations (usually) occur only in current scope What operations do we need?
remembers current state of table restores table to state at most recent scope that
Attribute information
Attributes are internal representation of declarations Symbol table associates names with attributes Names may have different attributes depending on their meaning:
variables: type, procedure level, frame offset types: type descriptor, data size/alignment constants: type, value procedures: formals (names/types), result type, block information (local decls.), frame size
157
Type expressions
Type expressions are a textual representation for types: 1. basic types: boolean, char, integer, real, etc. 2. type names 3. constructed types (constructors applied to type expressions): (a) arrayI T denotes array of elements type T , index type I e.g., array1 10 integer
(b) T1 T2 denotes Cartesian product of type expressions T1 and T2 (c) records: elds have names e.g., record integer real (d) pointerT denotes the type pointer to object of type T (e) D R denotes type of function mapping domain D to range R e.g., integer integer integer
158
Type descriptors
Type descriptors are compile-time structures representing type expressions e.g., char char pointerinteger
!
char char pointer integer
or
!
char pointer integer
159
Type compatibility
Type checking needs to determine type equivalence Two approaches:
Name equivalence: each type name is a distinct type Structural equivalence: two types are equivalent iff. they have the same structure (after substituting type expressions for type names)
s t iff. s and t are the same basic types arrays1 s2 arrayt1 t2 iff. s1 t1 and s2 s1 s2 t1 t2 iff. s1 t1 and s2 t2 pointers pointert iff. s t s1 s2 t1 t2 iff. s1 t1 and s2 t2
t2
160
and have the same type , and have the same type and have different type
Under structural equivalence all variables have the same type Ada/Pascal/Modula-2/Tiger are somewhat confusing: they treat distinct type denitions as distinct types, so
each constructor or basic type creates a node each name creates a leaf (associated with the types descriptor)
next
last
link
= pointer
pointer pointer
cell
Type expressions are equivalent if they are represented by the same node in the graph
162
We may want to eliminate the names from the type graph Eliminating name from type graph for record:
cell
= record
cell
163
cell
= record
164
165
Application of binary operator o: PLUS, MINUS, MUL, DIV AND, OR, XOR [bitwise logical] LSHIFT, RSHIFT [logical shifts] ARSHIFT [arithmetic right-shift] to integer operands e1 (evaluated rst) and e2 (evaluated second) Contents of a word of memory starting at address e Procedure call; expression f is evaluated before arguments e1 Expression sequence; evaluate s for side-effects, then e for result en
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 166
Evaluate e1 then e2 , yielding a and b, respectively; compare a with b using relational operator o: EQ, NE [signed and unsigned integers] LT, GT, LE, GE [signed] ULT, ULE, UGT, UGE [unsigned] jump to t if true, f if false Statement s1 followed by s2 Dene constant value of name n as current code address; NAMEn can be used as target of jumps, calls, etc.
167
SEQ s1 s2 LABEL n
Kinds of expressions
Expression kinds indicate how expression might be used Ex(exp) expressions that compute a value Nx(stm) statements: expressions that compute no value Cx conditionals (jump to true and false destinations) RelCx(op, left, right) IfThenElseExp expression/statement depending on use Conversion operators allow use of one form in context of another: unEx convert to tree expression that computes value of inner tree unNx convert to tree statement that computes inner tree but returns no value unCx(t, f) convert to statement that evaluates inner tree and branches to true destination if non-zero, false destination otherwise
168
Translating Tiger
Simple variables: fetch with a MEM: MEM BINOP Ex(MEM((TEMP fp, CONST k)))
PLUS TEMP fp CONST k where fp is home frame of variable, found by following static links; k is offset of variable in that level Tiger array variables: Tiger arrays are pointers to array base, so fetch with a MEM like any other variable: Ex(MEM((TEMP fp, CONST k))) Thus, for e i : Ex(MEM((e.unEx, (i.unEx, CONST w))))
i is index expression and w is word size all values are word-sized (scalar) in Tiger Note: must rst check array index i sizee; runtime will put size in word preceding array base
169
Translating Tiger
Tiger record variables: Again, records are pointers to record base, so fetch like other variables. For e. : Ex(MEM((e.unEx, CONST o))) where o is the byte offset of the eld in the record Note: must check record pointer is non-nil (i.e., non-zero) String literals: Statically allocated, so just use the strings label Ex(NAME(label )) where the literal will be emitted as:
e2 fn en in the (preferably GCd) heap, rst allocate
Ex( ESEQ(SEQ(MOVE(TEMP r, externalCall(allocRecord, [CONST n])), SEQ(MOVE(MEM(TEMP r), e1 .unEx)), SEQ(. . . , MOVE(MEM(+(TEMP r, CONST n 1w)), en .unEx))), TEMP r)) where w is the word size Array creation: e1 e2 : Ex(externalCall(initArray, [e1 .unEx, e2 .unEx]))
170
Control structures
Basic blocks :
a sequence of straight-line code if one instruction executes then they all execute a maximal sequence of instructions without branches a label starts a new basic block
Overview of control structure translation: control ow links up the basic blocks ideas are simple implementation requires bookkeeping some care is needed for good code
171
while loops
while c do s: 1. evaluate c 2. if false jump to next statement after loop 3. if true fall into loop body 4. branch to top of loop e.g., test : if not(c) jump done s jump test
done: Nx( SEQ(SEQ(SEQ(LABEL test, c.unCx(body, done)), SEQ(SEQ(LABEL body, s.unNx), JUMP(NAME test ))), LABEL done))
repeat e1 until e2
for loops
for 1. 2. 3. 4. 5. 6. := e1 to e2 do s evaluate lower bound into index variable evaluate upper bound into limit variable if index limit jump to next statement after loop fall through to loop body increment index if index limit jump to top of loop body t1 e1 t2 e2 if t1 t2 jump done body : s t1 t1 1 if t1 t2 jump body done:
For break statements: when translating a loop push the done label on some stack break simply jumps to label on top of stack when done translating loop and its body, pop the label
173
Function calls
f e1 en: Ex(CALL(NAME label f , [sl,e1,. . . en ])) where sl is the static link for the callee f , found by following n static links from the caller, n being the difference between the levels of the caller and the callee
174
Comparisons
Translate a op b as: RelCx(op, a.unEx, b.unEx) When used as a conditional unCxt f yields: CJUMP(op, a.unEx, b.unEx, t, f ) where t and f are labels. When used as a value unEx yields: ESEQ(SEQ(MOVE(TEMP r, CONST 1), SEQ(unCx(t, f), SEQ(LABEL f, SEQ(MOVE(TEMP r, CONST 0), LABEL t)))), TEMP r)
175
Conditionals
The short-circuiting Boolean operators have already been transformed into if-expressions in Tiger abstract syntax: e.g., x 5&a b turns into if x 5 then a b else 0 Translate if e1 then e2 else e3 into: IfThenElseExp(e1, e2, e3) When used as a value unEx yields: ESEQ(SEQ(SEQ(e1 .unCx(t, f), SEQ(SEQ(LABEL t, SEQ(MOVE(TEMP r, e2.unEx), JUMP join)), SEQ(LABEL f, SEQ(MOVE(TEMP r, e3.unEx), JUMP join)))), LABEL join), TEMP r) As a conditional unCxt f yields: SEQ(e1.unCx(tt,ff), SEQ(SEQ(LABEL tt, e2.unCx(t, f )), SEQ(LABEL ff, e3.unCx(t, f ))))
176
Conditionals: Example
Applying unCxt f to if x 5 then a b else 0:
SEQ(CJUMP(LT, x.unEx, CONST 5, tt, ff), SEQ(SEQ(LABEL tt, CJUMP(GT, a.unEx, b.unEx, t, f )), SEQ(LABEL ff, JUMP f ))) or more optimally: SEQ(CJUMP(LT, x.unEx, CONST 5, tt, f ), SEQ(LABEL tt, CJUMP(GT, a.unEx, b.uneX, t, f )))
177
translates to:
MEM(+(TEMP fp, +(CONST k 2w, (CONST w, e.unEx))))
where k is offset of static array from fp, w is word size In Pascal, multidimensional arrays are treated as arrays of arrays, so is equivalent to A[i][j], so can translate as above.
178
Multidimensional arrays
Array allocation: constant bounds allocate in static area, stack, or heap no run-time descriptor is needed dynamic arrays: bounds xed at run-time allocate in stack or heap descriptor is needed dynamic arrays: bounds can change at run-time allocate in heap descriptor is needed
179
Multidimensional arrays
Array layout:
Contiguous: 1. Row major Rightmost subscript varies most quickly: Used in PL/1, Algol, Pascal, C, Ada, Modula-3 2. Column major Leftmost subscript varies most quickly: Used in FORTRAN By vectors Contiguous vector of pointers to (non-contiguous) subarrays
180
which can be rewritten as variable part i1D2 Dn i2D3 Dn in1Dn in L1D2 Dn L2D3 Dn Ln1Dn Ln
constant part address of i1 in : address( ) + ((variable part constant part) element size)
181
case statements
case E of V1: S1 . . . Vn : Sn end 1. evaluate the expression 2. nd value in case list equal to value of expression 3. execute statement associated with value found 4. jump to next statement after case Key issue: nding the right case
sequence of conditional jumps (small case set) O cases binary search of an ordered jump table (sparse case set) Olog2 cases hash table (dense case set) O1
182
case statements
case E of V1: S1 . . . Vn : Sn end One translation approach: t := expr jump test L1 : S1 jump next L2 : code for S2 jump next ... Ln : code for Sn jump next test: if t V1 jump L1 if t V2 jump L2 ... if t Vn jump Ln code to raise run-time exception next:
183
Simplication
Goal 1: No SEQ or ESEQ. Goal 2: CALL can only be subtree of EXP(. . . ) or MOVE(TEMP t,. . . ). lift ESEQs up tree until they can become SEQs turn SEQs into linear list
ESEQ(s1, ESEQ(s2, e)) BINOP(op, ESEQ(s, e1 ), e2 ) MEM(ESEQ(s, e1 )) JUMP(ESEQ(s, e1 )) CJUMP(op, ESEQ(s, e1 ), e2 , l1 , l2 ) BINOP(op, e1 , ESEQ(s, e2 )) CJUMP(op, e1 , ESEQ(s, e2 ), l1 , l2 ) MOVE(ESEQ(s, e1), e2 ) CALL( f , a) ESEQ(SEQ(s1,s2), e) ESEQ(s, BINOP(op, e1 , e2 )) ESEQ(s, MEM(e1 )) SEQ(s, JUMP(e1 )) SEQ(s, CJUMP(op, e1 , e2 , l1 , l2 )) ESEQ(MOVE(TEMP t, e1 ), ESEQ(s, BINOP(op, TEMP t, e2 ))) SEQ(MOVE(TEMP t, e1 ), SEQ(s, CJUMP(op, TEMP t, e2 , l1 , l2 ))) SEQ(s, MOVE(e1 , e2 )) ESEQ(MOVE(TEMP t, CALL( f , a)), TEMP(t))
184
Transformations:
185
Register allocation
IR instruction selection errors register allocation machine code
Register allocation:
have value in a register when used limited resources changes instruction choices can move loads and stores optimal allocation is difcult NP-complete for k 1 registers
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 186
Liveness analysis
Problem:
Approach: temporaries with disjoint live ranges can map to same register if not enough registers then spill some temporaries (i.e., keep them in memory)
The compiler must perform liveness analysis for each temporary: It is live if it holds a value that may be needed in future
187
Control ow analysis
Before performing liveness analysis, need to understand the control ow by building a control ow graph (CFG):
nodes may be individual program statements or basic blocks edges represent potential ow of control
Out-edges from node n lead to successor nodes, succ n In-edges to node n come from predecessor nodes, pred n
Example: a 0 L1 : b a 1 c cb a b2 if a N goto L1 return c
188
Liveness analysis
Gathering liveness information is a form of data ow analysis operating over the CFG:
liveness of variables ows around the edges of the graph assignments dene a variable, v: defv def n usev use n set of graph nodes that dene v set of variables dened by n set of nodes that use v set of variables used in n
Liveness : v is live on edge e if there is a directed path from e to a use of v that does not pass through any defv
v is live-in at node n if live on any of ns in-edges v use n v is live-out at n if live on any of ns out-edges
189
Liveness analysis
Dene: in n : variables live-in at n in n : variables live-out at n Then:
out n succ n
Note:
ssuccn
in s
out n
in n in n
use n
out n
def n def n
190
use n
out
def n
in n
out n
out n
should order computation of inner loop to follow the ow liveness ows backward along control-ow arcs, from out to in nodes can just as easily be basic blocks to reduce CFG size could do one variable at a time, from uses back to defs, noting liveness along the way
191
N nodes in CFG N variables N elements per in/out ON time per set-union for loop performs constant number of set operations per node ON 2 time for for loop each iteration of repeat loop can only add to each set sets can contain at most every variable sizes of all in and out sets sum to 2N 2, bounding the number of iterations of the repeat loop worst-case complexity of ON 4 ordering can cut repeat loop down to 2-3 iterations ON or ON 2 in practice
192
v has some later use downstream from n v outn but not the converse
Conservatively assuming a variable is live does not break the program; just means more registers may be needed. Assuming a variable is dead when it is really live will break things. May be many possible solutions but want the smallest: the least xpoint. The iterative liveness computation computes this least xpoint.
193
Register allocation
IR instruction selection errors register allocation machine code
Register allocation:
have value in a register when used limited resources changes instruction choices can move loads and stores optimal allocation is difcult NP-complete for k 1 registers
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 194
195
196
Coalescing
Can delete a move instruction when source s and destination d do not interfere: coalesce them into a new node whose edges are the union of those of s and d In principle, any pair of non-interfering nodes can be coalesced unfortunately, the union is more constrained and new graph may no longer be K-colorable overly aggressive
197
build
done
any
coa spil l
aggressive coalesce
simplify
done
spill
any
select
lesc
198
Conservative coalescing
Apply tests for coalescing that preserve colorability. Suppose a and b are candidates for coalescing into node ab
George: coalesce only if all signicant-degree neighbors of a already interfere with b simplify can remove all insignicant-degree neighbors of a
remaining signicant-degree neighbors of a already interfere with b so coalescing does not increase the degree of any node
199
remove associated move instruction if resulting node is non-move-related it can now be simplied repeat simplify and coalesce until only signicant-degree or uncoalesced moves
4. Freeze : if unable to simplify or coalesce (a) look for move-related node of low-degree (b) freeze its associated moves (give up hope of coalescing them) (c) now treat as a non-move-related and resume iteration of simplify and coalesce 5. Spill : if no low-degree nodes (a) select candidate for spilling (b) remove to stack and continue simplifying 6. Select : pop stack assigning colors (including actual spills) 7. Start over : if select has no actual spills then nished, otherwise (a) rewrite code to fetch actual spills before each use and store after each denition (b) recalculate liveness and repeat
200
build simplify
actual spill
an y
spills
done
201
Spilling
Spills require repeating build and simplify on the whole program To avoid increasing number of spills in future rounds of build can simply discard coalescences Alternatively, preserve coalescences from before rst potential spill, discard those after that point Move-related spilled temporaries can be aggressively coalesced, since (unlike registers) there is no limit on the number of stack-frame locations
202
Precolored nodes
Precolored nodes correspond to machine registers (e.g., stack pointer, arguments, return address, return value)
select and coalesce can give an ordinary temporary the same color as a precolored register, if they dont interfere
e.g., argument registers can be reused inside procedures for a temporary
So, treat precolored nodes as having innite degree This also avoids needing to store large adjacency lists for precolored nodes; coalescing can use the George criterion
203
204
calls interfere with caller-save registers a cross-call variable interferes with all precolored caller-save registers, as well as with the fresh temporaries created for callee-save copies, forcing a spill choose nodes with high degree but few uses, to spill the fresh calleesave temporary instead of the cross-call variable this makes the original callee-save register available for coloring the cross-call variable
205
Example
Temporaries are , , , , Assume target machine with K 3 registers: , (caller-save/argument/resul (callee-save) The code generator has already made arrangements to save explicitly by copying into temporary and back again
206
Example (cont.)
Interference graph:
r3 b c e
r2
r1
No opportunity for simplify or freeze (all non-precolored nodes have signicant degree K) Any coalesce will produce a new node adjacent to degree nodes Must spill based on priorities: Node uses defs uses defs outside loop inside loop 2 10 0 1 10 1 2 10 0 2 10 2 1 10 3 Node has lowest priority so spill it degree 4 4 6 4 3 priority 0.50 2.75 0.33 5.50 10.30 K signicant-
207
Example (cont.)
removed:
r2
r1
Only possibility is to coalesce and : will have K signicantdegree neighbors (after coalescing will be low-degree, though highdegree before)
r3 b
r2
r1
ae
208
Example (cont.)
and ):
r2b
r1
ae
Coalescing
r3
with ):
r2b
r1ae
209
Example (cont.)
Cannot coalesce with because the move is constrained : the nodes interfere. Must simplify :
r3
r2b
r1ae
Graph now has only precolored nodes, so pop nodes from stack coloring along the way , , have colors by coalescing must spill since no color can be found for it Introduce new temporaries and for each use/def, add loads before each use and stores after each def
210
Example (cont.)
211
Example (cont.)
r1
As before, coalesce
r3c1c2
with , then
with :
r2b
r1
ae
212
Example (cont.)
As before, coalesce
r3c1c2
r2b
r1ae
Pop from stack: select . All other nodes were coalesced or precolored. So, the coloring is:
213
Example (cont.)
214
Example (cont.)
One uncoalesced move remains
215
216
allows us to build large programs keeps compile times reasonable requires independent procedures
The linkage convention ensures that procedures inherit a valid run-time environment and that they restore one for their parents. Linkages execute at run time Code to make the linkage is generated at compile time
Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and full citation on the rst page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specic permission and/or fee. Request permission to publish from hosking@cs.purdue.edu. 217
on entry, establish s environment at a call, preserve s environment on exit, tear down s environment in between, addressability and proper lifetimes
procedure P prologue procedure Q prologue
epilogue
epilogue
Procedure linkages
higher addresses incoming arguments argument n . . . argument 2 argument 1 local variables previous frame
frame pointer
outgoing arguments
Assume that each procedure activation has an associated activation record or frame (at run time) Assumptions: RISC architecture can always expand an allocated block locals stored in frame
stack pointer
return address current frame temporaries saved registers argument m . . . argument 2 argument 1 next frame lower addresses
219
Procedure linkages
The linkage divides responsibility between caller and callee
Caller Call Callee
pre-call
1. 2. 3. 4. allocate basic frame evaluate & store params. store return address jump to child
prologue
1. 2. 3. 4. 5. save registers, state store FP (dynamic link) set new FP store static link extend basic frame (for local data) 6. initialize locals 7. fall through to code
Return
post-call
1. copy return value 2. deallocate basic frame 3. restore parameters (if copy out)
epilogue
1. 2. 3. 4. 5. store return value restore state cut back to basic frame restore parents FP jump to return address
At compile time, generate the code to do this At run time, that code manipulates the frame & data areas
220
Data space xed-sized data may be statically allocated variable-sized data must be dynamically allocated some data is dynamically allocated in code
Control stack dynamic slice of activation tree return addresses may be implemented in hardware
221
free memory
allows both stack and heap maximal freedom code and static data may be separate or intermingled
222
Upward exposure: can I return a reference to my variables? functions that return functions continuation-passing style
With only downward exposure, the compiler can allocate the frames on the run-time call stack
223
Storage classes
Each variable must be assigned a storage class Static variables: (base address )
addresses compiled into code (usually ) allocated at compile-time limited to xed size objects control access with naming scheme
(relocatable)
Global variables: almost identical to static variables layout may be important naming scheme ensures universal access (exposed )
224
if sizes are xed if lifetimes are limited if values are not preserved
call-by-reference, pointers, lead to non-local lifetimes (usually ) an explicit allocation explicit or implicit deallocation
225
Lexical nesting
view variables as (level,offset ) pairs chain of non-local access links more expensive to nd
(compile-time)
(at run-time)
226
How do we map a name into a (level,offset ) pair? Use a block-structured symbol table (remember last lecture?) look up a name, want its most recent declaration declaration may be at current level or any lower level
Given a (level,offset ) pair, whats the address? Two classic approaches access links displays (or static links )
227
need current procedure level, k k k k l local value l cannot occur l nd ls activation record
Maintaining access links: calling level k 1 procedure 1. pass my FP as access link 2. my backward chain will work for lower levels calling procedure at level l 1. nd link to level l 1 and pass it 2. its access link will work for lower levels k
(static links )
228
The display
To improve run-time access costs, use a display :
table of access links for lower levels lookup is index from known offset takes slight amount of time at call a single display or one per frame for level k procedure, need k 1 slots
nd slot as
l l o )
229
1. Call includes bitmap of callers registers to be saved/restored (best with save/restore instructions to interpret bitmap directly) 2. Caller saves and restores its own registers Unstructured returns (e.g., non-local gotos, exceptions) create some problems, since code to restore must be located and executed 3. Backpatch code to save registers used in callee on entry, restore on exit e.g., VAX places bitmap in callees stack frame for use on call/return/non-local goto/exception Non-local gotos and exceptions must unwind dynamic chain restoring callee-saved registers 4. Bitmap in callees stack frame is used by caller to save/restore (best with save/restore instructions to interpret bitmap directly) Unwind dynamic chain as for 3 5. Easy Non-local gotos and exceptions must restore all registers from outermost callee 6. Easy (use utility routine to keep calls compact) Non-local gotos and exceptions need only restore original registers from caller Top-left is best: saves fewer registers, compact calling sequences
230
Call/return
Assuming callee saves: 1. caller pushes space for return value 2. caller pushes SP 3. caller pushes space for: return address, static chain, saved registers 4. caller evaluates and pushes actuals onto stack 5. caller sets return address, callees static chain, performs call 6. callee saves registers in register-save area 7. callee copies by-value arrays/records using addresses passed as actuals 8. callee allocates dynamic arrays as needed 9. on return, callee restores saved registers 10. jumps to return address Caller must allocate much of stack frame, because it computes the actual parameters Alternative is to put actuals below callees stack frame in callers: common when hardware supports stack management (e.g., VAX)
231
at v0, v1 a0a3 t0t7 s0s7 t8, t9 k0, k1 gp sp s8 (fp) ra
Usage Constant 0 Reserved for assembler Expression evaluation, scalar function results rst 4 scalar arguments Temporaries, caller-saved; caller must save to preserve across calls Callee-saved; must be preserved across calls Temporaries, caller-saved; caller must save to preserve across calls Reserved for OS kernel Pointer to global area Stack pointer Callee-saved; must be preserved across calls Expression evaluation, pass return address in calls
232
non-leaf routines: routines that call other routines leaf routines: routines that do not themselves call other routines leaf routines that require stack storage for locals leaf routines that do not require stack storage for locals
233
low memory
234
235
local variables saved registers sufcient space for arguments to routines called by this routine
(b) Save registers (ra, etc.) e.g.,
where and (usually negative) are compiletime constants 2. Emit code for routine
236
3. Get return address
4. Clean up stack
5. Return
237