COMPILER CONSTRUCTION

Assignment by GROUP 9

NOVEMBER 28, 2019

SUBMITTED TO SIR AMMAR
 Introduction

Name:
Aitdaal Anwar
Roll no:
2201
Semester:
7th (Morning)
Topic:
Patterns & Lexical Errors

[1]
GOVT. Postgraduate College of Science
 Patterns

Patterns:
There is a group of strings in
the input for which same token can be
produced
This group is described by a rule called
pattern

Lexemes:
A lexeme is a sequence of
characters in the source program that is matched
by the pattern for a token.
For example:
const float pi=3.1416;
pi is lexeme for the token “identifier”.

[2]
GOVT. Postgraduate College of Science
 Lexical Errors

Lexical Errors:
Only few errors can be
detected at lexical analysis phase. For e.g.,
Suppose a string encountered:
fi (a==5)
Lexical analyzer cannot tell that “fi” is a misspelled
keyword or undeclared identifier.
Lexical analyzer must return the token for identifier.
This happens where variable declaration is not
mandatory.
Sometimes lexical analyzer is unable to proceed then
the Simplest technique is “panic mode” recovery. It
suggests the solution for the error.

[3]
GOVT. Postgraduate College of Science
 Introduction

Name:
Zain Wajid
Roll no:
2220
Semester:
7th (Morning)
Topic:
Lexical Analyzer Working

[4]
GOVT. Postgraduate College of Science
 How it Works

The working of a Front-End Compiler is shown by the

diagram:

[6]
GOVT. Postgraduate College of Science
 How it Works

First Step: Lexical Analysis

(Source code)

(Token Stream)
‘if’ and other keywords are
are written in separate blocks

Ad-hoc Lexer:
It is a Handwritten code to generate token.
It partitions the input string by reading left to right and
recognize one token at time.A “Lookahead” variable is
require to decide where one token end and next token
start.

[6]
GOVT. Postgraduate College of Science
 Introduction

Name:
Muhammad Maaz
Roll no:
2240
Semester:
7th (Morning)
Topic:
Ad-Hoc Lexer Program

[7]
GOVT. Postgraduate College of Science
 Ad-Hoc Lexer Program

Ad-hoc Lexer:
class Lexer
{
Inputstream s;
char next; //look ahead
Lexer(Inputstream _s){
s=_s;
next=s.read( );
}
Token nextToken(){
if(idChar(next))
return readId();
if(number(next))
return readNumber();
if(next==‘ ” ’)
return readString();
…..
……
[9]
GOVT. Postgraduate College of Science
 Ad-Hoc Lexer Program
Token readId( ){
string id=“ “;
while(true){
char c=input.read( );
if(idChar ( c ) ==false)
return
new Token{TID,id};
id=id + string( c );
}
}
boolean idChar(char c){
if( isAlpha ( c) )
return true;
if( isDigit( c ) )
return true;
if( c== ‘ _ ’)
return true;
else
return false; }
[9]
GOVT. Postgraduate College of Science
 Introduction

Name:
Muhammad Raza Ijaz
Roll no:
2232
Semester:
7th (Morning)
Topic:
Ad-Hoc Lexer Problems

[10]
GOVT. Postgraduate College of Science
 Ad-Hoc Lexer Problems

Problems:
1. We do not know what kind of token we are going to read
from seeing the first character

2. If a token begins with “=“, is it “=“ or “==“ ?

 How to start?
 What to do with following character?
 How to avoid quadratic complexity of repeated concatenation?
 How to recognize keywords?

3. We might not know what kind of token we are going to read

from seeing first character
 if a token begins with “i’’ is it an identifier? (what about int,
if )
 if token begins with “2” is it an integer constant?

4. Interleaved tokenizer code are hard to write correctly, harder

to maintain.

5. In general unbounded look-ahead may be needed.

6. How to specify (unambiguously) tokens.

 Once specified, how to implement them in a systematic
way?
 How to implement them efficiently?

[11]
GOVT. Postgraduate College of Science
 Ad-Hoc Lexer Problems

7.For instance, consider.

 How to describe tokens unambiguously
 2.e0 20.e-01 2.0000
 “” “x” “\\” “\”\’”
 How to break up text into tokens
 if (x == 0) a = x<<1;
 if (x == 0) a = x<1;
 How to tokenize efficiently
 want to look at each character ~1 time
8.Need a more principled approach
9.Use lexer generator that generate efficient tokenizer
automatically

The MiniJava compiler:

[12]
GOVT. Postgraduate College of Science