Lexical Analyzer

- A software which will read program as input stream and compare them with already
defined patterns and the generate tokens.

Sp ---> LA ---> Tokens

How to implement LA ?
By hand
1- from Transition diagram
By tool
3- Using Lex Tools
.......................................
Writing Lexical Analyzer
1- from Transition diagram

Adv- Staright Forward code

- simple switch logic
Dis
- Too much code
- diagram Depenedent code
Example :
Assignment 2
DeVelop Lexical Specification for xyz language
1- Identifiers - start with letter, followed by lettre or digits
2- Keywords int, print, get, if, while
3- int-literals - start with non zero digit
07 not a int literal
7 is intliteral
4- operators = and + < <=, *, -
5- Punctuations ; { }
6- delimiters (ignored)

Tokenize the following programs

Sample Program1
{int a23;
a=25
print a
}
Sample program 2
{int a;
int b;
int c;
get a
get b
c= a+b;
print c

}
................................
Source Transition from lexical Specification
(See on board)

int state=0;
file.open();
char ch;
char lexeme[50];
int i=0;
while ((ch=file.getChar())!=eof){
switch (state){
case 0 : if (isletter(ch))
state=1;
else if (isdigit(ch))
state=3;
else if (ch=='+')
state=5;
else if (ch=='{')
state=6;
else if (ch=='}')
state=7;
else if (ch==';')
state=8;
break;
case 1 : if (isdigit(ch)) || (islet(ch))
state=1;
else
state=2;
break;
case 2 : ungetch(ch);
state=0;
i=0;
if (isKeyword(lexeme)){
cout<<" keyword found";
cout<<lexeme;
}else
cout<<" identifier found";
cout<<lexeme;
cout<<"Token = id ";

}// end switch

lexeme[i]=ch;
i++;
} //end while