Lexical Analysis

Lex Compiler

2
Lex Compiler
Lex program consists of three parts:-
%{
Declaration
%}

%%
Rule Section
%%

Auxiliary Procedure

3
Lex Compiler

●
Translation Rules are of the form:-
P1 {action1}
P2 {action2}
....
Pi {action3}
where each Pi is a regular expression and
each actioni is a program fragment describing what action the
lexical analyzer should take when pattern pi matches a lexeme

4
Lex Specification

Auxiliary Procedures

●
This section holds whatever auxiliary functions are used in
the actions
●
These functions can be compiled separately and loaded with
the lexical analyzer

5
Working of Lexical analyzer with parser

●
When called by the parser, the lexical analyzer begins reading its
remaining input, one character at a time, until it finds the longest
prefix of the input that matches one of the patterns
●

6
Working of Lexical analyzer with parser

●
When called by the parser, the lexical analyzer begins reading its
remaining input, one character at a time, until it finds the longest
prefix of the input that matches one of the patterns
● It then executes actioni which returns the control to the parser
●

7
Working of Lexical analyzer with parser

●
When called by the parser, the lexical analyzer begins reading its
remaining input, one character at a time, until it finds the longest
prefix of the input that matches one of the patterns
● It then executes actioni which returns the control to the parser
●
But if it does not then the lexical analyzer proceeds to find
additional lexemes, until one of the corresponding actions causes
a return to the parser.
●

8
Working of Lexical analyzer with parser

●
When called by the parser, the lexical analyzer begins reading its
remaining input, one character at a time, until it finds the longest
prefix of the input that matches one of the patterns
● It then executes actioni which returns the control to the parser
●
But if it does not then the lexical analyzer proceeds to find
additional lexemes, until one of the corresponding actions causes
a return to the parser.
●
The lexical analyzer returns a single quantity i.e the token to the
parser
●

9
Working of Lexical analyzer with parser

●
When called by the parser, the lexical analyzer begins reading its
remaining input, one character at a time, until it finds the longest
prefix of the input that matches one of the patterns
● It then executes actioni which returns the control to the parser
●
But if it does not then the lexical analyzer proceeds to find
additional lexemes, until one of the corresponding actions causes
a return to the parser.
●
The lexical analyzer returns a single quantity i.e the token to the
parser
●
To pass an attribute value with information about the lexeme, a
global variable is set known as yylval

10
Lex Program for tokens
%{
/* Definitions of manifest constants
LT, LE, EQ, NE, GT, GE, IF, THEN, ELSE, ID, NUMBER, RELOP
*/
%}

/* Regular definitions */

delim [\t \n]

ws {delim}+
letter [A-Za-z]
digit [0-9]
id {letter}( {letter} | {digit} )* 11
number {digit}+(\ . {digit} +) ? (E [+-] ? {digit}+) ?
Lex Program for tokens
%%

{ws} { /* No action and No Return */ }

if { return (IF); }
else { return (ELSE); }
then { return (THEN); }
{id} { yylval = (int) installID(); return (ID); }
{number} { yylval = (int) installNum(); return (NUMBER); }
“<” { yylval = LT; return(RELOP); }
“<=” { yylval = LE; return(RELOP); }
“=” { yylval = EQ; return(RELOP); }
“<>” { yylval = NE; return(RELOP); }
“>” { yylval = GT; return(RELOP); }
“>=” { yylval = GE; return(RELOP); }

12
%%
Lex Program for tokens
int installID()
{
/* Function to install the lexeme whose first character is pointed to by
yytext, and whose length is yyleng, into the symbol table and return
a pointer therto
*/
}

int installNum()
{
/* Similar to installID but puts numerical constants into a separate table */
}

13
Design of Lexical analyzer
generator
●
Aim: Design of a software tool that automatically constructs a
lexical analyzer from a program in the Lex language

14
Lex Program Example 1
%{

%}

%%
\n { printf(“\n Hello Good Morning”);
%%

void main()
{
yylex();

} 15
Lex Program Example 2
%{
char name[10];
%}
%%
[\n] {printf("\n Hi........%s......Good Morning\n",name); return 1;}
%%

void main()
{
char opt;
do
{
printf("\nWhat is your name?");
scanf("%s",name);
yylex();
printf("\nPress y to continue");
scanf("%c",&opt);
}
while(opt=='y');
} 16
Lex Program Example 3
%{
int linecount = 0, charcount = 0;
%}

%%
. {charcount++;}
\n {linecount++; charcount++;}
%%

void main()
{
yylex();
printf("# of lines = %d, # of chars = %d", linecount, charcount);

} 17
Lex Program Example 4
%{ void display(int flag,char *t)
void display(int, char *);
{
int flag;
if(flag==1)
%}
{
%% printf("\nThe given character %s is vowel\n",t);
[a|e|i|o|u] {flag =1; display(flag,yytext);}
}
. {flag =0; display(flag,yytext);}
%% else

printf("\nThe given character %s is not vowel\n",t);

void main()
}
{
printf("\nEnter the word:"); }
yylex();
}
18
Data Structure used in Lexical Analyzer

●
Symbol: Identifier used in the Source Program
– Examples: Names of variables, functions and Procedures
●

19
Data Structure used in Lexical
Analyzer
●
Symbol: Identifier used in the Source Program
– Examples: Names of variables, functions and Procedures
●
Symbol Table: To maintain information about attributes of symbol
●

20
Data Structure used in Lexical
Analyzer
●
Symbol: Identifier used in the Source Program
– Examples: Names of variables, functions and Procedures
●
Symbol Table: To maintain information about attributes of symbol
●
Operations on Symbol Table:
i. Add a symbol and its attributes
ii. Locate a symbol’s entry
iii. Delete a symbol’s entry
iv. Access a symbol’s entry

21
Data Structure used in Lexical
Analyzer
●
Design Goal of Symbol Table
i. The Table’s organization should facilitate efficient search
ii. Table should be compact (Less Memory)
●
Time Space Trade off
●
To improve search efficiency allocate more memory to symbol table
●
Organization of Entries:
i. Linear Data Structure
ii. Non Linear Data Structure

22
Data Structure used in Lexical
Analyzer
●
Symbol Table Entry Format
– Number of Fields to accommodate attributes of one symbol
– Symbol Field: The symbol to be stored
– Key Field: Basis for the search in table
– Entry of Symbol is called record
●
Entry Format
– Fixed Length
– Variable Length
– Hybrid

23
Data Structure used in Lexical
Analyzer

Symbol Class Attributes

Type, Length, number and bounds of

Variable
dimensions
Number of parameters, address of
Procedure
parameter list
Number of parameters, address of
Function parameter list, type and length of
returned value

Label Statement Number

24