Compiler Design

Role of Lexical Analyzer

 Session Objectives
• To learn
– Role of Lexical Analyzer
– Specification of Tokens

2 v 1.2
 Session Outcomes
• At the end of this session, participants will be able to
– How to specify tokens

3 v 1.2
 Agenda
• Role of Lexical Analyzer and Specification of Tokens

4 v 1.2
 Specification of Tokens

Dr. B. Prabavathy
SSNCE

5 v 1.2
 Lexical Analyzer

6 v 1.2
 Lexeme, Pattern, Token
• Lexeme
– is a sequence of characters in the source program that
matches the pattern for a token
• Pattern
– is a description of the form that the lexemes of a token may
take
• Token
– is a pair of a token name and an optional token value

7 v 1.2
 Example

8 v 1.2
 Specification of token
• Regular expressions are important notation for patterns

9 v 1.2
 Regular Definition
• Regular expressions are given names
• These names will be used in another regular expression

10 v 1.2
 Transition diagram

11 v 1.2
 Transition diagram
• Transition diagram for identifier

• Transition diagram for unsigned number

12 v 1.2
 Simple Questions
1. How to specify token for integer constant
2. To which phase this token would be useful?
3. What is Regular Definition ?

13 v 1.2
 Summary
• Language Processing System
• Phases of the Compiler

14 v 1.2