Token:

identifiers
constants (literals)
keywords (reserved words)
operators, separators
operator symbols arithmetic, relational,
punctuation symbols
----------------------------
- comments
white spaces

Lexical analysis (scanning)

10/15/2017 1
Lexical analysis
Data: text file
Results: program internal form (PIF)
symbol table ST
or: error message

Lexical analysis
10/15/2017 2
 detect

For each token in the source file

classify atom
codify atom
endForEach

Lexical analysis (version 1)

10/15/2017 3
While there are unprocessed characters in the source file do
detect token
If token is keyword or operator or delimiter then
addPIF (cod(atom), 0)
Else
If token is identifier then
ind:=poz(atom,TS)
addPIF ((cod_id, ind)
Else If token is constant then
ind:=poz(atom,TS)
addPIF ((cod_const, indice)
else
Error message
EndIf
EndIf
EndIf
EndWhile
Lexical analysis (version 2)
10/15/2017 4
Remark.: many searches in ST
ST:
++ and in all phases !!
comp
sequential search x3
organize ST for a quick search forma
ordered (lexicographic ordering) b
an
lexicographic ordered table
medie
balanced search tree
first
hash table
search hash function

.
10/15/2017 5