Scribd
Upload
77 views5 pages

Count Tokens in C Using Lex Program

Lex is a program that generates a lexical analyzer in C from a program written in the Lex language, which reads an input stream and produces tokens as output; the document provides an example Lex program that counts the total number of tokens in a C program by defining rules to identify keywords, identifiers, operators, separators, integers and floats; it also provides instructions for installing Lex and related tools, and compiling and running Lex programs on Windows and Linux systems.

Uploaded by

Apurva Ankushrao
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
77 views5 pages

Count Tokens in C Using Lex Program

Lex is a program that generates a lexical analyzer in C from a program written in the Lex language, which reads an input stream and produces tokens as output; the document provides an example Lex program that counts the total number of tokens in a C program by defining rules to identify keywords, identifiers, operators, separators, integers and floats; it also provides instructions for installing Lex and related tools, and compiling and running Lex programs on Windows and Linux systems.

Uploaded by

Apurva Ankushrao
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Experiment No.

Aim: Write a lex program to count total number of tokens in C.


Theory:

LEX
o Lex is a program that generates lexical analyzer. It is used with YACC parser
generator.
o The lexical analyzer is a program that transforms an input stream into a sequence of
tokens.
o It reads the input stream and produces the source code as output through
implementing the lexical analyzer in the C program.

The function of Lex is as follows:


o Firstly lexical analyzer creates a program lex.l in the Lex language. Then Lex
compiler runs the lex.l program and produces a C program lex.yy.c.

o Finally C compiler runs the lex.yy.c program and produces an object program a.out.

o a.out is lexical analyzer that transforms an input stream into a sequence of tokens.

o
o Tokens: A token is a group of characters forming a basic atomic chunk of
syntax i.e. token is a class of lexemes that matches a pattern.
o Eg – Keywords, identifier, operator, separator.
o
/*Lex code to count total number of tokens */

  %{

int n = 0 ;

%}

// rule section

%%

//count number of keywords

"while"|"if"|"else" {n++;printf("\t keywords : %s", yytext);}

// count number of keywords

"int"|"float" {n++;printf("\t keywords : %s", yytext);}

// count number of identifiers

[a-zA-Z_][a-zA-Z0-9_]* {n++;printf("\t identifier : %s", yytext);}

// count number of operators

"<="|"=="|"="|"++"|"-"|"*"|"+" {n++;printf("\t operator : %s", yytext);}

// count number of separators

[(){}|, ;]    {n++;printf("\t separator : %s", yytext);}

// count number of floats

[0-9]*"."[0-9]+ {n++;printf("\t float : %s", yytext);}

// count number of integers

[0-9]+ {n++;printf("\t integer : %s", yytext);}

.    ;
%%

int main()

    yylex();

    printf("\n total no. of token = %d\n", n);

 }

Output:
Commands to run lex program in windows:
1. Click on Execute CMD directly button in the IDE.
2. Compile the Lex File by typing the command
lex <filename>. l.
3. Build the Lex File by gcc/cc command in the CMD
 e.g gcc lex.yy.c -o <executable name for program>
4. Execute the program by typing <executable name for the program>.exe.
5.
Commands to run lex program in linux:

 write the lex program in a file and save it as file. l (where file is the name of the file).
 open the terminal and navigate to the directory where you have saved the file.l.
 type - lex file.l.
 then type - cc lex.yy.c -ll
 then type - ./a.out

Installing Softwares: (http://gnuwin32.sourceforge.net/packages.html


https://sourceforge.net/projects/tdm-gcc/files/Installer%20Supplement/gdb-5.2.1/)
1. Download Flex 2.5.4a
2. Download Bison 2.4.1
3. Download DevC++
4. Install Flex at "C:\GnuWin32"
5. Install Bison at "C:\GnuWin32"
6. Install DevC++ at "C:\Dev-Cpp"
7. Open Environment Variables.
8. Add "C:\GnuWin32\bin;C:\Dev-Cpp\bin;" to path.

Compilation & Execution of your Program:


1. Open Command prompt and switch to your working directory
where you have stored your lex file (".l") and yacc file (".y")
2. Let your lex and yacc files be "hello.l" and
"hello.y". Now, follow the preceding steps to compile and
run your program.
1. For Compiling Lex file only:
1. flex hello.l
2. gcc lex.yy.c
2. For Compiling Lex & Yacc file both:
1. flex hello.l
2. bison -dy hello.y
3. gcc lex.yy.c y.tab.c
3. For Executing the Program
1. a.exe

You might also like