Experiment Name: Tokenization of a string using “strtok()” function.

Code:
#include <stdio.h> #include<string.h> 
int main()
{
    char str1[1000] = "Today is very cold in daffodil international university(Ashulia),khagan(sav
ar),etc";
    char *word;
    word = strtok(str1, " ,()");
    printf("%s\n", word);
    while (word != '\0')
    {
        word = strtok('\0', " ,:()");
        printf("%s\n", word);
    }
    return 0;
}

Output:
Discussion:
We know, Tokenization means representing all the meaningful parts of a code
individually. Each part then called a “Token”. Similarly, Tokenization of a string is a
process where a string is broken into several parts. In this experiment, we have
written a program that does the job. For this we have used strtok() function. We
need to know that strtok() function has two parameters. First parameter takes the
string that we need to tokenize and the second parameter is called “Delimiter”.
Delimiter is the element that separates one word from another. It can be a blank
space or any punctuation mark.
There can be more than one delimiter for a single string. Strtok() function mainly
returns the address of each word of a string. That’s why we have taken a string
type “pointer variable” in the program. Strtok() function works in a way where it
puts a null character in the place of the delimiter and separate each word of a
string. We have used a while loop for repeated steps of the code and finally got
our output successfully.