SHRI RAMSWAROOP MEMORIAL UNIVERSITY

Topic : PAN Card Tampering Detection

Submitted To: Submitted By:

Mr. Neelesh Mishra
Soumya Rai
202110101110714
Jayendra Pratap Singh
PAN CARD TAMPERING DETECTION
• The purpose of this project is to detect tampering of PAN card using computer vision. This project
will help different organization in detecting whether the Id i.e. the PAN card provided to them by
their employees or customers or anyone is original or not.

• For this project we will calculate structural similarity of original PAN card and the PAN card
uploaded by user.
 STEPS
• Loading original and user provided images.

• Converting the format of tampered image similar to original image.

• Here, we checked the format and size of the original and tampered image.

• Converting the size of tampered and original image.

• User provided image which will be compared with PAN card.
ORIGINAL PAN CARD IMAGE USED
FOR COMPARISON.
APPLYING STRUCTURAL SIMILARITY INDEX (SSIM)
TECHNIQUE BETWEEN THE TWO IMAGES
SSIM
The Structural Similarity Index (SSIM) is a perceptual metric that quantifies the image quality
degradation that is caused by processing such as data compression or by losses in data transmission.

How SSIM helps in detection ?

SSIM actually measures the perceptual difference between two similar images. It cannot judge
which of the two is better: that must be inferred from knowing which is the original one and which has
been exposed to additional processing such as compression or filters.
● Structural similarity index helps us to determine exactly where in terms of x,y coordinates location,
the image differences are. Here, we are trying to find similarities between the original and tampered
image.
● The lower the SSIM score lower is the similarity, i.e SSIM score is directly propational to
similarity between two images
● We have given one threshold value of "45" i.e if any score is >= 80 it will be regarded as original
pan card else tampered one.
● Generally SSIM values 0.97, 0.98, 0.99 for good quality reconstruction techniques.
FLOW CHART
CREATING BOUNDING BOXES (CONTOURS)
Bounding rectangle helps in finding the ratio of
width to height of bounding rectangle of the
object. We compute the bounding box of the
contour and then draw the bounding box on both
input images to represent where the two images
are different or not.

Tampered Image
NEED OF EDGE DETECTION
Edge detection is an image processing
technique for finding the boundaries of objects
within images. It works by detecting
discontinuities in brightness. Edge detection is
used for image segmentation and data
extraction in areas such as image processing,
computer vision, and machine vision.
WE CONVERT THEM INTO
GRAYSCALE
Converting images into grayscale is very much beneficial in accuracy of image processing, because in
image processing many applications doesn't help us in identifying the important, edges of the coloured
images also coloured images are bit complex to understand by machine because they have 3 channel while
grayscale has only 1 channel.
INFERENCE :
Here is another very interactive way to show the
contours in terms of heated threshold i.e. by finding
the heated zone (text/image zone) and normal
zone(without text/image).

The heated zone i.e the zone which has text/images

will be shown in dark (black) region, and the other
one as light (kind of white) zone.
Different Image
INFERENCE :
Everything here is just same all we can see is
the change in role of color, here white color
is showing the heated zone and black color is
showing the normal zone.

Threshold Image
 SUMMARY
Finding out structural similarity of the images helped us in finding the difference or similarity in the
shape of the images. Similarly, finding out the threshold and contours based on those threshold for the
images converted into grayscale binary also helped us in shape analysis and recognition.

As, our SSIM is ~31.2% we can say that the image user provided is fake or tampered.

Finally we visualized the differences and similarities between the images using by displaying the
images with contours, difference and threshold.
CONCLUSION
This project can be used in different organizations where customers or users need to provide any kind
of id in order to get themselves verified. The organization can use this project to find out whether the ID
is original or fake. Similarly this can be used for any type of ID like Aadhar, voter id, etc.
FUTURE SCOPE

We going to enhance the detection of digital signature and digital watermarking further in our project
and also linked it with government database which will make it more reliable because of more training
and testing data.
REFERENCE

• https://www.w3schools.com/

• https://www.javatpoint.com/

• https://www.geeksforgeeks.org/python-programming-language/

• https://www.programiz.com/python-programming/examples

• https://github.com/

• https://www.youtube.com/