Best Product Glean using Beautiful Soap Module

An Application Development Report Submitted

In partial fulfillment of the requirement for the award of the degree of

Bachelor of Technology
In
Computer Science and Engineering -Artificial Intelligence
and Machine Learning
by

Eediga Akash Goud - 21N31A6646

ChittiBomma Revanth Srinivas -21N31A6640

BhupathiRaju Adithya Hima Varma – 21N31A6623

Under the Guidance of

Dr.G.L.N. Jayaprada
Assoc. Professor
Computational Intelligence Department
MRCET

DEPARTMENT OF COMPUTATIONAL INTELLIGENCE

MALLA REDDY COLLEGE OF ENGINEERING AND TECHNOLOGY
(Affiliated to JNTU, Hyderabad)ACCREDITED by AICTE-NBAMaisammaguda,
Dhulapally post, Secunderabad-500014.
2021-2025

1
 DECLARATION

I hereby declare that the project entitled “ Best Product Glean” submitted to Malla Reddy
College of Engineering and Technology, affiliated to Jawaharlal Nehru Technological
University Hyderabad (JNTUH) for the award of the degree of Bachelor of Technology in
Computer Science and Engineering- Artificial Intelligence and Machine Learning is a result
of original research work done by me.
It is further declared that the project report or any part thereof has not been previously
submitted to any University or Institute for the award of degree or diploma.

Eediga Akash Goud(21N31A6646)

ChittiBomma Revanth Srinivas (21N31A6640)
BhupatiRaju Aditya Hima Varma(21N31A6623)

2
 CERTIFICATE

This is to certify that this is the bonafide record of the project titled “Best Product Glean”
submitted by Eediga Akash Goud(21N31A6646), Chitti Bomma Revanth Srinivas
(21N31A6640), BhupatiRaju Adithya Hima Varma(21N31A6623) of B.Tech in the partial
fulfillment of the requirements for the degree of Bachelor of Technology in Computer Science
and Engineering- Artificial Intelligence and Machine Learning, Dept. of CI during the year
2023-2024. The results embodied in this project report have not been submitted to any other
university or institute for the award of any degree or diploma.

Dr. G.L.N Jayaprada Dr. D. Sujatha

Assoc. Professor Professor
INTERNAL GUIDE HEAD OF THE DEPARTMENT

EXTERNAL EXAMINER

3
 ACKNOWLEDGEMENT

We feel honored and privileged to place our warm salutation to our college Malla Reddy
College of Engineering and technology (UGC-Autonomous), our Director Dr. VSK
Reddy who gave us the opportunity to have experience in engineering and profound
technical knowledge.

We are indebted to our Principal Dr. S. Srinivasa Rao for providing us with facilities to
do our project and his constant encouragement and moral support which motivated us to
move forward with the project.

We would like to express our gratitude to our Head of the Department Dr. D. Sujatha
for encouraging us in every aspect of our system development and helping us realize our
full potential.

We would like to thank our application development guide as well as our internal guide

Dr. G.L.N Jayaprada(Assoc. Professor), for her structured guidance and never-
ending encouragement. We are extremely grateful for valuable suggestions and
unflinching co-operation throughout application development work.

We would also like to thank all supporting staff of department of CI and all other
departments who have been helpful directly or indirectly in making our application
development a success.

We would like to thank our parents and friends who have helped us with their valuable
suggestions and support has been very helpful in various phases of the completion of the
application development.

By

Eediga Akash Goud(21N31A6646)

ChittiBomma Revanth Srinivas (21N31A6640)
BhupatiRaju Adity Hima Varma(21N31A6623)

4
 ABSTRACT

This project introduces a user-friendly online ecommerce price comparison tool

implemented in Python, specifically designed for the terminal environment using the
BeautifulSoup module. The system enables users to effortlessly compare prices of
desired products across major ecommerce platforms such as Amazon and Flipkart. By
employing web scraping techniques, the tool extracts relevant pricing details without the
need for complex interactions or graphical interfaces. Users input their desired product
in the terminal, and the system retrieves and displays prices from various sources in a
clear and concise format. This implementation simplifies the process of making
informed purchasing decisions directly from the terminal, offering a convenient and
accessible solution for users who prefer command-line interfaces.

5
 TABLE OF CONTENTS

S.No. Topic Page No.

CHAPTER 1: INTRODUCTION 1-3

1.1: PURPOSE 1

1.2: BACKGROUND OF PROJECT 1

1.3: SCOPE OF PROJECT 2

1.4: PROJECT FEATURES 2

CHAPTER 2: SYSTEM REQUIREMENTS 3-5

2.1 HARDWARE REQURIEMENTS 3

2.2 SOFTWARE REQUIREMENTS 4

2.3 EXISTING SYSTEM 4

2.4 PROPOSED SYSTEM 5

CHAPTER 3: SYSTEM DESIGN 6-7

3.1: SYSTEM ARCHITECTURE 6

3.2: UML DIAGRAMS 7-8

CHAPTER 4: : IMPLEMENTATION 9-12

4. 4. SOURCE CODE 9-11

4.2 4.2 OUTPUT SCREENS 12

CHAPTER 5: CONCLUSION 13

BIBLIOGRAPHY 13

6
1. INTRODUCTION

1.1 Purpose:

The main purpose of this document is to present the requirements of the project “Best Product
Glean”. This study's goal is to evaluate the effectiveness of making informed purchasing
decisions. This tool prioritizes simplicity and accessibility, allowing users to input desired
product details directly into the terminal. By employing web scraping techniques, it retrieves
and presents product prices from popular ecommerce platforms like Amazon and Flipkart
without requiring user logins, ensuring privacy. The system streamlines the process of
comparing prices, saving users time and effort by eliminating the need to visit multiple
websites individually. Additionally, the tool's design facilitates scriptability, enabling
integration into automated workflows or scripts for users with specific needs. Overall, this
project aims to enhance the shopping experience by providing a user-friendly and customizable
solution for ecommerce price comparison in a terminal environment.

1.2 Background of project:

The project stems from the common frustration of consumers facing the tedious task of
manually comparing prices across different ecommerce websites. Recognizing the need for a
more efficient solution, this project utilizes the BeautifulSoup module in Python to create a tool
that streamlines the process of price comparison. By focusing on a terminal-based interface, the
project aims to provide a user-friendly and accessible experience, allowing individuals to input
product details directly into the terminal for quick and hassle-free comparisons.

The decision to employ web scraping techniques using BeautifulSoup is driven by the tool's
ability to effectively parse HTML content, extracting pertinent pricing information from
popular ecommerce platforms like Amazon and Flipkart. Moreover, the project prioritizes user

7
 privacy by eliminating the necessity for logins, ensuring that individuals can compare prices
without compromising personal information. The platform-independent nature of the tool,
designed for use within the terminal, also enhances its versatility, making it scriptable for users
who prefer automation in their workflows. Overall, the project seeks to simplify the price
comparison process, making it more efficient and accessible for users.

1.3 Scope of project:

The scope of the project encompasses providing users with a robust and versatile tool for
ecommerce price comparison, leveraging the BeautifulSoup module in Python within a terminal
environment. The primary goal is to streamline the process of comparing product prices across
major ecommerce platforms like Amazon and Flipkart, enhancing user convenience. The tool
aims to be platform-independent, allowing users to access it from any operating system, and
scriptable, enabling integration into automated workflows. The project also emphasizes user
privacy by eliminating the need for logins during the price comparison process. Overall, the
scope is to create an accessible, efficient, and privacy-conscious solution for users seeking a
seamless method of comparing product prices through a simple terminal interface.

1.4 Project Features:

The system features are as follows:
 The Best Product Glean tool, developed using the BeautifulSoup module in Python
within a terminal environment, offers a range of features:

 Terminal Interface: Users can input product details directly into the terminal, providing
a straightforward and accessible means of initiating price comparisons.

 Web Scraping Techniques: The tool employs web scraping to extract pricing
information from popular ecommerce platforms like Amazon and Flipkart, ensuring
comprehensive coverage and accuracy.

8
 Platform Independence: Designed to work across various operating systems, the tool is
platform-independent, allowing users to access it from different environments.

 Privacy Conscious: The project prioritizes user privacy by eliminating the need for logins
during the price comparison process, ensuring a secure and confidential user experience.

 Scriptable Design: The tool's scriptable nature allows users to integrate it into automated
workflows, enhancing adaptability for users with diverse needs.

 User-Friendly Interface: With a focus on ease of use, the tool features a user-friendly
interface for a seamless and intuitive price comparison experience in the terminal.

2.SYSTEM REQUIREMENTS

2.1 Hardware Requirements:

The hardware interfaces of this project consist of architecture, processing power, memory,
secondary storage, display adapter, peripherals like CD-ROM drivers, keyboards, pointing
devices, network devices, etc.
 Processor: i5 and above
 Ram: 4gb and above
 Hard Disk: 25 GB in local drive

9
2.2 Software requirements:

The software interface core programming language is Python, with a recommended version of
3.x. Key libraries include BeautifulSoup for parsing HTML and XML, installed via pip install
beautifulsoup4, and Requests for making HTTP requests, installed with pip install requests
 Operating System: Windows Only,macOs,Linux

2.3 Existing System:

The existing system for the Best Product Glean project using the BeautifulSoup module in
Python within a terminal environment primarily involves manual online shopping and price
comparison. In the absence of the developed tool, users typically navigate through multiple e-
commerce websites individually, searching for specific products and manually comparing prices.
This process is time-consuming, tedious, and may lead to inefficiencies due to the lack of a
centralized mechanism for real-time price updates. The existing system lacks automation and the
ability to swiftly aggregate and analyze product prices from various sources. The introduction of
the BeautifulSoup-powered tool aims to address these limitations by automating the price
comparison process, providing users with a more efficient and streamlined experience in
obtaining up-to-date information from different online retailers.

2.3.1 Drawbacks of existing system:

1)Manual Labor Intensive:

The existing system relies on manual efforts for price comparison, requiring users to visit each e-
commerce website individually. This manual process is time-consuming and inefficient.

10
2)Limited Automation:
Lack of automation means users miss out on real-time updates and struggle to keep track of
changing prices. This limitation reduces the effectiveness of the comparison process.

3)Potential Data Inconsistencies:

Due to the manual nature of the price collection, there is a higher risk of data inconsistencies and
errors. Users may encounter outdated or inaccurate information, leading to suboptimal decision-
making.

2.4 Proposed System:

The proposed system for the best product glean project in Python brings several enhancements
to the existing approach:

The proposed system automates the process of fetching and comparing prices from different e-
commerce websites using BeautifulSoup. This ensures real-time updates, allowing users to
access the most recent product pricing information without manual efforts.

The system introduces a user-friendly interface in the terminal, providing a seamless and
intuitive experience for users. This interface simplifies the interaction with the program, making
it easy for users to input product names and receive comprehensive price comparison results.

Leveraging the parsing capabilities of BeautifulSoup, the proposed system improves data
accuracy and consistency. It reduces the chances of errors in price extraction, ensuring that users
receive reliable and precise information for effective decision-making in their online shopping
endeavors.

11
3. SYSTEM DESIGN

3.1 System Architecture

Fig 3.1

12
3.3 UML Diagrams

3.3.1 Use case diagram

The use case diagram for Best Product Glean illustrates how users interact with the system to
search, view product details, and compare prices. It showcases the key functionalities,
emphasizing user actions and system responses.

13
Fig 3.3 (a)

14
3.3.2 Activity Diagram
The sequence diagram for best product glean depicts the step-by-step interactions between the
user, the system, and the web scraper, showcasing the flow of actions in searching, extracting,
and comparing product prices from different online sources.

Fig 3.3 (b)

15
4.IMPLEMENTATION

4.1: Code

from bs4 import BeautifulSoup

import requests
headers = {'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36'}
flipkart=''
amazon=''
def flipkart(name):
try:
global flipkart
name1 = name.replace(" ","+")
flipkart=f'https://www.flipkart.com/search?
q={name1}&otracker=search&otracker1=search&marketplace=FLIPKART&as-
show=off&as=off'
res = requests.get(f'https://www.flipkart.com/search?
q={name1}&otracker=search&otracker1=search&marketplace=FLIPKART&as-
show=off&as=off',headers=headers)

print("\nSearching in flipkart....")
soup = BeautifulSoup(res.text,'html.parser')

if(soup.select('._4rR01T')):
flipkart_name = soup.select('._4rR01T')[0].getText().strip().upper()
if name.upper() in flipkart_name:
flipkart_price = soup.select('._30jeq3')[0].getText().strip()
flipkart_name = soup.select('._4rR01T')[0].getText().strip()
print("Flipkart:")
print(flipkart_name)
print(flipkart_price)
print("---------------------------------")

elif(soup.select('.s1Q9rs')):
flipkart_name = soup.select('.s1Q9rs')[0].getText().strip()
flipkart_name = flipkart_name.upper()
if name.upper() in flipkart_name:
flipkart_price = soup.select('._30jeq3')[0].getText().strip()

16
flipkart_name = soup.select('.s1Q9rs')[0].getText().strip()
print("Flipkart:")
print(flipkart_name)
print(flipkart_price)
print("---------------------------------")
else:
flipkart_price='0'

return flipkart_price
except:
print("Flipkart: No product found!")
print("---------------------------------")
flipkart_price= '0'
return flipkart_price

def amazon(name):
try:
global amazon
name1 = name.replace(" ","-")
name2 = name.replace(" ","+")
amazon=f'https://www.amazon.in/{name1}/s?k={name2}'
res = requests.get(f'https://www.amazon.in/{name1}/s?k={name2}',headers=headers)
print("\nSearching in amazon...")
soup = BeautifulSoup(res.text,'html.parser')
amazon_page = soup.select('.a-color-base.a-text-normal')
amazon_page_length = int(len(amazon_page))
for i in range(0,amazon_page_length):
name = name.upper()
amazon_name = soup.select('.a-color-base.a-text-normal')[i].getText().strip().upper()
if name in amazon_name:
amazon_name = soup.select('.a-color-base.a-text-normal')[i].getText().strip()
amazon_price = soup.select('.a-price-whole')[i].getText().strip().upper()
print("Amazon:")
print(amazon_name)
print("₹"+amazon_price)
print("---------------------------------")
break
else:
i+=1
i=int(i)
if i==amazon_page_length:
amazon_price = '0'
print("amazon : No product found!")

17
print("-----------------------------")
break

return amazon_price
except:
print("Amazon: No product found!")
print("---------------------------------")
amazon_price = '0'
return amazon_price

def convert(a):
b=a.replace(" ",'')
c=b.replace("INR",'')
d=c.replace(",",'')
f=d.replace("₹",'')
g=int(float(f))
return g

name=input("Product Name:\n")
flipkart_price=flipkart(name)
amazon_price=amazon(name)

if flipkart_price=='0':
print("Flipkart: No product found!")
flipkart_price = int(flipkart_price)
else:
print("\nFlipkart Price:",flipkart_price)
flipkart_price=convert(flipkart_price)
if amazon_price=='0':
print("Amazon: No product found!")
amazon_price = int(amazon_price)
else:
print("\nAmazon price: ₹",amazon_price)
amazon_price=convert(amazon_price)

18
4.2: Output Screens:

19
5. CONCLUSION & FUTURE SCOPE

In conclusion, the Best Product Glean project employing the BeautifulSoup module in Python
has proven to be a practical and efficient solution for comparing prices across different online
platforms. The implementation successfully showcased the capabilities of web scraping in
extracting relevant product details, such as names and prices, from popular eCommerce websites
like Flipkart and Amazon. The ability to obtain real-time pricing information enables users to
make informed purchasing decisions, saving both time and money.

While the current project provides a solid foundation for eCommerce price comparison, there is
ample room for future enhancements and expansions. One avenue for improvement involves
incorporating more eCommerce platforms, diversifying the range of products, and refining the
search algorithm for better accuracy. Additionally, the project could explore incorporating
machine learning techniques to analyze pricing trends, predict potential discounts, and offer
users personalized recommendations. Implementing a user interface for a more user-friendly
experience and extending the project to a web application would further enhance accessibility.
Continuous updates to adapt to changes in the target websites and exploring mobile application
development are also potential future directions for the project. Overall, the project has
promising prospects for evolution and adaptation to meet the dynamic needs of online shoppers.

7. BIBILOGRAPHY

Rossum, G. van. (2001). Python Programming Language. Python Software Foundation.

https://www.python.org/

Beautiful Soup Documentation. (n.d.). Retrieved from

https://www.crummy.com/software/BeautifulSoup/bs4/doc/

Hunter, J. D. (2007). Matplotlib: A 2D Graphics Environment. Computing in Science &

Engineering, 9(3), 90–95. https://matplotlib.org/

Requests: HTTP for Humans. (n.d.). Retrieved from https://docs.python-requests.org/en/master/

20