2020 IEEE International Conference on Big Data (Big Data)
Item-Based Collaborative Filtering and Association
Rules for a Baseline Recommender in E-Commerce
Jessica Lourenco and Aparna S. Varde
Department of Computer Science
Montclair State University
Montclair, NJ, USA
2020 IEEE International Conference on Big Data (Big Data) | 978-1-7281-6251-5/20/$31.00 ©2020 IEEE | DOI: 10.1109/BigData50022.2020.9377807
(lourencoj1 | vardea)@montclair.edu
Abstract—In the ever-growing data-driven world today, data
increases in many forms, e.g. e-commerce sites uploading new
products, streaming services adding TV shows and movies, and
music platforms uploading new songs. It would be highly infeasible
for end users to quickly browse all this data. Hence recommender
systems can benefit end users (individuals as well as companies) in
efficiently finding suitable products. Rather than making end
users search through a vast array of items, recommender systems
can suggest suitable items to users based on popularity of the items
and the respective users’ buying behavior. Accordingly, in this
paper we explore two techniques widespread in recommender
systems, i.e. item-based collaborative filtering and association rule
mining, over Amazon review data on cellphones and accessories,
and build a baseline recommender system scalable to larger data.
Association rule mining is explored using the Apriori algorithm to
find patterns in the data from transaction history. Item-based
collaborative filtering is deployed using a correlation matrix to
find similar products. Both these techniques yield useful results as
evident from our baseline experiments. This work constitutes an
exploratory study with longtime products in e-commerce and sets
the stage for mining online data on relatively new products
pertinent to the Covid-19 pandemic. These include face masks,
hand sanitizers, disinfectant sprays, antibacterial wipes etc. Since
multiple vendors are designing such crucial products today, it is
important to provide recommendations to potential buyers. An
ultimate goal in our work is to build a recommender app for e-
commerce based on interesting results from our findings. This
work constitutes intelligent data mining scalable over big data in
e-commerce. It makes broader impacts on smart cities, since this
fits the smart living and smart economy characteristics.
Index Terms—Apriori algorithm, baseline recommender,
collaborative filtering, Covid-19, data mining, decision support
system, e-commerce, exploratory research, scalability, smart cities
I. INTRODUCTION
All of us today have encountered recommendation systems
in some facet of our daily lives. Netflix suggests movies we can
watch based on those we have already watched. Pandora
deploys user feedback of liking or disliking a song to play music
that we like. Facebook suggests friends to add considering our
current friends and their friends. YouTube recommends videos
similar to the ones we are currently viewing. An online store
offers products or services based on our purchase history. These
are just a few of the examples that might sound familiar. As data
gets bigger, it is even more significant to receive automated
recommendations, since the volume, velocity and variety of
such big data is directly proportional to the amount of time users
would spend browsing through it manually.
Recommender systems thrive largely on machine learning
algorithms that are employed to provide relevant suggestions to
users based on their own actions and those of other users
interested in similar products. Recommenders are very popular
on e-commerce websites where receiving an automated
978-1-7281-6251-5/20/$31.00 ©2020 IEEE
Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
recommendation makes browsing the content much easier for
an end user. Rather than always having to search through an
enormous range of items, it is useful if the system is able to
suggest other items similar to the one that a given user just
bought or browsed, based on those that are frequently bought
with the given item. This is extremely helpful as new items are
added constantly. Not having recommendations would make it
harder for users to find all they need. While this is helpful to
individual users, it is noticed that companies make a substantial
percentage of their revenue through user recommendations. For
example, Amazon.com states that 40% of their sales increased
using their recommendation engine [1].
Amazon currently uses item-to-item collaborative filtering,
which is one of the techniques we explore in this paper. They
use this technique to match each of a user’s purchased and rated
items to similar items which then appear to the user as
recommendation lists. Some types of recommendation lists on
Amazon.com are: a “frequently bought together” suggestion
based on pertinent items that other users generally buy along
with the items in the given user’s shopping cart and browsed
items; a “recommended for you” list based on a previous
purchase by the same user; and a “related to items you’ve
viewed” list based on items similar to those the given user has
already seen. Their real time recommendations given by item-
to-item collaborative filtering are surely powerful e-commerce
methods. Our work in this paper is orthogonal to such work and
is conducted with the long-term goal of mobile application
(app) development in order to enhance the shopping
experiences of targeted users.
Although this continues to be a constantly developing area,
there are some popular data mining techniques prevalent in
building good recommender systems. As is widely known, data
mining is the process of discovering interesting patterns and
trends from large amounts of data where the data sources can
include databases, data warehouses, the web, various other
information repositories, and data streamed into a system
dynamically [2]. Among the various data mining techniques, a
widespread one for years is association rule mining with
numerous applications in many domains. Collaborative
filtering is yet another technique that is useful in capturing user
and item preferences and hence is helpful in the design of
recommender systems. In this paper, we conduct an exploratory
study on these two techniques in order to build a simple baseline
recommender system in e-commerce that can be scaled to other
larger data. Our approach for developing this e-commerce
recommender system is illustrated in Fig. 1.
We explore association rule mining with the Apriori
algorithm and item-based collaborative filtering with a
4636
 Fig 1. Illustration of approach for developing baseline e-commerce recommender (green text
indicates frequent patterns using Apriori for association rule mining, and same-colored arrows
denote related products using item-based collaborative filtering)
correlation matrix on e-commerce data from the Amazon
review website focusing on products in the category of
cellphones and accessories. In association rule mining, the
focus of our work is on finding patterns with products from
previous transactions to offer recommendations to users based
on what they are currently buying or have previously bought.
In item-based collaborative filtering, we focus on using the
product ratings to recommend interesting products to buyers,
taking into account a similar / related product they recently
browsed or bought. We present our approach and experiments
in this paper leading to the development of the baseline e-
commerce recommender system.
Our work in this paper is a precursor to further e-commerce
research on new products relevant to the Covid-19 global
pandemic. These include items such as full personal protective
equipment (PPE), gloves, hand sanitizers, N-95 and other face
masks, antibacterial wipes and disinfectant sprays. Since this
pandemic has continued longer than expected, and the demand
for these products is increasing, several vendors are designing
new items in these product categories in order to keep up with
the supplies as per the demand. Hence, recommendations are all
the more crucial based on the use of these products and the
needs of the respective buyers. Accordingly, our ultimate aim
in this research activity is to design a recommender app in the
domain of e-commerce for the relevant product categories
based on the outcomes of our work. This would aid in decision
support for online shopping. This work on the whole fits the
characteristics of smart living and smart economy within the
realm of smart cities, thereby making broader impacts.
The rest of this paper is structured as follows: Section II
overviews related work on the data mining approaches and
recommender systems. Section III gives the details of the data
description along with statistics about the customers and the
product reviews. Section IV explains the details of our approach
and experimentation, i.e. the techniques deployed with respect
to the data in our work, namely, association rule mining and
4637
Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
item-based collaborative filtering, along with the experiments
conducted. Section V presents applications of recommender
systems using association rules and item-based collaborative
filtering with respect to companies as well as individual
customers, especially considering Covid-19 perspectives.
Section VI states the conclusions and future work.
II. RELATED WORK
Within the area of data mining and machine learning there
are numerous works related to recommender systems. An
interesting article [3] addresses the technology used to generate
recommendations and focuses on the application of data mining
techniques. The techniques discussed therein include cluster
analysis, classification, association rules, and a similarity
graph-based technique called Horting. Clustering techniques
are used such that consumers who appear to have similar
preferences are first grouped together, and certain predictions
can then be made for users based on those groups. The author
mentions how clustering methods usually produce relatively
less personal recommendations than other methods, and in
some cases the clusters have worse accuracy than collaborative
filtering based algorithms. In using classifiers, a category is
assigned to an output that can be used to classify new items.
Furthermore, the author states that association rules are among
the best-known examples of data mining in recommender
systems [3], which motivates us to explore them in our work.
Association rules identify items frequently found in
“association” with items in which a user has expressed interest
and hence using this technique can recommend items based on
current interest rather than using previous customer history.
Finally, they discuss Horting, a graph-based technique using
edges between nodes, such that the nodes depict users, in order
to determine the similarity between the respective users’ actions
and behaviors.
Another well-known technique used within recommender
systems is collaborative filtering. Many studies have compared
user-based and item-based collaborative filtering. The authors There are already apps and websites developed to aid with
in [4] experiment with several different item-based algorithms the Covid-19 pandemic. We find some online GIS (geographic
and the user-based nearest neighbor algorithms. They discuss information system) sources with mapping dashboards and
some challenges that are encountered when implementing user- applications [11] for tracking coronavirus around the world.
based collaborative filtering algorithms. One challenge is There is work [12] that provides free continuing professional
sparsity, where the result of the accuracy of recommendations development courses related to the Coronavirus disease.
may be poor if the recommender system is based on nearest
The brief survey of the literature herewith inspires us to
neighbor algorithms and unable to make recommendations for
explore data mining techniques for recommender systems
a particular user. Another challenge is scalability since the
within the context of online shopping, falling in the overall
nearest neighbor algorithms require computation that grows
realm of e-commerce. Though we conduct some experiments
with both the number of users as well as the number of items
with other techniques as well, the work we present in this paper
[4]. Furthermore, they conduct experiments on item-based
focuses on two techniques, i.e. item-based collaborative
recommendation algorithms and dive into the similarity
filtering and association rule mining. This is because we get
computation and prediction generation. Their experiments
more interesting results with these as compared to the other
suggest that item-based algorithms provide dramatically better
techniques, in addition to the fact that they are highlighted as
performance than user-based algorithms, while also providing
being among the most useful by the works in the literature. We
better quality than the best available user-based algorithms [4].
conduct exploratory research in this paper using existing data
Based on such research, we choose to explore item-based
from online shopping of cellphones and accessories in order to
collaborative filtering within our work.
study these techniques in detail and thus set the stage for their
Some recommender systems rely on sentiment analysis, also
use within other crucial products, i.e. those useful during the
known as opinion mining, which can classify emotions toward
Covid-19 pandemic, so that we can eventually build a
an item and determine whether there is a positive or negative
recommender app based on our findings. Some of this is
opinion expressed. Different methods have been harnessed for
orthogonal to our earlier work on app development [13, 14].
sentiment analysis in the literature. In [5], the authors provide
The work in this paper constitutes an initial study in which
an overview of sentiment analysis in social media that is also
patterns and product correlations are discovered from online e-
applicable to recommenders. They analyze the progress of
commerce data to provide relevant suggestions to users via a
techniques, review the advances in applications and discuss the
baseline recommender system, potentially useful for other
limitations as well. Overall, a sentiment analysis system can
products as well as for a recommender app development.
benefit from implementation using NLP (Natural Language
Processing) techniques along with machine learning in order to
predict a user rating from their written review. We could III. DATA DESCRIPTION
consider sentiment analysis as potential future work for The data used in this exploratory study is from Amazon
enhancing our simple baseline recommender system. review data sources (2018) as this is a widely used source found
Combining techniques can also be useful. In [6], a study is in the literature, e.g. [15]. The data includes reviews within the
aimed at designing a recommender system based on the explicit date range of May 1996 through October 2018. The raw review
and implicit preferences of the customers in order to increase data is of the size 34GB with 233.1 million reviews. An
prediction accuracy using both sentiment analysis and example of the data included in a product review in the category
collaborative filtering. The system uses NLP along with a of cellphones and accessories can be seen in Fig. 2 next.
supervised classification approach over hotel guest reviews in
order to create recommendations. In [7], recommendations are
provided based on sentiment analysis over partially labeled
training data, constituting a combination of supervised and
unsupervised learning tending towards a hybrid approach.
Furthermore, recommender systems are most useful when
users need to sort through large amounts of data. The Covid-19 Fig. 2. Example of an Amazon product review
pandemic has been accompanied by rapidly growing data. As
provided in [8] and [9], there is an enormous amount of data This raw data is subjected to preprocessing. Consider the
associated with the pandemic. Along with that comes a new following code snippet.
demand for online shopping due to social distancing. Due to merged_data = pd.merge(reviewdata, metadata, on=’asin’, how=’left’)
this e-commerce has been at the forefront of retail affecting Here, the review data json file is joined with the product
consumer behavior. According to [10], many countries can see metadata json file for the cellphones and accessories category.
the major shift to e-commerce, e.g. China where 50% of the These are then merged through the product ID, in order to have
people are using e-commerce more frequently today than all the attributes needed, in one data-frame named merged data.
earlier. Knowing this, recommender systems will certainly play Next, any null values are removed. Due to a large amount of
an important role in e-commerce users’ lives. On the whole, data comprising 10,063,255 reviews, we downscale the data to
they will make it easier for users to shop online.

4638

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
a sample of 99,077 reviews for this initial study. This is This approach has been illustrated in Fig. 1. Its implementation
conducted using supervised instance-based filtering. and experimentation are conducted using Python programming
Before implementing the techniques on the dataset, it is in Jupyter Notebook [16], an open-source interactive web
important to explore the data in order to comprehend the application that allows combining code, output and explanatory
attributes and fathom the content of the data. Specifically, it is text in one document. Details are described next.
useful to get information regarding the number of customers,
reviews per customer, number of products, reviews per product
and information about the ratings. In order to attain this, we first
check for the unique number of customers and products. Fig. 3
depicts the retrieval of those numbers through a function which
counts the unique values from a list. We also calculate the
average number of reviews per customer and per product.
In the data sample shown herewith, the number of unique
customers is 69,346, the number of unique products is 2,106,
the average number of reviews per customer is 1.4 and the
average number of reviews per product is 47. While this is a
small sample, it is well-representative of the data and presents
an example of how a subset can be useful in dealing with much
larger datasets. This helpful later in our further work on
Fig. 5. Distribution of ratings histogram
extending this study to bigger datasets and hence is important
from a scalability angle.
A. Association Rule Mining
1) Overview of Technique: As widely known, association
rule mining involves discovering associations of the type X ⇒Y
(X implies Y) where X is the antecedent and Y is the consequent
[2]. It is a powerful method for market basket analysis, which
aims at finding regularities in shopping behavior of customers
in supermarkets, mail-order companies, online shopping and so
on [17]. This analysis can especially help companies increase
Fig. 3. Sample calculations for number of unique customers and products as their sales by discovering buying patterns in their products and
well as average number of reviews per customer and per product
hence recommending other products. The classical Apriori
algorithm for discovering association rules is deployed in our
Thereafter, the summary statistics is retrieved pertaining to
work. There are three important aspects here: (1) confidence,
the ratings column in the data data-frame. As observed here in
i.e. the probability that, given X occurs, Y also occurs, (2)
Fig. 4, the total count of ratings is 99,077 with a mean of 4.3,
support, i.e. probability of both X and Y occurring together in
standard deviation of 1.16, minimum rating of 1, and maximum
dataset and (3) lift, i.e. the probability of the items X and Y
rating of 5. Also, more than half of the products have a rating
occurring together divided by a multiplication of their
of 5 which is good. As depicted in Fig. 5, a histogram is created
individual probabilities. The calculations of each are as given
to view distribution of the ratings for at-a-glance analysis.
in Equations (1) to (3) where P denotes probability, #X and #Y
are the number of transactions containing X and Y respectively
and #T is the total number of transactions in the dataset.

Confidence (X⇒Y) = 𝑃(𝑌 | 𝑋) = #(𝑋 ∩ 𝑌)/ #𝑋 (1)

Support (X⇒Y) = 𝑃(𝑋˄𝑌) = #(𝑋 ∩ 𝑌)/ #𝑇 (2)
Lift (X⇒Y) = 𝑃(𝑋˄𝑌) / (𝑃(𝑋) × 𝑃(𝑌)) (3)

The Apriori algorithm allows discovery of sets of products

Fig. 4. Summary of statistics on the ratings column in the data data-frame
IV. APPROACH AND EXPERIMENTATION
The approach deployed in this exploratory study entails
harnessing the techniques of association rule mining as well as
item-based collaborative filtering over the data in this work.
regardless of the order in which they are bought. The rules
discovered can then be used to make retail recommendations to
users based on the items bought. Implementing this algorithm
in our work involves using the apriori class from the apyori
library and setting parameters to get the association rules for the
products. The adaptation of association rule mining in our work
appears in Algorithm 1 herewith and is explained next.

4639

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.

Algorithm 1: Association Rule Mining for E-com Recommender
Input: E-commerce website of category cellphone & accessories from Amazon
review data; min-sup, min-conf, min-lift, min-len thresholds for confidence c,
support s, lift t, number of products p respectively
1. Preprocessing
a. Merge review data R and meta data M into data-frame F
b. Delete tuples with null values
c. Select a sample of data D from F
2. Select asin, also_buy attributes from D /** product id, items that
the user also buys */
3. For (i = 1 to n) in D ε F /** n = number of rows */
a. Merge asin, also_buy cols into new col bought_items
b. Delete asin and also_buy cols
c. For (j = 1 to a) /** a = number of asin values */
i. Split asin in bought_items into b separate
cols /** b = number of bought_items */
ii. Hence obtain altered data-frame D`
4. Convert D` to list L
5. Deploy Apriori algorithm on L using min-sup, min-conf, min-lift,
min-len thresholds
6. Output products such that s > min-sup, c > min-conf , t > min-lift,
and p > min-len /** include values of support s, conf c, lift t, and
number of products p in output */
2) Deployment on Dataset: In the given dataset, the following
two attributes are chosen from the many available ones in the
cellphone and accessories category (see Fig. 6 for a sample).
• asin: the ID of a given product bought by users
• also_buy: IDs of products that users also bought with it
Fig. 6. Data attributes used in the deployment of association rules
Given these two attributes, data transformation is performed
using merging and splitting of attributes. Consider the
following code snippet.
data[‘bought_items’]=data[[‘asin’,‘also_buy’]].agg(“, ”.join, axis=1)
data=data.drop([‘asin’, ‘also_buy’], axis=1)
Here, the asin and also buy attributes are merged in order to
create a new column named bought_items. Next, the column
bought_items is split into separate columns, one for each asin
that occurs within bought_items as shown in Fig. 7.
The last step in the deployment of Apriori is to change the
format of the dataset. Heretofore, the data is in a pandas data-
frame but the apyori library requires the dataset to be in the
form of a list. Thus, our code converts the pandas data-frame
into a list, iterating through each row in the data-frame.
3) Experimentation: In order to implement association
rule mining, pandas, numpy, and the apyori library as well as
the dataset are all imported. The Apriori algorithm is deployed
on the dataset by using the apriori class from the apyori library.
The parameters are set as needed. The first parameter is the list
named records from which rules are to be extracted. The
second parameter, min-sup, i.e. the minimum threshold for
4640
Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
support is an experimental value that is altered in multiple
executions. The third parameter, min-conf, i.e. minimum
threshold for confidence is also altered as an experimental
value in several executions. However, confidence is set at a
much higher value than support in our experiments shown here,
e.g. min-sup=0.02 and min-conf=0.90, both varied around
these approximate values. The fourth parameter, min-lift, which
shows the strength of a rule, is varied in some experiments and
is set to 3 in the examples shown here. The last parameter, min-
length, which specifies the number of products in the rule, is set
to 2 in all our experiments, since in these set of experiments,
we deal with 2 products. Based on these experiments, we
observe the output with association rules.
Fig. 7. Splitting bought_items into separate columns for each product ID
4) Output: By experimenting with the Apriori algorithm
on the product purchase history for the category of cellphones
and accessories from Amazon review data, we discover some
association rules. These rules can be used to provide suitable
recommendations through the users’ purchase history by
recommending what they should buy next or by recommending
additional products when they are considering a given one. The
recommendations through association rules are all based on
patterns that previous users have created through buying
products. These recommendations can be extremely helpful to
a given user and are likely to bring more profit to companies
implementing them through recommendation systems.
Fig. 8 shows a few sample outputs from association rules. It
includes two products and the calculated support, confidence
and lift values. The products are shown in the first two columns
with their asin (the product ID). The original dataset does not
provide product titles for asin under the attribute of also_buy
which has the product IDs of items the user also bought. Using
row index 0 as an example, the output of the association rules
can be interpreted as follows:
 If a user buys product B00KR9FT4Q then there is a 0.93 probability
that user also buys B00BW0X892 (based on confidence).
 The probability of product B00KR9FT4Q occurring in the data set
is 0.021 (based on support).
 The probability of product B00BW0X892 being bought together
with product B00KR9FT4Q is 37.7 (based on lift).
The product IDs are then mapped in our program to the
corresponding product names using the literature on these
products. A snapshot of this mapping is presented in Fig. 9 with
the respective products. These values can therefore be used to
stipulate recommendations via the users’ purchase history by
suggesting what they should buy next or by suggesting other
relevant products when they are considering a certain product.
Such recommendations are incorporated into the output panel
of our simple baseline e-commerce recommender system to
offer meaningful suggestions to users about products.
Fig. 8. Sample outputs from association rule mining
Fig. 9. Mapping of product IDs to product names
B. Item-based Collaborative Filtering
1) Overview of Technique: There are two filtering techniques
used in recommendation systems, namely, content-based
filtering and collaborative filtering. Content-based filtering
analyzes and constructs portraits of users and items through
extra information, such as user-item profiles, and content
analysis. Collaborative filtering instead derives suitable
recommendations based on interactive information between
users and items such as browsing, rating, and clicks [18]. There
are two types of approaches in collaborative filtering: user-
based and item-based. User-based collaborative filtering finds
users with similar consumption patterns as a given user and
offers the content that these similar users found interesting.
Item-based collaborative filtering uses the similarity between
the items to determine whether a given user would like it or not
[19]. Fig. 10 visually showcases the difference between the two
collaborative filtering techniques as found in the source
[https://medium.com/@cfpinela/recommender-systems-user-based-and-item-
based-collaborative-filtering-5d5f375a127f].
4641
Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
Fig. 10. User-based versus item-based collaborative filtering
In this paper, we focus on item-based collaborative filtering.
The reasons for this are twofold. First, we notice from the
literature that it is the preferred method in most works. Second,
we already consider association rule mining to find user-based
preferences, so we explore another method here. Both these
techniques are useful in providing recommendations from two
different perspectives. We apply item-based collaborative
filtering on our dataset of cellphones and accessories from
Amazon. This involves first creating a user-item rating matrix,
then applying truncated Singular Value Decomposition (SVD),
a dimensionality reduction matrix factorization technique, and
lastly calculating similarity measures through a correlation
matrix. Entering the product ID then returns recommendations
of top similar products. This is conveyed as the output to users.
Item-based collaborative filtering is adapted in our work as
shown in Algorithm 2 herewith, and is elaborated next.
Algorithm 2: Item-Based Collaborative Filtering for E-com Recommender
Input: E-commerce website of category cellphone and accessories from
Amazon review data, k = number of products in top-k ranking
1. Preprocessing
a. Merge review data R and meta data M
b. Delete tuples with null values
c. Select a sample of data D
2. Select asin, title, reviewerID and rating attributes /** product id,
name, reviewer info, ratings by reviewers */
3. Create user-item matrix U
a. Set index as reviewerID, columns as asin and values as
the product rating
b. Replace null values with 0s
4. Create new matrix N = UT /** Transpose the matrix */
5. Perform dimensionality reduction on N
6. Generate correlation matrix C based on N using Pearson’s
correlation coefficient ρ
7. For (i = 1 to k) in C /** products from 1 to k, where k is a
parameter for top-k */
a. Find asin with correlation value vi /** values v1 to vk as
top-k correlation values */
b. Map asin to title for k products /** top-k product titles
found from C */
c. Output O as title for recommended product /** top-k
product names */
8. Convey output O to the user
2) Deployment on Dataset: Among the many attributes
available in the original data of the cellphones and accessories
category, the ones considered in the item-based collaborative
filtering herewith are as follows (See Fig. 11 for a sample).
• reviewerID - the identification of the reviewer
• asin - the identification of the product
• title - the name of the product
• rating - the rating of the product

Fig. 12. Sample output of a pivot table with a user-item matrix

Fig. 11. Data attributes used in deployment of item-based collaborative filtering

In addition to the obvious attributes of asin and title depicting

the product ID and product name respectively, it is important to
consider the reviewer ID and the rating given by the reviewer
to the product. The reviewer ID serves to identify each
reviewer, thereby helping to distinguish between reviews (e.g.
we do not want all the reviews to come from the same few
reviewers). The rating serves to assess the product which is
clearly significant in providing recommendations.

3) Experimentation: The implementation of item-based

Fig. 13. Sample output of transposing the user-item matrix
collaborative filtering in this work entails importing pandas,
numpy, and TruncatedSVD from Sci-Kit Learn, in addition to
importing the review dataset. In order to maintain a simple
implementation, the product IDs, i.e. asin values are considered
initially. These are subsequently mapped to the title values in
the recommender output.
Next, a user-item matrix is created by making a pivot table.
In this table, the index is set as the reviewerID, columns as asin
and values as the product rating. In this implementation, any
null values are replaced with 0s. Fig. 12 shows a sample output
of the pivot table creating a user-item matrix. However, note
that for the item-based collaborative filtering technique, the
main interest is in the items. Hence, this matrix needs to be
transposed. Fig. 13 shows the transposed matrix that now has
the reviewerID as the columns and asin as the index. Fig. 14. Sample correlation matrix output
Thereafter, dimensionality reduction is performed by using
truncated singular value decomposition (SVD), a matrix
4) Output: The end goal here is to use the item-based
factorization technique. By using Scikit-Learn, the truncated
collaborative filtering to recommend similar products to those
SVD is implemented such that it produces a factorization where
that a user considers. By incorporating the product ratings for
the number of columns is the same as the specified truncation.
the category of cellphones and accessories from Amazon
Lastly, a correlation matrix is generated as portrayed in Fig. 14
review data, we have seen that a correlation matrix has been
in order to find similar products. Here, Pearson’s correlation
generated as found in Fig. 14 to show how each product
coefficient ρ is used via numpy’s corrcoef function to generate
correlates to another. This matrix is useful in proffering
this matrix. This correlation matrix shows how each product
recommendations. By incorporating this, the output panel of
correlates to another based on the reviewer ratings.
our simple baseline e-commerce recommender system is

4642

Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
implemented. This locates the product a user is considering and
finds similar products using the output of the item-based
collaborative filtering. The product ID, i.e. asin is mapped to
the product name, i.e. title in order to get a meaningful
recommendation of the products that the system is suggesting
to the user. This sets the stage for the final output that includes
the names of recommended products. Fig. 15 shows an excerpt
from the final output to the user for the product Insten
SportBand with Case as an example. In this figure, that product
itself appears at index 0 with a correlation value of a full 1.0
which is obvious.
As observed in Fig. 15, that there are 9 recommendations that
show the highest correlations to the given product. Based on
these recommendations, there are similar / related products on
the list. For instance, the considered product is a band and the
first recommendation at index 1 shows another band. Another
detail is that the considered product is a general phone case for
cellphones, hence the recommendations show different phone
case options. As evident from this example, the item-based
collaborative filtering approach suggests correlated products to
the user. Likewise, it has been noticed in our exploratory study
that several useful recommendations are provided by our simple
baseline e-commerce recommender.
Fig. 15. Recommender output example for given product at index 0
V. RECOMMENDER SYSTEM APPLICATIONS IN COVID-19
Based on our initial exploratory research in this paper as well
as a general survey of classical and state-of-the-art literature
relevant to recommender systems, e.g. [3, 4, 6, 7, 20, 21, 22],
we briefly outline a few applications herewith. We focus on the
Covid-19 angle here, incorporating the need of the hour as seen
in recent works, e.g. [8-12]. Similar claims can apply to other
general perspectives as well. These applications pertain to using
association rules and item-based collaborative filtering in
4643
Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
within recommender systems, leveraging benefits from both the
company perspective and a customer perspective.
A. Usefulness of Association Rule Mining
1) Company: If a company would implement a Covid-19
based recommender system using association rules techniques,
there would be many advantages. A company could use the
patterns in user purchase history to create new promotions, e.g.
if “Tzumi antibacterial wipes” are bought very frequently with
“Germ-X hand sanitizers”, there could be a promo of buy two
get one free in this context, provided there is enough supply of
these items. These marketing strategies can be used by an
online shopping site such as Amazon to jointly promote the sale
of these items. In the case of a physical store such as CVS
Pharmacy, the association rules would help identify where the
products should be placed on the shelves for an easier shopping
experience and for product marketing to customers. In an online
aspect, a company such as Amazon could recommend items
based on the item the customer currently has in their cart or has
previously purchased, offering suggestions of Covid-19 related
items frequently bought together. All these benefits could bring
companies such as Amazon and CVS more revenue, in addition
to helping them in better understanding customer behavior and
conveying suggestions to product vendors on manufacturing
specific items in high demand. Most importantly, this would
have the significant broader impact of producing more items
really needed by users based on their buying behavior.
2) Customer: If a customer uses a recommender system
that deploys association rules, focusing on Covid-19 related
transactions, the customer would have a seamless shopping
experience. In times of the Covid-19 lockdown as well as in its
recovery phase and the aftermath, customers are likely to spend
less time in places such as grocery stores and pharmacies,
preferring to be at home or outdoors in the open air. Hence, in
the case of any physical store, e.g. a pharmacy, if frequently
bought items, especially those much needed during this
pandemic, are placed next to each other customers would have
the pleasure of shopping with greater efficiency. Moreover, if a
certain item is placed on a shelf instead of another similar /
related item that is out-of-stock, then that would make things
easier. For example, Lysol disinfectant sprays have been out-
of-stock for quite some time and customers have been using
Windex cleaners instead on kitchen tabletops, light switches
etc. Such a fact could be discovered by association rules on
earlier data when both items were available, e.g. customers who
bought Lysol sprays also bought Windex cleaners. Hence,
placing Windex cleaners on the same shelf as Lysol sprays used
to be (when in-stock) would provide customers a better, quicker
shopping experience and would also save the store staff the
time of attending to those customers’ requests about finding
similar items. Likewise, when shopping online it is good to
have items that are frequently bought together (or used in place
of each other) to be recommended. This would make the
customers’ life easier; if they were also going to get one of the
recommended items in addition to or instead of a given item,
they would not have to spend much time searching. Overall,
such arrangements would give the customers a more pleasant
time while shopping and therefore they would be likely to shop
at the same place again, online or in-person.
B. Usefulness of Item-based Collaborative Filtering
1) Company: If a company would implement a Covid-19
related recommender system using item-based collaborative
filtering techniques, it would be beneficial in various ways. A
huge plus is that companies with online shopping facilities
would benefit from ratings provided on relatively new items
made by possibly new vendors, especially since some such
items would be imperative during this pandemic, e.g. face
masks. While N95 masks have been recommended by health
organizations, it is found that they are now more prevalent
among medical professionals mainly due to the scarcity of these
masks. Instead washable cloth masks and disposable surgical
masks are more in use by the general public. Also, since N95
masks are not comfortable to wear for too long, it is observed
that almost equally compatible KN95 masks are more common
and are recommended for longtime daily use due to their safety,
comfort and availability. In fact, KN95 masks come with
descriptions such as “perfect for office”. They are indeed
particularly good for offices and also for all-day use in homes,
especially where there are senior citizens etc. Likewise, the
correlations between such pertinent items as captured through
item-based collaborative filtering are crucial here, especially
with respect to the brands of specific products and their
relationships that would provide guidance in online marketing.
Customers would be delighted that they do not have to spend
too much time on browsing and searching, and would thus be
likely to buy more than one item if similar ones are displayed
via the results of item-based collaborative filtering. For
example, if customers are shopping for “N95 masks” and the
recommendations display “KN95 masks”, the customers would
see highly relevant and useful products and buy them. Also,
they would be inclined to revisit the concerned site due to liking
item-based recommendations therein, thus possibly buying
more items and bringing the company more revenue.
2) Customer: If a customer would come across a
recommender system pertinent to Covid-19 that uses item-
based collaborative filtering, the customer would appreciate the
personalization. Nowadays people prefer to have personalized
recommendations rather than general ones. For example, if the
customer is a healthcare professional (or is shopping on behalf
of one through a hospital), it would be beneficial to get ratings
on items such as full PPE (personal protective equipment) and
different brands of gloves from other healthcare professionals
who have bought similar items online. Having similar items
recommended based on user ratings would imply that the
recommendations would be geared towards that customer’s
likings. By having such recommendations, a customer would
easily be able to compare and browse more such products
without having to search again. Overall, the customer would
4644
Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
enjoy the shopping experience and would also save valuable
time, much needed during this pandemic and its aftermath.
VI. CONCLUSIONS AND FUTURE WORK
This paper constitutes exploratory research that applies and
demonstrates intelligent data mining via the techniques of
association rules and item-based collaborative filtering
employed within an approach for a baseline recommender
system in e-commerce. This focuses on the online shopping of
cellphones and accessories and uses the Amazon review data
(2018) as its dataset for exploratory study. The association rule
mining technique through the Apriori algorithm allows for the
discovery of patterns of products frequently bought together by
scrutinizing the purchase history of users. The item-based
collaborative filtering technique suggests highly correlated
products, based on a given product of interest, calculated herein
from users’ ratings of similar / related products. The baseline
recommender system built in this work offers interesting
suggestions to users considering the outputs of association rules
and item-based collaborative filtering. This system is quite
seamlessly scalable based on the manner in which techniques
have been adapted and implemented therein, hence the
respective techniques built into this approach can be feasibly
deployed on other larger datasets with suitable modifications.
Some future work includes experimenting with sentiment
analysis for opinion mining of users’ reactions to products,
which entails the polarity classification of emotions in text.
Since the dataset used in this work includes written reviews, the
classifications of the attitudes towards the products can
potentially be useful to improve recommendations. The work in
this paper sets the stage for conducting experiments with data
relevant to the recent Covid-19 pandemic, e.g. products such as
face masks, hand sanitizers, antibacterial wipes and others. The
baseline recommender system implemented herein using
intelligent data mining via the techniques of association rules
and item-based collaborative filtering can be potentially helpful
in analyzing Covid-19 related data as well. Additionally, the
accuracy of the recommender systems in this paper will be
evaluated when we extend this work to Covid-19 related
datasets. We may also explore advanced machine learning
techniques as needed to improve the accuracy.
Given this research, we aim to build an app that would
potentially serve as a recommender system for e-commerce
data based on such products. We would incorporate aspects
from HCI (Human Computer Interaction) and IoT (Internet of
Things) in implementing the proposed app. Studies on accuracy
and effectiveness would be conducted therein since this work
would entail full-fledged app development that would go
beyond our simple baseline recommender in this paper. Some
of this further work could be in line with our earlier work on
apps [13, 14, 23] where detailed user studies have been
conducted and on other topics [24, 25] where case studies and
surveys have been presented.
 On the whole, this paper presents exploratory research on
intelligent data mining techniques to enhance decision support
in e-commerce, provides a simple baseline recommender
system scalable to larger datasets, paves the ground for further
work especially relevant to Covid-19, and fosters suitable app
development on recommender systems. The work in this paper
broadly fits the characteristics of smart economy and smart
living in the overall paradigm of smart cities. The research
conducted here is supplementary to our other pertinent work on
smart city aspects such as smart governance with ordinance
mining [26], smart environment with green computing [27],
smart people with development of software tools supporting
21st century education and learning [28], and smart mobility
with peripheral contributions to autonomous vehicles [29] and
significant ones to object detection [30]. We anticipate that
further research pertinent to Covid-19 related products and
recommender app development would be significantly
beneficial to decision support in e-commerce and would also
make broader impacts on smart cities.
ACKNOWLEDGMENTS
This work is partly supported by a Graduate Assistantship
and a Student Scholarship for Jessica Lourenco, as well as a
Faculty Scholarship Program and a Doctoral Faculty Program
for Aparna Varde, all from internal sources at Montclair State
University. There is also partial support through an internal
grant (SGPD) and an external grant (MRI) in an area relevant
to this project. We thank all our funding sources.
REFERENCES
[1] S. Tareq, M. Noor and C. Bepery, “Framework of dynamic
recommendation system for e-shopping”, International Journal of
Information Technology, Vol. 12, 2020, pp. 135-140.
[2] J. Han, M. Kamber, and J. Pei, “Data Mining: Concepts and Techniques”,
2012, pp. 8.
[3] J. B. Schafer, “The application of data-mining to recommender systems”,
Encyclopedia of Data Warehousing and Mining, 2005, pp. 44-48.
[4] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl, “Item-based collaborative
filtering recommendation algorithms”, The WWW Conference, 2001, pp.
285-295.
[5] L. Yue, W. Chen, X. Li, W. Zuo, and M. Yin, “A survey of sentiment
analysis in social media”, Knowledge and Information Systems (KAIS)
Journal, 2019, Vol. 60, pp. 617-663.
[6] F. Abbasi, A. Khadivar, and M. Yazdinejad, “A grouping hotel
recommender system based on deep learning and sentiment analysis”,
Journal of Information Technology Management, 2019, 11(2), pp. 59–78.
[7] K. Gandhe, A. Varde, and X. Du, “Sentiment Analysis of Twitter Data
with Hybrid Learning for Recommender Applications, IEEE UEMCON
Conference, 2018, pp. 57-63.
[8] P. Lin, P. Moonwy, C. Schoenick, S. Kohlmeier, D. Rishi, T. Bozsolik,
B. Hamner, “COVID-19 Open Research Dataset Challenge (CORD-19)”,
Allen Institute for Artificial Intelligence, https://www.kaggle.com/allen-
institute-for-ai/CORD-19-research-challenge, 2020.
[9] English Corpora, “The Coronavirus Corpus”, https://www.english-
corpora.org/corona, 2020.
[10] B. Anam, S. Akhter, A. Hassan Qurashi, M. Shaheen, and U. Malaysia,
“Coronavirus affects e-commerce globally”, Journal of Xi’an Shiyou
University, 2020, ISSN No: 1673-064X.
4645
Authorized licensed use limited to: Raytheon Technologies. Downloaded on May 14,2021 at 05:02:23 UTC from IEEE Xplore. Restrictions apply.
[11] K. Boulos, and E.M. Geraghty, “Geographical tracking and mapping of
coronavirus disease COVID-19”, International Journal of Health
Geographics, Springer Nature Journal, 2020, Vol. 19, No. 8, DOI:
10.1186/s12942-020-00202-8.
[12] BMJ Learning, “Coronavirus disease 2019 (COVID 2019): Supporting
online courses”, https://new-learning.bmj.com/covid-19, 2020.
[13] Pathak, D., Varde, A., Alo, C., and Oteng F., “Ubiquitous Access for
Local Water Management through HCI Based App Development”, IEEE
UEMCON Conference, 2020, pp. 227-233.
[14] Varghese C., Varde, A. and Du, X., “An Ordinance-Tweet Mining App
to Disseminate Urban Policy Knowledge for Smart Governance”, I3E
2020 Conference, Vol. 2, pp. 389-401.
[15] J. Ni, J. Li, J. McAuley, “Justifying recommendations using distantly-
labeled reviews and fined-grained aspects,”, EMNLP-IJCNLP
Conference, 2019, pp. 188-197.
[16] T. Kluyver, B. Ragan-Kelly, F. Perez, B. Granger, M. Bussonnier, J.
Frederic, K. Kelley, J. Hamrick, J. Grout, S. Corlay, P. Ivanov, D. Avila,
S. Abdalla, C. Willing, Jupyter Development team, “Jupyter notebooks –
a publishing format for reproducible computational workflows”,
Positioning and Power in Academic Publishing: Players, Agents and
Agendas, 2016, pp. 87-90.
[17] R. Agrawal, and R. Srikant, “Fast Algorithms for Mining Association
Rules”, VLDB Conference, 1994, pp. 487-499.
[18] F. Xue, X. He, X. Wang, J, Xu, K. Lu, and R. Hong, “Deep Item-based
Collaborative Filtering for top-N Recommendation”, ACM Transactions
on Information Systems, 2019, Article no. 33, doi.org/10.1145/3314578.
[19] L. Candillier, F. Meyer, and M. Boullé, “Comparing State-Of-The-Art
Collaborative Filtering Systems”, MLDM 2007, pp. 548-562.
[20] K. Hammond and A. Varde, “Cloud Based Predictive Analytics: Text
Classification, Recommender Systems and Decision Support”, IEEE
ICDM Conference (workshops), 2013, pp. 607-612.
[21] J. B. Schafer, J. Konstan, J. Riell, “Recommender Systems in E-
Commerce”, ACM E-COMMERCE Conference. 1999, pp. 158-166.
[22] J. Lu, D. Wu, M. Mao, W. Wang, and G. Zhang, “Recommender System
Application Developments: A Survey”, Decision Support Systems
Journal, 2015, Vol. 74, pp. 12-32.
[23] D. Karthikeyan, S. Shah, and A. Varde, “Interactive Visualization and
App Development for Precipitation Data in Sub-Saharan Africa”, IEEE
IEMTRONICS conference, 2020, pp. 302-308.
[24] S. Chandra, A. Varde, and J. Wang, “A Hive and SQL Case Study in
Cloud Data Analytics”, IEEE UEMCON conference, 2019, pp. 112-118.
[25] P. Basavaraju, and A. Varde, “Supervised Learning Techniques in Mobile
Device Apps for Androids”, ACM SIGKDD Explorations Journal, 2016,
Vol. 18, No. 2, pp. 18-29.
[26] M. Puri, A. Varde, X. Du, and G. de Melo, “Smart Governance Through
Opinion Mining of Public Reactions on Ordinances”, IEEE ICTAI
Conference, 2018, pp. 838-845.
[27] M. Pawlish, and A. Varde, “The DevOps Paradigm with Cloud Data
Analytics for Green Business Applications”, ACM SIGKDD
Explorations Journal, 2018, Vol. 20, No. 1, pp. 51-59.
[28] Varghese, A., Varde, A., Peng, J. and Fitzpatrick, E., “A Framework for
Collocation Error Correction in Web Pages and Text Documents”, ACM
SIGKDD Explorations Journal, 2015, Vol. 17, No. 1, pp. 14-23.
[29] P. Persaud, A. Varde, and S. Robila, “Enhancing Autonomous Vehicles
with Commonsense: Smart Mobility in Smart Cities”, IEEE ICTAI
Conference, 2017, pp. 1008-1012.
[30] A. Garg, N. Tandon, and A. Varde, “I Am Guessing You Can’t Recognize
This: Generating Adversarial Images for Object Detection Using Spatial
Commonsense”, AAAI Conference, 2020, pp. 13789-13790.