0 Up votes0 Down votes

12 views5 pagesClick Prediction and Preference Ranking of RSS Feeds

Dec 11, 2017

© © All Rights Reserved

PDF, TXT or read online from Scribd

Click Prediction and Preference Ranking of RSS Feeds

© All Rights Reserved

12 views

Click Prediction and Preference Ranking of RSS Feeds

© All Rights Reserved

- A Comparative Analysis of Opinion Mining and Sentiment Classification in Non-English Languages
- Development of an Expert System for Diagnosis and appropriate Medical Prescription of Heart Disease Using SVM and RBF
- cec2007ok
- Ho 3114511454
- ObjectNess Bing
- Ya Seen 2015
- IR - Unsupervised-chi Square
- 54
- A COMPARATIVE STUDY ON HUMAN ACTION RECOGNITION USING MULTIPLE SKELETAL FEATURES AND MULTICLASS SUPPORT VECTOR MACHINE
- Survey Newsletter
- smarter science step 1 - peoe template
- Report 1
- clef2014wn-qa-papanikolaouet2014
- d 1428042413
- thomassarah 5
- Diff Criteria
- 016A REVIEW ON SENTIMENT ANALYSIS OF SOCIAL MEDIA DATA USING TEXT MINING AND MACHINE LEARNING
- Lecture 3 - Linear Methods for Classification
- FORECASTING1.pdf
- Data Science Scenarios and Solution Templates

You are on page 1of 5

Steven Wu

1 Introduction

RSS (Really Simple Syndication) is a family of data formats used to publish frequently updated works. RSS

is often used for temporally relevant information updates. An RSS document consists of a series of entries,

each tagged with a title, description, link, date of publication, and sometimes other metadata.

However, it is often the case that a large amount of RSS feeds in aggregate will create considerably more

information than the average user will be able to follow. In this case the user will read only those feeds that

he or she finds interesting. However, the volume of information involved can sometimes make this search for

interesting reading a time-consuming and laborious task.

Several machine learning methods have been applied with some success to text categorization problems,

such as Bayesian spam filters for email[1]. Here we explore text categorization algorithms for the purpose of

ranking user preferences for RSS entries.

In our problem we are given a series of RSS feeds and click information and must derive from these the

preferences of the user. We are given a set of candidate entries X = {X (i) }. For each X (i) we define y (i) to

correspond to the number of clicks made on X (i) . We also store publication time t(i) , ie the time at which

(i)

the entry was produced, and associated timestamp vectors z (i) such that zj is the time of j 1th click on

x(i) . Hence, y (i) = |z (i) |. We will also specifically discuss the time to click c(i) , defined as

(

(i)

(i) z0 t(i) for |z (i) | > 0,

c =

for |z (i) | = 0

Each entry X (i) also has a corresponding feed ID f (i) which specifies the specific feed that it belongs to. The

importance of this is that different RSS feeds are treated in different ways by the user - a user may check

certain feeds more often or be more likely to click entries from it due to topical reasons.

We are attempting to predict preference using click data as an indicator for preference. However, clicks

are not a perfect indicator of preference, in that the average user will not have an opportunity to click every

article that interests them. It is also the case that even if an article is to be clicked, it will likely not happen

for some nonzero amount of time after that article has been published on RSS.

Hence we will make a few assumptions regarding click behavior. First, we assume that there exists some

latent variable I (i) tied to X (i) which is an indicator for whether an entry will ever be clicked. We also

assume that if an entry X (i) , then c(i) is distributed as an exponential. This is to deal with the following

difficulty: given some time t, if we have some entry Xi where yi 1, then we know both ci and Ii . However,

if, conversely, yi = 0, then we have 3 possible explanations. First, the entry may not ever be read, ie Ii = 0.

Second, the entry may be too old, and the reader may have seen the entry elsewhere, ie t ti is too large.

Lastly, the reader may eventually read this entry, but not have had the opportunity to yet, ie ti + ci > t.

We may now formally state that our goal is to determine P (I (i) = 1), i.e. the chance that the user will

ever read a given feed entry.

(i)

With these assumptions we can find probability p(yt > y (i) ) by specifying functions for the rate

parameter of the exponential, f (X), and a preference function fI (X) returning a vector of and I

respectively. We then need to decompose the distribution into a time distribution (exponential) and an

binomial indicator. The standard Bayesian technique here would be to construct a generative model for y (i) ,

z (i) , and c(i) based on X (i) . However, the exact motivations behind RSS preference are not always very clear

and often vary from case to case, and so this is not feasible in our case. Hence, it would be ideal to train

a classifier under this model. Naively, we can inject features into our data by creating for each data X (i) a

(i)

large number of yt indicating the number of clicks at each time t. However, this is not entirely feasible, as

we would have to increase an already large dataset by another very large constant factor. Hence, the naive

approach fails here.

However, in general we are viewing y (i) and zi as time series, and attempting to model y and z. By

MLE, given the rate parameter , we can train a regressor on

(

(i)0 1 for y (i) 1,

y = R

t

Exp(t0 , )dt0 for y < 1

Note that here y (i)0 = p(I (i) = 1) given our assumptions. In this case we can determine merely by looking

at all entries X (i) that can no longer be clicked (ie are no longer on the RSS feed) and their respective c(i) .

We can then simply determine using any number of analytical methods.

With the knowledge of this distribution in mind we can then adapt traditional text categorization

algorithms to use the soft labels y 0 .

Our dataset consists of 112,828 RSS entries from a total of 163 unique RSS feeds, of which 2,607 have been

clicked. These were collected over a period of three months and represent a cross section of real-life RSS

usage.

For evaluation purposes we pick a time t and partition the dataset such that we train on Xi where

zi,0 < t and test on the remainder. In all the cases we use a modified vector space model where the first

dimension of our feature space for X (i) is f (i) , and each following dimension corresponds to a word in the

corpus. For all tests here we train on the first 500 entries chronologically and then attempt to predict future

clicks on entries beyond those 500.

A naive Bayes classifier is a special case of Bayesian network applied to the task of classification. Naive

Bayes assumes that every feature X (i) is conditionally independent from every other feature given y, or,

formally,

(i)

Y

P(X (i) = X|y 0 = c) = P(Xi = Xi |y 0 = c)

i

Q

(i) 0

h(X) = P(I = 1|X) = P(y = 1|X) = Q

i P(Xi = xi |C = ck )

In our case we are attempting to classify based on soft labels y 0 rather than discrete labels. To account

for this when training, for each entry X (i) we consider it instead to be y (i)0 entries with y (i)0 = 1 and 1 y (i)0

entries with y (i)0 = 0.

Logistic regression is a conditional probability model in which we assume that the posterior is logistic, ie

that

1

p(I (i) |, xi ) =

1 + exp(T x)

Here we can generate by minimizing the negative log-likelihood

X

l() = log(1 + exp(T xy 0 ))

i

across our dataset. Note in this case we use the calculated y 0 rather than y. We l1-regularize our dataset

by using the lasso algorithm, which is known to have good performance with text categorization[2], hence

l()lasso = l() + ||

where we use Mallows Cp as . We can then easily solve for the maximizing value of using this

function with gradient descent.

For the purposes of generating decision trees we use Quinlans C4.5 algorithm [3]. Again, to account for the

use of soft labels when maximizing information gain, for each entry X (i) we consider it instead to be y (i)0

entries with y (i)0 = 1 and 1 y (i)0 entries with y (i)0 = 0.

We create random forests as described by Breiman et al in [4]. In the random forests technique, we

construct each individual decision tree as above, bootstrapping the data in each case, to create the complete

ensemble.

The case of SVMs is a unique one since SVMs cannot consider soft labels. Hence we train the SVM only

by using those entries in X that have been removed from user access and hence can no longer be clicked,

using a Gaussian kernel with Automatic Relevance Determination (ARD). Apart from this pre-processing

the training is a reasonably straightforward application of Platts SMO algorithm on these points [5].

We run each of these algorithms on the data set as described. The performance of each algorithm can be

seen in Table 1.

Perhaps one of the most interesting results here is the unusually good performance of naive Bayes.

Support vector machines and random forests have been shown to significantly outperform naive Bayes in

[6]. However, here we see that naive Bayes performs only slightly worse than random forests and in fact

outperforms the SVM.

We suspect this performance gap is caused by most of the predictive power of our features being in the

feed ID f (i) rather than in the actual bag-of-words. This is intuitively sensible, as feeds, like human interests,

tend to be restricted to a few topics, and an interesting feed will tend to be clicked significantly more often.

This is also an explanation of the poor performance of logistic regression: logistic regression cannot account

for the feed IDs properly as it can only separate in the feature space, in which a meaningful distinction and

ordering does not exist for the feeds - hence, it is forced to operate entirely on the textual information given.

SVMs suffer a similar plight. The remaining algorithms that can consider a feature independent of any given

space fare much better.

The strong predictive capability of the feed ID intuitively makes sense. Most RSS feeds tend to be

focused around one or two topics, as do most user preferences. Hence, if a user clicks on entries from a given

feed it is quite likely that he is interested in the topics that feed provides. One can think of the feed ID as

a latent variable linked directly to the topic of the feed.

We also note a strong mutability in preferences. Figure 1 shows the ability of three of the classifiers

trained on data, and then tested on increasingly more recent data. We note an almost exponential degra-

dation of prediction quality as we increase our prediction horizon. Hence, in order to properly predict clicks

we need a dynamic model, or at least one that is frequently re-trained.

Prediction degradation over time

1.0

Random Forests

Decision Tree

SVM

0.8

0.6

clicks predicted

0.4

0.2

0.0

prediction horizon

Algorithm RMSE

Naive Bayes 0.79

Regularized Logistic Regression 0.92

Decision Trees 0.86

Random Forests 0.76

SVM 0.82

5 Further Directions

As it seems the most performant feature we observed was the feed ID, which seems to derive its predictive

power from being directly linked to topic, the next logical step seems to be topic models. We plan on

exploring Latent Dirichlet Allocation and the hierarchical version thereof in order to create topic models

with RSS feeds, which will allow us to classify with potentially greater accuracy than before. Other possible

approaches include collaborative filtering; however, these require a volume of data that we, at this moment,

do not have access to.

6 References

1. Sahami, M., Dumais, S., Heckerman, D. and Horvitz, E. (1998) A Bayesian Approach to Filtering Junk

E-mail. AAAI Workshop on Learning for Text Categorization.

2. Genkin, A., Lewis, D. and Madigan, D. (2004) Sparse Logistic Regression for Text Categorization.

3. Quinlan, J. (1993) C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers.

4. Breiman, L. (2001). Random Forests. Machine Learning.

5. Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization.

Technical Report MSR-TR-98-14.

6. Caruana, R. and Miculescu-Mizil, A. (2006) Proceedings of the 23rd international conference on Ma-

chine learning

- A Comparative Analysis of Opinion Mining and Sentiment Classification in Non-English LanguagesUploaded bynayeem074413
- Development of an Expert System for Diagnosis and appropriate Medical Prescription of Heart Disease Using SVM and RBFUploaded byijcsis
- cec2007okUploaded byBanon Tri Kuncahyo Hariadi
- Ho 3114511454Uploaded byAnonymous 7VPPkWS8O
- ObjectNess BingUploaded byAntonio Cangiano
- Ya Seen 2015Uploaded byAntonio Soto Leon
- IR - Unsupervised-chi SquareUploaded byResa Septiari
- 54Uploaded byNajmul Hasan
- A COMPARATIVE STUDY ON HUMAN ACTION RECOGNITION USING MULTIPLE SKELETAL FEATURES AND MULTICLASS SUPPORT VECTOR MACHINEUploaded bymlaij
- Survey NewsletterUploaded bygabriel.ridgeway
- smarter science step 1 - peoe templateUploaded byapi-246864303
- Report 1Uploaded byNiravSinghDabhi
- clef2014wn-qa-papanikolaouet2014Uploaded byapi-278914751
- d 1428042413Uploaded bySeminarsPunters
- thomassarah 5Uploaded byapi-269396544
- Diff CriteriaUploaded byNabeel Asim
- 016A REVIEW ON SENTIMENT ANALYSIS OF SOCIAL MEDIA DATA USING TEXT MINING AND MACHINE LEARNINGUploaded byIJAR Journal
- Lecture 3 - Linear Methods for ClassificationUploaded bymissinu
- FORECASTING1.pdfUploaded byNischalGNischal
- Data Science Scenarios and Solution TemplatesUploaded bydm2003
- decay labUploaded byapi-309621828
- 00031___805ec346c35605b43c3ddc82923cbf06Uploaded byEmmanuel Galicia
- A Novel Feature Extraction Technique of Electronic Nose for Detecting of Wound Infection Based on Phase SpaceUploaded byEditor IJRITCC
- A hybrid gene selection approach formicroarray data classification using cellular learning automata and ant colony optimizationUploaded byhassan
- nihms-522647Uploaded byMahendra Singh Meena
- A contour-color-action approach to automatic retrieval of several common video genres.pdfUploaded byAnonymous ntIcRwM55R
- 5765217565532733091.pdfUploaded byNicholas Logan
- Stark Assessment of Lifestyle Based Human Disorders Using Data Mining Based Learning TechniquesUploaded byDr Manik Sharma
- Dnicholson Rparanjpe Finalpaper References CorrectedUploaded byAbhay Kukade
- Design of Traffic Sign DetectionUploaded bySunil Raghu Veer Paduchuri

- Why Most Unit Testing is WasteUploaded byGuruuswa
- Games and ReputationUploaded byroberto86
- 04 Benjamin a. Engel Viewing Seoul From Saigon1Uploaded byroberto86
- Ej 982678Uploaded byLalitha Pramaswaran
- 1802.01315Uploaded byroberto86
- 0153-01-0002-0002-7Uploaded byroberto86
- Keyphrase Extraction Using Word EmbeddingUploaded byroberto86

- Choral Reading of PoetryUploaded byfordal
- Aluminium Sheet Grade 65032 SuppliersUploaded bysanghvi overseas inc
- SCA GO_MS89Uploaded bySoitda Bcm
- Gen Chem II Exam 4 Titration, Ksp Practice Problems f08Uploaded byDiego Marcelo Aragon Caqueo
- resume updated 10-24-14Uploaded byapi-296057161
- AnnUploaded bySebastin Ashok
- Article ToolsUploaded byPei Sing Ang
- Altis MBA GBS Application Letter Reference SiteUploaded bypartha_biswas_uiu
- Writing a Dissertation-ResourcesUploaded bytahereh1363
- Buffalo Public Schools updated organizational chartUploaded bySandra Tan
- observation form payton dohertyUploaded byapi-315002438
- Provisional Allotment List 2Uploaded byBar & Bench
- EAS 2Uploaded byaamicho
- nursing theory essay 2 weeblyUploaded byapi-305752422
- Educating Children at HomeUploaded byEcha Dian Anggraini
- GO.no.54 Sanction of 291 Warden and 23 District Coordinator Posts Along With Supporting Staff in APSWR InstitutionsUploaded byrameshpunna_117386
- Media Flash CardUploaded bySayyidah Balqies
- Problem Set 7 (Key)Uploaded bywheeler8921
- Resume,_Matthew_ShawUploaded byInternMatt
- BG Chapter 5Uploaded bygaurnityananda
- bbm-3A978-3-642-40255-5-2F1Uploaded bySibasish Sahu
- RUJUKANUploaded byFaiz Jibam
- Career Planning & DevelopmentUploaded byAmit Sharma
- SHS Rubric AP Lang Rhetorical Analysis 11 23 2016Uploaded byKorey Bradley
- The Transformation Curve Digital - Learning and DevelopmentUploaded bydee
- 3 Emotional Intelligence Exercises 1Uploaded byamigirl16
- Color and Personality Strongs Interest InventoryUploaded byLouri Minora
- Study Guide 1ºs AnosUploaded byPatríciaMiranda
- mckeown eng 1di course outlineUploaded byapi-262936479
- MemLaw-BankFraudUploaded bySalaam Bey®

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.