KEY PERFORMANCE INDICATORS FOR MARKET INTELLIGENCE FROM SOCIAL MEDIA USING OPINION MINING AND SENTIMENT ANALYSIS

TECHNIQUES

ABSTRACT Opinion mining or Sentiment Analysis is a specific application of text mining that has gained interest in the recent years. It is an important technique that can be used to leverage the intensity or emotional polarity of opinions expressed by users on social media. Research articles on the applications of opinion mining and sentiment analysis techniques in social media have been reviewed, in conjunction with market intelligence and metrics. A conceptual framework has been developed for obtaining Key Performance Indicators (KPI) for extracting market intelligence from social media using the main aspects of opinion mining and sentiment analysis. The important facets of this technique that can be used for measurement were identified. These were then used in the derivation of KPIs for specific social media objectives. The model was applied for a few scenarios available on the public domain. It is established that using opinion mining and sentiment analysis provide a granular metric in identifying the sentiments of the customers. Keywords: Opinion Mining, Sentiment Analysis, Social Media, Key Performance Indicators, Market Intelligence 1. INTRODUCTION Web 2.0 is a rich environment provided by the internet where users share their experiences through either textual or multimedia content. Web 2.0 holds valuable information for companies and thereby enables a new kind of market research (Bodendorf and Kaiser 2010). The voluminous user generated content on the Internet has therefore made this a perfect medium for

collaborative communication between marketing professionals and consumers. This is due to the wealth of consumer opinions available in the web as textual data, either as reviews, messages in social networks, blogs, microblogs, consumer forums. User Generated Content is fundamentally altering how audiences interact with the Internet, and how advertisers reach those audiences. In 2007 user generated content sites generated $1 billion in advertising revenue and by 2011 these sites are projected to attract 101 million users in the U.S. and earn $4.3 billion in ad revenue (Verna 2007). 2. SOCIAL MEDIA A RECENT ELEMENT IN THE MARKETING MIX Social Media is an integral part of Web 2.0. Kaplan and Haenlein (2009) define social media as a group of Internet based applications that build on the ideological and technological foundations of Web 2.0 and allow the creation and exchange of user generated content. The growth of social media sites in the recent years has triggered a big difference in the way companies and brands interact with their customers. In a traditional sense it enables companies to talk to their customers, while in a non-traditional sense it enables customers to talk directly to one another (Mangold and Faulds 2009). Apart from acting as a non traditional communicational channel social media is perceived by consumers as a more trustworthy source of information regarding products and services than corporate-sponsored communications transmitted via the traditional elements of the promotion mix (Mangold and Faulds 2009). A McKinsey report (Bughin et al. 2007) identifies that word of mouth communication is the primary factor behind 20 to 50 percent of all purchasing decisions. The motivation for mining opinions is extremely significant for the organisation when this word of mouth communication has a significant impact on sales (Davis and Khazanchi 2008) and the bottom line of the organisation.

It is therefore of great importance to organisations to sift through the unstructured textual content on social media and capture pertinent and authentic opinions on social media. Online conversation sources provide business with timely and authentic opinions and behaviours of current customers (Berkman 2008). It is also necessary for marketing professionals to manage negative online consumer reviews (Lee et al. 2008) to avoid adverse impact on the brand and product. Thus marketers can utilize social media in two main ways - as passive
marketing tools i.e. as sources of market intelligence and as active marketing tools as platforms of communication/promotion, customer interaction and customer feedback (Constantinides 2009).

The field of opinion mining and sentiment analysis is well-suited to various types of intelligence applications and would therefore be useful for handling many business-intelligence applications that need to extract opinions from unstructured human-authored documents (Pang and Lee 2008). It is a new discipline that uses techniques from Natural Language Processing, Machine Learning and Data Mining to automatically extract opinions or sentiments from text. Hence it is established that opinion mining and sentiment analysis can be used effectively in social media to extract market intelligence. Opinion mining and Sentiment Analysis focuses on classifying sentiment expressed in text- for e.g. a review, blog, microblog, comment expressing views on a particular topic. It can be classified as positive or negative. Sentiment analysis is also referred to as emotional polarity computation (Li and Wu 2010) as it detects the polarity of emotions and classifies them into positive and negative categories. This paper aims to review research articles on the applications of opinion mining and sentiment analysis techniques in social media. It also attempts to illustrate the suitability of opinion mining and sentiment analysis techniques as ideal tools to listen on social media and

specifically describes the kinds of business objectives that can be mapped to this technique. A conceptual model for extracting market intelligence from social media using opinion analysis and sentiment analysis has been identified. Key Performance Indicators are then derived for measuring the market intelligence using opinion analysis and sentiment analysis. 3. LITERATURE REVIEW There has been extensive work in Opinion mining and Sentiment Analysis since the year 2001 (Pang and Lee 2008). The literature review on opinion mining and sentiment analysis has been classified based on techniques and relationships with market intelligence and metrics. 3.1 Classification Techniques used in Opinion Mining and Sentiment Analysis Sentiment analysis either uses a domain dependent lexicon or machine learning technique to classify sentiment in text. The evolution of these techniques used for determining sentiment and opinion and the accuracy related to them is presented. Turney (2002) has classified sentiment of movie reviews based on the semantic orientation of the identified phrase using mutual information. The unsupervised classification algorithm developed by Turney (2002) achieves an average accuracy of 74% when evaluated on 410 reviews from Epinions (www.epinions.com), sampled from four different domains (reviews of automobiles, banks, movies, and travel destinations). This method can be suitably applied to summarisation of reviews and other automated opinion tracking systems. The accuracy of the technique is a limitation as it provides varying accuracy for different domains. Pang, Lee and Vaithyanathan (2002) show that sentiment classification through machine learning techniques perform comparatively better than human generated baselines and they conclude that the SVM (Support Vector Machine) is the most suitable method for sentiment classification method as it provides an accuracy of 81%. These machine learning methods and features were not based on

specific domain and hence can be extended to other domains unlike the PMI-IR (Pointwise Mutual Information (PMI) and Information Retrieval (IR)) method proposed by Turney (2002). However it needs sufficient training data to provide the required accuracy in classification. In addition to machine learning and information retrieval, researchers have progressed to add other techniques such as natural language processing (NLP), sentiment lexicon, to improve the classification accuracy. Dave, Lawrence and Pennock (2003) have developed a classifier that uses information retrieval techniques for feature extraction and scoring. This study has used machine learning, NLP and Information Retrieval techniques to analyse sentiments. Nasukawa and Yi (2003) illustrate sentiment analysis approach to extract sentiments associated with polarities of positive or negative for specific subjects from a document, instead of classifying the whole document into positive or negative. The system has high precision with limited number of domains and simple text data. In scenarios where domains were expanded to include documents that consisted of complex sentences the precision decreased from 95% to 75%. The system also includes an additional task of manual development of domain dependent sentiment lexicons and there is a necessity for maintaining it over a period of time in a systematic manner. In a similar type of analysis, Yi et al (2003) present a sentiment analysis system that detects all references to the given subject, and determines sentiment in each of the references using natural language processing (NLP) techniques. This system uses a sentiment lexicon. It demonstrated high quality results of 87% for review articles and 86 91% precision and 91 93% accuracy was obtained for the general web pages and news articles respectively. Pang and Lee (2004) have proposed a novel machine-learning method that applies text-categorization techniques. Subjective portions of the document i.e. around 60% of the reviews words were extracted and analysed. This technique maintains a good level of performance for the polarity

classification task. However the technique needs a highly processed text input for sentiment classification as it uses cut-based classification a graph based formulation to avoid feature generation. Mullen and Collier (2004) demonstrate that hybrid SVMs namely unigram-style feature-based SVMs obtain superior performance than those based on real-valued favourability measures. They conclude that using topic based information in conjunction with features would greatly improve the performance of the classification systems. Wilson, Wiebe and Hwa (2004) have identified opinions at the sentence level using identification of clauses. The most important contribution of this work is the classification based on the strength of the opinions expressed. They have ordered opinions into four categories namely: neutral, low, medium, and high. This is the first study that focuses on classifying the strength of emotions in clauses of every sentence in the corpus. This assignment of strength provides a finer level of detail to the detected sentiments. This study has achieved improvements in mean-squared error over baseline ranging from 57% to 64% using support vector regression. A system for phrase level polarity detection has also been proposed by Wilson, Wiebe and Hoffman (2005) that automatically identifies the contextual polarity for a large subset of sentiment expressions. This research highlights the need for identifying contextual polarity in sentiment detection. The most important contribution of this work is the classification based on the strength of the opinions expressed using phrase level and sentence level corpora. Determination of subjective terms in text is seen to be essential in classifying the sentiment of the opinion. The definition of most of these subjective terms is present in online dictionaries (gloss). Esuli and Sebastiani (2005) propose a new semi-supervised method for determining the orientation of subjective terms. The method is based on the quantitative analysis of the glosses of such terms i.e. the definitions that these terms from dictionaries and on the use

of the resulting term representations for term classification. This method significantly outperforms other known methods. SentiWordNet developed by Esuli and Sebastiani (2006) is a publicly available lexical resource that helps in determining sentiment from glosses. SentiWordNet is based on the quantitative analysis of the glosses associated to synsets. This is a very useful tool for opinion mining applications because of its comprehensive nature. Denecke (2009) has concluded that machine learning approaches, are more suitable for cross domain analysis and rule based approaches with fixed opinion lexica, are more suitable for single domain analysis. A lexicon enhanced method for sentiment classification that combines machine learning and semantic-orientation approaches into one framework significantly improves sentiment classification performance (Dang, Zhang and Chen 2009). In case a combination framework of lexicon and machine learning methods is selected, building a domain specific lexicon manually is a time consuming task. Towards this objective there has been significant research in the direction of automating this task. Kanayama and Nasukawa (2006) have proposed an unsupervised lexicon building method that can be built automatically for different domains. Their results indicate that the precision of polarity assignment with the automatically acquired lexicon was 94% on average, and the method is robust for corpora in diverse domains. An unsupervised method to induce a context-aware sentiment lexicon by utilizing the semi-structuredness of user-generated product reviews was proposed by Bross and Ehrig (2010). This method outperforms SentiWordNet as a baseline and is highly suitable for a review mining system. These methods can be considered to reduce the effort spent in manual definition of lexicons. It can therefore be inferred that a combination of lexicon and learning approach is suitable for detecting sentiments with accuracy in text. In case the domain to be analysed is fixed

then a lexicon would be appropriate for the analysis of sentiment. A number of automated approaches can also be used to compress the time spent in developing lexicons. SentiWordNet can be employed as a first step before the development of customised lexicons. Most of the studies based on lexicon and machine learning methods have used review based data as the underlying corpus. Research is required in this area to understand if these techniques can be extended to provide the same accuracy with other social media data and user generic content. Further studies on a framework based on combinatorial methods need to be explored for social media data. 3.2 Opinion Mining and Sentiment Analysis: Market Intelligence and Metrics Market intelligence is a special case of competitive intelligence and is the information relevant to a companys markets, gathered and analyzed specifically for the purpose of accurate and confident decision-making in determining market opportunity, market penetration strategy and market development metrics (Baumgartner, Gottlob and Herzog 2009). Morinaga et al. (2002) have proposed a framework for mining product reputations on the web using opinion mining. The comparison is shown in a positioning map which shows the clusters of competitors grouped based on reputation. Glance et al. (2005) have developed an exemplary framework that extracts market intelligence from the web. Sentiment analysis computation is an important component of this system. A system that gathers specific types of online content and delivers analytics based on classification, NLP, phrase finding and other mining technologies in a marketing intelligence application has been described with the aid of a case study. Although market intelligence studies have been conducted very few studies have focussed on the development of metrics. Glance et al. (2005) have provided an interesting set of

metrics for organisations to gauge the generated buzz, polarity of the comments, author dispersion (a measure how spread out the discussion of a topic is) and board dispersion (to measure viral issues and their polarity). Another proposal for a metric relating to the detection of topic and polarity has been used as an aggregate measure of authorial sentiment on a particular topic drawn from online messages (Nigam and Hurst 2004). This metric has been designed to capture real world market research and provide an aggregate score of opinions towards a topic. This is an interesting application of sentiment analysis technique as a market intelligence metric that uses the true frequency of positive and negative expressions towards the topic. A review of research done using Opinion mining and sentiment analysis over the past two years consisting of over hundred papers revealed that an increasing number of studies focus on blogs, microblogs, forums, news articles and bulletin based systems. Extensive research is presently being undertaken in product review data on the social media. However the review highlights there is a dire need to define precise metrics in gathering market intelligence from social media since there is scarce research work done in this area. 4.0 CONCEPTUAL MODEL Market Intelligence is defined as information obtained from external sources that can be used for identifying problems, changes and opportunities in the external marketing environment (Talvinen 1994). If a company utilizes marketing intelligence systems, the output can result in sound marketing decisions which can be one of the best sources of competitive advantage (Lackman, Saban and Lanasa 2000). Wood (2001) identifies pertinent information regarding the usage of the internet for market intelligence. Swedowsky (2009) has identified that companies across all industries can listen to social media to gain insight into key areas of business.

It is also clear from the extensive review of literature that opinion mining and sentiment analysis techniques is suitable for gaining strategic market intelligence from social media data by aiding in measuring consumer and prospective consumer opinions. It is also found that very little work has been carried out in either the development of indicators for measuring social media. Figure 1 presents a conceptual model on using opinion mining and sentiment analysis techniques in deriving market intelligence from social media.

Figure 1 Conceptual Model Opinion holders denote the users of social media who leave opinions or sentiments towards various objects including products and services in various sites on social media. These opinions and sentiments are summarised using opinion mining and sentiment analysis techniques and this summary is further used to gain market intelligence. The first stage Stage1 depicts the creation of opinions and sentiments on social media. Opinion holders leave a massive amount of opinion data on social media in an unstructured format. Opinion mining and sentiment analysis can be applied in the second stage (Stage2) to summarise the opinion data in a structured format that can be used by business to capture market intelligence. The third stage is the development of Key Process Indicators to measure and extract meaningful market intelligence data from social media. The following sections discuss the aspects of opinion mining and sentiment analysis suitable for the development of Key Performance Indicators for market intelligence.

5.0

KEY PERFORMANCE INDICATORS Key Performance Indicators (KPI) help organizations achieve organizational goals

through the definition and measurement of progress. The KPIs selected must reflect the organization's goals, they must be key to its success, and they must be measurable (Burby and Atchison 2007). Recent research during the period 2008-10 reveals that the following facets of opinion mining and sentiment analysis can be used to measure social media to gain market intelligence from social media. Table 1 Facets of Opinion Mining and Sentiment Analysis suitable for social media measurements These aspects from Table 1 have been used to derive Key Performance Indicators (KPIs) for measuring opinions and sentiments in social media to gain market intelligence. When brands and customers turn towards social media for increasing communication it is imperative that companies define and measure KPI for this channel. It is also essential that business goals are defined unambiguously and clearly before setting out to develop KPIs. If the objective of any business is to utilise social media as an external source for gathering market intelligence then suitable business goals for using this media should be defined at the outset before the definition of KPIs. This implies that listening to social media for the customers voice without predefined goals would not obtain meaningful metrics. Hence all social media business goals need to be entirely defined before the development of appropriate KPIs. The framework proposes Key Performance Indicators that are aligned to four precise social media business objectives (Owyang and Lovett 2010) namely Foster Dialog, Promote Advocacy, Facilitate Support and Spur Innovation. Opinion Mining and Sentiment Analysis can

be used to refine the KPIs to accurately assess and determine market intelligence from social media which is discussed in following sections. 5.1 Social Business Objective: Foster Dialog In the age of disruptive technologies and the collaborating consumer it is necessary to foster dialog on social media. Empowerment makes the consumer voice louder: many social media web sites have become forums of dialog, criticism and confrontation of commercial organizations or governments with their social, ethical and commercial responsibilities (Constantinides 2009). Dialog with the consumer is therefore indispensable today and is required to build awareness and word of mouth activity, obtain feedback and represent the brand online to gain competitive advantage in social media. Opinion Mining and Sentiment Analysis contributes to measuring the dialog by detecting the polarity and intensity of the opinion. Foster Dialog Key Performance Indicator #1: Share Of Voice Share of Voice is a marketing intelligence metric that is the relative percentage of brand mentions in social channels (i.e., articles, blogs, comments, Tweets, videos, etc.) among a competitive set (Owyang and Lovett 2010). Equation 1 represents this indicator.

Share of Voice =

Brand Mentions Total Brand Mentions ( Brand +Competitor A, B, C..n) . Equation 1

This measure can be used to gauge the interest levels and capture the buzz around the companys ad campaign or Youtube video. Similarly buzz around a competitors campaign can be monitored and corrective action can be taken and this can serve as a learning experience for future campaigns. Opinion mining and sentiment analysis can be used to redefine the metric as follows:

Positive Share of Voice = Positive Brand Mentions

Total Positive Brand Mentions ( Brand +Competitor A, B, C ..n)

. Equation 1
Negative Share of Voice = Negative Brand Mentions
Total Negative Brand Mentions ( Brand +Competitor A, B, C ..n)

. Equation 2

This metric can now be used to understand the underlying tone of the buzz. Equations 2 and 3 include sentiment as an indicator for detecting the type of share of voice. It provides a more granular measure of the opinions expressed on the channel. It would be an oversight to measure only the share of voice as there might be cases where the buzz seems loud but the negative mentions exceed the positive mentions indicating a negative buzz. Organizations can therefore use sentiment analysis as a true indicator to set thresholds for declining positive share of voice scenarios and increasing negative buzz scenarios. Figures 2 and 3 illustrate this scenario. The figures are screenshots of the result of performing a social media search for two competing beverage brands. Social Mention (http://www.socialmention.com)a social media search and analysis platform that aggregates user generated content from across the universe into a single stream of information and provides sentiment scores is the tool used for search. This site posts an aggregate sentiment score of the user related posts and this can be used to calculate the positive and negative Share of Voice.

Figure 2 Social Mention Result for Beverage Brand1

Figure 3 Social Mention Result for Beverage Brand2 The social mentions for two leading beverages are obtained. Based on the indicated sentiment from Figures 2 & 3 the share of voice is calculated using Equations 1 (Table 2a) and using Equation 2&3 (Table 2b).
Table 2a Share of Voice

Table 2b Share of Voice using Opinion Mining and Sentiment Analysis

Beverage Brand2 seems to enjoy a good of share of voice when compared with Brand1 (0.504 vs 0.496). It can be seen from Table 2b that Brand1 also enjoys a positive share of voice

(0.63).However the negative buzz for this brand (0.58) exceeds Brand1. This illustrates that even though the share of voice metric looks good (0.5) for Brand2, sentiment analysis adds an important dimension by indicating the negative share of voice is high comparatively. This metric will help the brand identify and address the issues pertaining to negativity. Another application is that in case of an increasing negative buzz around a brand, organisations can also analyse if it is the result of a maligning spam campaign using sentiment analysis techniques. As identified in Table 1 sentiment analysis can be used to identify spam. This technique can be extended to social media channels that have user generated content similar

to review text i.e. short text. Youtube comments, Blog comments, social network comments can be reviewed for suspicious behaviour. This methodology can be used by organisations to detect unwarranted campaigns and identify spam immediately. Equation 4 suggests a means for detecting actual negative share of voice.
Actual Negative Share of Voice =

Negative Brand Mentions Spam Comments Total Negative Brand Mentions ( Brand +Competitor A, B, C ..n)

. Equation 4

Foster Dialog Key Performance Indicator #2: Audience Engagement Audience Engagement is the proportion of visitors who participate in a specific marketing initiative by contributing comments, sharing or linking back (Owyang and Lovett 2010) and is measured using Equation 5.
Comments + Shares +Trackbacks Total Views

Audience Engagement =

. Equation 5

Based on the number of audience engaged organizations can understand and modify expectations based on advertising, search efforts and promotional activity accordingly. Using sentiment analysis the audience engagement can be determined as follows:
PositiveComments +Shares +Trackbacks Total Views

Positive Audience Engagement =

. Equation 6
Negative Audience Engagement =
NegativeComments +Shares +Trackbacks Total Views

All Positive Comments where Positve Comments =

. Equation 7

Negative Comments =

All Negative Comments

Equations 6 and 7 redefine the comment value to reflect the true measure of the comment value since negative comments undermine the object under scrutiny. This measure now can be used to detect if the engaged audience admire or loathe the brand. Li and Wu (2010) have developed an algorithm for identifying detecting hotspots in a forum or community discussion within a specific time span and have developed predictive methods for hotspots in a forum. This can be utilised to cluster discussion based on topics that are of great interest to the audience. Topicwise Audience Engagement with Sentiment Analysis (Equation 8) is a broad level KPI that seeks to understand hot issues that are discussed most in a conversation.
Topicwise Audience Engagement = Clusters based on Hot Spot Detected in specified time frame

. Equation 8

Measuring audience engagement also facilitates appropriate context based advertising. In case the audience are negatively engaged with a particular brand, dissatisfaction-oriented advertising based on sentiment analysis (Qiu et al. 2010) can be employed to advertise competing brands. Equation 9 is used to calculate the degree of intensity in the nature of engagement of the audience. Certain negative sentiments can be ignored if they are weak in nature or they can be addressed appropriately in case the intensity is of a stronger nature.
Audience Engagement with Intensity =
Degree Of Intensity towards Topic Total Intensity Score of Comments

. Equation 9

Foster Dialog Key Performance Indicator #3: Conversation Reach Conversation Reach (Equation 10) is the measure of the number of unique visitors who participate in a specific brand/issue/topic conversation across one or more social media channels (Owyang and Lovett 2010).

Conversation Re ach =

Total People Participating Total Audience Exposure

. Equation 10

An important characteristic of this measure is that it identifies the unique number of people who participate in the conversation. Owyang and Lovett (2010) explain that the use of this measure is superior when the scope of the conversation is restricted by associating with a specific marketing initiative, a topic and/or keywords. The impact of a campaign can be addressed using this measure. The modified KPI when opinion mining and sentiment analysis is used shall be:

Positive Conversation Reach =

Total People Participating Total Positive Audience Exposure

. Equation 11
Negative Conversation Reach =

Total People Participating Total NegativeAudience Exposure

. Equation 12
Audience with Positive Feedback where Total PositiveAudience Exposure =

Total NegativeAudience Exposure =

Audience with Negative Feeback

Equations 11 and 12 are used to measure positive and negative conversation reach. This would ensure that this metric would expose the nature of reach that is being exhibited for the brand / idea / topic under consideration. The Conversation Reach metric can be extended to capture the influencers who take the initiative in a positive direction with the use of sentiment analysis. Such influencers can be rewarded and thus their engagement with the brand can be reinforced. 5.2 Social Business Objective: Promote Advocacy

The business objective of consumer advocacy enlists the support and dedication of individuals that are ambassadors for certain products, brands or organizations, despite having no official connection (Owyang and Lovett 2010). This objective is focussed on developing

relationships with individuals who have an affinity towards the brand and encourage word of mouth behaviour through these individuals. These individuals can also be identified as segments and can be targeted for preferential campaigns and perks. Marketers should establish ties
and working relationships with leading blogs or users forums so that they review, discuss, comment on or even recommend the usage of the firms products and successfully engage social media personalities (Constantine 2009).

Promote Advocacy Key Performance Indicator #1: Percent Active Advocates

# Of Active Advocates( w / in Total Advocates past 30 days)

Active Advocates =

. Equation 13

The KPI in Equation 13 (Owyang and Lovett 2010) uses sentiment analysis by default to determine advocates. However this KPI can be further extended to include the tone of the advocates and develop a degree for advocacy. Such a measure is included in the Advocate Influence KPI (Equation 14). Promote Advocacy Key Performance Indicator #2: Advocate Influence This metric is used to derive the degree of influence wielded by unique influencers. The metric proposed by Owyang and Lovett (2010) is
Unique Advocate ' s Influence Total Advocate Influence

Advocate Influence =

. Equation 14 where Influence = Volume of relevant content, comments, share and reach

The authors suggest that this metric be used identify new individuals for the organisations advocacy program, incentivize participants or penalize/ motivate others. The Equation 14 is modified to include opinion mining and sentiment analysis.
Degree of Influence of Unique Advocate = Re lative Positive Score of Comments

. Equation 15

Emotional intensity of opinions can be determined by the approach used by Carrillo de Albornoz, Plaza and Gervs (2010) as mentioned earlier. This can be applied to rate the degree of influence by including an additional component on total positive sentiment towards the product as shown in Equation 15. Based on the degree of influence we can rate advocates and determine the reward and incentive structure for the advocate based on his influence level. This would ensure that we also nurture a relationship with influential advocates. Organisations can also try to identify and motivate advocates with lower rating by providing sufficient incentives. Promote Advocacy Key Performance Indicator #3: Advocate Impact This metric is associated with defining the impact of individual advocates is imperative in determining overall effectiveness of the business objective (Owyang and Lovett 2010). Equation 16 assesses advocate impact by tracking referral information. Monitoring traffic generated from advocacy programs can provide some kind of correlation measure to determine advocate impact. Advocate Impact
Advocate impact =
Number Of Advocacy Driven Conversations Total Volume Of Advocacy Traffic

. Equation 16

Advocate Impact with Opinion mining and Sentiment Analysis is shown in Equation17.
Advocate impact with Opinion Leader Weights = Leader Weight of Advocate Total Opinion Weights

. Equation 17

Equation 17 is used to find out leader weights. Credibility Evaluation and Leader Weight assignments (Cho et al. 2010) can be used to assess the credibility of the traffic generated by advocates and also assign weights to discern the nature of the opinion authors to determine if they are opinion leaders. 5.3 Social Business Objective: Facilitate Support Individuals can now turn to social media channels to expose their service issues to the world, they either support each other directly, or the company uses the same channels to support customers (Owyang and Lovett 2010). Opinion mining and sentiment analysis will not be of much use here as the objective is more towards solving an existing problem. Foster Dialog KPIs will capture the problem is being discussed. 5.4 Social Business Objective: Spur Innovation This is an objective that is specific to social media. It reflects the usage of the collective intelligence gathered from social media sources to activate innovation from organisations (Owyang and Lovett 2010). It is part of the prosumer culture promoted by Web 2.0 that enables customers play an active role in participating in decision making activities. For example the consumer can suggest design changes to products or services that are accepted by the companies who credit the consumers with the idea. This business objective is used to understand if the organisations listen to the voice of the customer and use it in innovations in a transparent manner acknowledging the customer. Spur Innovation Key Performance Indicator #1: Topic Trends The KPI proposed by Owyang and Lovett (2010). Topic Trends
Topic Trends =
Number Of Specific Topic Mentions All Topic Mentions

. Equation 18

When opinion mining and sentiment analysis are used: Topic Trend with Opinion mining and Sentiment Analysis
Hot Topic Trends =
Number Of Hotspot Topic Mentions All Topic Mentions

. Equation 19

Equation 19 utilises the methodology proposed by Li et al. (2009) to evaluate hotness of blogs based on the number of reviews, comments and sentiment of the comments. This can be used to understand the interest of the social media users towards topics. This measure can now help evaluate features of product or service that captures the interests of users and use them for innovation. Spur Innovation Key Performance Indicator #2 & #3: Sentiment Ratio & Idea Impact These KPIs use sentiment analysis by default. It is therefore apparent that opinion mining and sentiment analysis can be implemented to develop comprehensive KPIs to obtain market intelligence from social media. 6.0 DIRECTIONS FOR FUTURE RESEARCH

Opinion mining and Sentiment analysis is a key technique for developing meaningful metrics for extracting market intelligence from social media. There is a definite need to develop measures for collating intelligence from Social Media data. Social Media objectives need to be domain specific based on the nature of business and new Key Performance Indicators based on these domains is also an interesting area for future research. It would benefit researchers to adopt opinion mining and sentiment analysis techniques combined with other text mining techniques to the development of comprehensive metrics for social media. Apart from positive and negative sentiments, classification can be extended to include degrees of sentiments with more specific categories like sarcasm. Another application that requires extensive studies is the opinion holder. Much work needs to be done to obtain meaningful information about the opinion holder. Opinion

Mining and Sentiment Analysis techniques in conjunction with Social Network Analysis techniques can play a role in identifying information about the opinion holder. Business can use opinion holder data to conduct profiling and segmentation studies for social media. 7.0 CONCLUSION Opinion mining or sentiment analysis is a specific application of text mining that can be used to understand the opinions of the new age consumer on social media. This technique can be used to capture information and understand customer views and listen to the customer intelligently on the new media. We have proposed a conceptual framework for using this technique to summarise opinions of social media users and gain market intelligence from the new media. Based on the facets of opinion mining and sentiment analysis techniques a set of Key Performance Indicators were obtained to acquire competitive market intelligence from social media. These indicators can be used by organisations to maintain a meaningful dialog with the consumers, reward advocates and successfully innovate along with an active customer on social media. ACKNOWLEDGEMENTS The authors gratefully acknowledge Department of Science and Technology (DST), New Delhi for providing financial support to carry out this research work under PURSE scheme. One of the authors Ms. Angeline G Fernando is thankful to DST, New Delhi for the award of DSTPURSE fellowship. REFERENCES Baumgartner, Robert, Georg Gottlob, and Marcus Herzog 2009, Scalable web data extraction for online market intelligence, Proceedings of the VLDB Endowment, 2, 2, 1512-1523.

Berkman, Robert (2008), The Art of Strategic Listening: Finding Market Intelligence through Blogs and Other Social Media, Paramount Market Publishing, Inc. Bodendorf, Freimut and Carolin Kaiser (2010), Mining Customer Opinions on the Internet - A Case Study in the Automotive, International Workshop on Knowledge Discovery and Data Mining, 24-27. Broder, Andrei, Marcus Fontoura, Vanja Josifovski, and Lance Riedel (2007), A semantic approach to contextual advertising, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '07), 559-566. Bross, Jurgen and Heiko Ehrig (2010), "Generating a Context-Aware Sentiment Lexicon for Aspect-Based Product Review Mining," Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, Vol.1, 435-439. Bughin, Jacques, Jonathan Doogan, and Ole Jrgen Vetvik (2010), A new way to measure word-of-mouth marketing, April 2010, http://www.mckinseyquarterly.com/A_new_way _to_measure_word-of-mouth_marketing_2567, accessed March 10, 2011. Burby, Jason and Shane Atchison(2007), Actionable web analytics: using data to make smart business decisions, Sybex. Carrillo de Albornoz, Jorge, Laura Plaza, and Pablo Gervs (2010), A hybrid approach to emotional sentence polarity and intensity classification, Proceedings of the Fourteenth Conference on Computational Natural Language Learning, 153-161. Constantinides, E.(2009), Social Media / Web 2.0 as Marketing Parameter: An Introduction, 8th International Congress Marketing Trends, Paris, France.

Dang,Yan, Yulei Zhang, and HsinChun Chen (2010), A Lexicon-Enhanced Method for Sentiment Classification: An Experiment on Online Product Reviews, IEEE Intelligent Systems, 25, 4, 46-53. Dave, Kushal, Steve Lawrence, and David M. Pennock (2003), Mining the peanut gallery: opinion extraction and semantic classification of product reviews, Proceedings of the 12th international conference on World Wide Web (WWW '03), 519-528. Davis, Alanah and Deepak Khazanchi (2008), An empirical study of online word of mouth as a predictor for multi-product category e-commerce sales, Electronic Markets, 18, 2, 130141. Denecke, Kerstin (2009), Are SentiWordNet Scores Suited for Multi-Domain Sentiment -Classication?, 4th International Conference on Digital Information Management, ICDIM. Esuli, Andrea and Fabrizio Sebastiani (2005), Determining the semantic orientation of terms through gloss classification, Proceedings of the 14th ACM international conference on Information and knowledge management (CIKM '05), 617-624. Esuli, Andrea and Fabrizio Sebastiani (2006), SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining, In Proceedings of the 5th Conference on Language Resources and Evaluation LREC06 (2006), 417-422. Fan, Teng-Kai and Chia-Hui Chang (2009), Sentiment-Oriented Contextual Advertising, Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval. Glance, Natalie, Matthew Hurst, Kamal Nigam, Matthew Siegler, Robert Stockton, and Takashi Tomokiyo (2005), Deriving marketing intelligence from online discussion, Proceedings

of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, 419-428. Jindal, Nitin, Bing Liu, and Ee-Peng Lim (2010), Finding unusual review patterns using unexpected rules, Proceedings of the 19th ACM international conference on Information and knowledge management. Lee, Jumin, Do-Hyung Park, and Ingoo Han (2008), The effect of negative online consumer reviews on product attitude: An information processing view, Eighth International Conference on Electronic Commerce, 341-352. Kanayama, Hiroshi, and Tetsuya Nasukawa (2006), Fully automatic lexicon expansion for domain-oriented sentiment analysis, Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 355-363. Kaplan, Andreas M. and Michael Haenlein (2010), Users of the world, unite! The challenges and opportunities of Social Media, Business Horizons, 53, 1, 59-68. Lackman, Conway, Kenneth Saban, and John Lanasa (2000) The contribution of market intelligence to tactical and strategic business decisions, Marketing Intelligence & Planning, 18, 1, 6 9. Li, Jianjiang, Xuechun Zhang, Yu Weng, and Changjun Hu (2009), Blog Hotness Evaluation Model Based on Text Opinion Analysis, 2009 Eighth IEEE International Conference on Dependable, Autonomic and Secure Computing, 235-240. Li, Nan, and Desheng Dash Wu (2010), Using text mining and sentiment analysis for online forums hotspot detection and forecast, Decision Support Systems , 48, 2, 354-368. Mangold, Glynn W. and David J. Faulds (2009), Social media: The new hybrid element of the promotion mix, Business Horizons, 52, 4, 357-365.

Morinaga, Satoshi, Kenji Yamanishi, Kenji Tateishi, and Toshikazu Fukushima (2002), Mining product reputations on the Web, Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 341349. Nasukawa, Tetsuya, and Jeonghee Yi (2003), Sentiment analysis: capturing favorability using natural language processing, Proceedings of the 2nd international conference on Knowledge capture, 70-77. Nigam, Kamal and Matthew Hurst (2004), Towards a Robust Metric of Opinion, Proceedings of the AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications. Nguyen Thin, Dinh Phung, Brett Adams, Truyen Tran, and Svetha Venkatesh (2010), Hypercommunity detection in the blogosphere, Proceedings of second ACM SIGMM workshop on Social media, 21-26. Owyang, Jeremiah and John Lovett (2010), Social Marketing Analytics:A New Framework for Measuring Results in Social Media, April 2010, http://www.web-strategist.com/ blog/2010/04/22/altimeter-report-social-marketing-analytics-with-web-analyticsdemystified/, accessed March 10, 2011. Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan(2002), Thumbs up?: sentiment classification using machine learning techniques, Proceedings of the ACL-02 conference on Empirical methods in natural language, 79-86. Pang, Bo and Lillian Lee (2004), A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, 271-278.

Pang, Bo and Lillian Lee (2008), Opinion Mining and Sentiment Analysis, Foundations and Trends in Information Retrieval, 2, 12, 1-135. Qiu, Guang, Xiaofei He, Feng Zhang, Yuan Shi, Jiajun Bu, and Chun Chen (2010), DASA: Dissatisfaction-oriented Advertising based on Sentiment Analysis, Expert Systems with Applications, 37, 9, 6182-6191. Seki, Yohei, Noriko Kando, and Masaki Aono (2009), Multilingual opinion holder identification using author and authority viewpoints, Information Processing & Management, 45, 2, 189-199. Soo Cho, Kyung, Joon-Suk Ryu, Jae-Ho Jeong, Young-Hee Kim, and Ung-Mo Kim (2010), Credibility Evaluation and Results with Leader-Weight in Opinion Mining, International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, 5-8. Swedowsky, M. (2009), A Social Media How To for Retailer, http://blog.nielsen.com/ nielsenwire/online_mobile/a-social-media-how-to-for-retailers, accessed March 10, 2011 Talvinen, Jari M. (1994), Information systems in marketing, European Journal of Marketing, 29, 1, 8-26. Turney, Peter (2002), Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews, Proceedings of ACL-02, 40th Annual Meeting of the Association for Computational Linguistics, 417-424. Verna, Paul (2007), User-Generated Content: Will Web 2.0 Pay Its Way?, eMarketer, June 2007, http://www.researchandmarkets.com/reports/497613/user_generated_content_will_ web_2_0_pay_its.pdf accessed March 10, 2011.

Wilson, Theresa, Janyce Wiebe, and Paul Hoffmann (2005), Recognizing contextual polarity in phrase-level sentiment analysis, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, 347-354. Wilson, Theresa, Janyce Wiebe, and Rebecca Hwa (2004), Just how mad are you? finding strong and weak opinion clauses, Proceedings of the 19th national conference on Artifical intelligence, 761-767. Wood, Emma (2001), Marketing Information Systems in Tourism and Hospitality Small- and -Medium-sized Enterprises: a Study of Internet use for Market Intelligence, International Journal of Tourism Research, 3, 4, 283-299. Xu, Kaiquan, Stephen Shaoyi Liao, Jiexun Li, and Yuxia Song (2011), Mining comparative opinions from customer reviews for Competitive Intelligence, Decision Support Systems, 50, 4, 743-754. Yi, Jeonghee, Tetsuya Nasukawa, Razvan Bunescu, and Wayne Niblack (2003), Sentiment Analyzer: Extracting Sentiments about a Given Topic using Natural Language Processing Techniques, Third IEEE International Conference on Data Mining (ICDM'03).