Analysis of Pulse Asia and Social Weather Station survey methodsBy Billy Almarinez
Sample Size - Part 1
Not only have I been studying and teaching statistics as a college instructor, but I havealso been using it as a scientist and researcher for quite some time now. However, it wasonly during the early morning of March 26, 2010 that I thought of trying to scrutinizeSWS and Pulse Asia surveys from a statistical standpoint. Although SWS and Pulse Asianever reveal how they actually conduct the surveys (aside from indicating the questionsasked, the number of respondents, and the margin of error they set) and that they usuallyargue that their methods are "tried and tested" ones, I think it would not hurt if we try totake a look at how representative their survey results are of the entire population of registered voters, using another generally accepted and "tried and tested" method we usein statistics.I'm talking about Slovin's formula.It is only from members of a sample (as respondents) that data would be obtained througha survey, since a census (or gathering data from the entire population) is not feasible for data gathering given a short span of time and limited resources. It is important, however,that the sample used be as representative of the population as possible, so that inferencesderived from analysis of sample data may be more or less applicable to the whole population. This may be ensured by using appropriate sampling methods and usingappropriate sample sizes.In statistics, Slovin's formula is a generally accepted way of how to determine the sizeappropriate for a sample to ensure better representation of the population of a known size.The formula may be expressed as follows:n = N / (1 + Ne^2)where n = sample size N = population sizee = margin of error Again, in the context of SWS and Pulse Asia surveys, these groups could argue that theyuse formulas other than Slovin's formula in coming up with sample sizes of 2,100 and1,800 respondents respectively (I did a review of news clips from the ABS-CBN Newsweb site, and noted that these two figures are the most commonly used sample sizes of the two survey groups). However, if we use Slovin's formula (which is, again, a generallyaccepted and commonly used method in statistics), rather alarming ideas may be derived(alarming, considering how some Filipinos base and defend their decisions on who tovote for on survey results).Considering that both survey groups usually set the margin of error at plus or minus 2
percent (or 0.02), here're what Slovin's formula says about the SWS and Pulse Asiasample sizes (anybody with a considerable aptitude in algebra may verify these):
•
SWS's survey over 2,100 respondents, with margin of error set at 0.02, assumesa population composed of only 13,125 individuals.
In the context of election-related surveys, that would point to the survey results being possibly representative of a population of 13,125 registered voters nationwide.
•
Pulse Asia's survey over 1,800 respondents, with margin of error set at 0.02,assumes a population composed of only 6,429 individuals.
In the context of election-related surveys, that would indicate that the survey results may berepresentative of a population of 6,429 registered voters nationwide. Now let's see.... Based on Slovin's formula, the SWS sample size seems to assume thatthere are only 13,125 registered voters, while the Pulse Asia sample size seems to assumethat there are only 6,429 registered voters nationwide. How many registered voters arethere in the country? Can anyone provide the actual population size of registered voters?Is there anybody reading this who knows anyone from COMELEC? I'm sure there's adefinite figure.Well, with or without the actual figures from COMELEC, I believe
13,125 and 6,429 aregross underestimations of the actual number of registered voters in the country
.I'm not trying to disprove SWS or Pulse Asia here. Again, it is highly likely that they areusing methods that do not include Slovin's formula. However, here's my case in point:
Before you believe that SWS and/or Pulse Asia survey results are what canactually be expected if elections were held then and there, think more than twice; itis also highly likely that the results may not really be reflective of what the entireFilipino electorate may actually and ultimately reflect, from a statistical standpoint
.And that is not yet considering the sampling method employed by these survey groups.Hence, to the SWS and Pulse Asia survey frontrunners and their supporters, I suggest for you not to keep your hopes too high, or you may end up disappointing yourselves if theactual results of the elections will not reflect the trends reflected by those survey results.And to survey tailenders and their supporters, there may actually be valid bases for younot to give much credence to these survey results. Quoting from Sen. Gordon: "The real'survey' is on May 10, 2010."'Nuff said!http://www.facebook.com/note.php?note_id=383911232421
ANOTHER TAKE ON SWS AND PULSE ASIA SURVEYS BY A STATISTICSINSTRUCTOR Sampling Method Part 2
Last time, I attempted to discuss the questionable sample sizes being employed by SWSand Pulse Asia in their surveys. Here, I am going to give my take on the samplingmethod.Again, as an instructor of college statistics and methods of research, I am inclined toquestion the methodology used by SWS and Pulse Asia in their conduct of surveys.Why? Because it is actually dubious how these two survey firms come up with resultsalmost every few weeks if they're actually using scientifically and statistically soundmethodologies. The question is rooted mainly in the reluctance (for some reason or another) to disclose the details of the procedures they employed. Scientific technicalreports like those that present results of surveys should include a detailed description of how sampling was carried out, and unfortunately every time SWS and Pulse Asia comeup with results of their survey they only provide findings without specifying in detail howthey conducted their study. They only indicate the sample size used and the margin of error they set as well as the question they gave a respondent, but the sampling methodand how they actually conducted the survey (i.e., how they distributed the surveyquestionnaires) seem to remain undisclosed to the general public.Recently, news reports (mainly from ABS-CBN and GMA 7) on the most recent PulseAsia survey results indicate that the survey firm used a "multistage random samplingmethod". What did Pulse Asia mean by that? And how did they actually determine whothe respondents would be? It is very easy for a researcher to say that he/she used or isgoing to use a random sampling method, but conducting such is actually not that simple.It is not as simple as going out in the street and handing out a survey form to somebodythe researcher meets "randomly". Such an activity is not a probability (random) samplingmethod, in which all members of the population are supposed to have an equal probability of being selected into the sample. In the case of surveys conducted via truerandom sampling, all members of the population of registered voters (including me andyou) should have an equal chance of being selected as a respondent.How should sampling for an election-related survey (like the ones Pulse Asia and SWSsupposedly conduct) be carried out in order for the results to be valid and reflective of thecharacteristics of the population? Here's my take, and my attempt to discuss why it is notas simple and as easy as how Pulse Asia and SWS want us and gullible voters to believe:
•
of 00
Analysis of Pulse Asia and Social Weather Station Survey Methods
Unfortunately, the alarming thing is that gullible Filipinos readily believe that Pulse Asia and SWS survey results are reflective of what the results of the election would be, ...
(More)
2,388 Reads
Info and Rating
| Category: | |
| Rating: | |
| Upload Date: | 04/11/2010 |
| Copyright: | Attribution Non-commercial |
| Tags: |
Leave a Comment