You are on page 1of 16

Jessica R.

Dreistadt Critical Review Analysis: Mass Media’s Effect on the Polarization of Social Groups SSP 411 – Advanced Research Methods October 12, 2004 Problem and Hypothesis a. What is the problem? Vincent Price investigates the effect of the media on people’s association with social groups in conflict. This study seeks to understand the extent of influence the media has on public opinion when group conflict is or is not present in the reporting mechanism. Jeffrey C. Hubbard, Melvin DeFleur, and Lois B. DeFleur investigate if public opinion of specific social problems is closer in number to mass media portrayals of those social problems or actual statistics. Specifically, they seek to determine if the media unduly influences public opinion about the incidence of social problems in their community. Martin Gilens conducts his research in 1996, at the height of welfare reform. He examines the content of print and broadcast newsmedia and compares the race, age, and occupational status of people in stories about poor people with census data. He then used secondary data to determine if the media influences perception about low-income people and whether or not this leads to opposition to welfare. Rosalee Clawson and Rakuya build on Gilens research four years later. They examine photographs in newsmagazine stories about poor people and compare the race, age, work status, gender, family size, and residence to government data. They also count the number of times people in these photographs are engaging in stereotypical behavior. Like Gilens, the problem

Dreistadt 1

under examination is “whether the media perpetuate inaccurate and stereotypical images of the poor.” (53) Thomas E. Ford studies the detrimental effect of stereotypical portrayals of African Americans in television sitcoms on white opinion about individual African American people. Specifically, he examines whether priming subjects effects their judgment as to the guilt of a crime suspect. b. What are the variables? Price devises two fictitious student newspaper articles about changing Stanford’s core requirements - a nonexistent issue of limited concern to students. One article states a difference in opinion between ‘hard’ or ‘soft’ majors (205). The other article notes that there are mixed opinions about the change, but does not attribute those opinions to any particular social group. Participants were asked to rate a set of student’s comments about the issue; each student was identified by their major and class year. Half of the conflict group and half of the no conflict group questionnaires stated that this issue was actually being considered for implementation and the other half stated that it was not currently being considered (this article was five years old). The questionnaire also contained a fictitious list of students’ comments about the issue, with each student’s major and class year identified. To further complicate matters, some (a number or percentage is not specified by the author) were given questionnaires that did not identify the major and class of the students who offered comments. The independent variables were the presence/absence of group conflict, the level of consequence of the issue to the respondent, and the identification of major and class in statements about the issue. The dependent variables were the subjects’ opinion about the social groups’ position and their personal opinion about the issue. Hubbard et. al. do not conduct an experiment. They compare content analysis of local

Dreistadt 2

television and newspapers, along with agency records, to survey results. Gillens also uses content analysis, along with secondary data; thus, there are not independent and dependent variables. Similarly, Clawson and Trice conduct content analysis rather than an experiment. In Ford’s study, participants are categorized into four groups. Half watch a TV skit that portray stereotypical African American behavior and the other half watch a skit that portray an African American person engaging in neutral behavior. All of the participants are then given a vignette to read; half receive one with a white suspect and the rest receive one with an African American suspect. The independent variables are the comedy skits show to participants and the race of the suspect in their vignette. The dependent variable is the subjects’ assessment of the guilt of the suspect in the vignette. c. Does this research have a theoretical base? Price develops a theory about the process by which the media influences public opinion in three parts. In the first step, the message recipients react to cues in the media by determining their relationship to the people in the news story. Their level of salience to the group is determined by the presence or absence of group conflict in the message, the pre-existence of group identification in the message recipient, and the recipient’s stake in the issue. In the second part of this process, recipients process the information presented about other people’s opinions using stereotypes about the groups in conflict. In the third step, recipients conform to their expectation of the norms of the group to which they have determined they belong. (203-4) Hubbard and colleagues base their research on a theory that organizes the process of social problem development into three stages: emergent, legitimization, and institutionalized. During the first stage, groups work to raise public awareness of an issue. In the legitimization stage, a social problem is accepted and recognized by the general public. In the institutionalized

Dreistadt 3

stage, “official social machinery is established to alleviate the problem on a continuing basis.” (23) They also present theory related to the media’s indirect effects on defining social problems and public opinion about those problems. Gilens bases his study on the assumption that Americans have a better understanding of poor people who are white, employed, and/or elderly (522). He also notes that, “media distortions of social conditions are…likely to result in public misperceptions that reinforce existing biases and stereotypes.” (516) He further states that this, “increases white Americans’ opposition to welfare.” (517) Finally, Gilens states that, “negative stereotypes of African Americans as lazy and misperceptions of the poor as predominately black reinforce each other.” (518) Clawson and Trice present several theoretical bases for their research. The authors state that, “in our society, citizens believe poor people have many undesirable qualities that violate mainstream American ideals.” (54) They also note that “the visual representation of a political issue is an integral part of the definition of that issue.” (54) They continue, “the pictures provide texture, drama, and detail, and they illustrate the implicit, the latent, the ‘taken for granted.” (55) Ford presents theory about the learning process related to television viewing and opinion formation. He summarized this theory by stating that, “in the case of priming an abstract representation of a social group in general, the target person must belong to the primed category. He or she must fit the activated stereotypical representation if priming is to influence social judgment.” (268) d. Do any hypotheses or predictions follow logically from the theory that is presented?

Dreistadt 4

Price hypothesizes that “news reports emphasizing group conflict, by providing members of the public with a depiction of the “sides” they may adopt in thinking about a responding to an issue, may plan an important role in organizing public opinion.” (200) Hubbard, DeFleur, and DeFleur hypothesized that most survey respondents would heavily rely on mass media for information about social problems, due to limited personal experience, and therefore their opinion would be different from actual agency records (25-6). They also hypothesize that media reports would not reflect public opinion and that the significance of the media’s influence would be demonstrated by a high correlation between public opinion of the incidence of social problems and media reports about those problems (26). Gilens does not explicitly state a hypothesis. Based on the information presented, he appears to be hypothesizing that the content of national print and broadcast news media inaccurately reflects the race, age, and employment status of poor people. Although not a part of his original research, Gilens also offers evidence that public opinion is tied to these inaccuracies (528) and this leads to lack of support for government aid programs (517). Clawson and Trice clearly do not take a solid position on the issue under investigation. The authors state that they are studying, “whether or not common stereotypical traits or behaviors associated with the poor are portrayed in the media” (54) and “whether the media perpetuate inaccurate and stereotypical images of the poor.” (53) The background research and theory that they present would lead the reader to ascertain that their position on these questions is that yes, the media does these things. Ford refines his theory by predicting that white people who are exposed to stereotypical images of African Americans on television will judge an African American suspect to be guilty. This same group, he speculates, would not make the same judgment about a white suspect. (268)

Dreistadt 5

Method a. Is the method appropriate and adequate to examine the problem or hypothesis? All components of Price’s methodology were relevant. This experiment that examines the process of group identification and level of salience to that group. By having respondents’ freely write their reactions to the articles, he is able to better understand the thought process that effects choice of group and position on an issue. Given the hypothesis, Hubbard and colleagues select an appropriate methodology. The hypothesis relies on simple analysis of media and agency records’ content compared to a public opinion survey. If Gilens purpose is to merely compare the content of newsmedia to actual statistics, his methodology is adequate. If, however, he also seeks to show causality between the newsmedia and public opinion, the secondary data he presents is not sufficient. Original research would need to be conducted in order to adequately control the sampling process and research instruments. The method employed in Clawson and Trice’s research adequately examines if a specific component of the media misrepresent the poor. They perform content analysis on pictures featured in stories about poor people in five national news magazines. Ford’s methodology was adequate to examine his hypothesis. By exposing participants to different television skits and measuring their judgment of a crime suspect afterwards, they can reasonable determine the effect of priming on opinion. b. Are the tests or measures appropriate tests?

Dreistadt 6

The tests used in Price’s research design thoroughly investigate all three steps of the theory he seeks to affirm. The operationalization of the hypothesis into these specific measures is appropriate and comprehensive. The measurements presented in Hubbard and colleagues’ research are appropriate to evaluate agency records and media content. Additional analysis would be needed to determine the content of national news, sitcoms, and other TV programs in order to analyze their impact. The hypothesis should clearly state that only local news is under investigation. They do not divulge sufficient information about the content of the survey used to make a determination about its appropriateness. Gilens uses appropriate measures to catalogue print and broadcast newsmedia. He composed an exhaustive list of topics related to poor people and looked up news stories using a reliable reference guide. He breaks race down into white, black, and undetermined – ignoring Hispanic, Asian American, and Native American populations. Clawson and Trice improve on this shortcoming by adding Hispanic and Asian American to their race categories. They also build on Gilens research by analyzing additional characteristics of the people in the photographs: gender, family size, urban/rural residence, and stereotypical activity. Ford’s measures are also appropriate for his purposes. After viewing the television skits, participants read a vignette, respond with their verdict, and then evaluate the skits they saw in terms of comedy and stereotypical behavior. c. Are the tests reliable and valid? Price conducted sufficient pretesting. He tested students’ opinions about the choice of subject for the article by surveying a “convenience sample of 42 Stanford undergraduates.” (204)

Dreistadt 7

Because he does not divulge his exact process for deriving this sample, its validity cannot be ascertained. Before reading the articles, students were asked about their position on the issue. This may have influenced their response by sensitizing them to the subject. Participants were randomly assigned to the control and noncontrol groups. Because the study investigated students’ affiliation with specific college majors, participants were selected from a variety of departments. In debriefing, students noted that they did not suspect that the articles, and the issue itself, were fabricated. Students’ narrative comments were categorized by two coders who exhibited a high degree of intercoder reliability (80%); their differences were averaged. When further divided into ‘positive’ and ‘negative comments, intercoder reliability diminished to 6675%. Again, the differences were averaged. Ratings for the student statements about the issue are divided into ‘hard’ and ‘soft’ majors, as well as high and low salience, and then correlated; this further affirms the reliability of the measure. Because Hubbard et. al. are using content analysis and a survey, their results would presumably be reliable. However, reliability may be limited because the authors do not detail the nature of, or selection process for, the survey questions that were used. In addition, because their sample is drawn from a city that is significantly not representative of the US population, the results cannot be generalized to other people and places – a threat to external validity. The authors state that they selected respondents from specific census tracts but do not explain why these census tracts were chosen. They do not indicate whether the content analysis was doublechecked by a second coder. They also do not consider the possibility that many manifestations of the social problems under investigation may not be reported to authorities. Gilens process for content analysis was highly reliable. A straightforward system of recording the race, age, and employment status of people featured in photographs was used. A

Dreistadt 8

random sample of 25% of photographs were double-checked by a second coder - intercoder realiability was high – from .94 to .98. Gilens devotes significant space in his discussion to public opinion about the actual composition of poor people. Original research would have to be conducted to exert more control over all components of that key factor in order to for the research to be valid. Clawson and Trice do not provide sufficient information about their categorization and coding process. They do not name the topics referenced in the Reader’s Guide to Periodical Literature. The process for determining various characteristics of the people in the photographs is unknown. Therefore, it is difficult to judge the level of reliability. The study has validity because they are indeed measuring the content of newsmagazine photographs. However, external validity may be limited if conducting this research with television, the Internet, alternative media, or local/regional media. There was an insufficient sample size in Ford’s study. There were a total of forty subjects, divided into four distinct groups. A larger sample size would improve the validity. In addition, participants were paid $4.00 and given extra credit for participating. This might have influenced results. Reliability is threatened because the instrument gauging subjects’ opinion of the suspect’s guilt consists of only two questions. The instrument that measures their impression of the skit also consists of just two questions. Participants were told they were participating in two studies – comedy and judicial review. They watched the skits, read and responded to the vignette, and then rated the skit in terms of comedy and stereotypes. Further analysis could determine if this ordering is most appropriate. During debriefing, one (in 40) participants revealed that s/he suspected that the two studies were related. Pretesting could have identified if preexisting prejudices were evident in the sample. The race of the suspect in the vignettes was

Dreistadt 9

implied by his name: Tood or Tyrone. A test was performed to assess the way this sample would categorize those names. d. Is the research design adequate? Perhaps Price’s research design is overadequate. Although he does thoroughly investigate the interrelationship of the variables at hand, the design is cumbersome and difficult to understand. It would be helpful to reduce the complexity of the research design, if this could be done without jeopardizing the integrity of the experiment. The research conducted by Hubbard and colleagues is fairly straightforward and the research design is adequate for their purposes. Research design could be improved to increase validity and reliability. Again, Gilens’ research design is adequate if he only seeks to demonstrate the difference between national newsmedia content and the actual race, age, and employment status of poor people. If he is additionally seeking to demonstrate causality between this and public opinion about poor people, additional research in a controlled environment is needed. Clawson and Trice are able to adequately assess their problem with their research design. It would be interesting to compare the content of the five national newsmagazines they analyzed to other media, such as newspapers or television. Ford’s research design is adequate. It measures and analyzes the variables stated in the hypothesis. e. Are all extraneous variables controlled? Because of the complexity of the research design, extraneous variables were minimized in Price’s study. Additional means of controlling such variables are unknown.

Dreistadt 10

In contrast, the research of Hubbard et. al. is fairly simple. They do not adequately control for the influence of media other than those analyzed or for potential confounding factors. There is only one relevant reference in Gilens paper. When using the Reader’s Guide to Periodical Literature to develop a list of news stories about poor people, he did not include African American or Black in his subject list (519). He notes that therefore “the body of stories…will underestimate the true degree to which poverty is presented as a black problem.” (519) However, he does not consider that many stories about poor people would like be cross referenced under the other topics he searched. Clawson and Trice do not have any extraneous variables to control. Ford’s paper does not identify extraneous variables or discuss how they are controlled. Results a. Are the results clear? The results in Price’s paper are clearly presented. This is aided by charts that present the data in an easy to read and interpret format. He consistently relates the results back to his original hypothesis. Hubbard and colleagues reiterate the hypothesis and relate the results accordingly. Because the results are simple numbers, they are clearly communicated via a chart. Gilens clearly presents his findings, using narrative comments and tables that capture the information. He breaks the results down into the three areas of interest – race, age, and occupational status. Clawson and Trice present their data clearly. There are a lot of numbers discussed and more charts would be helpful.

Dreistadt 11

Ford presents his results in a clear manner. He computes Cronbach’s alpha for the comedy and stereotype ratings of the skits and the guilt ratings, noting the presence or absence of main effects and interaction effects. b. Are the control variables really controlled? Because the independent variables were presented in questionnaire format, they were effectively controlled in Price’s study. This is not applicable to Hubbard’s, Gillens’, or Clawson and Trice’s research as they are not experiments. Ford does control the independent variables. Because they are prerecorded or preprinted, this is easily done. c. Did the experimental procedures (if any) “work?” Price’s experimental procedures reveal information about to support his hypothesis. Participants identified with social groups much more strongly when the article contained group conflict. Significant main effects were found for presence of conflict and level of personal consequence. Interaction effects were found between personal consequences and salience, as well as presence of conflict, personal consequences, and salience. This demonstrates that the presence of conflict did in fact cause people to choose a position in addition to effecting the magnitude of that position. Subjects’ reaction to the student statements about the issue further supported the hypothesis. Price uses regression to analyze conformity to group norms; in this instance, part of his hypothesis is rejected. Hubbard et. al., Gilens, and Clawson and Trice did not conduct an experiment; thus, this question is not applicable.

Dreistadt 12

Ford’s experimental procedures examine the underlying constructs presented in the hypothesis. The results can then be used to accept or reject that hypothesis. Discussion (and/or interpretation) of Results a. How does the interpretation follow from the results? Price’s interpretation further explains the results – both expected and unexpected. He also offers areas that require further explanation. He reiterates that the influence of media on public opinion formation is a systematic process, and that this research investigates a portion of that process. Because the results do not support Hubbard and colleagues’ hypothesis, the results explain possible alternative hypotheses. They also relate the results back to the theory, stating that the “problems under study…are clearly in the institutionalized stage.” (30) Gilens paper has a section between the results and conclusion that presents a lot of new ideas and data. This both adds to the findings just presented and confuses the issue he is really interested in examining. His research was based on content analysis and comparing that to census data; this section offers additional secondary data about public opinion related to race and poor people. Gilens also explores several possibilities that might explain why the media misrepresent poor people. Again, he is presenting new data. His summary neatly ties up the key data uncovered through his research. He restates the problem in his last sentence, “by implicity identifying poverty with race, the news media perpetuate stereotypes that work against the interests of both poor people and African Americans.” (538) In the discussion, Clawson and Trice also tie in the connection between media and public opinion, citing Gilens and others. They provide a summary paragraph in the results section,

Dreistadt 13

which may have been more useful as a beginning to the discussion section. In the discussion, Clawson and Trice note that other types of media also misrepresent poor people. Ford explains the results obtained in terms of the theory about priming. He also notes the significance of using comedic portrayals of stereotypical African American behavior, suggesting that, “disparagement of social groups through humor increases our tolerance or acceptance of discrimination against outgroups.” (272) This could be explored through further research. He offers much additional theory and research support in the discussion section. b. How do methodological problems influence the results? Price admits that students tend to view his chosen subject in a negative manner. This may have effected participants’ responses. He also notes that subjects may have altered their response because they viewed their group’s position as “selfish or mundane” (220). Again, Hubbard’s failure to include national news media may distort the results. Their process for evaluating content was adequate. As the authors suggest on page 30-31, the media may have a stronger influence on public opinion during the first two stages of social problem formation and this project investigated problems in the institutionalized stage. They also speculate that “media may play a role in shaping conceptions of importance, but little role in producing beliefs about prevalence.” (31) These factors could have been incorporated into the research design by examining a variety of issues and comparing public opinion about their prevalence and importance. Significant methodological problems were not detected in the content analysis of news media in Gilens’ study. His results may have been different if original research was used to learn more about public opinion in this area.

Dreistadt 14

In Clawson and Trice’s paper, they do not give sufficient information about the coding process to determine if it influenced the results. Their selection of five newsmagazines in the universe of all media may have influenced their results. The sampling technique used could possibly influence results in Ford’s study. The order of questions presented after viewing the television skits could also have skewed the results. c. Is the interpretation appropriate to the original hypothesis? The interpretation of Price’s research presents significant support for most of the original hypothesis. It offers possible explanations for particular components of the three-prong hypothesis that did not yield expected results. Hubbard and colleagues summarize their findings in relationship to the original hypothesis. They suggest several reasons why their hypothesis may not be true, and offer additional areas for investigation. Gilens interpretation is appropriate given his original hypothesis, with exception to the presentation of new data after the results section. He restates the problem and his perspective, based on the research just conducted. In Clawson and Trice’s paper, they do not discuss their research in the discussion section, but the research of others. Analysis about what they saw in the magazines they reviewed and speculation into why this happens should have been included. Ford clearly supports his original hypothesis in the discussion. He also offers additional hypotheses that could add to this research. He raises the question about the effect of humor at the end; perhaps this could have been used as a control variable, or at least introduced as a possible influence, earlier in the paper.

Dreistadt 15

Works Cited

Clawson, Rosalee A. and Trice, Rakuya. 2000. “Poverty as We Know It: Media Portrayals of the Poor.” Public Opinion Quarterly 64:53-64. Ford, Thomas E. 1997. “Effects of Stereotypical Television Portrayals of AfricanAmericans on Person Perception.” Social Psychology Quarterly 60(3):266-278. Gilens, Martin. 1996. “Race and Poverty in America: Public Misperceptions and the American News Media.” Public Opinion Quarterly 60:515-541. Hubbard, Jeffrey C, DeFluer, Melvin L., and DeFleur, Lois B. 1975 “Mass Media Influences on Public Conceptions of Social Problems.” Social Problems 23(1):22-34. Price, Vincent. 1989. “Social Identification and Public Opinion: Effects of Communicating Group Conflict.” Public Opinion Quarterly 53:197-224.

Dreistadt 16