the people wlia qrraducest ae seieshle"study of me What is Content Analysis? useful definitions of content analysis have An enduringly popular one by Bernard Berelson is as fol ent analysis is a Teseareh tec words are objective, systematic, quantitative the words which dist scientific eontent anelysis the ordinary, informal anal s do every day in reading news papers or magazines or list the conrarsations of our friend Odjective means that the categories used to aualyze content must iMferent persons can analyze the seme fas and get the sime results. It also categories rnd terms (good-bad, avoided both because these terms are hi aoing terds to chan lection of content to anelyze must analysis, supposedly, from the argumentative, Diased collection of data to prose a point, Quantitative means that the results of the analysis are usually expressed numerically in come way: in frequency distributions, in contingency tables, in correlation coefficients, in ratios and percent, of various sorts, ‘The preference for quantification is unde! there is simply much moore agreement as to whet iz correct and incorrect procedure within the precise language of mathematics. It is always advisable, ‘of course, to be sure that the assumptions under lying the use of numbers have been met; a fine frosting of statistics 5 sometimes used to conceal a crumbling analytical cake. Manifest means that the cemantic anslysi analysis is ordinarily of fairly dizect enmean that the hat indicates an area of meat rato the history For content analysis, ory running back to to the establishment of the first chools of jou the place of his acefully examinin; the troublesome problems ered before and have been solved in the history of c before; thus, he may avoid haviog to nly of subject matter example of a study long CONTENT ANALYSIS a! patie In put ia, an ee fs ra substitu Assun sag then, ¢ ueratizel fashion, y Uhroweh classifications of newspaper content.z olitieal co and + the editor of ection fed to look: fo ig a cleac In method involves far too itulged in frivol «la colntion. Imm content elton worth $0 ardvous an rough’ cont. bo ef a problera bef t nnalysir the same rule h work of a rigure purposes of fat notion wader igor of the pr sy coaly of effort. ¢ the last analy2is, re tie major problems insel a abstract,MASS COMMUNICATIONS SEARCH, Sore one escape the necescity for | a 03 hig ~vhewe, Such a plan is especially important ia wordy. And one of problems s, in aluost aay study he anderta ollection of wo! 1, book's, oF m ie 6a of mene gazines, or ia time on radio oF ‘The quaterial canyot be coded in its entirety; aap f the problem is to be con- 1 was italicized above because one of the chief ampling. of course, is the availability of relevant, es of the mass media are ephemeral products fash out upon the scene and then are gon is is being ‘ritten, for example, there is no p America where one can Sind a complete current collection of Ameri vets, to sa nothing of a collection with deep run Librar estige papers of the tend to sare only the couse: Thus, es a mule, analysts who want to generalize ta th \nevieun press, must plan their study ip advance and subscribe to & nple of newspaper were interested in generalizing American uewspapers, a sample could be drawn by going tot bee oi dexwing uimbers randdoualy:Erom. the Since there are many more small dailies iiss ia the nation, however, such a plan would no! reptesnt the content typically seen by readers of Americas 3, most of whom live in large urban areas and read the \ desi lerge cice newspaper 1d to reprosent what is avail- able to readers, therefore, would have to be weighted in such a way har papers of lasger circulation bad a greater chance of being selected. Sulch w sample is described in the International Pross Institute Flow the 3 dy" Other nei in the references. fer titles of publications have been selected, issues must still bi ampled iu fime. ‘This presents additional problems to the content analyst. It is well known, for example, that American daily news- 5 tary in thickness during the varias days of the meek, due primarily to the Gow of advertising which reflects the weekend buying spaper sampling methods are giver pap ally ha all this in tore of the me ontent from S. ys, hod of time s is the “co Tn th apers. Then a Monday papers pepers from the yea fnucted weeks” pews and a ple the total-tiad section, th re often ward so that others ¢ operation is simply * in some of the ¥ fer seliability hhave used words as Often the analyst i individual words, bo"CONTENT ANALYSIS 185 papirs of Thursday are typically fat s ads papers are slim. Similarly, maga. cally have their biggest issues of the year just before and 1c Christmas chopping season, Radio eud television station every day with progra onteat Ww with the hours and with tl le of newspapers which included Ip content from Saturday afternoon issues would be @ poor sample for most analyses, for esanip metimes used in newspaper content eek” technique of Robert Jones and arter!? Tn this method a “calendar” is constructed sho Tuesday issues, ete, fa the total sa naple is taken by randomly choo itt on the ealenda y. In this manner ich will respresent th days of the week, and sed in th For some analyses sic coding unit must be established. This is t division or segment of the content which is to receive a score, ‘Typical coding units are the word, th theme or assertion, the item, space and time units, and the character.** Words are often easier coding units to work with than the more encompassing subdivisions of content, Tt is fairly easy to define a vword so that others cam identify it in all its variations. If the coding 1 is simply to note the presence or absence of certain words (as in some of the readability formulas, for example), a high degree of coder reliability can usually be obtained. Some studies which hhare used words a3 basic coding units appear in the references." Oftes the analyst is interested in larger units of meanis, individual words, however, and he feels that he must move to a largorIf e cour of individual wo If the hypo A theme is define ax a sine sentence . an 4 subjectmatten"* For example, ¢ advice columnist toward ravious re, mothers, sisters, brothers might be ‘The statement, ~ Your father, tho ‘ant nian,” contains at least thre handsome,” © Your father ant,” It is evident chat, unless tho p analysis cam rapisly explod jody of content into a large smozat of data. TY hould e he starts to hat b ertaia that be is coding oaly the he st s within the content) that ax wth may wonder why content onal relatively al geanimiotica! upits suck as the sentence or the 1a hy these units should not dl indeed they are occasionally ficult to put these units into f the categories are at all comples. What i neutral, or unfavorable) is to be assizued to a nit? Obriowely the sentence a tements, joined by such connectors as “but,” “how nevertheless.” Coders, faced by such conflicting statement contain cuntict niay not operete relisbly in coding the sentence, One tine they may decide that the orer-all weight of the sentence is favorable; che nest time they may decide that the weight is neutral or even unfavorable. ‘The seme criticism applies even more strongly to paragraplis es units Why then. is the most popular unit in content analysis the entire item, story. article, or editorial? Does not the cary CONTENT ANALY plied to voliag 2 unit isd he claesication terested in nieas requeatly the ste ‘ably into a six ries Usi cof material ace to tolerated because cer Sling spice und Be ew it have found that « precisely) correla item coding is be A character, 6 For example, Ber “approved” charaCONTENT ANALYSIS 1k ayphe nly, when coders attempt to place an entice asi ples eategory. Attempts to write directions for 7 rniag such feats nearly always produce fei veul directions are not written, coders tend to approach the subjectivels, and their classifications aie not relinble. The wise student will etways look for aad exsasiue with care the eoder rei ability figures in studies whish report that comples categories we applied to coding units es large a» witble stories. Often he will f that reliabilities simply are not reported, Using the entire article 2 uuit is defensible, loweret, in a sariety of other situations wh-n the classifeation categories are tuuad, For esample, if one interested in measuring atfention paid to various entegories of a: frequently the story has unity enough to enable coders to place i reliably into 4 single desctiptive news category, or at most into ts categories Using the large unit is defensible also if gtrat aus ‘of miaterial are to be coved. Someschat lower reliabilities can the tolerateil becouse enough cases will be collected so that true relatic: vill show up in spite of a considersile amount of rtom cod: \ covling bins, of cousse, will continue to appear im spite v lange usiuber of waits cod Spice and time units such as the column tis io aul television), were widely: ua ccrigtive wouter: analyses and are still used today.” When subtle estegorios are to b employed, time end space nieasuremtent suffers in the sane way th item units tical units sailez, Moreover, severul researcher hhave found that space and time uaits (which are difficult to me precively) correlate highly with item units; thus, the simpler natur: item coding is being more frequently employed today. A character, oF person, or a class of persons is sometimes used ss 8 coding unit in content analysis. All relevant information the character is sifted out of the article or story and cl For example, Berelson and Patricia Salter classified charactors i American magazine fiction according 4p whether they were “Ameri cans,” “Anglo-Saxons and Nordivs,” or “Others.” ‘Then the examined each character as to his gosls, hia occupation, whether he was “approved” or not, etc. ‘They found that in the stories analyze’, the Americans, Anglo-Saxons. and Norilies were mote often majo> “approved” characters and cere mote often interested ia heart goals188 MASS COMMUNICATIONS RESEARCH ‘hers. Recently, Jack Schwartz studied the in motion pictures in such term romantic involvement, telationships with students, ete, Oue usions was that it the movies, “, . . a successful romance for the eduestor exists .~'v outside the academic pale; he leaves the teaching profession or rsrties someone with less edueation.”? The character unit has often been used, us the foregoing examples show, to esamine the extent to which the mass inedia perpetuate the stereo. types of our society. The Contest Unit ding unit is the smallest divis a score, Sometims an examination of the basic unit; the unit can be coded reliably ouly conlezt, Hence, # contest unit—the largest division sulted by a coder in order to assign a g unit—is often used in content analysis. rat you were to categorize all sxsertions made by a source about Communists as being favorable, unfavorable, or neutral. Suppose, further, that you encountered the following asser ion: “The Communists are taking over the world bit by bit." In olatiou, it is impo: ‘assertion accurately. It sounds neutral or ambiguous, Only when it is placed in a context unit is the dizection apparent. Fur example, if the sentence appears in a ech entitled, “Toe Red Menace,” delivered at a National Associa. Yon of Manufacturers Convention by a conservative Republican nator; then the correct code probably would be unfacorable. If the sertion appeared in @ speech by a Russian astronaut to » world therigg of Communist youth, however, the correct code would \doubredly be favarable. Some limits are usually placed on the size of the context unit which the coder is permitted to scan. Ultimately the unit could become so large that the coding of the basie unit would again become unreliable. A historian trying to analyze Jefferson's quotations concerning the press in the context of the president's entire life might encounter this problem. ‘Then, too, as the context unit become: larger, more time is spent scoring each coding wait. Usually, therefore, a compromise must be sought be ‘ween the desirable and the possible al word on the selection of units of analysis is in order. of content to receive , however, a score cannot be given solely from he ba Supt le to code th CONTENT ANALY It is certainly ‘eGnitions, unit fore him. He rwcasutes others bi past. From tin atvenspt deliberate problem vith fre couse, than to d that insights axe ales Testing Coder Re At neatly the method being bility, ‘This me different coders to apply the nie Tt may also men required to re I do it extremely impoRey being used are MSY on the face of iv niet investigator subtle gualifcatio tliat these qualifice aad incorporated i: cated to others, 2 unreligble, A cod g newly traine is intuitions and concrete verbal exp Jouraals are gradu: coder reliabilities Various method in the refereaces: given here>! Sup arerage nusiber of of a number of nes theCONTENT ANALYSIS Tr is certainly desirable that the student be familiar with the definitions, units, codes, and categories other researchers have used before him. He can eave salusble tine by following the diseipliaed mes hers have devised. Yet he should uot slavishly follow the Fast. From time to time, as be is developing bis idee, he shoutd attempt deliberately to break with the conseational and try to se his Problem with fresh and clear visiou. Tt is easier to say thin, of course, thaa to do it is oftea in such contemplative moments that in ith advance science not by inches hut by Testing Coder Reliability AL an arly stage in the research, and at intervals thereafter, the method being used in the analysis should fuce tests of its rel Ghat | This means, ia general terms, that the investigator puts ‘lflerent covers to work on th» saiue couteut and checks to see whee they apply the zuethod in way and obtain the same results. FE may clso nvean that the ssiue coders, after a time lapse, are Top ige’ te tecode some matevial coded clier to eee whether they Sil do it the come way the second time. Coder reliability rate wee {kitemels impottant ad should never be negleoted unless the codes tring used are so obviaus that even a sévece ectic will eoncode ther can, he ace of it” they can be reliably used. In many cases the chiet investigator or his main analysts gradually learn to sasha many aubtle qualifeations and restrictions ia coding content. ‘To the cutee, fhat thee quilifcations and restrictions ate not plainly spelled sat and incorporated in the coding directions they canaot be tommune cies icles, and the method is likely to become inereasingy Garelisbe, coder reliability check tekea at intervals i the rseatel, feing nerly trained coders, will forcibly remind the researcher thay bis intuitions and subtle feelings for coutent must allimataly fal Stuerete cerbal exprostion if others ate to use his method. Repatalic Journals are gradually becoming reluctant to print aualyces in which coder reliabilities have not been tested. ethous of making coder reliability tests are mentioned in the references: hence illustrations of only two methods will he given here.*t Suppose one puryose of the analysis is (o extimate the arerage uuuuber of page oue uatiooulinteruatioual stories per i of a number of newspapers. We miight devide to test ceterin this instance by teking fifty newspapers from the sainple vod ‘oder go through the papers, counting the number of = on t having each sationel-international st MASS COMMUNICATIONS RESEARCH front page of each. We then could make a correlation matrix giving the scores assigned to exch b pair of coders. In Figure 1, for example, each newspaper by mark in the « front page of paper by Coder A (on the Inspection shows a high degree of agree ment between these two coders, ‘The Pearsoniam produet-mom correlation between the two sets of scores is .961. We migh (on the bor otal asi ttergram represents the scores given to a particula: jeal axis) and Coder B from this point, and figure the product-moment correlations between the scon and 59 on, estimate of the co Scattergeam given by Co A and Coder C, then Coder B and Coder (, Fivally, we might present the average correlation as ou, found on thy Coder A 1B ua 8 4 ul 10 bility this study FIGURE 1 the number of national-international stories front page of fifty newspapers by two coders i 678 9WU Riis Coder B CONTENT ANALY: What if the ws coders axe disapy tunities for erzors The process of stories but coding it iy often helptu but to look into t involved aro codi cedute often resu percentage of agr of stories on whi of stories found b tested. Thus, su 829 national-inter found 341. Of ¢ found by Coder these two code? should be divi whic Og <= (Io the esampl 2 inches long ou In some te (for example, ¢ 6 a3 juvenile, adult, involved ia fndia, be merely the ui divided by the to: all possible pairs st since theo analysis report. i perointage of age coer rclability te ceiling conditions to the fact that Testing the Valid Validity suppe technique measurehetwee vow ¢, CONTENT ANALYSIS What ifthe cortelation vocfficieut is law aud the scores uf the coders are disappointingly different? Ubsiously, at least eo oppor: s for errors are present: (1) A coder may be missiug stories in ess of scanning the paper; or (2) A coder may be finding stories but coding them erroneously’ for one reason or another. ‘Thus, it is often helpful not ouly to check the total score given a paper but to look into the matter further aud fad out whether the coders invilad are coding the same stories in the same way. Such a pro cedure often rezults in a statistical measure referred to as the avery percentage of agreement, which is, in one definition, the number of stories on which two coders agree divided by the total number of stories found by both the coders, averaged oxex all pairs of coders testod. Thus, suppose that in the example given, Coder B found 323 national-internetional stories in the fifty papers and Coder Found S41, Of the 323 stories found by Coder B, 321 were al found by Coder A. What is the percentage of agreenunt between hese two coders? ‘The answer is 94 per cent, because the 321 stories should be divided by the total number of stories found by Lot. coders which is S42 An analysis of this type helps the researcber ud which coders are meking mistakes and what those mistakes a (Un the example, Coler 1 was cosnting stories wwere lets thaw 2 inches long and Coder B was not.) In some tests coders are given a certain number of waits to code (for example, a certain number of stories about crime, to be coved juvenile, adult, or mised crime). In this case, there is uo error involved in finding the stories, and the percentage of ayreerueat will le merely the number of items coded the same way by two covers divided by the total number of items in the test and averaged over all possible pairs of colers. ‘This test obviously is easier than the first since the opportunity for error is less. In reading content analysis reports, it is important to note specifically how the reported percentage of agreement was obtained. Generally speaking, those coder reliability tests are most respected which are done under natural coding conditions, using natural materials, and without alerting covers to the fact that they are participating in a special test of their ability Testing the Validity of Results Validity supposedly answers the question, “Does the measurement technique messure what if is supposed to messure?”* In the fore- /