You are on page 1of 6

Pharm World Sci (2008) 30:216–221 DOI 10.



Critical appraisal of apparently evidence-based written advertising in Pakistan
Dileep Kumar Rohra Æ Muhammad Umair Bashir Æ Ummey Aymen Khwaja Æ Muhammad Ressam Nazir

Received: 26 March 2007 / Accepted: 23 September 2007 / Published online: 12 October 2007 Ó Springer Science+Business Media B.V. 2007

Abstract Objectives The objective of the study was to critically assess references cited in support of claims in drug advertisements. Methods Drug advertising brochures were collected from privately practicing General Practitioners from different parts of Karachi. Three blinded reviewers then categorized each reference in the brochure according to the sources viz: journals (both Medline indexed and non-indexed), medical reference books, web addresses, personal communications or testimonials, abstracts presented at symposia/conferences, WHO and National Health Guidelines, ‘data on file’ and ‘others’ (which included a diverse set of references). Each reviewer then assessed and analyzed the references further into 2 broad categories: traceable and non-traceable. Traceable references were appraised and, depending upon the claim with which the reference was attached, were classified into justifiable, inaccurate/false, exaggerated and ambiguous. Results We collected a total of 175 different brochures. Thirty-nine (22.3%) brochures did not cite any references and were not subjected to further analysis. The remaining 136 (77.7%) contained a total of 559 references. 305 (54.6%) of these references were from Medline-indexed journals; 67 (12.0%) were from non-indexed journals; 55

(9.8%) references quoted medical reference books; 27 (4.8%) references cited web addresses; 12 (2.1%) references were personal communications/testimonials; 11 (2.0%) references referred to abstracts presented at symposia/conferences; 12 (2.1%) references were from WHO and National Health Guidelines; 8 (1.4%) references were listed as ‘data on file’, while the remainder that could not be defined were classified as ‘others’ (13.1%). Out of a total of 559 references, 249 (44.5%) could not be traced. After critically analyzing the 310 traceable references, 197 (63.5%) were adjudged justifiable, 30 (9.7%) inaccurate/ false, 79 (25.5%) exaggerated and 15 (4.8%) ambiguous. Conclusion Results of this study show for the first time that the claims substantiated with references in the pharmaceutical advertisements in Pakistan are highly unreliable. Keywords Advertisements Á Drug promotion Á Evidence based advertising Á Marketing Á Pakistan

Impact of this paper • • All claims by pharmaceutical companies need to be verified independently. The practice of critical evaluation of claims of pharmaceutical companies by health care providers may contribute in promoting rational prescribing. Publication of this paper may help minimize the irrational claims. When pharmaceutical marketers realize that their claims are monitored and critically analyzed, they may be more vigilant on their practices. This publication may help the Ministry of Health of the Pakistani Government (and maybe other countries as well) to be more aware of a need to monitor marketing practices of pharmaceutical companies.

D. K. Rohra (&) Department of Biological and Biomedical Sciences, Aga Khan University, Stadium Road, P.O. Box 3500, Karachi 74800, Pakistan e-mail: M. U. Bashir Á M. R. Nazir Faculty of Health Sciences, Aga Khan University, Stadium Road, P.O. Box 3500, Karachi 74800, Pakistan U. A. Khwaja Dow University of Health Sciences, Karachi, Pakistan


Pharm World Sci (2008) 30:216–221


Introduction Drug advertisements have been criticized for their lack of reliability and their potential in influencing physicians’ prescribing habits [1], and also for lapses in ethics [2, 3]. Pharmaceutical companies today adopt a number of methods to advertise their products. In Pakistan, like many other countries, the most important means used by pharmaceutical companies to promote their products is direct approaching to medical practitioners. One extremely effective manner of marketing when contacting practitioners is to distribute advertising brochures. The usefulness of the drug is highlighted through a series of promotional claims about the drug’s efficacy, convenience and safety. Although drug companies in Pakistan advertise their products through journals, newspapers, and medical seminars, physicians here rate pharmaceutical representatives along with their advertising brochures as their most important source of information about new drugs [4]. These brochures typically highlight the usefulness of the drug by displaying emotional messages and drug statistics. Realizing the importance attached to evidence-based medicine in professional circles, drug advertising brochures cite various references to substantiate their claims and leave a lasting impact on the physician. Advertisements can be highly informative as long as they are critically appraised [5], and the advertisers follow established guidelines. Policies exist regarding the guidelines for drug marketing in Pakistan (Drugs: Licensing, Registering and Advertising Rules 1976) [6] as in other developed countries like the USA [7] and UK [8]. These policies are clear as far as the content of the promotional claims is concerned, stating that the claims must not be exaggerated and must be accompanied by contraindications and adverse effects. However, there are no well-defined guidelines for the regulation of the references cited in drug advertising brochures, creating grey areas that enable manipulative companies to exploit evidence-based advertising. Furthermore, pharmaceutical companies are not required to obtain approval for the content of advertisements from the Pakistan Ministry of Health. These problems are compounded by the absence of a proper monitoring system and lack of punitive measures in cases of non-compliance with drug advertising regulations in Pakistan. References that accompany advertising brochures have been criticized for inadequacy and inaccuracy [9]. The usefulness of these references is undeniable, as they serve as means for critical assessment of the claims. However, studies have shown that most of the references do not justify the use of the drug [10, 11]. Practitioners in developing world, who are still not effectively trained in evidence-based medicine, are very

vulnerable to such promotional tactics. Hence such marketing, though a problem of the developed world as well, is a bigger dilemma for the developing world. There is a potential for inappropriate prescribing practices, which contribute, not only to escalating national health care costs, but also increased frequency of adverse effects and drug resistance. Thus, in view of the serious impact advertisements have on drug prescriptions [12, 13], the aim of our study was to assess the quality of references that accompany promotional claims in advertising brochures in Pakistan, and to further analyze the accessibility, accuracy and validity of the quoted references. Our findings may then contribute to a more comprehensive understanding of promotional marketing and continued improvement in standards in the developing world.

Methods Brochures were collected from 45 privately practicing General Practitioners’ (GPs) offices distributed in various parts of Karachi from July 1 to 10, 2006. Since, in Pakistan, no database exists of the practicing GPs, randomization was not possible. However, it was consciously ensured that all quarters of the city were covered and the sampled offices were not concentrated in a particular location. A convenient sample of 175 different brochures was procured, with an average of about 4 (range; 2–6) brochures per physician. Since no data exists as to the total number and kind of advertising brochures distributed to physicians over a fixed span of time, our expectation was that a sample size of 175 collected over 10 days was representative of the advertising pamphlets physicians received. From the set of collected brochures, those that did not cite any references (n = 39) were not subjected to further analysis. A protocol was then established for the 3 reviewers (DKR, MUB and MRN) to follow during assessment and analysis of the citations quoted in the remaining brochures as shown in Fig. 1. Following categorization into sources of references, each reviewer was then asked to trace the cited reference and validate its accessibility. Each reviewer tried separately to trace each reference using all available databases. These databases included Pubmed, library catalogues and the internet. All the searching was done in the library of Aga Khan University, with an access to almost all of indexed journals. In case of inaccessibility to the full text of the paper, Abstracts were retrieved from the Pubmed. The reviewers were then able to divide the references into ‘traceable’ and ‘non-traceable’. A reference was adjudged non-traceable when it was either not available in the database or was incomplete and hence could not be


A ll b r o c h u r e s c o lle c te d

Pharm World Sci (2008) 30:216–221

R e fe r e n c e c ite d Medline -indexed Journals

R efe r e n c e s N o t C ite d

A cc u r a t e
I n a c c u r a te / F a ls e T r a c e a b le

Non-indexed Journals
M e d ic a l R e fe r e n c e B o o k s W eb A d d resses

Exaggerated Ambiguous

N o n -tr a c e a b le
Personal Communication / Testimonial

W H O / N a ti o n a l H ea lt h G u i de li n e

D a ta o n F ile

O th e r s

Fig. 1 Flow chart showing the methodology of analysis of the drug references

accessed. These were only judged incomplete when they could not be accessed because of a lack of one or more of the following requirements of standard bibliographic reference: author’s name, journal’s title, page number, journal’s year of publication, journal’s issue, and volume or supplement number. Despite deficiencies in standard requirements, some references were traced through trial and error; such references were then classified into incomplete but also into one of the justifiable/inaccurate/ exaggerated/ambiguous classes. The reviewers also encountered situations where the journal’s title or author’s name had been misspelled. Such discrepancies were ignored when a provision existed in the database for correcting a misspelled word, such as in Pubmed. However, when the misspelling was so blatant that it impeded the searching, that reference was included in the category of incomplete. All traceable references were further analyzed by each of the reviewers separately and based on the analysis were categorized into 4 classes according to the following definitions: (1) Justifiable: A reference to a claim was adjudged ‘justifiable’ when it accurately supported the claim made. Inaccurate/False: A reference was labeled as ‘inaccurate or false’ when the information contained within the reference could not validate the claim. For example, a claim in a brochure advertising lansoprazole claimed that the drug had 4 times




greater bacteriostatic effect against H.pylori than omeprazole. Analysis of the reference cited for the claim proved that the actual study undertaken dealt with triple therapy involving metronidazole, lansoprazole, clarithromycin or amoxicillin and their efficacy in the eradication of H.pylori; the study did not involve a straight forward comparison of the efficacy of omeprazole and lansoprazole. Exaggerated: A reference was adjudged ‘exaggerated’ when the study or information in the reference was not in proportion to the claim; it lacked one of several factors that the claim highlighted. For example, a brochure advertising co-trimoxazole cited the results of a study on the efficacy of co-trimexazole in treating shigellosis, but instead of mentioning the limitation of the research, the advertisement claimed that thr drug in question could be used for any diarrhoeal disease. Ambiguous: Finally, a reference was adjudged ‘ambiguous’ when the information in the reference was presented in vague manner such that the reviewers found it difficult to clearly link the information contained in the reference to the claim. For example, a brochure advertising sibutramine claimed that it did not disrupt the normal eating cycle without specifying what the normal eating cycle was and what kind of hormones and chemicals were involved. The study quoted in reference to the claim described the mechanism of action of sibutramine. Hence, no distinct relationship could be determined. A few


Pharm World Sci (2008) 30:216–221 Table 1 Sources of citations with analysis of claims Source


n (% of 559 citations Incomplete/nonTraceable citations analyzed) traceable citations (%)* Justifiable Inaccurate Exaggerated Ambiguous claims claims claims claims n (% of traceable citations) 305 (54.6) 67 (12.0) 55 (9.9) 27 (4.8) 12 (2.1) 12 (2.1) 11 (2.0) 8 (1.4) 62 (11.1) 559 (100) 94 (30.8) 41 (61.2) 17 (30.9) 19 (70.4) 2 (16.7) 7 (58.3) 4 (36.4) 8 (100.0) 57 (78.1) 249 (44.5%) Total = 570*** 147 (69.7) 6 (23.1) 24 (63.2) 5 (62.5) 2 (20.0) 4 (80.0) 2 (28.6) 0 7 (43.7) 14 (6.6) 6 (23.1) 2 (5.3) 0 1 (10.0) 0 3 (42.9) 0 4 (25.0) 46 (21.8) 13 (50.0) 8 (21.1) 2 (25.0) 5 (50.0) 1 (20.0) 2 (28.6) 0 2 (12.5) 79 (25.5%) 4 (1.9) 1 (3.8) 4 (10.5) 1 (12.5) 2 (20.0) 0 0 0 3 (18.8) 15 (4.8%)**

Pubmed indexed Non-pubmed indexed Books Internet WHO & National Health Guidelines Professional communication Symposia Data on file Others

197 (63.5%) 30 (9.7%)

*Refers to number and percentage of incomplete/non-traceable citations in that particular source **The total number of traceable references in the above categories adds up to be more than 310 because at times, a single reference was quoted for more than one claim and in such situations the reference was treated separately for each claim ***The total for traceable and non-traceable references also adds to be more that 559 because some references, although ‘incomplete’ according to standard reference citations, were traced and analyzed as explained in the ‘Methods’ section

claims were encountered that included reference to papers published in non-English language journals with no translated abstracts or full text translations available. Keeping in mind the fact that practitioners in Pakistan are highly unlikely to be fluent in German or Russian, these references were classified as ambiguous too, as there was no way to validate their authenticity. Hence, first we described the references cited, and then characterized each reference in the light of its promotional claim. Finally, when the reviewer thought that a claim was not exactly supported by the reference cited, the reasons were described.

monographs, non-indexed journals or non-published data on file. 137 (55%) of the 249 were incomplete or misprinted. Traceable references numbered 310 (55.5%). These were further classified as defined in the methods. Upon analysis, traceable references were adjudged as justifiable (197; 63.5%), inaccurate/false (30; 9.7%), exaggerated (79; 25.5%), or ambiguous (15; 4.8%). The breakdown of the analysis of claims is shown in Table 1. Representative examples of categories of claims are presented in Table 2.

Discussion Pharmaceutical companies spend huge sums on marketing in the developed world, with immense amounts targeting clinicians [14]. Similar trends exist in the developing world. Since no databases are available to analyze such trends in developing countries, it is difficult to verify them. Data gathered from international studies clearly prove that it would, indeed, be a mistake to regard drug advertisements as trivial, as was once believed [15]. A previous study has shown that medical practitioners rely on the pharmaceutical industry for much of their drug information in Pakistan [4]. Despite apprehensions about the truthfulness of the advertised claims, Pakistani GPs rate pharmaceutical advertisements as their most important source of information [1].

Results A total of 175 different brochures were collected. Thirtynine did not cite any reference, whereas 136 contained a total of 559 references. Breakdown of the source of citation is depicted in Table 1. The references classified under ‘others’ are those for which no single class definition could be determined. These references quoted newspapers, various circulations, ‘physician desk references’ and a number of product bulletins and newsletters. Of the references identified, we could not retrieve 249 (44.5%). Out of the 249, 112 (45.0%) were not found to exist on the available databases because they were from



Pharm World Sci (2008) 30:216–221

Table 2 Some examples of inaccurate, exaggerated and ambiguous extrapolation of references in drug advertisements Brand name Generic name Urocit-K Potassium citrate Claim/statement Reference (as written on the brochure) Comments False—there is not a single reference to the B.I.D. dosing in the article.

B.I.D. dosing provides therapeutic Nicar MJ, Peterson R, Pak CY. coverage for the all important Use of potassium citrate as midnight and early morning potassium supplement during periods. thiazide therapy of calcium nephrolithiasis. J Urol 1984;131(3):430–3 Unique mode of action

Vastarel MR Trimetazidine

Kantor PF, Lucien A, Kozak R, Exaggerated—in the article, Lopaschuk GD. The antianginal authors suggested a possible drug trimetazidine shifts cardiac mode of action with the energy metabolism from fatty conclusion that the exact acid oxidation to glucose mechanism is not known. oxidation by inhibiting mitochondrial long-chain 3– ketoacyl coenzyme A thiolase. Circ Res 2000; 86(5):580–8.



In the treatment of complicated Infection. 1990; 18 Suppl 3: S132– Incomplete—information given on UTI CEFSEL (Cefixime) is 9 the brochure is not enough to effective antibiotic if infections search the article on the internet. are caused by sensitive strains. GLA is growth inhibitory both in Pancreotology. 2003 vitro and in vivo, at doses nontoxic to non-cancer cells. Incomplete and true—the address was incomplete but if we can limit our search with the word GLA and 2003, we could find an article which contains this exact statement.




Crotamiton/sulphur Crotamiton is the only scabicide Konstantinov D, Stanoeva L, Justifiable—exact statement is also available today which displays Yawalkar SJ. Crotamiton cream there in the article, with the not only antiprurtic but also and lotion in the treatment of same meaning. marked antibacterial properties. infants and young children with scabies. J Int Med Res 1979;7(5):443–8 Insulin Life expectancy reduced by 8– 10 years in diabetic patients. Koskinen SV, Reunanen AR, False—article dealt with relative Martelin TP, Valkonen T. mortality but not with life Mortality in a large populationexpectancy. based cohort of patients with drug-treated diabetes mellitus. Am J Public Health. 1998; 88(5):765–70 Coscelli C, Lostia S, Lunetta M, Nosari I, Coronel GA. Safety, efficacy, acceptability of a prefilled insulin pen in diabetic patients over 60 years old. Diabetes Res Clin Pract. 1995;28(3):173–7 Plempel M (1981) Expert Report Microbiology Mycospor Exaggerated—study was about elderly patients above 60 years of age but the statement on the brochure was generalized.




Easy to teach and simple to use.



Active substances concentrate in individual layers…

Ambiguous—The article was in German language and we couldn’t find the related content in the article’s translated abstract.

The pharmaceutical industry has added evidence-based medicine to its promotional strategies. Though this is in line with WHO guidelines which state that all claims in drug advertisements be supported by suitable documentary evidence, questions have repeatedly arisen as to how much references cited in the advertising brochures actually

correspond to this guideline. The findings of our study show that although 77% of brochures did cite references, 45% of the references were not traceable, hence of little value to any practitioner who would wish to verify a claim. The fact that 55% of the references were cited from Pubmed indexed journals and the finding that a large number of the references


Pharm World Sci (2008) 30:216–221


were not traceable suggest that the references were cited more to add an impression of authenticity to the claims rather than to offer a means for research on the part of the practitioner. Quoting references from a published clinical trial and expressing the results of the trial in usually attractive graphs in a drug brochure serves to influence physicians into changing their prescribing behaviors. During analysis of the references classified as exaggerated, we found that at times the cited reference included a study involving a particular group of people, such as a geriatric or pediatric group, but the advertisement generalized the results to an entire population. Further, most of the claims that cited textbook references were a few chosen statements that were presented out of context. These statements highlighted the drug’s efficacy, convenience or usefulness, while adverse effects, important warnings and contraindications were ignored. Taking into account the inaccessibility of 44.5% of the citations, justification of only 197 (35.2%) of 559 claims could be confirmed. This small number is disturbing as it indicates that pharmaceutical companies were blatantly exploiting the biomedical literature to substantiate claims. This also highlights the fact that drug advertising in Pakistan is not properly regulated and adherence to established guidelines is not being monitored. The objective of our study was to analyze and assess claims in light of references provided. However, assessment of the truthfulness of the promotional claims itself is complex and beyond the scope of this study. For this, each reference needs to be considered separately and researched thoroughly. Thus in our view, some of the non-traceable references could have substantiated claims. Furthermore, some of the claims could have been accurate, but were supported by papers of poor methodological quality.

courses that highlight the importance of evidence-based medicine during the undergraduate medical school years would be one step that could thwart manipulative advertising. However, the most effective measures would be those that specify well-defined policies for regulation of advertising content and for which there is also a monitoring system in place to oversee the implementation of the guidelines.
Acknowledgement The authors wish to thank the Department of Biological and Biomedical sciences, Aga Khan University, Karachi for bearing the costs of this study and publication.
Funding No funding received. None declared.

Conflicts of Interest

1. Rohra DK, Gilani AH, Memon IK, et al. Critical evaluation of the claims made by pharmaceutical companies in drug promotional material in Pakistan. J Pharm Pharm Sci 2006;9:50–9. 2. Silverman M, Lee PR, Lydecker M. The drugging of the Third World. Int J Health Serv 1982;12:585–96. 3. Lee PR, Lurie P, Silverman MM, Lydecker M. Drug promotion and labeling in developing countries: an update. J Clin Epidemiol 1991;44 Suppl 2:S49–55. 4. Rohra DK, Jawaid A, Rehman T, Sukkurwala AQ, Palanpurwala AS, Gangwani R. Prescription of new drugs by general practitioners in Pakistan: an exploration into information sources, prescription influences and general attitudes. Pak J Med Res 2007;46:5–10. 5. Levy R. The role and value of pharmaceutical marketing. Arch Fam Med 1994;3:327–32. 6. Accessed on July 10, 2007. 7. Accessed on July 10, 2007. 8. = SS_GET_PAGE &nodeId = 165/. Accessed on July 10, 2007. 9. Gutknecht DR. Evidence-based advertising? A survey of four major journals. J Am Board Fam Pract 2001;14:197–200. 10. Bero LA, Galbraith A, Rennie D. The publication of sponsors symposiums in medical journals. N Engl J Med 1992; 327:1135–40. 11. Wilkes MS, Doblin BH, Shapiro MF. Pharmaceutical advertisements in leading medical journals: experts’ assessments. Ann Intern Med 1992;116:912–9. 12. Gonul F. Promotion of prescription drugs and its impact on the physician’s choice behaviour. J Market 2001;216:79–90. 13. Jones MI, Greenfield SM, Bradley CP. Prescribing new drugs: qualitative study of influences on consultants and general practitioners. Br Med J 2001;323:378–81. 14. Health IMS. Total US promotional spend by type, 2004, 2005. [,2777,6599_ 49695992_75406357,00,htm]. Accessed on July 30, 2006). 15. Herxheimer A, Stalsby Lundborg C, Westerholm B. Advertisements for medicines in medical journals; a collaborative international study. Report for director-general of World Health Organisation, Sweden: WHO,1992.

Conclusion and suggestions In conclusion, the results of this study show for the first time that references quoted in support of claims in pharmaceutical advertisements are highly unreliable in Pakistan. Marketing practices adopted by the pharmaceutical companies need to be closely monitored and laws governing drug marketing currently in place should be properly enforced. It is recommended that the content of the advertisement of drugs should first be approved by the Ministry of Health. Doctors should also critically analyze the references that accompany drug claims in brochures before accepting them as a reliable proof. This, however, may be difficult for Third World doctors, since physicians in developing nations are often overworked and time-constrained. Most physicians, as pointed out earlier, are also not conversant with the concept of evidencebased advertising, as no such courses exist in undergraduate or postgraduate medical education in Pakistan. Inclusion of