This action might not be possible to undo. Are you sure you want to continue?
Robert M. May Office of Science & Technology, Albany House, 94-98 Petty France, London SW1H 9ST, UK.
The United States took much pleasure last summer from its performance in the Olympic Games, where it w than any other country. But was this the right measure of performance? Counting four points for gold, two f bronze, and calculating the score relative to population size, a different picture emerges. Tiny Tonga was fir among the larger economies, and overall the United States ranked 37th, well behind most of the European co United Kingdom, a lamentable 48th). Similar questions arise when we ask about the quality of the scientific research output of countries purposes, most notably overall advance in our understanding of nature, it is total output that matte purposes--for example, in producing trained people or for underpinning industrial advances--outpu size [measured by population, gross domestic product (GDP), or other things] is more relevant. In this paper I offer comparisons, from a variety of viewpoints, of scientific research outputs amon I derive my analysis from a recent United Kingdom benchmarking study (1), which in turn draws he work (2-4), particularly an analysis of Australian research (5). These studies are all based largely o Citation Index (SCI), established by the Institute for Scientific Information (ISI). This database cove research publications from 79 countries and more than 4000 journals since 1981. The database h shortcomings and biases (6), but overall it gives a wide coverage of most fields. We studied the 14 1981 to 1994, in which the ISI database totaled 8.4 million papers and 72 million citations.
Publications and Citations
The top 15 countries, ranked by the contribution of their scientists to the world's total number of publication engineering, and medicine, from 1981 to 1994 (7, 8) accounted for 81.3% of the world's papers (Table 1). T were the world's seven largest economies, the so-called G7 countries. The United States was dominant, publ the world's science. The United Kingdom was second, ahead of more populous countries like Germany, Fran 15 countries that constitute the European Union (EU) rivaled the United States and produced ~32% of all pa
Table 1. The world's top countries, ranked by their share of the world's papers in science, medic engineering (6). The table also shows citation shares, RCI (9), expenditure on R&D (22), and a m effectiveness: citations per unit of government expenditure on R&D in G7 countries (in boldface) are for the yearly average for the period from 1981 to 1984; expenditures are in £million and for %GDP spent on R&D are for 1994, except for 1993 for Netherlands, Denmark, and Sweden and and Switzerland.
Share of papers
Share of citations
GDP spent on R&D
United States United Kingdom Japan
US UK JP
34.6% 8.0 7.3
49.0% 9.1 5.7
1.42 (1) 1.14 (5) 0.78 (18)
2.5% 2.2 2.9
1 0.5 1.9 2.5% 2.5 2. which are also the five largest economies. RCI [citations divided by publications (6. In terms of the RCI.1 1.9 0.78 (18) 0.2 0.3 2. percentages ranged from 70% (in psychology) to 37% (in United Kingdom was second in 15 of the 20 fields. China Denmark Finland US UK JP GE FR CA IT IN AU NE SE SW PR DE FN 34.16 (4) 0.9 (12) 2.0 4.0 1.4 1. the United States still ranks first (10).9 3.7 6.7 49. the rankings are similar except for I relative citation impact.37 (2) 0. The above analysis can be broken down by field (Table ) (7).75 (19) 0. In low R&D investment and also have a low RCI.42 (1) 1.27 (66) 0.87 (14) 1.1 5.7 1. and Canada) rank in the top 10 by RCI.5 2. and the top 15 countries by publi include the top 8 by RCI.7 2.14 (5) 0.86 (15) 0. the fourth is France.6% 8.9 0.6 1. The top five countries by publication shares.0 7.1 2. 9)]. Only three of the G7 countries (United S Kingdom.24 (3) 1.8 0. and others. although significantly reshuffled.7 1. The smaller countries wit notably Switzerland and Sweden (second and third by RCI). gives some measure of the qu paper. although the United States still showed strongly.1 2.10 (6) 1.3 2.R. at 14th. followe Sweden.27 (65) 1.4 0.0 0.0 5.2 4. The ranking when the quality measure of RCI is used.2 2.4 In terms of these countries' percentage shares of all citations. are relatively high investors in R&D.97 (8) 1.3 1.00 (7) 0.7 0.7 2.United States United Kingdom Japan Germany France Canada Italy India Australia Netherlands Sweden Switzerland P.4 2. the United States rank shares in all 20 fields we discriminated.6 1. .2 2. and placed lowest (fifth) in physics.8 2.0% 9.6 1.5 4. As expected. invest pro research and development (R&D) than do most other countries (Table ). United Kingdom.3 7.
FR. a comparative advantage is revealed.IS US.UK. BE.NO.UK.CA.GE.SW.UK.UK.26 US. Mathematics Microbiology Molec.DE. if the R 1.SE.UK. science base (12).DE.CA SE. This with a separate recent review of the U.96 US.US. Also shown are the top fiv ranked by share of the world's citations and by RCI in each of the 20 fields.AU US. and vice versa.CH.SE.SW.BE.FR 1.UK.GE. RCI 20 fields of science defined by the ISI (6.JP.IS SW. 23).02 US.FR 1.SW.13 US.US.SW.05 1. Engineering Geosciences Immunology Materials sci.GE.AU.FR.JP.GE 1. Top five countries Field RCA RCI By total citations Agriculture Astrophysics Biol.AU DE.GE By SE.US.SE.CA.SE US.UK.US.FR 1.FR 1.03 1.CA.UK.13 US.22 US. Psychology 1.04 US.CA.27 0.SE.US.DE.UK.FR.UK.NZ.CA.FR.SE.GE.GE 1.NE.CA.GE 0. As shown in Table . Abbreviations are giv Fig. and has absolute strength across a broad range of scientific disciplines.CA. Chemistry Clinical medicine Computer sci.UK.US.UK.UK.DE SE.98 US. & environ.92 1.SW.CA. RCA (11).GE US.22 0.FR. for the United Kindom.UK.CA. biol.13 US.CA 1.JP.SW.39 US.IS.05 US.NE.UK.99 1.05 US.18 0.DE SW. Ecol.UK.GE.GE.87 1.US. Belgium.CA 1.FR 0.96 0. .NE US.GE. compared with the world average.10 US. relative to the fraction of the world's citations that are in that field (11).05 0.79 0.DE IS.67 1. The Australian study (5) defined a country's "revealed c advantage" (RCA) in a specified scientific field as the fraction of all that country's citations (or pape particular field.UK.UK. & biochem.DE.NE.UK.09 1. and a measure of absolute quality.UK.SW.UK.CA.GE. on the basis of this in Kingdom has a strong comparative advantage in pharmacology.NE US.65 0.GE.US.37 US.SW.09 US.JP.GE 1. 7.JP.AU.SE US.UK.GE.IS.FR.GE.SW.CA.DE.JP.AU US. Thus.SE.NE.SW DE.13 US.AU 0.12 1.K.80 0.JP.11 US.UK.UK.GE.UK.JP.IS UK.UK. clinical medicine.CA 1.USSR.US.GE.69 US.UK. Comparative advantage in citations.UK US.GE.GE.AU 1.DE.Table 2.JP.FR 1.NE. & genetics Multidisciplinary Neuroscience Pharmacology Physics Plant & animal sci.UK.CA Bibliometric analysis can also uncover patterns of relative investment or relative advantage of a co subject. it appears that the United Kingdom is strong in bi biomedical research. In view of both the RCI and RCA.NO.06 0.US.FR 1.07 1.87 1.AU.UK.96 0. 1.JP 1.CA 1.SE SW.FR.FR SW.44 US. plant and anima neuroscience.UK.56 US.UK 1.SW.
This with a separate recent review of the U. and Japan come low in the rankings.K. and Germany. Switz. As in the opening analogy with the Olympic Games. Switz. science has thresholds of critical siz Table 3. it appears that the United Kingdom is strong in bi biomedical research. and United States (16). the top 12 countries ranked accord person are smaller. Similar analyses for other countries (5) suggest that some smaller European countries (Denmark. Some countries (Au New Zealand. chemistry. science base (12). China. we ask about performance in rela size. Netherlands. mainly northern European countries. Netherlands. and South Korea). Country Country C countries are in parentheses. plant and anima neuroscience.. Similarly.Paps. This is understandable.Kingdom has a strong comparative advantage in pharmacology.. The United followed by Germany. Top 12 countries are listed first for each index. the top G7 country is Canada. in contrast. cits. fol Kingdom at eighth and United States at ninth. computing. It is not surprisin with larger economies show such relative evenness. Most of the Asian countries that are emerging as scientific powers. Unlike the case for Olympic medals. Israel 167 152 Switz. Switzerland. New Zealand. Measures of relative performance (1. Consideration of a country's share of the world's total papers or citations tends to focus attention o economies. is followed by the United Kingdom. Japan. United States. and has absolute strength across a broad range of scientific disciplines. T above and in Table indicate that the distribution of the data for the 20 fields for the United Kingdom clustered than for any other country. per person Switzerland. and materials). Netherlands. 5). but the presence of small countries such as t Switzerland (where many would expect preponderant investment in pharmacologically related rese surprising. Switzerland) have prominence in biomedical research. Singapore. Thailand. Italy. France. citations. Sweden . In view of both the RCI and RCA. On this basis (Table ). or money spent on basic research. The Asian economies have prominence in certain industries (such as engineering. A similar pattern holds in terms of citations per pers Germany. Neth. The greatest evenness in quality. Indonesia. A measure of evenness of a country's scientific effort is given by the variance of the distribution of papers or citations) for the 20 fields specified by the ISI (13). for oth scientifically strong United States. N. Finland. The United Kingdom has the lowest s papers) or greatest evenness in its patterns of scientific capability among the 20 fields. France. Whether this is a good thing is another question. Philippines. Canada.. have un example. Zealand. and the scientifically weak Papua Thailand) no particular pattern of specialization emerges. by this measure. and France (14). and South Africa) have prominence on research based on natural resources. at fifth. clinical medicine. there is performance by a very small country. evenness in quality can be estimated by calculating the variance in the appropriately no of a country's RCIs among the 20 fields (15).
we note States was ahead of the United Kingdom at the start. 2.S. The United Kingdom does particularly well in these performance ratio because the number of citations of papers by U. A similar. The United States' share of world papers decreased by 1.K. U. and France. 5 terms of publications relative to population size (Table ). of the top three G7 countries (United Kingdom.S.7% an increased by 2. are rescaled ag Patterns of Change From 1981 to 1994. the United States ranked first. Canada) and the other four. and Germany's by 0. >10% per year. Finland Iceland Norway Australia France (15) Germany (16) Japan (19) Italy (20) Comparison of scientific output relative to government money spent on research and developmen and excluding defense funds) is arguably the best measure of the cost-effectiveness of spending i and strategic research (Table . There is a marked gap--by a factor of 3 or more--be measured in citations per unit of civil expenditure.9%. Germany. with the exception of the Netherlands Germany from the top 12) and also with minor reordering in the rankings (chauvinistically. were exhibited by the scientifically emerging countries China. decrease in wo .7% per year.7%. 3. Zealand Norway Australia France (15) Germany (17) Japan (19) Italy (21) 152 147 127 127 109 107 104 100 99 96 93 72 67 49 41 Sweden Israel Denmark U. however. though still average growth in publications: United States. the world's output of scientific papers increased by 3. behind at the end).4%. Japan third. scientists is relatively high and partly because t on R&D is relatively low. Singapore. Finland U. absolute rankings have not changed much. Canada U. South Korea. the countries that were already the leaders in 1981 growth. gap is also seen if total government e as the scaling factor [and similar patterns are seen if papers. and Taiwan. the top 12 c were top at both the beginning and end of the 14-year span. The U.K. The scientifically established countries had lower. This rate correspond 19 years. 3.K. In terms of papers per population.S. In terms of share total papers. although slightly smaller. Overall.Israel Sweden Denmark Canada Neth. United Kingdom. The greatest growth rates. Neth. rather than citations. Such increases in output from newer players have meant that the United States. scientifically developed Western countries have seen their share of the world's papers decrease so 1981 to 1994. whereas France's share increased by 0. last columns). and Germany fou beginning and at the end of the 1981-1994 interval.0%. United Kingdom second.0% per year from 1981 to Kingdom's share by 0. United Kingdom. N.1%.3%.
SK. NO.1% per year. New Zealand. 1. Up to the past few dec essentially solely of Nobels. Although still the best known and richest.9% per year. Mexico. South Korea. Malaysia. But such considerations seem to us unlikely to introduce tim than a couple of years [a supposition supported by the average half-life of citations (17)]. Singapore. for the larger scientifically developed. Phillipines. while U. In the decades around World War II. Japan third. or more years earlier. proportionally French scientists won. or in citations or papers per tended to show little change. German scientists won most of the awards in the early decades. and Sweden. because citations ac (17). MX. The greatest gain among the top 12 countries in Table was by New Z showed an average annual growth in RCI of 1.000 (United S mathematics' Fields Medal (18). and Germany fou beginning and at the end of the 1981-1994 interval. 2). NZ. only phy medicine (broadly construed as the biomedical end of the life sciences) are recognized. Thailand. Norway. the emerging As mentioned above tended to show large increases in RCI over the 14a relatively low base. RCI increased at 0. In general. ID. all of about 0. countries.8 the United Kingdom's share decreased 1. that the relative paucity of Nobel prizes won by U. South Africa.4% per year. Abbreviations are in Chile. P PH. Change in the inherent quality of a country's science from 1981 to 19 indicated by changes in the RCI. Fig. TA. the United States ranked first. 20.S. Its scope is. moreover. SA. We thus counted all internationally recognized scientific prizes worth more than $200. United Kingdom second.total papers. for example. As shown in Fig. all else being equal. measured as average percentage change in RCI (9) versus (average 1981-1985). HK. [View Larg Image (14K GIF file)] Corresponding analyses of trends in citations need to be interpreted with care. Another way of assessing a country's science base is to look at its success in winning major interna often asserted. with the exception of the Netherlands Germany from the top 12) and also with minor reordering in the ranki we note that the United States was ahead of the United Kingdom at t the end).K. Other countries started the period with low RCI further back. A different researchers who did this work were trained 10. the top 12 c were top at both the beginning and end of the 14-year span. The U. the Nobel Prize is growing number of notable awards. PN. Today's performance may well the new generations of scientists are being nurtured [see (1) for further discussion]. SI. Taiwan.S. decrease in world share of citations was about 0. there is thus little suggestion of any marked de of the scientific output of the top-ranking countries in Table . scientists over the evidence of declining scientific strength. and sc States began to win many awards in the 1930s. In terms of papers per population. The countries that rank highest in average RCI (Table ).1% per year in the United Sta at 0. Hong Kong.2% per year in the United Kingdom. The greatest declines were for Austra Norway. papers published in the early 1980s (or earlier) should contribute m than those published in the early 1990s. 1. scientists established a continuing command of around half of the Scientists in the United Kingdom have maintained a steady fraction of about 10% of all awards thro this total is rescaled for population size. Indonesia. Thus. the United Kingdom has been the leader throughout the ce . TH. by decade through the 20th century (Fig. and developing. MA. Change in the quality of scientific output. restricted: among the sciences.
despite the rising performance of new players and the G5 countries' diminishing share of the world's papers and citations. proportionally fewer German and F while U. last columns. Italy. and France none. If this total is rescaled for populatio Kingdom has been the leader throughout the century. last columns)? But the same broad patterns of performance among the G7 countries are also seen in the analysis in Table . Germany. Despite these concerns. scientists established a continuing command of around hal prizes.S. over the 20th century (1). both in the journals included and in patterns of citation. for example. but a large proportion do this work in dedicated research institutes: Max Planck and CNRS Institutes. German scientists won most of the award decades. Japan and United States each give two large p United Kingdom. Order of curves is as listed in the legen Version of this Image (28K GIF file)] Analysis of prizes gives a time-delayed measure of performance. Could this explain why the United States. has been won largely by Australians. 26% of papers with first authors in the United Kingdom were the product of transnational collaborations (4).essentially solely of Nobels. Another concern is that there is an English language bias in the ISI database. United Kingdom. The figure shows the fraction of the world's major international science prizes (18) won by each of the (the G5 countries and Australia). and Japan (Table . Scientists in the United Kingdom have maintained a steady fra all awards throughout the century. 2. By contrast. Germany and France have superb scientists who do outstanding work. In 1994. Fig. and scientists in the United States began to win many awar the decades around World War II. This consideration may explain why the G5 countries have continued to win about 80% of the awards over the past three decades. by decade. and Canada do so much better than France. is . where the four leaders are not the English-speaking countries. most basic research in North America. One bias in this analysis is that one recent lar Australia Prize. the Scandinavian countries. are surprising. United Kingdom. Discussion The above comparisons are to a degree confounded because a large and growing fraction of scientific work involves international collaborations [see discussion in (8)]. and others among the top countries in Table . the large differences in performance indicated in Table . Germany. My view--and it is no more than a guess--is that a large part of the difference in performance between the top dozen or so countries in Table and the lower ranking of the G7 countries arises from differences in the nature of the institutional settings where the scientific research is done.
. in press). such as the United Kingdom. research libraries. UK. by forging new connections between the two. 2. The peace and quiet to focus on a mission in a research institute. The nonhierarchical nature of most North American and northern European universities. S. and so on) to all departments in these more numerous universities. among the scientifically advanced countries. London. 1995). 3. rather than in research institutes. Ed. Katz et al. may be a questionable blessing. J. coupled with the pervasive presence of irreverent young undergraduate and postgraduate students. World Science Report 1996 (Unesco. Paris. (UK Office of Science and Technology. October. University of Sussex. I thus suggest that. although it helps create wealth around the world. 1994). The Technology Foresight enterprise in the United Kingdom currently seeks to remedy this. More generally. This raises questions about how to focus such infrastructure support and indirect costs upon the best people and groups. Sussex. and the accessibility. is not consistently translated into strong industrial performance within the United Kingdom itself. To end on a parochial note. This issue deserves further analysis. It seems unlikely that governments can afford to supply the previously customary level of infrastructure for research (equipment. there are significant implications for those countries. I observe that the United Kingdom does well in attracting inward investment (20). 1994). I think the alternative of hiving off most fundamental research into dedicated institutes could be a suboptimal solution. the acknowledged strength of the science base in the United Kingdom. better value for money (in terms of papers or citations per person or per dollar spent) might be associated with performing basic research mostly in universities. from two or more institutions. in the United Kingdom. Luxembourg. undistracted by teaching or other responsibilities.done in universities (19). If so. of the science base is a large factor in this success. Australian Science: . could be the best environment for productive research. but a convincing and objective analysis is more difficult. and Australia. The Changing Shape of British Science (Science Policy Research Unit. properly furnished laboratories. Sweden. REFERENCES AND NOTES 1. Anecdotes abound. technicians. It is believed that the strength. The question is sharpened by the observation that. 5. which recently have seen great enlargement in the number of tertiary institutions designated as universities. Moore. European Report on Science and Technology Indicators 1994 (European Commission Publications. Analysis of the Quality of the UK Science Base. H. by the year 2000 half of all scientific papers will have three or more authors. 2. But this is another story (21).
for country i. but [following (5)] these are omitted from our analysis. Canada 4. Citation Indexing: Its Theory and Application in Science. For country i. is RCIi multiplied by the ratio of all citations to all papers (namely 8. interesting errors can. Here. while pathbreaking papers may be cited only slightly for many years. Spectacular scientific errors may attract many citations. Journals from nonEnglish-speaking countries are not as well represented as those from Englishspeaking countries. and E. moreover. Other publications.3. average citation rates are far higher in molecular biology and genetics than in materials science). It includes citations of books and chapters in edited books.1%. and Italy 3. Citation patterns vary among fields (for example. Germany 6. These proportions changed relatively little from 1981 to 1993 and may be compared with the corresponding figures in Table . Self-citation (which accounts for at least 10% of all citations) may bias some of the results. the ISI database preserves the 1981 list of countries through time. Other problems are associated with citation patterns. Bourke and L. Technology and Humanities (Wiley. Japan 8. but it does not include the citations in such publications. The EU study (2) attributes fractions of a paper among countries in proportion to authorial numbers. There are several ways of counting authors of collaborative papers.3%. the breakup of the USSR). Papers that describe technical methods may attract thousands of reflexive citations. ignoring changes in boundaries. Review articles can mask the primary papers they review. Ci/Pi . see (2). 1996). the average number of citations per paper. Some problems have to do with the compilation of the database.0%. 6. let total papers be Pi and citations Ci . many Australian journals are not covered (5)]. That is.7%.2%. The share of the world's papers is then pi = Pi /P. spark new ideas and developments in science. following (5). Canberra. New York. A survey of different accounting methods by P.5%. For a more detailed discussion of these questions. It does not cover all significant scientific journals [for example. (5). Similarly. In order that percentage shares add up to 100. such as government and other agency reports and working papers.8%. such papers are attributed to each of the countries represented among the authors. Hence RCIi = ci/pi = (Ci /Pi)(P/C). United Kingdom 8. . There are problems with shifting definitions of countries (for example. although we think this may distort overall counts less than is sometimes suggested. Garfield. Butler [A Crisis for Australian Science? (Australian National University Press. 4. The ISI database has many acknowledged biases and limitations. 1994)] suggests that these various approaches have relatively little effect on conclusions. 1979). 5. where P = Pj is the worlds' total number of papers. citation share is ci = Ci /C. the 1993 world shares of papers for the G7 was the United States 35. are essentially omitted. The ISI database covers some additional fields of social science. Canberra. Yet another method is simply to assign papers to first authors. The United Kingdom performs even better on this basis. even though this means some double counting. France 5. with C = Cj .53). the world total of papers in defined as the sum of papers across all countries. Performance from Published Papers (Australian Government Publishing Service.
The first column follows (5). Put another way. are narrow. accounting for the different values of their overall RCI. It has been suggested that U. as defined in (11). Data from ISI database. Bower (U. include many subfields. 8. this would tend to elevate the RCI for U. Kyoto (Japan). The Director General of Research Councils' Review of the Science Budget Portfolio (UK Office of Science and Technology. the United Kingdom receives roughly 17% of all outward investment from the United States. This bias is further complicated because the citation half-life of a paper tends to vary among disciplines. 12. including roughly 40% of all such investment from the United States. on average. around 11% from research council and government institutes. Define Pi(j) as the number of papers published in field j by country i . like chemistry or physics. London.2% of all publications in that field. We must use the variance of the distribution in the indices of revealed comparative advantage. King Faisal (Saudi Arabia). while others. 19. We do not simply use the distribution of RCI(j) by fields j. 13. 17. About 80% of UK papers come from universities and teaching hospitals. Here pi is the overall publications share of country i. 50% from South Korea. 5). Then the quantity Pi of (9) is Pi = Pi (j). as defined in (9). . Fields Medal. Australia.3b and discussion in (5). see also (1) and (3). I define RCAi(j) = [Pi(j)/Pi] [P(j)/P]. The United Kingdom accounts for more than a third of all inward investment into the EU. May 1995). See figure 2. which is a relative measure of performance. It follows that RCAi(j) = [Pi(j)/P(j)] pi(j)/pi. Volvo (Sweden). and shorter in subjects like molecular biology (average around 2 years). Such a measure of evenness of scientific capabilities cannot sensibly be obtained simply from the variation in citation or publication shares among fields because the ISI's 20 fields are themselves of variable breadth: some.). December 1996). like astronomy. and P is the overall total of (9). and 15% of Germany's. 7% of Japan's outward investment. 11. Figure 2 includes the following awards: Nobel (Sweden). where P(j) = Pi(j) is the world's total of papers in field j.7. based on RCI.S. London. scientists. Balzan (Italy). Draper (U.S. RCAi(j). 40% from Japan. Crafoord (Sweden). and 9% from industry (4). tend to be more insular than their European and other counterparts. we have excluded countries that contribute less than about 0. Wolf (Israel). 18. papers. 16. Japan.S. 9. 10. and pi(j) is the corresponding publications share in field j. See figure 2. Chorofas (Switzerland). 12. 14. tending to focus more on their own (United States) literature than is the case in other countries (5). but rather RCI(j) normalized by the overall citation impact of a country across all fields [in analogy to that in (13)]. Following (2. A Strategy Taking the Foresight Programme to the Millennium (Office of Science and Technology. Data mainly from (5).). UK Department of Trade and Industry.S. For the second column. If so.3a and discussion in (5). Such normalization allows us to make meaningful comparisons of variances among different countries. 15. being longer in subjects like ecology (average around 7 years).
one (or a few) highly cited multiauthored paper that has one author from a small country can produce a high RCI ranking for that country.Otherwise. we follow (5) in assigning multiauthored papers to each country with a contributing author (8). . which arguably is misleading.