You are on page 1of 14

A Macro Analysis of Productivity Differences Across Fields: Challenges in the Measurement of Scientific Publishing

Fredrik Niclas Piro, Dag W. Aksnes, and Kristoffer Rørstad

Nordic Institute for Studies in Innovation, Research and Education (NIFU), N-0302 Oslo, Norway.

E-mail: {fredrik.piro,dag.w.aksnes,kristoffer.rorstad}@nifu.no

While many studies have compared research productiv- ity across scientific fields, they have mostly focused on the “hard sciences,” in many cases due to limited pub- lication data for the “softer” disciplines; these studies have also typically been based on a small sample of researchers. In this study we use complete publication data for all researchers employed at Norwegian univer- sities over a 4-year period, linked to biographic data for each researcher. Using this detailed and complete data set, we compare research productivity between five main scientific domains (and subfields within them), across academic positions, and in terms of age and gender. The study’s key finding is that researchers from medicine, natural sciences, and technology are most productive when whole counts of publications are used, while researchers from the humanities and social sci- ences are most productive when article counts are frac- tionalized according to the total number of authors. The strong differences between these fields in publishing forms and patterns of coauthorship raise questions as to whether publication indicators can justifiably be used for comparison of productivity across scientific disciplines.

Introduction

The purpose of this study is to further the knowledge base about differences between scientific domains in levels of scientific/scholarly publishing productivity. The study is based on the total scientific publication output of 11,571 researchers, at the main Norwegian universities, over a 4-year period. Most previous productivity studies are restricted to the data available from larger bibliometric databases, such as the ISI Web of Science (Thomson Reuters). Such analyses often produce a distorted picture of the research output in some

Received April 11, 2012; revised May 25, 2012; accepted May 26, 2012

© 2013 ASIS&T Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/asi.22746

disciplines, in particular for the social sciences, arts, and humanities, where a substantial share of relevant books or journals is not indexed in the Web of Science (Hicks, 2004). The novelty of this study is the completeness and volume of publication data it draws on, and the level of detail included about academic employees in every disci- pline. Although our focus is on differences in average pro- ductivity between major areas of research at the macro level, researchers’ academic position, age, and gender are also considered, as they have been shown to be important vari- ables associated with productivity differences (Kyvik, 1991, 1993). We also investigate publishing practices (books, book chapters, and journal articles), coauthorship practices, and publication frequencies across scientific domains and subfields. These questions have been analyzed in several previous studies, but usually on the basis of more limited data sets; some studies cover a broad range of fields, but based on a much smaller sample (e.g., Dundar & Lewis, 1998; Shin & Cummings, 2010), some studies only cover selected disci- plines (e.g., Bourke & Butler, 1996; Costas, van Leeuwen, & Bordons, 2010; Seglen & Aksnes, 2000), while the study most comparable to ours in terms of sample size and cover- age of disciplines does not include books and book chapters (Archambault & Larivière, 2010). In Norway, scientific productivity has been investigated in several previous studies by Kyvik (e.g., Kyvik, 1991, 1993, 2003), but in contrast to our study, these studies used self-reported, nonverified data sets. Productivity studies at the individual level have shown that the number of publications per person depends on a wide range of factors, such as: age, gender, academic posi- tion and rank, the availability of research funds, teaching loads, equipment, research assistants, workload policies, departmental culture and working conditions, departmental size, and organizational context (Bonaccorsi & Daraio, 2005; Dundar & Lewis, 1998; Kyvik, 1993; Ramesh &

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, ••(••):••–••, 2013

Singh, 1998; Smeby & Try, 2005). Nevertheless, few studies consider disciplinary differences as a key factor for produc- tivity in their analytical models of research performance; in effect, they treat faculty demographics as control variables (Shin & Cummings, 2010, pp. 582–583). The completeness of our data set makes it possible to investigate productivity from the opposite point of view: research productivity is analyzed across scientific domains, using individual researcher characteristics as mediating variables.

Methods and Data

In this study we use publication data from the national Norwegian database, FRIDA. In 2004, Norway imple- mented a bibliometric model for performance-based funding of research institutions. These institutions’ budget alloca- tions are now partially based on their scientific and scholarly publishing, as documented in FRIDA. This database includes all types of scientific publications, in all fields of research in the higher education sector. Bibliographic data are collected through a common documentation system used by all institutions, resulting in complete, verifiable, and structured data for use in bibliometric analysis. In the FRIDA database, publication activity is reported by the institutions as standard bibliographic references, which are analyzable by publication channel and type of publication. A dynamic authority record, covering 19,000 controlled scientific and scholarly publication channels ensures that references from nonscientific publications are not entered into the system. Publication data from profes- sional bibliographic data sources (e.g., the Web of Science) are imported to the FRIDA system, to facilitate the registra- tion of publications by the employees. The database is there- fore well suited to productivity analyses across subject fields, as a large-scale database, with complete data covering not only journal articles, but also monographs and book chapters. To provide information about authors’ individual charac- teristics (institutional affiliation, position, age, and gender), the bibliographic database was coupled to another database, the Norwegian Research Personnel Register (providing the official Norwegian R&D statistics, compiled by NIFU), to introduce individual-level data. This database has biographi- cal information for all researchers in the higher education sector and institute sector in Norway (with a few exceptions, such as students and foreign guest researchers). The researchers have unique IDs in both the Research Personnel Register and the FRIDA publication database. However, the IDs are not identical and the linking is based on information on the full name of the persons as well as their institutional affiliations. For a large number of persons, there is a one- to-one correspondence, and homonyms (different persons with identical names) do not represent a problem. In a previous study, based the Norwegian Research Personnel Register, we found that the frequency of homonyms using the full name of the persons was only 1.4% of the Norwe- gian population of researchers (Aksnes, 2008). In our study

we have linked persons with identical names manually using available data and information. The publications of staff at individual departments were assigned to five broad fields, the humanities, social sciences, natural sciences, medicine, and technology; this assignment was based on the departments’ reporting of their scientific profile in national R&D statistics (departments select from a list of 58 scientific subfields, within the five main domains). The sixth domain used in national statistics (agriculture, fishery, and veterinary sciences) was not used because there were too few researchers in these subfields. Researchers from veterinary medicine were included in the medicine category, while researchers from agriculture and fishery sciences were included in the natural sciences. The analyses cover the 4-year period 2005–2008 and the study is limited to the four major research universities in Norway (Oslo, Bergen, Trondheim, and Tromsø), which account for 72% of national research output in the higher education sector. Only research personnel with at least one publication during the time period were included. Scientific productivity can, in principle, be measured relatively easily, by quantifying research output in terms of publications. In practice a number of issues make this more difficult, and must be addressed (Seglen, 2001). In particu- lar, the choice and weighting of publication types and the attribution of author credit are important. Many publications are the result of collaborative efforts involving more than one researcher, and are thus multiauthored. Two different principles and counting methods are typically applied for multiauthor papers, in bibliometric studies. The most common is “whole” counting, where all contributing authors receive credit for a whole publication. A second approach is “fractionalized counting” where credit is divided equally between all authors; if an article has five authors, each author is credited with 1/5 (0.2) of the article. It can be argued that these counting methods are complementary: the whole or integer count gives the number of papers in which each person “participated”; a fractionalized count gives the number of papers “creditable” to each person, assuming all authors made equal contributions to the paper, and that all contributions add up to one (Moed, 2005). While these are the two main counting methods applied in bibliometric studies, many other variants have been used, and a study by Gauffriau, Larsen, Maye, Roulin-Perriard, and von Ins (2007) identified 19 different methods of counting publica- tions. As Larsen (2008, p. 235) observes, “40 years of pub- lication counting have not resulted in general agreement on definitions of methods and terminology nor in any kind of standardization.” In this study, three simple and commonly applied count- ing methods are employed, allowing the following publica- tion indicators to be calculated and compared:

1. Average number of publications (whole counts) per person. 2. Average number of fractionalized publications per person (each publication divided by the number of authors).

TABLE 1. Average and median number of authors per publication by field, and publication types (article equivalents) across disciplines (n = 41,066).

 

Median authors per

Avg. authors per

Avg. authors per publication (excl. 100+ authors

Percentage journal

Percentage book

Percentage

Subject field

publication

publication

publications)

articles

articles

monographs

Humanities

1

1.4

1.4

39.0

37.5

23.5

Social Sciences

2

2.6

2.6

49.2

31.2

19.6

Technology

3

4.3

4.3

69.2

28.1

2.7

Natural Sciences

4

19.6

5.6

88.8

9.2

2.0

Medicine

6

8.4

8.3

93.9

4.6

1.5

3. Average number of article equivalents per person (frac- tionalized publication counts combined with a weighting of monographs as equal to 5 articles—both in journals and books).

The latter weighting system of publication types (also applied in the Norwegian funding model) attempts to make the research efforts behind articles and books comparable to article publications. Kyvik’s (1991) review of such weighting procedures from several other studies shows that it is standard to consider a full monograph as equivalent to 4–6 articles.

Results

Between 2005 and 2008 almost 60,000 publications (n = 59,861) were produced by the researchers in this study (n = 11,571). About 60% of the researchers are based in the two largest scientific fields, medicine (33%) and the natural sciences (28%). Another 12% of the researchers were from technology. A total of 27% of the researchers come from the “softer” disciplines: 16% from social sciences and 11% from the humanities. The use of different publication forms varies substan- tially between scientific areas (Table 1). In the humanities, more than 60% of publications are books and book chapters. In the social sciences the figure is ~50%. In contrast, books and book chapters play a minor role in the natural sciences and medicine. In technology, book chapters are more common due to many “proceedings papers” in conference series produced in this field. The overall conclusion that is apparent in Table 1 is that, while the natural sciences and medicine may be reasonably compared based on their pro- duction of journal articles only, any comparison with the humanities and social sciences will be inadequate and partial, unless book chapters and monographs are included in measures of publication activity. The five scientific fields also differ substantially in their coauthorship practices. Research in humanities subjects is typically an individual game, while social science publica- tions are typically coauthored by two or three researchers. Publishing in other disciplines is generally the result of teamwork, with the highest median number of authors found in medicine (six authors). Looking at the average number of authors, the natural sciences have by far the highest number (19.6), although this is largely due to the “CERN-papers”

which often have several hundred authors; if the average number of publications is calculated for each field excluding all publications with more than 100 authors there is a sub- stantial drop in the average number of authors in the natural sciences, and medicine emerges as the scientific domain with the highest average number of authors. The results in Table 1 illustrate that whole counts of publications may skew the data, and should therefore be fractionalized by the number of contributing authors. In Figures 1–5 and Table 2 we compare research produc- tivity across scientific domains and by academic position. All figures can be found the Appendix. Large variations in scientific publishing productivity are apparent between fields (Figure 1). As expected, the number of publications (whole counts) per researcher is much higher in the natural sciences, medicine, and technology than in the humanities and social sciences. However, due to the high number of authors per publica- tion in technical/medical/natural science publications, pro- ductivity drops significantly when we use fractionalized counts. In contrast, fractionalized productivity measures result in substantially higher averages in both the humanities and social sciences than in the other fields. This impact of fractionalization on productivity measures for the natural sciences is particularly significant in the subfield of physics, due to the effect of the “CERN-papers” (Table 1).At the other end we find mathematics and informatics with the lowest differences between these two measures. A second observa- tion that can be made from Figure 1 is that differences between disciplines are much greater when looking at frac- tionalized publications per person compared to whole counts. The average number of publications based on fractionalized counts varies from 1.03 in medicine to 3.18 in humanities (see Appendix), that is a ratio of 3.08, while the ratio for the lowest and highest whole counts at the field level was 1.69 (3.64 in humanities and 6.16 in natural sciences). Previous studies have also shown large differences in publication patterns according to researchers’ academic position. In order to compare “like with like” we added data on a range of common academic positions, and a category “Other personnel” which includes adjunct professors and other scientific positions (e.g., researchers), administrative and technical personnel as well as retired persons. Scientific productivity varies greatly between the various academic positions (Figure 2).

FIG. 1. Mean number of publications per person: total, all research personnel. FIG. 2. Mean number

FIG. 1. Mean number of publications per person: total, all research personnel.

FIG. 1. Mean number of publications per person: total, all research personnel. FIG. 2. Mean number

FIG. 2. Mean number of publications per person by academic position.

Regardless of the publication measure used, professors constitute the most productive academic group; this is perhaps to be expected as full professors at Norwegian research universities are expected to devote ~50% of their time to research. Postdoctorates exceed associate pro- fessors’ publishing based on whole counts, but associate professors are more productive after fractionalization. The largest decrease in productivity after fractionalization is found in the case of medical doctors/physicians, who turn

out to be less productive than PhD students based on fractionalized counts. Figure 3 shows how the research productivity varies for professors only across scientific domains. The pattern for professors is very similar to that observed for the whole sample: the professors in the humanities and social sciences are less productive than their colleagues in the “hard” sciences based on whole counts, but this picture is reversed after fractionalization, where productivity is once

FIG. 3. Mean number of publications per person, full professors ( n = 2,405). FIG. 4.

FIG. 3. Mean number of publications per person, full professors (n = 2,405).

FIG. 3. Mean number of publications per person, full professors ( n = 2,405). FIG. 4.

FIG. 4. Mean number of article equivalents per year by scientific field and academic position.

again highest in the humanities and lowest in medicine. One noticeable exception in this professors-only analysis is found in the results for technology, where the gap in- publishing output compared to the humanities and social sciences is much smaller than that found in the analysis of all researchers (Figure 1).

When we compare between academic positions using article equivalents only (Figure 4), another confirmation of previous findings emerges. Across every academic position, personnel from the humanities and social sciences are most productive. In every academic position (except for medical doctors/physicians, where there are no personnel from the

FIG. 5. Mean number of publications per person by subfield (n = 37).
FIG. 5. Mean number of publications per person by subfield (n = 37).

TABLE 2. Average number of publications per person by subfield.

Fractionalized article

Subfield

Scientific domain

No. of researchers

Whole counts

Fractionalized counts

equivalents

Archaeology

Humanities

109

3.09

2.60

3.21

History

205

3.72

3.15

3.87

Languages

356

3.50

2.93

3.40

Literary sciences

164

4.55

4.17

5.43

Musicology

50

2.60

2.27

2.85

Philosophy sciences

146

3.55

3.16

3.99

Theology/history of religion

94

4.24

3.93

4.91

Economics

Social Sciences

142

4.39

2.32

2.49

Educational sciences

286

4.28

2.84

3.49

Human geography

71

3.75

2.16

2.41

Jurisprudence/criminology

279

3.61

3.25

4.40

Media/journalism sciences

73

3.96

3.00

3.93

Political sciences

194

5.04

3.29

3.86

Psychology

412

5.94

2.10

2.22

Social anthropology

126

3.11

2.54

3.16

Sociology

206

4.61

3.00

3.48

Biosciences

Natural Sciences

1,048

5.21

1.34

1.35

Chemistry

463

6.56

1.68

1.70

Fisheries

97

3.37

0.83

0.83

Geosciences

419

5.08

1.23

1.25

Informatics

301

6.65

2.27

2.31

Mathematics

305

5.29

2.15

2.21

Physics

515

9.48

1.72

1.75

Basic medicine/odontology

Medicine

1,543

5.04

0.93

0.94

Health sciences

488

7.08

1.80

1.83

Clinical medicine

1,560

5.30

0.87

0.88

Clinical odontology

132

4.50

1.24

1.24

Biotechnology

Technology

113

4.80

1.02

1.06

Building

111

4.40

1.50

1.64

technology/architecture Chemical technology

126

6.57

1.80

1.88

Electro/technical sciences

93

4.14

1.31

1.37

Engineering

103

4.57

1.75

1.82

Environment technology

69

4.65

1.78

1.80

ICT

428

5.96

2.28

2.32

Marine technology

78

6.14

2.27

2.32

Material technology

139

5.26

1.32

1.32

Mineral resources/petroleum

72

4.31

1.60

1.60

humanities) we also find the same pattern of productivity rankings by disciplinary area: humanities are first, followed by social sciences, technology, natural sciences, and lastly medicine. The results so far have been based on five broad scientific domains. Further analyses were performed for all subfields containing 50 researchers or more (accounting for a total of 11,116 out of the full data set of 11,571 researchers). Table 2 helps to illustrate (1) whether the overall ranking described is also valid at the subfield level, and (2) whether there are any substantial variations in subfield productivity within each scientific domain. When looking at publication whole counts, physics (9.48) and health sciences (7.08) have the highest scores. Psychology has the highest score in the social sciences (5.94), while literary science (4.55) is the humanities sub- field with the highest score. The highest median score is found in the natural sciences (5.29, ranging from 3.37–

9.48), but this is also the scientific domain with the largest spread in subfield scores. Medicine has the second highest median score (5.17, ranging from 4.50–7.08), followed by technology (4.72, ranging from 4.14–6.57), social sciences (4.28, ranging from 3.11–5.94), and lastly humanities (3.55, ranging from 2.60–4.55). The whole counts for subfields are thus in line with the overall trends found for broad fields (Figure 1). The fractionalized scores show some interesting results. Within the humanities and social sciences, psychology gets the lowest score (2.21), indicating that this subfield, although classified as part of the social sciences, has a pub- lication pattern more in line with medicine. The highest fractionalized scores at the subfield level are found in the subjects literary sciences (4.17), and theology/history of religion (3.93) from the humanities. Outside of the humani- ties and social sciences, Information and Communication Technologies (ICT) has the highest fractionalized score

(2.28). The subfields from the humanities also have the highest median fractionalized score (3.15, ranging from 2.27–4.17), followed by social sciences (2.84, ranging from 2.10–3.39). Natural sciences (1.68, ranging from 0.83– 2.27), and technology (1.67, ranging from 1.02–2.82) are both above medicine (1.08, ranging from 0.87–1.80). This is also in accordance with the overall trends (Figure 1). The fractionalized article equivalents for subfields show a near identical pattern to those of fractionalized scores:

the median score in humanities subfields is highest (3.87, ranging from 2.85–5.43), followed by social sciences (3.48, ranging from 2.22–4.40). The median value for technology subfields (1.72, ranging from 1.06–2.32) is slightly above that of natural sciences (1.70, ranging from 0.83–2.31), the opposite relationship than for the fractionalized scores. Again, the lowest median value is found in medicine (1.09, ranging from 0.88–1.83). The highest average score is found for literary sciences (5.43), and again, ICT (2.32) is the subfield outside humanities and social sciences with the highest score. The three lowest scores are found in fisheries (0.83), clinical medicine (0.88), and basic medicine/ odontology (0.94). The main picture that emerges from these results is that there is a reverse association for subfield scores based on whether publications have been fractionalized or not (Figure 5). That is, the higher the whole counts scores are for a subfield, the lower its fractionalized scores will be. In Figure 5 the subfields are listed by the number of whole counts (regardless of scientific domain).

Discussion

We have identified large productivity differences between five main scientific domains and their subfield areas. In line with previous studies, we find that whole counts of publications are higher in the technical, natural, and medical fields (compared to humanities and social sciences) (e.g., Archambault & Larivière, 2010; Dundar & Lewis, 1998; Shin & Cummings, 2010). The analysis goes on to indicate, however, that simple adjustments to the methods used in counting publications (i.e., fractionalization based on number of authors and weighting of publication types) lead to very substantial changes in the comparative performance across disciplinary areas. A noticeable example is that of full professors in medicine and the humanities:

  • 1. Whole counts: a full professor in medicine publishes on average 138% more publications than a professor in humanities.

  • 2. Article equivalents: a full professor in the humanities publishes on average 75% more than a professor in medi- cine once the publications are fractionalized and weighted by publication type.

Our results imply that it is more demanding in terms of research effort to produce e.g., 10 articles each with 10 authors than a single-authored article, which both would result in one article equivalent. This is in line with Lee and

Bozeman’s (2005) study where the strong relationship between research collaboration and research productivity (measured by the number of publications) was no longer significant when the publications were fractionalized. However, we do not know of any other studies that show that the relationship between scientific domains and productivity is completely reversed after fractionalization, as is found in our study. One possible reason for this is the fact that we included data on monographs and book chapters. In Archambault and Larivière’s (2010) study of 13,479 researchers in Quebec, it was shown that productivity by domain changed after fractionalization in the same direction as in our study. That is, the superiority in productivity observed in medicine, natural sciences, and engineering compared to social sciences and humanities was drastically reduced. In the Canadian study, based on journal articles only, the productivity ratio between mean number of whole counts per researcher in social sciences (6.9) and natural sciences (17.4) was 2.52, but was reduced to 1.95 after fractionalization. In the humanities, the ratio compared to natural sciences was 6.96 when using whole counts and 1.95 after fractionalization. Because our study also includes monographs and book chapters, which together account for a very large share of the publications in the social sciences and humanities, the initial productivity differences when looking at whole counts were smaller than found in the Canadian study. The whole count ratio between the natural sciences and social sciences was 1.38, but after fractional- ization the ratio was 1.67 in favor of the social sciences, and increasing to 1.96 after weighting the publications by type. The same pattern was seen for the humanities. At first, using whole counts, the productivity ratio between the humanities and natural sciences was 1.69 in favor natural sciences, which changed to 2.01 in favor of the humanities after frac- tionalization, and 2.44 after weighting of publication type (see Appendix). Comparing results like this to the Canadian study is interesting, because our study takes it further by stating that the remaining productivity differences are com- pletely eliminated as books and book chapters are also included; they even contribute to a reversed productivity landscape. Our results clearly demonstrate that the choice between whole counts and fractionalized counts (with or without a weighting to take account of different publication types) determines the outcome of the productivity analysis. This suggests that none of the three indicators (whole counts, fractionalized counts, and article equivalents) used here provides good indicators by themselves for comparing pro- ductivity across scientific domains. However, it is possible that other weighting principles might work to make such comparisons more valid and meaningful. In citation analy- ses, the average citation rate varies considerably between the different scientific disciplines. As a response, various refer- ence standards and normalization procedures have been developed. The most common is the average citation rate of the journal or field in which the particular papers have been published (Moed, 2005). It may be that similar reference

standards can be developed for scientific productivity. The main problems that need to be addressed relate to how much credit each author should be given in coauthored publica- tions, and the relative weight that should be given to the differing publication formats of journals, book chapters, and monographs. As seen in our analyses, if scientific domains

are to be comparable, one needs to attribute to each author a higher score for each paper than the fraction according to the number of authors. In order to adjust for the identified field differences, the extra weighting would have to be substan- tial. To test this approach, we performed an analysis where coauthored publications gave 0.5 points to each author, regardless of the total number of authors, to investigate whether this would neutralize the field differences. Analyz- ing fractionalized article equivalents we found that the mean score increased in all scientific domains, but the increase was lowest in domains with a low share of coauthored pub- lications; the humanities (which was up 0.09 points: from

  • 3.93 to 4.02) and social sciences (up 0.46 points: from 3.16

to 3.62). The largest increase was found in medicine, where the mean number went up by 164% (from 1.04 to 2.75), followed by natural sciences (from 1.61 to 3.27) and tech-

nology (from 1.81 to 2.82). The main focus of this exercise, however, is to compare the distances between the fields when all coauthored publications are boosted to 0.5 points per author. To investigate the gap between fields, their rela- tive publication output can be expressed as ratios between the most productive field based on the original analysis, the humanities. When humanities output was taken as equal to 1.00, the ratios for the other disciplines were: 0.80 for social sciences, 0.46 for technology, 0.40 for natural sciences, and

  • 0.26 for medicine. The new analysis drastically reduces the

gaps between subject ratios, to: 0.90 for social sciences (up 0.10), 0.70 for technology (up 0.24), 0.81 for natural sci- ences (up 0.41), and 0.68 for medicine (up 0.42). Although it does not eliminate the differences between fields, the new weights given to coauthored papers reduced much of the field differences, although the ranking remained fairly stable: humanities and social sciences at the top, medicine at the bottom, and natural sciences changing position to over- take technology. When we performed the same analysis for professors only, the field differences almost completely disappeared. Professors from the humanities and social sciences were no longer rated as the most productive groups, as they were overtaken by the professors in natural sciences and medi- cine. Originally, professors in humanities had the highest average rate of publications (5.99), followed by social sciences (5.33, ratio: 0.89), technology (4.25, ratio: 0.70), natural sciences (3.61, ratio: 0.60), and medicine (2.75, ratio: 0.46). The distance between the highest and lowest mean number was thus more than two. After providing all authors with 0.5 points for coauthored publications, the results changed markedly: the highest value was found in natural sciences (7.10, more than twice as high compared to the original analysis); medicine was second (6.64, which is 2.4 times as high as originally, ratio: 0.90, compared to

natural sciences); technology was third (6.44, which is 1.5 times as high as originally, ratio: 0.90); the social sciences were fourth (6.15, compared to 5.33 originally, ratio: 0.87); and the humanities came out at the bottom (6.10, compared to 5.99 originally, ratio: 0.86). These findings are interesting because they show that when it comes to the most research- intensive and productive academic group, field differences can be adequately neutralized by using the 0.5 point method. When all academic groups and not just full professors were included, large field differences still remained, which in the case of medicine may be exemplified by many physicians in hospitals who do publish occasionally, but at a rather low frequency compared to other scientific personnel at univer- sities who are not professors. Another option to adjust measures of publication rates would be to introduce standardized scores, e.g., Hays (1970) suggested the equation Z = (x – M)/s, where “Z” is a researcher’s standardized score; “x” is the person’s raw score; “M” is the average score for the discipline to which s/he belongs; and “s” is the standard deviation for that discipline. Based on such standardization, almost all differences between scientific fields would disappear (see Figure 6; real numbers have been standardized around 100 for visual purposes). Even though field differences in productivity have been more or less eliminated, two exceptions stand out: the first is that the humanities remain above other scientific domains in fractionalized publications. The second, and more substantial exception, is the low values seen for tech- nology compared to other fields after publications have been fractionalized. Nevertheless, it seems likely that it will be more difficult to provide acceptable reference standards for productivity than for citation rates. This is because scientific productivity has been shown to depend on numerous factors, such as academic position, gender, age, time for research/teaching loads, availability of research funds (and equipment, research assistants, etc.), department culture and working conditions, and departmental size and organizational context. These findings suggest that productivity differences across fields are not a matter of weighting procedures and fractionalization schemes, but may be the result of differ- ences in other meaningful variables—at both the individual and contextual levels. In the next sections we discuss our results in light of former studies that have looked at produc- tivity differences, based on some of the issues discussed here. In general, our results differ from past studies based on the same Norwegian universities, covering the same scien- tific domains (Kyvik, 1991, 1993, 2003; Kyvik & Olsen, 2008). This is likely due to important differences between these studies and ours: this study is based on a much larger data set (11,500 compared to 1,200–1,500 persons) and also uses registration databases for data on individual research- ers, while the other studies used questionnaire data and nonverified self-reported numbers of publications. Finally, it may be that productivity patterns may have changed over time, and these studies cover differing time periods. Thus,

FIG. 6: Standardized number of publications by scientific domains. the results from former Norwegian studies are
FIG. 6: Standardized number of publications by scientific domains. the results from former Norwegian studies are
FIG. 6: Standardized number of publications by scientific domains. the results from former Norwegian studies are

FIG. 6: Standardized number of publications by scientific domains.

the results from former Norwegian studies are not directly comparable to the current study.

Productivity and Gender

Previous studies have shown large gender differences in scientific productivity (Aksnes, Rørstad, Piro, & Sivertsen, 2011a; Kyvik & Teigen, 1996) with female researchers being shown to be less productive. In Aksnes et al. (2011a), this pattern was found for almost all age groups and scien- tific domains. Male researchers in our study appear to be more productive when using all three productivity measures (see Appendix); this holds for all scientific domains (not shown in the tables), thus making it important to look at how the gender balance differs between fields (gender is also highly correlated with academic position, where women are less represented in the most productive group, the profes- sors), because a high share of women is associated with low productivity. A total of 35.6% of the sample are women, and the highest shares of women are found in medicine (44.5%), the humanities (41.3%), and the social sciences (40.3%). The share of female researchers is much lower in natural sciences (27%) and technology (19.5%). These results do not support gender as an explanatory factor for subject field differences in productivity, since women are better repre- sented in the two disciplines where productivity is highest, based on fractionalized counts.

Productivity and Age

Although the results of previous studies have not always been entirely consistent, a curvilinear relationship between age and productivity is quite firmly established. The average

production of publications increases with age and reaches a peak at some point during an academic career, from which it then declines (see, e.g., Barjak, 2006; Gingras, Larivière, Macaluso, & Robitaille, 2008; Gonzalez-Brambila & Veloso, 2007; Aksnes, Rørstad, Piro, & Sivertsen, 2011b). Based on the same data used in this study, Aksnes et al. (2011b) found an identical pattern of age-related rise and decline across scientific domains; but despite similari- ties in overall age patterns, there were also noticeable dif- ferences. For example, scholars in the humanities and social sciences remained productive for a longer period. In these fields, researchers aged between 65 and 69 remained very productive. In this study, looking at 10-year age intervals, productiv- ity (fractionalized counts) increases with age in all scientific domains, up to 50–59 years (humanities and natural sci- ences) or 60–69 (all other fields). Therefore, the presence of many young researchers appears to dilute rates of publica- tion productivity. Technology is a particularly interesting case here—the most productive age group (60–69) consti- tutes only 7.5% of researchers in this field (the mean share of 60–69-year-olds in all fields is 13.9%). In technology we also find the highest share of researchers in both the 20–29 and the 30–39 age groups. Both in that study and the present one, we may question whether the decision to exclude researchers without a paper has affected the age-specific results. Arguably, there are scientific staff in the four main universities in Norway in the older age groups who did not publish any papers throughout the years 2005–2008. A study by Gringas et al. (2008) found that productivity for research- ers who kept publishing increased in line with our results, but once researchers without a paper were included in the denominator, the productivity began to decrease from age

50 years. While the lack of nonproductive researchers in our data in particular may affect the results for the older age groups, we do not believe that it impacts the interpretation of the key result, which is the divide between young and more established researchers. The two most productive scientific domains measured by fractionalized counts (humanities and social sciences) have a much lower share of researchers under 40 years (28.6 and 30.2%, respectively) than the natural sciences and technol- ogy (51.6 and 62%, respectively), which supports the idea that some of the productivity differences between these fields may be related to a different age composition, with the humanities and social sciences benefiting from a more mature pool of researchers. However, medicine also has a low share of young researchers (29.9% under 40 years) and still has a low productivity level (possibly because many physicians are in dual positions in universities/hospitals and publish at a lower frequency than full-time university employees).

Productivity and Academic Position

Many studies have shown that individual publication pro- ductivity tends to increase moving up the hierarchy of aca- demic positions (Kyvik, 1991; Aksnes et al. 2011a), where professors are the most prolific personnel. It is also well established that PhD students are much less productive in terms of the number of publications they produce than more established researchers. This is not surprising, as the PhD students are still in training as researchers. In our study, we find that technology has a completely different composition of research roles when compared to the other scientific fields. In technology, 19.5% are professors and 39.9% PhD students, while in the humanities the distribution is 31.3% and 13.9%, respectively. The share of professors is also low in medicine, as many of the more experienced researchers are employed at university hospitals as medical doctors/ physicians. The composition in the natural sciences and technology includes postdoctorates and PhD students as the largest proportion of personnel, while in the other domains, professors/associate professors are in the majority. Hence, in the humanities and social sciences the majority of research- ers are found to be in the most productive academic positions, whereas in natural sciences and technology, the majority of the personnel are found in the least productive academic positions. This discussion indicates that some of the excess produc- tivity in humanities and social sciences may be explained by different compositions of research personnel based on gender, age, and academic position. The differences also indicate that the reduced productivity in technology may be due to the large number of younger and less advanced researchers (PhD students) and the low share of professors and associate professors. However, this should not automati- cally be interpreted as a management argument for hiring more senior personnel. PhD students represent an important work force in the science system and do a lot of time-

consuming work on behalf of their supervisors, which would otherwise reduce these senior researchers’ own publication output. Within a group of researchers led by a professor, the professor will typically get her/his name on all publications, while the PhD students will be authors only in the publica- tions they have been directly involved in (Kyvik, 1991). This means that, instead of high numbers of PhDs being seen as an impediment to the overall productivity of a group (i.e., PhD students and postdocs) they may enhance productivity in some segments of that group. This was also found (with reversed causation) in Smeby and Try (2005), where pro- ductivity decreased with age, but where older researchers had a positive impact on the overall group level, Further- more, Gingras et al. (2008) showed that the decline in researchers productivity about age 50 was subsequently suc- ceeded by an increase in the average number of papers in highly cited journals and among highly cited papers (which rises continuously until retirement), meaning that productiv- ity and quality/impact are two features that should both be considered before describing a development as being negative due to reduced productivity. Norwegian R&D statistics also include data on time available for research, according to academic positions, institutions, and domains (R&D equivalents). These data show that the tenured personnel tend to have ~40% of their time to use on research, while the figures for PhD students and postdocs are around 70–80%. These differences are mainly due to the fact that the tenured personnel have much larger teaching obligations. If we had applied the R&D equivalents on the publication data in our study the differ- ences between the natural sciences and technology, on the one hand, and the humanities and the social sciences on the other would be even larger (due to the different composi- tion of the academic personnel). This reinforces the obser- vation that the work effort needed to produce one article equivalent in the first domain is significantly higher than in the latter.

Different Publishing Practices

Using the Norwegian Research Personnel Register, we were able to interpret our results in light of different compositions of researchers in the different disciplinary fields, based on researchers’ age, gender, and academic position. In the next sections we briefly discuss two other explanatory factors behind productivity differences across fields: the share of unproductive researchers and patterns of collaboration. Our study included only those researchers who had pub- lished at least one or more publications during the 4-year period, which represents a limitation in our data set, as only researchers who have published are registered in the data- base. Arguably, many of the researchers who are not found in the database (and have not published) are newly appointed, and thus unproductive. Nevertheless, many researchers are not in the database simply because they have not published in the time period. This is only a problem for

our conclusions if there are systematic differences between the scientific domains in their share of unproductive researchers. We do not have data to verify/falsify this, but in Kyvik’s (1991) 20-year old study at the same university departments there were certainly field differences in the share of unproductive researchers. The highest share of unproductive researchers was found in the humanities (19%, ranging from 4% in theology to 28% in languages), followed by the social sciences (14% unproductive researchers, ranging from 8% in sociology to 20% in psychology), natural sciences and technology (13%, ranging from 8% in geosciences to 20% in physics) and medicine (10%, ranging from 5% in biomedical sciences to 12% in odontology). We therefore cannot conclude that the share of unproductive researchers is similar in all fields. If this pattern described above is also valid for our study sample, the superiority of the “hard sciences” for the whole counts analysis would be even greater, while the “soft sciences” superiority in frac- tionalized counts analysis would be reduced. Such patterns of unproductiveness may thus have skewed our data. Another aspect that must be considered in trying to explain productivity differences between scientific fields is the great difference in the average number of authors con- tributing to a paper (as seen in Table 1), which obviously has a great impact on the number of whole count publications (Lee & Bozeman, 2005). There are various reasons for these field differences in publishing practices. Research in the natural sciences and medicine is often large scale: it may require contributions from different research laboratories with different experimental equipment, or field research based on observations in different parts of the world carried out by several research teams. This results in a large number of authors in many publications. Kyvik (1991, p.69–70) argues that scientists in the “hard sciences” have become more dependent on each other to make progress than have researchers in the “soft sciences.” In the latter fields, inde- pendence may actually be regarded as fruitful, in order to develop scientific paradigms. This idea was supported in Kyvik’s study, in that several of the few humanities researchers who had tried to coauthor a publication reported poor experiences. One possible explanation for this pattern is that the final products in the humanities often have an essayistic and individual form, making it difficult to agree on both content and style for the collaborating persons. In other fields, where content is more codified and standardized, it may be easier to coauthor publications. Still, the expectation that collaboration automatically generates more publications may be questioned, as it has been by Lee and Bozeman (2005, p. 694), who point out that: “The number of collabo- rators is not the same as the number of collaborations. If one collaborates ten times with one person, how does that differ, in productivity effects, from collaborating ten times with ten different persons?” Permanent collaboration/collaborators seems to be the key factor in publication patterns, as in Rey-Rocha, Martin-Sempere, and Garzón (2002), where researchers belonging to consolidated teams were shown to be more productive than their colleagues in nonconsolidated

teams. These again were more productive than individuals without a team. We have shown in our paper that productivity results are highly dependent on whether publication whole counts or fractionalized publication counts are used. This is due to the large differences in the average number of authors behind each article, across the different scientific fields. However, designing weighting procedures that are able to resolve this coauthorship dilemma (or field differences in book vs. journal article publishing) may still not be enough to unify the two indicators, as several factors at the individual level (age, gender, academic position) are both strongly related to productivity and are unevenly distributed across scientific fields. Our study is based on data on Norwegian scientists. Although it is a large-scale study, the question remains whether these findings have general validity in other set- tings. As shown above, several previous studies have pro- vided consistent results concerning the publication patterns across fields. The higher productivity in some fields based on whole counts has been documented in numerous other studies, given different coauthorship practices across fields. However, our inclusion of monographs and book chapters not only added further evidence to this, but also showed that what may be an unfavorable gain of much coauthorship, may be an unreasonable disadvantage after fractionalization once the publication numbers are based on books as well. The findings in this article of various author characteristics being associated with productivity, such as academic posi- tion, gender, and age, are also much in line with previous studies, although the limitation in our study of not including the unproductive researchers may have led to an overesti- mation of the productivity especially in the older age groups, where productivity numbers are lower in studies that have included the unproductive researchers as well. Although empirically robust, it should be acknowledged that the exact figures in terms of average productivity are likely to vary among countries. Cultural factors related to gender equality, academic opportunities, professional inte- gration, availability of resources, time for research, etc. differ at national levels and may affect scientific perfor- mance. Moreover, it is evident from our findings that differ- ent compositions within research populations (e.g., in terms of academic position and age) would have given different overall results.

Conclusion

In our study, scientists from medicine, the natural sci- ences, and technology are found to be more productive than their colleagues from the humanities and social sciences when productivity is measured using whole counts of pub- lications. This is arguably due to the large share of multiau- thored papers in these disciplines. After fractionalization of publications, the picture is completely reversed, with schol- ars from the humanities and social sciences being rated as the most productive. We have therefore argued that these

simple productivity measures cannot be used in comparative assessments of productivity levels across domains, as they are not neutral in terms of the varying research efforts involved in single-authored and coauthored publications. In contrast to citation indicators, where advanced and valid methods for adjusting for field differences exist, it is much more complex to produce reference standards for scientific productivity. Not only are there large differences in publish- ing practices (both in types of publication and the extent of collaboration) across domains, but variables such as position and age of researchers also play a role. In evaluations of research performance, and increasingly in funding alloca- tions, the productivity of scientists is usually one of the issues reviewed. In this respect, studies like ours may be useful in assessing the productivity levels of groups or personnel.

Acknowledgment

Preliminary results of this study were presented at the 11th International Conference on Science and Technology Indicators, Leiden, September 10th, 2010: Dag W. Aksnes, Kristoffer Rørstad, Gunnar Sivertsen, Fredrik Niclas Piro (2010): Productivity differences across fields — a macro analysis. Book of abstracts 11th International Conference on Science and Technology Indicators, Leiden (pp. 14–16).

References

Aksnes, D.W. (2008). When different persons have an identical author name. How frequent are homonyms? Journal of the American Society for Information Science and Technology, 59(5), 838–841. Aksnes, D.W., Rørstad, K., Piro, F.N., & Sivertsen, G. (2011a). Are female researchers less cited? A large-scale study of Norwegian scientists. Journal of the American Society for Information Science and Technol-

ogy, 62(4), 628–636. Aksnes, D.W., Rørstad, K., Piro, F.N., & Sivertsen, G. (2011b). Age and scientific performance. A large-scale study of Norwegian scientists. In E. Noyons, P. Ngulube, & J. Leta (Eds.), Proceedings of the 13th Inter- national Conference of the International Society for Scientometrics and Infometrics (ISSI 2011) (pp. 34–45). Durban, South Africa: ISSI. Archambault, E., & Larivière, V. (2010). Individual researchers’ research productivity: A comparative analysis for counting methods. In Book of Abstracts – 11th International Conference on Science and Technology Indicators (STI), Leiden 2010 (pp. 22–24). Leiden, Netherlands: Univer- siteit Leiden, CWTS. Barjak, F. (2006). Research productivity in the internet era. Scientometrics, 68(3), 343–360. Bonaccorsi, A., & Daraio, C. (2005). Exploring size and agglomeration effects on public research productivity. Scientometrics, 63(1), 87–

120.

Bourke, P., & Butler, L. (1996). Publication types, citation rates and evaluation. Scientometrics, 37(3), 437–494. Costas, R., van Leeuwen, T.N., & Bordons, M. (2010). A bibliometric classificatory approach for the study and assessment of research perfor- mance at the individual level: The effects of age on productivity and impact. Journal of the American Society for Information Science and Technology, 61(8), 1564–1581. Dundar, H., & Lewis, D.R. (1998). Determinants of research producti- vity in higher education. Research in Higher Education, 39(6),

607–631.

Gauffriau, M., Larsen, P.O., Maye, O., Roulin-Perriard, A., & von Ins, M. (2007). Publication cooperation and productivity measures in scientific research. Scientometrics, 73(2), 175–214. Gonzalez-Brambila, C., & Veloso, F.M. (2007). The determinants of research output and impact: A study of Mexican researchers. Research Policy, 36(7), 1035–1051. Gringas, Y., Larivière, V., Macaluso, B., & Robitaille, J-P. (2008). The effects of aging on researchers’ publication and citation patterns. PLoS ONE, 3(12), e4048. Hays, W.L. (1970). Statistics. London: Holt, Rinehart and Winston. Hicks, D. (2004). The four literatures of social science. In H.F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 473–495). Dordrecht, Netherlands: Kluwer

Academic Publishers.

Kyvik, S. (1991). Productivity in academia. Scientific publishing at Norwegian universities. Oslo, Norway: Universitetsforlaget. Kyvik, S. (1993). Academic staff and scientific productivity. Higher Education Management, 5(2), 191–202. Kyvik, S. (2003). Changing trends in publishing behaviour among univer- sity faculty, 1980-2000. Scientometrics, 58(1), 35–48. Kyvik, S., & Bruen Olsen, T. (2008). Does the aging of tenured academic staff affect the research performance of universities? Scientometrics, 76(3), 439–455. Kyvik, S., & Teigen, M. (1996). Child care, research collaboration and gender differences in scientific productivity. Science, Technology & Human Values, 21(1), 54–71. Larsen, P.O. (2008). The state of art in the publication counting. Sciento- metrics, 77(2), 235–251. Lee, S., & Bozeman, B. (2005). The impact of research collaboration on scientific productivity. Social Studies of Science, 35(5), 673–702. Moed, H.F. (2005). Citation analysis in research evaluation. Dordrecht, Netherlands: Springer. Ramesh, B.A., & Singh, Y.P. (1998). Determinants of research productivity. Scientometrics, 43(3), 309–329. Rey-Rocha, J., Martin-Sempere, M.J., & Garzón, B. (2002). Research productivity of scientists in consolidated vs. non-consolidated teams:

The case of Spanish university geologists. Scientometrics, 55(1), 137–

156.

Seglen, P.O. (2001). Evaluating biology. A scientometric study of a univer- sity department. NIFU Report 6/2001. Oslo, Norway: NIFU. Seglen, P.O., & Aksnes, D.W. (2000). Scientific productivity and group size: A bibliometric analysis of Norwegian microbiological research. Scientometrics, 49(1), 125–143. Shin, J.C., & Cummings, W.K. (2010). Multilevel analysis of academic publishing across disciplines: Research preference, collaboration, and time on research. Scientometrics, 85(2), 581–594. Smeby, J.C., & Try, S. (2005). Departmental context and faculty research activity in Norway. Research in Higher Education, 46(6), 593–619.

Appendix. Average number of publications per person 2005–2008, by field and academic position.

 

Fractionalized

Fractionalized article

Field

Position a

Number of persons

Whole counts

counts

equivalents

Humanities

Professors

407

5.40

4.82

5.99

Social sciences

542

7.58

4.41

5.33

Natural sciences

651

13.43

3.53

3.61

Medicine

537

12.88

2.74

2.75

Technology

268

11.97

4.09

4.25

Humanities

Associate professors

282

3.51

3.12

3.94

Social sciences

366

3.99

2.61

3.10

Natural sciences

318

5.25

1.65

1.66

Medicine

226

5.85

1.53

1.60

Technology

86

5.27

1.87

1.91

Humanities

Post docs

115

3.40

2.86

3.49

Social sciences

143

4.53

2.65

3.32

Natural sciences

484

5.58

1.50

1.51

Medicine

376

4.71

0.88

0.88

Technology

173

5.39

1.83

1.85

Humanities

PhD-students

180

1.76

1.57

1.80

Social sciences

340

2.06

1.27

1.46

Natural sciences

782

3.20

0.74

0.74

Medicine

809

2.60

0.46

0.46

Technology

548

2.89

0.97

0.97

Humanities

Medical doctor/ Physician

0

N/A

N/A

N/A

Social sciences

12

1.75

0.38

0.38

Natural sciences

3

1.00

0.19

0.19

Medicine

886

4.63

0.69

0.70

Technology

0

N/A

N/A

N/A

Humanities

Others b

315

2.64

2.16

2.62

Social sciences

500

3.13

1.75

2.03

Natural sciences

956

4.23

0.97

0.99

Medicine

966

4.34

0.81

0.82

Technology

300

3.56

1.08

1.13

Humanities

Total

1,299

3.64

3.18

3.93

Social sciences

1,903

4.47

2.65

3.16

Natural sciences

3,194

6.16

1.58

1.61

Medicine

3,800

5.37

1.03

1.04

Technology

1,375

5.27

1.77

1.81

Total

All fields

11,571

5.23

1.78

1.96

Professors

2,405

10.47

3.83

4.28

Associate professors

1,278

4.61

2.24

2.58

Post docs

1,291

4.99

1.61

1.75

PhD students

2,659

2.71

0.82

0.87

Medical doctor/physician

901

4.58

0.68

0.69

Others b

3,037

3.85

1.18

1.29

Men

All fields

7,449

6.00

1.99

2.19

Women

4,122

3.85

1.39

1.54

a Persons who have changed academic position during the time period are classified and included in more than one category. b Includes a variety of other positions: adjunct professors and other scientific positions (e.g., researchers), administrative, and technical personnel, as well as retired persons.