You are on page 1of 20

Scientometrics (2014) 101:1695–1714 DOI 10.1007/s11192-014-1338-z

Post-interdisciplinary frames of reference: exploring permeability and perceptions of disciplinarity in the social sciences

Timothy D. Bowman Andrew Tsou Chaoqun Ni Cassidy R. Sugimoto

Received: 11 November 2013 / Published online: 4 June 2014 Akade´ miai Kiado´ , Budapest, Hungary 2014

Abstract The ProQuest Dissertations and Theses database contains records for approx- imately 2.3 million dissertations conferred at 1,490 research institutions across 66 coun- tries. Despite the scope of the Dissertations and Theses database, no study has explicitly sought to validate the accuracy of the ProQuest SCs. This research examines the degree to which ProQuest SCs serve as proxies for disciplinarity, the relevance of doctoral work to doctoral graduates’ current work, and the permeability of disciplines from the perspective of the mismatch between SCs and disciplinarity. To examine these issues we conducted a survey of 2009–2010 doctoral graduates, cluster-sampled from Economics, Political Sci- ence, and Sociology ProQuest SCs. The results from the survey question the utility of traditional disciplinary labels and suggest that scholars may occupy a post-interdisciplinary space in which they move freely across disciplinary boundaries and identify with topics instead of disciplines.

Keywords

Disciplinarity ProQuest Permeability Disciplines Subject

categories

  • T. D. Bowman (&) A. Tsou C. Ni C. R. Sugimoto

School of Informatics and Computing,

e-mail: tdbowman@indiana.edu

Indiana University Bloomington, Bloomington, IN, USA

  • A. Tsou

e-mail: atsou@indiana.edu

  • C. Ni

e-mail: chni@indiana.edu

  • C. R. Sugimoto

e-mail: sugimoto@indiana.edu

123

1696

Scientometrics (2014) 101:1695–1714

Introduction and background

Disciplines are conceptual entities that are institutionalized into academe in the forms of schools, departments, and centers. As such, these units are sustained by the propagation of new scholars under the same disciplinary label (Turner 2000). This label serves as a symbol of students’ knowledge and skills, as well as facilitating the exchange of students on the academic labor market (Abbott 1999). However, given the emphasis on interdis- ciplinarity from institutions and funding agencies (e.g., Morillo et al. 2003), it is no wonder that disciplinary lines are being continually crossed as doctoral students from one disci- pline are hired on the faculties of new disciplines (Sugimoto et al. 2011). Furthermore, many of these doctoral students go into non-academic sectors—perhaps only partially relying on the domain expertise of their doctoral work. Studies of disciplinary mobility frequently rely on the disciplinary classification systems embedded in various indexing systems. For example, citation indices such as the Web of Science (WoS) and Scopus are often used to extract data on disciplines. The disciplinary variables are operationalized as the Subject Categories (SCs) of these databases—that is, SCs are proxies for disciplines. Criticisms of the validity of SCs as proxies for disci- plinarity have been raised across many decades (e.g., Garfield 1978; Ni et al. 2013). However, SCs for major databases such as PubMed and WoS remain the gold standard in evaluative bibliometrics, and practices of using these as proxies for micro and macro evaluations of the system of science persist (e.g., Leydesdorff et al. 2013, 2012; Ley- desdorff and Rafols 2009). These issues are intensified by the growth of interdisciplinarity across sectors. Research pertaining to interdisciplinarity has become more common in the past forty years with ‘‘a corresponding increase in the discussion of interdisciplinarity across disciplinary, profes- sional, and general literature’’ (Klein 1990, p. 38). Porter and Rafols (2009) have suggested that ‘‘science is indeed becoming more interdisciplinary, but in small steps—drawing mainly from neighboring fields and only modestly increasing the connections to distant cognitive areas’’ (p. 719). However, other studies suggest the ubiquity of interdisciplinarity in modern scholarly activities. Even disciplines such as physics can no longer be con- sidered to be ‘‘a single, isolated discipline’’ (Klein 1993, p. 201); ‘‘modern research knowledge’’ is no longer ‘‘exchange restricted within narrow silos’’ (Porter and Rafols 2009, p. 740). The degree to which disciplines interact has been described in levels of ‘‘permeability’’ (Baumann 1975, p. 15; Klein 1996, p. 38). One investigation of perme- ability has analyzed the degree to which doctoral students from one field are hired in another and begin training new students (Sugimoto et al. 2011). However, few studies have used dissertation data as a platform for studies of disciplinarity and interdisciplinarity. One large-scale index providing information on doctoral education is the ProQuest Dissertations and Theses database. This dataset contains records for approximately 2.3 million dissertations conferred at 1,490 research institutions across 66 countries (Ni and Sugimoto 2012), a scope that ‘‘cover[s] 40 % of all dissertations from major universities’’ (Andersen and Hammarfelt 2011, p. 374). This represents the largest historical database of dissertation and theses, covering works as far back as the early nineteenth century (Su- gimoto 2014). Despite the scope of the Dissertations and Theses database, it has rarely been used to analyze the growth and connectivity of disciplines (exceptions include:

Andersen and Hammarfelt 2011; Ni and Sugimoto 2013a, b; Ying and Xiao 2012), and no study has explicitly sought to validate the accuracy of the ProQuest SCs. To investigate these issues, we conducted a survey of recent doctoral graduates, cluster- sampled from three ProQuest SCs: Economics, Political Science, and Sociology. The

123

Scientometrics (2014) 101:1695–1714

1697

resultant findings were supplemented with data unobtrusively gathered about individuals whose dissertations were classified as being from Chemistry, Computer Science, and Biology. The primary objective was to investigate the degree to which ProQuest SCs could serve as proxies for disciplinarity. The secondary objective was to analyze the relevance of doctoral work to doctoral graduates’ current work and the degree to which they use theories, methods, and literature from their home discipline and other disciplinary areas. Finally, the work sought to analyze the permeability of disciplines from the perspective of the mismatch between SCs and disciplinarity. This work serves to examine the use of ProQuest SCs for disciplinary analyses, as well as to advance our understanding of in- terdisciplinarity in the careers of doctoral graduates. Implications from this research can be used to inform practices and policies of higher education and indexing of scholarly communication.

Methods

Subject categories

When a doctoral student submits a dissertation to ProQuest, they are instructed to select relevant subject areas. These suggestions are then used by ProQuest dissertation editors, who assign SCs to the dissertation. As noted by a ProQuest representative, ‘‘we use the author’s subjects unless our scope for a subject is entirely different from the author’s interpretation of it (as sometimes occurs). Subjects are assigned based on the dissertation topic, not the university department, although naturally those are most often in sync’’ (McLean 2012). Each dissertation in the database is associated with at least one SC, although an increasing proportion is associated with more. Each SC is identified by a four-digit unique identification number, known as the SCID. Figure 1 provides a schema of a dissertation assignment to more than one SC. Figure 2 lists an example dissertation with ID = 15 in the ProQuest database, as well as the three SCs (identified by SCID) to which it is assigned. Each subject category listed in Fig. 2 has two levels: the more general first level category and the qualifying second level category. The first level subject category is considered as a proxy for the discipline, while the second level subject category is considered a proxy for the specialty. Therefore, dissertation 15 is assigned to two disciplines and three specialties. It is a multi-disciplinary dissertation, and can be considered simultaneously as a sociology and health science dissertation. The first item listed (SCID 0344; Sociology) is considered to be the primary category (McLean

2012).

Data

We constructed a sampling frame of all dissertations completed in 2009 or 2010 for each of three disciplines: Economics, Political Science, and Sociology. The time frame was selected in order to ensure that recent graduate students had sufficient time to transition from doctoral work into employment. A dissertation was considered as part of the disci- pline if it contained the disciplinary label as at least one of the first level SCs (not necessarily as the primary SC; that is, where the discipline was listed in the first position). Five hundred dissertations from each SC were randomly selected for inclusion. Given that most dissertations from this time period are associated with multiple SCs, we resampled to

123

1698

Scientometrics (2014) 101:1695–1714

1698 Scientometrics (2014) 101:1695–1714 Fig. 1 Data schema of ProQuest dissertation Fig. 2 Example dissertation w/multiple
Fig. 1 Data schema of ProQuest dissertation Fig. 2 Example dissertation w/multiple subject categories
Fig. 1
Data schema of ProQuest dissertation
Fig. 2
Example dissertation w/multiple subject categories

ensure that dissertations were only assigned to one discipline. The three disciplines were selected in order to examine an array of social sciences.

Identifying sampling frame

An online web application was constructed using PHP, jQuery, and HTML, allowing a research assistant (RA) to search, retrieve, and record additional information about each subject by using the information in ProQuest as a starting point. The RA then attempted to find current information about the subject by examining the links on the first three pages returned from a Google search. The Google searches were conducted using the person’s first name and last name, their dissertation title, or both. We collected information about each subject’s current URL, current email, current departmental affiliation, and current CV link; all of the new information was saved in a PostgreSQL database and linked to the ProQuest data using bridge tables. This allowed us to identify contact information and obtain additional contextual information regarding career trajectories. Contact e-mail

123

Scientometrics (2014) 101:1695–1714

1699

addresses for each person could not be located. Therefore, the final sample included 348 individuals in political science, 338 individuals in economics, and 283 individuals in sociology.

Questionnaire

The questionnaire was designed to gather (a) demographic information, (b) career infor- mation, and (c) information regarding disciplinary identity: that is, the degree to which the individual self-associated with the discipline in which they were trained (see Appendix for the full questionnaire—note that not all questions were utilized in the present study). The questionnaire was pilot-tested with one faculty member and four graduate students and built using Qualtrics 1 survey software. The questionnaire was available electronically, and the 969 participants were invited to participate between March 29 and April 18, 2013. No reminders were sent as the use of follow-ups has been shown to be negatively related to response rates (Anseel et al. 2010). The response data were exported from Qualtrics into Excel and SPSS for further evaluation. The main analysis of the survey results was to determine the degree to which the exact disciplinary labels were used by respondents to describe their area of work. That is, an exact match between disciplinary labels would be one in which the discipline under study was Economics and the respondent said that they received their degree in Economics. A close match would be one that demonstrated the direct application of the subject area (for example, economics of education). Non-matches were those that were either specialized (e.g., forecasting) or distinct disciplines (e.g., History).

Validation exercise

Data from three additional disciplines were selected as a validation metric. Using the same mechanism as for the three social science disciplines, 100 dissertations for each of three disciplines (Biology, Chemistry, and Computer Science) were selected at random. These were selected from the same timeframe as the initial dataset and on the basis of the same criteria (that is, containing at least one instance of the SC, but not necessarily in the primary position). We then conducted a series of Google searches to identify the current departmental affiliations of these dissertations’ authors, as well as to identify the discipline in which they received their doctorate degree.

Results

This section is organized into three main parts. The first part provides a basic description of all ProQuest dissertations (in the entire database, not just in the timeframe under analysis) in the three SCs so as to provide context. The second part provides descriptive data regarding the respondents. The last part is an analysis of the questionnaire.

123

1700

Scientometrics (2014) 101:1695–1714

ProQuest description

There were a total of 96,434 dissertations (including all of ProQuest dissertations and theses until 2011) that included Sociology as a SC in ProQuest. Sociology was associated with 13 s-level SCs (specialities) (Table 1). Seventy-five percent of dissertations contained another first-level category (discipline). The ten first-level SCs most commonly occurring with the Sociology SC are listed in Table 2. There were 76,336 dissertations with the Economics SC. Economics contains 8 s-level SCs, present in Table 3. Only 34 % of dissertations labeled Economics contained another first-level SC. The most frequently co-occurring SCs are listed in Table 4. Political science had the lowest number of dissertations in ProQuest at 55,626, as well as the least number of second-level SCs (4). These are listed in Table 5. 55 % of Political Science dissertations list another discipline. Table 6 shows the most frequently co-occurring disciplines. Based on the evaluation of the ProQuest SCs, Sociology is both the most specialized and the most interdisciplinary of the three selected disciplines; in addition, Economics is the least interdisciplinary (having the lowest proportion of dissertations with another SC), and Political Science is the least specialized (as represented by having the fewest number of second-level SCs).

Description of survey respondents

Our sampling frame of 500 dissertations from each discipline reflects the general patterns of interdisciplinarity, although the sampled dissertations were more likely to contain multiple SCs than the numbers reflected here. This is due to the fact that more recent dissertations are more likely to be associated with multiple SCs. Within the sampling frame, 68 % of Economics dissertations, 78 % of Political Science dissertations, and 92 % of Sociology dissertations were associated with another first-level SC. A 20 % response rate was obtained for the survey, with lower levels for Economics (n = 45; 13 %) compared to Political Science and Sociology [n = 81, n = 64 (respec- tively); both 23 %]. This counts only those surveys that were fully completed. The gender, race, and age of respondents are presented in Table 7, broken down by discipline. Overall, responders were 51 % male (n = 96) and 49 % female (n = 94); 75 % of the respondents fell between 31 and 45 years old. As shown, Economics was more male-dominated, whereas Sociology was more female-dominated. Caucasians were the majority in all disciplines, and respondents from Economics tended to be the youngest. Nearly all respondents (98 %) were currently employed, and respondents were from a diverse set of affiliated institutions. Table 8 shows the respondents’ identification with various types of organizations. Respondents were highly associated with academic insti- tutions, particularly when the individuals in question were associated with Sociology. Counting partially and fully complete response sets, the 48 respondents from Economics represented 46 different institutions. The majority of institutions (n = 38; 83 %) were in the USA: ten other countries were also represented (Brazil, Canada, Chile, Germany, Italy, Lebanon, Mexico, Sweden, Switzerland, and UK). Similarly, 67 affiliations were located for Sociology doctorates. These were also primarily in the USA (n = 62; 93 %), with only five other countries represented (Canada, France, Peru, United Arab Emirates, and South Africa). Political Science doctorates were associated with 81 institutions (with one

123

Scientometrics (2014) 101:1695–1714

1701

Table 1

Specialities of Sociology by number of occurrences

Specialty

#Diss

1st appear

Sociology, Individual and Family Studies

23,395

1934

Sociology, General

12,768

1930

Sociology, Social Structure and Development

11,882

1894

Sociology, Criminology and Penology

11,325

1934

Sociology, Public and Social Welfare

9,600

1912

Sociology, Industrial and Labor Relations

7,602

1914

Sociology, Theory and Methods

5,065

1922

Sociology, Demography

4,108

1935

Sociology, Organizational Theory

417

1971

Sociology, Sociolinguistics

166

2008

Sociology, Environmental Justice

67

2008

Table 2

Subject Categories most often occuring with Sociology by number of occurrences

 

Subject category

# Co-occurrences

Psychology

20,685

Education

18,022

Women’s Studies

9,003

Health Sciences

8,149

Political Science

7,992

History

6,946

Business Administration

5,185

Economics

4,838

Anthropology

4,650

Social Work

4,333

Black Studies

4,132

Table 3

Specialties of Economics by number of occurrences

Specialty

#Diss

1st appear

Economics, General

29,920

1930

Economics, Finance

18,038

1932

Economics, Theory

11,984

1938

Economics, Agricultural

10,773

1929

Economics, Commerce Business

8,637

1899

Economics, Labor

6,368

1912

Economics, History

3,910

1909

Economics, Environmental

202

2008

1702

Scientometrics (2014) 101:1695–1714

Table 4

Subject Categories most often occuring with Economics by number of occurrences

Subject category

# Co-occurrences

Business Administration

 

7,233

Political Science

5,104

Sociology

4,838

History

2,628

Education

1,887

Health Sciences

1,798

Agriculture

1,514

Energy

1,444

Environmental Sciences

1,303

Urban and Regional Planning

1,157

Geography

1,041

Table 5

Specialties of Political Science by number of occurrences

Specialty

#Diss

1st appear

Political Science, General

31,362

1895

Political Science, Public Administration

15,984

1894

Political Science, International Law and Relations

12,023

1927

Political Science, International Relations

639

1948

Table 6

Subject Categories most often occuring with Political Science by number of occurrences

Subject category

# Co-occurrences

History

8,035

Sociology

7,992

Economics

5,104

Education

3,443

Business Administration

2,257

Law

1,958

Philosophy

1,941

Environmental Sciences

1,469

Health Sciences

1,376

Mass Communications

1,370

Women’s Studies

1,271

Switzerland, and UK). While respondents identified with institutions in 19 countries, 91 % were located in the US, 7 % were in Canada, and only 2 % represented the other 17 countries; there is a clear US bias in the response. These high degrees of association with academic institutions across all categories should not be taken to be representative of career trajectories for doctoral students; rather, they are possibly an artifact of collecting the sample based on identifiable contact

123

Scientometrics (2014) 101:1695–1714

1703

Table 7

Socio-demographic characteristics of respondents, by discipline

 
 

Economics (n = 45) (%)

Political Science (n = 81) (%)

Sociology (n = 64) (%)

Gender

Female

35.6

47.5

60.0

Male

64.4

52.5

40.0

Race

Caucasian

85.4

90.1

79.7

Non-caucasian

14.6

9.9

20.3

Age (years)

26–35

72.7

42.0

30.8

36–45

18.2

35.8

36.9

\45

9.1

22.2

12.3

Table 8

Current institutional affiliation of respondents

 
 

Acad. (%)

Corp. (%)

Govt (%)

Non-profit (%)

Other (%)

No answer (%)

Economics

75.6

6.7

11.1

2.2

2.2

2.2

Political Science

79.0

3.7

8.6

3.7

2.5

2.5

Sociology

86.2

3.1

1.5

6.2

1.5

1.5

information, which was potentially more easily located for those holding academic posts (due to the public availability of curricula vitae and other relevant material).

Questionnaire results

Figure 3 depicts the degree to which respondents who completed dissertations in certain SCs identified with the same disciplinary label. The first column represents responses to the question: ‘‘What was the field of study of your doctorate degree?’’ Questions 2 and 3 asked respondents to identity the disciplines from which they consumed (2) and to which they disseminated (3) information. Questions 4 and 5 asked respondents to identify the disci- plines from which they primarily drew theories (4) and methods (5). Question six asked:

‘‘How would you self-identify your expertise or research area to a future employer?’’ Question seven asked: ‘‘How would you self-identify your expertise or research area to persons outside the workplace?’’ These used the same coding scheme, where explicit mention of the disciplinary label was indicated with a darker color and applications including the disciplinary label (e.g., sociology of x) were indicated in a lighter color. Our goal was to assess the degree to which the respondents identified with the broad disciplinary label (i.e., Political Science, Sociology, and Economics), as well as applied and specialized terms that would be recognizable to an outsider (e.g., sociology of x, economics of y, politics of z). We did not attempt to differentiate specializations, as our aim was only to see how much the broad category was used as an identify marker for the respondents. For respondents in the Economics set, 50 % (23/46) answered ‘‘Economics’’ to the first question (i.e., ‘‘What was the field of study of your doctorate degree?’’). These are indi- cated in dark red in the figure. Ten respondents (shown in lighter red in Fig. 3) identified a

123

1704

Scientometrics (2014) 101:1695–1714

Fig. 3

Association with

disciplinary labels, by each survey question. Grey indicates that no response was received. Cases with non-response for all seven categories were eliminated

1704 Scientometrics (2014) 101:1695–1714 Fig. 3 Association with disciplinary labels, by each survey question. Grey indicates

disciplinarity perspective of economics (e.g., agricultural and resource economics, business economics, nursing economics, and health economics). A few identified specialties of economics, including macroeconomics and time series econometrics (also identified in the

123

Scientometrics (2014) 101:1695–1714

1705

lighter shade). Other areas included industrial organization, history, urban and regional planning, finance, operations research, sociology, and marine biology. No color is repre- sented for these. Grey indicates that the question was not answered. Similarly, 45 % of those with Political Science SCs on their dissertations said that their doctoral degree was in ‘‘Political Science.’’ Three provided specialized terms, such as comparative politics or political theory. Many listed disciplines outside of Political Sci- ence, such as Anthropology, Education, Geography, French, and Philosophy. Others listed specialties that are closely related (and possibly considered within) the field of political science: public administration, public policy, and international relations (these were not shaded). Sociology demonstrated the highest permeability. Only 15 % of respondents with Sociology SCs indicated that they received their degrees in Sociology. There were a few instances of specific applications of sociology (e.g., sociology of gender and sexuality; sociology of culture and inequality; sociology of politics); however, most respondents were outside of the field of sociology (for example, Anthropology, Nursing, or Business). Social Work was also fairly highly represented in the responses (this was considered a distinct discipline). In Economics, the highest proportion of respondents associated with the explicit dis- ciplinary label was regarding the use of theories, with 60 % of respondents indicating ‘‘Economics’’ as the primary discipline from which they use theories. The majority of respondents indicated ‘‘Economics’’ for all questions except question 6 and 7, where only 20 % reported using the term ‘‘Economics’’ to describe their expertise to an employer. Only a third said that they would use this term to describe themselves to people outside their workplace. More individuals identified Political Science as the area of their doctorate (44 %) than those who indicated that they primarily used political science theories in their work (38 %). Very few used this label to self-identify to employers (11 %) or to those outside the workplace (17 %). The strongest association with Sociology was in terms of methods, with 19 % of respondents indicating that they primarily drew upon sociological methods in their work. Nearly 18 % indicated that they drew from sociological theories, fewer than those who indicated that they received a degree in sociology (15 %). A very small percentage self- identified with Sociology to describe themselves to employers (5 %) or others (4 %). The majority of respondents whose dissertations were granted in Sociology did not indicate any apparent association with Sociology. Given that the SC that appears first is considered the ‘‘primary SC,’’ it is possible that more reliable results might have been generated if the analysis were restricted to primary SCs. This was investigated by analyzing the percentage of all 500 dissertations in the sampling frame that were associated with the degree as a primary SC, the proportion of dissertations among the respondents with the selected discipline as the primary SC, and the percent of these that were ‘‘exact matches’’ with the degree in which the respondent received their degree. By exact match, we refer to those respondents which mentioned the disciplinary label in their description of the discipline in which they received their degree. Table 9 provides the percentages of all dissertations (from the sampled 500) that had the given discipline as the primary SC (i.e., the first-listed first-level SC); the percent of dissertations among respondents with the given discipline as the primary SC; the percent of respondents who gave an ‘‘exact match’’ response to question 1 on the survey; the percent of respondents with the designated primary SC dissertation who gave an ‘‘exact match’’

123

1706

Scientometrics (2014) 101:1695–1714

answer; and the percentage of respondents without the given discipline as a Primary SC who gave an ‘‘exact match’’ answer. As shown, our respondent set matched the proportions of dissertations from the entire set with the discipline as the primary SC. In all cases, the proportion of respondents with exact matches improved if only those dissertations with the discipline in the primary position were included. For example, whereas only 50 % of respondents stated that they received their degree in Economics, this improved to 84 % if only those respondents whose dissertations had Economics as a Primary SC were considered. However, although this eliminates some false positives, it does not completely mitigate the false negative situation:

half of the respondents whose dissertations did not list Economics as a Primary SC identified Economics as their discipline. There was also large variability across the dis- ciplines: even when looking at only dissertations with Sociology as a Primary SC, only 36 % were exact matches (however, this is limited by the small sample size of this test).

Validation exercise

In order to supplement these findings, we investigated three disciplines (Biology, Chem- istry, and Computer Science) that lay outside of the social sciences. A series of Google and ProQuest searches were conducted on the authors of 100 dissertations for each of these three disciplines in order to determine the degree to which their field of study accorded with the SC associated with their dissertation. Table 10 presents the percentage of our sample that had the designated discipline as a Primary SC; the percentage for whom the Primary SC and the field of study were an ‘‘exact match’’; the percentage of dissertations that had the designated discipline as a Primary SC that were an ‘‘exact match’’; and the percentage of people who had a dissertation with another Primary SC but had received a degree in Biology, Chemistry, or Computer Science (respectively). As with our previous analysis, there was an improvement in accuracy when only the primary SC was considered, although the gain was minimal in Biology and Chemistry. There was also a high degree of potential false negatives in the field of Chemistry. We further analyzed the instances in which each discipline was listed as a Primary SC but was not considered an exact match. There were 55 such instances in Biology. Many of the degrees in this category were from related areas: Biochemistry, Bioinformatics, Bio- logical Chemistry, Biological Oceanography, Biomedical Sciences, Biophysics, and Bio- statistics (these represented 16 of the dissertations). There were also other disciplines such as Chemistry, Computer Science, Psychology, Ecology, Geology, and Zoology. Finally, there were specializations such as Neuroscience (8 of the sampled dissertations). There were nineteen instances in which Chemistry was listed first, but it was not an exact match. These included some aspects of chemistry (e.g., Chemical and Biochemical Engineering, Chemical and Biomolecular Engineering, Chemical Biology, and Chemical Engineering). Other disciplines represented among these dissertations were Physics, Biology, and Information and Library Science. There were only two instances in which Computer Science was listed as a primary SC but was not an exact match with the discipline of the degree. In both cases it was a very similar term: Engineering (Computer Engineering) and Informatics and Computing. The validation study confirmed the results of the main study by: (1) suggesting improved accuracy when Primary SCs were used; and (2) identifying large variability in both false positives and false negatives across disciplinary areas.

123

Scientometrics (2014) 101:1695–1714

1707

Table 9

Relationship between the primary SCs assigned to the sampled dissertations and the disciplinary

labels used by respondents to describe the discipline in which they received their degree (i.e., Question 1 in

Fig. 3)

 

Primary SC

Primary SCs

Exact matches

% of primary SC

% of non-primary

(out of 500)

(from

(from

that were exact

SCs that were exact

(%)

respondents)

respondents)

matches

matches

Economics

68

69 % (n = 33)

50 % (n = 23)

84 % (n = 26) a

50 % (n = 7)

Political

42

48 % (n = 40)

45 % (n = 37)

64 % (n = 25)

41 % (n = 17)

Science

Sociology

20

23 % (n = 17)

15 % (n = 11)

36 % (n = 5)

10 % (n = 5)

a Percentage of those who responded to the question for this and all other similar calculations

Table 10 Relationship between the primary SCs assigned to dissertations and the discipline in which dissertation authors received their degree

 

Primary

Exact

% of primary SC that were

% of non-primary SC that were

SC (%)

matches (%)

exact matches

exact matches

Biology

90

37

39 % (n = 35)

20 % (n = 2)

Chemistry

82

72

77 % (n = 63)

50 % (n = 9)

Computer

48

70

96 % (n = 46)

46 % (n = 24)

Science

Discussion and conclusion

The results of our survey question the utility of traditional disciplinary labels in a time of heightened specialization and interdisciplinary. In examining the entire ProQuest dataset, Sociology was shown to be the most specialized and also the most likely to appear in the presence of another first-level SC. These SCs were spread across the disciplinary spectrum, including medical sciences, social sciences, humanities, and professional fields. Economics was slightly narrower, with little more than one-third of the dissertations in this area appearing in conjunction with another SC. Political Science was the least specialized, according to ProQuest’s schema, but it was still fairly interdisciplinary, with more than half of the dissertations occurring in the presence of another first-level SC. The goal of the questionnaire was to identify the ways in which the dissertation authors associated with the SC listed on the dissertation. Many elements were investigated: indi- viduals’ degrees, their use of theory and methods, the disciplines from which they con- sumed and disseminated knowledge, and the way they described their expertise to employers and the public. As shown in Fig. 3, there were varying degrees to which respondents identified with the disciplinary labels into which they were classed. Sociology was the most diffuse, with very few people using this term to describe their scholarly activities or identities. A majority of respondents did not refer to ‘‘sociology’’ in any of their responses, even though their dissertation was classified under this SC. One concern could be that Social Sciences are particularly permeable, interdisciplinary, and more diffuse than STEM disciplines. To test this, we examined a random sample of dissertations in Biology, Chemistry, and Computer Science. As was shown, there was high variability in the degree to which the SCs captured the disciplinary labels under which the disciplines were conferred. Biology was the most diffuse and had the highest potential for

123

1708

Scientometrics (2014) 101:1695–1714

obtaining false positives, even when restricting the analysis to primary SCs. Computer Science had the most reliable results (although still exhibiting a fairly high level of false negatives) when the sampling was restricted to primary SCs. These findings suggest that caution be exercised when using ProQuest SCs as proxies for disciplinarity. However, the variability across the disciplines suggests that ProQuest may not be systematically misclassifying dissertations; rather, it may well be that the traditional disciplinary labels no longer hold. In the qualitative component of this study, many respondents provided detailed and specialized responses—rather than giving a single word disciplinary label, they described at length the topic that they studied. It is possible that topics carry more weight than disciplines in specialized environments. Specialization may also be characterizing the marketplace, as epitomized by the degree labels for Biology. Degrees were no longer conferred in ‘‘Biology,’’ but rather in Bioin- formatics, Biological Oceanography, and Biostatistics. There is a ‘‘balance between spe- cialization and integration’’ (Klein 1990, p. 38) that must be carefully managed. Furthermore, it begs the question of what institutions gain by specializing their degrees. Is there a newcomer’s advantage to being the first school to offer a degree in a specialized topic? Perhaps traditional disciplines are no longer the Gold Standard, thus reflecting a marketplace that is now nimble enough to evaluate people on their topical distinctions rather than on their disciplinary affiliations. In this case, ‘‘interdisciplinary’’ may not be the appropriate term to characterize the new marketplace for doctoral students. We may live in a post-interdisciplinary space in which scholars wish to move more freely across disciplinary boundaries and aggregate topically more than disciplinarily. Whitley suggested this was the case as early as 1984 when he wrote that ‘‘the discipline as a set of research activities has outgrown the departmental basis of employment and careers’’ and that the ‘‘identification of intellectual disciplines with university undergraduate departments is no longer a social reality in many science’’ (Whitley 1984, p. 18). There remain, however, some disciplines that are less permeable and present more barriers for entry (Klein 1993), and it seems that disciplines still remain a dominant organizing agent for institutions of higher education and retain ‘‘enormous influence over the organization and production of knowledge’’ (Klein 1993, p. 185). This begs an analysis of the ways in which disciplinary labels may connote something different between academic markets and professional labor markets. There is a historically complicated relationship between academic knowledge and professionalism. Abbott (1988) argued that the ‘‘ability of a profession to sustain its jurisdictions lies partly in the power and prestige of its academic knowledge’’ (pp. 53–54). However, there are numerous conflicts over jurisdictional boundaries both among and within disciplines, one that is complicated by the growing specialization of research (Jacobs 2013). There are also many professionals who continue to contribute to academic knowledge once they have formally left academe (Breimer and Leksell 2013). Future research is needed to understand whether specialization in the labor market is parallel to specialization in the academic research market.

Limitations

There are, of course, a number of limitations to the present study. As noted above, our sample is restricted to those with an available e-mail address. It is likely that those with an e-mail address are more likely to be (a) employed and (b) associated with an academic institution. Given the ‘‘risks’’ associated with interdisciplinary research (Kniffin and Hanks 2013), it is possible that the sample might skew towards those who are more ‘‘disciplinary’’

123

Scientometrics (2014) 101:1695–1714

1709

in nature. However, current research has shown that individuals conducting interdisci- plinary dissertation research are more likely to be employed in academe (Millar 2013). Our results seem to confirm this. The mode of delivery is also a limitation as e-mail has been shown to have lower response rates than other modes of survey research (Schaefer and Dillman 1998). Our response rates were similar to those found in a review of survey research with academic participants (Dykema et al. 2013), but still lower than averages for other modes of delivery. We used a number of tested incentives (i.e., personalizing the survey and a monetary incentive), but it’s possible that our response rate could have been improved with another incentive structure (e.g., advance notice). The choice of fields is also a limitation. Our objective was to choose a wide variety of disciplines, each with their own particularistic identities—in both the main and validation studies. However, there is always a somewhat arbitrary nature to discipline selection and we must acknowledge that different results may have been generated given a different choice of disciplines. Furthermore, we have classified Economics, Sociology, and Political Science as social sciences and Biology, Computer Science, and Chemistry as non-social sciences. However, these distinctions may not hold across all subareas of these disciplines.

Future research

Future research should investigate how people can appropriately evaluate these topic-based scholars, what implications exist for the reorganization of institutions, how these topic distinctions play into information consumption and disciplinary identity, and how repu- tation and authority can be conferred given this highly permeable space. Although inter- disciplinarity is often seen as a positive thing (Rafols and Meyer 2010), there are dangers and consequences for a lack of disciplinary grounding. Disciplines are not only institu- tional conventions, but are also bound by shared paradigms: i.e., the types of questions that can be asked, the ways in which they can be asked, and the answers that can be retrieved (Kuhn 1970). Members of disciplines establish their own sets of cultural rules and norms, allowing both members and outsiders to make sense of activities and interactions through the application of disciplinary-relevant frames (Goffman 1974). Disciplines provide the- oretical and methodological frameworks for the education and training of doctoral stu- dents. One might question how students can move freely across disciplinary boundaries— appropriating theories and methods as necessary—while still maintaining a deeply con- textualized knowledge of the appropriate use of these theories and methods. Furthermore, there is the conceptual problem of what happens when disciplinary work is abandoned in favor of interdisciplinary research. If interdisciplinary research represents ‘‘the juxtapo- sition of disciplines’’ (Klein 1990, p. 56), who is contributing to the tool chest from which these researchers are drawing theories and methods? Can interdisciplinary research exist in the absence of disciplines? Klein (1993) suggested that boundary work involves not only ‘‘crossing’’ and ‘‘deconstructing,’’ but also ‘‘reconstructing boundaries’’ (p. 196). There- fore, it is possible that interdisciplinary areas are merely pre-disciplinary areas in the process of reconstructing boundaries. Alternatively, we may be transitioning into a post- interdisciplinary environment, where identities are constructed in topical, rather than disciplinary, frames of reference.

Acknowledgments This work was funded by the Science of Science Innovation and Policy (SciSIP) program of the National Science Foundation (Grant no. 1158670).

123

1710

Scientometrics (2014) 101:1695–1714

Appendix: Questionnaire

1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123
1710 Scientometrics (2014) 101:1695–1714 Appendix: Questionnaire 123

Scientometrics (2014) 101:1695–1714

1711

Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123
Scientometrics (2014) 101:1695–1714 1711 123

123

1712

Scientometrics (2014) 101:1695–1714

1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123
1712 Scientometrics (2014) 101:1695–1714 123

123

Scientometrics (2014) 101:1695–1714

1713

Scientometrics (2014) 101:1695–1714 1713 References Abbott, A. (1988). The system of professions: An essay on the
Scientometrics (2014) 101:1695–1714 1713 References Abbott, A. (1988). The system of professions: An essay on the
Scientometrics (2014) 101:1695–1714 1713 References Abbott, A. (1988). The system of professions: An essay on the
Scientometrics (2014) 101:1695–1714 1713 References Abbott, A. (1988). The system of professions: An essay on the
Scientometrics (2014) 101:1695–1714 1713 References Abbott, A. (1988). The system of professions: An essay on the
Scientometrics (2014) 101:1695–1714 1713 References Abbott, A. (1988). The system of professions: An essay on the

References

Abbott, A. (1988). The system of professions: An essay on the division of expert labor. Chicago, IL: The University of Chicago Press. Abbott, A. (1999). Department and discipline: Chicago sociology at one hundred. Chicago, IL: The Uni- versity of Chicago Press. Andersen, J. P., & Hammarfelt, B. (2011). Price revisited: On the growth of dissertations in eight research fields. Scientometrics, 88, 371–383. Anseel, F., Lievens, F., Schollaert, E., & Choragwicka, B. (2010). Response rates in organizational science, 1995–2008: A meta-analytic review and guidelines for survey researchers. Journal of Business and Psychology, 25(3), 335–349. Baumann, B. (1975). Imaginative participation: The career of an organizing concept in a multidisciplinary context. The Hague: Martinus Nijhoff. Breimer, L. H., & Leksell, J. (2013). Longitudinal and cross-sectional study of registered nurses in Sweden who undertake a PhD showing that nurses continue to publish in English after their PhD but male nurses are more productive than female nurses. Scientometrics, 87(2), 337–345. Dykema, J., Stevenson, J., Klein, L., Kim, Y., & Day, B. (2013). Effects of e-mailed versus mailed invitations and incentives on response rates, data quality, and costs in a web survey of university faculty. Social Science Computer Review, 31(3), 359–370. Garfield, E. (1978). The Gordian Knot of journal coverage: Why we can’t put all the journals you want into the Current Contents edition you read. Current Contents, 13, 5–7. Goffman, E. (1974). Frame analysis: An essay on the organization of experience. New York, NY: Harper & Row. Jacobs, J. A. (2013). In defense of disciplines: Interdisciplinarity and specialization in the research uni- versity. Chicago, IL: The University of Chicago Press. Klein, J. T. (1990). Interdisciplinarity: History, theory, and practice. Detroit: Wayne State University Press.

123

1714

Scientometrics (2014) 101:1695–1714

Klein, J. T. (1993). Blurring, cracking, and crossing: Permeation and the fracturing of discipline. In E. Messer-Davidow, D. R. Shumway, & D. J. Sylvan (Eds.), Knowledges: Historical and critical studies in disciplinarity (pp. 185–211). Charlottesville, London: The University Press of Virginia. Klein, J. T. (1996). Crossing boundaries: Knowledge, disciplinarities, and interdisciplinarities. Char- lottesville, VA: University of Virginia Press. Kniffin, K. M., & Hanks, A. S. (2013). Boundary spanning in academia: Antecedents and near-term consequences of academic entrepreneurialism. Cornell Higher Education Research Institute (CHERI) Working Paper 158. Kuhn, T. S. (1970). The structure of scientific revolutions. Chicago: University of Chicago Press. Leydesdorff, L., Carley, S., & Rafols, I. (2013). Global maps of science based on the new Web-of-Science categories. Scientometrics, 94(2), 589–593. Leydesdorff, L., & Rafols, I. (2009). A global map of science based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348–362. Leydesdorff, L., Rotolo, D., & Rafols, I. (2012). Bibliometric perspectives on medical innovation using the medical subject headings of PubMed. Journal of the American Society for Information Science and Technology, 63(11), 2239–2253. McLean, A. (2012). Personal correspondence to Cassidy Sugimoto via email on Oct 26, 2012. Millar, M. M. (2013). Interdisciplinary research and the early career: The effect of interdisciplinary dis- sertation research on career placement and publication productivity of doctoral graduates in the sci- ences. Research Policy, 42(5), 1152–1164. Morillo, F., Bordons, M., & Gomez, I. (2003). Interdisciplinarity in science: A tentative typology of disciplines and research areas. Journal of the American Society for Information Science and Tech- nology, 54(13), 1237–1249. Ni, C., & Sugimoto, C. R. (2012). Using doctoral dissertations for a new understanding of disciplinarity and interdisciplinarity. Poster. In Annual Meeting of the American Society for Information Science and Technology. Baltimore, MD. Ni, C., & Sugimoto, C. R. (2013a). Exploring interdisciplinarity in Economics through academic genealogy:

An exploratory study. Poster. In Proceedings of the International Society for Scientometrics and Informetrics Conference. Ni, C. & Sugimoto, C. R. (2013b). Academic genealogy as an indicator of interdisciplinarity: A preliminary examination of sociology doctoral dissertations. In Poster iConference. Ni, C., Sugimoto, C. R., & Jiang, J. (2013). Venue–author–coupling: A novel measure for identifying disciplines through social structures. Journal of the American Society for Information Science and Technology, 64(2), 265–279. Porter, A. L., & Rafols, I. (2009). Is science becoming more interdisciplinary? Measuring and mapping six research fields over time. Scientometrics, 81(3), 719–745. Rafols, I., & Meyer, M. (2010). Diversity and network coherence as indicators of interdisciplinarity: Case studies in bionanoscience. Scientometrics, 82, 263–287. Schaefer, D. R., & Dillman, D. A. (1998). Development of a standard e-mail methodology: Results of an experiment. The Public Opinion Quarterly, 62(3), 378–397. Sugimoto, C. R. (2014). Academic genealogy. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics:

Metrics-based evaluation of research. Boston, MA: MIT Press. Sugimoto, C. R., Ni, C., Russell, T. G., & Bychowski, B. (2011). Academic genealogy as an indicator of interdisciplinarity: An examination of dissertation networks in Library and Information Science. Journal of the American Society for Information Science and Technology, 62(9), 1808–1828. doi:10.

Turner, S. (2000). What are disciplines? And how is interdisciplinarity different? In P. Weingart & N. Stehr (Eds.), Practising interdisciplinarity (pp. 46–65). Toronto: University of Toronto Press. Whitley, R. (1984). The rise and decline of university disciplines in the sciences. In R. Jurkovich & J. H. P. Paelinck (Eds.), Problems in interdisciplinary studies: Issues in interdisciplinary studies (pp. 10–25). Aldershot, Hampshire: Gower. Ying, T. Y., & Xiao, H. G. (2012). Knowledge linkage: A social network analysis of tourism dissertation subjects. Journal of Hospitality and Tourism Research, 36(4), 450–477.

123