SPECIAL ARTICLE www.jasn.

org

Quantity and Reporting Quality of Kidney Research

Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Markos Kyriakos Tomidis Chatzimanouil, Louise Wilkens, and Hans-Joachim Anders

Division of Nephrology, Medizinische Klinik and Poliklinik IV, Klinikum der Universität München, Ludwig Maximilians
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023

University München, Munich, Germany

ABSTRACT
Background In 2004, researchers reported that the number of nephrology clinical have become available, and adherence
trials was low and that the reporting quality of such trials was suboptimal. Further- is requested by the leading journals in
more, the number or quality of preclinical kidney-related studies has not been sys- internal medicine and nephrology.2,3
tematically evaluated. Thus, we speculated on an increasing
Methods We performed a systematic review of randomized clinical trials published quantity and reporting quality of kid-
in 1966–2017 (listed in the Cochrane Library) and preclinical studies published in ney-related clinical trials within the last
1945–2017 (listed in PubMed). For reporting quality analysis, we evaluated the final 15 years as well as preclinical studies, the
main paper of 118 clinical trial reports and 135 preclinical studies published in lead- latter not being systematically evaluated
ing journals in 1996, 2006, and 2016 on the basis of criteria from the widely used before.
CONSORT and ARRIVE guidelines.
Results The annual number of reports of clinical kidney-related trials more than
METHODS
doubled between 2004 and 2014 along with reports in other medical disciplines.
Hypertension remains the dominant focus of study, but ongoing trials also center on
The study is composed of two parts. One
CKD, ESRD, and AKI. The reporting quality analysis revealed improvements, but
is the quantitative analysis regarding the
deficits in reporting of clinical trial design, mode of randomization, and intention-to-
numbers (both total and per year) of ran-
treat analysis remain. Annual numbers of kidney-related preclinical studies remained
domized, controlled trials (RCTs) and
low between 1945 and 2017 compared with other disciplines. Reporting quality
preclinical studies in the field of nephrol-
analysis of preclinical studies revealed substantial reporting deficits across all lead-
ogy compared with other specialties of
ing journals, with little improvement over the last 20 years, especially for group size
internal medicine as well as the distribu-
calculations, defining primary versus secondary outcomes, and blinded analysis.
tion of nephrology studies inside the
Conclusions Nephrology studies keep increasing in number but still lag behind field. The second part is a qualitative
other medical disciplines, and the quality of data reporting in kidney research can analysis of a sample of papers (RCTs
be further improved. and preclinical studies) on the basis of
J Am Soc Nephrol 30: 13–22, 2019. doi: https://doi.org/10.1681/ASN.2018050515 criteria for trial reporting using estab-
lished guidelines, such as the CONSORT
Statement and the ARRIVE guidelines. A
test analysis using ten RCTs and ten pre-
In 2004, Strippoli et al.1 presented quan- for trial reporting, greater attention to clinical studies was performed by two
titative and qualitative analyses of the the trial methods and not just the re- investigators (M.K.T.C. and L.W.), and
clinical trials performed in the field of sults.”1
nephrology. As a disappointing finding, Increasing abstract submissions to in-
kidney-related clinical trials were not ternational kidney conferences, a sub-
only low in number compared with stantial increase in nephrology journals, Published online ahead of print. Publication date
other medical disciplines but also of and increased scientific productivity of available at www.jasn.org.

poor quality, either as conducted or re-
ported.1 The authors concluded: “The
challenges of improving the quality and
quantity of trials in nephrology are sub-
stantial, but they can be overcome by us-
ing standard guidelines and checklists
evolving countries give the impression
that the number of kidney-related clini-
cal trials may have substantially increased
compared with 15 years ago. In addition,
guidelines for the conduct and reporting
of clinical trials and preclinical studies
Correspondence: Dr. Hans-Joachim Anders,
Medizinische Klinik and Poliklinik IV, Klinikum der
Universität München—Innenstadt, Ziemssenstrasse
1, 80336 Munich, Germany. Email: hjanders@med.
uni-muenchen.de
Copyright © 2019 by the American Society of
Nephrology

J Am Soc Nephrol 30: 13–22, 2019 ISSN : 1046-6673/3001-13 13

 SPECIAL ARTICLE www.jasn.org
it defined an interobserver analytical co-
herence of 90.3%.
Quantitative Analyses of Clinical
RCTs
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW
Databases were selected for the RCTs as
well as the preclinical analyses from which
the data were extracted. For the RCTs, the
database selected was the Cochrane Li-
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
brary, mainly because it uses an integrated
system of trees and subtrees for each MeSH
term, which provided a better overview of
the research process compared with other
databases. Because the Cochrane Library is
limited to clinical trials, PubMed was used
for the preclinical studies.
Number and Proportion of RCTs in Ne-
phrology Compared with Other
Specialties
For the question on the number of RCTs
in nephrology in comparison with other
specialties, an MeSH term, thought to
most accurately summarize the data in
question, or a combination of two or
more relevant MeSH Terms was used
for each specialty as listed in Supplemen-
tal Material.
Coverage of RCTs within Nephrology
RCTs from nine areas were retrieved for
the years 1966–2016 using the following
MeSH terms: “Renal Insufficiency,
Chronic,” “Kidney Transplantation,”
“Diabetic Nephropathies,” “AKI,” “Peri-
toneal Dialysis,” “GN,” “Hypertension,
Renal,” “Kidney Calculi,” and “Hyper-
tension.” Because “Renal Dialysis,”
“Peritoneal Dialysis,” and “Kidney
Transplantation” were subterms of the
more general MeSH term “RRT,” we cre-
ated another term, named “Kidney
Exp.,” to include all of them.
Quantitative Analyses of Preclinical
Studies
A similar search regarding the preclinical
studies was conducted using PubMed by
applying the same MeSH terms (before
the conversion to match the Cochrane
System) used for the RCTsearch. To limit
the results specifically to animal trials,
the limit “Animals” in the PubMed in-
terface was used, and the results were
extracted for the timeline 1945–2016. A
14 Journal of the American Society of Nephrology
per year categorization was applied for
the construction of the per year
diagrams.
Qualitative Analyses
Paper Selection Criteria
We searched the PubMed database. As
representative RCT sample periods, we
selected the years 1996, 2006, and 2016
and identified all kidney-related RCTs
from the top five journals as assessed
by impact factor and the number of kid-
ney-related RCTs per year: The New Eng-
land Journal of Medicine (NEJM) The
Lancet, Kidney International, Journal of
American Society of Nephrology (JASN),
and American Journal of Kidney Diseases
(AJKD). For the qualitative analysis of
preclinical studies, we used the same
years (1996, 2006, and 2016) but limited
sample collection to January to March of
each year from the journals JASN, Kid-
ney International, Nephrology Dialysis
Transplantation (NDT; selected by im-
pact factor and tendency to publish pre-
clinical studies in the kidney domain)
using the PubMed database. The search
for the RCTs was conducted using the
MeSH term “kidney diseases” and the
limits “Randomized Controlled Trial”
and “Humans,” limited each time by
the specific journal of publication and
the appropriate year. For the animal tri-
als, the same MeSH term “kidney
diseases” was used and limited by “Ani-
mals,” and the search was conducted per
year for each of the three journals. Only
results published inside the first 3
months of each year were included. In-
clusion criteria for both the RCTs and
the preclinical studies were that the pa-
pers were original articles, in English,
and published online. Review articles,
editorials, special articles, and commen-
taries were excluded. From the RCT
analysis, observational, prospective,
and nonrandomized trials were ex-
cluded. Because the ARRIVE criteria
apply to in vivo animal experiments, pre-
clinical studies exclusively mechanistic or
only in vitro studies were excluded.
Quality Assessment
The methodological quality of all of these
studies was assessed by one independent
Significance Statement
Turning the research lens inward to assess
the state of nephrology research, this sys-
tematic review assesses the quantity of
clinical trials and preclinical studies re-
ported over the past five to seven decades,
finding that, although the numbers of clin-
ical and preclinical studies have sub-
stantially increased with time, nephrology
still lags behind other medical disciplines. In
addition, the authors’ quality analysis of
nephrology research reports published by
leading journals in the past two decades
(on the basis of applying criteria from the
widely used CONSORT and ARRIVE re-
search guidelines to assess the main body
of the final paper) reveals that, although
deficits remain, clinical trial reporting
quality has improved. However, many gaps
persist in reporting quality of preclinical
studies. Identifying deficits may help im-
prove quantity and quality of kidney re-
search and accelerate advancement.
investigator (M.K.T.C.) using the criteria
mainly on the basis of the original CON-
SORT 2010 checklist for RCTs2 and the
ARRIVE guidelines for the preclinical
studies.3 Only the main body of the final
paper was analyzed, and supplementary
information and previously published
study protocols were not analyzed. For
the RCT analysis, we extracted data by
allocating points (zero for insufficient,
one for unclear or insufficiently report-
ed, and two for adequate reporting) to
each of the 19 core items of the CON-
SORT Statement regarding “Title and
Abstract,” “Introduction, ” “Methods,”
and “Results” and each of the subitems.
In addition, to match the previous anal-
yses from Strippoli et al.1 and Deo et al.,4
two extra items were added (i.e., “Inten-
tion-to-treat analysis” and “Loss-to-
analysis”). “Intention-to-treat analysis”
was rated as adequate when sufficient
data were included to confirm that the
analysis regarding the primary end point
was undertaken according to the treat-
ment assignment and that the numbers
of participants randomly assigned were
identical to the numbers of participants
analyzed, irrespective of whether “inten-
tion-to-treat” was stated or not. We also
tried to calculate the “Loss-to-analysis”
for each trial included so as to try to de-
termine what percentage of participants
J Am Soc Nephrol 30: 13–22, 2019

randomly assigned was not included in
the analysis. In total, 35 items regarding
trial reporting for RCTs were graded.
For the analysis of the preclinical stud-
ies, the 15 core items of the ARRIVE
guidelines regarding “Introduction,”
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW
“Methods,” and “Results” were included.
In addition, we added to the core items 3,
6 (subitem “d: A time line diagram or
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
flow chart” was included by initiative of
the researchers), 10, and 13. The subi-
tems were assessed separately; other-
wise, they were included in the grading
method of the core item (zero for insuf-
ficient, one for unclear or insufficiently
reported, and two for adequate report-
ing) for a total of 23 graded items regard-
ing trial reporting for animal trials. A
more detailed view of the criteria used
to assess the quality of the trials and the
grading system used to evaluate each
item separately can be found in Supple-
mental Tables 1–3.
Statistical Analyses
Data are presented as mean6SD. Com-
parison of groups was performed using
ANOVA, and post hoc Bonferroni correc-
tion was used for multiple comparisons.
A value of P,0.05 was considered to in-
dicate statistical significance.
RESULTS
Number and Trends of Clinical Trials
in Nephrology Compared with
Other Medical Disciplines
Our Cochrane MeSH tree analysis dis-
played the reported clinical trials across
medical disciplines (Figure 1A). Neurol-
ogy and cardiology kept reporting the
most clinical trials since the early
1990s, whereas both MeSH term “kidney
diseases” and its definition extended by
RRTs (“Kidney Exp.”) remained at the
lowest ranks of medical disciplines (Fig-
ure 1A). The average slope of additional
clinical trials reported each year from
1966 to 2003 was 27.7614.6 for all dis-
ciplines, and slopes were 7.5 and 11.0 for
kidney diseases and expanded kidney
diseases, respectively (both P values ver-
sus all disciplines ,0.001). Since the last
evaluation in 2003 (years 2004–2014),
J Am Soc Nephrol 30: 13–22, 2019
the slope has increased to 68.4642.2
for all disciplines, indicating a profound
increase in number and spread among
the disciplines in annual clinical trial re-
porting. In the same period, kidney dis-
eases and their expanded definition
increased to 23.3 and 31.2, respectively
(both P values versus all disciplines
,0.001). The years 2015 and 2016
were excluded from this analysis, be-
cause delays in MeSH term indexing
within the PubMed database led to de-
clining curves for all disciplines (M.
Collins, United States National Library
of Medicine, personal communication).
Over the entire period, all disciplines
displayed an annual increase of reported
clinical trials of 38.4621.2, whereas
kidney diseases displayed an annual in-
crease of reported clinical trials of 11.3
and expanded kidney diseases displayed
an annual increase of reported clinical
trials of 15.8 (both P values versus all
disciplines ,0.001). Together, the num-
ber of kidney-related clinical trials has
increased but less so compared with
other disciplines.
Clinical Trial Coverage of Different
Kidney Disease Entities
Cochrane database MeSH tree analysis
for trial coverage of different disease en-
tities within the field of nephrology
showed that clinical trials addressing hy-
pertension have been the highest in
number starting from the 1970s and
that they continue to predominate (Fig-
ure 1B). RRT-related trials were already
much lower in number, and disease
entities, such as GN, AKI, or kidney cal-
culi, although all being prevalent, con-
tribute only a few papers to clinical trial
activity (Figure 1B).
Number of Preclinical Studies in
Nephrology Compared with Other
Medical Specialties
Preclinical research activity was analyzed
using PubMed from 1945 to 2017. On-
cology, infectious diseases, and neurol-
ogy have reported the most studies since
the 1990s, whereas kidney disease–re-
lated studies, in narrow and expanded
definitions, remain at the low end
among the medical disciplines since
www.jasn.org SPECIAL ARTICLE
1945 (Figure 2A). The average slope of
additional preclinical studies reported
each year from 1945 to 2003 was 95.06
60.2 for all disciplines, and the average
slopes were 26.7 and 30.5 for kidney dis-
eases and expanded kidney diseases,
respectively (both P values versus all dis-
ciplines ,0.001). Since the last evalua-
tion in 2003 (years 2004–2014), the
slope has increased to 313.86217.0 for
all disciplines, indicating a profound in-
crease in number and spread among the
disciplines in annual preclinical study
reporting. In the same period, studies
on kidney diseases and their expanded
definition increased to 88.5 and 89.3,
respectively (both P values versus all
disciplines ,0.001). Over the entire
period, all disciplines display an an-
nual increase of reported clinical trials
of 133.66 85.1, whereas kidney dis-
eases display an annual increase of
reported clinical trials of 37.8 and ex-
panded kidney diseases display an an-
nual increase of reported clinical trials
of 40.5 (both P values versus all
disciplines ,0.001). Thus, although
increasing in number, the count of
kidney-related preclinical studies is
lagging behind other disciplines.
Preclinical Trial Coverage of
Different Kidney Disease Entities
Studies focusing on hypertension by far
outweigh all other kidney disease entities
and preclinical research as well (Supple-
mental Figure 1), but preclinical kidney
research topics covered a broader range
of topics than seen in clinical trials (Figure
2B). Whereas studies on kidney trans-
plantation revealed an early peak in the
mid-1960s, preclinical research activity
on GN substantially increased starting
from the early 1980s (Figure 2B). As a
more recent trend, studies addressing
AKI, CKD, and diabetic nephropathy
have become the most popular preclinical
research topics within the last decade
(Figure 2B).
Quality of Clinical Trial Reporting
To assess the quality of trial reporting, we
selected 125 publications from top med-
ical and nephrology journals as listed in
Supplemental Figure 2. Seven had to be
Anti-Peroxidasin Autoantibodies 15
 SPECIAL ARTICLE www.jasn.org

A
4500
Nervous System
Cardiovascular
4000 Neoplasms
Infection
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

3500 Nutrition and Metabolism

Respiratory Tract
Immune System
3000 Digestive System
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023

Skin and Connective Tissue

No. of RCTs

2500 Endocrine System

Musculoskeletal
Kidney Exp.
2000
Hemic and Lymphatic
Kidney
1500

1000

500

0
1966 1971 1976 1981 1986 1991 1996 2001 2006 2011 2016
Year

B
500
Hypertension
Renal Dialysis
450
Renal Insufficiency, Chronic
400 Renal Dialysis w/o Peritoneal Dialysis
Kidney Transplanation
350 Diabetic Nephropathies
Acute Kidney Injury
300
No. of RCTs

Peritoneal Dialysis
Glomerulonephritis
250
Kidney Calculi
Hypertension, Renal
200

150

100

50

0
1966 1971 1976 1981 1986 1991 1996 2001 2006 2011 2016
Year

Figure 1. Quantitative analysis and topic coverage of clinical trials in medical disciplines identified from the Cochrane database. Several
MeSH terms were applied to best cover each discipline as described in Methods. Nephrology (“Kidney”) is represented by the MeSH term
“kidney diseases.” Expanded (Exp.) nephrology (“Kidney Exp.”) also covers the MeSH term “RRT” (subterms included “renal dialysis,”
“peritoneal dialysis,” and “kidney transplantation”). (A) Annual number of clinical trials per discipline from 1966 to 2016. The sudden
decline in numbers in the years 2015–2016 should relate to delays in data inclusion. (B) Disease entities as defined by the available MeSH
terms were quantified as described in Methods. Annual number of clinical trials per kidney disease entity from 1966 to 2016. The sudden
decline in numbers in the years 2015–2016 should relate to delays in data inclusion. RCT, randomized, controlled trial; MeSH, medical
subject heading.

16 Journal of the American Society of Nephrology J Am Soc Nephrol 30: 13–22, 2019
 www.jasn.org SPECIAL ARTICLE

A
25000
Infection
Neoplasms
Nervous System
Cardiovascular
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

20000
Digestive System
Nutrition and Metabolism
Immune System
Respiratory Tract
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023

15000
No. of Trials

Endocrine System
Skin and Connective Tissue
Hemic and Lymphatic
Musculoskeletal
10000
Kidney Exp.
Kidney

5000

0
1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015
Year

B
500
Glomerulonephritis
Kidney Transplanation
450
Acute Kidney Injury
Renal Insufficiency, Chronic
400
Diabetic Nephropathies
350 Hemodialysis
Hypertensive Nephropathy
300 Peritoneal Dialysis
No. of Trials

Kidney Calculi
250

200

150

100

50

0
1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015
Year

Figure 2. Quantitative analysis of preclinical studies identified from PubMed. Several MeSH terms were applied to best cover each
discipline as described in Methods. Nephrology (“Kidney”) is represented by the MeSH term “kidney diseases.” Expanded (Exp.) ne-
phrology (“Kidney Exp.”) also covers the MeSH term “RRT.” (A) Annual number of preclinical studies per discipline from 1945 to 2016 and
(B) topic coverage among the preclinical studies trials of the nephrology domain (B). The sudden decline in numbers in the years
2015–2016 should relate to delays in data inclusion. MeSH, medical subject heading.

J Am Soc Nephrol 30: 13–22, 2019 Anti-Peroxidasin Autoantibodies 17

 SPECIAL ARTICLE www.jasn.org
deleted for invalidity criteria (two pro-
spective trials, one observational trial,
one nonrandomized trial, and three
nonoriginal articles). The main bodies
of the remaining 118 papers were scored
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW
for the CONSORT criteria as not report-
ed, insufficiently reported, or sufficiently
reported as listed in Supplemental Table
1.2 The title identified studies as RCTs in
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
only 23.7%. Abstracts provided a struc-
tured summary of the trial design in
51.7% (Supplemental Table 2). The in-
troduction named the objectives and a
clear hypothesis in 82.2%. The method
sections specify the precise trial design,
including allocation ratio, in only 17.8%
and any changes to trial design and the
respective reasons after trial commence-
ment in #1.7%. A precise description of
how and when the intervention was ad-
ministered was reported in 66.9% of the
studies. A precise definition of the pre-
specified primary and secondary out-
comes was reported in #60.2% of the
trials. Information about sample size
calculation was lacking in 52.5% of tri-
als. The modes of randomization, such
as sequence generation, the mechanisms
of allocation concealment, and imple-
mentation, were reported only in
35.6%, 19.5%, and 14.4%, respectively.
Modes of blinding were reported in
,15% of trials, whereas the statistical
methods used for group comparisons
on primary and secondary outcomes
were reported in 90.7%. However, only
77.1% of studies report the numbers of
participants included in each analysis.
An intention-to-treat analysis regarding
the primary end point was performed in
only less than one half (45.8%) of the
studies. Adverse events of the interven-
tion were reported in only 55.9.%.
Analyzing trends over time revealed
linear improvements from 1996 to
2016 in reporting trial nature in the title,
structuring abstracts, reporting trial de-
sign, prespecifying primary and second-
ary outcomes, sample size calculations,
analysis adjustments, presenting flow di-
agrams, defining periods of recruitment
and follow-up, baseline data, providing
denominators (number of participants)
for each analysis, and side effects (Figure
3A, Supplemental Table 2). Reverse
18 Journal of the American Society of Nephrology
trends were found for describing pre-
cisely how and when the interventions
were performed.
Among the leading journals reporting
kidney-related clinical trials, a great di-
versity of matching CONSORT quality
criteria was found. Only The Lancet,
NEJM, and AJKD provided structured
abstracts, whereas Kidney International
and JASN did not (Figure 3B, Supple-
mental Table 2). Trial design was insuffi-
ciently described in all aforementioned
journals. In JASN, proper descriptions of
sample size calculations and the randomi-
zation procedures were less frequent than
in the other journals. Clinical trials reported
in JASN also more frequently lacked an in-
tention-to-treat analysis compared with
those in the other journals. Unlike
NEJM, the other journals insufficiently re-
ported the statistical methods. Finally, we
checked the frequency of kidney trial reg-
istration before enrolment of the first
participant, a request by the International
Committee of Medical Journal Editors
since 2005.5 Of 100 randomly picked hu-
man kidney RCTs from PubMed (years
2008–2018), 93 were registered in a public
database, one was an observational trial
that did not need registration, and the
other six had been concluded before the
recommendation had been published.
Quality of Preclinical Trial Reporting
For quality assessment of preclinical
studies, we selected 209 publications
from JASN, Kidney International, and
NDT. Seventy-four had to be deleted
for invalidity criteria (Supplemental
Figure 3). The remaining 135 papers
were graded for the ARRIVE criteria as
specified in Methods and Supplemental
Table 3.3 Objectives or a hypothesis were
specified in the introduction in 73%
(Supplemental Table 4). The methods
specify a detailed ethical statement in
81% and the number or type of experi-
mental groups in 65%. However, details
on randomization, details on blinding,
and a timeline diagram were provided
in only 17%, 1%, and 12%, respectively.
Experimental procedures and type of an-
imals were generally well described (98%
and 84%, respectively), although infor-
mation on housing and husbandry were
largely lacking (34%). Information on
sample size calculation was always absent
(0%), and also, numbers of independent
replications were rare (19%). Only a mi-
nority of studies (13%) specified the pri-
mary and secondary outcomes. Regarding
statistical methods, the types of tests were
almost always reported (95%), but the
unit of analysis for each dataset and
whether the data met the prespecified as-
sumptions of the statistical approach were
not reported (0%). Baseline data were re-
ported in only 12% of the studies, and the
numbers analyzed for each test were report-
ed in only 33%. Adverse events were hardly
ever reported (2%).
Analyzing trends over time revealed
improvements from 1996 to 2016 in re-
porting ethical statements, blinding, and
timeline diagrams (Figure 4A, Supple-
mental Table 4). Reverse trends were
found for describing the primary and sec-
ondary objectives or a research hypothe-
sis and defining primary or secondary
outcomes. Completely neglected across
the entire study period was reporting de-
tails on group size calculations and
whether the data obtained met the as-
sumptions of the statistical approach.
Instead, rationale, experimental proce-
dures, statistical methods, and outcomes
were generally well reported.
Among the evaluated journals (JASN,
Kidney International, and NDT), report-
ing of preclinical studies according to the
ARRIVE quality criteria revealed some but
no profound differences (Figure 4B, Sup-
plemental Table 4). JASN scored higher on
ethics statement reporting. NDT reported
group allocation methods better. All jour-
nals rarely reported details on group size
calculations, whether the data obtained
met the assumptions of the statistical ap-
proach, blinding, and adverse events.
DISCUSSION
We had hypothesized that the numbers
of kidney-related clinical trials would
have increased within the last 15 years,
maybe also compared with other medi-
cal disciplines. We had further specu-
lated that reporting quality improved
and that both assumptions also apply
J Am Soc Nephrol 30: 13–22, 2019
 www.jasn.org SPECIAL ARTICLE

A B
Title Title
Abstract Abstract
Background a Background a
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Background b Background b
Trial Design a Trial Design a
Trial Design b Trial Design b
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023

Participants a Participants a
Participants b Participants b
Interventions Interventions
Outcomes a Outcomes a
Outcomes b Outcomes b
Sample Size a Sample Size a
Sample Size b Sample Size b
Randomization a1 Randomization a1 JASN
Randomization a2 Randomization a2 NEJM
Randomization b 1996 Randomization b Lancet
Randomization c 2006 Randomization c Kidney Int.
2016
Blinding a Blinding a AJKD
Blinding b Blinding b
Blinding c Blinding c
Statistical Methods a Statistical Methods a
Statistical Methods b Statistical Methods b
Participant flow a Participant flow a
Participant flow b Participant flow b
Participant flow c Participant flow c
Recruitment a Recruitment a
Recruitment b Recruitment b
Baseline Data Baseline Data
Numbers Analyzed Numbers Analyzed
Intention-to-treat Intention-to-treat
Loss - to - Analysis Loss - to - Analysis
Outcomes & Estimation a Outcomes & Estimation a
Outcomes & Estimation b Outcomes & Estimation b
Ancillary Analyses Ancillary Analyses
Harms Harms

0 50 100% 0 50 100%

Figure 3. Quality assessment of reporting clinical trials in the main final paper according to the Consolidated Standards of Reporting
Trials (CONSORT) criteria reveals improvements with time. Each of the CONSORT criteria was assessed as nonreported, unclear/in-
sufficiently reported, or sufficiently reported in representative samples selected from The New England Journal of Medicine (NEJM), The
Lancet (Lancet), Journal of the American Society of Nephrology (JASN), American Journal of Kidney Disease (AJKD), and Kidney In-
ternational (Kidney Int.) of the years 1996, 2006, and 2016. Shown are the percentages of papers fulfilling the criterion “sufficiently re-
ported” for (A) all journals in each of the 3 years to detect changes over time or (B) each of the journals across all time points. The selected
number of papers was too small to also analyze trends over time for each journal.

J Am Soc Nephrol 30: 13–22, 2019 Anti-Peroxidasin Autoantibodies 19

 SPECIAL ARTICLE www.jasn.org

A B
Background a Background a

Background b Background b
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Objectives Objectives

Ethical statement Ethical statement

nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023

Study design a Study design a

Study design b 1996 Study design b JASN

2006 Kidney Int.
Study design c Study design c
2016 NDT
Study design d Study design d
Experimental Experimental
procedures procedures
Experimental animals Experimental animals

Housing and husbandry Housing and husbandry

Sample size a Sample size a

Sample size b Sample size b

Sample size c Sample size c

Allocating animals to Allocating animals to
exp. groups exp. groups
Experimental outcomes Experimental outcomes

Statistical methods a Statistical methods a

Statistical methods b Statistical methods b

Statistical methods c Statistical methods c

Baseline data Baseline data

Numbers analysed Numbers analysed

Outcomes and Outcomes and

estimation estimation

Adverse events Adverse events

0 50 100% 0 50 100%

Figure 4. Quality assessment of reporting preclinical studies in the main paper according to the Animal Research: Reporting In Vivo
Experiments (ARRIVE) criteria does not reveal improvements with time. Each of the ARRIVE criteria (except for title and abstract criteria)
was assessed as nonreported, unclear/insufficiently reported, or sufficiently reported in representative samples selected from Journal of
the American Society of Nephrology (JASN), Kidney International (Kidney Int.), and Nephrology Dialysis and Transplantation (NDT) of the
years 1996, 2006, and 2016. Shown are the percentages of papers fulfilling the criterion “sufficiently reported” for (A) all journals in each of
the 3 years to detect changes over time or (B) each of the journals across all time points. The selected number of papers was too small to
also analyze trends over time for each journal.

to preclinical studies, which had not clinical study numbers. Analysis of deficits persist across leading kidney
been assessed before. In contrast, ne- reporting quality of clinical trials has journals.
phrology takes last rank among the dis- improved over time, whereas for pre- In 2004, Strippoli et al.1 first docu-
ciplines regarding clinical trial and pre- clinical studies, numerous reporting mented the low quantity of kidney

20 Journal of the American Society of Nephrology J Am Soc Nephrol 30: 13–22, 2019

disease–related clinical trials among the
other medical disciplines, which raised
some disappointment in the field. To ex-
clude that this could have been a false neg-
ative result due to omitting trials on RRTs,
we introduced an “expanded nephrology”
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW
definition; however, this did not substan-
tially change the result. Among the kidney
trials, topic coverage remains biased to-
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
ward hypertension and RRTs, probably be-
cause these entities allow for shorter trials
for outcome evaluation. In contrast, trials
addressing the progression of CKD face
the problem of primary outcomes that suf-
ficiently predict ESRD. Another factor
may be industry activity. Disciplines with
strong publication output may benefit
from strong industrial involvement (e.g.,
funding support), which may be also a fac-
tor favoring publications in the areas of
hypertension and RRTs in contrast to
CKD prevention or GN.
The reporting quality of clinical trials
remains a concern. It is of note that our
analysis did not include supplementary
information or previously published
study protocols; hence, our results do
not necessarily question the quality of
trial design but mostly, question under-
reporting in the main body of the final
paper, which may be hampered by space
limitations. However, precise definitions
of prespecified end points and perform-
ing an intention-to-treat analysis are es-
sential to avoid erroneous conclusions,
and these important aspects should be
reported in main body of the final paper.
Indeed, most journals now routinely re-
quest the trial design to be registered be-
fore enrolment of the first participant
(http://www.icmje.org/about-icmje/
faqs/clinical-trials-registration/) and
provision of CONSORT criteria check-
lists on submission, which may help to
improve this aspect. Analyzing clinical
trial reporting over time revealed signif-
icant improvements for some but not all
criteria. Interestingly, reporting quality
of how and when the intervention was
performed showed even a reverse trend.
These data imply that defining standards
and assuring them during the review
process and at the level of the editorial
processing are both needed to further
improve trial reporting. This is supported
J Am Soc Nephrol 30: 13–22, 2019
by comparing reporting quality in differ-
ent journals. NEJM seems to request more
accuracy of statistical methods reporting
than other journals. However, none of the
journals scored high for all criteria. As one
limitation, seven of 125 studies (5.6%)
during our quality analysis did not match
the “interventional clinical trial” nature.
It is likely that there is a similar rate of
misclassified articles in the quantitative
analysis. However, this rate should be dis-
tributed equally among all the disciplines
or kidney disease entities, and therefore, it
should not affect the conclusions.
An identical quantitative analysis for
preclinical studies revealed similar find-
ings. Expanding the nephrology-related
MeSH term also did not result in a rele-
vant increase in study numbers. Ne-
phrology completely lacks the profound
increases in published preclinical studies
reported from other medical disciplines
in the given timeframe. The reasons for
this remain uncertain but may include
structural problems in fundraising or num-
ber of related institutions. The coverage of
research topics varied over time, mimicking
major trends in nephrology, such as the
implementation of kidney transplantation
in the 1960s, novel classifications for AKI
and CKD, and the evolving global epidemic
of type 2 diabetes.
The increasing awareness of the lack of
proper reporting and reproducibility of
preclinical studies6–10 fueled concerns
about the reliability of preclinical re-
search as a predictor of human outcomes
as a whole.11,12 To this aim, the ARRIVE
guidelines on reporting of preclinical
studies were published in 2010.3 Our
quality analysis revealed significant defi-
ciencies in adhering to these guidelines
before and after this date. In particular,
providing precise information on ani-
mal substrains and housing conditions,
naming the assumptions for group size
calculations, randomization, and defining
primary end points that are also relevant
for human disease in animal studies are
important deficits. 13 Over time, only
the reporting of ethical statements,
blinding, and timeline diagrams im-
proved, whereas many other criteria did
not. No major differences in strengths
and deficiencies between JASN, Kidney
www.jasn.org SPECIAL ARTICLE
International, and NDT were found. En-
forcing adherence to the ARRIVE stan-
dards and reporting results accordingly
may help to improve what has been called
the “reproducibility crisis” in preclinical
research.6,8,9,12 Assuring proper group
size calculations and a blinded analysis
and considering sex disparities in experi-
mental animals should be important in this
context.12,14,15 It would have been interest-
ing to also analyze other important
journals, such as Journal of Clinical Investi-
gation, Nature Medicine, Nature, Science,
JCI Insight, etc., but these multidisciplinary
journals publish kidney-related preclinical
studies less frequently and only in low
numbers within the set time periods.
In summary, clinical and preclinical
research papers in nephrology have re-
mained low in number compared with
those of other medical disciplines. The
quality of data reporting in the main body
of papers presenting clinical trials keeps
improving but is still suboptimal in many
ways. The quality of data reporting of
preclinical studies is still in its infancy
and may contribute to reproducibility
problems. Efforts at all levels are needed
to overcome these deficits in the future.
Given the central role of kidney disease–
related morbidity and mortality, as well
as health care costs, greater investments
in kidney research are needed.
ACKNOWLEDGMENTS
H.-J.A. was supported by Deutsche For-
schungsgemeinschaft grant AN372/24-1.
This project was prepared as a doctoral
thesis at the Faculty of Medicine, Ludwig
Maximilians University Munich by M.K.T.C.
DISCLOSURES
None.
This article contains the following supplemental
material online at http://jasn.asnjournals.org/
lookup/suppl/doi:10.1681/ASN.2018050515/-/
DCSupplemental.
SUPPLEMENTAL MATERIAL
Supplemental Figure 1. Quantitative analysis
of preclinical studies identified from PubMed.
Anti-Peroxidasin Autoantibodies 21
 SPECIAL ARTICLE www.jasn.org
Supplemental Figure 2. Flow chart illustrating
the identification and selection of clinical trial reports
papers for the quality analysis.
Supplemental Figure 3. Flow chart illustrat-
ing the identification and selection of preclinical
study reports papers for the quality analysis.
Downloaded from http://journals.lww.com/jasn by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW
Supplemental Material. Supplementary
methods.
Supplemental Table 1. Criteria for assessment
of CONSORT recommendations.
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
Supplemental Table 2. Quality assessment of
clinical trials according to the CONSORT criteria.
Supplemental Table 3. Assessment criteria for
ARRIVE recommendations.
Supplemental Table 4. Quality assessment of
preclinical studies according to the ARRIVE
criteria.
REFERENCES
1. Strippoli GF, Craig JC, Schena FP: The
number, quality, and coverage of randomized
controlled trials in nephrology. J Am Soc
Nephrol 15: 411–419, 2004
2. CONSORT. Available at: http://www.consort-
statement.org/. Accessed September 9, 2017
22 Journal of the American Society of Nephrology
3. Kilkenny C, Browne WJ, Cuthill IC, Emerson
M, Altman DG: Improving bioscience re-
search reporting: The ARRIVE guidelines
for reporting animal research. PLoS Biol 8:
e1000412, 2010
4. Deo A, Schmid CH, Earley A, Lau J, Uhlig K: Loss
to analysis in randomized controlled trials in
CKD. Am J Kidney Dis 58: 349–355, 2011
5. International Committee of Medical Journal
Editors: Clinical Trials. Available at: http://
www.icmje.org/recommendations/browse/
publishing-and-editorial-issues/clinical-
trial-registration.html. Accessed September
22, 2018
6. Braybrook J: Reproducibility: Developing
standard measures for biology. Nature 551:
168, 2017
7. Hsieh T, Vaickus MH, Remick DG: Enhanc-
ing scientific foundations to ensure repro-
ducibility: A new paradigm. Am J Pathol
188: 6–10, 2018
8. Vasilevsky NA, Brush MH, Paddock H, Ponting
L, Tripathy SJ, Larocca GM, et al.: On the re-
producibility of science: Unique identification
of research resources in the biomedical litera-
ture. PeerJ 1: e148, 2013
9. Begley CG, Ellis LM: Drug development:
Raise standards for preclinical cancer re-
search. Nature 483: 531–533, 2012
10. Kilkenny C, Parsons N, Kadyszewski E,
Festing MF, Cuthill IC, Fry D, et al.: Survey of
the quality of experimental design, statisti-
cal analysis and reporting of research using
animals. PLoS One 4: e7824, 2009
11. van der Worp HB, Howells DW, Sena ES,
Porritt MJ, Rewell S, O’Collins V, et al.: Can
animal models of disease reliably inform hu-
man studies? PLoS Med 7: e1000245, 2010
12. Anders HJ, Jayne DR, Rovin BH: Hurdles to
the introduction of new therapies for im-
mune-mediated kidney diseases. Nat Rev
Nephrol 12: 205–216, 2016
13. Hirst JA, Howick J, Aronson JK, Roberts N,
Perera R, Koshiaris C, et al.: The need for ran-
domization in animal trials: An overview of sys-
tematic reviews. PLoS One 9: e98856, 2014
14. Miller LR, Marks C, Becker JB, Hurn PD, Chen
WJ, Woodruff T, et al.: Considering sex as a
biological variable in preclinical research.
FASEB J 31: 29–34, 2017
15. Shah K, McCormack CE, Bradbury NA: Do
you know the sex of your cells? Am J Physiol
Cell Physiol 306: C3–C18, 2014
See related editorial, “The Quality of Reporting of
Kidney Research: A Challenge to JASN,” on pages
1–2.
J Am Soc Nephrol 30: 13–22, 2019