38 Scientific Basis for Ayurvedic Therapies

4. Description of product and procedure

5. Description of sample selected
6. Description of outcome measures
7. Data report
8. Attention to possible bias in experimental design
9. Bias present in design or operationalization of design
10. Comparison of dropouts and competitors

Of the factors listed, 1, 2, 3, 5, and 6 have two points possible due to the specific
subfactors listed (see Table 3.1 for more details). The safety score was derived by using
the SAS-CT system which queries six global issues with subfactors being scored to
produce a possible point score of 100. As the clinical trials reviewed here were not
controlled, the scores were modified for the safety assessment. Full scores were given if
the scoring category was satisfied for the single group receiving the therapy. Table 3.1
gives both summed scores for each data set reviewed within the monograph for both
scales and gives partials across factors for all data sets reviewed.
All authors and three research assistants acknowledged here participated in the system-
atic review process using both scales. Each article was primarily reviewed by two people
with a third person as an arbitrator for reconciliation purposes. This mechanism guaran-
teed that four to five people reviewed each article in order to produce the scoring offered
in the table.
The studies reviewed here with the Singh Q-E Quality Assessment Scale received full
points on the factors of the background and significance of study, sample description,
clarity of outcome measures utilized, and clarity of tables and data reported. Twelve of
thirteen data sets utilized outcome measures with established, known, or reported reli-
ability and validity. Ten of the thirteen articles were believed to be without operational-
ization flaws and presented the protocol in sufficient detail to allow for replication. Nine
of the data sets optimized treatment to patients and seven of the thirteen justified the
dosage of herbal treatments or therapies offered. Overall, this is quite a good showing.
These results should encourage others to read the literature that is originating in India
and other non-Western venues with the attitude that the data presented are going to be
consistent with the quality of trials available in the West.
The SAS-CT scores were a bit more mixed. This factor is important, because people in
the West are particularly interested in safety information on herbals and procedures that
are less well known to them. Of the 13 studies reviewed, the range of scores was from 0
to 100. Four studies got perfect scores for reporting safety issues and five got no points.
Other scores were 17, 21, 44.1, 47, and 88. It must be noted that an investigator should
address the issues around safety: the occurrence of adverse events, the severity of them
should they occur, whether they are likely related to the trial or not, and what the
dropout rate is and were these due to adverse events minimally. For example, it is
impossible to tell whether five articles did not report adverse events because none
occurred, or the information was merely not reported. A simple sentence saying (1) that
there were no adverse events and (2) there were no dropouts would go a long way to
assist those reading data to assess safety. A reader cannot assume that a product is safe
if there is nothing about adverse events mentioned; he or she can only wonder.