ArgMed-Agents: Explainable Clinical Decision Reasoning with Large Language
Models via Argumentation Schemes
Shengxin Hong1 Liang Xiao2 , Xin Zhang1 and Jianxia Chen2
1
Detroit Green Technology Institute, Hubei University of Technology, China
2
School of Computer Science, Hubei University of Technology, China
seikin.shengxinhong@gmail.com, lx@mail.hbut.edu.cn
arXiv:2403.06294v1 [cs.AI] 10 Mar 2024
Abstract
There are two main barriers to using large language
models (LLMs) in clinical reasoning. Firstly, while
LLMs exhibit significant promise in Natural Lan-
guage Processing (NLP) tasks, their performance
in complex reasoning and planning falls short of
expectations. Secondly, LLMs use uninterpretable
methods to make clinical decisions that are fun-
damentally different from the clinician’s cognitive
processes. This leads to user distrust. In this
paper, we present a multi-agent framework called
ArgMed-Agents, which aims to enable LLM-based
agents to make explainable clinical decision rea-
soning through interaction. ArgMed-Agents per-
forms self-argumentation iterations via Argumen-
tation Scheme for Clinical Decision (a reasoning
mechanism for modeling cognitive processes in
clinical reasoning), and then constructs the argu-
mentation process as a directed graph representing
conflicting relationships. Ultimately, Reasoner(a
symbolic solver) identify a series of rational and
coherent arguments to support decision. ArgMed-
Agents enables LLMs to mimic the process of clin-
ical argumentative reasoning by generating expla-
nations of reasoning in a self-directed manner. The
setup experiments show that ArgMed-Agents not
only improves accuracy in complex clinical deci-
sion reasoning problems compared to other prompt
methods, but more importantly, it provides users
with decision explanations that increase their con-
fidence.
1 Introduction
Large Language Models (LLMs) [OpenAI, 2023] have re-
ceived a lot of attention for their human-like performance in
a variety of domains. In the medical field especially, prelim-
inary studies have shown that LLMs can be used as clinical
assistants for tasks such as writing clinical texts [Nayak et
al., 2023], providing biomedical knowledge [Singhal et al.,
2022] and drafting responses to patients’ questions [Ayers et
al., 2023]. However, the following barriers to building an
LLM-based clinical decision-making system still exist: (i)
LLMs still struggle to provide secure, stable answers when
faced with highly complex clinical reasoning tasks [Pal et
al., 2023]. (ii) There is a perception that LLMs use unex-
plainable methods to arrive at clinical decisions (known as
black boxes), which may have led to user distrust [Eigner and
Händler, 2024].
To address these barriers, exploring the capabilities of
LLMs in argumentative reasoning is a promising direc-
tion. Argumentation is a means of conveying a compelling
point of view that can increase user acceptance of a posi-
tion. Its considered a fundamental requirement for build-
ing Human-Centric AI [Dietz et al., 2022]. As compu-
tational argumentation has become a growing area of re-
search in Natural Language Processing (NLP) [Dietz et al.,
2021], researchers have begun to apply argumentation to
a wide range of clinical reasoning applications, including
analysis of clinical discussions [Qassas et al., 2015], clini-
cal decision making [Hong et al., 2023; Zeng et al., 2020;
Sassoon et al., 2021], address clinical conflicting [Čyras et
al., 2018]. In the recent past, some work has assessed the abil-
ity of LLMs in argumentation reasoning [Chen et al., 2023;
Castagna et al., 2024] or non-monotonic reasoning [Xiu et
al., 2022]. Although LLMs show some potential for compu-
tational argumentation, more results show that LLMs perform
poorly in logical reasoning tasks [Xie et al., 2024], and bet-
ter ways to utilise LLMs for non-monotonic reasoning tasks
need to be explored.
Meanwhile, LLM as agent studies have been surpris-
ingly successful [Zhang et al., 2023; Wang et al., 2023;
Shi et al., 2024]. These methods use LLMs as computational
engines for autonomous agents, and optimise the reasoning,
planning capabilities of LLMs through external tools (e.g.
symbolic solvers, APIs, retrieval tools, etc.) [Pan et al., 2023;
Shi et al., 2024], multi-agent interactions [Tang et al., 2024]
and novel algorithmic frameworks [Gandhi et al., 2023].
Through this design, LLMs agents can interact with the envi-
ronment and generate action plans through intermediate rea-
soning steps that can be executed sequentially to obtain an
effective solution.
Motivated by these concepts, we present ArgMed-Agents,
a multi-agent framework designed for explainable clinical de-
cision reasoning. We formalised the cognitive process of clin-
ical reasoning using an argumentation scheme for clinical de-
cision (ASCD) as a prompt strategy for interactive reasoning
by LLM agents. There are three types of agents in ArgMed-
Agents: the Generator, the Verifier, and the Reasoner. the
Generator generates arguments to support clinical decisions
based on the argumentation scheme; the Verifier checks the
arguments for legitimacy based on the critical question, and
if not legitimate, it asks the Generator to generate attack ar-
guments; Reasoner is a symbolic solver that identifies reason-
able, non-contradictory arguments in the resulting directed ar-
gumentation graph as decision support.
In our method, we do not expect every proposed argument
or detection of Generator or Verifier to be correct, instead we
consider their generation as a assumption. The LLM agents
are induced to recursively iterate in a self-argumentative man-
ner through the prompt strategy , while the newly proposed
assumptions always contradict the old ones, and eventually
the Reasoner eliminates unreasonable assumptions and iden-
tifies coherent arguments, leading to consistent reasoning re-
sults. ArgMed-Agents enables LLMs to explain its own out-
puts in terms of self-cognitive profiling by modelling their
own generation as a prompt for question recursion. The gen-
erative process derives the attack relations of the arguments
are determined based on the ASCD prompt strategy. For ex-
ample, when there exist two decisions as arguments a and b,
they attack each other based on the decision exclusivity de-
fined in ASCD.
Our experiment was divided into two parts, evaluating ac-
curacy and explainability of ArgMed-Agents clinical reason-
ing, respectively. First, we conducted experiments on two
datasets, including MedQA [Jin et al., 2020] and PubMedQA
[Jin et al., 2019]. To better align with real-world applica-
tion scenarios, our study focused on a zero-shot setting. Sec-
ond, we used predictability and traceability as measures of
explainability by manually assessing whether the clinical rea-
soning process of ArgMed-Agents is meaningful in terms of
explanation for users. The results show that ArgMed-Agents
achieves better performance in both accuracy and explain-
ability compared to direct generation and Chain of Thought
(CoT).
2 Preliminaries
2.1 Abstract Argumentation Framework
Abstract Argumentation (AA) frameworks [Dung, 1995] are
pair ⟨A, R⟩ composed of two components: a set of abstract
arguments A, and a binary attack relation R. Given an AA
framework ⟨A, R⟩, a, b ∈ Aand(a, b) ∈ R indicates that a
attacks b. On top of that, Dung defines some notions:
• ∃a ∈ A is acceptable if f ∄b ∈ A such that (b, a) ∈ R
or ∃b, c ∈ A and (b, a) ∈ R such that (c, b) ∈ R.
• A set of arguments is conflict-free if there is no attack
between its arguments.
In order to decide whether a single parameter can be ac-
cepted or whether multiple parameters can be accepted at the
same time, the argumentation system allows for the use of
various semantics to compute the set of parameters (called
extensions). For example, Given an AA framework ⟨A, R⟩,
a, b ∈ Aand(a, b) ∈ R, E ⊆ A is a prefered extension only
if it is a maximal element (with respect to the set-theoretical
inclusion) among the admissible sets(a set of all arguments
are acceptable and conflict-free) with respect to A.
2.2 Argumentation Scheme
The concept of Argumentation Scheme (AS) originated
within the domain of informal logic, stemming from the semi-
nal works of [Walton et al., 2008; Walton, 1996]. Argumenta-
tion Scheme serves as a semi-formalized framework for cap-
turing and analyzing human reasoning patterns. Formally de-
fined as AS = ⟨P, c, V ⟩, it comprises a collection of premises
(P ), a conclusion (c) substantiated by these premises, and
variables (V ) inherent within the premises (P.V ) or the con-
clusion (c.V ). A pivotal aspect of the argumentation scheme
is the delineation of Critical Questions (CQs) pertinent to
AS. Failure to address them prompts a challenge to both the
premises and the conclusion posited by the scheme. Conse-
quently, the role of CQs is to instigate argument generation;
when an AS is contested, it engenders the formulation of a
counter-argument in response to the initial AS. This iterative
process culminates in the construction of an attack argument
graph, facilitating a nuanced understanding of argumentative
dynamics. Figure 1 illustrates three templates of argumenta-
tion schemes.
3 Argumentation Scheme for Clinial Decision
In the past, numerous studies [Oliveira et al., 2018; Sassoon
et al., 2021; Qassas et al., 2015] have explored the application
of argumentation in the clinical domain. In this section, we
provide a summary of these endeavors, focusing on the de-
velopment of Argumentation Schemes for Clinical Decision
(ASCD). ASCD encapsulate various argumentation scheme
tailored for clinical decision-making processes and the rea-
soning process. Additionally, we propose to refine and adapt
these schemes to enhance their suitability for LLM. ASCD
consists of three argumentation schemes which are Argumen-
tation Scheme for Decision (ASD), Argumentation Scheme
for Side Effect (ASSE) and Argumentation Scheme for Bet-
ter Decision (ASBD). Figure 1 includes the components of
the three argumentation schemes and the derivations between
them.
ASCD formalizes the decision-making process for clini-
cians. Clinical decision reasoning begins with ASD, where
decisions are related to clinical goals. Formally, in Figure
1 we model the derivation of CQs. When the CQs are re-
jected, new ASs are generated as arguments to refute the first
proposed AS based on the derivation relation. It is worth not-
ing that when a CQ without a derivation relationship (e.g.,
ASD.CQ1, ASSE.CQ1) is rejected, the AS will be refuted by
itself (on the grounds of the CQ’s answer). We illustrate how
to reason using ASCD with an example:
• Joey is a 50 year old male patient who has suffered a
stroke and has high blood pressure. For Joey, the goal is
to control his blood pressure;
First, we represent the initial ASD reasoning result as
ASD1 (joey, control blood pressure, ACEI) → ACEI
meaning that ACEI drugs should be considered as treatment
for joey. Given the known patient facts and clinical guide-
lines, both ASD1 .CQ1 and ASD1 .CQ2 were accepted,
 knowledge to build knowledge bases and rule. In this sec-
Argumentation Scheme for Decision
ASD (F, D, G)
tion, we propose a multi-agent framework called ArgMed-
Premise - Given the patient Fact F Agents, which supports the seamless integration of prompt
Premise - To achieve Goal G
Premise - Decision D promotes the goal G strategy designed based on ASCD into agent interactions.
Conclusion: Decision D should be considered Our approach enhances LLMs to be able to perform explain-
CQ1: Is there any evidence to support Decision able clinical decision reasoning without the need for expert
D promote goal G? involvement in knowledge encoding.
derive
CQ2: Has Decision D caused side effects on the
patient? 4.1 ASCD-based Multi-agent Interaction
CQ3: Can Goal G be achieved by completing the
Decision D?
ArgMed-Agents framework includes three distinct types of
derive
CQ4: Are there alternative Decision to achieve LLMs agent:
derive
the same goal G? • Generator(s): Instantiate the AS according to the current
situation.
Argumentation Scheme for Side Effect Argumentation Scheme for Better Decision
ASSE (SE, D, F) ASBD (F, G, D1, D2)
• Verifier: Whenever Generator generates an instantiation
Premise - Given the patient Fact F Premise - Given the Decisions D1 and D2 of an AS, Verifier checks the accuracy of the instantia-
Premise - Decision D results in side effect SE Premise - Given the patient Fact F, for Goal G,
Conclusion: Decision D needs to be used with caution
Decision D2 is better than D1 tion of that AS via CQ. When the CQ validation is re-
Conclusion: Decision D2 should be considered jected, return to the generator the reason why the CQ
CQ1: Is there any evidence to support Decision D
results in side effect SE?
CQ1: Is there any evidence to support Decision D2 rejected, so that the generator generates a new AS based
better than Decision D1 for Goal G?
on the derivation rules of that CQ. The process iterates
CQ2: Is this side effect SE unacceptable?
until no more new AS are generated.
CQ3: Are there ways of ameliorating the side effects
observed? • Reasoner: A symbolic solver for computational argu-
mentation. At the end of the iteration, the arguments
Add New presented by the generator constitute a complete argu-
Goal to G mentation framework, and it identifies a subset of coher-
ent and reasonable arguments within that argumentation
framework.
Figure 1: The derivation rules between the argumentation schemes We convert ASCD into a step-by-step reasoning interac-
in ASCD. tion protocol as a prompt strategy and define the interac-
tion behavior between LLM agents based on ASCD prompt
strategy, which describes the specification of the interac-
and ASD1 .CQ4 was found to have substituted decision- tion behavior between different types of agents under the
making thta ARB can also control blood pressure. Ac- ASCD reasoning mechanism. Figure 2 depicts how ArgMed-
cording to the derivation rules of ASD1 .CQ4, first gen- Agents interacts. In this example, GeneratorA receives
erate the ASD2 (joey, control blood pressure, ARB) → the question at first, which uses ASD to generate an initial
ARB. Since decisions are exclusive, ASD1 and ASD2 decision Paroxetine for the patient. Then Verifier detects
are mutually attacks. Based on known clinical evidence, that the argument presented by GeneratorA is rejected by
ACEI have shown more significant effects in reducing the ASD.CQ2 and ASD.CQ4, which results in argument from
risk of cardiovascular events (e.g., heart attack, stroke). GeneratorB , GeneratorC and GeneratorD . The output
Therefore, another derivation of ASD1 .CQ4 is denoted as from GeneratorB indicates that Paroxetine exists side ef-
ASBD(joey, control blood pressure, ACEI, ARB) → fect derived from ASD.CQ2. On the other hand, accord-
ACEI. From this, we get a arguments graph in which ASD1 ing to the derivation rules of ASD.CQ4, firstly, GeneratorC
and ASD3 attack each other and ASBD attacks ASD2 . No- is generated to suggest that there exists an alternative deci-
tably, since we set exclusivity between ASD, in order for sion Trazodone; secondly, GeneratorD finds that Trazodone
ASD.CQ1 and ASD.CQ3 to be valid, when these two CQs is more suitable for the patient than Paroxetine by compar-
are rejected, that ASD first refutes itself (forming a closed ing Paroxetine and Trazodone. Moreover, the Verifier of
loop) and then generates a new ASD. GeneratorB suggests that GeneratorB ’s argument is re-
jected by ASSE.CQ3, but this CQ does not have a derivation
4 ArgMed-Agents: a Multi-LLM-Agents rule, so the argument representing GeneratorB would attack
Framework itself. The Verifier facilitates an iterative process of mutual
debate between Generators, which forms an argumentation
In Section 3, we discussed Argumentation Schemes for clini- framework that can be solved by the symbolic Reasonner. We
cal decision(ASCD). ASCD is a semi-formal reasoning tem- provide a formal description of the interaction process in Ap-
plate for clinical decision making by defining the logical pendix A.
structure and the reasoning mechanism in the clinical rea- In ArgMed-Agents, we do not expect single generation or
soning process. However, the clinical decision support sys- single verification to be correct. ArgMed-Agents uses gener-
tems based on argumentation scheme in the past [Sassoon et ation as a assumption to recursively prompt the model with
al., 2021; Qassas et al., 2015] have often been implemented critical questions, identifying conflicting and erroneous ar-
through knowledge-based approaches, which rely on expert guments in an iterative process. Ultimately, such mutually
 Question
Premise - Given the patient Fact F [A 40-year-old woman with symptoms of
A 40-year-old woman presents with difficulty falling asleep, diminished depression including difficulty falling asleep, diminished appetite, Premise - Given the Decisions Trazodone and
appetite, and tiredness for the past 6 weeks. She says that, despite going tiredness, irritability, feelings of hopelessness, diminished Paroxetine
to bed early at night, she is unable to fall asleep. She denies feeling concentration and interest, and weight loss.]
anxious or having disturbing thoughts while in bed. Even when she manages to Premise -
Generator
fall asleep, she wakes up early in the morning and is unable to fall back Premise - To achieve Goal G [G: Treat the patient's condition] Effectiveness in Treating Depression: Generator
A
asleep. She says she has grown increasingly irritable and feels increasingly Both Trazodone and Paroxetine have evidence D
hopeless, and her concentration and interest at work have diminished. The Premise - Paroxetine promotes the goal G [ Paroxetine is a selective supporting their effectiveness in treating
patient denies thoughts of suicide or death. Because of her diminished serotonin reuptake inhibitor (SSRI) antidepressant known to effectively depression.
appetite, she has lost 4 kg (8.8 lb) in the last few weeks and has started treat depression by increasing serotonin levels in the brain.]
drinking a glass of wine every night instead of eating dinner. She has no Side Effect Profile:
significant past medical history and is not on any medications. Which of the Trazodone may be more sedating but generally has
following is the best course of treatment in this patient? Conclusion: Paroxetine should be considered fewer sexual side effects compared to Paroxetine.
However, both medications have their own set of
Options: (a) Diazepam (b) Paroxetine (c) Zolpidem (d) Trazodone potential adverse effects.
CQ1: Is there any evidence to support Decision D (Paroxetine) promote
goal G? Symptom Relief:
YES Both medications can potentially address the
Argumentation CQ2:Has Decision D (Paroxetine) caused side effects on the patient?
patient's symptoms, but Trazodone may be more
specifically targeted towards sleep disturbances.
Framework YES Verifier
CQ3:Can Goal G be achieved by completing the Decision D (Paroxetine)? Conclusion: Trazodone should be considered
YES
A CQ4: Are there alternative Decision of (Paroxetine) to achieve the
C same goal G?
YES
CQ1: Is there any evidence to support Decision D2
(Trazodone) better than Decision D1 (Paroxetine)
for Goal G?
YES
Premise - The patient is experiencing symptoms of depression, including Verifier
B difficulty falling asleep, diminished appetite, tiredness, irritability,
hopelessness, diminished concentration, and loss of interest in work.

D Generator
She has also lost weight due to decreased appetite and has started using
alcohol as a coping mechanism.
CQ1: Is there any evidence to support Decision D
B
(Paroxetine) results in side effect SE?
Premise - The decision to prescribe Paroxetine, a selective serotonin
YES
reuptake inhibitor (SSRI) commonly used to treat depression and anxiety
disorders, is considered as a potential course of treatment. Verifier
CQ2:Is this side effect SE not acceptable?
NO
Premise - Paroxetine, like many medications, can have side effects.
Common side effects of Paroxetine include nausea, drowsiness, dizziness,
CQ3: Are there ways of ameliorating the side effects
dry mouth, sweating, blurred vision, constipation, or headache.
observed?
Additionally, more serious side effects such as serotonin syndrome,
NO
suicidal thoughts or behavior, and withdrawal symptoms can occur,
although they are less common.

Symbolic Conclusion: Considering the patient's symptoms and the potential side
effects of Paroxetine, the decision to prescribe Paroxetine needs to be
CQ1: Is there any evidence to support Decision D
(Trazodone) promote goal G?

Solver made with caution. YES

CQ2:Has Decision D (Trazodone) caused side effects on the Verifier

patient?
Premise - Given the patient Fact F [A 40-year-old woman with symptoms YES
Reasoner of depression including difficulty falling asleep, diminished
appetite, tiredness, irritability, feelings of hopelessness, CQ3:Can Goal G be achieved by completing the Decision D
diminished concentration and interest, and weight loss.] (Trazodone)?
Generator YES
C Premise - To achieve Goal G [G: Treat the patient's condition]
CQ4: Are there alternative Decision of (Paroxetine,
Premise - Trazodone promotes the goal G [Trazodone is primarily Trazodone) to achieve the same goal G?
Decision prescribed as an antidepressant due to its serotonin modulating YES
properties and its sedative effects, which can help with sleep
disturbances associated with depression.] //The iteration will continues because this CQ2 and CQ4 are
Trazodone should be considered as treatment rejected but is not shown here due to space constraints

Conclusion: Trazodone should be considered

Figure 2: An example from the MedQA USMLE dataset, with the entire process of ArgMed-Agents reasoning about the clinical problem.
Notably, the letters in the argumentation framework correspond to the serial numbers of the four generators on the right, representing the
premises and conclusion generated by that generator. In the argumentation framework, the red nodes (A and C) represent arguments in
support of the decision and the yellow nodes (B and D) represent arguments in support of the beliefs.

attacking arguments are converted into a formal framework ized process consistent with clinical reasoning (i.e., following
that solves for a subset of reasonably coherent arguments via the ASCD prompt strategy). This design not only facilitates
Reasoner. LLMs to engage in clinical reasoning in a critical-thinking
manner, which in turn maintains a cognitive processes consis-
The theoretical motivation for our method stems from tent with clinicians to improve the explainability of decisions,
non-monotonic logic, logical intuition and cognitive clinical. but also allows for the effective highlighting of some implicit
Studies have shown that LLM performs reasonably well for knowledge in LLMs that cannot be readily accessed through
simple reasoning, single-step reasoning problems, however, traditional prompts.
as the number of reasoning steps rises, the rate of correct
reasoning decreases in a catastrophic manner [Creswell et
al., 2022]. Thus, we consider that LLM is primed with ra- 4.2 Explainable Argumentative Reasoning
tional intuition, but lacks true logical reasoning ability. In ArgMed-Agent obtains a complete argument graph by iter-
light of this, ASCD Prompt Stratygy guides LLM in a re- ating through the ASCD Prompt Strategy. In this section,
cursive form to generate a series of casual reasoning steps as we describe how Reasoner reasons and explains decisions
a tentative conclusion, with any further evidence withdraw- through these arguments. First, we map the reasoning results
ing their conclusion. The process ultimately leads to a di- of ASCD to the AA framework.
rected graph representing the disputed relationships. In addi-
tion this, agents with different LLMs roles within ArgMed- Definition 1. An AA framework for ASCD is a pair ⟨A, R⟩
Agents interact collaboratively with each other in a formal- such that:
 • ∃arg = ⟨P, c, V ⟩ ∈ A that arguments set A is consti-
tuted by the argumentation schemes.
• A = Argsd (A) ∪ Argsb (A) such that arguments in ar-
gument set A are distinguished between two types of ar-
guments: arguments in support of decisions Argsd (A)
and arguments in support of beliefs Argsb (A).
In Definition 1, arguments in support of decisions
Argsd (A) correspond to ASD in the argumentation scheme,
and arguments in support of beliefs Argsb (A) correspond to
ASSE and ASBD in the argumentation scheme. These two
types of arguments play different roles; arguments in sup-
port of decisions build on beliefs and goals and try to jus-
tify choices, while arguments in support of beliefs always try
to undermine decision arguments. Therefore, based on the
modeling of ASCD, we make the following setup for the at-
tack relation between these two types of arguments: first, ar-
guments in support of different decisions are in conflict with
each other, which is consistent with our previous description
of decisions being exclusive in ASCD. Second, arguments
in support of decisions are not allowed to attack arguments
in support of beliefs, which is dictated by the modeling of
ASCD, as can be seen in Figure 1, where ASSE and ASBD
are always trying to disprove ASD, while the converse does
not work. Formalized as:
Definition 2. Given an AA framework for ASCD ⟨A, R⟩,
R ⊆ Argsd (A) × Argsb (A) such that:
• ∀arg1 , arg2 ∈ Argsd (A) such that arg1 ̸= arg2 ,
(arg1 , arg2 ) ∈ R.
• ∄(arg1 , arg2 ) ∈ R such that arg1 ∈ Argsd (A) and
arg2 ∈ Argsb (A).
Next, we use the preferred semantics in the AA framework
to illustrate how the Reasoner Agent selects the set of possible
admissible arguments to support clinical decision.
Definition 3. Given an AA framework for ASCD ⟨A, R⟩, E ⊆
E is a set of preferred extension such that Argsb (E) is the
support decision for E.
Example 1. An AA framework for ASCD is given below such
that Argsd (A) = {A, B, C} and Argsb (A) = {D, E}. In
this case, decisions B and C are optional because they are
supported by preferred extension.
D E
Admissible Sets: {B, D, E},
{C, D, E},{B, D},{B, E},
A {C, D},{C, E},{D, E},{B},
{C},{D},{E},{}
Prefered Extension:{B, D, E},
B C {C, D, E}
In this regard, we present a proposition to justify the Rea-
soner agent’s use of preferred semantics for decision making.
Proposition 1. Given an AA framework for ASCD ⟨A, R⟩
and E ⊆ E is a set in preferred extension E where,
• ∃a ∈ Argsd (A) is acceptable such that Argsd (E) ̸= ∅
• ∃a ∈ Argsd (E) such that ∀b ∈ Argsd (A), b ̸= a that
b∈
/ Argsd (E).
Proposition 1 states that using preferred semantics on the
AA framework for ASCD always selects extensions that con-
tain the only acceptable decision, which is consistent with our
expected reasoning. Meanwhile, arguments from Argsb (E)
can be regarded as explanation of decision Argsd (E) [Fan
and Toni, 2015]. The proof of the proposition proceeds as
follows: We first prove that there will be only one decision in
the preferred extension. According to Definition 2, we know
that decisions are exclusive, so different decisions do not ap-
pear in the same admissible set. Secondly, we prove that the
preferred extension will always contain a argument in sup-
port of decision. According to the modeling of ASCD as well
as Definition 2, it can be seen that arguments in support of
decision (ASD) are built on top of arguments in support of
beliefs (ASSE and ASBD), i.e., the inclusion of ASD argu-
ments does not lead to the ASSE and ASBD arguments be-
coming unacceptable. Therefore, the set is maximized when
the acceptable set includes pro-decision arguments.
Definition 4. Given an AA framework for ASCD ⟨A, R⟩ and
E is its prefered extension. There are errors in the clinical
reasoning of ArgMed-Agents in this case iff Argsd (E) = ∅.
Clinical reasoning errors are discussed in [Tang et al.,
2024]. They consider that most clinical reasoning errors in
LLMs are due to confusion about domain knowledge. On
the other hand, research by [Singhal et al., 2022] points out
that LLMs may produce compelling misinformation about
medical treatment, so it is crucial to recognize this illusion,
which is difficult for humans to detect. For this purpose, we
define the relevant mechanisms for identifying clinical rea-
soning errors in Reasoner as follows: When any decision in
the AA framework is not accepted, we consider there are er-
rors in the reasoning by ArgMed-Agents. This mechanism
assists ArgMed-Agents in identifying the capability bound-
aries of LLMs when their knowledge reserves are insufficient
to address certain issues.This helps ArgMed-Agents avoid
the risks associated with adopting erroneous decisions and
achieving more robust and safe clinical reasoning.
5 Experiments
In this section, we demonstrate the potential of ArgMed-
Agents in clinical decision reasoning by evaluating the ac-
curacy and explainability.
We implemented Generator and Verifier in ArgMed-
Agents using the APIs GPT-3.5-turbo and GPT-4 provided
by OpenAI [OpenAI, 2023], and according to our setup, the
LLM Agents are implemented with the same LLM and dif-
ferent few-shot prompts. Each agent is configured with spe-
cific parameters: the temperature is set to 0.0, as well as a
dialogue limit of 4, indicating the maximum number of de-
cisions allowed for ArgMed-Agents to generate, which is to
prevent the agents from getting into loops with each other. On
the other hand, we using python3 to implemente a symbolic
solver of abstract argumentation framework as Reasoner.
 Model Method MedQA PubMedQA
Direct 52.7 68.4
GPT-3.5-turbo CoT 48.0 71.5
ArgMed-Agents 62.1 78.3
Direct 67.8 72.9
GPT-4 CoT 71.4 77.2
ArgMed-Agents 83.3 81.6
Table 1: Results of the accuracy of various clinical decision reason-
ing methods on MedQA and PubMedQA datasets.
5.1 Accuracy
The following two datasets were used to assess the accuracy
of ArgMed-Agents clinical reasoning:
• MedQA [Jin et al., 2020]:Answering multiple-choice
questions derived from the United States Medical Li-
cense Exams (USMLE). This dataset is sourced from of-
ficial medical board exams and encompasses questions
in English, simplified Chinese, and traditional Chinese.
The total question counts for each language are 12,723,
34,251, and 14,123, respectively.
• PubMedQA [Jin et al., 2019]: A biomedical ques-
tion and answer (QA) dataset collected from PubMed
abstracts. The task of PubMedQA is to answer
yes/no/maybe research questions using the correspond-
ing abstracts (e.g., Do preoperative statins reduce atrial
fibrillation after coronary artery bypass graft surgery?).
We randomly selected 300 examples in each dataset for our
experiments. We set the baseline in our experiments to com-
pare with gpt direct generation and Chain of Thought(CoT)
[Wei et al., 2023]. It is worth noting that we focussed on eval-
uating the performance of ArgMed-Agents on clinical deci-
sion reasoning, however, there were some biomedical general
knowledge quiz-type questions in MedQA and PubMedQA,
and so we intentionally excluded this part of the questioning
in selecting the example.
Table 1 shows the accuracy results of MedQA and Pub-
MedQA. We compared ArgMed-Agents to several baselines
in direct generation and CoT settings. Notably, both GPT-3.5-
turbo and GPT4 models showed that our proposed ArgMed-
Agents improved accuracy on a clinical decision reasoning
task compared to a baseline such as CoT. This result demon-
strates the effectiveness of ArgMed-Agents to enhance the
clinical reasoning performance of LLMs. Interestingly, after
we repeated the experiment several times, we found that the
introduction of CoT sometimes leads to a surprising drop in
performance. The reason for this may be that when LLMs
experience hallucinatory phenomena, the use of CoT will
amplify such hallucinations indefinitely. In contrast, our
approach of using mutual argumentation iterations between
multiple agents effectively mitigates this problem. We anal-
ysed the data from our experiments and found that each AS’s
CQ1 (i.e., asking the Verifier whether there is evidence to sup-
port the argument made by the Generator) acts as a halluci-
nation detector in the reasoning process. On the other hand,
Method Pre. Tra. % Arg.
CoT 0.63 2.8 59.5%
ArgMed-Agents 0.91 4.2 87.5%
Table 2: Human evaluation result on 20 examples.
we analysed the examples in which both Direct Generation,
CoT and ArgMed-Agents reasoned wrongly. We found that
ArgMed-Agents can report 76% of errors in these examples
according to Reasoner’s definition 4. This provides initial
evidence that ArgMed-Agents not only has higher accuracy
compared to baseline, but also that it is safer for clinical de-
cision reasoning.
5.2 Human Evaluation for Explainability
The main focus of Explainable AI (XAI) is usually the rea-
soning behind decisions or predictions made by AI that be-
come more understandable and transparent. In the context
of artificial intelligence, Explainability is defined as follows
[ISO, 2020]:
...level of understanding how the AI-based system came
up with a given result.
Based on this criterion, we define two measures of LLMs’
explanability:
• predictability(Pre.): Can the explanations given by
LLMs help users predict LLM decisions?
• proof-based traceability(Tra.): Can the model reason
about the correct decision based on the correct path of
inference (i.e. does the path of inference as an explana-
tion have relevance to the LLM’s decision)?
For the evaluation of predictability, our experimental setup
is as follows: we recorded the inputs and the reasoning pro-
cess (e.g., the complete dialogue between Generator and Ver-
ifier in ArgMed-Agents and the corresponding argumentation
framework). We invited 50 undergraduate and graduate stu-
dents in medical-related disciplines to answer questions given
only the inputs to the LLMs as well as the explanations (four
questions were distributed to each individual), and counted
to what extent they were able to predict the decisions of the
LLMs.
Our team has produced a fully functional knowledge-based
clinical decision support system at an early stage that can
be used to assist in treatment decisions for complex dis-
eases such as cancer, neurological disorders, and infectious
diseases. Figure 3 illustrates this knowledge-based CDSS,
consisting of computer-interpretable guidelines in the form
of Resource Description Framework (RDF) [Klyne and Car-
roll, 2004] and the corresponding inference engine. We use
the knowledge-based CDSS’s process of performing reason-
ing on the 20 examples in MedQA as a logical criterion for
measuring the proof-based traceability of LLMs. We manu-
ally evaluated the consistency of ArgMed-Agents’ reasoning
paths with logical criteria and the relevance of explanations
to decision making, scoring each example on a scale of one
to five. CoT performance was also used as a comparison.

Figure 3: A screenshot of the knowledge-based CDSS for diagnos-
ing depression, where prismatic nodes represent enquiry, circular
nodes represent decision, and square nodes represent action.
In Table 2, it is shown that ArgMed-Agents outperforms
CoT in both measures of explainability. Specifically, in
the experiment evaluating predictability, 50 participants an-
swered the question a total of 200 times, of which partici-
pants succeeded 91 out of 100 times in predicting ArgMed-
Agents decisions, compared to 63/100 times in predicting the
CoT. On the other hand, most of the reasoning nodes of the
knowledge-based CDSS can be found to correspond in the
reasoning process of ArgMed-Agents, whereas the reasoning
process of the CoT method to arrive at an answer is often log-
ically incomplete. Interestingly, we found that the reasoning
of the knowledge-based CDSS on many examples can be re-
garded as a reasoning subgraph of ArgMed-Agents, probably
because ArgMed-Agents traverses all the reasoning paths in
a randomly generated manner, and thus the reasoning graph
includes not only the reasoning paths of the correct decisions,
but also the explanations of the incorrect decisions.
6 Related Work
6.1 LLM-based Clinical Decision Support System
Extensive research highlights the potential for LLMs to be
used in medicine [Bao et al., 2023; Nori et al., 2023;
Jin et al., 2023]. However, LLMs still struggle to make safe
and trustworthy decisions when they encounter clinical deci-
sions that require complex medical expertise and good rea-
soning skills [Singhal et al., 2022]. Therefore, much work
has begun to focus on ways to enhance clinical reasoning in
LLMs. [Tang et al., 2024] proposes a Multi-disciplinary Col-
laboration (MC) framework that utilises LLM-based agents
in a role-playing environment that engages in collaborative
multi-round discussions until consensus is reached. De-
spite the results achieved, the method is unable to formalise
the iterative results in such a way as to enhance the infer-
ence performance of the LLM using inference tools. [Sav-
age et al., 2024] proposes a method that uses diagnostic
reasoning prompts to improve clinical reasoning abilities
and interpretability in LLM. However, their approach does
not focus on critical reasoning in clinical decision-making,
which allows LLM to generate explanations of why one de-
cision is ”good” but not why others were not chosen. In
addition to this, the work such as [Singhal et al., 2023;
Li et al., 2023] fine-tuned LLMs using extensive datasets col-
lected from the medical and biomedical literature. Unlike our
approach, ours focuses more on exploiting the latent medi-
cal knowledge inherent in LLMs to improve their reasoning
ability in a training-free environment.
6.2 Logical Reasoning with LLMs
An extensive body of research has been dedicated to utiliz-
ing symbolic systems to augment reasoning, which includes
code environments, knowledge graphs, and formal theorem
provers [Pan et al., 2023; Pan et al., 2024; Wu et al., 2023].
In the study by Jung et al. [Jung et al., 2022], reasoning is
constructed as a satisfiability problem of its logical relations
through inverse causal reasoning, with consistent reasoning
enhanced using SAT solvers to improve the reasoning abili-
ties of Large Language Models (LLMs). Zhang et al.’s work
[Zhang et al., 2023] employs cumulative reasoning to decom-
pose tasks into smaller components, thereby simplifying the
problem-solving process and enhancing efficiency. Xiu et al.
[Xiu et al., 2022] delve into the non-monotonic reasoning
capabilities of LLMs. However, despite the initial promise
exhibited by LLMs, their performance falls short in terms
of generalization and proof-based traceability, with a signif-
icant decline in performance observed with increasing depth
of reasoning. In line with our study, some recent works have
started to explore the potential for argumentative reasoning
in LLMs [Chen et al., 2023], aiming to enhance LLMs ar-
gumentative reasoning abilities [de Wynter and Yuan, 2023;
Castagna et al., 2024].
7 Conclusion
In this work, We propose a novel medical multi-agent interac-
tion framework called ArgMed-Agents, which inspires agents
of different roles to iterate through argumentation and critical
questioning, systematically generating an abstract argumen-
tation framework. Then, using a Reasoner agent to identify a
set of reasonable and coherent arguments in this framework as
decision support. The experimental results indicate that, com-
pared to different baselines, using ArgMed-Agent for clini-
cal decision-making reasoning achieves greater accuracy and
provides inherent explanations for its inferences.
Despite the success of ArgMed-Agents, there are still some
limitations. In particular, abstract arguments alone may strug-
gle with clinical reasoning tasks that numerical calculations
or probabilistic reasoning. Therefore, future research will fo-
cus on enhancing the capabilities of ArgMed-Agents to ad-
dress these challenges. One promising avenue involves ex-
ploring the integration of value-based argumentation or prob-
abilistic argumentation techniques. Our goal is to provide
healthcare professionals with powerful tools to enhance their
decision-making process and ultimately improve patient out-
comes.
Ethical Statement
There are no ethical issues.
References
[Ayers et al., 2023] John W. Ayers, Adam Poliak, Mark
Dredze, Eric C. Leas, Zechariah Zhu, Jessica B. Kel-
ley, Dennis J. Faix, Aaron M. Goodman, Christopher A.
Longhurst, Michael Hogarth, and Davey M. Smith. Com-
paring Physician and Artificial Intelligence Chatbot Re-
sponses to Patient Questions Posted to a Public Social Me-
dia Forum. JAMA Internal Medicine, 183(6):589–596, 06
2023.
[Bao et al., 2023] Zhijie Bao, Wei Chen, Shengze Xiao,
Kuang Ren, Jiaao Wu, Cheng Zhong, Jiajie Peng, Xu-
anjing Huang, and Zhongyu Wei. Disc-medllm: Bridg-
ing general large language models and real-world medical
consultation, 2023.
[Castagna et al., 2024] Federico Castagna, Nadin Kokciyan,
Isabel Sassoon, Simon Parsons, and Elizabeth Sklar. Com-
putational argumentation-based chatbots: a survey, 2024.
[Chen et al., 2023] Guizhen Chen, Liying Cheng, Luu Anh
Tuan, and Lidong Bing. Exploring the potential of large
language models in computational argumentation, 2023.
[Creswell et al., 2022] Antonia Creswell, Murray Shanahan,
and Irina Higgins. Selection-inference: Exploiting large
language models for interpretable logical reasoning, 2022.
[Čyras et al., 2018] Kristijonas Čyras, Brendan Delaney,
Denys Prociuk, Francesca Toni, Martin Chapman, Jesús
Domı́nguez, and Vasa Curcin. Argumentation for ex-
plainable reasoning with conflicting medical recommen-
dations. CEUR Workshop Proceedings, 2237, January
2018. 2018 Joint Reasoning with Ambiguous and Con-
flicting Evidence and Recommendations in Medicine and
the 3rd International Workshop on Ontology Modularity,
Contextuality, and Evolution, MedRACER + WOMoCoE
2018 ; Conference date: 29-10-2018.
[de Wynter and Yuan, 2023] Adrian de Wynter and Tommy
Yuan. I wish to have an argument: Argumentative reason-
ing in large language models, 2023.
[Dietz et al., 2021] Emmanuelle Dietz, Antonis Kakas, and
Loizos Michael. Computational argumentation and cogni-
tion, 2021.
[Dietz et al., 2022] Emmanuelle Dietz, Antonis Kakas, and
Loizos Michael. Argumentation: A calculus for human-
centric ai. Frontiers in Artificial Intelligence, 5, 2022.
[Dung, 1995] Phan Minh Dung. On the acceptability of ar-
guments and its fundamental role in nonmonotonic rea-
soning, logic programming and n-person games. Artificial
Intelligence, 77(2):321–357, 1995.
[Eigner and Händler, 2024] Eva Eigner and Thorsten
Händler. Determinants of llm-assisted decision-making,
2024.
[Fan and Toni, 2015] Xiuyi Fan and Francesca Toni. On
computing explanations in argumentation. In Proceedings
of the Twenty-Ninth AAAI Conference on Artificial Intelli-
gence, AAAI’15, page 1496–1492. AAAI Press, 2015.
[Gandhi et al., 2023] Kanishk Gandhi, Dorsa Sadigh, and
Noah D. Goodman. Strategic reasoning with language
models, 2023.
[Hong et al., 2023] Shengxin Hong, Liang Xiao, and Jianxia
Chen. An interaction model for merging multi-agent
argumentation in shared clinical decision making. In
2023 IEEE International Conference on Bioinformatics
and Biomedicine (BIBM), pages 4304–4311, 2023.
[ISO, 2020] ISO, IEC: AWI TS 29119-11: Software and sys-
tems engineering - software testing - part 11: Testing of AI
systems. Technical report, 2020.
[Jin et al., 2019] Qiao Jin, Bhuwan Dhingra, Zhengping Liu,
William W. Cohen, and Xinghua Lu. Pubmedqa: A dataset
for biomedical research question answering, 2019.
[Jin et al., 2020] Di Jin, Eileen Pan, Nassim Oufattole, Wei-
Hung Weng, Hanyi Fang, and Peter Szolovits. What dis-
ease does this patient have? a large-scale open domain
question answering dataset from medical exams, 2020.
[Jin et al., 2023] Qiao Jin, Yifan Yang, Qingyu Chen, and
Zhiyong Lu. Genegpt: Augmenting large language mod-
els with domain tools for improved access to biomedical
information, 2023.
[Jung et al., 2022] Jaehun Jung, Lianhui Qin, Sean Welleck,
Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras,
and Yejin Choi. Maieutic prompting: Logically consistent
reasoning with recursive explanations, 2022.
[Klyne and Carroll, 2004] Graham Klyne and Jeremy J. Car-
roll. Resource description framework (rdf): Concepts and
abstract syntax. W3C Recommendation, 2004.
[Li et al., 2023] Chunyuan Li, Cliff Wong, Sheng Zhang,
Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan
Naumann, Hoifung Poon, and Jianfeng Gao. Llava-
med: Training a large language-and-vision assistant for
biomedicine in one day, 2023.
[Nayak et al., 2023] Anupama Nayak, Michael S Alkaitis,
Karthik Nayak, Martin Nikolov, Kevin P Weinfurt, and
Kevin Schulman. Comparison of history of present ill-
ness summaries generated by a chatbot and senior internal
medicine residents. JAMA Intern Med, 183(9):1026–1027,
Sep 2023.
[Nori et al., 2023] Harsha Nori, Yin Tat Lee, Sheng Zhang,
Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas
King, Jonathan Larson, Yuanzhi Li, Weishung Liu, Ren-
qian Luo, Scott Mayer McKinney, Robert Osazuwa Ness,
Hoifung Poon, Tao Qin, Naoto Usuyama, Chris White, and
Eric Horvitz. Can generalist foundation models outcom-
pete special-purpose tuning? case study in medicine, 2023.
[Oliveira et al., 2018] Tiago Oliveira, Jérémie Dauphin, Ken
Satoh, Shusaku Tsumoto, and Paulo Novais. Argumen-
tation with goals for clinical decision support in multi-
morbidity. In Proceedings of the 17th International Con-
ference on Autonomous Agents and MultiAgent Systems,
2018. 2031–2033.
[OpenAI, 2023] OpenAI. Gpt-4 technical report, 2023.
[Pal et al., 2023] Ankit Pal, Logesh Kumar Umapathi, and
Malaikannan Sankarasubbu. Med-halt: Medical domain
hallucination test for large language models, 2023.
[Pan et al., 2023] Liangming Pan, Alon Albalak, Xinyi
Wang, and William Yang Wang. Logic-lm: Empowering
large language models with symbolic solvers for faithful
logical reasoning, 2023.
[Pan et al., 2024] Shirui Pan, Linhao Luo, Yufei Wang, Chen
Chen, Jiapu Wang, and Xindong Wu. Unifying large lan-
guage models and knowledge graphs: A roadmap. IEEE
Transactions on Knowledge and Data Engineering, page
1–20, 2024.
[Qassas et al., 2015] Malik Al Qassas, Daniela Fogli, Massi-
miliano Giacomin, and Giovanni Guida. Analysis of clin-
ical discussions based on argumentation schemes. Pro-
cedia Computer Science, 64:282–289, 2015. Conference
on ENTERprise Information Systems/International Con-
ference on Project MANagement/Conference on Health
and Social Care Information Systems and Technologies,
CENTERIS/ProjMAN / HCist 2015 October 7-9, 2015.
[Sassoon et al., 2021] Isabel Sassoon, Nadin Kökciyan, San-
jay Modgil, and Simon Parsons. Argumentation schemes
for clinical decision support. Argument and Computation,
12(3):329–355, November 2021.
[Savage et al., 2024] Thomas Savage, Abhishek Nayak,
Robert Gallo, and et al. Diagnostic reasoning prompts re-
veal the potential for large language model interpretability
in medicine. npj Digital Medicine, 7:20, 2024.
[Shi et al., 2024] Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue
Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce Ho, Carl
Yang, and May D. Wang. Ehragent: Code empowers large
language models for few-shot complex tabular reasoning
on electronic health records, 2024.
[Singhal et al., 2022] Karan Singhal, Shekoofeh Azizi, Tao
Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung,
Nathan Scales, Ajay Tanwani, Heather Cole-Lewis,
Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul
Gamble, Chris Kelly, Nathaneal Scharli, Aakanksha
Chowdhery, Philip Mansfield, Blaise Aguera y Arcas,
Dale Webster, Greg S. Corrado, Yossi Matias, Kather-
ine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu,
Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan
Karthikesalingam, and Vivek Natarajan. Large language
models encode clinical knowledge, 2022.
[Singhal et al., 2023] Karan Singhal, Tao Tu, Juraj Gottweis,
Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark,
Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, Mike
Schaekermann, Amy Wang, Mohamed Amin, Sami Lach-
gar, Philip Mansfield, Sushant Prakash, Bradley Green,
Ewa Dominowska, Blaise Aguera y Arcas, Nenad Toma-
sev, Yun Liu, Renee Wong, Christopher Semturs, S. Sara
Mahdavi, Joelle Barral, Dale Webster, Greg S. Corrado,
Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam,
and Vivek Natarajan. Towards expert-level medical ques-
tion answering with large language models, 2023.
[Tang et al., 2024] Xiangru Tang, Anni Zou, Zhuosheng
Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman
Cohan, and Mark Gerstein. Medagents: Large language
models as collaborators for zero-shot medical reasoning,
2024.
[Walton et al., 2008] Douglas Walton, Chris Reed, and Fab-
rizio Macagno. Argumentation Schemes. Cambridge Uni-
versity Press, New York, 2008.
[Walton, 1996] Douglas Walton. Argumentation Schemes for
Presumptive Reasoning. Routledge, 1st edition, 1996.
[Wang et al., 2023] Lei Wang, Chen Ma, Xueyang Feng,
Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Ji-
akai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei
Wei, and Ji-Rong Wen. A survey on large language model
based autonomous agents, 2023.
[Wei et al., 2023] Jason Wei, Xuezhi Wang, Dale Schuur-
mans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc
Le, and Denny Zhou. Chain-of-thought prompting elicits
reasoning in large language models, 2023.
[Wu et al., 2023] Qingyun Wu, Gagan Bansal, Jieyu Zhang,
Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun
Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadal-
lah, Ryen W White, Doug Burger, and Chi Wang. Auto-
gen: Enabling next-gen llm applications via multi-agent
conversation, 2023.
[Xie et al., 2024] Jian Xie, Kai Zhang, Jiangjie Chen,
Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao,
and Yu Su. Travelplanner: A benchmark for real-world
planning with language agents, 2024.
[Xiu et al., 2022] Yeliang Xiu, Zhanhao Xiao, and Yongmei
Liu. LogicNMR: Probing the non-monotonic reasoning
ability of pre-trained language models. In Yoav Goldberg,
Zornitsa Kozareva, and Yue Zhang, editors, Findings of the
Association for Computational Linguistics: EMNLP 2022,
pages 3616–3626, Abu Dhabi, United Arab Emirates, De-
cember 2022. Association for Computational Linguistics.
[Zeng et al., 2020] Zhiwei Zeng, Zhiqi Shen, Benny
Toh Hsiang Tan, Jing Jih Chin, Cyril Leung, Yu Wang,
Ying Chi, and Chunyan Miao. Explainable and
Argumentation-based Decision Making with Qualita-
tive Preferences for Diagnostics and Prognostics of
Alzheimer’s Disease. In Proceedings of the 17th In-
ternational Conference on Principles of Knowledge
Representation and Reasoning, pages 816–826, 9 2020.
[Zhang et al., 2023] Yifan Zhang, Jingqin Yang, Yang Yuan,
and Andrew Chi-Chih Yao. Cumulative reasoning with
large language models, 2023.
A Formal Description of ArgMed-Agents

Algorithm 1 ArgMed-Agents Interaction Reasoning

Require: Question Q, Argumentation Schemes AS∈{ASD , ASSE, ASBD}
Ensure: Decision D
1: function argGeneration(Q, AS)
2: arg ← Generator(Q, AS.prompt) ▷ Arguments presented by Generator
3: AA.add(arg) ▷ Add arguments to AA framework
4: for CQ in AS.CQs do
5: if Verifier(arg, CQ) is rejected then ▷ Verifier proposes the arg rejected by CQ
6: if AS.derive(CQ) = True then ▷ When CQ exist derived role
7: AS ← AS.derive(CQ) ▷ Update AS to derived role
8: arg2 ← argGeneration(Q, AS) ▷ recursive
9: AA.addAttack(arg2 , arg) ▷ arg2 attack arg
10: else
11: AA.addAttack(arg, arg) ▷ Self-Attack (Defeasible)
12: end if
13: end if
14: end for
15: return arg
16: end function
17:
18: function main
19: AA ← ∅ ▷ initialize the AA framework
20: argGeneration(Q, ASD)
21: D ← Reasoner(AA)
22: end function

B ASCD Reasoning Mechanisms

Argumentation Scheme Critical Questions Reject Rule Derive Role Attack Rule
ASD.CQ1 NO (no evidence to support the argument) none self-attack
ASD.CQ2 YES (there are side effects of decision-making) ASSE ASSE attack ASD
Argumentation Scheme
for Decision(ASD) ASD.CQ3 NO (incomplete decisions to fulfil goals) none self-attack
ASD.CQ4 YES (existence of alternative decisions) ASD ASBD attack the worse decision
in ASD 1 and ASD 2
ASSE.CQ1 NO (no evidence to support the argument) none self-attack
Argumentation Scheme
ASSE.CQ2 NO (the side-effect is acceptable) none self-attack
for Side Effect(ASSE)
ASSE.CQ3 YES (there are ways to ameliorate side effects) ASD none
Argumentation Scheme ASBD.CQ1 NO (no evidence to support the argument) none self-attack
for Better Decision(ASBD)

Table 3: List of specialized schemes, CQs, when CQs will be rejected and their derived roles and attack rules between derived roles and initial
schemes. Notably, every ASD is attacking each other by default, so it is not shown in the attack rules.

C More Case Details in Experiment

Question: An 18-year-old woman presents with recurrent headaches. The
Generator D [Derive from ASD(B).CQ4]:
pain is usually unilateral, pulsatile in character, exacerbated by
ASD(F, G, D)
light and noise, and usually lasts for a few hours to a full day. The
Premise - Given the patient Fact F [An 18-year-old woman presents with recurrent headaches, described as
pain is sometimes triggered by eating chocolates. These headaches unilateral, pulsatile, exacerbated by light and noise, sometimes triggered by eating chocolates, and
disturb her daily routine activities. The physical examination was accompanied by essential tremors.]
within normal limits. She also has essential tremors. Which drug is
suitable in her case for the prevention of headaches? Premise - To achieve Goal G [G: Prevent the recurrence of the patient's headaches and improve quality of
life.]

Options: a: "Ergotamine", b: "Verapamil", c: "Valproic, d: Premise - Valproic acid promotes the goal G [Valproic acid is an anticonvulsant and mood-stabilizing drug
"Propranolol" that has been shown to be effective in migraine prophylaxis and could also have a benefit on essential
tremors.]

Generator A: Conclusion: Valproic acid should be considered.

ASD(F, G, D)
Premise - Given the patient Fact F [An 18-year-old woman with recurrent headaches Verifier:
that are unilateral, pulsatile, exacerbated by light and noise, lasting for hours ASD.CQ1- Are there any evidence to support Decision D promote goal G?
to a day, sometimes triggered by chocolates, and disturb daily activities. YES
Diagnosis suggestive of migraines. She also has essential tremors.] ASD.CQ2- Has Decision D caused side effects on the patient?
YES
Premise - To achieve Goal G [G: Prevent the patient's headaches and manage ASD.CQ3- Can Goal G be achieved by completing the Decision D?
essential tremors.] YES
ASD.CQ4- Are there alternative Decision to achieve the same goal G?
Premise - Propranolol promotes the goal G [Propranolol is a beta-blocker that is NO
effective in preventing migraines and is also used to treat essential tremors,
which the patient has.] Generator E [Derive from ASD(B).CQ4]:
ASBD(F, G, D1, D2)
Conclusion: Propranolol should be considered. Premise - Given the decisions Valproic Acid (D1) and Propranolol (D2)

Verifier: Premise -
Effectiveness in Migraine Prophylaxis:
ASD.CQ1- Are there any evidence to support Decision D promote goal G?
Both Valproic Acid and Propranolol are effective in the prevention of migraines, which align with the patient's
YES
goal of preventing headaches.
ASD.CQ2- Has Decision D caused side effects on the patient?
NO
Side Effect Profile:
ASD.CQ3- Can Goal G be achieved by completing the Decision D?
Valproic Acid is associated with a variety of potential side effects including weight gain, hair loss, and
YES
teratogenicity, which may not be ideal for a young woman. Propranolol, while also having potential side
ASD.CQ4- Are there alternative Decision to achieve the same goal G?
effects, is less likely to cause these particular issues and may provide a dual benefit for this patient's
YES
existing essential tremor condition.

Generator B [Derive from ASD(A).CQ4]: The medical profile of the Patient:

The patient is an 18-year-old woman with essential tremors, which is also an indication for Propranolol. As
ASD(F, G, D) Propranolol is a non-selective beta-blocker, it is not only effective for migraine prophylaxis but also for
Premise - Given the patient Fact F [An 18-year-old woman presents with recurrent treating essential tremors.
headaches characterized as unilateral, pulsatile, exacerbated by light and noise,
lasting a few hours to a full day, sometimes triggered by eating chocolates, and
Conclusion: Propranolol should be considered
disrupting her daily activities; she also has essential tremors.]

Premise - To achieve Goal G [G: Prevent the patient's recurrent headaches] Verifier:
ASD.CQ1- Are there any evidence to support Decision D promote goal G?
Premise - Verapamil promotes the goal G [Verapamil is a calcium channel blocker YES
that can be effective in the prophylaxis of headaches, specifically in migraines ASD.CQ2- Has Decision D caused side effects on the patient?
and cluster headaches; it can also be beneficial for patients with coexisting YES
conditions where beta-blockers like propranolol are contraindicated or ASD.CQ3- Can Goal G be achieved by completing the Decision D?
ineffective.] YES
ASD.CQ4- Are there alternative Decision to achieve the same goal G?
Conclusion: Verapamil should be considered. NO

Verifier: Generator F [Derive from ASD(D).CQ2]:

ASD.CQ1- Are there any evidence to support Decision D promote goal G? ASSE(SE, D, F)
YES Premise - The patient is an 18-year-old woman with recurrent, unilateral, pulsatile headaches that are
ASD.CQ2- Has Decision D caused side effects on the patient? exacerbated by light and noise, lasting for several hours to a day, and triggered by eating chocolates. She has a
NO history of essential tremors and her physical examination was within normal limits.
ASD.CQ3- Can Goal G be achieved by completing the Decision D?
YES Premise - The decision to prescribe Valproic acid is made for the prevention of headaches. Valproic acid is a
ASD.CQ4- Are there alternative Decision to achieve the same goal G? medication that can be used to prevent migraines, which her symptoms suggest, as well as for treating seizures
YES and bipolar disorder.

Premise - Valproic acid can have side effects such as weight gain, hair loss, tremors, and liver toxicity. The
liver toxicity is particularly concerning and requires regular monitoring of liver function tests. In women of
Generator C [Derive from ASD(A).CQ4]: childbearing age, valproic acid also carries a risk of teratogenicity and thus must be used with effective
ASBD(F, G, D1,D2) contraception to prevent pregnancy.
Premise - Given the decisions Verapamil and Propranolol
Conclusion: Given the potential risk of significant side effects, including liver toxicity and teratogenicity, as
Premise - well as the patient's existing essential tremors which could potentially be exacerbated by valproic acid, the
Effectiveness in Preventing Headaches: decision to use Valproic acid as a preventive treatment for her headaches needs to be made with caution. Careful
Both Verapamil and Propranolol can be used to prevent headaches, but their consideration should be given to the patient's health status, potential for pregnancy, and the need for regular
mechanisms of action and efficacy in specific types of headaches may differ. monitoring if Valproic acid is chosen as a therapy.

Additional Medical Condition (Essential Tremors): Verifier:

Propranolol is not only used for the prophylaxis of headaches but also for
essential tremors, which the patient has. While Verapamil is a calcium channel ASD.CQ1- Is there any evidence to support Decision D results in side effect SE?
blocker used in various other conditions, it's not particularly indicated for YES
essential tremors. ASD.CQ1- Is this side effect SE not acceptable?
YES
Patient-specific factors and side effect profile: ASD.CQ1- Are there ways of ameliorating the side effects observed?
Considering the patient's age, gender, and the type of headaches, which are likely NO
migraines based on the description, Propranolol is often used as a first-line
preventive treatment for migraines. Propranolol might also help control her
essential tremor symptoms. Verapamil can be used for migraine prevention but does
not address her tremors.
AA framework: Decision:

Conclusion: Propranolol should be considered F A C A

Verifier:
Propranolol should
ASD.CQ1- Is there any evidence to support Decision D2 better than Decision D1 for
E B
Goal G?
YES D be considered

Figure 4: Example of correct reasoning by ArgMed-Agents. We show the flow of the multi-agent dialogue and the resulting formal AA
framework.
 Question: A 14-year-old girl is brought to the physician by her Generator C [Derive from ASD(A).CQ4]:
mother for the evaluation of recurrent episodes of nose bleeding ASD(F, G, D)
for several months. The episodes occur unexpectedly and stop after Given the patient Fact F [A 14-year-old girl with recurrent nose bleeding
a few minutes by elevating the upper body and bending the head episodes and heavy menstrual flow, normal vital signs, a slightly reduced
hemoglobin level, a normal platelet count, prolonged partial thromboplastin
forward. Menses occur at regular 27-day intervals with heavy flow. time, and prolonged bleeding time.]
Her last menstrual period was 3 weeks ago. Vital signs are within
normal limits. Physical examination shows no abnormalities. Premise - To achieve Goal G [G: Stop the patient's bleeding and prevent future
Laboratory studies show:\nHemoglobin 11 g/dL\nHematocrit 34%\ episodes.]
nLeukocyte count 7,000/mm3\nPlatelet count 180,000/mm3\
Premise - Desmopressin promotes the goal G [Desmopressin is a medication that
nProthrombin time 13 sec\nPartial thromboplastin time 45 sec\ acts on the vasopressin receptors and can increase factor VIII levels, which
nFibrin split products negative\nThe bleeding time is 10 minutes. can help improve coagulation in bleeding disorders such as von Willebrand
Which of the following is the most appropriate next step in disease, which is suspected in this girl due to the symptoms and prolonged
treatment? bleeding time.]

Conclusion: Desmopressin should be considered

Options: a: "Oral prednisone", b: "Intravenous immunoglobulins",
c: "Tranexamic acid" , d: "Desmopressin"
Verifier:
ASD.CQ1- Are there any evidence to support Decision D promote goal G?
YES
Generator A: ASD.CQ2- Has Decision D caused side effects on the patient?
NO
ASD(F, G, D)
ASD.CQ3- Can Goal G be achieved by completing the Decision D?
Premise - Given the patient Fact F [A 14-year-old girl with a history of
YES
recurrent nosebleeds and heavy menstruation, presenting with laboratory findings
ASD.CQ4- Are there alternative Decision to achieve the same goal G?
that suggest a potential bleeding disorder given the prolonged bleeding time and
NO
slightly prolonged partial thromboplastin time.]

Premise - To achieve Goal G [G: Stop the patient's current bleeding episode
effectively and reduce the likelihood of future bleeding episodes.]
Premise - Tranexamic acid promotes Goal G [Tranexamic acid is an
antifibrinolytic agent known to reduce bleeding by inhibiting the activation of
plasminogen to plasmin, thereby aiding in the stabilization of blood clots and
is indicated in cases of heavy menstrual bleeding and situations where excessive
bleeding is a concern.]
Conclusion: Tranexamic acid should be considered.
Verifier:
ASD.CQ1- Are there any evidence to support Decision D promote goal G?
YES
ASD.CQ2- Has Decision D caused side effects on the patient?
YES
ASD.CQ3- Can Goal G be achieved by completing the Decision D?
YES
ASD.CQ4- Are there alternative Decision to achieve the same goal G?
YES
Generator B [Derive from ASD(A).CQ2]:
ASSE(SE, D, F)
Premise - The patient is a 14-year-old girl with a history of recurrent
nosebleeds and heavy menstruation, presenting with laboratory findings
suggestive of a potential bleeding disorder, including prolonged bleeding time
and slightly prolonged partial thromboplastin time.
Premise - The decision to administer Tranexamic acid, an antifibrinolytic
agent known to reduce bleeding, is considered as a potential course of
treatment to achieve the goal of stopping the patient's current bleeding
episode effectively and reducing the likelihood of future bleeding episodes.
Premise - Tranexamic acid, like many medications, can have side effects.
Common side effects of Tranexamic acid include nausea, diarrhea, headache,
muscle pain, or weakness. Additionally, more serious side effects such as
blood clots, visual disturbances, or allergic reactions can occur, although
they are less common.
Generator D [Derive from ASD(A).CQ4]:
ASBD(F, G, D1, D2)
Premise - Given the decisions Tranexamic acid and Desmopressin
Premise -
Effectiveness in Treating Bleeding Disorders:
Desmopressin is effective in treating patients with bleeding disorders because
it increases the levels of factor VIII and von Willebrand factor, which helps
in improving coagulation. Tranexamic acid helps by inhibiting the breakdown of
fibrin clots, but does not address underlying deficiencies in clotting
factors.
Directness of Treatment:
Desmopressin directly addresses the potential underlying problem in this
patient—impaired function of platelets or factor VIII deficiency—by increasing
factor VIII levels. Tranexamic acid would not correct the underlying
coagulation deficiency; it only helps to stabilize clots that have already
formed.
Potential for Recurrence Prevention:
Desmopressin can prevent or reduce the severity of future bleeding episodes by
increasing factor VIII levels, particularly before known triggers (like
surgical procedures). Tranexamic acid is more of a temporary measure during an
acute bleeding episode and does not have a prophylactic effect.
Given the Fact the patient has a prolonged partial thromboplastin time,
indicating a possible deficiency in intrinsic pathway coagulation factors, and
Desmopressin can increase factor VIII levels, which could be deficient in this
patient:
Conclusion: Desmopressin should be considered
Verifier:
ASD.CQ1- Is there any evidence to support Decision D2 better than Decision D1
for Goal G?
YES

Conclusion: Considering the patient's symptoms and the potential side effects
AA framework: Decision:
of Tranexamic acid, the decision to administer Tranexamic acid needs to be
D
C
made with caution.

Verifier:
ASD.CQ1- Is there any evidence to support D ecision D results in
B A
side effect SE? Desmopressin should be
YES
Is this side effect SE not acceptable? considered
C
ASD.CQ1-
NO
ASD.CQ1- Are there ways of ameliorating the side effects observed?
NO

Figure 5: (continued) Example of correct reasoning by ArgMed-Agents. We provide runnable demo files with available GPT-4 API Keys in
the code that include these cases.