Prompt Engineering in Large Language Models

Devansh Bansal
Computer Engineering Department, San Jose State University,
San Jose, USA
Email: devansh.bansal@sjsu.edu

Abstract: Within the dynamic realm of natural
language processing (NLP) and artificial intelligence
(AI), large language models (LLMs) have emerged as
revolutionary instruments that have increased the
human-AI interaction manifold. These models have an
exceptional capacity to produce text that closely
resembles human language, utilizing input prompts as a
basis. They can understand information related to any
domain and provide suitable and correct responses, for
e.g: understanding mathematical formulas, writing code,
summarizing text, and so on. However, the crucial factor
in realizing their complete capabilities and guaranteeing
a smooth collaboration between humans and AI is in the
field of prompt engineering, which combines both
artistic and scientific elements. The goal of this study is
to delve into the world of LLMs, understand how they
work, and examine how prompts affect performance in
terms of correctness and speed. In this study, we
examine different prompt engineering methods and
propose a layered architecture approach toward
prompting the foundational models such as GPT and
DALLE.
these apps and services essentially provide three main
functionalities:
1. Text Understanding
2. Content generation
3. Reasoning
Currently, there are two main problems that researchers are
trying to solve in this domain. The first is that these models
are very expensive to train and the training data really
affects the performance of these models. So it’s crucial that
the data is clean in all aspects. It should not be biased or
incorrect. The second problem is that once these models are
trained, the only way to interact with them is via text
prompts or attachments comprising text files.
Consider the following two prompts given to ChatGPT
running on GPT 3.5

Index Terms—Prompt Engineering; Demystifying Large

Language Models; Improving LLM performance using prompt
enginering; Prompts in LLMs
I. INTRODUCTION
Recent advances in the fields of Natural Language
Processing and Generative AI have led to the development
of models like GPT-3, DALL-E, Claude, etc. that
demonstrate strong abilities in language understanding, text
generation, and reasoning. However, successfully leveraging
their capabilities is contingent on effective prompt
engineering—the careful construction of text prompts to
provide optimal context and direction. These powerful
models have the potential to improve the user experience in
different areas such as:
1. Students are using different LLMs as study guides or
personal AI tutors. [3]
2. Content creators and artists are leveraging models like
DALLE to generate new content and ideas.
3. Data engineers are using these models to breeze through
large datasets and texts.
We envision a future comprising of multiple AI agents
providing services in specific niches. We will see rise of
multiple tools, and application built on the top of the
foundational models such as GPT and DALLE. Most of
Prompt 1
Prompt 2
The responses to the above prompts greatly varied from
each other. In the first case, ChatGPT gave a very abstract
answer without delving into the critical details, whereas in
the second, the model gave a very specific and convincing
answer. This sensitivity illustrates the need for further
research into systematic and equitable prompt engineering
methods that facilitate more robust human-AI collaboration.
[1]
Currently, a lack of consensus on standards, best practices,
and transparency in prompt design presents barriers to
real-world deployment. Prior studies have also tended to
focus only on narrow aspects of prompt optimization or
tasks, with insufficient consideration of fairness and
accountability. Recent analysis reveals high variability in
outputs even from the same LLM when prompted
differently, underscoring the levels of control that prompt
engineering permits.
In this research, we first study the history of AI evolution,
and then we study how LLMs came into being and their
applications in today’s world. Then we talk about Prompt
Engineering and evaluate how it affects the quality of
response in terms of correctness and speed.
2. Evolution of AI
Humans started talking about AI in the early 20th century.
At that time, cinema and art used to present this magical
creature that could think like humans or even better than
them. This thinking could materialize into something real
only around the late 1950s. That was the time when the first
AI language was also introduced, called LISP, by John
McCarty.
From the 1980s-2011, the world witnessed a couple of
breakthroughs in the world of AI. The first expert agent,
known as XCon, was introduced. One of the AI agents beat
the human world chess champion. These events acted like a
fire and ignited the AI boom around 2010. Since then, there
has been an exponential advancement in AI, both in terms of
scientific breakthroughs and its adoption in the real world.
Last year, when OpenAI released its first AI chat application
called ChatGPT, which was powered by its most powerful
large language model called GPT, the entire world was
shaken. The app got its first million users in a span of five
days, the fastest ever in the world’s history. This also led to
a spurt in the growth of research in the area. Many people
started researching these LLMs, how they work, and how to
make them work better. This is how Prompt Engineering
came into being. Some people even say that Prompts are the
new language of today’s world and that humans should use
better prompts while communicating amongst themselves as
well.
3. Related Work
To understand the current state of work and different
sub-domains in this large domain of artificial intelligence,
we studied multiple research papers. From the background
research, it was evident that the following four themes have
direct correlation with the topic of this study and need some
attention. Understanding these themes help in navigating the
large and still immature world of large language models and
generative artificial intelligence.
A. Non-determinism in LLMs
Large Language Models (LLMs) possess an inherent degree
of opacity, akin to a black box, where the intricacies of their
functioning are not fully transparent. While their
fundamental operation mirrors the interconnected networks
and weighted nodes within the human brain, the precise
mechanism by which these models generate text responses
based on user prompts remains largely uncharted. Our
research endeavors delved into this mystery by employing
varied prompts for a singular use case. Remarkably, we
uncovered a robust correlation between the internal thought
processes of these models as they formulate responses and
the dynamics of human-human interaction. This revelation
underscores the paramount importance of acquiring a
nuanced understanding of how to pose effective questions to
LLMs.
One of the techniques that really stood out was the chain of
reasoning. Researchers tried a chain of reasoning to build
the prompts, similar to how humans structure their thoughts
to form sentences. Surprisingly, just by restructuring the
prompts, the performance of the model improved by 60%. It
generated more correct responses with verified facts.
A. Correctness and Reliability of LLMs
In light of the increasing integration of Large Language
Models (LLMs) into practical applications, ensuring the
accuracy and dependability of model outputs is crucial. A
notable challenge in this regard is the occurrence of
hallucinations, where the model interprets user prompts in
unintended ways. Effectively addressing this challenge is
essential for the meaningful use of LLMs across various
fields. Prompt Engineering emerges as a strategic
framework to tackle this issue by refining user inputs for
more aligned outcomes.
Researchers have explored diverse architectural approaches
to alleviate hallucinations. Investigating innovative model
designs serves as a critical avenue for minimizing
interpretative deviations in LLM responses. Apart from
architectural considerations, two prominent factors
significantly influencing response quality have surfaced: the
quality and diversity of training data and the precision of
input prompts. The interplay between these factors is
complex, as the model's ability to generate accurate
responses depends on both the depth and variety of its
training data and the specificity and clarity of the prompts
encountered in real-world scenarios.
As researchers work towards optimizing the accuracy and
reliability of LLMs, the exploration of various architectural
approaches and a nuanced understanding of the intricate
relationship between training data and prompts take
precedence in ongoing investigations. By enhancing insights
into these multifaceted aspects, we can advance effective
Prompt Engineering strategies, contributing significantly to
the broader objective of refining the precision and reliability
of LLM-generated outputs.
2
 C. Prompting techniques and model behavior
Within this section, we conducted an in-depth examination
of various prompting techniques proposed by fellow
researchers. A noteworthy study in this domain focused on
prompt engineering tailored for text-to-image generative AI
models. The authors of this research undertook a
comprehensive exploration, executing five distinct
experiments that varied prompts, random seeds,
optimization lengths, styles, and subjects. Notably, their
findings indicated that the incorporation of prompt seeds
and iteration lengths in the final prompts resulted in
improved model responses. Moreover, the study highlighted
that diverse styles were interpreted distinctively by the
models, revealing nuances in the models' responsiveness.
Subsequently, the researchers proffered insightful prompt
guidelines, aiming to inform the design of applications
utilizing such models. [2]
D. Applications of LLMs
Our research is based on the assumption that the adoption of
AI in different spheres of human life will increase. In
today’s world, the mobile apps that we use will be taken
over by these AI agents, providing specific experiences to
the users.
3. Project Description and Motivation
A. Purpose and Significance
In today’s world, AI is significantly changing experiences
and systems across different spheres. In a couple of years,
this interaction between humans and AI in different realms
of life will increase much more. It’s important that everyone
is aware of this amazing technology and how to leverage it
for their own use cases. LLMs represent a fundamental shift
in how we interact with intelligent systems and it’s critical
that we well understand how they work to ensure their full
utilization. In this project, we scope our goals to understand
these models from prompting perspective and provide a set
of guidelines for the users so that they can write better
prompts and get better results.
B. Problem Statement
In today's AI landscape, prompts play a crucial role. Large
Language Models (LLMs) tend to respond differently to
questions depending on who's asking. This variability can
lead to different user perceptions about the model's
performance. We see a lot of potential in these models, and
there are many interesting ways in which we can use these
models if we make user interactions simpler and intuitive. In
an ideal situation, the model should accurately understand
and respond to user questions in a way that feels human. To
achieve this, it's crucial to understand how these models
function, how they react to different inputs and prompts, and
the algorithmic tweaks needed to improve response
accuracy and speed.
C. Objectives
This study aims to achieve the following main objectives:
● Comprehend the workings and behavior of Large
Language Models (LLMs) across diverse
scenarios.
● Explore the applications of LLMs to gain a
comprehensive understanding of their practical
uses.
● Investigate the interaction dynamics between
prompts and LLMs.
● Analyze strategies for crafting improved prompts
to enhance model responses.
● Propose guidelines for architecture and design to
facilitate the development of applications that
optimize prompt-based interactions.
D. Motivation
AI has advanced rapidly in the last few years but awareness
and user adoption are still far behind. 80% of the total
human population in the world still doesn’t know about it
and even among the latter 20%, most don’t know how to ask
the right questions to these models. It’s very important to do
research in this area and simultaneously create awareness
about it in the community.
4. Technical Aspects
We propose an iterative, prompt optimization pipeline with
reinforced active learning for automated generation and
testing customized to target outcomes. A model-agnostic
waveform controller architecture updates prompts based on
modular validation signals, allowing rapid tuning without
human input. Before discussing the architecture and the
implementation plan, let’s first discuss the foundational
models that we will use interchangeably under the hood.
3
 A. GPT (Generative Pre-trained Transformer):
GPT, a pioneering model in the landscape of language
models, has significantly contributed to the advancements in
Prompt Engineering for Large Language Models (LLMs).
Developed by OpenAI, GPT employs transformer
architecture and is pre-trained on massive datasets, enabling
it to generate coherent and contextually relevant text. Its
ability to understand and respond to natural language
prompts has made it a cornerstone in the field. Researchers
have extensively explored techniques to fine-tune GPT for
specific tasks, highlighting its flexibility in adapting to
diverse applications.
B. DALL-E:
DALL-E, an innovative model by OpenAI, extends the
capabilities of generative models into the realm of images.
Unlike traditional language-centric LLMs, DALL-E
operates as a generative model for visual content. By
conditioning on textual prompts, DALL-E generates images
that correspond to the given descriptions. Understanding
and manipulating prompts in the visual domain present
unique challenges, and DALL-E's architecture offers
insights into the intersection of language and image prompt
engineering.
C. CLAUDE (Contrastive Language-Image
Pre-Training):
CLAUDE represents a novel approach to prompt
engineering by bridging the gap between language and
image modalities. Developed to understand cross-modal
relationships, CLAUDE is pre-trained on datasets
containing both textual and visual information. This model
showcases the potential for prompt engineering in scenarios
where input involves both language and image components.
Analyzing CLAUDE sheds light on strategies to effectively
engineer prompts in multimodal settings.
D. GEMINI (Generative Multimodal Interaction):
GEMINI, a cutting-edge model, stands at the forefront of
multimodal prompt engineering. By integrating information
from both textual and visual prompts, GEMINI aims to
produce coherent and contextually relevant responses across
different modalities. The study of GEMINI provides
valuable insights into the challenges and opportunities
presented by prompt engineering in scenarios requiring
interaction between diverse modes of input.
Each of these foundational models contributes uniquely to
the understanding and advancement of prompt engineering
in LLMs. Their architectures and pre-training approaches
offer valuable insights for researchers aiming to optimize
prompt-based interactions in various applications.
4.1 Working of LLMs
Large Language Models (LLMs) represent a transformative
leap in natural language processing, with their intricate
architectures enabling them to comprehend and generate
human-like text. A cornerstone in this domain is the
transformer architecture, widely adopted in models like
GPT (Generative Pre-trained Transformer). This
architecture relies on self-attention mechanisms, allowing
the model to weigh the significance of different words in a
sequence, fostering improved contextual understanding.
Pre-training is a crucial phase in the development of LLMs.
It involves training the model on massive corpora to learn
linguistic patterns and contextual relationships. This process
equips the model with a broad understanding of language,
enabling it to generate coherent text across various contexts.
GPT exemplifies this pre-training paradigm, leveraging a
diverse range of internet texts to acquire language
proficiency.
Fine-tuning is the subsequent step, tailoring the pre-trained
model for specific tasks. This phase involves exposing the
model to task-specific datasets, allowing it to adapt its
knowledge to particular domains. Researchers have
explored diverse fine-tuning strategies to enhance the
performance of LLMs in applications such as text
completion, summarization, and question-answering.
LLM server system architecture
4.2 System architecture and design
Upon comprehending the intricacies of Large Language
Models (LLMs) and their applications in contemporary
settings, exemplified by platforms like ChatGPT and Bard,
the significance of furnishing pertinent parameters within
the user prompt becomes evident. Essential components
such as context and related data play a pivotal role in
steering the model towards generating contextually relevant
and accurate responses. In our proposed solution, we
advocate for the incorporation of an additional layer in the
implementation pipeline. This layer is strategically designed
to refine the user's input, augmenting it with the necessary
metadata, before being transmitted to the underlying LLM.
[4] The objective is to enhance the contextual understanding
of the model by ensuring that vital information is explicitly
provided, thereby optimizing the prompt for more precise
4
and tailored responses. This augmented layer acts as a
crucial intermediary step, fine-tuning the user's input to
better align with the nuanced requirements of the LLM and,
consequently, fostering improved performance in various
application scenarios.
Addition of Prompt Engineering layer
The pipeline centers on an ensemble of parallel Prompt
Optimizer modules consisting of 1) Prompt Generator 2)
Performance Validators 3) Risk Validators 4) Active
Learning Controller. This enables both targeted optimization
for metrics like relevance as well as holistic safeguards
against potential harms.
The prompt generator contains a lattice-based sequencer for
creative exploration guided by the controller’s learned
reward patterns. Performance validators then score prompts
on dimensions like semantics, fluency, and accuracy, while
risk validators estimate the likelihood of toxicity, bias, and
deception. The controller aggregates validation signals into
overall reward signals, approving high-quality prompts for
downstream use while rejecting or regenerating harmful
ones. Active learning cycles continually expand dataset
diversity. Human-in-the-loop options are also available for
preference learning.
4.3 Prompt Optimization Approach
We employ a reinforced algorithm with Monte Carlo
rollouts for the Controller to learn superior prompt design
patterns, using interim scores from Validators as reward
signals. Cross-entropy method: hyperparameter tuning
prevents overfitting. Transfer learning from human
demonstrations augments policy learning.
We also introduce novel Hybrid Reward Smoothing
regularization, blending both short- and long-term reward
estimates to prevent instability while allowing for sharp
optimization.
More formally: Rt = α*R't + (1-α)*R ̃t
Where Rt is the reward at time t; R’t is the Monte Carlo
rollout estimate; R ̃t is the exponentially smoothed
long-term reward; and α is a tuned weighting coefficient.
This framework outperforms prior programmatic
optimization techniques for the breadth of language use
cases while maximizing safety.
4.4 Safety and Fairness Considerations
We implement proactive measures for risk reduction in
prompts, including
● Toxicity classifiers assess stereotyping,
microaggression, and politeness. [5]
● Entailment detectors identify logically unsupported
claims.
● Confidence estimators score uncertainty levels.
● Adversarial sanity checks with contrastive
candidates and boundary testing.
The documentation required for release also encompasses
fairness assessments like subgroup specificity analysis,
prompt diversity metrics, and intended failure mode
declarations. Structured vetting procedures prior to
deployment are mandated, along with monitoring
requirements.
These codify transparency best practices for ethical prompt
engineering as AI becomes further entwined with decision
systems. Restricting optimization solely for performance
risks overlooks complex harms. Our validators and
proactive mitigations address this while allowing rapid and
adaptable prompting.
5. Conclusion
The use of prompt engineering further refines LLM
behavior. Prompt engineering involves formulating inputs in
a manner that elicits desired outputs from the model.
Researchers have investigated various prompt engineering
techniques to improve response accuracy and relevance.
Understanding the inner workings of LLMs is crucial for
effective prompt engineering, as it allows researchers to
manipulate inputs strategically.
Despite their remarkable capabilities, LLMs are not devoid
of challenges. Non-determinism in model responses poses a
significant hurdle. Identical prompts may yield different
responses due to the model's inherent stochastic nature. This
introduces unpredictability, necessitating careful
consideration in applications where consistency is
paramount.
Interpreting LLM behavior involves delving into the
attention mechanisms within the transformer architecture.
Attention maps provide insights into which parts of the
input sequence the model focuses on during processing.
This interpretability aspect is vital for gaining transparency
5
into the decision-making process of LLMs, particularly in
critical domains such as healthcare and legal applications.
In conclusion, users need to follow a four-step approach
while thinking/formulating prompts:
1. Ensure that you are setting up as much context as possible
in the question
2. Ensure you are using correct spelling and grammar. (to
prevent hallucination)
3. Start small and gradually build up the conversation (chain
of reasoning)
4. Imagine you are talking to a human, not AI.
6. References
1. M. K. Pehlivanoğlu, M. A. Syakura and N. Duru,
"Enhancing Paraphrasing in Chatbots Through Prompt
Engineering: A Comparative Study on ChatGPT, Bing, and
Bard," 2023 8th International Conference on Computer
Science and Engineering (UBMK), Burdur, Turkiye, 2023,
pp. 432-437, doi: 10.1109/UBMK59864.2023.10286606.
2. Y. Feng et al., "PromptMagician: Interactive Prompt
Engineering for Text-to-Image Creation," in IEEE
Transactions on Visualization and Computer Graphics, doi:
10.1109/TVCG.2023.3327168.
3. L. Rai, C. Deng and F. Liu, "Developing Massive Open
Online Course Style Assessments using Generative AI
Tools," 2023 IEEE 6th International Conference on
Electronic Information and Communication Technology
(ICEICT), Qingdao, China, 2023, pp. 1292-1294, doi:
10.1109/ICEICT57916.2023.10244824.
4. M. Swaroop, D. Khashabi, C. Baral, Y. Choi, & H.
Hajishirzi, "Reframing instructional prompts to gptk's
language", 2021. https://doi.org/10.48550/arxiv.2109.07830.
5. J. Hernández-Orallo, "AI Safety Landscape From
short-term specific system engineering to long-term
artificial general intelligence," 2020 50th Annual IEEE/IFIP
International Conference on Dependable Systems and
Networks Workshops (DSN-W), Valencia, Spain, 2020, pp.
72-73, doi: 10.1109/DSN-W50199.2020.00023.
6. Beurer-Kellner, L., Fischer, M. L., & Vechev, M. (2023).
Prompting is programming: a query language for large
language models. Proceedings of the ACM on Programming
Languages, 7(PLDI), 1946-1969
https://doi.org/10.1145/3591300.
7. M. Kuzlu, Z. Xiao, S. Sarp, F. O. Catak, N. Gurler and O.
Guler, "The Rise of Generative Artificial Intelligence in
Healthcare," 2023 12th Mediterranean Conference on
Embedded Computing (MECO), Budva, Montenegro, 2023,
pp. 1-4, doi: 10.1109/MECO58584.2023.10155107.
8. J. Mange, "Effect of Training Data Order for Machine
Learning," 2019 International Conference on Computational
Science and Computational Intelligence (CSCI), Las Vegas,
NV, USA, 2019, pp. 406-407, doi:
10.1109/CSCI49370.2019.00078.
9. K. K. Phokela, S. Sikand, K. Singi, K. Dey, V. S. Sharma
and V. Kaulgud, "Smart Prompt Advisor: Multi-Objective
Prompt Framework for Consistency and Best Practices,"
2023 38th IEEE/ACM International Conference on
Automated Software Engineering (ASE), Luxembourg,
Luxembourg, 2023, pp. 1846-1848, doi:
10.1109/ASE56229.2023.00019.
10. Yang, Z. G. and Ligeti-Nagy, N. (2023). Improve
performance of fine-tuning language
models with prompting. Infocommunications Journal,
15(Special Issue), 62-68.
https://doi.org/10.36244/icj.2023.5.10
11. Y. Miao, S. Li, J. Tang and T. Wang, "MuDPT:
Multi-modal Deep-symphysis Prompt Tuning for Large
Pre-trained Vision-Language Models," 2023 IEEE
International Conference on Multimedia and Expo (ICME),
Brisbane, Australia, 2023, pp. 25-30, doi:
10.1109/ICME55011.2023.00013.