2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI

The Possibility of Now –

Learning, Teaching and Developing Learning Tools with Gen AI
Dr Aneesha Bakharia
Senior Lecturer
School of Electrical Engineering and Computer Science (EECS)
The University of Queensland
Australia

Twitter: @aneesha

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 1

 A new human AI interface
• OpenAI ChatGPT was released on 30 Nov 2022

A prompt is a piece of text that triggers

a specific response or action from an LLM

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 2

Rarely has a technology possessed such profound
potential to both revolutionize and challenge the
boundaries of education
• Answer questions • Provide direct answers across
multiple domains
• Feedback on writing
• Write essays for you
• Feedback on programming
• Write programming code for you
• Brainstorm new ideas
• Personalised Chatbot tutors

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 3

Initial Flaws Identified by Educators
• Not even a year old and already rapid progress…

Can’t take Can’t do

Can’t reflect
feedback math

The text will

I’ll be able I'll include
have false
to detect it images
facts

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 4

Feedback Processes and Evaluative Judgement
Initial Prompt

Incorrect Format

• My initial
thoughts on
Feedback & Instructions higher level
. thinking skills
.
.
Usable Output

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 5

Prompting has changed and is changing…
and will further impact Education
• Reusable Prompt templates
• Persona’s are important
(e.g. Act as a …)
• Prompt techniques
(e.g. Chain of Thought, Chain of Density, etc)
• Programmatically access via an API

• This are having a huge transformative

impact across all industries
Especially on Education:
Teaching, Learning and Research
https://github.com/f/awesome-chatgpt-prompts

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 6

 Possibility of Now

• Empowered Educators • Very simplistic prompts required to get

sophisticated results
• Empowered Learners
• Direct answers given to learners
• Out of domain creators
(No development of problem solving
• Tools to scaffold learning and critical thinking skills)
(via personalized Chatbot tutors and other
• Prompt Injection and lack of safeguards
custom tools e.g. writing, coding, etc)
• Hallucination and Bias
• Addressing the “Clicks to Construct” issue
(Ability to analyse the learning process) • Paid Services (lack of equity)

RESEARCH
2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 7
 Possibility of Now

Empowered Educators
(and Learning Designers)

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 8

How are Educators using ChatGPT?

Communication with Parents Lesson and Unit Planning Classroom Management and
Compose a message to a parent Develop a 1-week unit plan for Environment
requesting a meeting to discuss their teaching high school students about Outline 3 techniques for managing
child’s disruptive behaviour. the French Revolution. classroom noise levels.

Assessment and Feedback Professional Development Teaching Strategies and

Write feedback for a student who and Collaboration Techniques
struggled with a persuasive essay Describe 5 professional development Provide 5 strategies for fostering a
assignment. goals for a new teacher. growth mindset in your students.

Subject-Specific Activities Gamification and Interactive Integrating Subject Areas and

and Projects Learning Co-Teaching
Suggest 3 collaborative projects for Create a list of badges students can Advice on how to incorporate physical
high school students studying earn for a unit on constructing circles. education into lessons.
environmental science.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 9

Teaching
Example:
Create a
WORKSHEET
on AREA with
solutions
• Persona
• Specific task with a topic and
output quantity
• Audience
• Format/Structure of output
• Include a worked example
(if just the structure does not
work)
• Use the words “think in
steps” or
”think step by step”

Text version of prompt:

https://raw.githubusercontent.com/aneesha/prompt_engineering/main/mathworksheet_solutions.md

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 10

Course Content Generation

https://nolej.io/
2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 11
Eduweaver: Content in LiaScript

https://github.com/aneesha/eduweaver https://liascript.github.io/

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 12

Eduweaver: Prompts
Prompt: Generate TOC
Please act as a subject matter expert on the topic of {topic} that also
has learning designer skills and knowledge of markdown and json.
You know how to design online modules. You can break up the
content into chunks, explain concepts so that can easily be
understood, write quizzes to test knowledge and write creative
learning activities for students.
Could you please come up with {no_sub_topics} sections for an
online module on the topic of {topic} for {audience} in json format?
{sub_topics_to_include_text}
{sub_topics_to_not_include_text}
Here is an example of the format you need to return:
[{{"section_name": "The name of the section 1",
"section_description": "The descriptions of the section 1"}}]
Please don't return any intro text before or after the json.
https://github.com/aneesha/eduweaver
2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI
Prompt: Generate Content
Could you please write the content for {section_name} that will be
understood by {audience}?
The description of the topic that you must write is {section_description}.
{teaching_method_to_include_text}
{sub_topics_to_not_include_text}
Please don't output the section name but do include an introductory
paragraph.
Prompt: Generate Quiz: Generate Quizzes
Could you please write quiz questions for the content in {section_name}
that will be appropriate for {audience}?
The description of the topic that you must write quiz questions for is
{section_description}.
Please write {no_quiz_questions_in_sub_topic} questions in the json
format that is given in the example below.
[{{"question": "Question 1","question_type": "single_option","options":
[{{"option_name": "Option 1", "correct": "false"}},
{{"option_name": "Option 2", "correct": "true"}}]
}}]
The incorrect answers should not be obvious or easy.
13
 A reminder to have humans in the loop

Proceedings of the Workshop on International Conference of Artificial Intelligence in Education co-located with The 24th International Conference on
Artificial Intelligence in Education (AIED 2023) https://ceur-ws.org/Vol-3491/

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 14

 Possibility of Now

Empowered Learners

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 15

ChatGPT as a Tutor

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 16

Prompting Approaches for Students

Mollick, E., & Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts. arXiv preprint arXiv:2306.10052.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 17

Prompting Approaches for Students

Mike Sharples - https://www.slideshare.net/sharplem/generative-ai-for-teaching-learning-and-assessment

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 18

 Possibility of Now

Out of Domain Creators

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 19

Teaching Example:
Creating interactive
learning activities -
Glossary
• Persona
• Task Description
• If an application or tool,
describe what it needs to do
in steps and know domain
terminology for where it will
be run (eg on a web page or
a desktop application).

• Audience
• Output structure

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 20

 Creating
Teaching interactive
Example:
learning
Creating activities:
interactive learning
Glossary
activities - Glossary

• Single HTML file that can be

pasted into an LMS
• Wrote the interactive to the
specification
• Your role is to embed in an LMS
and test

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 21

 Possibility of Now

Tools to Scaffold Learning

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 22

API Access and the System Prompt
• Use a persona and create your own chatbot

Try at https://platform.openai.com/playground?mode=chat
Text version of prompt :
https://raw.githubusercontent.com/aneesha/prompt_engineering/main/grammar_tutor.md
2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 23
Educational Chatbots

class.com khanmigo

Quizlet – Q-Chat Duo Lingo – MAX

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 24

 LamPost - Email Writing Assistant for
Adults with Dyslexia

Goodman, S. M., Buehler, E., Clary, P., Coenen, A., Donsbach, A., Horne, T. N., ... & Morris, M. R. (2022, October). Lampost: Design and
evaluation of an ai-assisted email writing prototype for adults with dyslexia. In Proceedings of the 24th International ACM SIGACCESS
Conference on Computers and Accessibility (pp. 1-18).

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 25

 Metaphorian – Metaphor Creation for
Science Writing

Kim, J., Suh, S., Chilton, L. B., & Xia, H. (2023, July). Metaphorian: Leveraging Large Language Models to Support Extended Metaphor
Creation for Science Writing. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (pp. 115-135).

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 26

 Spellburst - coding with ui driven prompts

Angert, T., Suzara, M. I., Han, J., Pondoc, C. L., & Subramonyam, H. (2023). Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language
Prompts. arXiv preprint arXiv:2308.03921.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 27

AUTOGPT
• Multiple versions of ChatGPT
write prompts to meet a goal
or perform a task
• The user only needs to give
a high-level task and/or goals

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 28

 LLM-based Autonomous Agents

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., ... & Wen, J. R. (2023). A survey on large language model based
autonomous agents. arXiv preprint arXiv:2308.11432.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 29

 MetaGPT

Hong, S., Zheng, X., Chen, J., Cheng, Y., Zhang, C., Wang, Z., ... & Wu, C. (2023). Metagpt: Meta programming for multi-agent collaborative
framework. arXiv preprint arXiv:2308.00352.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 30

Can we develop Learning Tools with
Agents?
• Integrations with an LMS are expensive to create and maintain
• Creating LTI compliant learning tools are expensive to design
and maintain

• Current Masters Research Thesis Project (Mr Junhao Li)

• AutoLTI
• Can we use concepts from ChatDEV and MetaGPT to develop LTI
tools?
• Educators enter a specification and review the generated tool.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 31

 Possibility of Now

Impact of Improved Prompting

Techniques

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 32

Many Prompting Techniques
Freely Available Templates

• How do you think “Chain of Density”

impacts the analysis of a ChatGPT
transcript?

https://twitter.com/moritzkremb/status/1712099649262358789

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 33

Many Prompting Techniques
Freely Available Templates
• How do you think
“Chain of Density”
impacts the analysis of a
ChatGPT transcript?

https://twitter.com/moritzkremb/status/1712099649262358789

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 34

https://twitter.com/moritzkremb/status/1713902866497167366

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 35

 https://twitter.com/Damn_coder/status/1710920561143250988
2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 36
 Promptbreeder

Fernando, C., Banarse, D., Michalewski, H., Osindero, S., & Rocktäschel, T. (2023). Promptbreeder: Self-Referential Self-Improvement
Via Prompt Evolution. arXiv preprint arXiv:2309.16797.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 37

Prompt Optimisation

Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., & Chen, X. (2023). Large language models as optimizers. arXiv preprint
arXiv:2309.03409.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 38

Multimodal Capabilities

https://twitter.com/GregKamradt/status/1711772496159252981

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 39

 Multimodal Capabilities

Image from : https://en.m.wikipedia.org/wiki/File:Simple_photosynthesis_overview.PNG

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 40

Generatavism

Pratschke, B. M. (2023). Generativism: the new hybrid. arXiv preprint arXiv:2309.12468.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 41

 Possibility of Now

• Empowered Educators • Very simplistic prompts required to get

sophisticated results
• Empowered Learners
• Direct answers given to learners
• Out of domain creators
(No development of problem solving
• Tools to scaffold learning and critical thinking skills)
(via personalized Chatbot tutors and other
• Prompt Injection and lack of safeguards
custom tools e.g. writing, coding, etc)
• Hallucination and Bias
• Addressing the “Clicks to Construct” issue
(Ability to analyse the learning process) • Paid Services (lack of equity)

RESEARCH
2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 42
LLM’s vs “Clicks to Construct” Dilemma
• Challenge faced by in
translating raw data, like clicks,
into meaningful constructs that
reflect the learning process.
• This issue often led to a failure
in delivering high-level insights
or understanding the learning
process.
Knight, S., & Shum, S. B. (2017). Theory and learning Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., ... & Gašević, D.
analytics. Handbook of learning analytics, 17-22. (2023). Practical and ethical challenges of large language models in education: A
systematic literature review. British Journal of Educational Technology.

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 43

An Open Call to Researchers
• We are building on a rapidly changing landscape!
• How do we evaluate LLM based products for learning?
• How do we design LLM based products to enhance learning, problem solving and
higher level thinking given the changing nature of prompting?
• How do we track learner interactions with LLM based products so that we can use
LLM’s to classify to higher-level learning constructs?
• We need educators and tutors in the loop, making decisions with GenAI and not just
automated classification!
• How do we ensure safeguards, address bias and hallucinations?
• How do we ensure equity?
We need to leverage open source LLM’s eg Meta’s Llama and have the ability to train
and finetune our own models.
We can’t rely on blackbox models.
• How can this research agenda be funded?

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 44

 Questions

Thank You

2023 GLOBAL ONLINE CONFERENCE | EMPOWERING LEARNERS FOR THE AGE OF AI 45