Professional Documents
Culture Documents
April 5, 2024
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 1/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
In this blog, we'll talk about how to create your own version
of Devin or something similar. I'll break down the blog into
very fundamental concepts as follows:
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 2/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Task Object
OPEN_STATE = 'open'
COMPLETED_STATE = 'completed'
ABANDONED_STATE = 'abandoned'
IN_PROGRESS_STATE = 'in_progress'
VERIFIED_STATE = 'verified'
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 3/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
class Task:
id: str
goal: str
parent: "Task | None"
subtasks: List["Task"]
Args:
parent: The parent task, or None if it
is the root task.
goal: The goal of the task.
state: The initial state of the task.
subtasks: A list of subtasks associated
with this task.
"""
if parent is None:
self.id = '0'
else:
self.id = parent.id + '.' +
str(len(parent.subtasks))
self.parent = parent
self.goal = goal
self.subtasks = []
for subtask in (subtasks or []):
if isinstance(subtask, Task):
self.subtasks.append(subtask)
else:
goal = subtask.get('goal')
state = subtask.get('state')
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 4/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
subtasks = subtask.get('subtasks')
self.subtasks.append(Task(self,
goal, state, subtasks))
self.state = OPEN_STATE
Args:
indent: The indentation string for
formatting the output.
Returns:
A string representation of the task and
its subtasks.
"""
emoji = ''
✅
if self.state == VERIFIED_STATE:
emoji = ' '
🟢
elif self.state == COMPLETED_STATE:
emoji = ' '
❌
elif self.state == ABANDONED_STATE:
emoji = ' '
💪
elif self.state == IN_PROGRESS_STATE:
emoji = ' '
🔵
elif self.state == OPEN_STATE:
emoji = ' '
result = indent + emoji + ' ' + self.id + '
' + self.goal + '\n'
for subtask in self.subtasks:
result += subtask.to_string(indent + '
')
return result
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 5/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
def to_dict(self):
"""Returns a dictionary representation of
the task.
Returns:
A dictionary containing the task's
attributes.
"""
return {
'id': self.id,
'goal': self.goal,
'state': self.state,
'subtasks': [t.to_dict() for t in
self.subtasks]
}
Raises:
ValueError: If the provided state is
invalid.
"""
if state not in STATES:
raise ValueError('Invalid state:' +
state)
self.state = state
if state == COMPLETED_STATE or state ==
ABANDONED_STATE or state == VERIFIED_STATE:
for subtask in self.subtasks:
if subtask.state != ABANDONED_STATE:
subtask.set_state(state)
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 6/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Returns:
The current task in progress, or None if
no task is in progress.
"""
for subtask in self.subtasks:
if subtask.state == IN_PROGRESS_STATE:
return subtask.get_current_task()
if self.state == IN_PROGRESS_STATE:
return self
return None
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 7/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Plan Object
class Plan:
"""Represents a plan consisting of tasks.
Attributes:
main_goal: The main goal of the plan.
task: The root task of the plan.
"""
main_goal: str
task: Task
Args:
task: The main goal of the plan.
"""
self.main_goal = task
self.task = Task(parent=None, goal=task,
subtasks=[])
def __str__(self):
"""Returns a string representation of the
plan.
Returns:
A string representation of the plan.
"""
return self.task.to_string()
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 8/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Args:
id: The ID of the task.
Returns:
The task with the specified ID.
Raises:
ValueError: If the provided task ID is
invalid or does not exist.
"""
try:
parts = [int(p) for p in id.split('.')]
except ValueError:
raise ValueError('Invalid task id, non-
integer:' + id)
if parts[0] != 0:
raise ValueError('Invalid task id, must
start with 0:' + id)
parts = parts[1:]
task = self.task
for part in parts:
if part >= len(task.subtasks):
raise ValueError('Task does not
exist:' + id)
task = task.subtasks[part]
return task
Args:
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 9/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Args:
id: The ID of the subtask.
state: The new state of the subtask.
"""
task = self.get_task_by_id(id)
task.set_state(state)
def get_current_task(self):
"""Retrieves the current task in progress.
Returns:
The current task in progress, or None if
no task is in progress.
"""
return self.task.get_current_task()
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 11/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Now the next step for us is to define actions for the LLM to
take. So, a coding agent can have many possible actions.
Some of them can be executable and some non-executable.
Actions like "think", "add_task", "modify_task" are non-
executable while actions like "read_file", "write_file",
"run_code" are executable. We also need to return
observations after each running each action.
@dataclass
class Action:
def run(self) -> "Observation":
raise NotImplementedError
def to_dict(self):
d = asdict(self)
try:
v = d.pop('action')
except KeyError:
raise NotImplementedError(f'{self=} does
not have action attribute set')
return {'action': v, "args": d, "message":
self.message}
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 12/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
@property
def executable(self) -> bool:
raise NotImplementedError
@property
def message(self) -> str:
raise NotImplementedError
@dataclass
class Observation:
"""
This data class represents an observation of the
environment.
"""
content: str
@property
def message(self) -> str:
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 13/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
@dataclass
class ExecutableAction(Action):
@property
def executable(self) -> bool:
return True
@dataclass
class FileWriteAction(ExecutableAction):
path: str
content: str
action: str = "write"
@property
def message(self) -> str:
return f"Writing file: {self.path}"
@dataclass
class FileWriteObservation(Observation):
"""
This data class represents a file write
operation
"""
path: str
observation : str = "write"
@property
def message(self) -> str:
return f"I wrote to the file {self.path}."
@dataclass
class NullAction(NotExecutableAction):
"""An action that does nothing.
This is used when the agent need to receive user
follow-up messages from the frontend.
"""
action: str = "null"
@property
def message(self) -> str:
return "No action"
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 15/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
@dataclass
class NullObservation(Observation):
"""
This data class represents a null observation.
This is used when the produced action is NOT
executable.
"""
observation : str = "null"
@property
def message(self) -> str:
return ""
@dataclass
class AgentThinkAction(NotExecutableAction):
thought: str
action: str = "think"
@property
def message(self) -> str:
return self.thought
@dataclass
class AgentFinishAction(NotExecutableAction):
action: str = "finish"
@property
def message(self) -> str:
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 16/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Memory
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 17/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
@dataclass
class State:
plan: Plan
iteration: int = 0
history: List[Tuple[Action, Observation]] =
field(default_factory=list)
updated_info: List[Tuple[Action, Observation]] =
field(default_factory=list)
LLM Agent
import requests
import os
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 18/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
class LLM:
def __init__(self):
self.client = OpenAI()
self.url =
"https://api.openai.com/v1/chat/completions"
max_tokens=kwargs.get('max_new_tokens', 1000),
temperature=kwargs.get('temperature', 0)
)
response =
result.choices[0].message.content
except Exception as e:
print(str(e))
return
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 19/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
return response
prompt = """
# Task
You're an AI blogger. You can't see, draw, or
interact with a
browser, but you can write files, and you can think.
%(task)s
## Plan
As you complete this task, you're building a plan
and keeping
track of your progress. Here's a JSON representation
of your plan:
```json
%(plan)s
```
%(plan_status)s
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 20/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
If, and only if, ALL tasks have already been marked
verified,
you MUST respond with the `finish` action.
## History
Here is a recent history of actions you've taken in
service of this plan,
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 21/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
## Action
What is your next thought or action? Your response
must be in JSON format.
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 22/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
%(hint)s
"""
history_dicts.append(observation.to_dict())
history_str = json.dumps(history_dicts,
indent=2)
hint = ""
current_task = plan.get_current_task()
if current_task is not None:
plan_status = f"You're currently working on
this task:\n{current_task.goal}."
if len(current_task.subtasks) == 0:
plan_status += "\nIf it's not achievable
AND verifiable with a SINGLE action, you MUST break
it down into subtasks NOW."
else:
plan_status = "You're not currently working
on any tasks. Your next action MUST be to mark a
task as in_progress."
hint = plan_status
latest_action_id = latest_action.to_dict()
['action']
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 24/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
return prompt % {
'task': plan.main_goal,
'plan': plan_str,
'history': history_str,
'hint': hint,
'plan_status': plan_status,
}
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 25/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
actions = (
FileWriteAction,
AgentThinkAction,
AddTaskAction,
ModifyTaskAction,
)
ACTION_TYPE_TO_CLASS =
{action_class.action:action_class for action_class
in actions}
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 26/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
action_dict = json.loads(response)
if 'contents' in action_dict:
action_dict['content'] =
action_dict.pop('contents')
action = action_from_dict(action_dict)
return action
Alright! So, we have all the building blocks for running this
agent now. Let's write a function to make this Agent work
end2end.
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 27/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
def update_state_for_step(i):
state.iteration = i
def update_state_after_step():
state.updated_info = []
update_state_for_step(i)
action: Action = NullAction()
observation: Observation = NullObservation("")
try:
pr = get_prompt(state.plan, state.history)
action_res = llm.generate(pr)
action = parse_response(action_res)
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 28/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
if action is None:
raise ValueError("Agent must return an
action")
print(f"ACTION: {action}")
except Exception as e:
print(f"ERROR: {str(e)}")
return True
update_state_after_step()
if isinstance(action, AddTaskAction):
try:
state.plan.add_subtask(action.parent,
action.goal, action.subtasks)
except Exception as e:
print(f"ERROR: {str(e)}")
return True
elif isinstance(action, ModifyTaskAction):
try:
state.plan.set_subtask_state(action.id,
action.state)
except Exception as e:
print(f"ERROR: {str(e)}")
return True
if action.executable:
try:
observation = action.run()
except Exception as e:
print(f"ERROR: {str(e)}")
return True
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 29/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
add_history(action, observation)
max_iterations = 30
llm = LLM()
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 30/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
References
https://github.com/OpenDevin/OpenDevin
https://openai.com/
https://www.cognition-labs.com/introducing-devin
Report this
Published by
Praphul Singh
Data Science @Oracle | IIT Kanpur 6 articles Following
Published • 1w
✨
Happy to share my new blog post about creating your very own Autonomous LLM
Agent like Devin Learn step-by-step how to build an Autonomous Large
Language Model (LLM) agent - it's more than just a text generator! Discover how to
make an assistant like Devin, who can not only chat, but write codes for you end2end,
run them and more! #LLM #GPT4 #Devin #OpenDevin #LargeActionModels
#GenerativeAI
Reactions
+16
2 Comments
Most relevant
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 31/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
Add a comment…
Good one 🙌🏻
Like · 1 Reply · 1 Reply
Thanks Aman! 😀
Like Reply
Praphul Singh
Data Science @Oracle | IIT Kanpur
Following
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 32/33
4/16/24, 9:54 PM (25) Building your own Autonomous LLM Agents | LinkedIn
https://www.linkedin.com/pulse/building-your-own-autonomous-llm-agents-praphul-singh-uompc/ 33/33