The Report is Generated by DrillBit Plagiarism Detection Software

Submission Information

Author Name Arjun

Title plag
Paper/Submission ID 515740
Submission Date 2022-05-10 23:08:04
Total Pages 64
Document type Article

Result Information

Similarity 8%
1 10 20 30 40 50 60 70 80 90

Sources Type Report Content

Student
Paper
0.14%
Internet
2.11%

Journal/
Publicatio
n 3.39% Words <
14,
4.75%

Exclude Information

Quotes Not Excluded

References/Bibliography Not Excluded
Sources: Less than 14 Words Similarity Not Excluded
Excluded Source 0%
Excluded Phrases Not Excluded

A Unique QR Code use to View/Download/Share Pdf File

DrillBit Similarity Report

A-Satisfactory (0-10%)
B-Upgrade (11-40%)

8 41 A C-Poor (41-60%)
D-Unacceptable (61-100%)
SIMILARITY % MATCHED SOURCES GRADE

LOCATION MATCHED DOMAIN % SOURCE TYPE

1 en.wikipedia.org Internet Data

2

2 en.wikipedia.org Internet Data

1

3 ijera.com Publication
<1

4 Osteoarthritis, obesity and weight loss evidence, hypotheses and horizons Publication
<1
- a s by Bliddal-2014

5 www.mdpi.com Publication
<1

6 Security of Intelligent Building Network Based on Wireless Sensor Publication

<1
Network by Xu-2018

7 Phenotype as Developmental Bridge Whither Nature and Nurture by Publication

<1
Revie-2005

8 Open Source Digital Camera on Field Programmable Gate Arrays by Publication

<1
Ababei-2016

9 Effectiveness of Social Welfare Programmes in East Asia A Case Study Publication

<1
of Taiwan by Chien-Chun-2011

10 Subtext and Countertext in Muriels Wedding by Mackey-2001 Publication

<1

11 Deadly Liver Mob opening the door improving sexual health pathways Publication
<1
for Aborig by Biggs-2016
12 An Embodied Approach to the Study of Media Forms Introducing a Publication
<1
Social Scientifi by Holbert-2004

13 A New Semisupervised-Entropy Framework of Hyperspectral Image Publication

<1
Classification Bas by Sun-2018

14 A Textbook of Advanced Oral and Maxillofacial Surgery Volume 3 Publication

<1
State-of-the-A

15 The midwife as doula A guide to mothering the mother by Dan-1981 Publication

<1

16 Special issue on the Ethnography of Healing Genealogies of Affliction Publication

<1
Aetiol by Zo-2007

17 Pairs trading strategy optimization using the reinforcement learning Publication

<1
method a c by Fallahpour-2016

18 Optimal Corridor Selection for a Road Space Management Strategy Student Paper
<1
Methodology and Tool

19 Finite-Time Average Consensus Based Approach for Distributed Convex Publication

<1
Op by Ma-2018

20 downloads.hindawi.com Publication
<1

21 Assessment of railway vibrations using an efficient scoping model by Publication

<1
Connolly-2014

22 Application of a Chaotic Quantum Bee Colony and Support Vector Publication

<1
Regression to Mul by Gao-2020

23 www.scribd.com Internet Data

<1

24 From privacy to anti-discrimination in times of machine learning by Publication

<1
Hagendorff-2019

25 A Generalized Theory of Sorption Phenomena in Biological Materials Publication

<1
Part II - Ad by P-1972
26 XXVII Wandering Home to Manhattan 1867 (LETTERS 1674 TO 1750) Publication
<1
by -2020

27 www.radioeng.cz Publication
<1

28 Strategic transformations in the media by Oliver-2018 Publication

<1

29 Research on AGC Performance During Wind Power Ramping Based on Publication

<1
Deep Reinforcemen by Zhang-2020

30 Research Article Published by 'Jahng Bullen' - www.eurodl.org Publication

<1

31 Quantum game of Go could baffle even an AI by Crane-2020 Publication

<1

32 Present Status and Future Perspectives of Heart Transplantation by Publication

<1
Toyoda-2013

33 PALATIN a platform for interactive algorithms Student Paper

<1

34 Marine Surveying in Britain during the Seventeenth and Eighteenth Publication

<1
Centuries by A-1957

35 Insight into consumer experience on UK train transportation services by Publication

<1
Mogaji-2019

36 How Low Can You Go Reducing Rates of Hypoglycemia in the Non- Publication
<1
critical Care Hosp by Kulasa-2017

37 Discriminative and Robust Competitive Code for Palmprint Recognition Publication

<1
by Xu-2018

38 Deep learning from chemoinformatics to precision medicine by Kim- Publication

<1
2017

39 A 32-electrode data collection system for electrical impedance Publication

<1
tomography by Koukourlis-1995

40 www.oapen.org Publication
<1
41 PDF File Data pdf.usaid.gov Internet Data
<1
 Abstract

As our topic says we are trying to develop a self-learning/ self-aware/

self-evolving Artificial intelligence. It is under development right now. With the existing
technology this specific task is a challenge, but it is doable. Basically, what we are doing in
this project is creating a virtual species that much like humans is aware of its surroundings
and doings. This can be achieved by firstly making the basic Environment, i.e., the
22
simulation those entities would be in, and then proceeding with the entities themselves. For
the environment there is still an internal debate about what engine would be best to run this
simulation, but it’s been narrowed down to either unity or unreal engine 4/5(depending on
when the project is finished, and which version has more support). Now let’s see in detail
each one’s pros cons and details.

12
Pose estimation is a computer vision technique for predicting and tracking the location of a
17
person or an item. This is done by looking at a person's or an object's stance and orientation
together. Model analysis, image processing, digital signal processing, artificial intelligence,
and other areas fall under this category.

In high-tech domains including word processing, office automation, machine translation, and
13
real-time surveillance systems, it has great application value and theoretical relevance. As a
result, computer vision in natural settings has taken on a whole new meaning. Text
identification and detection algorithms were formerly mostly reliant on artificially built
functions and traditional image processing techniques.

Advances in computer vision such as picture distribution, material recognition, and semantic
separation have been made in recent years, thanks to the rapid development of increasingly
powerful learning algorithms.

A data-driven algorithm is the detailed learning algorithm. In-depth learning algorithms

7
automatically discover and understand the rules for characteristics hidden in vast volumes of
data through repetitive learning without requiring significant human intervention, resulting
 in superior summaries than typical image processing algorithms.

4
As a result, researchers seeking to train advanced AI/ML models are frequently limited by both
the quality and quantity of data available. So, if you want to demonstrate that a vehicle can
drive itself, you'll need a lot of kilometres of human driving data. Researchers, on the other
20
hand, may construct a large number of training data sets and come up with creative models
using simulation, exactly as humans do in their brains.

INTRODUCTION

In today’s world, AI is gradually being used for more technological purposes that can
15
assist us humans for almost all our daily needs, be it automation companies like Tesla
which use AI to store user data to assist their customers in their daily needs or various
10
other companies which follow the trend.
It was this very scenario that led to the idea of creating an environment where we teach
and train an AI to understand and make use of various situations around itself. We will
be utilizing the power of strong and complex Deep Learning models, and it will be
12
trained on a decently big dataset of various staging environments to make predictions
about how the species would react for different situations.
8
The problem statement involves researching, studying, trying and selecting the most
optimal way To create a virtual species which is self aware and can sustain under various
environments and situations.

Despite deep learning's enormous contributions to artificial intelligence, there's a

major flaw: It necessitates massive volumes of data. Both proponents and opponents
of deep learning agree on this point. Deep learning didn't become a popular AI
technique until recently, owing to a dearth of suitable data and processing power to
handle it. One of AI researchers' key aims is to lessen deep learning's reliance on
data.

Supervised deep learning can help with computer vision and various aspects of
natural language processing, for example. Deep learning is becoming increasingly
crucial when it comes to sensitive applications like cancer detection. Other
applications, including analyzing vast amounts of content posted on social
networking sites every day, are particularly beneficial (with some caveats).
30
According to academics, deep reinforcement learning has yielded outstanding
outcomes in games and simulations. In recent years, reinforcement learning has
mastered a variety of games previously thought to be beyond the reach of artificial
intelligence. AI programmes have already defeated human world champions in
11
StarCraft 2, Dota, and the ancient Chinese board game Go.

It's worth noting, though, that AI programmes learn to solve issues in a fundamentally
different way than humans do. This is due to the fact that the reinforcement learning
agent starts with a blank slate and is limited in its capabilities. Finally, the AI is left to
16
learn on its own by trial and error and how to get the most out of it.

When the issue area is simple and you have adequate processing power to execute a
number of trial-and-error sessions, this model has a good chance of working.
Usually, reinforcement learning agents need a ridiculously large number of sessions
31
to master a single game. Reinforcement learning research has been restricted to
wealthy tech companies' research labs due to the high costs.

This is primarily executed with the aid of a Machine Learning model, more specifically a
sub-field of ML called Deep Learning, which uses many more hidden layers and requires
a lot more resources than traditional Machine Learning.

The way a Deep Learning model is trained is via Supervised Learning i.e. using hundreds
and thousands of sample images with their truth labels, categorized into that we want to
make predictions on ultimately and then feeding all these images to the model and using
an optimization algorithm like Gradient Descent, Adam etc. to update the model’s
parameters in this way that it captures the input to output mapping and when new inputs
start coming in the future, the model applies the knowledge that it has gained from past
examples to make predictions.

One of the notable and renowned Deep Learning techniques that is used for the task of
image classification is something called Convolutional Neural Networks. CNNs are a
form of Neural Network created specifically for capturing and processing pictures and
38
photographs. It contains kernels that are a square matrix of learnable parameters which are
convolved with the current image to move the network forward.

Objective:

To create a virtual species that much like Humans are aware of its surroundings and
doings. We proceed by first making the basic Environment, i.e., the simulation those
22
entities would be in, and then proceeding with the entities themselves. For the
environment we would be utilizing the unity or unreal engine 4/5.

Project Scope

24
To create a virtual species that is aware of its surroundings and actions in the same way that
humans are. We begin by creating the basic Environment, which is the simulation that those
entities will be in, before moving on to the entities themselves. We would use Unity or Unreal
Engine 4/5 to create the environment.

A huge amount of data is required when the aim is to design a network capable of classifying
34
between all the categories, that too with state-of-the-art accuracy. Not only that, since the model
is also to be deployed, research also needs to be done about how to most efficiently design the
model so as to minimize the size and inference time (i.e., the time that it takes to make a
prediction given a new input).

Feasibility Study
The most crucial phase in every project is establishing whether or not the concept is viable. A
Feasibility Study is what it's called. The feasibility study is used to evaluate a proposed project's
21 33
inadequacies and prospective capabilities, as well as to recommend activities that will assist the
41 5
project realize its goals. The type and content of viable studies are largely determined by the
locations where the initiatives in question are carried out. The ability of a concept to succeed or
fail is referred to as "feasibility."
It recognises whether the product/service under development has a realistic market cap, what the
investment requirements are and where to get the funds, whether and where the technical
know-how to turn the idea into a physical product is accessible, and so on. It's a strategy for
making good judgments and achieving goals.

Technical Feasibility

9
The most critical pillar for a project's future success is technical feasibility. A correct and
comprehensive analysis of the programme and project that will be built determines whether the
project's future remains stable or if it is a project with a limited scope, just as a firm foundation
invariably leads to tall skyscrapers.

A true developer and his or her team will become technically sound before arranging and
25
working with software. Rather than wasting time tinkering with the issues that lead to a project's
14 32
failure, a quick review of the project's technical aspects, life cycle, and any other relevant
components is far more productive.

14
This not only helps with the technical aspects of a project, but it also acts as a link between
departments, encouraging and motivating them to contribute their time and effort to the project's
other tasks.

20
Python also has the benefit of being cross-platform and having a large number of extended
libraries that appear to do a better job than others. Furthermore, the library in question is simple
to set up and has a number of built-in features that have aided the project's constructive learning
and development.

Due to the limited workforce, development is both cost-effective and technologically advanced,
as it is based on Python's most recent technology, which allows us to create machine learning
models and 'Convolutional Neural Networks (CNN)'. The proposal can be delivered as a
technically sound and implementable project with the right hardware and software.

Economic Feasibility

If the predicted benefits equal or surpass the expected expenses, a system is called economically
7
feasible. The cost-benefit analysis method is used to weigh the predicted costs and benefits of a
project to determine its economic feasibility.

9
Economic analysis is used to analyze the cost and effectiveness of the proposed system. The
most significant part of economic feasibility is the cost-benefit analysis. It is, as the name
suggests, an examination of the system's costs and advantages. Developing an IT application is a
long-term endeavor.

8
The application has generated revenue for the company since its inception. Profits can come in
the form of money or a better working environment. It does, however, come with dangers, as
11
estimations can be off, and the project may not be lucrative. The cost-benefit analysis aids
management in identifying a project's worth, rewards, and dangers.

The ability to use a system after it has been created and installed is referred to as "operational
18
feasibility." Will there be any user resistance, which could have an impact on the app's potential
benefits? When it is integrated into daily routine, it is quite yielding and will be readily accepted
by individuals and others.

Gantt Chart
Tools Used

19
Python: Python was used in the project as a scripting language as well as the development
15 3
language for creating the web application hosted on the internet. The reason Python was used
for the Machine Learning part of the project was because it is easy to use, has a lot of
23
scientific and advanced statistics and mathematical libraries, Machine Learning tools and
frameworks along with a very active and supportive community of developers worldwide.

3
Python was used for training the models as well as scraping information about the
various environments and constructing the database using the MediaWiki API. Python
was used due to it being easy to read, dynamically-typed and the vast number of libraries
present,which help developers focus on what matters the most instead of boilerplate
code.

Also, when we are working on something related to Machine Learning and/or Big Data, we
2
need to iterate on various experiments quickly and thus, need a language which can be used to
develop prototype solutions relatively quickly and easily, the exact facility which is provided
by Python.

Apart from this, Python was also used as the development language for the web application
17
because of Streamlit. It is a python library that allows easy development and maintenance of
web applications by using pure Python and nothing else like HTML, CSS, JS etc.

Scikit Learn: Scikit-learn or sklearn, as it is better called by the developers, is a very

important and famous Python library, which provides a vast array of statistical and advanced
mathematical tools which are used frequently in the realm of ML while developing a model.
Apart from the usual tools and algorithms, it also provides various ML models straight out of
the box like Logistic Regression, Linear Regression, Random Forests, Decision Trees, SVM
Classifier etc, which come in handy while working on a classical Machine Learning project.

NumPy: If someone is a Python and/or ML developer, there’s almost a guarantee that they
have worked with NumPy. It provides all sorts of features and functionality to create and
manipulate arrays in python and for doing mathematical calculations like mean, median,
mode, minimum, maximum, variance, standard deviation etc.
The way NumPy differs from doing these calculations in native python itself, is because of its
speed, since NumPy has been made in such a way that it tries to first optimize the calculations
26
in the best possible way and then execute them in parallel to achieve greater performance.

Matplotlib: Matplotlib is a popular Python library which is used for creating and drawing
plots, visualizing images and drawing conclusions from them. It provided the functionality to
have different types of plot like bar graph, scatter plot, histogram, pie charts etc.
TensorFlow: It is a Machine Learning framework written in Python, which helps in creating,
manipulating and managing the process of model creation, training and evaluation as well as
creating the complete end-to-end data pipeline, right from feeding the data to the ML model
to evaluating it on the test dataset.

Tensorflow framework was used for training and evaluating the models. After training,
the models were converted using Tensorflow Lite to their portable versions. The final
models had an accuracy of ~95%. Tensorflow is one the Big 3 frameworks that any ML
developer uses for developing their ML models, be it from scratch or the ones provided
by Tensorflow itself.

TFLite: Tensorflow Lite in is offering from Tensorflow itself, which basically functions
as an intermediate exchange format for the ML model. The best thing about TFLite
models is that they are much faster and smaller in size than the traditionally trained
Tensorflow Model, since it uses various techniques like Graph Optimization, Weights
Clustering and Pruning etc.

Git & GitHub: Git is a version tracking system which is used to keep track of all the
different changes being made to the project’s source code as well as all the different
experimental and production branches that the developers are working on.
4 27 37
Github, on the other hand, is an online software hosting tool, built on top of Git, which the
developers can use to collaborate with each other while working on a common project.

Kaggle Kernel: The entire modelling process was carried out on Kaggle Kernels, which
are similar in nature to Jupyter Notebooks. Kaggle provides free GPU and TPU
resources (30 hours per week each) which can train the model many times faster than on
CPU.

Libraries like Numpy, Matplotlib, PIL etc. were used to data and image processing as
well as visualizing the input and output images.
1) For the environment:
36
a) Unreal Engine: Unreal Engine is a video game engine that debuted in 1998 with the
first-person shooter Unreal. It has since been embraced by a variety of businesses,
including the film and television industries, and is now used in a variety of
three-dimensional (3D) game genres.
6
The C++-based Unreal Engine is incredibly portable, supporting a wide range of
3
platforms including PC, mobile, console, and virtual reality. One of the major
enhancements anticipated for UE4 is real-time global illumination using voxel cone
tracing, which eliminates pre-computed lighting.
2
The Sparse Voxel Octree Global Illumination (SVOGI) feature, which was
demonstrated in the Elemental demo, was replaced with a similar but less
computationally expensive technique due to performance concerns.

One of its key features is Nanite, an engine that allows high-detail photographic
source material to be integrated into games. Epic may be able to employ the Nanite
virtualized geometry technology to leverage its prior acquisition of Quixel, the
world's largest photogrammetry collection as of 2019.

By letting the engine software manage these things, Unreal Engine 5 aimed to make
it as easy as possible for developers to create detailed game worlds without having to
spend a lot of time creating new sophisticated assets.

Nanite can import practically any other three-dimensional representation of objects

and environments, including ZBrush and CAD models, allowing film-quality assets
to be used.

The levels of detail (LODs) of these imported goods are handled by Nanite, who
adjusts them to the target platform and draw distance, a job that would normally be
done by an artist.

Another component is Lumen, which is defined as a "completely dynamic global

lighting system that adapts instantly to scene and light changes." Lumen does away
 with the requirement for artists and developers to create a lightmap for a scene,
instead calculating light reflections and shadows on the fly, allowing for real-time
light source behavior. Niagara is used for fluid and particle dynamics, while Chaos is
used as a physics engine.

Epic also developed the Unreal Engine 5 to take advantage of the upcoming high
speed storage solutions with the next-generation console hardware, which will use a
mix of RAM and custom solid-state drives. With potentially tens of billions of
polygons on a single screen at 4K resolution.

Epic also developed the Unreal Engine 5 to take advantage of the upcoming
high-speed storage solutions with the next-generation console hardware that will use
a mix of RAM and custom solid state drives.

The Unreal Engine, developed by Epic Games, debuted in 1998 with the first-person
shooter Unreal. Originally intended for PC first-person shooters, it has since been
embraced by a variety of businesses, including the film and television industries, and
is currently used in a variety of three-dimensional (3D) game genres. The Unreal
6
Engine is very portable, supporting a wide range of desktop, mobile, console, and
virtual reality platforms.

It is developed in C++. One of the most significant enhancements anticipated for

UE4 is real-time global illumination using voxel cone tracing, which removes
pre-calculated lighting. The Sparse Voxel Octree Global Illumination (SVOGI)
feature from the Elemental demo was replaced with a similar but less
computationally expensive approach due to performance concerns.

b) Unity: Unity Technologies has developed a cross-platform game engine with an

integrated development environment. It's used to make games for web browsers,
desktop platforms, consoles, and mobile devices. It is considered easy to use for
beginner developers and is popular for indie game development. Film, automobile,
architecture, engineering, construction, and the United States Armed Forces are
among the industries that have utilised the engine.
 Unity is the union of:

1. a gaming engine that enables a game to operate (and so be played) in a variety of

settings.
2. a programme that allows you to put together the "visible components" of a game
(the IDE) with a graphical preview and a controlled "play it" option.
3. a code editor (MonoDevelop is given, but developers can use other editors).

c) Blender is a 3D modeling programme that is both free and open-source. Rigging,

Modeling, animation, simulation, rendering, compositing, and motion tracking are all
supported, as well as video editing and game production. Advanced users utilize
Blender's Python scripting API to tweak the application and create specialized tools,
which are frequently included in subsequent Blender editions.

Individuals and small studios will profit from Blender's unified pipeline and
responsive development method. Blender is a multi-platform application that works
on Linux, Windows, and Macintosh computers. Its user interface is based on
OpenGL, which ensures a consistent experience. The list of supported systems
highlights those that are routinely evaluated by the development team to ensure
particular compatibility.

Blender is a 3D modeling application that is both free and open-source. Video editing
and game development, as well as modeling, rigging, animation, simulation,
rendering, compositing, and motion tracking, are all supported.

Blender's Python scripting API is used by advanced users to alter the application and
build specialised tools, which are regularly included in later Blender editions.
Blender's unified pipeline and flexible development style will benefit individuals and
small studios.

Blender is a cross-platform software that runs on Linux, Windows, and Macintosh

computers. It has an OpenGL-based user interface, which offers a consistent
experience. The list of supported systems emphasises those that the development
team tests on a regular basis to assure compatibility.
d) Figma is a vector graphics editor and prototype tool that is mostly web-based, with
desktop apps for macOS and Windows providing extra offline features. The Figma
mobile app for Android and iOS allow viewing and interacting with Figma
prototypes in real-time on mobile devices.

The feature set of Figma focuses on use in user interface and user experience design,
with an emphasis on real-time collaboration.Figma's collaboration platform can be
used for UI design, ux design, prototyping, graphic design, wireframing,
brainstorming, and templates.

Figma allows users to draw vector networks in any direction, create custom fonts,
and make instant arc designs for clocks, watches, and pie charts using the Arc tool.

Many features of Figma are automat tasks such as resizing buttons, text, layouts,
padding, direction, and spacing, and translating design changes into computer code
for handing off designs to software developers.

e) Adobe XD (formerly known as Adobe Experience Design) is a vector-based user

experience design tool created and released by Adobe Inc. for online and mobile
apps. It is available for macOS and Windows, although there are versions for iOS
and Android to help preview the result of work directly on mobile devices.

Adobe XD enables website wireframing and creating click-through

prototypes.Adobe XD creates user interfaces for mobile and web apps. Many
features in XD were previously either hard to use or nonexistent in other Adobe
applications like Illustrator or Photoshop.

XD supports and can open files from Illustrator, Photoshop, Photoshop Sketch, and
AfterꢀEffects. In addition to the Adobe Creative Cloud, XD can also connect to other
tools and services such as Slack and Microsoft Teams to collaborate. XD may also
auto-adjust and transition from macOS to Windows.
 Self-learning AI

Self-learning Artificial intelligence (AI) is a type of machine learning that can learn from
unlabeled data. It works on a high level by evaluating a dataset and looking for patterns
from which it might derive inferences. It basically teaches itself to "fill in the blanks."

While a student who studies Spanish for five years in school may have a solid grasp of the
language and how to apply it, the student will have learned considerably more slowly than
someone who simply moves to Mexico for a few months. The principle of
learning-by-doing is being applied to AI in the form of self-learning AI.

When training a machine on a concept for which there isn't a lot of training data,
self-learning AI comes in handy. It can also be useful for training computers on
processes that academics are unfamiliar with, making labelled training samples
difficult to come by.

Another advantage of self-learning AI is that once a new talent is mastered, it may be applied
to other related skills with greater ease. When deep learning is done in a supervised
environment, the machine must start from scratch and gradually add new actions to their
repertoire. When the setting changes, though, the abilities don't often transfer as easily.

Because AI uses unsupervised learning from the entire environment rather than a specific
40
dataset, it can be on the lookout for additional anomalies that human researchers may be
unaware of. Due to this, it is better than most people at spotting changes and patterns that
28
indicate a breach, cybersecurity is one of the top areas where self-learning AI is now being
deployed.

METHODOLOGY AND ALGORITHMS

Research and Planning
 35
We read many research papers that were published with similar problem statement in view to
better understand better the Self-learning AI in a simulated Environment and how the algorithms
pertaining to it perfrom. There were various findings that made clear what approachand
algorithms to use.
29
In this project, we have researched, studied and presented some of the most novel and state of the
art approaches currently being employed by researchers worldwide making use of a wide variety
of CNN architectures as backbone including EfficientNet, MobileNet v2 and v3, ResNet 18,
VGG 16 etc.
After the research and planning the project was divided into two parts, the first one being
designing a deep neural net and training the model on the training data and converting it to a
TensorFlow model so that it can be ported to the stimulated environment

Learning Rate Scheduler:

While using Transfer Learning, one thing to keep in mind is that the model architecture that is
being used already has balanced and trained parameters. Using a usual or static learning rate
could distort the pre-trained parameters, therefore, instead of using a single learning rate, we need
to create a learning rate scheduler which increases the learning rate gradually up to a certain
epoch and then starts decreasing it exponentially.

We also need to decide the starting, minimum, maximum learning rate along with the number of
ramps up epochs and sustain epochs as well as the value of the exponential decay constant used
for decreasing the learning rate.

Creating and training the model:

● Downloading the EfficientNet package using pip.

● Creating a strategy using tf.distribute depending on the type of accelerator being used
by the kernel i.e. TPU, GPU or CPU.
● Creating an ImageDataGenerator variable with a rescaling factor of 255.0 so as to
normalize the images before training.
● Initialize the train, validation and test datagen by using the ImageDataGenerator’s
 flow from directory feature specifying the path of the images, the batch size, target
size of the image and the class mode (categorical in this case).
● Within the scope of the strategy, instantiate the EfficientNet-B0 with the ‘imagenet’
weights and remove the last layer. Set the trainable option to True.
39
● Now, create a Keras Sequential model with the first layer being the EfficientNet-B0
followed by a GlobalAveragePooling2D layer and then end with a dense layer
consisting of as many neurons as the number of target classes.

(Summary of the created model)

● Then, using the categorical cross entropy loss function, Adam optimizer, and
Accuracy as the evaluation measure, we must compile the model.
● Create a callbacks list consisting of a Learning Rate Scheduler and a Model
Checkpoint object which evaluates the model on its validation accuracy.
● Use the fit generator function to start training the model specifying the train datagen,
steps to take per epoch, validation datagen, validation steps, the number of epochs to
run the training for (10 epochs in this case) and the callbacks list (if any).

The trained model can then be easily evaluated on the validation datagen using the evaluate
generator.
RESULT AND OUTPUT

The Deep Learning model, after going through the training for 10 epochs, produced an accuracy
of 94.2% on flower dataset, 94% on animal dataset and 97.4% on the bird dataset. For the
size of the data that the model has been trained on, it is pretty good accuracy.
The training curves for the loss and accuracy have been visualized below for the best model
amongst the three, the bird predictor:

(Cross Entropy Loss vs Epochs curve)

 (Validation Accuracy vs Epochs curve)
.

Limitations of this training method

To date, the majority of artificial intelligence (AI) applications that have emerged have relied on
supervised machine learning to learn rules and exceptions to norms. This is where the AI is
taught a certain topic using tagged data sets.

From AI's ability to recognise our friends' faces in our social media feeds to its ability to
remove background noise from video meetings – a useful feature given how many of us now
work from home – this approach of teaching has produced fantastic results. While labelling
and cleaning data sets so that computer algorithms can understand them takes time, the
accuracy can be rather good.
On the other hand, supervised learning only works when the consequence is fully understood
by humans. Even though their looks are identical, we can differentiate a friend's face from a
stranger's face, and we can rapidly distinguish between two voices, even if their accents are
similar.

The most visible example of AI's self-learning component is also its most vulnerable victim,
vulnerable to different attacks and cyber threats. Cyber security is vulnerable because the more
data we exchange, the more vulnerable we are to attack; at the same time, the more attackers
innovate, the more difficult it is to detect them.

It is impossible to turn off data flows. Global businesses and, especially in today's epidemic,
social relationships necessitate Internet-based communication. Given that all data is linked
and, to some extent, vulnerable to intrusion, the challenge is identifying malicious behaviour
among all the legitimate activity. In order to gain our trust, we must not only detect the overtly
hostile threat actors, but also the more subtle intruders–the insider danger or the smart attacker
who impersonates someone he or she is not.

Precautions we need to take while training this model

Given a set of parameters, supervised machine learning is successful at learning. However,

when dealing with cyber-threat actors, those boundaries become more difficult to define. The
goalposts are always moving.

While artificial intelligence may never be able to perfectly replicate human intelligence,
unsupervised machine learning can come close. It does not learn from training data sets, but
rather from the data environment in which it is put.
It can also recognise different shades of grey in data flows, as well as new patterns that aren't
pre-defined.

Therefore, in order to curb the problems that we may during the training and pruning
of the model we opt for:
1. Select the appropriate learning model for the situation.

Every AI model is diverse for a reason: each challenge necessitates a different solution and
requires various data resources. There is no one-size-fits-all approach to eliminating bias, but
there are several guidelines that can aid your team's formation.

Supervised and unsupervised learning methods, for example, each have their own set of benefits
and drawbacks. Unsupervised clustering or dimension reduction models can learn bias from
their input data. If belonging to group A substantially correlates with behavior B, the model can
blend the two. While supervised models provide you more control over data selection bias, they
also introduce human bias.

2. Select a training data set that is representative.

While your data scientists will do the majority of the job, everyone involved in an AI project
must actively defend against data selection bias. You must cross a very fine line. The training
data must be diverse and cover a wide range of categories, although segmentation in the model
can be problematic unless the real data is classified similarly.

It is inadvisable to have different models for different groups, both computationally and in
terms of public relations. When there isn't enough data for one group, you can use weighting to
make it more significant in training, but be cautious. It can lead to unexpected new prejudices.

3. Use real-time data to track performance.

All discriminatory models most likely operated as planned in controlled conditions. Regulators
(and the general public) rarely recognise good intentions when assigning blame for ethical
transgressions. As a result, you should aim to emulate as many real-world events as possible
when building algorithms.
Using test groups on algorithms that are currently in use, for example, is not a good idea.
Instead, test your statistical procedures on real data wherever possible.

It's easier to verify result equivalence this way, but it also implies that you're willing to tolerate
skewed data. While demonstrating equality of opportunity is more challenging, it is morally
acceptable.

Although achieving both types of equality is difficult, diligent oversight and real-world testing
of your models should give you the best opportunity.

How to mimic a system similar to self learning AI

Traditional artificial intelligence (AI) makes use of computer algorithms to create computer
programmes that are designed to make judgments or solve issues based on pre-programmed
knowledge.

Self-learning AIs, on the other hand, build on top of powerful neural networks and deep
learning frameworks like Google's Tensorflow with machine learning algorithms. At first
glance, AI appears to be the pinnacle of invention, if it weren't for one basic fact: even
self-learning AIs can't learn everything.

Self-learning AI programmes that must identify objects or images are fed massive volumes of
data in order to build a neural network that matches the situation at hand. This means that if
the AI system is to recognise a cat in a photograph, more photographs of cats must be
submitted.

You'll need to submit 100 images of the object you want to identify and 100 photos of an object
that is opposite or different from the original object into the system to construct an image
recognition software employing self-learning AI technologies.
And that isn't all. These things will take days to train on the system. The system will need test
data to see if the machine has been properly trained to recognise the correct object. Before
they start delivering correct results, self-learning AIs can spend weeks training images and
attempting countless tests.

Self-learning AIs must sustain a continual and ongoing learning process even after generating
accurate results. Without this, the self-learning AI will become increasingly inaccurate over
time, becoming frequently confused by newly obtained photos.

Implementation of self learning AI in games

By teaching a virtual agent to outperform human players, we can learn how to optimise a
variety of processes in a variety of interesting subfields. It was Google DeepMind's popular
AlphaGo that did it, defeating the world's best Go player and scoring an unattainable goal at the
time.

We'll build an AI agent that can educate itself to play the popular game Snake from scratch. To
do so, we built a Deep Reinforcement Learning approach using Keras on top of Tensorflow and
PyTorch (both versions are available, you can choose the one you prefer).

This method involves the interaction of two elements: the environment (the game itself) and the
agent (Snake). The agent gathers information about its present condition (which we'll explain
later) and takes appropriate action.

The agent gradually learns which acts maximise the payoff (in our case, what actions lead to
eating the apple and avoiding the walls). There are no game rules provided. Snake is at first
unsure of what to do and conducts haphazard actions.

The goal is to come up with a way to maximise the score or reward. We'll see how a Deep
Q-Learning system picks up Snake in just 5 minutes, scoring up to 50 points and displaying a
reasonable strategy.
 Implementation theory of the game

Reinforcement Learning is a collection of algorithms and techniques used in control and

decision-making (e.g., robotics, autonomous driving, etc.). These methods are used to tackle
problems that require the use of a Markov Decision Process (MDP). It means that our game
must be represented by a set of states S, a set of actions A, a reward function R, and, if
desired, a state transition function T.

The environment and the agent are the two main components of Reinforcement Learning.
Every time the agent performs an action, the environment rewards the agent, which might
be positive or negative depending on how good the activity was from that specific state.
The agent's goal is to determine which behaviours maximise reward in all possible
scenarios.

At each iteration, the agent receives observations from the environment, which are
referred to as states. A state can be defined as its location, speed, or any combination of
environmental elements. The agent's decision-making process is referred to as policy to be
more explicit and to use a Reinforcement Learning notation.

On a theoretical level, a policy is a mapping from the state space to the action space.
A Q-table is a matrix that links an agent's present state to the various actions the agent
can perform. The probabilities of success for the actions in the table were modified based
on the rewards the agent earned during training.

To put it another way, these numbers represent the average reward the agent will receive if it
acts on the state s. This table shows the agent's policy, which specifies what actions should
be taken at each stage to maximise the expected outputs.
 Code
Food collection

public Material normalMaterial;

public Material badMaterial;
public Material goodMaterial;
public Material frozenMaterial;
public GameObject myLaser;
public bool contribute;
public bool useVectorObs;
[Tooltip("Use only the frozen flag in vector observations. If \"Use Vector Obs\" " +
"is checked, this option has no effect. This option is necessary for the " +
"VisualFoodCollector scene.")]
public bool useVectorFrozenFlag;
 m_ResetParams = Academy.Instance.EnvironmentParameters;
SetResetParameters();
}

sensor.AddObservation(localVelocity.x);
sensor.AddObservation(localVelocity.z);
sensor.AddObservation(m_Frozen);
sensor.AddObservation(m_Shoot);
}
else if (useVectorFrozenFlag)
{
sensor.AddObservation(m_Frozen);
}
}

public Color32 ToColor(int hexVal)

{

public void MoveAgent(ActionBuffers actionBuffers)

{
m_Shoot = false;

if (Time.time > m_FrozenTime + 4f && m_Frozen)

{
Unfreeze();
}
if (Time.time > m_EffectTime + 0.5f)
{
if (m_Poisoned)
{
Unpoison();
}
if (m_Satiated)
{
Unsatiate();
}
}

dirToGo = transform.forward * forward;

dirToGo += transform.right * right;
rotateDir = -transform.up * rotate;

var shootCommand = discreteActions[0] > 0;

if (shootCommand)
{
m_Shoot = true;
dirToGo *= 0.5f;
m_AgentRb.velocity *= 0.75f;
}
m_AgentRb.AddForce(dirToGo * moveSpeed, ForceMode.VelocityChange);
transform.Rotate(rotateDir, Time.fixedDeltaTime * turnSpeed);
}

{
 hit.collider.gameObject.GetComponent<FoodCollectorAgent>().Freeze();
}
}
}
else
{
myLaser.transform.localScale = new Vector3(0f, 0f, 0f);
}
}

void Freeze()
{
gameObject.tag = "frozenAgent";
m_Frozen = true;
m_FrozenTime = Time.time;
gameObject.GetComponentInChildren<Renderer>().material = frozenMaterial;
}

void Unfreeze()
{
m_Frozen = false;
gameObject.tag = "agent";
gameObject.GetComponentInChildren<Renderer>().material = normalMaterial;
}

void Poison()
{
m_Poisoned = true;
m_EffectTime = Time.time;
gameObject.GetComponentInChildren<Renderer>().material = badMaterial;
}

void Unpoison()
{
m_Poisoned = false;
gameObject.GetComponentInChildren<Renderer>().material = normalMaterial;
}

void Satiate()
{
m_Satiated = true;
m_EffectTime = Time.time;
gameObject.GetComponentInChildren<Renderer>().material = goodMaterial;
}

void Unsatiate()
{
m_Satiated = false;
gameObject.GetComponentInChildren<Renderer>().material = normalMaterial;
}

var discreteActionsOut = actionsOut.DiscreteActions;

discreteActionsOut[0] = Input.GetKey(KeyCode.Space) ? 1 : 0;
}

public override void OnEpisodeBegin()

{
Unfreeze();
Unpoison();
Unsatiate();
m_Shoot = false;
 m_FoodCollecterSettings.totalScore += 1;
}
}
if (collision.gameObject.CompareTag("badFood"))
{
Poison();
collision.gameObject.GetComponent<FoodLogic>().OnEaten();

AddReward(-1f);
if (contribute)
{
m_FoodCollecterSettings.totalScore -= 1;
}
}
}

public void SetLaserLengths()

{
m_LaserLength = m_ResetParams.GetWithDefault("laser_length", 1.0f);
}

public void SetAgentScale()

{
float agentScale = m_ResetParams.GetWithDefault("agent_scale", 1.0f);
gameObject.transform.localScale = new Vector3(agentScale, agentScale, agentScale);
}
 public void SetResetParameters()
{
SetLaserLengths();
SetAgentScale();
}
}

Survival

public class Chiklz : Agent

{
//The audio source and all the audio clips used
public AudioSource audioSource;
public AudioClip[] audioClips;

//Static variable for the highest score so it is only one for all the instances
public static float longest_Score;
//Current iterration
public static int iter;

//Train yard ID
public int id;

//Spider Object
public Spider spider;
//Transforms for every location
public Transform mine, forest, farm, camp;
Transform current_Target;

//Gradient for the light and the light object

public Gradient gradient;
public Light light;

//Some texts
public TextMeshProUGUI[] owned_Resources, equipped_Resources, stats;
public TextMeshProUGUI time_Text, iter_Text, record_Text;

//Current day, hour and previous hour

int day;
[SerializeField]
float hour, prev_Hour;

//Animator object
public Animator animator;

//Bool is_Moving is true when the agent is moving

//Bool is_Busy is true when the agent is performing an action
[SerializeField]
bool is_Moving, is_Busy;

//Ingame stats
public float health, camp_Health, hunger, maxHealth_Camp, max_Hunger, max_Health;
float speed, range, attack_Speed, attack_Damage;

[SerializeField]
int inv_Items, food, wood, stone_E, wood_E, food_E;

//Some more booleans

bool is_Spider, was_Spawned;
//Spawn position, for restarting the level
Vector3 spawn_Pos;

//Aren't we all?
bool is_Dead;

//Time scale for faster training

float scale;

//This method is called at the beginning and initializes all the variables
public override void Initialize()
{
scale = 1.5f;
iter = 0;
Time.timeScale = scale;
current_Target = farm;

maxHealth_Camp = 100;
max_Health = 100;
max_Hunger = 100;

health = max_Health;
hunger = max_Hunger;
camp_Health = maxHealth_Camp;

speed = 10;
range = 1;
 attack_Speed = 1;
attack_Damage = 1;
}

//This method Is called at the end of each level and resets everything
public void Reset_Level()
{
iter++;
iter_Text.text = "Iteration Nr.: " + iter.ToString();

is_Dead = false;
was_Spawned = false;
is_Moving = false;
is_Busy = false;

transform.localPosition = Vector3.zero;
current_Target = farm;

maxHealth_Camp = 100;
max_Health = 100;
max_Hunger = 100;

health = max_Health;
hunger = max_Hunger;
camp_Health = maxHealth_Camp;

inv_Items = 1;

stone_E = 0;
food_E = 0;
wood_E = 1;
 food = 0;
wood = 0;

day = 0;
hour = 0;
prev_Hour = 0;
}
// Update is called per frame
void Update()
{
//Changing color of the sky
light.color = gradient.Evaluate(hour / 24);

//Basic Life Stuff

if (camp_Health > 0)
camp_Health -= Time.deltaTime;
if (hunger > 0)
hunger -= Time.deltaTime / 2;
if (hunger <= 0)
health -= Time.deltaTime;
if (hunger >= 80)
if (health < max_Health)
health += Time.deltaTime;
if (health <= 0)
Die("Low Health");
if (camp_Health <= 0)
Die("Camp Fire Died");

//Incrementing time
hour += Time.deltaTime / 60;
 if(hour >= 18)
{
if (!was_Spawned)//if the spider was not spawned - spawn it.
{
spider.is_Awake = true;
is_Spider = true;
was_Spawned = true;
}
}
if(hour >= 24)// day passed
{
day++;
hour = 0;
was_Spawned = false;

AddReward(10f); //Reward for passing through the day

}
//changing some text
if(id == 0)
time_Text.text = "Day " + day + "(" + (int)hour + "h)";

//Moving
if (is_Moving && !is_Busy && !is_Dead)
{
if(Vector3.Distance(transform.localPosition, current_Target.localPosition) > range)
{
transform.localPosition = Vector3.MoveTowards(transform.localPosition,
current_Target.localPosition, speed * Time.deltaTime);
}
else
{
 is_Moving = false;
}
}
}

//Called at beginning of every episode and resets the level

public override void OnEpisodeBegin()
{
Reset_Level();
}

//All the observations the agents collects before making a decision

sensor.AddObservation(food);
sensor.AddObservation(food_E);

sensor.AddObservation(wood_E);
sensor.AddObservation(wood);
 sensor.AddObservation(inv_Items);

sensor.AddObservation(max_Health);
sensor.AddObservation(max_Hunger);
sensor.AddObservation(maxHealth_Camp);

sensor.AddObservation(is_Spider);
}

//Similar to update but with the frequency set in Decission Requester Component
public override void OnActionReceived(float[] vectorAction)
{
if(id == 0)
Update_Text();

//Rewarding for every survived hour

if (Mathf.Floor(hour) == prev_Hour + 1)
{
prev_Hour = Mathf.Floor(hour);
AddReward(1f);
}

//If there is wood in storage - use it

if (camp_Health <= maxHealth_Camp - 25 && wood > 0)
{
wood--;
camp_Health += 25;
}
if (!is_Busy && !is_Dead)
{
if (vectorAction[1] != 1)
{
if (vectorAction[0] == 0)//Farm
{
if (Has_To_Move(farm))
{

}
else
{
StartCoroutine(Farm());
}
}
else if (vectorAction[0] == 1)//Enemy
{
if (Has_To_Move(mine))
{

}
else
{
StartCoroutine(Mine());
}
/*if (is_Spider && hunger > 35)
{
if (Has_To_Move(spider.transform))
{
 }
else
{
if (vectorAction[1] == 0)
{
StartCoroutine(Deposit());
}
else
{
StartCoroutine(Eat());
}
}
}*/
}
else if (vectorAction[0] == 2)//Forest
{
if (Has_To_Move(forest))
{

}
else
{
StartCoroutine(Chop());
}
}
else if(vectorAction[0] == 3)//Camp
{
if (Has_To_Move(camp))
{

}
 else
{
if (vectorAction[1] == 0)
{
StartCoroutine(Deposit());
}
else
{
StartCoroutine(Eat());
}
}
}
else//Mine
{
if (Has_To_Move(mine))
{

}
else
{
StartCoroutine(Mine());
}
}
}
else
{
StartCoroutine(Eat());
}

}
}
//Allows to control the agent for testing purpouses

public void Update_Text()

{
owned_Resources[0].text = "Food: " + food;
owned_Resources[1].text = "Wood: " + wood;

equipped_Resources[0].text = "Food: " + food_E;

equipped_Resources[1].text = "Stone: " + stone_E;
equipped_Resources[2].text = "Wood: " + wood_E;
 stats[0].text = "Health: " + (int)health;
stats[1].text = "Hunger: " + (int)hunger;
stats[2].text = "Camp Health: " + (int)camp_Health;

stats[3].text = "Max Health: " + (int)max_Health;

stats[4].text = "Max Hunger: " + (int)max_Hunger;
stats[5].text = "Max Camp Health: " + (int)maxHealth_Camp;

//Checks if the agents needs to move by calculating the distance between it and the target
public bool Has_To_Move(Transform target_Position)
{
if (Vector3.Distance(transform.localPosition, target_Position.localPosition) > range)
{
current_Target = target_Position;
transform.LookAt(target_Position);
if (animator)
animator.Play("Walk");

is_Moving = true;
return true;
}
else
{
return false;
}

}
 public void Die(string reason)
{
if (!is_Dead)
{
audioSource.clip = audioClips[5];
audioSource.Play();

is_Dead = true;
if(animator)
animator.Play("Die");

print("Died because of " + reason);

if (longest_Score < (day * 25 + hour))

{
longest_Score = (day * 25 + hour);
record_Text.text = "Max. Time Survived: " + day + " days and " +
String.Format("{0:0.00}", hour) + " hours";

}
AddReward(-10f);

if (id == 0)
{
print("Result: " + GetCumulativeReward());
}
}
//Reset_Level();
//EndEpisode();
 StartCoroutine(Death());
}

//I did that for animation... i think

IEnumerator Death()
{
yield return new WaitForSeconds(5);
//Reset_Level();
EndEpisode();
}

//Store stuff at base

IEnumerator Deposit()
{

if (inv_Items > 0)// if the agents has items in inventory it will take him .5 seconds to store
them and he will be rewarded
{
audioSource.clip = audioClips[1];
audioSource.Play();

is_Busy = true;
if (animator)
animator.Play("Action");

yield return new WaitForSeconds(.5f);

max_Health += stone_E * 10;

max_Hunger += stone_E * 10;
maxHealth_Camp += stone_E * 10;
 stone_E = 0;
food += food_E;
food_E = 0;

//camp_Health += wood_E * 25;

wood += wood_E;
wood_E = 0;

inv_Items = 0; ;

is_Busy = false;
AddReward(0.05f);
}
else
{
if (animator)
animator.Play("Idle");
}
}

IEnumerator Eat()
{
if (hunger < max_Hunger - 10)//Eat only if there is need for that
{
if (Vector3.Distance(transform.localPosition, camp.localPosition) > range)//if we are not
in range of the campfire look if there is any food in inventory to eat
{
if (food_E > 0)
{
audioSource.clip = audioClips[0];
 audioSource.Play();

is_Busy = true;
if (animator)
animator.Play("Action");

yield return new WaitForSeconds(.5f);

food_E--;
inv_Items--;
if (hunger < max_Hunger - 10)
{
hunger += 10;
}
else
{
hunger = max_Hunger;
}
is_Busy = false;
}
}
else
{
if (food > 0)//look if there is food in the campfire to eat
{
audioSource.clip = audioClips[0];
audioSource.Play();

is_Busy = true;
if (animator)
animator.Play("Action");
 yield return new WaitForSeconds(.5f);
food--;
if (hunger < max_Hunger - 20)
{
hunger += 20;
}
else
{
hunger = max_Hunger;
}
is_Busy = false;
}
}
}
}

IEnumerator Mine()
{
if (inv_Items < 2 && hunger > 25) //if not hungry - mine
{
audioSource.clip = audioClips[2];
audioSource.Play();

is_Busy = true;
if (animator)
animator.Play("Action");

yield return new WaitForSeconds(1);

stone_E++;
 inv_Items++;
hunger = Mathf.Max(hunger - 25, 0);

is_Busy = false;
AddReward(0.01f);
}
else
{
if (animator)
animator.Play("Idle");
}
}

IEnumerator Farm()
{
if (inv_Items < 2)//if not hungry - farm
{
audioSource.clip = audioClips[3];
audioSource.Play();

is_Busy = true;
if (animator)
animator.Play("Action");

yield return new WaitForSeconds(1);

food_E++;
inv_Items++;

is_Busy = false;
 AddReward(0.01f);
}
else
{
if (animator)
animator.Play("Idle");
}
}

IEnumerator Chop()
{
if (inv_Items < 2 && hunger > 25)//if not hungry - chop
{
audioSource.clip = audioClips[4];
audioSource.Play();

is_Busy = true;
if (animator)
animator.Play("Action");

yield return new WaitForSeconds(1);

wood_E++;
inv_Items++;

hunger = Mathf.Max(hunger - 25, 0);

is_Busy = false;
AddReward(0.01f);
}
else
 {
if (animator)
animator.Play("Idle");
}
}

//Not used in the final version

IEnumerator Attack()//if not hungry - attack
{
if (hunger > 35)
{
is_Busy = true;
if (animator)
animator.Play("Action");

yield return new WaitForSeconds(1);

spider.Die();
print("Kileld");
hunger = Mathf.Max(hunger - 35, 0);

is_Spider = false;
is_Busy = false;
}
else
{
if (animator)
animator.Play("Idle");
}
}
}
Animal AI

{
Walking AI
public class WalkerAgent : Agent
{
[Header("Walk Speed")]
[Range(0.1f, 10)]
[SerializeField]
//The walking speed to try and achieve
private float m_TargetWalkingSpeed = 10;

public float MTargetWalkingSpeed // property

{
get { return m_TargetWalkingSpeed; }
set { m_TargetWalkingSpeed = Mathf.Clamp(value, .1f, m_maxWalkingSpeed); }
}

const float m_maxWalkingSpeed = 10; //The max walking speed

//Should the agent sample a new goal velocity each episode?

//If true, walkSpeed will be randomly set between zero and m_maxWalkingSpeed in
OnEpisodeBegin()
//If false, the goal velocity will be walkingSpeed
public bool randomizeWalkSpeedEachEpisode;

//The direction an agent will walk during training.

private Vector3 m_WorldDirToWalk = Vector3.right;

[Header("Target To Walk Towards")] public Transform target; //Target the agent will walk
towards during training.

[Header("Body Parts")] public Transform hips;

public Transform chest;
 //This will be used as a stabilized model space reference point for observations
//Because ragdolls can move erratically during training, using a stabilized reference transform
improves learning
OrientationCubeController m_OrientationCube;

//The indicator graphic gameobject that points towards the target

DirectionIndicator m_DirectionIndicator;
JointDriveController m_JdController;
EnvironmentParameters m_ResetParams;

public override void Initialize()

{
m_OrientationCube = GetComponentInChildren<OrientationCubeController>();
m_DirectionIndicator = GetComponentInChildren<DirectionIndicator>();
 m_ResetParams = Academy.Instance.EnvironmentParameters;

SetResetParameters();
}
 //Set our goal walking speed
MTargetWalkingSpeed =
randomizeWalkSpeedEachEpisode ? Random.Range(0.1f, m_maxWalkingSpeed) :
MTargetWalkingSpeed;

SetResetParameters();

//GROUND CHECK
sensor.AddObservation(bp.groundContact.touchingGround); // Is this bp touching the
ground

//Get velocities in the context of our orientation cube's space

//Note: You can get these velocities in world space as well but it may not train as well.
sensor.AddObservation(m_OrientationCube.transform.InverseTransformDirection(bp.rb.velocity
));

sensor.AddObservation(m_OrientationCube.transform.InverseTransformDirection(bp.rb.angular
Velocity));

//Get position relative to hips in the context of our orientation cube's space

sensor.AddObservation(m_OrientationCube.transform.InverseTransformDirection(bp.rb.position
- hips.position));

if (bp.rb.transform != hips && bp.rb.transform != handL && bp.rb.transform != handR)

{
sensor.AddObservation(bp.rb.transform.localRotation);
sensor.AddObservation(bp.currentStrength / m_JdController.maxJointForceLimit);
}
}

/// <summary>
/// Loop over body parts to add them to observation.
/// </summary>
public override void CollectObservations(VectorSensor sensor)
{
var cubeForward = m_OrientationCube.transform.forward;

//velocity we want to match

var velGoal = cubeForward * MTargetWalkingSpeed;
//ragdoll's avg vel
var avgVel = GetAvgVelocity();
 //current ragdoll velocity. normalized
sensor.AddObservation(Vector3.Distance(velGoal, avgVel));
//avg body vel relative to cube
sensor.AddObservation(m_OrientationCube.transform.InverseTransformDirection(avgVel));
//vel goal relative to cube

sensor.AddObservation(m_OrientationCube.transform.InverseTransformDirection(velGoal));

//rotation deltas
sensor.AddObservation(Quaternion.FromToRotation(hips.forward, cubeForward));
sensor.AddObservation(Quaternion.FromToRotation(head.forward, cubeForward));

//Position of target position relative to cube

sensor.AddObservation(m_OrientationCube.transform.InverseTransformPoint(target.transform.p
osition));

foreach (var bodyPart in m_JdController.bodyPartsList)

{
CollectObservationBodyPart(bodyPart, sensor);
}
}

public override void OnActionReceived(ActionBuffers actionBuffers)

//Update OrientationCube and DirectionIndicator

void UpdateOrientationObjects()
{
m_WorldDirToWalk = target.position - hips.position;
m_OrientationCube.UpdateOrientation(hips, target);
 if (m_DirectionIndicator)
{
m_DirectionIndicator.MatchOrientation(m_OrientationCube.transform);
}
}

void FixedUpdate()
{
UpdateOrientationObjects();

var cubeForward = m_OrientationCube.transform.forward;

// Set reward for this step according to mixture of the following elements.
// a. Match target speed
//This reward will approach 1 if it matches perfectly and approach zero as it deviates
var matchSpeedReward = GetMatchingVelocityReward(cubeForward *
MTargetWalkingSpeed, GetAvgVelocity());

//Check for NaNs

if (float.IsNaN(matchSpeedReward))
{
throw new ArgumentException(
"NaN in moveTowardsTargetReward.\n" +
$" cubeForward: {cubeForward}\n" +
$" hips.velocity: {m_JdController.bodyPartsDict[hips].rb.velocity}\n" +
$" maximumWalkingSpeed: {m_maxWalkingSpeed}"
);
}

// b. Rotation alignment with target direction.

//This reward will approach 1 if it faces the target direction perfectly and approach zero as it
deviates
var lookAtTargetReward = (Vector3.Dot(cubeForward, head.forward) + 1) * .5F;

//Check for NaNs

if (float.IsNaN(lookAtTargetReward))
{
throw new ArgumentException(
"NaN in lookAtTargetReward.\n" +
$" cubeForward: {cubeForward}\n" +
$" head.forward: {head.forward}"
);
}

AddReward(matchSpeedReward * lookAtTargetReward);
}

//Returns the average velocity of all of the body parts

//Using the velocity of the hips only has shown to result in more erratic movement from the
limbs, so...
//...using the average helps prevent this erratic movement
Vector3 GetAvgVelocity()
{
Vector3 velSum = Vector3.zero;

//ALL RBS
int numOfRb = 0;
foreach (var item in m_JdController.bodyPartsList)
{
numOfRb++;
velSum += item.rb.velocity;
}
 var avgVel = velSum / numOfRb;
return avgVel;
}

//normalized value of the difference in avg speed vs goal walking speed.

public float GetMatchingVelocityReward(Vector3 velocityGoal, Vector3 actualVelocity)
{
//distance between our actual velocity and goal velocity
var velDeltaMagnitude = Mathf.Clamp(Vector3.Distance(actualVelocity, velocityGoal), 0,
MTargetWalkingSpeed);

//return the value on a declining sigmoid shaped curve that decays from 1 to 0
//This reward will approach 1 if it matches perfectly and approach zero as it deviates
return Mathf.Pow(1 - Mathf.Pow(velDeltaMagnitude / MTargetWalkingSpeed, 2), 2);
}

/// <summary>
/// Agent touched the target
/// </summary>
public void TouchedTarget()
{
AddReward(1f);
}

public void SetTorsoMass()

{
m_JdController.bodyPartsDict[chest].rb.mass =
m_ResetParams.GetWithDefault("chest_mass", 8);
m_JdController.bodyPartsDict[spine].rb.mass =
m_ResetParams.GetWithDefault("spine_mass", 8);
 m_JdController.bodyPartsDict[hips].rb.mass =
m_ResetParams.GetWithDefault("hip_mass", 8);
}

public void SetResetParameters()

{
SetTorsoMass();
}
}

Animal Attack AI

target = camp;

spawn_Pos = transform.localPosition;
if (animator)
animator.Play("Idle");
distance_Player = 100;
InvokeRepeating("Calculate_Distance",0,.1f);
}

// Update is called once per frame

 void Update()
{
if (is_Awake)
{
if (distance_Player < 2)
target = player;
else
target = camp;

if (Not_In_Range())
{
transform.localPosition = Vector3.MoveTowards(transform.localPosition,
target.localPosition, 3 * Time.deltaTime);
if (animator)
animator.Play("Walk");
}
else
Attack();
}
}

public bool Not_In_Range()

{
transform.LookAt(target);
return Vector3.Distance(transform.localPosition, target.localPosition) > 1;
}
public void Calculate_Distance()
{
distance_Player = Vector3.Distance(transform.localPosition, player.localPosition);
}

public void Attack()

{
if (attack_Dealt < 5)
{
cooldown += Time.deltaTime;
if (animator)
animator.Play("Attack");
if (target == camp)
{
if (cooldown > 1f)
{
player.gameObject.GetComponent<Chiklz>().camp_Health -= 10;
cooldown = 0;
attack_Dealt++;
}
 }
else
{
if (cooldown > 1f)
{
player.gameObject.GetComponent<Chiklz>().health -= 10;
cooldown = 0;
attack_Dealt++;
}
}
}
else
{
Die();
}
}
public void Die()
{
print("Ded");
is_Awake = false;
if (animator)
animator.Play("Die");
}

public void Reset_Spider()

{
if (animator)
animator.Play("Idle");

transform.localPosition = spawn_Pos;
attack_Dealt = 0;
cooldown = 0;
}