See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/320703991

Artificial Intelligence and Augmented Reality based Skin Rash Diagnosis System

Article · July 2017

CITATIONS READS
0 2,120

4 authors:

Alekhya Avadhanula Prajakta Morale

San Jose State University San Jose State University
1 PUBLICATION   0 CITATIONS    1 PUBLICATION   0 CITATIONS   

SEE PROFILE SEE PROFILE

Pooja Shah Sneha kasetty sudarshan

San Jose State University San Jose State University
3 PUBLICATIONS   0 CITATIONS    3 PUBLICATIONS   8 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

AI and AR based Skin Rash Diagnosis System View project

All content following this page was uploaded by Sneha kasetty sudarshan on 29 October 2017.

The user has requested enhancement of the downloaded file.

 Artificial Intelligence and Augmented Reality
based Skin Rash Diagnosis System
Alekhya Avadhanula, Prajakta Morale, Pooja Shah and
Sneha Kasetty Sudarshan
San Jose State University
Abstract
In the field of healthcare, prevention and
cure of rare and dangerous diseases
generally requires detecting the symptoms
at right time. Many times, immediate
decisions and actions are required. It is
always better to treat a disease at an early
stage. This paper proposes the use of
Artificial Intelligence (AI) and Augmented
Reality (AR) for the detection and
prediction of the severity of a skin
disease/rash. The paper primarily on two
skin diseases - sun burns, and chicken pox.
AR will help in detecting the rash, while
artificial intelligence will help in predicting
the intensity of the rash. AI and AR provide
powerful tools. Blending these two robust
technologies will improve the diagnosis
techniques drastically.
Keywords - Artificial Intelligence;
Augmented Reality; Decision Making;
Rash Diagnosis.
I. INTRODUCTION
Patients often find it difficult to describe
their skin problems to the doctor. Sometimes
when the patient is in a remote area and if he
encounters some skin problem, it is very
difficult for him to contact the doctor and
get appropriate treatment. Even if the
problem is not serious, a person may panic
about a simple skin rash, which is not
intense and could be cured without
treatment.
The proposed augmented reality application
for medical diagnosis is empowered with the
predictive analysis of AI, and
superimposition technique of AR. The
intensity and type of the skin rash will be
detected by AR while AI will tell the user
about the next steps to be taken towards the
cure. The application is built using the Bot
AI, which will give immediate preventive
results for the analysis done by AR. The
process of detection, extraction, and
processing the data it is done by Wikitude in
AR [1].
Artificial Intelligence - Any application can
be considered an AI application, if it can
‘think’ and make decisions like humans.
How the application works is not an issue, it
just needs to make smart decisions like
humans [2].
People started exploring the field of artificial
intelligence after the second world war. So,
AI is comparatively a new technology to
explore. AI can solve the problems which
need intelligent behavior or need to make
decisions like humans. The first AI
application was developed for organic
chemistry in 1970. This program-solving
introduced AI in the field of science. The
implementation of AI in healthcare
increased due to the growth of genomic
databases, better computation power for the
processing of databases, improved natural
language processing, enabling system to act
like humans.
Augmented Reality - Augmented Reality
technology allows the users to interact with
a virtual object based on the real-world
object. Because of AR, the virtual object can
be presented to the user with a variety of
additional information in various
environments [3]. Augmented reality is a
combination of virtuality and reality. Due to
AR, we can now blend the information from
our senses and mobile devices in myriad
ways, which was impossible before [4].
Fig. 1: Order of concepts Reality and Virtuality
Augmented reality differs from its
commonly known “relative”, virtual reality
(VR). VR creates a 3D world completely
detaching the user from reality. There are
two respects by which AR is unique: users
do not lose touch of reality and it puts
information into eyesight as fast as possible.
These distinctive features enable AR to
become a driving force in the future of
medicine.
The Remainder of the paper is organized as
follows. Section II discusses the different
rash diagnosis. In chapter III we have
declared the existing systems which are
available for medical diagnosis. Chapter IV
we give an overview of the system. Chapter
V we have described our proposed System
Algorithm. chapter VI we have enlisted the
general concerns in AI and AR technologies.
Finally, chapter VII concludes the paper.
II. RASH DIAGNOSIS
A. Acne
Acne, aka acne vulgaris, is caused by the
clogging of hair follicles with oil, dead skin
cells and can be a long-term skin disease. It
usually appears in the form of pimples,
greasy skin, scarring, blackheads or
whiteheads.
Consequences of Acne:
● May lead to scars.
● Can affect the appearance of a
person and therefore lower self-
esteem.
● Can spread around the area if not
treated properly.
B. Chicken pox
Chickenpox is not just a skin disease but, is
very serious and can lead to damage of
important body organs like liver, stomach. It
is highly recommended for everyone to take
vaccination to avoid chickenpox. It is very
contagious and very likely to occur at least
once in a lifetime to those who haven’t been
vaccinated. It can be identified by symptoms
like itchy skin rashes with blister-like
appearance.
How it spreads:
● From mother to baby during
pregnancy or nursing.
● Through airborne disease carriers.
● Through skin contact.
● Through infected person’s saliva.
● Through contact with contaminated
objects.
Dangers of chicken pox:
In most cases, people infected with
chickenpox recover completely, but for
people with weak immune systems, if could
be a life threat. It is difficult to differentiate
between chicken pox and acne as both have
similar effects on skin.
How to differentiate between chicken pox
and acne?
In the early stages, the skin rash caused by
chickenpox often appears like the red rash
formed due to acne but, the spots will
develop into watery blisters in case of
chickenpox. These blisters spread very
quickly and appear all over the body unlike
acne. After four or five days, these blisters
get old and turn into scabs with a crusty
outer layer. Mostly, these blisters first occur
on chest and stomach. Fever is the main
symptom that differentiates chickenpox
from acne.
C. Sunburn
When a person is exposed to severe sunlight
or UV rays, it results in a sunburn which
causes serious skin damage and makes the
skin too hot to touch. It can be identified by
red, itchy, painful skin. Sometimes, it may
lead to blisters.
Dangers of sunburn:
It can not only be painful, but also result in
dangerous skin diseases like skin aging,
DNA damage and increases the risk of skin
cancer. If a person suffers five or more
sunburns, then there is a high chance of
increasing the risk of lifetime melanoma by
about 80%.
III. EXISTING SYSTEMS
Implementation of augmented reality for
skin diagnosis can revolutionize the
diagnosis procedure. There are very few
applications which have used augmented
reality to study a person’s skin through the
camera. In this section, we have discussed
about few such already existing applications.
First is Olay skin advisor which, tells the
age of a person’s skin by analyzing the
picture of their face. It not only tells the age,
but also focuses on specific areas of the face
and tells up to what extent they have been
damaged. Through this application, the
company aims at increasing their sales for
the cosmetic products by suggesting
cosmetics based on their facial problems. It
detects the skin damage based on the fine
lines and the changes in the texture. The
main drawback of this application is that it
cannot detect any dangerous moles or other
skin diseases.
Another interesting application named
DoctorMole was recently developed, which
can differentiate between a normal mole and
a mole caused due to skin cancer. In this
application, the users can take a picture of
their suspicious mole and input it to the
application which decided whether it is a
normal mole or a skin cancer mole [6]. The
formula used behind this is that, if it is a
normal mole, then it will have symmetric
dimensions otherwise, it can be a cancer
mole. So, depending on the dimensions of
the mole, the application alerts the user
whether it is a sign of skin cancer. The main
drawback of this application is that the
pictures and results cannot be exported from
the application to get doctor’s advice or
consent. These results cannot be substituted
for a physician’s final verdict and should be
diagnosed properly.
Similarly, there is another application called
DermaCompare which can detect signs of
skin cancer just with the help of a picture of
the mole [7]. Unlike the previous one, this
application uses augmented reality with big
data technology to compare the existing
mole with previous cases of suspicious
cancer moles. This application beats the
previous DoctorMole because it is easy to
use and the pictures can be shared outside
the application. It is very user-friendly and
can be used on smartphones and google
glass.
All the above-mentioned applications
concentrate only on a single use-case, but
cannot differentiate between different skin
problems. Our application focuses on
providing advice to the users based on the
observed skin problem and suggest them on
how to proceed for can diagnosis based on
the severity.
IV. PROPOSED SYSTEM OVERVIEW
The 2 main technologies which we
researched for the rash diagnosis are
Artificial Intelligence and Mobile
Augmented Reality.
A. Artificial Intelligence
Artificial intelligence is the ability for
machines to possess human like intelligence
and mimic the human brain [8]. Artificial
intelligence has advanced over the past years
and helps to solve many challenging
problems in computer science.
Artificial Intelligence contains a mechanism
behavior which has the capacity of
perception and learning, decision making
and intelligent reasoning. The feature
intelligent reasoning of human being comes
with a rational approach. It creates tools
which function better than humans as their
mechanism are part of mathematics which
include logic, reasoning and engineering and
are not prone to human errors [2].
Benefits of AI in Medical Diagnosis
Health is an important aspect of everyone's
life and central for our well- being.
Medicine is a domain where predictions and
immediate actions are very important [2].
Automatic system is useful for two reasons.
● Simple usual things which are time
consuming can be performed by a
machine and avoid overloading the
medical staff.
● Few medical condition are hard to be
detected by humans. An alert or a
suggestion can make a huge
difference between life and death.
For example, detecting some
cancerous cell in an image [2].
A Medical decision made by an AI machine
are not perturbed by causes that are specific
to humans like stress, reduced attention and
fatigue. It is efficient, can be repeated,
superior speed and can store a large amount
of data. These systems can select and
generate important data which will be very
useful for physicians [2].
In this paper, we use Artificial Intelligence
(AI) technology support for decision making
of the next steps required in diagnosing a
rash.
Overview of AI Mechanism for Medical
Predictions
In this paper, we discuss two AI approaches
suitable for medical reasoning along with its
advantages and disadvantages.
1. The Knowledge-Based solution
This approach implements the process of
human reasoning applied to a representation
of knowledge [2]. The important component
in this approach is the knowledge base. The
Knowledge base consists of a set of related
sentences to the natural language. Therefore,
it knows the things that an automated
decision system should know to generate an
optimal solution.
In Figure 2, is an example of medical
diagnosis of three types of Hepatitis: B,
B+D and C. Rules are created for decision
making based on some laboratory tests. The
nodes of the graph consist of the knowledge
pieces or logical operators. The edge
connects a pair of nodes.

Figure 2: Structure of a knowledge based system for
Medical diagnosis [2].
Advantages
The knowledge representation is in the form
of natural language which is easy to create
and understand and therefore readily
accepted and used in various fields. It is
easy to implement for a knowledge base
with simple rules. The results are clear, uses
logic and states whether the output is true or
false. A simple system represented by this
mechanism always produces a correct result.
Disadvantages
If the task to be solved has many data with
complex connections, the graph becomes too
complex. It becomes difficult to express the
logic to be used to decide on conclusions.
Because of its complexity, the information
content could be damaged while getting
converted into rules. If the graph is too
complex, unexpected results can be
encountered [2].
2. The Probabilistic Reasoning
If the information system is too complex
then probabilistic approach is used for
decision making. For example, let's consider
a complex scenario where a patient is
diagnosed with hepatitis B. The problem in
the condition arises with the evolution of the
patient's medical condition which involves
different evolutionary types and several
severity levels. It would be useful to have
certain predictions on the patient's evolution
for the treatment. In such a scenario, due to
the complexity representing the patient's
premises in clear rules for Knowledge-based
system diagnosis is difficult. The solution
for this situation would be provided by
statistical methods. All the standardized
premises for as many patients as possible is
stored in a database to calculate the
probability of evolutionary type and severity
level [2]. The statistical approach provides a
probability of each possible output. Bayes
theorem is used to implement the
probabilistic reasoning and the formula is as
shown below.
The theorem describes the connection
between two events Dk (which is the
diagnosis of many possible diseases) and S
(which stores patient's symptoms and
laboratory test results) [2].
Advantages
The solution offered by this mechanism is
more accurate than Knowledge-based
mechanism.
Disadvantages
An important feature of this system is that it
does not provide a clear result, it just
provides a probability. For example, it is not
sure based on this mechanism if a person
will develop a severity regarding a disease,
but it is more probable based on the system
output [2].
B. Augmented Reality for Future Medicine
The concept of Augmented Reality is very
popular in various fields. The advancement
and popularity of mobile tech devices such
as tablet PC, smartphones and google
glasses provides the community to develop a
lot of possible AR applications. Augmented
Reality applications is gaining its popularity
in Medical science field for education,
diagnosis, pre-planning and as well as in
operation room.
Benefits of using Augmented Reality for
Medicine
AR is unique from Virtual Reality in two
aspects; the user gets information as fast as
possible and as well as the information is in
eyesight, and user does not lose in touch
with reality. These feature enables AR in
becoming a driving factor for many future
medical applications [7].
Augmented Reality acts as an assistant to
surgeons in the OR. While operating in the
surgery room time and precision are two
very important requirements. AR healthcare
app can save lives by providing the right
information to the surgeons in a timely
manner. An AR developed sunglasses
creates a 3-dimensional construction of a
tumor which is visible by the surgeon
without any radiation exposure. The
application could provide useful information
regarding the operation to the surgeon [7].
Patients can visually see how a drug will
affect their body in 3D. Instead of reading
the long boring description of the medicine
on prescription bottle, the user can hover
their mobile device on the label and see its
details in 3D [7].
AR technology can make the lives of both
patients and nurses easier through AccuVein
AR technology. The AccuVein uses AR to
act as scanner that projects over the skin.
This projection shows the nurse where the
veins are present in the patient's body. This
makes finding the vein on the first stick 3.5
times more likely [7].
In case of an emergency if a person has
collapsed, getting medical care in a timely
manner is very critical for the patient's life.
It takes time to call an emergency number
and explain the situation and provide current
address to get help. In such a situation, an
Augmented Reality application can provide
the nearest hospital from its database. You
can even project the nearest hospitals on the
applications screen which provides the
directions to the nearest hospital as well [7].
Proposed system for Rash Diagnosis using
AI and Augmented Reality
In this paper, we propose the use of
Artificial Intelligence and Augmented
Reality in diagnosing a rash which will be
useful for a person living in a remote area
who does not have access to local doctor for
inspection. Our approach attempts to mimic
a doctor’s medical assessment through
technology.
System Architecture Components
1. AI
ChatBOT AI is a computer program which
conducts a conversation via textual or
auditory methods. It is designed to simulate
a human conversational pattern. In our
system, we ask few questions to the user for
diagnosis. Based on the answers given by
the user it can reason and give diagnosis for
the rash.
2. Augmented Reality
After the initial medical assessment of the
user based on chatBOT, further additional
details are required from the user to
diagnose the rash. Using mobile Augmented
Reality, we collect additional data about the
users’ rash through image processing. An
AR equipped mobile device with camera is
hovered over the infected area. Using the
image processing component of AR, we
compare the existing rash image with a
database of all rashes available to identify
the type of rash. Based on the answers
collected from chatBOT along with rash
image processing through AR we can give
diagnosis to the user.
3. Mobile Sensors
After we conclude the diagnosis based on
the severity of the rash we collect the user's
GPS location through the location sensors
available on the device. Based on the current
user's location we provide the nearest
doctor's location by communicating with
third party API’s. Additional data about the
user's health is collected from other sensors
present in the users’ device and mailed to
the doctor.
Figure 3: System Architecture diagram for Rash
diagnosis using AR and AI
V. PROPOSED SYSTEM ALGORITHM
Since Skin Disease Diagnosis system
includes variety of components including
Augmented Reality, Artificial Intelligence,
and Mobile Sensors, this section describes
each component in detail.
A. Rash Diagnosis: An Augmented Reality
System
This component basically needs to detect the
type of the rash on the skin. Numerous
algorithms use skin color as one of the basic
features for detecting and tracking human
being [2].
● Target: In our system, rash on the
skin which needs to be diagnosed,
should be the target image. The
extracted rash area is used by the
tracker to recognize type of the rash.
● Target collection: An archive storing
a collection of different types of rash
that can be recognized by the tracker.
● Tracker: The tracker analyzes the
live camera image and detects the
targets stored in its associated target
collection. Multiple trackers can be
created, however, only one tracker
can be active for recognition at any
given time.
1. Tracking Methodology
This is the first phase of Augmented Reality
in which the images of the real scene are
taken by tracking cameras. There are three
main approaches; Marker based, Model
based and Motion based approach [13].
Figure 4 shows the tree structure of Mobile
Augmented Reality Taxonomy.
Marker based approach: Due to binary
nature of a marker on any image, it is
relatively simple to detect the marker in an
image. So, in this approach, mainly the
image is identified and converted into binary
format, then it is processes to connect the
connected components. Lastly the
information of the marker is fitted to the
processed image. This is a very robust
technique and extensively used in all AR
applications.

Figure 4: MAR Taxonomy [13]
Model based approach: It requires a prior
knowledge of the scene. This comprises
edge detection of an object, texture
information and optical flow of an image.
This computation is performed using the
RANSAC algorithm, and recursively
determines which homograph is correct.
SFM Based Method: Structure for motion
(SFM) approach does not require any prior
knowledge of the scene. Instead, the
processing happens online and does not need
to rely on offline content. So, it is possible
to reconstruction of an image on the fly. Due
to this nature, this approach is very complex.
2. Detection Methodology
After tracking, the data obtained is used to
detect the real-world coordinate system. The
algorithm such as SURF, FAST, extracts
feature from the template image and the
current image [13]. Then the features from
both set are matched based on a threshold
having distinct matching. Then based the
matching, RANSAC is used to calculate the
homograph between the template image and
the final image.
With a calibrated camera and project model,
if a point has (x, y, z)r coordinates, projection
onto the image plane is (x/z, y/z,1)r. In this
system, there are two coordinate systems; The
world coordinate system W and the 2D image
coordinate system [5]. Point constraints for the
camera pose problem can be explained in the
following figure.
Figure 5: Detection Methodology [12]
3. SLAM: Actual Methodology
In augmented reality, computer vision
methods are the presence of markers and 3D
geometry in the environment. Static systems
already know the positions of the scene,
whereas dynamic systems use Simultaneous
Localization and Mapping (SLAM) [12].
SLAM is not an algorithm but it works on
set of algorithms to simultaneously find the
position/orientation of object with respect to
the surrounding and at the same time
mapping the structure of the scene is
performed. Marker based or Model based
approach is different than SLAM because
SLAM needs to work in real time. In the
rare case, we can relate the SLAM with
SFM method but the trick here is, unlike
SFM, in SLAM, processing is done on each
incoming image by the time the next one
arrives. So, the pose of the camera is
immediately available. This is must in the
case of creating MAR applications and
which is why Wikitude works perfectly
based on SLAM.
How SLAM works?
The goal of the SLAM is, using a single
camera, combine measurements of points
made over multiple frames and recover the
pose of the camera with high accuracy. The
basic working of SLAM is, firstly, the
estimating the current camera pose
(localization) is performed in real time on
one thread. Secondly, the mapping thread
runs the adjustment on the map in the
background. Once the mapping thread
completes its job, it updates the map used
for tracking, which in turn the tracker adds
new observations to expand the map [16].
Suppose, xt represents the position of agent
and mt represents a map of the environment
[7]. Objective is to compute P(mt, xt | o1:t),
so after applying Bayes rules :
B. Image Recognition
The image recognition engine is an
important component of ay AR application.
Wikitude uses Natural Feature Tracking
(NFT) principle for image recognition [13].
Steps
To incorporate and interact with as much of
the “real world” as possible, several steps
need to be executed:
1. Preprocessing: The target image that
should be tracked is analyzed.
Significant areas in the image, Feature
Points, are extracted and stored. The
extraction algorithms are already
explained in the previous chapter. This
step is executed only once per tracked
image, and can run offline and
asynchronously.
2. Feature Point Detection: In this step, the
current camera image is analyzed for
unique points.
3. Tracking: These recognized points are
then compared with the key points
generated from the target images in step
1. If they are similar then the image is
considered a positive match and then
tracked.
Local vs. Cloud Image Recognition
NFT comes in two types; offline tracking
and online cloud recognition. Cloud
recognitions is where one single image is
sent from mobile device to the wikitude
cloud. Cloud server scans for the matched
from its database and returns the best fit.
Advantage of cloud recognition is, images
can be stored in huge database through
distributed manner whereas disadvantage is
this asynchronous process can take several
seconds to complete this transaction, and has
a network delay. Contrary to this, offline
tracking solutions store the information of
the target images on the device itself and
constantly compare the camera image with
the database. This allows for a smooth
tracking experience, but also puts heavy
requirements on the computational power of
the device.
Fig 6: The Wikitude Tracking Engine
C. Machine Learning for Rash Diagnosis
Natural Language Processing(NLP), a part
of Machine Learning, can translate and
interpret human natural language input. The
machine learns and uses its algorithms
through supervised and unsupervised
learning. Supervised learning is to train the
machine to convert the input into a required
output. Unsupervised learning is to discover
new patterns in the data without any prior
information. The machine itself allocates a
function to the data through raw data
pattern. This is done via extracting the
hidden layers which are part of the data
processing layers in a neural network.

Neural Network Algorithm Fig 7: Retrieval based model

Machine learning technologies include
neural networks as one of the popular
algorithms which consist of different layers
for studying and learning data [8]. Every
hidden layer tries to detect patterns on the
image. When a pattern is spotted, the next
hidden layer is initiated and so on. The more
layers in a neural network, the more is
learned and the more accurate the pattern
detection is. Neural Network learns and
prioritizes the connections between the
different neurons each time the network
processes data.
ChatBOT Framework
A Computer program which responds to you
in a natural language via a chat interface. To
build a chatbot application, you can either
go for existing tool which takes care of
AI/conversational part or create an own
machine learning system.
Generative models
This method is harder because they generate
new responses from scratch and does not
depend on predefined set of responses. [15]
Open Domain:
In this setting, user can take the conversation
anywhere without any goal or intention. E.g.
Social media sites like Twitter and Reddit
are typically open domain
Closed domain:
The space of possible inputs and outputs is
limited because the system is trying to
achieve a very specific goal. E.g. Technical
Customer Support or Shopping Assistants
are examples of closed domain problems
where they have a goal to fix a problem.

Methodology
There are four types of chatbot models and
generally this framework categorized in a
square model [12].

Retrieval-based models
This is an easier method in which responses
are stored in a repository and response is
picked based on the input and context. It
could be a rule-based expression match.
These systems don’t generate any new text, Fig 8. ChatBot Machine Learning model [15]
they just get picked from a fixed set of
responses [15]. Figure 8 explains how ChatBot works; the
first square block is contained, not complex
and can deliver both business and user
value. In the second square, questions are
asked and the smart machine technology
generates the response. Based on this
response, Chatbot handles both the common
questions and some unforeseen situations for
which there are no predefined responses.
The last block which is a smart machine can
handle longer chats and it behaves more like
a human. But generative response increases
complexity. [13]
D. Sensors
Consider a scenario where a doctor in Delhi
want to monitor the heart rate, and vitals
data such as blood pressure, body
temperature and respiratory rate of a patient
in Mumbai. This is done with the patient
having a wearable device such as apple
watch that collects the data and sends to a
central system. The system then sends this
data to the doctor either through a mobile
push notification or a live feed to his laptop
browser.
1. Location Service Detection
Using the standard location service in the
background, mobile application can use the
location based tracking of user to detect the
medical services nearby. The system
provides location updates to background
apps when the application is in the
foreground, running in the background, or is
suspended. If the application is in suspended
mode, the system wakes up the application
and delivers update to location manager’s
delegate and then goes back to the
suspended state. While running in the
background, application should process new
location data. Ideally, application should not
consume the location service continuously
as it degrades the battery life.
2. Health Data
The HealthKit framework for iOS was
announced at WWDC 2014 as a single place
where apps can store, share, and read health-
related data. HealthKit can effortlessly share
data between fitness and health sensors.
There are two main data types;
Characteristics and samples.
Characteristics: User's birth date or blood
type.
Sample: Represents data at a point in time
which includes the user's height and weight,
steps taken, temperature, pulse rate, etc.
Workouts, which belong to the samples
category, are intended specifically for
representing runs, walks, rides, etc. Workout
data might not be useful for doctor but
characteristic and samples are the important
factors for any doctor to correct diagnosis.
3. Search Engine
Most search APIs are multi-index, multi-
type which can be applied across multiple
types within an index, and across multiple
indices with support for the multi index
syntax. There are various technologies
available; Elasticsearch is one of the highly
scalable open-source full-text search and
analytics engines. It allows you to store,
search, and analyze big volumes of data
quickly and have complex search features
and requirements. There are some important
applications which uses Elasticsearch to
store large set of data and provides some
amazing techniques to search and
autocomplete suggestions, collect logs, and
visualize data using Kibana dashboard [14].
But recently, for standalone systems, there is
another simpler yet effective approach;
Location based social networks (LBSNs) are
popular with the fast deployment of mobile
networks. There are many LBSN providers
such as Facebook places, Google Earth,
Yelp. By exploiting the application
programming interfaces (APIs) provided by
Yelp, we will concentrate on Yelp and focus
on sampling and estimating the places
nearby patients. [10]. The POI search
knowledge services that we define are
wrappers around existing web services and
produces semantic web services [11]. In our
case, we have programmed the logic for
lifting our different knowledge services that
may choose from different sources like
Yelp, Google Places since these domains are
constant. So, when the search is called on
keywords such as “skin cancer doctor”,
“acne specialist” etc. the JSON response of
the list of nearby doctors is then parsed and
shown to user using appropriate logic.
VI. CONCERNS
Even though our Proposed solution which
uses AI and Augmented Reality
technologies which theoretically proves to
be convenient, efficient and exceeding
human ability to perform calculations. The
technology should not be substituted for
human judgment. The doctor is still the best
standard for medical diagnosis [2].
Privacy and security concerns arise in this
rash disease system. The patient might not
want to discuss the course of the treatment
and want to keep it confidentially. It needs
to be handled in a secure but simpler manner
[12].
Since Android based mobile devices are
Java based, the applications run on Java
virtual machine which consumes a lot of
computing resources on the loop of large
trunk of data, which is needed by the image
processing application [13]. Compared to
Android, iPhone has a better graphical
processing unit (GPU) and supports C/C++
natively which means it can run more
efficiently [13].
There are general issues with developing
mobile Augmented reality applications
related to Navigation and tracking. It is hard
to track the location of the user inside the
building. Usability is another concern as the
mobile device screen is very small, the user
interface for Augmented Reality will not
look good and it will be inconvenient for the
user to operate with a small screen [14].
VII. CONCLUSION
We believe that this system is an extremely
good implementation of AI and AR for skin
rash diagnosis. The proposed iOS
application can be used by a person who
needs immediate help in skin rash diagnosis.
The paper primarily focuses on skin diseases
like sun burns, which can lead to skin cancer
and acne. The application will be able to
check the intensity of skin rash, and notify
the person accordingly about the further
steps needed to be taken.
Further enhancements for this application
will be developing the system for Android
users. The system collects the health
information from the health app of the
phone. In future, the system will be
enhanced to send these details to the doctor
for quicker medical help to the user on
arrival to the doctor’s clinic.
REFERENCES
[1] Martin. (Oct 24, 2013). The wikitude
Tracking Engine and Wikitude studio- The
CTOs perspective [online]. Available:
https://www.wikitude.com/the-wikitude-
tracking-engine-and-wikitude-studio-the-
ctos-perspective/
[2] Adriana Albu, Loredana Stanciu. (2015).
Benefits of Using Artificial Intelligence in
Medical Predictions [online].Available:
http://ieeexplore.ieee.org.libaccess.sjlibrary.
org/stamp/stamp.jsp?arnumber=7391610&ta
g=1
[3] Bhagya Hettige, Hansika Hewamalage,
Chathuranga Rajapaksha, Nuwan
Wajirasena, Akila Pemasiri, Indika Perera.
(2015). Evaluation of Feature-Based Object
Identification for Augmented Reality
Applications on Mobile Devices[online].
 Available:http://ieeexplore.ieee.org/docume
nt/7399005/?reload=true&arnumber=73990
05
[4] Dimitris Chatzopoulos, Carlos Bermejo,
Zhanpeng Huang, and Pan Hui. (26 April
2017). Mobile Augmented Reality Survey:
From Where We Are to Where We Go.
[online].
Available:http://ieeexplore.ieee.org/stamp/st
amp.jsp?arnumber=7912316 .
[5] Yao XIao-yang. (2011). Study on
Development of Information Security and
Artificial Intelligence [online]. Available:
http://ieeexplore.ieee.org.libaccess.sjlibrary.
org/stamp/stamp.jsp?arnumber=5750602
[6] Madhavi Dhingra; Manisha Jain; Rakesh
Singh Jadon. (2016). Role of Artificial
Intelligence in Enterprise Information
Security: A Review [online]. Available:
http://ieeexplore.ieee.org.libaccess.sjlibrary.
org/stamp/stamp.jsp?arnumber=7913142 Vivian Novak, Peter Szul. (2012). Current
[7] The Medical Futurist. (2017).
Augmented Reality in Healthcare Will Be
Revolutionary [online].
Available:http://medicalfuturist.com/augme
nted-reality-in-healthcare-will-be-
revolutionary/ [15] Henk Pelk. (Feb 16).Machine Learning,
[8] Eric Marchand, Hideaki Uchiyama,
Fabien Spindler. (18 December 2015).Pose
Estimation for Augmented Reality: A
Hands-On Survey [online].Available:
https://hal.inria.fr/hal-01246370/file/survey-
ieee-v2.pdf
[9] (April 6, 2016). Deep Learning for
Chatbots Part 1- Introduction [online].
Available:
http://www.wildml.com/2016/04/deep-
learning-for-chatbots-part-1-introduction/
[10] Yanhua Li, Moritz Steiner,Jie Bao.(19
May 2014).Region sampling and estimation
of geosocial data with dynamic range
calibration [online]. Available:
http://ieeexplore.ieee.org/document/6816726
[11] Elena Simperl, Philipp Cimiano, Axel
Polleres, Oscar Corcho, Valentina Presutti
[online]. (2012). Available:
View publication stats
https://books.google.com/books?id=zHO7B
QAAQBAJ&pg=PA708&lpg=PA708&dq=
yelp+api+algorithm+ieee&source=bl&ots=
GPnUWzFRaF&sig=c2ltn9L72SrK3f7OaX
U3zYDC5yE&hl=en&sa=X&ved=0ahUKE
wjTkt6898_TAhUB6GMKHY8ADGIQ6AE
IMzAD#v=onepage&q=yelp&f=false
[12] Julie Carmigniani. (Dec 2011).
Augmented Reality Methods and algorithms
for hearing Augmentation[online].
Available:
https://fau.digital.flvc.org/islandora/object/fa
u%3A3762/datastream/OBJ/view/Augmente
d_reality_methods_and_algorithms_for_hea
ring_augmentation.pdf
[13] Fu Yu.Real Time Augmented Reality
on Mobile Phone.[online].Available:
http://web.eecs.umich.edu/~silvio/teaching/
EECS598_2010/final_report/Fisher.pdf
[14] Stan Kurkovsky, Ranjana Koshy,
Issues in Handheld Augmented Reality
[online].
Available:http://ieeexplore.ieee.org.libacces
s.sjlibrary.org/stamp/stamp.jsp?arnumber=6
285844
Neural networks and algorithms [online].
Available:
https://chatbotsmagazine.com/machine-
learning-neural-networks-and-algorithms-
5c0711eb8f9a
[16] Osian Haines. (May 13 2016). An
Introduction to Simultaneous Localisation
and Mapping[online]. Available:
https://www.kudan.eu/kudan-news/an-
introduction-to-slam/