MSc Summer Project Plan

Student Name: Lazzat Zholayeva

Student ID: 4309795
Course:

Project title:
Automatic Landmark classification using crowdsourcing images

Statement of Research Problem:

Given an image, find all of the same landmarks in a dataset automatically

Research background / motivation:

Deep Learning has evolved hand-in-hand with the digital era, which has brought
about an explosion of data in all forms and from every region of the world. This data,
known simply as Big Data, is drawn from sources like social media, internet search
engines, e-commerce platforms, online cinemas and more. This enormous amount of
data is readily accessible and can be shared through fintech applications like cloud
computing. However, the data, which normally is unstructured, is so vast that it could
take decades for humans to comprehend it and extract relevant information.
Companies realize the incredible potential that can result from unraveling this wealth
of information, and are increasingly adapting to Artificial Intelligence (AI) systems
for automated support.

Image retrieval is a fundamental problem in computer vision: given a query image,

can you find similar images in a large database? This is especially important for query
images containing landmarks, which accounts for a large portion of what people like
to photograph. The new dataset is the largest worldwide dataset for image retrieval
research, comprising more than a million images of 15K unique landmarks. All
existing approaches involve only traditional learning, I want go further and apply
deep learning.

Methodology:

- Traditional learning – user selects features

Retrieval similarities
Feature extraction
Classification
- Deep learning – features are learnt automatically. Deep learning, a subset of
machine learning, utilizes a hierarchical level of artificial neural networks to
carry out the process of machine learning. The artificial neural networks are
built like the human brain, with neuron nodes connected together like a web.
While traditional programs build analysis with data in a linear way, the
hierarchical function of deep learning systems enables machines to process
data with a nonlinear approach.

The model is going to be evaluated according to mean Average Precision @ 100

(mAP@100):
 mAP@100=1Q∑q=1Q1min(mq,100)∑k=1min(nq,100)Pq(k)relq(k)

where:

Q -is the number of query images that depict landmarks from the index set
mq is the number of index images containing a landmark in common with the query
image q -(note that this is only for queries which depict landmarks from the index set,
so mq≠0)
nq -is the number of predictions made by the system for query q
Pq(k) -is the precision at rank k for the q-th query
relq(k) -denotes the relevance of prediciton k for the q-th query: it’s 1 if the k-th
prediction is correct, and 0 otherwise

Work programme:

Work Package 1 (W1)-‘In-depth analysis of data’. How many labels and

the quality of labels
M1 (Milestone 1) – exact number of labels and sort labels according to their
quality (low, middle and good)
Work Package 2 (W2) – ‘Literature Review’. Conduct research into
‘Machine learning’and ‘computer vision’. Make notes on the key papers,
looking in particular for gaps in the research and write up.
M1 (Milestone 2) – Literature review chapter of dissertation completed.
Work Package 3 (W3) – ‘Classification methods’. Compare results of
variety of Classification methods
M1 (Milestone 2). Table of metrics of Classification methods
Work Package 4 (W4) – ‘Deep learning’. Create a self-adaptive algorithm
that gets increasingly better analysis and patterns with experience or with new
added data.
Work Package 5 (W5) – ‘User Testing’. Define relevant test subjects,
formulate plan for evaluatory testing, recruit subjects, pilot methodology,
collect data, analyse data, report results.
M5 – User evaluation test results chapter of thesis written.

Timeplan:
Week1 Week2 Week3 Week4 Week5 Week6 Week7 Week8 Week9 Week 10 Week 11 Week 12
W1
M1
W2
M2
W3
M3
W4
M4
W5
M5