Caption Generator

Introduction:
Image captioning involves machines to automatically describe the content of an image using an English
sentence. There are two main approaches to Image Captioning: bottom-up and top-down. Bottom-up
approaches, such as those by, generate items observed in an image, and then attempt to combine the
items identified into a caption. Top-down approaches, such as those by, attempt to generate a semantic
representation of an image that is then decoded into a caption using various architectures, such as
recurrent neural networks. The latter approach follows in the footsteps of recent advances in statistical
machine translation, and the state-of-the-art models mostly adopt the top-down approach. Our
approach draws on the success of the top-down image generation models listed above, where a
convolutional neural network (CNN) is used to encode the input image into a feature vector and Long
Short-Term Memory (LSTM) decodes the feature vector into a sequence of words. Statistical Machine
Translation (SMT) has always been a challenging task for language pairs with significant word ordering
differences and rich inflectional morphology, we present a web-based application that generates
captions of images of flicker 8k dataset and its translation. First, we use a deep convolutional neural
network to generate a vectored representation of an image that we then feed into a Long-Short-Term
Memory (LSTM) network, which then return captions. Then the resulted caption is translated into Urdu
through Google translator apart from this we maintain a parallel corpus in which manually post editing is
done.

Scope:
The System is based on Neural networks and Digital image processing. System will generate the captions
in Multiple languages English Urdu and Dutch. System will use google translator API for translating the
languages and for the analysis of image Digital image processing and Neural networks will be used.

System flow:
Image Input

Image analysis

Suggest Caption

Caption Translation