JOURNAL OF TELECOMMUNICATIONS, VOLUME 13, ISSUE 2, APRIL 2012 36

A new approach to annotate web images and how to search for the personal images
M. Yasrebi
AbstractThese Now a day we are facing the increase in the quantity of digital photos in the web further more numerous web sites hosting various images of individuals & groups, as an illustration we can point to the most popular ones like Flickr & the FaceBook. But the drawback we are encountering is annotation of those photos which is done based on visual features of the issue.In this article we are proposing a new approach to use the existing images in Photo sharing & social networking web sites originated from human annotation which makes us capable of searching family members pictures in the most possible efficient trend.The offered algorithm is based on the familial ties which make it possible to access distant members of an individuals family via his/her close relatives regarding the fact that the applicant must run his /her search through the cannel of close family members.Our empirical evaluation highlighted the fact that this approach can be considered as another step forward to help people in their search for their beloved ones.

Index Terms Semantic annotation, multimedia, personal image, social network, folksonomy

1 INTRODUCTION

dvanced digital capturing technologies have led to the explosive growth of image data. To retrieve the desired images from a huge amount of image data, textual query is handier to represent her/his interest than providing visually similar images for query. With the prevalence of personalized web sites such as MySpace, YouTube, FaceBook and others, personal multimedia objects of individuals in the forms of photos, videos, multimedia presentations, etc., have been growing very rapidly. As an example, in the case of photos alone, popular online photo sharing websites now host billions of photos. But, due to the significant diversity of a large amount of image data in daily life, effective image annotation is still a very challenging and open problem. To address this problem, in this paper, we propose novel approach to annotate the image by discovering the representative and discriminative visual features alike the genes for every person. Fig. 1 is an example of our basic idea. In Fig. 1 we consider every person has some specific and unique features (DNA/Gene) that will distinguish him from others. Three clusters are shown and each cluster contained images of a person. Several systems have added support for labeling photos with text-based tags in what are called folksonomies such as social networks. We use these folksonomies to put photos of each person in a separate cluster.

Fig.1 Clustering images per person based on folksonomies

Our intent behind this idea is to identify the conceptual distinctness of each gene. An issue of concern in this work is the quality of clustering since it actually makes a significant impact on the quality of search photos.

RELATED WORK

Mohammad Yasrebi is with the Department of Computer, Shiraz branch, Islamic Azad University, Shiraz, Iran.

As an example of the growth of multimedia objects on the web, there are currently a variety of popular commercial photo sharing websites on web hosting billions of personal photographs. All of them offer a limited and superficial operation in management of their library of Images like organizing photos into albums, adding photo captions, and performing simple keyword search, such as Fotolog, PhotoBucket, DeviantART, Snapfish and Flickr. Several systems have added support for labeling photos with

2012 JOT www.journaloftelecommunications.co.uk

37

text-based tags in what are called folksonomies as well as a social network of friends. Among these systems, the most advanced in terms of multimedia management is probably Flickr, which also supports user groups with group albums, nested albums, notes on regions within photos, differentiation between friends/family/other in a users contacts, some search options (date, content type, license), an extensive public API to allow users to access and manipulate data hosted on Flickr programmatically from their own applications or websites, and finally a machine tag format that allows annotating photos with arbitrary name-value pairs[2]. However, none of these popular sites directly support precise annotation of semantics and they all have only limited search options. In addition one of the most popular social network web sites is FaceBook which allows tagging as well as the annotation of people in photos, but there is no effective way to search for photos and only limited suggestions. There are two primary approaches to the annotation suggestion problem. The first is content-based, and uses features extracted directly from image to define similar images in the hopes of classifying or annotating them such as [4],[5],[6],[7],[8]. The other approach is metadatabased or context-based, which attempt to suggest candidate annotations based on annotation co-occurrence, popular annotations, geographic coordinates, keywords in the title, etc such as [9],[10]. There is also limited work available on the use of data from social networks for annotation suggestion. Shevade et al. [11] uses the WordNet and ConceptNet ontologies and makes suggestions by using content-based techniques to comparing new photos to photos in known clusters based on shared annotation.

One of the most popular social networks that gained popularity among users is FaceBook. This web site offers multiple operations with a user friendly interface. One of the aspects of the issue is the possibility of labeling the images to provide more details for the users hence friendship suggestions offered to the applicants in addition to the other features. It will be wise to mention that this kind of suggestion is base on the number of mutual friends in such a way that the more you have common friends in your list the higher number of suggestions you will receive. The main idea driven from the fact that the tags (technically called folksonomy) can be used to trace the family ties and friendship between people to search for related images.

The suggested model is based on the fact that the familial network of each person regarded as a directed graph in the form of = (V , E) . In this form Vs are defined as

family

clusters embracing the pictures of individual images, and Es are set as family relation between the members that included the following set. E={father, mother, brother, sister, son, daughter, spouse, friend}
The E Set is pointing to the fact that each person can have a direct link with 8 other people. In addition we appoint the factor contributing to the distance between members as D which defines as follows:

v = v or is n, where n is the number of edges between v , v .

zero, where
1 2 1 2

D(v1 , v2) is

3 ANNOTATION SUGGESTION
One of the big obstacles in the field of semantic web multimedia we are facing is managing its objects and specifically its photos & Images. Semantic annotation of Images gives the internet users the ability to search images in much precise and efficient way. This virtual gap is shorten by semantic annotation which users apply to label their pictures by them. In this section a new approach is introduced to give the opportunity to the users to leave no stone unturned in their search and access the photos of their family members and friends existing on web. As we know there exist a number of web sites which have the ability to perform the mentioned task by varying degrees but all of them offer a limited and superficial operation in management of their library of Images such as tagging , creating caption and classifying them in the form of albums in addition to the simple searches.

For instance the D factor between a person and his/her father set as 1 and another relation between him/her and his/her uncle is regarded as 2. The main advantage of the algorithm is that this model not only covers the relation between friends but also dominates the family bonds.

4 GIVING BIRTH TO THE GRAPH

The existing tagged images in social networks classified by the famous algorithm of k-mean. Regarding this matter we face a number of clusters named as genes in [1] article. But by applying a slight change in this trend we set a unique cluster for each individual by sorting specific genes including special chromosome like age factor and the geographic orientation of the image. As shown in Fig. 1 each cluster is recognized by a unique ID which comprised of a set of self images for each individual. In each image of each cluster the name, age and location of caption is mentioned.

38

HOW TO SEARCH FOR THE NODS?

Fig.2 Part of

family

graph

As an illustration Fig. 2 depicts 4 clusters of

family

s graph

that each belongs to an individual and for differing each cluster an item such as national Id number employed to create such a contrast. In this figure each box illustrate the related concepts to the image of each person as name, age & the location of the shot. In addition the direct edges between these clusters creates the directed graph.

After building the G family graph in order to traverse the directed graph to search for a specific nod with a unique ID, first we must calculate the distance parameter for that nod and on the next step the path of traversing should be determined. Calculation of the D factor is important because it guaranties the shortest possible way that will be chosen. For further explanation a typical query language is introduced to demonstrate the function. To traverse the graph at the beginning of the path the Id of the source nod is set and as we go forward the edges that must be traversed are set. In the querys argument the edges of the traversing path are organized according to their path by slashes which set between them. For instance if we are looking for johns uncle with the ID 1234 in Fig. 2 first by the use of a family ontology the concept of an uncle is defined and subsequently the factor D for john will be calculated and the amount 2 will acquired from the calculation. At the next stage the argument quarry will be set as follows: John1234/father/brother Now that will be sufficient to follow the direction on the traverse graph till we reach to the nod which belongs to the Johns uncle (if the nod is existed) and consequently search the related pictures belonging to the various places and ages. In this section it is also possible to apply our search in the levels of clusters themselves or even limit it by setting a boundary for a specific age or place. Last but not the least is that, the search is applicable not only in the level of graph but also in the level of a cluster.

family

In other words each cluster has maximum 8 directed edges which is designated as a member of E set. For instance in the figure 2 cluster c4 known as the father of the cluster c1 & subsequently cluster c3 is the brother of the cluster c4. The most outstanding point to regard is the quality of clustering for images in which the higher the quality is, the better result of the search will be. Also we can use the data model to manage the multimedia objects and their relationships. This model can be implemented as a relational database (ER model) or using the resource description framework RDF. Fig.3. below provides an example of ER diagram. In this diagram we can use rectangle boxes for the multimedia objects such as images which have some attributes such as Id, Name, Age and Location. Similarly, the diamonds can use for the family relations between the images of the persons such as father, mother and etc which discussed in previous section.

CONCLUSION

Indeed, an optimal solution to achieve high accuracy annotator is very difficult. This paper constitutes a new approach to give the opportunity to the users to leave no stone unturned in their search and access the photos of their family members and friends existing on web. The main idea driven from the fact that the tags (technically called folksonomy) can be used to trace the family ties and friendship between people to search for related images. The existing tagged images in social networks classified as clusers and the most outstanding point to regard is the quality of clustering for images in which the higher the quality is, the better result of the search will be.

ACKNOWLEDGMENT
The results presented in this paper are from a research project supported financially by Islamic Azad University, Shiraz Branch, Iran. Partial support of the author by the Islamic Azad University, Shiraz Branch is also appreciated.
Fig.3 An example of relational database diagram

39

REFERENCES
[1] Ja Hwung-su; Chien Lee Chou; Ching-Yung Lin; Vinsent S-Tseng. Effective semantic annotation by image-to-concept distribution model., 2011.IEEE. [2] Brendan Elliott; Z. Meral Ozsoyoglu. Annotation suggestion and search for multimedia objects on the web. CIVR 2008,Niagara falls, Ontario, Canada. [3] J. Golbeck; A. Alford; R. Alford; J. Hendler. Organization and Structure of Information using Semantic Web Technologies, in Handbook of Human Factors in Web Design, Robert W. Proctor and Kim-Phuong L. Vu (eds.). Lawrence Erlbaum Associates, NJ: 2004. [4] Jia-Yu Pan; Hyung-Jeong Yang; Christos Faloutsos; Pinar Duygulu. GCap: Graph-based Automatic Image Captioning. MDDE 2004. [5] Jia-Yu Pan; Hyung-Jeong Yang; Christos Faloutsos; Pinar Duygulu. Automatic Multimedia Cross-modal Correlation Discovery. KDD04. [6] J. Li; J. Z. Wang, 2006. Real-time computerized annotation of pictures. In Proceedings of the 14th Annual ACM international Conference on Multimedia (MULTIMEDIA 06), 2006, pp. 911-920. [7] X. Lu; P. Mitra; J. Z. Wang; C. L. Giles. 2006. Automatic categorization of figures in scientific documents. In Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06), 2006, pp. 129-138. [8] E. Izquierdo; A. Dorado. Climbing the Semantic Ladder: Towards Semantic Semi--Automatic Image Annotation using MPEG7 Descriptor Schemas. In Proceedings of the Seventh international Workshop on Computer Architecture For Machine Perception (Camp'05), 2005. [9] M. Naaman; R. B. Yeh; H. Garcia-Molina; A. Paepcke. Leveraging Context to Resolve Identity in Photo Albums, Proceedings of Fifth ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), 2005. [10] A. Graham; H. Garcia-Molina; A. Paepcke; T. Winograd. 2002. Time as essence for photo browsing through personal digital libraries. In Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 02), 2002, pp. 326-335. [11] B Shevade; H Sundaram; Min Yen-Kan. A Collaborative Annotation Framework. ICME 2005, pp. 1346-1349. M. Yasrebi is the head master of the department of computer engineering, Islamic Azad University, Shiraz branch, Iran. A researcher on the semantic annotionon on texts and multimedia objects, he has published over 10 papers in the field. Contact him at the Computer Eng. College, room No. 207, Univ. of Shiraz branch, St. Sadra, Pardis Academic Complex, Iran.