Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

N-gram Graphs: A generic machine learning tool in the arsenal of NLP, Video Analysis and Adaptive Systems. (Part II)
George Giannakopoulos1
1 University

of Trento, Italy ggianna@disi.unitn.it

04/05/2010 - SETN 2010

George Giannakopoulos

N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

In the previous episode...

N-gram Graphs and Operators Richer information Domain agnostic Generic applicability State-of-the-art performance in summary evaluation Promising for language-independent summarization Usable in classification, clustering, record linkage

George Giannakopoulos

N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

This episode

Representing behavior (using Optical Flow Proximity Graphs)

George Giannakopoulos

N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing This episode Representing behavior (using Optical Flow Proximity Graphs) Combining N-gram Graphs with the Vector Space George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing This episode Representing behavior (using Optical Flow Proximity Graphs) Combining N-gram Graphs with the Vector Space User Modeling with N-gram Graphs George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing This episode Representing behavior (using Optical Flow Proximity Graphs) Combining N-gram Graphs with the Vector Space User Modeling with N-gram Graphs The JINSECT toolkit: An open source LGPL toolkit for N-gram Graphs George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition Examples Assistive Environment George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition Examples Assistive Environment Super Market – Mall George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition Examples Assistive Environment Super Market – Mall Parking Lot George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition Examples Assistive Environment Super Market – Mall Parking Lot Vending Machines – ATMs George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition Examples Assistive Environment Super Market – Mall Parking Lot Vending Machines – ATMs Traffic George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition Examples Assistive Environment Super Market – Mall Parking Lot Vending Machines – ATMs Traffic Sports George Giannakopoulos N-Gram Graphs .

ning.com/files/ DPSX6QXHN*m77We5ozsvlC1V7uw5qyicb90jUaDEda2vMbj*cnWXOm9T8YtCG61DUl2ijCFR1n8OfnvFHaOjWokU5EXwtKxE/ sam200.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Optical Flow Image from http://api.jpg George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition and Video Indexing The Method In collaboration with Panagiota Antonakaki. NCSR Demokritos. George Giannakopoulos N-Gram Graphs .

No a priori information required George Giannakopoulos N-Gram Graphs .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition and Video Indexing The Method In collaboration with Panagiota Antonakaki. NCSR Demokritos.

NCSR Demokritos.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition and Video Indexing The Method In collaboration with Panagiota Antonakaki. No a priori information required No preprocessing steps required George Giannakopoulos N-Gram Graphs .

NCSR Demokritos.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Behavior Recognition and Video Indexing The Method In collaboration with Panagiota Antonakaki. No a priori information required No preprocessing steps required Only optical flow for feature vector calculation George Giannakopoulos N-Gram Graphs .

OFPGs) George Giannakopoulos N-Gram Graphs .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Representing Behavior — Variations Proposed Methods 1 Whole frame representation using graphs (Optical Flow Proximity Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Representing Behavior — Variations Proposed Methods 1 Whole frame representation using graphs (Optical Flow Proximity Graphs .OFPGs) Segmentation and representation using hierarchy of graphs (Symbolic) 2 George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Whole Frame Representation (1) Training Step George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Whole Frame Representation (2) Testing Step George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Graph Representation from Vectors Extraction of feature vector. f = (OFnorm . OFangle ) (1) George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Features where getBinForValue is a function that returns the name of the bin (quantization) George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Parametrically Determined Window George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs What is Used? Using N-gram Graph Operators Update operator George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs What is Used? Using N-gram Graph Operators Update operator Comparison operator George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs What is Used? Using N-gram Graph Operators Update operator Comparison operator Intersection operator George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs What is Used? Using N-gram Graph Operators Update operator Comparison operator Intersection operator All-not-in or delta operator George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Noise in Data Reasons for removal Background noise due to camera George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Noise in Data Reasons for removal Background noise due to camera Classification lies in the differences between classes George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Noise in Data Reasons for removal Background noise due to camera Classification lies in the differences between classes George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Merge: O(|Gm ||G M | + |Gm |c) George Giannakopoulos N-Gram Graphs .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs What is the Complexity of Trivially Implemented Graph Operators? Extraction from Source (of size N. |G M | = max(|G1 |. |G2 |) Size Similarity: O(1) Containment and Value Similarity: O(|Gm ||G M |) Update. |G2 |). m dimensions): O(Dwin m × N) Similarities |Gm | = min(|G1 |.

|G2 |) Size Similarity: O(1) Containment and Value Similarity: O(|Gm | log |G M |) Update. |G M | = max(|G1 |. Merge: O(|Gm | log |G M | + |Gm |c) George Giannakopoulos N-Gram Graphs . |G2 |).slower insert Extraction from Text (of length N): Dwin × N Similarities |Gm | = min(|G1 |.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Improved Complexity of Graph Operators? Using hash for nodes (or edges) Quick search .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs N-gram Graph – Value Similarity Example a 1.0 e d Result: 1.0 a 1.0 c 1.0 8.375 N-Gram Graphs George Giannakopoulos .0 4 = 1 4 + 1 8 = 0.0 b 8.0 c b 4.0 4 + 4.0 1.0 1.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Space Complexity and Related Considerations Vertices can be burdensome Edges can be burdensome Operators copying graphs Indexing increases memory requirement George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Space Complexity and Related Considerations Vertices can be burdensome Edges can be burdensome Operators copying graphs Indexing increases memory requirement Serialization George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Segmented Frame Representation — Hierarchy (1) Training Step George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs

Searching and Updating the Index

George Giannakopoulos

N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs

Segmented Frame Representation — Hierarchy (2)
Training Step

George Giannakopoulos

N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs

Segmented Frame Representation — Representing a Class
Training Step

George Giannakopoulos

N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Segmented Frame Representation — Testing Testing Step George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Frame as Symbols Example George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Symbol Graph Example George Giannakopoulos N-Gram Graphs .

Frames George Giannakopoulos N-Gram Graphs .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Size of Index vs.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Experiments and Inter-class Similarity George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Noise Removal Effect George Giannakopoulos N-Gram Graphs .

9298 F-measure 0.6741 0.9656 0.7178 0.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Experiments (Semveillance dataset) Behavior run walk abrupt Precision 0.9522 Recall 0.7746 0.9287 0.9408 George Giannakopoulos N-Gram Graphs .8231 0.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Experiments (PETS04 dataset [Fisher. 2004]) George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Supports disambiguation to real world entities in the Web.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Entity Name System (ENS) [Bouquet et al. Palpanas et al.. 2008. Provides for the reuse of identifiers.. 2008] An ENS: Maps real world entities to Unique identifiers. George Giannakopoulos N-Gram Graphs .

it/~themis affiliation : University of Trento George Giannakopoulos N-Gram Graphs .unitn.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Entity Entity in the ENS is a set of: Free-form attribute names Free-form attribute values Entity Example title : Dr firstName : Themis family_name : Palpanas homepage : http://dit.

the update of subscribers over changes to entities. informing subscribers over changes they are mostly interested in. George Giannakopoulos N-Gram Graphs . takes into account explicitly or implicitly declared user interests.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Entity Subscription Services An Adaptive Entity Subscription System (AESS) provides for: management of subscription to specific entities.

Attribute Insertion Content: affiliation→University of Trento George Giannakopoulos N-Gram Graphs .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Change Examples .Type and Content Type: Deletion (of entity) Content:(N/A) or Type: Entity Update. Attribute Update Content: title→Prof or Type: Entity Update.

defines predefined values for interest levels.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Our AESS expects user feedback for interest indication. George Giannakopoulos N-Gram Graphs . We need to create an architecture for the system. expresses interest as a real value.

free form strings) info of a change. expresses interest as a real value. defines predefined values for interest levels.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Our AESS expects user feedback for interest indication. We need to create an architecture for the system.e. represent efficiently the type and content (i. George Giannakopoulos N-Gram Graphs .

defines predefined values for interest levels. expresses interest as a real value. from user feedback. represent efficiently the type and content (i. We need to create an architecture for the system. that can use this representation.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Our AESS expects user feedback for interest indication. free form strings) info of a change. George Giannakopoulos N-Gram Graphs .e. create a user model.

e. We need to create an architecture for the system. George Giannakopoulos N-Gram Graphs . defines predefined values for interest levels. represent efficiently the type and content (i. free form strings) info of a change. take into account both simple and complex scenarios of preference. create a user model. that can use this representation. from user feedback.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Our AESS expects user feedback for interest indication. expresses interest as a real value.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Architecture Change Queue George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Architecture Change Queue User Profile DB George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Architecture Change Queue User Profile DB Adaptive Information Control George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Architecture Change Queue User Profile DB Adaptive Information Control Subscription Information Broker George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Representing Changes — Type The type of a change: deletion. merging or update updates can involve: attribute deletion. e. splitting. But what about Content? George Giannakopoulos N-Gram Graphs . 0 (abnormal) to 1 (normal) Feature Space A dimension indicative of each type/subtype of change. attribute insertion or attribute update given graded indication of normality of the change.g.

etc.) Fuzzy string matching Updatable model — if possible Graded similarity from comparison of instance to model We use Character N-gram Graphs George Giannakopoulos N-Gram Graphs . date.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Representing Changes — Content The content of a change: Instances of attribute names Instances of attribute values Problems and requirements Free form strings Other types (numeric.

00 1.00 _Bas 1.00 1.00 1.00 1.00 1.00 1.00 _irs 1.00 1.00 __Ba first name:Basil Graph Size: 39 bidirectional edges 1.00 1.00 1.00 1.00 1.00 1.00 _me_ 1.00 1.00 1.00 1.00 1.00 1.00 George Giannakopoulos N-Gram Graphs .00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 _asi 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 _ame 1.00 1.00 1.00 1. _nam __na 1.00 1.00 _rst 1.00 1.00 1.00 1.00 1.00 _t_n 1.00 1.00 1.00 ___B 1.00 1.00 _st_ 1.00 1.00 1.00 1.00 1.00 1.00 1.00 _e__ 1.00 _fir 1.00 1.00 1.00 1.00 1.00 _sil 1.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Content as a graph A character n-gram graph is a string model based on the coexistence of character n-grams in a string.00 1.00 1.00 1.00 1.00 1.00 1.00 1.

00 _Geo 1.00 1.00 _e__ 1.e.00 __na 1.00 _org 1.00 _fir 1.00 1.001.00 _irs _rge 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 _st_ _nam 1.00 1.00 1.00 1.00 1.00 1.00 1.00 _eor 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 first name:Basil first name:George Graph Size: 42 bidirectional edges 1.00 1.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Content as a graph: Updating 1. not bad scaling for normal user requirements.00 1.00 _ame 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 _me_ 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 i.00 _rst 1.00 1.00 _t_n 1.00 1.00 1.00 1.00 1.00 1.001.00 1.00 1.. George Giannakopoulos N-Gram Graphs .00 1.00 1.00 __Ge 1.00 1.00 1.00 1.00 ___G 1.00 1.00 1.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Text Size to Graph Size — Actual vs Random 0 0 2000 4000 6000 8000 0 q q q q qq qqq qq qq qq qqq qq qq qq qq qq qq qq qq qq q qq qq qq qq qq qq qq qq q qq qqq qqq qqq qqq qq q q q qq qq qq qq qq qq q q q q qq qq qq qq qq qq qq qq qq qq qq qq q q q q q qq qq qq qq qq qq q qq q qq qq qq q qq qq qq qq qq qq qq qq qq q qq qq q q q q q q q qq qq qq qqq qqq qq qq qq q q q qq q qq qq qq qqq qqqq qqqq qqq qqq qqq qq qq qq qq qq qq qq qq q qq qq qq qq qqq qq qq qqq qqq qqq q q qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq q q q q qq q qq q q qq qq qq qq q qq qq qq qq qq qq qq q q q q q qq qqqq qqq qqq qqq qqq qq q q q q qq qq qq qq qq qq qq qq qq qq qq qq q q q qq qq qq qq qq qq qq q q q qq qq q q q q q q q q q qq qqq qq qq qq qq q q q q q q qq qq qq qq qq qq q q q qq qq qq qq qq q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q qq q qq q qq qq q q q q q q q q qq qq qq qq qq qq qq qq qq qq qq qq qq qqq qqq qqq qqqq qqq qqq qq qq qq qq qq qq qq qq qq qq qq q q q q qq qq qq qq qq q qq qq qq qq q q qq qq q q q qq qq qq qq qq qq qq qq qq q q q q q q q qq qq qq qq qq qq q qq q qq qq qq q q q q q qq qq q q q q q q q qq qq qq qq qq qq q q q q qq qq qq qq qq q q q q q q q q q qq q q q qq qq q q qq q q q q q q qq qq qq qq qq qq q q q q q q qq qq qq qq qq qq qq q q qq q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq qq qq qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q qq qq qq qq qq qq q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 10000 20000 30000 40000 50000 60000 70000 q q q q q q q q q q q q q q q q q q q qq qq qq qq qq qq q q q q q q q q qq q qq qq qq qq qq q q qq q q q q qq q q q q qq q q qq qq q q q qq qq qq q q q q q q q q qq qq qq qq qq qq qq qq qq qq qq q q q qq qq qq qq qq qq qq qq qq q q q q qq q qq q qq qq qq q q qq qq qq qq qq qq qq qq q q q q q q q q q q qq q q qq qq qq qq qq qq qq qq qq qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q qq qq q qq q q q q q q q q qq qq qq qq qq qq qq qq q q q q q q q q q qq qq qq qq qq qq q q qq qq qq qq qq qq qq qq qq q qq q q q q q qq q q qq qq qq qq q q q q q q q q qq qq qq qq q q qq qq qq qq q q q q q q q q q q q q q q q q q q q q qq qq qq qq qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq qq qq qq q q q q q q q q q q q q q q q q q qq qq qq qq qq qq q qq qq qq qq qq qq q qq qq qq qq qq q q qq q q q q q q qq qq qq qq qq qq q q q q q q qq q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q qq qq qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Edges 6000 8000 10000 2000 4000 Edges 0 2000 4000 6000 8000 10000 Modeled Strings Modeled Strings Actual Text Random Text George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Mapping Content to the Feature Space Given a set of labeled changes and a new change. We have one similarity-based feature for each interest level. George Giannakopoulos N-Gram Graphs . N-gram Graph Normalized Value Similarity (NVS) Create a graph representing labeled instances for each level. We want dimensions indicative of content similarity.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments From Change to Feature Space: Workflow Overview To map a change instance to the feature space: Apply values to Type dimensions. George Giannakopoulos N-Gram Graphs . Add dimensions for Content graph similarity. Calculate Content graph similarities for every interest level.

Initialize new vector.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Update Model with New Data To update the user model with a new instance: Merge Content-based graph into corresponding interest level graph. Calculate an -SVR [Chang and Lin. 1998] regression model to estimate interest. 2001. How does this model perform? George Giannakopoulos N-Gram Graphs . Vapnik.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Experimental Setting Synthetic data for data changes 10-fold validation 1000 changes per fold Each iteration is mapped to a set of 10 changes We judge if learning occurs and its rate. if the use of content (graphs) is useful. George Giannakopoulos N-Gram Graphs .

2837) George Giannakopoulos N-Gram Graphs .2993715 (< 10−2 ) -0.02968072 (0.2601) -0.3734062 (< 10−2 ) -0.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Determining Learning Ability Table: Correlation between Emission-Iteration Number and Regression Mean Absolute Error per Subscriber Profile and Method Subscriber Type-based Attribute name-based Attribute name-value pair-based Complex Graphs Correlation (p-value) -0.5989662 (< 10−2 ) -0.3398559 (< 10−2 ) -0.03564642 (0.08581718 (< 10−2 ) -0.3484) -0.03393356 (0.

Gray line: Simple method. Figure: Type Based George Giannakopoulos Figure: Attribute Name-based N-Gram Graphs . Black line: Content aware method.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Determining Speed and Stability of Learning (1) Rate of Acceptable Errors (RAE): Percentage of errors in estimation that cannot cause ranking error.

Figure: Attribute Name-Value-based George Giannakopoulos N-Gram Graphs Figure: Complex .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Determining Speed and Stability of Learning (2) Rate of Acceptable Errors (RAE): Percentage of errors in estimation that cannot cause ranking error. Gray line: Simple method. Black line: Content aware method.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview of Senses Representation Overview of Solution Data and Experiments Importance of Representation Rate of Acceptable Errors (RAE): Percentage of errors in estimation that cannot cause ranking error.0 RAE 0.8 0 20 40 60 Iteration 80 100 Figure: Name.0 q q q q qq q q q q qq q q q q q qq q q q q q q q q q q qq q qq q q q qq q q qq q q q qq q qqq qqq q q qq qq q q qq q q q q q q q q q q q qq qq q q q q q q q q qq q q q q qq q q q 0.6 q 0.4 0.2 0. Value in Separate Graphs George Giannakopoulos N-Gram Graphs . 1. Value in One Graph Figure: Name.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview Why use it? Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview Why use it? Application suite AutoSummENG (summary evaluation) MUDOS-NG (language-independent summarization) Document Classifier Spam filter Grammaticality Estimator Entropy-based Chunk Splitter George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview Why use it? Library Character and Word N-gram Graphs N-Gram Distribution Graphs Operators Serializability Distributed Processing Examples (JADE) Multi-threading Utilities (file to string.) Interoperability (R. etc.) George Giannakopoulos N-Gram Graphs . etc. thesauri. Distribution class.

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview Why use it? Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview Why use it? Open Source LGPL Extendable Reusable Lots of examples Non-trivial implementations George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Overview Why use it? Easy to Apply Find what the vertices should be Define the neighborhood relation Use them George Giannakopoulos N-Gram Graphs .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

Summary and the Future Appendix

Outline
1

2

3

4

Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix
George Giannakopoulos N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

Summary and the Future Appendix

Almost there...

Flashback Optical Flow Proximity Graphs Proximity Graphs in a Hierarchy Combining Graphs with Vector Space JINSECT Toolkit and Library

George Giannakopoulos

N-Gram Graphs

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing

Summary and the Future Appendix

Into the future...

Indexing graphs Hierarchy and granularity criteria Expressiveness of proximity graph Recognition of n-gram graphs

George Giannakopoulos

N-Gram Graphs

and whatever you plan to make out of them. Indexing graphs Hierarchy and granularity criteria Expressiveness of proximity graph Recognition of n-gram graphs .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Summary and the Future Appendix Into the future.. George Giannakopoulos N-Gram Graphs ....

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Summary and the Future Appendix N-gram Graphs: A New Perspective Thank you George Giannakopoulos (ggianna@disi. 1 See http://tinyurl.com/2fna572 George Giannakopoulos N-Gram Graphs .unitn.it) Please provide your thoughts on the feedback form1 .

Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Summary and the Future Appendix Outline 1 2 3 4 Evolution of N-gram Graphs to Video Analysis The Optical Flow Proximity Graph Whole Frame Operators Revisited: Complexity Hierarchy in Graphs Modeling User Preferences Overview of Senses Representation Overview of Solution Data and Experiments JINSECT: A Toolkit for All Overview Why use it? Closing Summary and the Future Appendix George Giannakopoulos N-Gram Graphs .

10 0.30 George Giannakopoulos N-Gram Graphs .Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Summary and the Future Appendix Change Data Generation: User Simulation User type (Prob.02) Change type Attribute change (normal) Attribute insertion Attribute deletion Entity merge Entity split Entity deletion Attribute change (abnormal) Attribute deletion Probability 0.03) Malevolent (0.) Benevolent (0.10 0.60 0.45 0.95) Sys.30 0.(0.admin.45 0.70 0.

Default attribute (some attributes in the ENS are considered default — e. Any change concerning an attribute that contains the string “name”. Interesting Complex Critical Interesting George Giannakopoulos N-Gram Graphs . Attribute change or insertion on “isDeceased” attribute. Entity deletion. with a new value of “true”.. Default attribute deletion or normal update.g.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Summary and the Future Appendix Subscription User Simulation Subscriber Type-based Attribute name-based Attribute name-value pair-based Importance Critical Interesting Critical Interesting Critical Description Attribute deletion. the name of a person entity — while all the others non-default) update or insertion with an abnormal value. with a new value of “false”. (None) Attribute change or insertion on “isDeceased” attribute.

J.. T. An entity name system (ENS) for the semantic web.-C..ntu.csie. Fisher. Andritsos. C.tw/~cjlin/libsvm. Y. (2004).edu. A.-J. and Lin. and Velegrakis.. (2001). In ESWC. P. B.. Stoermer. R. George Giannakopoulos N-Gram Graphs . Entity data management in OKKAM. 6th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance. Palpanas. (2008). Chaudhry. H. pages 729–733. C. In DEXA Workshops. LIBSVM: a library for support vector machines. Software available at http://www.Video Analysis Adaptive Systems JINSECT: A Toolkit for All Closing Summary and the Future Appendix Bouquet. Chang. and Bazzanella. pages 1–5. (2008). The PETS04 surveillance ground-truth data sets. P.. pages 258–272. In Proc.

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.