Multimedia Database

Management System
Wei Tsang Ooi
CS731
 MMDBMS :
Querying Interface,
Indexing
and
Buffer Management
 Why MMDBMS ?
Concurrency Control
Recovery
Privacy
Query Support
Version Control
 Example of MMDBMS
Digital Library
News-On-Demand
Video-On-Demand
Music Database
Telemedicine
Geographic Information System
 No Intergration

id size fps title filename

000001 530M 30 soam l6.mpg

000002 450M 30 tibor l7.mpg
000003 600M 30 parag l5.mpg
000004 510M 30 wei l4.mpg
Semi-intergrated

BLOB

000001b70ae9902...
Fully Intergrated

 
index
 

buffers
 

 storage
Nature of Multimedia Data
Large amount of data
Time sensitive
Vague matching
 Database Components

 Query
Interface
Query
Processing

storage buffer index

manager manager
Query Interface
and
Processing
 Problems
Needs support for :
• temporal and spatial relations
• “natural” interface
• fuzzy query

SQL is not suitable.

 13 Temporal Relations
Allen 83
before starts

meets equals

overlaps during

finished by finishes
overlapped by
contains
met by
started by
after
 Spatio Relations
Papadias, Theodoridis 96
• Topological Relations
• Directional Relations
Topological Relations
disjoint inside

meet equal

overlap covers

covered by
contains
Directional Relations
North
NW NE

West East

SW SE

South
Spatio-temporal Relations
overlap-after
meet-during
...
 Querying
Image
Audio
• Music
• Sound
• Speech
Video
 Querying Image
Common approach
• allow query by sketches (color,
shape, texture) or examples.
• perform matching by Feature Vectors
F = (v1, v2, ... vn)
• e.g. Color Histogram
 Querying Image
Exisiting Systems :
• QBIC
• VisualSEEK
• PhotoBook
• Virage
• FourEyes
 Querying Music
Hawley 93
• Input by MIDI Keyboard
• Measure relative pitch (U, D, S)
• Perform exact match with existing
database.
 Querying Music
Ghias, Logan, Chamberlin & Smith
95
• Input by humming
• Extract relative pitch
• Perform approximate matching
 Querying Music
Chou, Chen, & Liu 96
• Query by chord
• Represents musics by chord
• C Am Em F C Am Em F ...
• Perform fuzzy match
 Querying Music
Chen & Chen 98
• Query by “rhythm” (tempo ?)
• Represents musics by rhythm
• | ¶¶— | ¶·¶·| ¶¶¶¶ | ...
• Perform fuzzy match
 Querying Sound
Wold, Blum, Keisar & Wheaton 96
• Analyze audio to extract features
• loudness, pitch, brightness, bandwidth
and harmonicity
• Segment the audio to pieces
• Feature Vector Matchings
 Querying Speech
Hauptmann & Witbrock 97
Informedia
• Use speech recorgnition to convert
audio to text
• segment audio using silence detection
Query by speaking keywords
 Querying Video
Query by speech
• Informedia
Visual approach
• VideoQ
 VideoQ
Chang et. al 97
• User can sketch objects
• Specifies
• color
• texture
• shape
• motion
• duration
• camera zoom and pan.
 VisualQ Example

Titanic Sinking Someone Skiing

Indexing
 Indexing
Requirements
• support spatio-temporal operations
• support fuzzy matches
 Indexing Images
N-dimentional indices for feature
vector
Well studied in DB/CG community
Two examples :
• VP-tree
• R-tree
 VP-tree

R
Q
P

R PQS VWTU

T
S

U
W
V
 VP-tree

R
Q
P

R P U

T
S VW
S Q T
U
W
V
R-tree
 Indexing Audio
Audio are modeled as strings
Inexact match is needed
Common indices for string search
can be used
Example
• PAT-tree
 PAT-tree

ab b c

abc c abc c

ababc abc babc bc c

 Indexing Video
Treat time as third dimension
We can use any multidimension
indexing structures
Buffer Management
 Buffer Management
Minimizing response time
Ensuring continuity and
synchronization

Prefetching & Replacement

 Glossaries
Presentation
Media Stream
Media Object
 Buffer Management
Relevance Based
Distance Based
Buffer Consumption
 Relevance Based
Moser, Kraiß & Klas 95 : L/MRP
Definition:
• State = (curr obj, skip)
• Relevance : State  Real
 Relevance Based
Prefetch :
• future objects with highest relevance
Replace :
• old objects with lowest relevance
 Example (5,+2)
Relevance

History
Referenced
Skip

Time Scale
 Distance Based
Özden, Rastogi & Silberschatz 96
Definition
• distance of a client C is the offset
between C and the immediate
following client.
 Distance Based
Replace :
• blocks consumed by clients with
largest distance
 Example

C1 C2 C3 C4
 Example

C1 C2 C3 C4
 Example

C1 C2 C3 C4
 Buffer Consumption
Wu & Yu 97
Definition :
• Buffer consumption = amount of
buffer used x total time
 Buffer Consumption
Result :
• Minimizing buffer consumption
increase system throughput
• Increase retrieval rate of current
stream is better then prefetching
next waiting stream
Open Problems
 Querying
Query video by action ?
Query music by
• relative pitch + chord + rhythm ?
Continuing quest to improve
accuracy
 Indexing
Well studied problem
Indexing for approximate string
search ?
 Buffer Management
Should take into consideration:
• object dependencies (MPEG)