JOURNAL OF COMPUTING, VOLUME 3, ISSUE 5, MAY 2011, ISSN 2151-9617

HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 47

Watermarking Algorithm for Secure Video

Retrieval
M.A. Mohd Shukran

Abstract—With the proliferation of digital multimedia content, issues of copyright protection have become more
important in video retrieval system because the copying of digital video does not result in the decrease in quality that
occurs when analog video is copied. One method of copyright protection is to embed a digital code, “watermark”, into
the video sequence. The watermark can then unambiguously identify the copyright holder of the video sequence. In
this paper we propose a new video watermark embedding algorithm that would insert watermarks into video
sequences and also detect if a given video sequence contains a given watermark. The video standard that has been
chosen is the H.264 standard as it offers a significant efficiency improvement over the previous video compression
standards. A couple of tests that can be considered representative for most image manipulations and attacks were
performed. The proposed algorithm has passed all attack tests. The watermarking mechanisms in this paper have
been proved to be robust and efficient to protect the copyright of H.264 coded video within the video retrieval system.

Index Terms— advance video coding, copyright, H.264, video retrieval, watermark.

——————————  ——————————

1 Introduction proposed. These methods take temporal redundancy

In today’s video delivery and broadcast networks,
issues of copyright protection have become more
urgent than in analog times due to increased analog
video retrieval technology. One method of copyright
protection is to use information hiding technique in
video sequences. The commonly used video
approaches used video watermarking can be
categorised as two main kinds, namely embedding
watermark in the compressed domain [1-3] and in the
uncompressed domain [5-7]. The compressed domain
approach, the typical information hiding method in [1]
embeds the watermark data into Video Length Coding
(VLC). Some other methods embed the watermark
data in the residual of the motion vectors [2], and
Lagendijk [3] proposed XDEW algorithm to embed
watermark in both I-frame and P-frame. These
methods in compressed domain can gain low
computational complexity, but they are not robust
against some common attacks such as image
manipulations and collusion attacks. In order to
improve the robustness of the video watermarking
algorithm, the uncompressed domain approach using
3D-DFT [4], 3-D DWT [5] and 3-D DCT [6] are
————————————————
 M.A. Mohd Shukran is with National Defence University of Malaysia,
Faculty of Defence Science and Technology, Kem Sungai Besi, 57000 Kuala watermark embedding algorithm that can then clearly
Lumpur, Malaysia.
into account, but they require too high computational
complexity to meet the real-time requirement.
Another problem relates to copyright protection in
video retrieval system. Content-based video retrieval
and identification is a computationally complex
problem. Digital video identification engines are
required to handle different variations while
performing their tasks. Monitoring of TV broadcasts,
for example, requires adaptation to the different
transformations which content may undergo because of
the different characteristics of transmission standards
(PAL, NTSC, DVB-T, etc). Network-aware
multimedia distribution systems increase the number of
possible scenarios in which the handling of this
dynamism is necessary. In these cases, handling that
complexity in real time requires the support of
indexing structures [7, 8] and acceleration techniques
[9, 10, 11]. Existing video retrieval systems make use
of very different features of the content to perform the
identification. Video structure, normally based on
syntactic units (shots), has been proposed to be used in
different ways by several works [12, 13]. Other
approaches reduce the video to a set of key frames,
normally based on shot structure, decreasing the
computational complexity of the task [14].
Considering these problems, this paper proposed
identify the copyright holder of the video sequence
without sacrificing the robustness of watermarking
and alleviating the computational complexities.
JOURNAL OF COMPUTING, VOLUME 3, ISSUE 5, MAY 2011, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG
This paper is structured as follows: Section 2
presents the concept of H.264 video standard. Section
3 presents the approach that is used in watermarking
algorithm in H.264 video sequence. Section 4 presents
the experimental result of image manipulations and
attacks such as basic testing, visual quality testing,
watermark sequence, double watermark insertion and
collusion tests on H.264 video sequence. Section 5
contains conclusion.
2 H.264 Video Standard
The H.264 standard (also known as MPEG4 AVC)
offers a significant improvement over the previous
video compression standards (i.e. the video sequences
get better bit rate/distortion ratios). Section 2.1 to 2.3
describes the most important parts of the H.264 video
standard used in the proposed video watermarking
system.
2.1. Sub blocks
A sub block is the basic part of a H.264 video
stream. There are two types of sub blocks: luminance
sub blocks and chroma sub blocks. Luminance sub
blocks consist of 4x4 pixels whereas chroma sub
blocks consist of 2x2 pixels. The blocks are transform,
prediction and entropy coded. There are several
different ways how a sub block can be chosen from the
prediction code [7], where the simplest one is that the
first transform coefficient of each sub block use the
first transform coefficient of neighbouring sub blocks
and the other transform coefficients do not use
prediction coding at all. Entropy coding can be done
using two different algorithms, named CAVLC
(Context Adaptive Variable Length Coding) and
CABAC (Context Adaptive Binary Arithmetic Coding)
[8].
2.2. Macro blocks
A macro block consists of 4x4 luminance sub
blocks and 2x2x2 chroma sub blocks. A macro block
contains information on what kind of prediction coding
is used in sub blocks use. Fig. 1 shows an example of
macro blocks.
Fig. 1. Macro block
48
2.3. Picture/slice management
H.264 has efficient methods in handling each
picture or slice. In its standard, it has a picture that
consist three types of slices. These slices are I slices, P
slices and B slices. A slice can be parsed and decoded
independently without using the data from other slices.
2.4. Network Abstraction Layer (NAL)
A network abstraction layer unit contains a header
and either a slice or a parameter set. The NAL units are
placed after each other in H.264 file as shown in Fig.
2.
Fig. 2. Example of NAL units
3 Proposed Approach
The proposed approach contains four components:
 Generate watermark – This will generate a random
watermark.
 Insert watermark – This will insert a watermark into
a video sequence and save the resulting watermarked
video sequence.
 Generate threshold – This will generate a threshold.
If the squared difference generated by a comparison
is higher than the threshold, there is a match.
 Compare watermark – This will, given a watermark,
a watermarked video sequence and the original non-
watermarked video sequence, insert the watermark to
the original video sequence and calculate the square
difference.
3.1. Generate watermark
In the proposed approach, the algorithm will
generate a watermark that is 240 coefficients long.
These coefficients in the watermark will be randomly
generated based on the amplitude of the watermark.
This is the same as the number of non-prediction
coded luminance coefficients in a H.264 macro block
that uses the simplest prediction coding scheme. Then
these coefficients will be stored in the text file so that it
JOURNAL OF COMPUTING, VOLUME 3, ISSUE 5, MAY 2011, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG
can be used later to integrate into the H.264 video
sequences.
3.2. Embedding watermark
The algorithm that was used to embed the
watermark in the video sequence is shown below:
vi* = vi (1 + αwi) (1)
Where v is the host video sequence to watermark, and
vi is its i:th transform coefficient. Moreover, v* is the
output watermarked video sequence, and vi* is its i:th
coefficient. w is the watermark and wi is its i:th
number. Parameter α is a constant.
3.3. Compare watermark
When the owners of a video sequence v finds an
illegal copy of it, v*, they might want to find out if it
contains a certain watermark. There are two basic ways
for them to do that:
 Extract wi* from vi* for every i by using the inverse
insertion formula and calculate the total squared
difference d as:
d = Σ(wi*– wi)2 (2)
 Insert w into v to get v** and calculate the total
squared difference d as:
d = Σ(vi** – vi*)2 (3)
If the squared difference d is less than some threshold
t, the comparison is defined as a match. A way to
generate a threshold is described in section 3.4.
3.4. Generate threshold
After calculating the squared difference d using
formula (2) of section 3.3, the next step is to generate
the threshold t and determine whether the value of d is
low enough to be considered as a match. This can be
done using the proposed algorithm:
1. Assume that the given watermark w is not the
watermark in the watermarked video sequence v*,
but some other random watermark. This means that
each coefficient wi is a stochastic variable.
2. Since the squared difference d is a sum of many
stochastic variables, it is likely using roughly a
49
normal distribution. Steps 3 to 5 are used to
calculate its mean m and standard deviation σ.
3. Generate M (where M is large) random watermarks,
w1 to wM, with the same amplitude as w.
4. Insert each wi into v and calculate each squared
difference di.
5. If M was chosen large enough, the sample mean of
the values of di is roughly equal to m and the
sample standard deviation is roughly equal to σ.
6. We now have that d is a stochastic variable with a
normal distribution with a known mean m and
standard deviation σ. This means that we can find a
t such that
P(d ≤ t) = e (4)
where e is the accepted error probability, and
should be chosen to something very small.
7. The desired threshold is t. For e = 0.01,
t = m -2.33 σ (5)
4 Experimental Result
4.1. Basic test
This testing concerns about testing the basic usage
of the program. For example, the performances of four
basic functions which are generating, inserting,
comparing and threshold calculation are tested. The
test that was performed was using the following
methods:
1. Two thousand watermarks with amplitude of
100 were generated, by executing the
following command for each i in 1 to 2000:
watermark generate 100 i.txt
2. The first of these watermarks was inserted
into video0.264 by executing:
watermark insert 1.txt video0.264 video1.264
3. A threshold was calculated by executing:
watermark threshold 1.txt video0.264
video1.264
Threshold: 6.2107
4. The following command was executed for
each i in 1 to 2000:
watermark compare i.txt video0.264
video1.264
The comparison results are shown in Figure 3:
JOURNAL OF COMPUTING, VOLUME 3, ISSUE 5, MAY 2011, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 50

16
15
of 100 was used in this experiment. Fig. 4 compares
the visual difference between the luminance of the
14
13
12

original videos and watermarked pictures. This proves

11
10
9
8
7
6
5
that the proposed method can embed watermarks into
4
3
2
various videos without degrade the video quality.
1
0
0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000

Moreover, we embedded the watermarks in various

Fig. 3. Diagram of results of basic insertion test videos shown in Fig. 4 with different amplitude
and comparison values. The graph in Fig. 5 shows the peak signal–to-
noise ratio (PSNR) values of watermarked pictures
According to Fig. 3, a dot at location (x, y) means embedded by using various amplitude values. The
that the x:th watermark got a squared difference of y. result shows that the proposed algorithm can preserve
The line represents the threshold which has the value high PSNR quality of watermarked videos by using
of 6.2107. Here, there are only a few false positives. wide range of embedding strength.
The first watermark, which is the one that was put into
the video sequence, got a much lower value for the 46.0000
mother

squared difference than the other watermark values. 45.0000 news

salesman
44.0000

4.2. Visual Quality Test 43.0000

PSNR(dB)
42.0000

41.0000
H.264 Original Watermarked
40.0000
Video H.264 Video Frames
Frames 39.0000

Mother.CIF test video in frame number 10 38.0000

37.0000
0 100 400 800 1200 1600 2000 2400 2800 3200
Amplitude

Fig. 5. PSNR in various amplitude values

PSNR = 44.8 PSNR = 39

News.CIF test video in frame number 10 4.3. Watermark sequences
The algorithm is supposed to be able to handle
watermark sequences instead of normal watermarks.
This section will test that it works. The following
commands were executed:

watermark generate 100 1.txt

PSNR = 44.2 PSNR = 39.8 watermark generate 100 2.txt
Salesman.CIF test video in frame number dir /b 1.txt 2.txt > 12.wms
10
dir /b 2.txt 1.txt > 21.wms
watermark insert 12.wms video0.264 video12.264
watermark insert 21.wms video0.264 video21.264
watermark threshold 12.wms video0.264
video12.264
Threshold: 750.041
PSNR = 44.2 PSNR = 39.2 watermark threshold 21.wms video0.264
video21.264
Fig. 4. Visual quality test of the Threshold: 807.612
watermarked video frames watermark compare 12.wms video0.264
video12.264
Square Difference: 0
In this experiment, we embedded the 240 watermark compare 12.wms video0.264
coefficients generated randomly by the program into video21.264
the I-frames of the tested videos. The amplitude value Square Difference: 847
JOURNAL OF COMPUTING, VOLUME 3, ISSUE 5, MAY 2011, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 51

watermark compare 21.wms video0.264 sequences, attempt to generate a video sequences

video12.264 whose watermark is too weak to be detected by the
Square Difference: 847 program. This method of collusion means that the
watermark compare 21.wms video0.264 pirates take the first macro block from the first pirate’s
video21.264 video sequence, the next from the second pirate’s
Square Difference: 0 sequence and so forth. Instead inserting each
watermark into the video sequence and then writing a
All these results are the expected ones. program that would swap macro blocks, the test will
insert a watermark sequence into the video sequence,
which has the same result. In more detail, the test was
4.4. Double watermark insertion test as follows:

There are many ways that a pirate can deceive the 1. Generate N watermarks, where N is the
watermark comparison algorithm. One way is to insert number of pirates, by executing the following
some random watermark in addition to the watermark for each i in 1 to N:
already in there. The following test was made: watermark generate 100 i.txt
1. A watermark 1.txt of amplitude 100 was 2. Make a watermark sequence, S.wms that
generated. includes all N watermarks.
2. 1.txt was inserted into video sequence 3. Insert S.wms by executing:
video0.264 to getvideoi.264 watermark insert S.wms video0.264
3. A couple of other watermarks, 2.txt to 6.txt of videoS.264
different amplitudes were generated and each 4. Generate the threshold as follows:
added to video1.264 to get video2.264 to watermark threshold S.wms video0.264
video5.264 videoS.264
4. The following commands was executed for 5. For each i in 1 to N, execute:
each i in 2 to 6: watermark compare i.txt video0.264
watermark compare 1.txt video0.264 videoS.264
videoi.264 6. Count the number of matches in 5.
watermark threshold 1.txt video0.264
videoi.264 Since the test generates a different number of
matches each run, it was repeated three times for each
The results were as follows: number of pirates.

Here are the results:

Table 1. Results of piracy attack in double
insertion test

Table 2. Results of piracy attack in collusion

test by swapping macro block

As can be seen in Table 1, the pirate did not manage

to defeat the watermark in this way using the
amplitudes up to 600.
As can be seen in Table 2, the implementation was
very robust against collusion attacks.
4.5. Collusion by swapping macro block

Collusion means that a group of pirates, who each

have a different watermarked copy of a video
JOURNAL OF COMPUTING, VOLUME 3, ISSUE 5, MAY 2011, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG
5 Conclusion
WIn this paper, a novel approach of video
watermarking technique for video retrieval system has
been proposed and developed. The hidden information
is inserted by altering the transform coefficients. In
section 4, some image manipulations and attacks such
as basic testing, visual quality testing, watermark
sequence, double watermark insertion and collusion
tests were performed on watermarked video sequences.
The proposed approach has passed all these attack tests
without any problem, despite using very small
watermark (amplitude of 100, which was used for
these tests, means that the transform coefficients are
only increased or decreased by 10% on average). Since
these tests can be the representative for most image
manipulations and attacks, this shows that the
watermark generated by the proposed algorithm is
robust and efficient for video retrieval system.
6 References
[1] D. Cross, B. G. Mobasseri, “watermarking for
selfauthentication of compressed video,” IEEE ICIP,
vol.2, pp. 913-916, 2002.
[2] Y. Dai, L. Zhang, Y. Yang, “A new method of
MPEG2 video watermarking technology,” IEEE ICCT,
pp. 1845-1847, 2003.
[3] I. Setyawan, R. L. Lagendijk, “Low bit rate video
watermarking using temporally extended differential
energy watermarking (DEW) algorithm,” Proc.
Security and Watermarking of Multimedia Contents
III, vol. 4314, pp. 73-44, 2001.
[4] F. Deguillaume, G. Csurka, J. O. Ruanaidh, and T.
Pun, “Robust 3D DFT video watermarking,” in
Proceedings of Security and Watermarking of
Multimedia Contents, SPIE, San Jose, vol. 3657, pp.
113-124, 1999.
[5] M. D. Swanson, B. Zhu, “Multiresolution scene-
based video watermarking using perceptual models,”
IEEE Journal on Selected Areas in Communications,
vol. 16(4), pp. 540-550, 1998.
[6] J. H. Lim, D. J. Kim, H. T. Kim, C. S. Won,
“Digital Video Watermarking Using 3D-DCT and
Intra-Cubic Correlation,” Security and Watermarking
of Multimedia Contents 3, Proceedings of SPIE vol.
4314, 2001.
52
[7] J. Yuan, Q. Tian, and S. Ranganath, “Fast and
robustsearch method for short video clips from large
video collection,” Proceedings of the 17th
International Conference on Pattern Recognition,
866–869. IEEE Comp Society, 2004.
[8] M. L. Miller, M. A. Rodriguez, and I. J. Cox,
“Audio fingerprinting: nearest neighbor search in high
dimensional binary spaces,” In IEEE Workshop on
Multimedia Signal Processing, pages 182–185, 2002.
[9] K. Kashino, T. Kurozumi, and H. Murase, “A
quick search method for audio and video signals based
on histogram pruning”, IEEE Transactions on
Multimedia, 5(3):348–357, 2003.
[10] C. Narayanaswami and M. Raghunath,
“Expanding the digital camera’s reach.” Computer,
37(12):65–73, 2004.
[11] P. N. Yianilos, “Locally lifting the curse of
dimensionality for nearest neighbor search (extended
abstract)” In SODA ’00: Proceedings of the eleventh
annual ACM-SIAM symposium on Discrete
algorithms, pages 361–370, Philadelphia, PA, USA,
2000.
[12] J. M. Sanchez, X. Binefa, and J. Vitri, “Shot
partitioning based recognition of tv commercials”
Multimedia Tools Appl., 18(3):233–247, 2002.
[13] J. S. Pedro, N. Denis, and S. Domínguez, “Video
retrieval using an edl-based timeline”, In Springer,
editor, Pattern Recognition and Image Analysis,
IbPRIA 2005, Estoril, Portugal. Lecture Notes in
Computer Science., volume 3522, pages 401–408,
2005.
[14] Y. Rui, T. S. Huang, and S. Mehrotra, “Exploring
video structure beyond the shots”, Proceedings of the
IEEE International Conference on Multimedia
Computing and Systems, page 237, Washington, DC,
USA, 1998. IEEE Computer Society.
[15] C. Y. Chen, S. Y. Chien, Y. W. Huang, T. C.
Chen, T. C. Wang, andL.-G. Chen, “Analysis and
architecture design of variable block-size motion
estimation for H.264/AVC,” IEEE Trans. Circuits
Syst. I, Reg. Papers, vol. 53, no. 3, pp. 578–593, Mar.
2006.
[16] T. Wiegang, G. J. Sullivan, G. Bjøntegaard, and
A. Luthra, “Overview of the H.264/AVC video coding
JOURNAL OF COMPUTING, VOLUME 3, ISSUE 5, MAY 2011, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 53

standard,” IEEE Trans. Circuits Syst. Video Technol.,

vol. 13, no. 7, pp. 560–576, Jul. 2003.

Mohd Afizi Mohd Shukran receives his B.InfoSyst from

University of Melbourne, Australia. Then, he did his Master of
IT in Sydney University, Australia. Finally he completed his
Doctor of Philosophy (Ph.D) in Sydney University. Currently,
he is the Senior Lecturer in National Defense University of
Malaysia. His expertises are in multimedia technology, signal
processing, data mining and information security.