|HAI LALA MONTALBUM DA MAMA HATI US009948977B2

(12)Mpr
United
et al.
States Patent ( 10 ) Patent No.: US 9 ,948, 977 B2
(45 ) Date of Patent: Apr. 17 , 2018
( 54 ) SYSTEM , METHOD , AND APPARATUS FOR (56 ) References Cited
DETERMINING PRESENTATION TIME FOR
PICTURE WITHOUT PRESENTATION TIME U .S . PATENT DOCUMENTS
STAMP 6 ,061, 399 A * 5/ 2000 Lyons .............. HO4N 21/ 23406
375/ 240
( 75 ) Inventors: Srinivas Mpr , Karnataka (IN );
@ 6 ,081 ,299 A * 6 / 2000 Kesselring .................... 348/512
Sherman Chen , San Diego , CA (US ); 6 ,320 , 909 B1 * 11/2001 Takabatake ....... ... H04N 19 /61
Sandeep Bhatia , Karnataka (IN ) 375 /240 .25
6 ,438, 169 B1 * 8/2002 Takashima ....... HO4N 7 /52
(73 ) Assignee : Avago Technologies General IP 375/ 240 . 05
6 ,542,549 B1 * 4/2003 Tan .......... HO4N 19 /20
( Singapore) Pte . Ltd ., Singapore (SG ) 348 /419 . 1
6 ,671, 323 B1 * 12 /2003 Tahara et al. ............ 375 / 240 .26
(*) Notice : Subject to any disclaimer, the term of this 6 , 674, 801 B1 * 1/ 2004 Kim .. . HO4N 21/4307
patent is extended or adjusted under 35 6 ,738 ,427 B2 * 5 /2004 Zetts ........
348 /423. 1
.... 375/240 .28
U .S .C . 154 (b ) by 2372 days . 6 , 865,747 B1 * 3 / 2005 Mercier ........... G11B 20 / 00086
348 /E5.007
( 21) Appl. No.: 11/024,561 2003/0067989 A1* 4 /2003 Yoshinari G11B 27/ 034
375 /240 . 25
( 22 ) Filed : Dec . 22 , 2004 2003/0667989 * 4/2003 Yoshinari Hiromi G11B 27/034
375/ 240 .25
(65 ) Prior Publication Data 2003/0138051 A1 * 7 / 2003 Chen et al. .............. 375 / 240. 26
2004 / 0013270 A1 * 1/ 2004 Bae et al. ............... 381/ 1
US 2006 /0133515 A1 Jun . 22 , 2006 2004 /0165665 A1* 8 /2004 Harumoto et al. ...... 375/ 240 . 26
US 2011/0064148 A9 Mar. 17 , 2011 2006 / 0098742 A1 * 5 /2006 Meenakshisundaram
et al. ..... ... 375/240. 27
Related U .S . Application Data * cited by examiner
(63 ) Continuation -in -part of application No. 10 /340 ,061, Primary Examiner — Jessica M Prince
filed on Jan . 9, 2003, now Pat. No. 7, 787,539. (74 ) Attorney, Agent, or Firm — Foley & Lardner LLP
(51) Int. Ci. (57 ) ABSTRACT
H04N 7 / 12 ( 2006 .01) Presented herein are system (s ),method (s), and apparatus for
H04N 21/43 (2011 .01 ) determining the presentation time for a picture without a
presentation time stamp. A first and second picture are
(52) U . S . CI.
??? decoded . The first picture is a reference picture for the
.. .. .. H04N 21/4307 (2013 .01) second picture . The presentation time for the second pictures
(58 ) Field of Classification Search is computed as a function of a presentation time and a
??? . . . . . .. .. . . . . . . . .. .. . .................. HO4N 21/ 4307 decode time for the first picture .
USPC ............... .............. 375 /28 ; 240 /25
See application file for complete search history . 18 Claims, 5 Drawing Sheets

Decoda la picture
410
Decode B , picture

415
15 PTS for Yes Use the PTS
H2O
B , pidur No interpolation required
tekent ?

No
Compute PTS - DTS difference
- 42S
of lo picture
434 440
Difference is 6 Interpolated PTS =
Mald times ? DTS + 2 held times
135 445
Difference is 7 Interpolated PTS -
field Ames DTS + 2 Meld times
IN Interpolaled PTS = DTS + 2 field times
Na
Difference is 8
field times
IN Inapolated PTS = DTS + 3 field times
460
Difference is 9
470
4 Interpolated PTS 41
field times DT$ + 3 field times

Iruct polated PTS 480

DIS + 2 field times
(DEFAULT)

DONE
U . S . Patent Apr. 17, 2018 Sheet 1 of 5 US 9 , 948 ,977 B2

Video Data Frame Frame Frame Frame Frame

101 103 103 103 103 103

113 115
105 107

-- -117

TH
119

anyon EF - S
Slice Group
119

Picture
MB
111
MB MB
111
MB
111

121
SG SG SG
119 119

GOP
123
EEE Pic
121
Pic
121
Pic
121
Pic
121

VES
125
GOP
123
GOP
123
A . GOP
123
GOP
123

FIG . 1a
U . S . Patent Apr . 17, 2018 Sheet 2 of 5 US 9 ,948,977 B2

Top field FIG . 1b

Bottom field

Io B Bg P B Bo Pt
FIGURE 1c

0000000
Io Pi By B3 P B BG
FIGURE 1d
 atent Apr . 17 , 2018 Sheet 3 of 5 | US 9, 948, 977 B2

??
SDRAM202 Data Video
Processor Transport
Presentation 205 Processor
Buffer 207
201

* Audio
204 6

o 215 CDB 208

Video
Decoder
2009

BDS Frame Display

212 Buffer Engine
210 211

Video
Encoder
216

FIGURE 2 DAC 217

atent Apr. 17 , 2018 Sheet 4 of 5 US 9 ,948, 977 B2

PTS of Pin
DTS of I/Pm Prs clupo
PTS of 1/P .

L P Bois Blancs Pravo

I/Pm Bm + 1 Bm + 2 Pm + 3

FIGURE 3
 atent Apr. 17 , 2018 Sheet 5 of 5 US 9 ,948, 977 B2

Decode lo picture
105
www

Decode B ; picture
4703
415
Is PTS for
Bi picture
nresent ?

No
Compule PTS - DTS difference
of lo picture
Yes
Use the PTS
No intcrpolation required

- 425
*** 420

Difference is 6 y Interpolated PTS 440

field timcs ? DTS + 2 field times
435 445
Difference is 7 Interpolated PTS =
field times DTS * 2 field times
IN 4644
-450 Interpolated PTS = DTS + 2 field times
Difference is 8
field times I$ RFF = 0 460
IN T Yes Interpolated PTS = DTS + 3 field times
Difference is 9
2470 Interpolated PTS . 475
field times DTS + 3 field times

Iruetpolated PTS * 480

DIS + 2 ficld times
(DEFAULT)

DONE
FIGURE 4
 US 9, 948,977 B2
SYSTEM , METHOD , AND APPARATUS FOR the present invention as set forth in the remainder of the
DETERMINING PRESENTATION TIME FOR present application with reference to the drawings.
PICTURE WITHOUT PRESENTATION TIME
STAMP BRIEF SUMMARY OF THE INVENTION
RELATED APPLICATIONS Presented herein are system ( s ), method ( s ), and apparatus
for determining the presentation time for a picture without a
presentation time stamp.
This application is a continuation-in - part of U .S . appli In one embodiment, there is presented a method for
cation for patent, Ser. No . 10 / 340 ,061, filed Jan . 9 , 2003 by determining a presentation time for pictures. The method
Chen , that issued as U .S . Pat . No. 7 ,787 ,539 on Aug. 31, 0 comprises decoding a first picture; decoding a second pic
2010 . ture , the first picture being a reference picture for the second
picture ; and computing a presentation time for the second
FEDERALLY SPONSORED RESEARCH OR picture , based on a presentation time and a decode time for
DEVELOPMENT the first picture.
15 In another embodiment, there is presented a circuit for
[Not Applicable] determining a presentation time for pictures. The circuit
comprises a processor and a memory connected to the
MICROFICHE /COPYRIGHT REFERENCE processor . The memory stores a plurality of executable
instructions. The execution of the instructions by the pro
[Not Applicable] 20 cessor causes: decoding a first picture ; decoding a second
picture , wherein the first picture is a reference picture for the
BACKGROUND OF THE INVENTION second picture ; and computing a presentation time for the
second picture , based on a presentation time and a decode
In MPEG -2 video , a B -picture can be encoded by one 2525 timeIn for the first picture .
another embodiment, there is presented a system for
picture displayed before , called the forward reference pic determining presentation times for pictures. The system
ture and one picture displayed after, called the backward comprises a video decoder, a frame buffer, and a buffer
reference picture . Encoded B -pictures are data dependent on descriptor structure . The video decoder decodes a first
these reference pictures . The reference pictures are decoded picture and a second picture , the first picture being a
prior to the B -picture . One of the reference pictures, how - 30 reference picture for the second picture . The frame buffer
ever is displayed after the B - picture . stores the first picture . The buffer descriptor structure stores
As a result, the decoding order and the displaying order a presentation time and a decode time for the first picture .
can be different. The decoder system receives the pictures in The video decoder computes a presentation time for the
the decoding order. After decoding the pictures, the decoder second picture , based on a presentation time and a decode
system reorders the pictures into the display order . A display 35 time for the first picture .
engine displays the reordered pictures . These and other features and advantages of the present
A multimedia program can include a video , multiple invention may be appreciated from a review ofthe following
audio channels , and data channels . The video , audio chan detailed description of the present invention , along with the
nels, and data channels are synchronized to make the pro accompanying figures in which like reference numerals refer
gram intelligible . Time stamps are used to achieve this 40 to like parts throughout.
synchronization .
As noted above , the video pictures have a decoding order BRIEF DESCRIPTION OF SEVERAL VIEWS OF
and a display time. Decoding time stamps (DTS ) indicate the THE DRAWINGS
decoding time. Presentation time stamps (PTS ) indicate the
display order. The video encoder encodes the DTS and PTS 45 FIG . 1a illustrates a block diagram of an exemplary
into the video program . The DTS and PTS are 33 -bit Moving Picture Experts Group (MPEG ) encoding process ,
numbers that are driven by a 27 MHz clock . in accordance with an embodiment of the present invention .
The DTS and PTS add overhead to the transmitted stream . FIG . 1b illustrates an exemplary interlaced picture , in
Encoders encode the time stamps on a periodic basis. MPEG accordance with an embodiment of the present invention .
allows as much as 700 ms between time stamps . For a 50 FIG . 1c illustrates an exemplary sequence of pictures in
display rate of 30 interlaced pictures / sec (60 fields /sec .), display order.
there can be as many as 41 consecutive fields without time FIG . 1d illustrates an exemplary sequence of pictures in
stamps . decoding order.
A decoder system can interpolate the DTS and PTS values FIG . 2 illustrates a block diagram of an exemplary circuit
of the pictures without time stamps. The decoding and 55 for decoding the compressed video data , in accordance with
presentation times are spaced evenly apart . The decoder an embodiment of the present invention .
system can interpolate the decode time because the pictures FIG . 3 is a timing diagram describing the presentation
are provided in decode order. The decoder system interpo- time and decoding times for exemplary pictures.
lates the presentation timeby evaluating the type of pictures . FIG . 4 is a flow diagram for determining a presentation
However, interpolating the PTS values involves decoding 60 time for a picture , in accordance with an embodiment of the
numerous parameters and pictures. This can be unfeasible present invention .
where a large number of consecutive pictures are without
time stamps . This is particular complex in cases where there DETAILED DESCRIPTION OF THE
are a large number of consecutive B -pictures . INVENTION
Further limitations and disadvantages of conventional and 65
traditional approaches will become apparent to one of ordi FIG . 1a illustrates a block diagram of an exemplary
nary skill in the art through comparison of such systemswith Moving Picture Experts Group (MPEG ) encoding process of
 US 9, 948,977 B2
video data 101, in accordance with an embodiment of the 103 (a predicted picture ) are limited to prediction from
present invention . The video data 101 comprises a series of portions of no more than two reference pictures 103. It is
pictures 103 . Each picture 103 comprises two -dimensional noted that pictures 103 used as a reference pictures for a
grids of luminance Y, 105 , chrominance red Cm, 107 , and predicted picture 103 can be a predicted picture 103 from
chrominance blue Ch, 109, pixels . 5 other reference pictures 103 .
FIG . 1b is an illustration of a picture 103 . A picture 103 The macroblocks 111 representing a picture are grouped
can either be captured as an interlaced picture or as a into different slice groups 119 . The slice group 119 includes
progressive picture . In an interlaced picture 103 , the even the macroblocks 111, as well as additional parameters
numbered lines are captured during one time interval, while
the odd -numbered lines are captured during an adjacent time 10 forming thethepicture
describing slice group . Each of the slice groups 119
form the data portion of a picture
interval. The even -numbered lines form the top field , while structure 103 . The picture 103 includes the slice groups 119
the odd -numbered lines form the bottom field of the inter as well as additional parameters that further define the
laced picture . picture 103 .
Similarly, a display device can display a picture in pro The picture 103 also includes a header 103h storing
gressive format or in interlaced format. A progressive dis- 15 various parameters that relate to the picture. The parameters
play displays the lines of a picture sequentially , while an
interlaced display displays one field followed by the other may include, for example , a picture structure indicator
field . In a special case , a progressive picture can be dis - (picture /top - field / bottom - field ), a progressive picture
played on an interlaced display by displaying the even sequence flag (usually comes in transport layer ), a progres
numbered lines of the progressive picture followed by the 20 sive picture flag, and a repeat first field parameter. It is noted
odd -numbered lines, or vice versa . that in varying standards there may be additional or less
Referring again to FIG . 1a , the two- dimensional grids are parameters.
divided into 8x8 blocks, where a group of four blocks or a The progressive picture parameter indicates whether the
16x16 block 113 of luminance pixels Y is associated with a picture has been encoded as a progressive picture . If the bit
block 115 of chrominance red Cm, and a block 117 of 25 is set, the picture has been encoded as a progressive picture .
chrominance blue Ch pixels. The block 113 of luminance If the bit is not set, the picture has been encoded as an
pixels Y , along with its corresponding block 115 of chromi- interlaced picture .
nance red pixels Cm, and block 117 of chrominance blue The picture structure parameter specifies the picture struc
pixels C , form a data structure known as a macroblock 111. ture corresponding to the image buffer. Pan scan vectors
The macroblock 111 also includes additional parameters, 30 specify the displayable part of the picture . The aspect ratio
including motion vectors , explained hereinafter. Each mac - indicates the aspect ratio of the image buffer. The decode and
roblock 111 represents image data in a 16x16 block area of display horizontal size parameters indicate the decoded and
the image. the displayable horizontal sizes of the image buffer, respec
The data in the macroblocks 111 is compressed in accor - tively.
dance with algorithms that take advantage of temporal and 35 The repeat first field is a one -bit parameter that specifies
spatial redundancies . For example , in a motion picture , whether the first displayed field of the picture is to be
neighboring pictures 103 usually have many similarities. redisplayed after the second field , for an interlaced
Motion causes an increase in the differences between pic sequence . The repeat first field is used to display motion
tures, the difference being between corresponding pixels of picture standard material on a National Television Standard
the pictures , which necessitate utilizing large values for the 40 Committee (NTSC ) display . The motion picture standard
transformation from one picture to another. The differences material includes 24 progressive pictures per second . The
between the pictures may be reduced using motion compen - national television standard displays 60 fields per second. To
sation , such that the transformation from picture to picture display the motion picture standard material on the NTSC
is minimized . The idea ofmotion compensation is based on display, a technique known as 3 :2 pull down is used .
the fact thatwhen an objectmoves across a screen , the object 45 In 3 : 2 pulldown, the even numbered lines and odd
may appear in different positions in different pictures , but numbered lines form the top field and bottom field , respec
the object itself does not change substantially in appearance , tively . Two pictures or four pictures from the motion picture
in the sense that the pixels comprising the object have very standard material correspond to 2 .5 picture periods or 5 field
close values , if not the same, regardless of their position periods for the NTSC display . For every two progressive
within the picture .Measuring and recording the motion as a 50 motion picture standard pictures, four fields are generated .
vector can reduce the picture differences . The vector can be One of the fields from the second picture is repeated . The
used during decoding to shift a macroblock 111 of one repeat first field indicates whether the first field is repeated .
picture to the appropriate part of another picture , thus 1 , B2, B3, P , B5, B6, and P4, FIG . 1c , are exemplary
creating movement of the object. Hence , instead of encoding pictures. The arrows illustrate the temporal prediction
the new value for each pixel, a block of pixels can be 55 dependence of each picture. For example, picture B2 is
grouped , and the motion vector, which determines the posi- dependent on reference pictures In , and P , . Pictures coded
tion of that block of pixels in another picture, is encoded using temporal redundancy with respect to exclusively ear
Accordingly, most of the macroblocks 111 are compared lier pictures of the video sequence are known as predicted
to portions of other pictures 103 (reference pictures ). When pictures (or P -pictures ), for example picture P , is coded
an appropriate (most similar, i.e . containing the same 60 using reference picture 1o . Pictures coded using temporal
object ( s )) portion of a reference picture 103 is found , the redundancy with respect to earlier and / or later pictures of the
differences between the portion of the reference picture 103 video sequence are known as bi-directional pictures (or
and the macroblock 111 are encoded . The location of the B -pictures ), for example, pictures B2 is coded using pictures
portion in the reference picture 103 is recorded as a motion 1o and P Pictures not coded using temporal redundancy are
vector. The encoded difference and the motion vector form 65 known as I-pictures, for example Io. In the MPEG -2 stan
part of the data structure encoding the macroblock 111 . In d ard , l-pictures and P -pictures are also referred to as refer
the MPEG - 2 standard , the macroblocks 111 from one picture ence pictures.
 US 9, 948,977 B2
The foregoing data dependency among the pictures B -pictures , B1, B2, the video decoder 209 decodes the
requires decoding of certain pictures prior to others. Addi pictures, I., B1, B2, P3, B4 , B5, P6 . . . , in an order that is
tionally , the use of later pictures as reference pictures for different from the display order. The video decoder 209
previous pictures requires that the later picture is decoded decodes each of the reference pictures, e .g ., I. , P3, prior to
prior to the previous picture . As a result, the pictures cannot 5 each picture that is predicted from the reference picture . For
be decoded in temporal display order , i.e. the pictures may example, the video decoder 209 decodes 10 , B1, B2, P3 , in the
be decoded in a different order than the order in which they order, 10, P3, B1, and B2. After decoding I, and P3, the video
will be displayed on the screen . Accordingly, the pictures are decoder 209 applies the offsets and displacements stored in
transmitted in data dependent order, and the decoder reor B , and B ,, to the decoded I , and P? , to decode B , and B ,.
ders the pictures for presentation after decoding . I., P1, B2, 10
B3, P4, B5, B6, FIG . 1d , represent the pictures in data " In order to apply the offset contained in B , and B2, to the
dependent and decoding order, different from the display decoded and Pz in
I, and P3, the video decoder 209 stores decoded I.
memory known as framebuffers 210 . The display
order seen in FIG . 1c. engine 211 , then displays the decoded images onto a display
Referring again to FIG . 1a , the pictures are then grouped
together as a group of pictures (GOP ) 123 . The GOP 123 15 device , e . g . monitor, television screen , etc ., at the proper
also includes additional parameters further describing the time and at the correct spatial and temporal resolution .
GOP. Groups of pictures 123 are then stored , forming what Since the images are not decoded in the same order in
is known as a video elementary stream (VES) 125 . The VES which they are displayed , the display engine 211 lags behind
125 is then packetized to form a packetized elementary the video decoder 209 by a delay time. In some cases the
sequence 130 . The packetized elementary sequence 130 20 delay timemay be constant. Accordingly , the video decoder
includes packets of varying length . 209 writes the decoded images in frame buffers 210 so that
The packets 130 include a header 130h storing a number the display engine 211 can display them at the appropriate
of parameters. The parameters can include decode time time. Additionally , the video decoder 209 writes parameters
stamps (DTS ) and presentation time stamps (PTS ). The DTS associated with the decoded pictures in buffer descriptor
indicates the decoding order. The PTS indicates the display 25 structures 212 . These parameters include the Presentation
order. Encoders encode the time stamps into the PES headers Time Stamp (PTS), the Decode Time Stamp (DTS), repeat
on a periodic basis . MPEG allows as much as 700 ms first field , and top field first parameters, to name a few .
between time stamps. For a display rate of 30 interlaced The display engine 211 uses the PTS to determine when
pictures/ sec (60 fields/ sec .), there can be as many as 41 to display the decoded pictures . However, not every picture
consecutive fields without time stamps. Flags in the PES 30 hash a PTS . Where a pictures does not have a PTS, the video
header 130h indicate the presence of the PTS and DTS .
The packetized elementary sequence 130 is then pack decoder 209 calculates the presentation time and writes the
etized into uniform length packets 135 . Each packet 135 is presentation time for the picture in the buffer descriptor
then associated with a transport header, forming what are structures 220 . The video decoder 209 calculates the PTS
known as transport packets. The transport packets 135 can 3535 based on the DTS and PTS of future reference pictures , and
be multiplexed with other transport packets 135 carrying repeat first field parameters , and writes the calculated pre
other content, such as another video elementary stream 125
sentation time to the buffer descriptor structures 220 . The
or an audio elementary stream . The multiplexed transport
computation will now be described .
packets form what is known as a transport stream . The Referring now to FIG . 3, there is illustrated a graph
transport stream is transmitted over a communication 40 describing the decoding and presentation times for an exem
medium for decoding and displaying. plary set of pictures , Pn, . . . Im , Bm + 1, Bm +2, Pm +3, Bm +4 ,
FIG . 2 illustrates a block diagram of an exemplary circuit Bm + 59 . . . , where the subscripts indicate the decoding order .
for decoding the compressed video data, in accordance with Picture Pn is from a previous sequence that the video decoder
an embodiment of the present invention . Data is received 209 did not encounter , and the video decoder 209 locks onto
and stored in a presentation buffer 201 within a Synchronous 45 the sequence starting 1o . The display sequence is Pm , Bm + 19
Dynamic Random Access Memory (SDRAM ) 202. The data B . , I. , B , B . , P . .= m + 3 *

can be received from either a communication channel or It would not be possible to determine the PTS value of
from a local memory, such as, for example , a hard disc or a picture B . . from P . , because picture P , is from a previous
DVD . sequence that the video decoder 209 did not encounter. It
The data output from the presentation buffer 201 is then 50 may be possible to compute the PTS value of picture Bm + 1
passed to a data transport processor 205 . The data transport from Pm + 3. However, in order to compute the PTS for Bm + 1,
processor 205 demultiplexes the transport stream into pack
information about picture Bm + 2 and Pm +3 is needed . If the
etized elementary stream constituents , and passes the audio number
transport stream to an audio section 215 and the video of B pictures is increased , the number of frames that
transport stream to a video transport processor 207 and then 55 would
" need to be examined also increases.
Picture Bm +1 is predicted from picture Im . Because Im is
to a compressed data buffer 208 for a video decoder 209 . The
audio data is then sent to the output blocks , and the video is displayed after Bm + 19 Im is the future reference picture for
sent to a display engine 211 . Bm + 1. Therefore , Im and information associated with Im are
The display engine 211 scales the video picture , renders stored in frame buffers and buffer descriptor structures ,
the graphics, and constructs the complete display . Once the 60 during decoding of Bm + 1
display is ready to be presented , it is passed to a video As can be seen , I and P pictures are displayed when the
encoder 216 where it is converted to analog video using a subsequent I or P picture is decoded . Therefore , the differ
digital to analog converter (DAC ) 217 . ence between the PTS and DTS associated with an I or P
The video decoder 209 decodes at least one picture, 10, B1, picture indicates the total number of displayable fields
B2, P3, B4, B5 , P6, . . . , during each picture display period , 65 between the previous I or P picture and the current I or P
in the absence of Personal Video Recording (PVR ) modes picture . For example , the difference between the PTS and
when live decoding is turned on . Due to the presence of the DTS associated with Im indicates the number of pictures
 US 9 ,948,977 B2
between the previous I or P picture , Pn, and In . The differ display times and writes the foregoing for the PTS in the
ence between the PTS and DTS is the time required to buffer descriptor structures. If 1 , the video decoder 209
display 6 fields. Therefore : calculates at 465 the presentation time as the DTS of I/Pm + 2
field display times and writes the foregoing for the PTS in
PTS for Bm + 1= DTS of picture Im + 2* Time to Display 5 the buffer descriptor structures.
Field If the difference is 9 times the display time for a field at
It is preferable to calculate the PTS for a Bm + 1 from the 470 , the video decoder 209 calculates at 475 , the presenta
future prediction picture, Ign , because the future reference tion timeas the DTS of I/P + 3 field display times and writes
picture is normally stored in a frame buffer when decoding the foregoing for the PTS in the buffer descriptor structures .
a predicted picture . Therefore , no additional memory is 10 If the difference is other than 6 -9 field display times, the
required . Additionally , the PTS for Bm +1 can be determined video decoder 209 calculates at 480 , the presentation time as
without information regarding subsequent pictures , Bm + 2 the DTS of I/P 7 + 2 field display times and writes the
and Pm + 3. foregoing for the PTS in the buffer descriptor structures .
Below is a decision table, indicating the decisionsmade The embodiments described herein may be implemented
by the video decoder 209 , based on the PTS and DTS of a 15 as a board level product, as a single chip , application specific
future reference picture , the repeat first field parameter. integrated circuit (ASIC ), or with varying levels of the
decoder system integrated with other portions of the system
as separate components. The degree of integration of the
PTS - DTS Presentation decoder system will primarily be determined by the speed
Im /Pm picture RFF – Bm + 1 Time Picture Sequence 20 and cost considerations. Because of the sophisticated nature
6 fields X DTS + Pn = F , Bm + 1 = of modern processor, it is possible to utilize a commercially
( 3 frames) 2 Field F , Bm + 2 = available processor, which may be implemented external to
time F an ASIC implementation . Alternatively , if the processor is
7 fields 0 DTS + Pn = F, Bm + 1 = available as an ASIC core or logic block , then the commer
( 3 .5 frames ) 2 Field F , Bm + 2 = 25 cially available processor can be implemented as part of an
time TBT
7 fields - DTS + Pn = F , Bm + 1 = ASIC device wherein certain functions can be implemented
(3.5 frames ) 2 Field TBT, Bm + 2 = in firmware .
8 fields 0
time
DTS +
F
Pn = TBT, While the present invention has been described with
(4 frames ) 3 Field Bm + 1 = F, reference to certain embodiments , it will be understood by
time Bm + 2 = TBT 30 those skilled in the art that various changes may be made and
8 fields - DTS + Pn = F, Bm + 1 = equivalents may be substituted without departing from the
(4 frames ) 2 Field TBT, Bm + 2 = scope of the present invention . In addition , many modifica
time BTB tions may be made to adapt a particular situation or material
9 fields X DTS + Pn = TBT, to the teachings of the present invention without departing
(4 .5 frames) 3 Field Bm + 1 = TBT,
time Bm + = TBT 35 from its scope . Therefore , it is intended that the present
> 9 fields X
Default: invention not be limited to the particular embodiment dis
DTS + 2 Field
time closed , but that the present invention will include all
embodiments falling within the scope of the appended
T = Top Field ,
B = Bottom Field ,
claims.
F = Frame/(Pair of Fields),
40
40

X = Don 't Care The invention claimed is:

1 . A method for determining a presentation time for
Referring now to FIG . 4 , there is illustrated a flow pictures, the method comprising :
diagram for determining the presentation time for B -picture, decoding a first picture with a decoder ;
Bm + 1, following an I/ P picture, Im or Pm , in accordance with 45 decoding a second picture, the first picture being a refer
an embodiment of the present invention . At 405 , the video ence picture for the second picture; and
decoder 209 decodes picture I/Pm , writing the decoded determining a presentation time for the second picture
picture to the frame buffers, and parameters associated with using a difference between a presentation time for the
the picture to buffer descriptor structures. At 410 , the video first picture and a decode time for the first picture .
decoder 209 decodes picture Bm + 1. At 415 , the video 50 2 . The method of claim 1 , wherein the first picture is a
decoder 209 determines whether pictureB includes a future reference picture for the second picture.
PTS . If picture Bmt, includes a PTS , the video decoder 209 3 . The method of claim 1 , wherein the first picture is
uses the PTS as the presentation time and writes the PTS to associated with a presentation time stamp indicating the
the buffer descriptor structures at 420. presentation time for the first picture , and a decode time
If the picture B .sub .m + 1 does not include a PTS , at 425 55 stamp indicating the decode time for the first picture .
the video decoder 209 computes the difference between the 4 . The method of claim 1, wherein the presentation time
PTS and DTS associated with picture I/P .sub. m . If the for the second picture is further determined using a repeat
difference is 6 or 7 times the display time for a field , at 430 first field parameter.
or 435 , respectively , the video decoder 209 calculates at 440 5 . A circuit for determining a presentation time for pic
or 445, the presentation time as the DTS of I/ P.sub .m + 2 field 60 tures, the circuit comprising :
display times and writes the foregoing for the PTS in the a processor; and
buffer descriptor structures. a memory coupled to the processor, the memory storing a
If the difference is 8 times the display time for a field , at plurality of executable instructions, wherein execution
450 , the video decoder 209 examines the repeat first field of the instructions by the processor causes :
parameter for Bm + 1 and determines ( 455 ) whether the 65 decoding a first picture ;
parameter is 0 or not. If 0 , the video decoder 209 calculates decoding a second picture , the first picture being a
at 460 the presentation time as the DTS of I/Pm + 3 field reference picture for the second picture ; and
 US 9 ,948,977 B2
10
determining a presentation time for the second picture 12 . The system of claim 9, wherein the presentation time
using a difference between a presentation time for for the second picture is further determined using a repeat
the first picture and a decode time for the first first field parameter .
picture . 13. The method of claim 4 , further comprising determin
6 . The circuit of claim 5 , wherein the first picture is a 5 ing whether the second picture is associated with the repeat
future reference picture for the second picture. first field parameter, based on the presentation time stamp of
7 . The circuit of claim 5 , wherein the first picture is the first picture and the decode time stamp of the first
associated with a presentation time stamp indicating the picture .
presentation time for the first picture , and a decode time
stamp indicating the decode time for the first picture . 10 14whether
. The method of claim 4 , further comprising determin
8 . The circuit of claim 5 , wherein the presentation time for ing a third picture is associated with the repeat first
field parameter, based on the presentation time stamp of the
the second picture is further determined using a repeat first first picture and the decode time stamp of the first picture.
field parameter.
9 . A system for determining a presentation time for 15 . The circuit of claim 8, further comprising determining
pictures , the system comprising: 15 whether the second picture is associated with the repeat first
a video decoder configured to decode a first picture and field parameter, based on the presentation time stamp of the
decode a second picture, the first picture being a first picture and the decode time stamp of the first picture.
reference picture for the second picture ; 16 . The circuit of claim 8, further comprising determining
a frame buffer configured to store the first picture ; and whether a third picture is associated with the repeat first field
a buffer descriptor structure configured to store a presen - 20 parameter, based on the presentation time stamp of the first
tation time for the first picture and a decode time for the picture and the decode time stamp of the first picture .
first picture , 17 . The system of claim 12 , further comprising determin
wherein the video decoder is configured to determine a ing whether the second picture is associated with the repeat
presentation time for the second picture using a differ - first field parameter, based on the presentation time stamp of
ence between the presentation time for the first picture 25 the first picture and the decode time stamp of the first
and the decode time for the first picture . picture .
10 . The system of claim 9, wherein the first picture is a 18 . The system of claim 12, further comprising determin
future reference picture for the second picture .
11 . The system of claim 9 , wherein the first picture inis ing whether a third picture is associated with the repeat first
associated with a presentation time stamp indicating the 30 field parameter, based on the presentation time stamp of the
first picture and the decode time of the first picture .
presentation time for the first picture , and a decode time
stamp indicating the decode time for the first picture. * * * * *