This action might not be possible to undo. Are you sure you want to continue?
Intel Confidential—Internal Use Only
Please note: The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. It is intended to be a general overview of media and video technology in its current state, intended for use by Intel employees, and is not intended to be a complete and definitive treatise. E-mail firstname.lastname@example.org with any questions.
Video Fundamentals Contents
1. Introduction ................................................................................................................. 3 2. Video Distribution Methods ........................................................................................ 3 3. What Enables Video Playback on a PC? ..................................................................... 4 4. How is Video Displayed? ............................................................................................ 5 5. The Technology Behind High Quality Video Playback ............................................ 12 6. The Auditory Experience........................................................................................... 26 7. Future Technologies .................................................................................................. 28 8. Summary.................................................................................................................... 30
Intel Confidential—Internal Use Only
Video playback is one of the most common usage models on a PC. From watching videos on YouTube to downloading full length movies from online retailers or viewing the latest hit Blu-ray movies, the user desires the best video experience possible. The PC video playback experience is dependent on the hardware, post-processing technologies, display, and audio capabilities of a user’s notebook or desktop PC. Intel® HD graphics with Intel® Clear Video HD technology has hardware acceleration for high definition (HD) playback as well as post-processing capabilities that enhance the PC video viewing experience with sharper images, precise color control, and advanced support for a wide range of digital displays. This document focuses on a PC’s ability to enhance and provide the best user experience when viewing videos on a display. Many of the technologies presented in this document can be used to describe different aspects of photograph viewing and editing; however, the purpose of this document is to educate and be a reference for a wide audience on the concepts of video playback and related technologies. This document will describe video post-processing, display, and audio technologies as well as possible future directions of video technology. This document should be used in conjunction with other material available on competition.intel.com to provide a detailed resource enabling effective conversations about PC video playback capabilities.
2. Video Distribution Methods
The primary difference in how a consumer views his or her videos is the delivery method of the video content. Common delivery methods for video are physical storage mediums (DVDs and Blu-ray discs), television (cable and satellite providers), online video, or personal content a consumer created with a camcorder. Both standard definition (SD) and high definition (HD) videos can be distributed on physical storage mediums which are read and played by a Blu-ray player, HD-DVD player, DVD player, or VCR. Videos distributed on physical storage mediums are typically professionally created movies but can also be documentaries, home videos, television series, or other types of video. Physical storage mediums are portable and can generally be used on any television or PC with hardware capable of reading the storage medium. Television content refers to a broad distribution of movies, television programming, sporting events, and many other types of video. Television content is delivered over a cable land line or satellite link to the consumer and can be either SD or HD, depending on the preferences of the consumer. Online video is any video content which is downloaded or streamed from the Internet in order to be viewed, generally (although not exclusively) on a PC. Streamed video can be either live or pre-recorded, but in both cases, the video is ‘streamed’ from the Internet for immediate viewing. Downloaded video content is pre-recorded and downloaded fully onto a user’s PC to be viewed at a later time. Examples of online content providers are Intel Confidential—Internal Use Only Page 3
recording of sporting events. 3D graphics is very similar to the CPU in that it uses general purpose hardware such as the shader pipelines in order to perform a wide range of tasks. more and more online content providers are switching some or all of their content to higher resolutions and even full HD at the cost of higher storage and bandwidth requirements. which provides movie and television programming over the Internet. creating and editing personal video content. Personal video content is created by the end user. in general terms. With a PC. The 2D graphics engine should not be confused with the PC’s 3D graphics capability.Netflix. Graphics solutions. 3. however. For more information please refer to Graphics Fundamentals and CPU Relevance. In contrast. home movies. Regardless of what hardware is used to perform video playback. home videos. which is the ability to render and output objects to a display in realtime for immediate viewing. What Enables Video Playback on a PC? A PC’s video playback capability typically resides as a part of a PC’s graphics solution or 2D graphics engine. performance is likely to lag behind using specific fixed function hardware. a user can enjoy video distributed in many ways: online video streaming and downloading. On most graphics solutions the 3D and 2D capabilities are entirely separate as many of the 2D capabilities such as the video decode pipeline and other video post-processing technologies are implemented as fixed function hardware. Examples of personal video content are wedding videos. Intel Confidential—Internal Use Only Page 4 . and YouTube. however. watching a video does not require real-time rendering since it is playing back pre-recorded content. Personal video content is not typically recorded in a professional manner and video postprocessing plays a greater role in providing the best viewing experience possible. and watching live TV from a satellite or broadband cable connection by using a TV tuner card. which provides user generated video content. have two parts to them: the 3D graphics capability and the 2D video capability. viewing 3D movies with a 3D enabled display. There is a common misconception that 3D gaming and graphics performance is tied to video playback performance. Online content is sometimes lower quality than standard definition video. Fixed function hardware is hardware which is designed specifically to perform one task. it is important to understand and remember that the performance is not related to 3D graphics performance and cannot be measured or assessed by metrics like 3DMark Vantage from Futuremark. etc. Some graphics solutions do use the shader pipelines and other 3D graphics components to run common video post-processing procedures but because the 3D hardware is general purpose. physical storage mediums like Blu-ray or DVD movies.
or 1080p.e. a high quality display must be part of a user’s PC. If a 720p HD video is shown on a display with 1080p resolution then the video must be scaled up from the original 720p resolution to the resolution of the larger display resulting in less overall video quality. One example of a high definition display is a PC monitor that supports resolutions above 640x480. Bandwidth requirements for transmitting high definition content scales with the amount of pixels being transferred over the connection. This capability is generally called upscaling. or downscaling. the proper display must be used. there are many different video display resolutions. Current high definition displays come in resolutions of 720p. 1080i. Video broadcasting companies have standard definition and high definition versions of many channels. Examples of devices with low screen resolution are Mobile Phones. also referred to as HDTVs. This is generally called downscaling. Standard definition is the most common resolution in which video is displayed. To address this. Intel Confidential—Internal Use Only Page 5 . Low resolution devices are becoming one of the main avenues for video consumption as they allow the user to view videos anywhere instead of being confined to the living room or home office. the bandwidth required to transfer 1080p video is approximately double that of 720p video and more than four times the bandwidth required for standard definition 480p video. The resolution of standard definition video is 480p (640 x 480 pixels) and is still the most widely used viewing resolution for television. For example. then an HD display that supports 1080p resolution must be used. SD and HD videos have to be scaled down in order to be properly displayed on these low resolution devices. Handheld Devices). In addition to the common display resolutions above. the video to the proper resolution which the display supports. Some of the most commonly known display resolutions are SD. popular before the introduction of LCD flat panel displays. For example. video processing can take care of displaying video on any size of display by upscaling. The display can show video in many different formats and resolutions. How is Video Displayed? In order to have the best video viewing experience. full HD 1080p is not yet being used by broadcast companies due to bandwidth limitations. High definition videos are displayed at any resolution higher than standard definition. In order to watch a video in its intended resolution.4. An example of a standard definition display is a cathode-ray tube TV. Low resolution devices are a broad category of devices which have display resolutions lower than standard definition 480p. PDAs. and Smartphones. If a smaller display than 1080p is used then only a small portion of the video (equal to the resolution of the smaller display) is seen. the original video must be scaled down from the HD resolution to the resolution of the smaller display (resulting in data loss). however. and Low Resolution (i. HD. if the desired output and visual experience is a 1080p high definition video. however. In summary. many videos also come in other resolutions.
These terms are used to describe the technology in the display as well as the capabilities of the display which can ultimately have an impact on the video viewing experience. From a compute resource perspective. Gamut describes the range of colors that can be created by a three color Red. the screen is drawn from top to bottom in logical order. Page 6 Intel Confidential—Internal Use Only . Progressive: Progressive describes a way in which an image can be drawn on the display. and Blue (RGB) display. and Blue can display 2^30 or approximately 1. The benefit of interlaced content is that the bandwidth required to transmit them is halved over progressive formats because odd lines are sent in one frame and then the even lines in another frame following the first. Gamut: The gamut of a display describes the total amount of colors the display is able to produce affecting the quality of images presented on the display. Odd lines are interlaced with the even lines for a completed picture. and one for the left eye. Displays enabled for 3D media are able to display two different visual streams offset from one another: one for the right eye. 720p. for example. 480p. When drawing an image on a display using interlaced scanning the rows are drawn for odd lines top to bottom before the rows are drawn for even lines top to bottom. Typical aspect ratios are widescreen 16:9. Stereo 3D (stereoscopic): Stereo 3D or stereoscopic 3D refers to the ability to project or display the illusion of a 3D image. Green. a touch display is not much different than a mouse since the touch display provides the XY coordinates instead of the mouse. For example. The combination of the streams on the display and viewing the display with polarized glasses gives the illusion to the human eye of a 3D image with depth.Display Concepts and Technologies There are many terms that are used to describe a display and determine the video capabilities of that display. means that for every 16 pixels of width there are 9 pixels of height in an image displayed on it. 480i and 1080i are examples of interlaced video display formats. a 30-bit display with 10-bits of color information each for Red.073 billion unique colors. When drawing an image on a display using progressive scanning. Aspect Ratio: The aspect ratio is important in a display because it describes the format in which video is seen or the ratio at which the screen is drawn. and 1080p are examples of progressive video display formats. A widescreen aspect ratio of 16:9. The following terms are commonly found on any description of a display and its associated hardware. Green. or standard screen 4:3. Touch: Touch displays recognize the touch of human skin and have the ability to register the exact location of the touch and map the location with specific functions such as buttons where the touch acts much like a PC mouse. Interlaced: Interlaced describes a way in which an image can be drawn on the display.
2006 and has a total bandwidth of 10. CbCr.org/manufacturer/terms. (Note: There are some Speed HDMI 1. The following technologies can affect how video is displayed. however. This list is by no means a comprehensive list of all connection types that are capable of transmitting video. February 2.jpg.2 HDMI Version 1. Intel® Clear Video HD technology currently supports all of the following technologies with the exception of HDMI 1.3 Deep Color: HDMI 1.3 was released June 22.As mentioned previously.factorydirectav. 2010 Intel Confidential—Internal Use Only Page 7 . 1 2 Source: http://en. a poor display can trump any enhancements made to the video because the display hardware is of bad quality. February 2. display types.org/wiki/HDMI.aspx.3 cables that can support ALL HDMI 1. eight channels of digital audio. and a Consumer Electronics Control (CEC) connection which allows HDMI devices to control one another when needed.1 HDMI is fast becoming the preferred connection type for high definition media. the display is one of the most important aspects of the video viewing experience. and various display technologies which affect video playback.4.3 cable. In order for a company to license HDMI.3 o HDMI 1. sRGP.wikipedia. Regardless of the video processing techniques used. This section focuses on display inputs.2 Gbit/s and a maximum resolution of 2560x1600p75 (2560x1600 at 75Hz).4 features except for HDMI Ethernet.3: HDMI version 1. 36-bit. 2010 3 Source: http://shop. High Definition Multimedia Interface (HDMI): HDMI media transmission allows for the transfer of high-definition video. LLC. it must pay a royalty to HDMI Licensing.hdmi. and 48-bit Deep Color in the following formats: xvYCC. February 2.3 supports 30-bit. 2010 Source: http://www.) Figure 2: HDMI 1. HDMI can encrypt signals using HDCP in order to provide content protection between the transmitting device and receiving device.com/images/hdmi3. Display Inputs The following is a list of commonly found display inputs and connection types used when connecting a computing solution to a display in order to transmit video and audio data. that capability will be supported in the future.
3 features HDMI 1.org/wiki/File:101783-UKHDMI-6-GB. o HDMI 1.jpg.3 supports output of DTS-HD Master Audio as well as Dolby TrueHD. 2009 and has a resolution up to 4096x2160p24 which is used in digital theaters but with HDMI 1.4 has the following enhancements: Figure 1: Worlds first HDMI 1.wikipedia. 2010 Intel Confidential—Internal Use Only Page 8 .4 Audio: HDMI 1. DisplayPort: DisplayPort is a royalty free standard for transferring digital data.4 Automotive Connection System: HDMI 1. 4 Source: http://en. February 2.4: HDMI version 1.4 cable4 o HDMI 1.4 has the ability utilized at home.3. HDMI 1. DisplayPort is capable of using HDCP content protection in order to protect and encrypt data transmission between devices. One key differentiating factor of DisplayPort versus HDMI or DVI is that it supports both external (PC to TV) and internal (Laptop Display) connections.4 adds support for an Audio Return Channel to the specifications of HDMI 1. o HDMI 1. HDMI Version 1.3 Audio: HDMI 1.4 also uses a Type E connector for use within a vehicle.3 currently only supports 3D stereoscopic playback up to 1080i resolution. o HDMI 1.4 supports Full HD 3D playback in 1080p24 (1080p at 24Hz) and in many other formats.3 3D: HDMI 1. o HDMI 1.o HDMI 1.4 3D: HDMI 1.4 introduces a cable which is specifically made for connections within an automotive system and can withstand the stresses of the automotive environment.4 was released May 28.4 Ethernet: Supports 100 Mb/s Ethernet between HDMI devices. In addition to HDMI 1.
i-chart. top. and HDMI 1. 2010 Intel Confidential—Internal Use Only Page 9 . VGA is the minimal standard which all PC graphics devices currently support. February 22. DVI is capable of using HDCP content protection. Figure 4: DVI connection.Figure 3: DisplayPort cable5 DVI: DVI is a digital video output connection which is used mostly with personal computer displays.3 connection.co.belkin. o Dual-Link DVI: Dual-Link DVI provides greater bandwidth in the form of a second data link for displays which output at high resolution.com/images2/S-HDMI-DVI-1.jpg Source: http://www.nz/images/belkin-VGA-cable. although support may be phased out as digital displays become more common. February 2. bottom6 Video Graphics Array (VGA): VGA is a graphics output standard introduced by IBM in 1987.jpg. VGA has 15 pins and can output 640x480 video with no audio. Expect native VGA support to be phased out in favor of newer display standards like DisplayPort in the next few years. DVI is a video only format and contains no ability to transfer audio.smelectronics. Figure 5: VGA cable7 5 6 Source: http://www. 2010 7 Source: http://www.com/pressroom/releases/uploads/assets/media/hi-res/F2CD00X-XX.jpg.
Modern televisions usually have their own video post-processor and TV tuner hardware embedded as part of its hardware platform. Digital Light Processing (DLP): DLP displays contain an optical semiconductor called a DLP chip which was invented by Texas Instruments. and are able to be constructed in greater sizes than realistic with a CRT. LED-backlit LCDs are much thinner than current Cold Cathode Fluorescent Lamp-backlit LCDs because the amount of space required for LED backlighting is much smaller than CCFL. portable. Monitors therefore assume that all technology related to the post-processing of any image is completed by the PC to which it is connected. greater dynamic contrast. and a wider color gamut than traditional CCFL-backlit LCD displays. Monitors are typically displays which are connected directly with a PC with their only purpose being the display of the data which is transferred to them. LCoS technology allows for lower cost.Display Types Videos are displayed on many different devices which can be broadly categorized as either televisions or PC monitors. Televisions typically make no assumption that the driving input takes care of any video post-processing or any other type of video enhancement. CRTs are primarily being replaced by LCD displays. Content Protection 8 9 Source: http://en. LCDs are known to be lightweight. the CRT is the oldest display type which can display media8. February 8.org/wiki/Liquid_crystal_display.org/wiki/Cathode_ray_tube#History.wikipedia.wikipedia. There are many display types which videos are viewed on. Liquid Crystal on Silicon (LCoS): LCoS is a technology similar to DLP but instead of using mirrors. there are some important differences. legacy Cathode Ray Tubes to ultra-thin and light weight LED-backlit LCD displays: Cathode Ray Tube (CRT): Invented in 1897. heavy. Although there are some similarities between televisions and monitors.9 LED-backlit LCD: LED-backlit LCDs are currently also called LED TVs by television makers such as Samsung and Panasonic. 2010 Intel Confidential—Internal Use Only Page 10 . CRTs were the standard viewing display for video until relatively recently. The DLP chip has up to two million mirrors (1920 x 1080 resolution (1080p) = 2073600 pixels) which are coordinated to reflect an image onto a screen. higher resolution displays. Liquid Crystal Display (LCD): LCDs are the most commonly found types of display today. LED-backlit LCDs also have lower power consumption. Displays range from bulky. Compared to other display technologies. LED-backlit LCDs should not be confused with true LED displays. February 2. 2010 Source: http://en. LCoS uses liquid crystals.
UDI. and DisplayPort as of version 1. Other Display Technologies These are technologies which may be located on the display or describe different aspects of the displays such as specific technologies which it may have which affect the video quality: Dual Simultaneous HDMI: Dual Simultaneous HDMI is the ability for a video device to simultaneously output two HDMI signals at once to two different displays or other devices. Flickering occurs when the image does not appear to blend into a single image but is still running fast enough that there only appears to be a flicker in the image. green. PAVP reduces processor utilization by off-loading decode functions from the CPU to the chipset. In a display. HDCP currently supports and protects content being transferred over DVI. GVIF. and blue. half the scene is shown and then the other half of the scene is shown immediately following the first half. The bit color depth is related to total number of colors able to be displayed by Colors = 2^BCD. the refresh rate of the display is generally fast enough that to the human eye the difference between the two is not noticed and it is perceived as a solid. magenta. De-flicker filtering removes flicker from interlaced media playback on a display. When interlaced content is displayed. blended image.3.Content protection is the ability for devices to protect the copyright of content by encrypting the data when it is transferred from one device to another such as a PC to its associated monitor. Brilliant Color adds three more: cyan. De-Flicker Filtering: Flickering occurs on screens which display interlaced content. There are several different content protection schemes which help protect the copyright of content and provide data encryption: Protected Audio video Path (PAVP): PAVP protects the data path within a PC during playback by encrypting the compressed video data when it sends it to the chipset and ensures that PCs which have hardware decode acceleration are utilized. Displays typically only have three color channels: red. Intel Confidential—Internal Use Only Page 11 . DPCP: DPCP is a 128-bit AES encryption content protection scheme designed specifically for DisplayPort and was developed by Philips. Brilliant Color: Brilliant Color is a technology from Texas Instruments. and yellow for greater color accuracy. HDMI. Bit Color Depth: The Bit Color Depth is the number of bits which are used to represent the color encoding of a single pixel in an image. High-Definition Content Protection (HDCP): HDCP protects audio and video content over a number of different interfaces from being copied.
Graphics devices which output underscanned images output an image at a lower resolution so it can fully fit onto the display area.org/wiki/Dither#Which_types_to_use. 10 Source: http://en. and the technologies in the video decoding pipeline on a graphics solution which affect a user’s video playback experience. Underscanned images have black margins between the edge of the image and the edges of the display area. 4-bit color) have the illusion of being in a much larger color palate (i. The Technology Behind High Quality Video Playback A number of key terms come up in any discussion on video capabilities on a PC platform.wikipedia. Overscanning. blue. In the figure below. Video Concepts The following concepts relate to the way in which color is described in a video as well as ways in which video can be changed or enhanced by video post processing.e. Dithering inserts colors which are part of the palate in such a way that to the human eye the way in which they blend together gives the illusion of a much richer color palate than is actually present in the image. February 3. Spatial Temporal Dithering: Dithering is the process of making an image which has a small color palate (i. the video decoding pipeline. Hue: Hue describes the overall color tone as the color closest to red. on the other hand. This section will help with understanding the concepts of video playback. yellow. the image of the cat on the left uses a very limited color palate while the image on the right is the same image with dithering applied. green. Figure seven shows an image in six different hue tones. occurs when the image output to the display is larger than the display and only a portion of the full image is seen. It is important to understand basic definitions from color theory as well as basic definitions relating to video post-processing in order to understand what technologies such as Intel® Clear Video HD can actually do. 2010 Intel Confidential—Internal Use Only Page 12 .e. or any combination of two of the four. 32-bit color). Figure 6: Dithering10 Underscanning/Overscanning: Underscanning is where the image displayed is smaller than the actual screen area of the display. 5.
wikipedia.Figure 7: Image with different hue tones11 Colorfulness: Colorfulness is the perception of how much Hue is present in an image. in figure seven the image on the middle right would be described to be more colorful than the image on the top left. An image with an excess of Hue would be described to be very colorful.jpg. 2010 Source: http://en. Figure 8: Chroma12 Saturation: Saturation describes if the color in an image is intense or dull. January 27. 2010 Intel Confidential—Internal Use Only Page 13 . Figure 9: Image at different saturation levels13 11 12 Source: http://en. Decreasing saturation is seen from top left to bottom left. In figure eight the image on the right has its chroma increased 50% in relation to the image on the left.org/wiki/File:Surfing_in_Hawaii_unmodified.jpg. Chroma: Chroma refers to how colorful a color is in reference to white.wikipedia.org/wiki/File:Hue_shift_six_photoshop. January 26. In figure nine the original image is top left. Increasing saturation is seen from top right to bottom right.
13 14 Source: http://commons.borisfx. Figure 11: Image with increased brightness on the right side versus left15 Contrast: Contrast describes the level of blending of objects in an image.jpg. January 26. January 26. January 26. 2010 Intel Confidential—Internal Use Only Page 14 . Figure 11 shows a full image with half of it having increased brightness. 2010 15 Source: http://www.org/wiki/File:Saturation_change_photoshop.com/images/fx/brightness_contrast.wikimedia. Figure 12 shows six different levels of contrast with decreasing contrast from top left to bottom left and increasing contrast from top right to bottom right. Figure 10: Grayscale14 Brightness: Brightness describes how much an object seems to be radiating or reflecting light.jspx.imagingassociates.com. 2010 Source: http://www. In images with low contrast it is harder to distinguish one object from another and in some cases they blend completely together.jpg. Lightness: Lightness describes if the color in an image is light or dark. One example of color going from dark to light is grayscale as seen in figure 10. Lightness is related to the brightness in an image.au/color/testpatterns. Images with high contrast have very apparent differences in color and it is easy to tell the difference between one object and another.
Figure 13: Image at different levels of gamma17 Artifact: Artifacts are departures from the expected or desired output of a display. January 26. Artifacts are anomalies which are not linked to a software failure but to the failure of the hardware to correctly output or display the video as it was originally created. to jagged lines in an image. Not only does compressed video takes less space to store.Figure 12: Image at different levels of contrast16 Gamma: Gamma is a variable which is changed to adjust the brightness in an image. but it requires less bandwidth to distribute.wikipedia. 2010 Source: http://en. to completely corrupt screens which are unusable. The data that is lost cannot be recovered upon 16 17 Source: http://commons. January 26. Figure 13 shows four different levels of gamma and their effect on an image. 2010 Intel Confidential—Internal Use Only Page 15 .jpg. In a lossy compression algorithm some data is lost in the compression process in order to maximize the storage space reduction. transmit or download from the PC to the TV or over the Internet. There are two types of video compression: lossy compression and lossless compression.org/wiki/File:Contrast_change_photoshop.wikimedia. Video Codecs and Formats Video compression is necessary to reduce the amount of storage space required for video.jpg.org/wiki/File:GammaCorrection_demo. Examples of artifacts range from bad pixels (wrong color).
but stores other file information in the wrapper. The current version of DivX is based on the H.pdf.18 H. H.de/imagecom_G1/assets/pdfs/csvt_overview_0305. software programs sometimes use a wrapper around a codec such as DivX which uses H.264 as its codec. The following video codecs and wrappers are typically used in video distribution: DivX: DivX is a video codec software package which takes care of the encoding and decoding of videos and audio. DV: Digital Video format is used primarily in video cameras.264 is a codec used in modern video products such as Blu-ray Disc.264 in order to view the video on an H. MPEG-4 AVC: MPEG-4 AVC (Advanced Video Coding) is a group of technologies defining the compression of video and audio digital data. videos compressed into a lossy compression format take less storage space than videos compressed with a lossless compression algorithm. One usage example would be converting a downloaded video from VC-1 format to H. Video transcoding is the process of converting one compressed video format to another.264 codec whereas previous versions relied on older codec’s such as MPEG-2. MVC: MVC is an extension to the H. January 27. In addition to Codecs. 2010 Intel Confidential—Internal Use Only Page 16 . Codec stands for COmpression-DECompression and is a full algorithm for the encoding and decoding of video and audio data. MPEG-2: MPEG-2 is a group of technologies defining the compression of video and audio digital data. The common algorithms used for compressing video files are called Codecs. however. This involves first decoding the file and then encoding it in the new format. Video decoding is the playback process where the compression is essentially reversed and the video file is sent to the display.decompression.264 only enabled player. 18 Source: http://ip. Video encoding is the process of taking the raw video format and compressing it to a smaller size using a Codec. In lossless compression. some newer lossy compression algorithms are very efficient such that the amount of data lost is very small while still realizing a large storage space benefit.hhi. MPEG-2 is the standard for digital video broadcasting over cable and satellite as well as movies on DVD. A related video concept is called transcoding.264: H.264 is also known as MPEG-4 part 10.264 video format which allows for the transmission of stereoscopic 3D video. Typically. There are. the original video is completely retrievable from the compressed video without any data loss.
video streaming from the Internet. In order to playback videos. a video container is used to pack all files together which are vital to video playback including the subtitles. and outputs the video to the display. and chapter information. For example. The Video Decoding Pipeline The video decoding pipeline is the process which is applied to playback a video stored on a disc. MPEG-4. Windows Media Video 9. MPEG-4.19 Typically. in the video encoding-decoding pipeline.mov and . The video decoding pipeline takes the encoded video from all of these sources and applies the correct decoding algorithm based on the video’s codec. audio streams. Inverse Discrete Cosine Transform (iDCT): Allows for the decompression of audio and video files which were compressed with a discrete cosine transform. video streams.org/wiki/Microsoft_Silverlight Intel Confidential—Internal Use Only Page 17 . and AVC video encoding. or any other method of viewing videos. Context-Adaptive Binary Arithmetic Coding (CABVC): Lossless compression technology found in H. Context-Adaptive Variable-Length Coding (CAVLC): Lossless compression technology found in H. VC-1: VC-1 is a video codec which is an alternative to the MPEG-4 H. VC-1 is found in Blu-ray Discs. A high level block diagram of the video decoding pipeline is shown below in figure 14. scales the video to the appropriate aspect ratio and resolution. and AVC video encoding.264. Video Compression and Decompression Techniques The following algorithms are found in video codecs and are responsible for the compression and decompression of video. 19 See http://en.264.wikipedia. VC-1 is known for its ability to be decoded faster than H. and Microsoft Silverlight. a Discrete Cosine Transform (DCT) is applied to the video on the encoding side when the video is first stored onto a disc.264 standard.264. the files in the container must be decoded in order to be displayed. the Inverse Discrete Cosine Transform is applied to the video stream from the disc undoing the Discrete Cosine Transform so that the video can be output to a display and viewed. applies video post processing techniques. To watch the video. video being broadcast over cable. Examples of video containers are . Developed originally by Microsoft and distributed as an open standard.avi.
Most recently. Video post processing and decoding technologies are meant to make the video appear as close as possible to the original video when it is decoded from its compressed state and output to a display.264. One example of typical use of PIP is a sports enthusiast watching one sporting event in HD while superimposing a smaller video of another sporting event in SD somewhere else on the display. PIP also refers to the ability to output two HD streams at the same time. If a codec is not specifically accelerated by the graphics. to a display. the CPU still decodes the video. Intel Confidential—Internal Use Only Page 18 . and AV-1 are accelerated by both integrated and discrete graphics solutions. Video decoding occurs in real-time poor video decoding hardware will cause the video to stutter which has a direct negative effect on the user experience. There are a variety of video processing techniques that can be applied to a video to make the video playback experience on a PC as good as possible. This section details technologies which are located in the video decoding pipeline which affect the way in which a video is sent to a display. one HD and one SD. These technologies are applied to video in addition to the decoding algorithm. If a user starts watching a video and experiences stutter or sees many jagged lines and blurring effects on the screen the user’s overall experience will be impacted dramatically. Video decoding refers to the process of taking compressed video and uncompressing it to its original form to be sent to a display and historically was always performed on the CPU.Figure 14: The video decoding pipeline Video Decoding Technologies The term video decoding can be used interchangeably with the term video playback. Picture-in-Picture (PIP): Picture-in-Picture is the ability of the hardware to output two video streams simultaneously. but most mainstream video codecs such as H. and post-processing technologies which affect the quality of the video. MPEG-2. Most graphics devices have adopted fixed function hardware in order to accelerate specific video codecs to take the decoding burden off of the CPU. This section also represents a list of common technology names which are seen in current specifications and marketing feature lists.
Block. runs post-processing procedures on the video stream. The following technologies are found in video post-processors which are found on almost all modern integrated and discrete graphics solutions. the noise in the video is reduced and the resulting video is clearer after the filter than before. the video decoder takes the reference frame and the directional information to reconstruct the following frames accurately even though they were not specifically stored. One example benefit of video post-processing is playing standard definition video on a high definition display. and then sends the resulting processed video stream to a display. that noise is especially noticeable.intel. Upon decompression of the video. and when displayed on an HD display. Dual-Stream Playback is used in 3D stereoscopic video where one stream being output to the display is the left eye and the other stream being output to the display is the right eye. Recording the direction of movement instead of the entire frame takes less space to store. The compute requirements for these technologies are dependent on the specific hardware implementations of each technology. Inverse Telecine is the process of detecting the 3:2 pulldown (see Pulldown Detection) from telecine videos and Intel Confidential—Internal Use Only Page 19 . and Overlapped Block Motion Compensation are different motion compensation algorithms to name a few. Video Post‐Processing Technologies Video processing refers to the portion of the video decoding pipeline which takes the decoded video stream. Motion compensation takes a reference frame and then makes it possible to reconstruct a number of following frames by recording the direction in which the frame changes. Many digital standard definition videos contain extra noise as a result of their conversion from analog to digital format. When the video is sent through a noise reduction filter as part of the video post-processing pipeline. A comparison of the capabilities of various integrated and discrete graphics solutions will be published by the end of Q2’10 in a video playback competitive positioning guide on competition. Quarter and Half Pixel. There are many different algorithmic variations of motion compensation but the concept stays the same: Global. Variable Block-Size. Motion compensations works under the idea that consecutive frames in a video have very little difference between them.com. Dual-Stream Playback: Dual-Stream Playback is the ability of the hardware to output two video streams at the same time. Motion Compensation: Motion Compensation is a technology used in video compression and decompression. All technologies in this section are meant to give an overall better perceived video quality to the viewer of the video. These technologies affect the perceived quality of video in various ways from enhancing or correcting the color of an image to increasing the apparent sharpness of an image. Telecine / Inverse Telecine: Telecine is the process in which a motion picture taken at 24 frames per second on film is converted into the National Television System Committee standard 30 frames per second stored in 60 interlaced frames digitally for distribution on television or DVD.
In addition to the 3:2 pulldown cadence.intel. 3:2 pull down converts 24 progressive frames per second into 60 interlaced fields per second. and then replicating the fields every other frame. Pulldown Detection: A special case of de-interlacing deals with pulled down content.20 Film Mode Detection and Cadence Detection: Film Mode Detection and Cadence Detection are the same as Pulldown Detection. or filmed at various other frame rates such as those used for documentaries and anime content. This is accomplished by splitting the progressive source content into fields. Splitting each film frame into two fields will yield 48 fields per second. Hence. The first frame is stored as 3 fields (with one field being used twice).cadence. By working with the original progressive content artifacts are minimized. By detecting the repetitive 3:2 cadence. Converting film that runs at 24 frames per second to 30 frames (60 fields of video) presents an obvious conversion problem. other film-to-TV specific cadences emerge when dealing with film content that has been edited. 3:2 pulldown. it is possible to recreate the original progressive frames. As analog television runs at approximately 60 fields per second.ppt 21 Source: http://www. Used when broadcasting a typical Hollywood movie over NTSC TV. 2010 Intel Confidential—Internal Use Only Page 20 . the name. and the second frame being stored as 2 fields. The number of cadences that could be encountered is further multiplied when considering both NTSC (60Hz) as well as PAL (50Hz) target television refresh rates. other cadences are often introduced into video streams as the result of editing. simply transferring each film frame onto each video frame would result in a film running about 25% faster than intended. So not only must the progressive film be interlaced.hqv. Likewise. non-NTSC broadcasts. Playing back such an encoded stream using typical de-interlacing methods misses an opportunity to achieve significantly enhanced visual quality.converting the video back into its original 24 frames per second version to be viewed on a display as it was originally intended. and numerous other influences.cfm?page=tech. Figure 15: Cadence Detection21 20 Source: http://smcr.com/index.com/SMCRDocs/WW3708_CSE_BoulderCreek_Montevina_Introduction_Rev1_0. January 28. the frame rate must also be converted.
Cathode Ray Tubes (CRTs) use interlacing to display video and the majority of video is still stored and broadcast in interlaced formats. etc. There are many different types of deinterlacing in which the algorithm to produce the results is different but the concept is the same. so that it can be seen at a different cadence without loss of data or negatively affecting the user experience. Digital video streams can also exhibit similar artifacts as a result of their original capture or their subsequent compression. De-Interlacing: Many modern displays use progressive scanning in order to display video.ppt Intel Confidential—Internal Use Only Page 21 . interlaced content must be first converted to progressive. however. De-interlacing is the method in which interlaced video is converted to progressive video. Motion Adaptive.23 22 23 Source: http://www.com/SMCRDocs/WW3708_CSE_BoulderCreek_Montevina_Introduction_Rev1_0.de-interlacing. i. Some types of de-interlacing are Vector Adaptive Per-Pixel. De-Interlacing is one of the most important aspects of a video processor as a poorly de-interlaced video can have a large negative effect on the user experience. 3:2 to 2:2:2:4. However. it is possible to leverage motion detection algorithms to dramatically reduce the appearance of randomized noise in video streams while accurately preserving fine detail. Figure 17 shows an image before noise reduction on the top and with noise reduction applied on the bottom. By realizing that noise artifacts are nondeterministic in their motion. or 5:4. Noise artifacts are most noticeable in regions of the image that contain large areas of solid colors. i. In order to be displayed on progressive displays. and duplicating the content will inevitably inject analog noise into the stream. de-noise filters are able to differentiate between noise and valid video data. 3:2 Cadence Correction is the ability of the hardware to change the cadence of a video.e. Traditional de-noise algorithms often suppress fine detail within an image by mistaking the detail for noise. Figure 16: De-Interlacing22 Noise Reduction: When working with analog video streams..cfm?page=tech. thus degrading the overall video quality. Spatial-Temporal. Cadence Correction: The cadence of a video is the same as the Pulldown or Film Mode of a video. capturing. converting.hqv.com/index.intel. and Non-Motion Adaptive De-Interlacing. 2010 Source: http://smcr.e. January 28. Advanced Pixel Adaptive.
Figure 18: Total Color Control Color Vibrance: Color Vibrance is the ability of hardware to even out the color of more saturated and less saturated areas in an image by increasing the apparent color of the less saturated areas. and contrast in an image. and Cyan. Figure 18 shows an example of total color control. 2010 Intel Confidential—Internal Use Only Page 22 . Total Color Control: Total Color Control allows for adjustment of saturation levels for six colors: Red. Green. 24 Source: http://www. Blue.Figure 17: Noise Reduction24 ProcAMP Color Control: ProcAMP Color Control is an Intel® technology package which allows for the adjustment of hue. January 29. saturation. Magenta.ambery. Figure 19 shows an image on the left and the same image on the right with increased color vibrance. Yellow. The increase of color vibrance is most noticeable in the color of the dog’s fur. brightness.html.com/di10visccohd.
In PC printing.Figure 19: Color Vibrance25 Color Correction: Color Correction is the process in which hardware alters the overall color of the light. For example. a different color space is used to create all the colors.com/Photoshop-CS4-Missing-Lesa-Snider/dp/0596522967. Color space conversion it is the ability to convert a color space such as RGB to another color space like CMYK in order for the different hardware to maintain the same representation of an image. magenta. Color space is the description on how all colors can be created. Figure 20: Skin Tone Enhancement (Flesh Tone) 25 Source: http://www. specifically flesh tone enhancement. Figure 20 shows an original image on the left and the same image with skin tone enhancement applied on the right. refers to the ability to recognize a specific range of typical flesh tones in an image and enhancing the areas in which those colors exist in order to give flesh a more realistic and vibrant appearance. Feb 8. Dynamic Tone Enhancement (Flesh Tone): Tone enhancement. CMYK (cyan. however. or the color temperature.amazon. 2010 Intel Confidential—Internal Use Only Page 23 . Color Space Conversion: Color Space Conversion is the process in which one color space is converted to another. and Blue. in a PC monitor which emits light the color space RGB is used because all colors can be created from a combination of Red. A higher color temperature provides a bluish hue to an image whereas a lower color temperature gives off a yellow or red hue. black) is the color space used in printers in order to print. yellow. Green.
Sharpness enhancement filters reduce the appearance artifacts by identifying and operating on the edges within an image.26 In figure 22. Adaptive Contrast Enhancement (Dynamic Contrast): Adaptive contrast enhancement increases the contrast of local areas of an image by decreasing the overall brightness of the image and amplifying the local features resulting in an overall greater contrast enhancement. and Polyphase Scaling. Figure 22: Edge Enhancement Video Scaling (Up-Conversion/Down-Conversion): Video scaling is the process of converting a video from one resolution and/or aspect ratio to another resolution and/or aspect ratio.intel. By applying noise reduction algorithms specifically on shape edges and improving contrast ratios in these specific regions. There are quite a few different types of video scaling which are only algorithmic differences but the concept is the same: Advanced Video Scaling. Panel Fitting.com/SMCRDocs/WW3708_CSE_BoulderCreek_Montevina_Introduction_Rev1_0. 26 Source: http://smcr. Horizontal and Vertical Scaling. High Quality Video Scaling. it is possible to mitigate artifacts that typically accompany high scale ratios. Figure 21: Adaptive Contrast Enhancement Edge Enhancement: Edge Enhancement increases the contrast of edges in a video which increases its apparent sharpness. the image on the left has edge enhancement applied whereas the image on the right does not.ppt Intel Confidential—Internal Use Only Page 24 . Figure 21 shows an image without adaptive contrast enhancement on the left and with adaptive contrast enhancement on the right. Adaptive contrast enhancement can be used in videos where the setting of the video is dark in order for objects to be seen.
February 8. lightness (luminance).29 When a video stream is output the data is conveyed in two forms.Up-Conversion is the process in which a movie in one resolution is up-converted to another resolution i.tw/en/images/products/chipsets/p4-series/img_videodeblock.com. Chroma subsampling is a scheme in which the full luminance information is sent but color information is either filtered or averaged to reduce bandwidth requirements. The human eye perceives changes in luminance much more than it perceives changes in color. Chroma Subsampling Format Conversion: Chroma subsampling format conversion is the conversion between one rate of chroma subsampling to another.org/patents/app/20090115906. Down-Conversion is where a higher resolution video is scaled down to a lower resolution to be viewed on a lower resolution display. Figure 23: De-Blocking27 Whites Processing (Blue Stretch): Blue stretch increases the blue component of white and near white pixels in video while avoiding hard transitions and without affecting flesh-tone colors. 2010 29 Source: http://www. 2010 Intel Confidential—Internal Use Only Page 25 .faqs.com/PDFs/Chroma_subsampling_notation. De-Blocking: When viewing lower quality video on a large monitor. Image changes for the human eye are barely perceptible when using chroma subsampling.e.28 Video Gamma Control: Video gamma control is necessary because many monitors and displays have different gamma ranges. and color (chroma). February 9.pdf. 27 28 Source: http://www. De-Blocking filters smooth out sharp edges between blocks when block coding techniques such as discrete cosine transform are used. Up-Conversion first scales the original video to the new resolution and then fills in the missing data. 480p SD is converted to 720p HD.poynton. Missing data is filled in by scanning the image and determining what pixels fit best in what locations.jpg Source: http://www. jagged edges sometimes appear between different ‘blocks’ of the video which gives the appearance of pixilation in video playback.via. Video gamma control ensures that video will be output at the correct gamma levels so that the video appears the same regardless of what display it is seen on and does not have a wide variation on gamma levels due to the wide variation of gamma levels on displays.
Many modern PCs are able to perform dynamic video range control which enables the hardware to limit range the audio volume in a video. in an image. In many cases. February 1.1492-7.com/reviews/avivo-vs-purevideo. Dynamic video range control increases the volume of quiet scenes in a video and decreases the volume of loud scenes in a video so that the entire video stays in the specified ‘Dynamic Range’ and certain scenes are not too quiet or too loud.tomshardware. 6. that user may have less need for high quality audio than someone who is using their computing solution to drive a home theater with surround sound speakers. 2010 Intel Confidential—Internal Use Only Page 26 . television. Detail Enhancement: Detail enhancement increases the apparent detail. If a user views online video. such as Hollywood movies. or sharpness. Figure 24: Detail Enhancement30 Bad Edit Correction: Bad edits occur when a video is edited and the edits also make changes to the pulldown cadence of the video.html. having a crisp and clear video is not enough to fully immerse the user in the video viewing experience. Blurry areas of video become sharp when detail enhancement is correctly implemented. Audio Codec’s and Formats The following audio codec’s and formats affect the quality of audio which can be transferred from the PC to the monitor. 30 Source: http://www. however. The Auditory Experience High quality audio which is well synced with the video playback is important in order to enable the best user experience possible when watching video. Figure 24 shows an image with detail enhancement on the bottom and without detail enhancement on the top. Bad edit correction is where the hardware detects the changes in the pulldown cadence and is able to recover the original 24 FPS video so that the changed cadence doesn’t affect the viewing experience. or speaker system.
PCM has the best sound quality of any format due to a lack of compression. MP3 is also known as MPEG-1 Audio Layer 3. however.1 channels of surround sound audio (seven speakers and one subwoofer). The following audio formats are found in video playback: Blu-ray: This is a comprehensive list of audio formats which are allowed in Bluray audio playback:31 Blu-ray is a standard format for high-definition video playback. it takes a significantly larger amount of space to store. o MP3: MP3 is a lossy audio compression format which is the non-official standard in portable audio player playback. o PCM: PCM is a complete copy of the studio master audio track and when used in video discs. MP3s can also be found in digital audio distribution online such as online music download websites. and is stored in an uncompressed form. o Dolby Digital Plus: Dolby Digital Plus is an expanded version of Dolby Digital which allows for a higher bit rate audio. o Dolby TrueHD / DTS-HD Master Audio: Dolby TrueHD and DTS-HD Master Audio are both lossless compression codecs.com/news/show/1064. Other Formats: Two other prominent audio formats used in audio encoding are MP3 used in devices such as MP3 players and AAC which is used on devices such as the iPhone. and PlayStation3. 2010 Intel Confidential—Internal Use Only Page 27 . more efficient compression of audio data. iPod. o DTS-HD High Resolution: DTS-HD High Resolution audio format is an expanded version of DTS which allows for a higher bit rate and more efficient compression of audio data.highdefdigest. February 3. The Blu-ray specification for DTS allows for higher bit rate audio which allows for greater sound fidelity. even if the user is viewing very high quality video. o DTS: DTS is an audio format found on DVD’s and is a required specification for Blu-ray high definition playback hardware. The lossless compression of the audio track means that they take up less space than a PCM track but upon decompression they are identical to the PCM studio master. MP3 was recognized for being able 31 Source: http://www. and supports up to 7.Audio transfer can range from scaled-down versions of a soundtrack to the original studio recording at full fidelity. o Dolby Digital / AC-3: Dolby Digital is the most commonly found audio format used in DVD’s and is the base standard for Blu-ray. Poor audio quality can ruin a video viewing experience.
Intel Wireless Display also has possible use in business applications where the projector is seamlessly linked with a laptop which allows for easier usage than before. Two connections used in modern video playback are HDMI and TOSLINK. Future Technologies The video industry is continuously evolving with new technologies enhancing the enduser experience. These two connections do not represent the entire range of audio connections. higher resolution for video playback (Quad HDTV).to reproduce audio quality close to the original soundtrack and at the same time provide good compression to save a significant amount of storage space. both useful for transferring video data. This section explores several new technologies in the immediate future for video: wireless display (Intel® Wireless Display). are not able to transfer audio data. TrueHD. 7. HDMI: Along with carrying video data. AAC is seen as the successor to MP3 as the standard audio encoding format. or DTS-HD. For more information on Intel Wireless display. Intel Wireless Display is just the start of new ways to deliver video to the user. and new ways to play more realistic video (Stereoscopic 3D). see the Intel Wireless Display Product Brief. Intel® Wireless Display Intel® Wireless Display is already in production and gives an easy way to wirelessly link different devices such as laptops and televisions in order to playback video. Other connections such as coax cables are used in audio data transfer but are not typically used in high definition audio data transfer as part of the video playback experience. HDMI connections are also capable of carrying Dolby TrueHD and DTS-HD Master Audio. new storage mediums for video (Holographic Versatile Discs). o AAC: AAC is a lossy compression codec for digital audio. VGA and DVI connections. TOSLINK: Toslink is an optical fiber connection created by Toshiba and is used mostly in audio data transfer. Intel Confidential—Internal Use Only Page 28 . HDMI is one of the only connection types which allow for the transfer of both audio and video data. Intel Wireless Display allows for the ease of sharing video content streaming seamlessly on a large screen television at the same time as on a laptop. Audio Connections Audio is distributed along with video as either part of the same connection or part of a separate connection. AAC is part of both the MPEG-2 and MPEG-4 specifications. Toslink can carry PCM audio but it cannot carry Dolby Digital Plus.
One possible next step for the video industry is adopting very high resolutions such as Quad HDTV 2160p. was created in 184033) it hasn’t been until the last decade that wide scale production of 3D Hollywood movies has been possible due to the extreme amounts of compute power which is required to render a Hollywood movie in 3D. and those problems are most notably bandwidth limitations and storage limitations on discs. William.maxellcanada. Four times the resolution means that television makers can either make displays which are four times larger at the same pixels per inch. Although the ability to create the illusion of 3D images has been around for quite a long time. Stereoscopic 3D video is video playback where the illusion of a 3D image is created on a display when viewed with specific glasses which are typically either polarized. Quad HDTV 2160p could possibly be enabled with the same advancement in storage technology with technologies such as Holographic Versatile Discs. The next evolutions in video viewing will more than likely include higher resolutions than today. (the first still 3D photograph. 2010 Source: Welling. or stereogram. however. or controlled with proprietary device drivers and connected to a PC. Many movie theaters are already playing 3D movies and the next step for the industry is to bring the 3D movie experience to the home theater.pdf.32 Stereoscopic 3D Video Playback Stereoscopic 3D video playback is a fast growing segment in video viewing. February 9. stereographic 3D. Going to higher resolutions has its problems. Dual-Layer Blu-ray Discs have a capacity of 50GB whereas a Holographic Versatile Disc such as the one which Maxell is releasing in 2010 have up to 1. page 23 Intel Confidential—Internal Use Only Page 29 . wide scale 1080p distribution was made possible with the advent of disc storage technology which could hold an entire feature length movie in high definition on just one disc. Stereoscopic video enabled televisions are already being developed and manufactured by many television companies and PC platforms are moving towards enabling stereoscopic 3D playback for personal PCs and home theater systems. NVIDIA 3D Vision Surround. Stereoscopic 3D has also been referred to as stereo 3D. 1920 x 1080 pixels). Using the entire volume of a disc to record information allows for orders of magnitude more data storage than the current Blu-ray format. as well as appearing as standalone monitors for desktops. 2160p has four times the resolution of 1080p today (3840 x 2160 pixels vs.6TB of capacity. Holographic Versatile Discs are an optical storage technology which takes advantage of the ability to not only record information on the surface of a disc like Blu-ray but HVDs can record information in the entire depth of the disc as well. Photography in America. Holographic Storage Technology Recently. 3D monitors are beginning to be embedded in laptops. NVIDIA® 3D Vision™. all-in-ones.Quad HDTV Video experience was improved dramatically when storage technology made it possible for the masses to move from standard definition to high definition video because of the dramatic increase in resolution. or just 3D. or have four times as many pixels per inch at the same size.com/pdfs/c_media/optical_stor_tech. 3D stereo. As a result of the increase in 3D movies being seen in movie 32 33 Source: http://www.
Intel® Core™ 2010 Processors with Intel® HD Graphics contain Intel® Clear Video HD Technology which enables smooth HD video playback. and implements many of the video post-processing technologies described in this document. to viewing it and sharing it with friends and family. Intel Confidential—Internal Use Only Page 30 . from creating and editing 3D video content. Videos come in many different formats and from many different channels of distribution. Expect stereo 3D video technology to gain in popularity throughout 2010 and 2011. but it can be expected that there might be many more. home stereographic 3D video playback is gaining in momentum. Intel® Core™ processors are poised to be able to provide the full spectrum of the home 3D experience. Summary Video playback is one of the most common forms of entertainment on a PC. premium audio quality.theaters. 8. This section discussed only four new and current technologies or advances which are pushing the boundaries of the current video experience. making PC video playback capabilities very important to achieving the best possible experience.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.