Project Report | Public Key Cryptography | Digital Signal Processing

CHAPTER 1 INTRODUCTION 1.

Introduction to Digital Image Processing:
   Vision allows humans to perceive and understand the world surrounding us. Computer vision aims to duplicate the effect of human vision by electronically perceiving and understanding an image. Giving computers the ability to see is not an easy task - we live in a three dimensional (3D) world, and when computers try to analyze objects in 3D space, available visual sensors (e.g., TV cameras) usually give two dimensional (2D) images, and this projection to a lower number of dimensions incurs an enormous loss of information.    In order to simplify the task of computer vision understanding, two levels are usually distinguished; low-level image processing and high level image understanding. Usually very little knowledge about the content of images High level processing is based on knowledge, goals, and plans of how to achieve those goals. Artificial intelligence (AI) methods are used in many cases. High-level computer vision tries to imitate human recognition and the ability to make decisions according to the information contained in the image.   This course deals almost exclusively with low-level image processing, high level in which is a continuation of this course. Age processing is discussed in the course Image Analysis and Understanding, which is a continuation of this course.

1.1 History:
Many of the techniques of digital image processing, or digital picture processing as it was often called, were developed in the 1960s at the Jet Propulsion Laboratory, MIT, Bell Labs, University of Maryland, and few other places, with application to satellite imagery, wire photo standards conversion, medical imaging, videophone, character recognition, and photo enhancement. But the cost of processing was fairly high with the computing equipment of that era. In the 1970s, digital image processing proliferated, when cheaper computers Creating a film or electronic image of any picture or paper form. It is accomplished by scanning or photographing an object and turning it into a matrix of dots (bitmap), the meaning of which is unknown to the computer, only to the human

viewer. Scanned images of text may be encoded into computer data (ASCII or EBCDIC) with page recognition software (OCR).

1.2 Basic Concepts:
 

A signal is a function depending on some variable with physical meaning. Signals can be
o o o o

One-dimensional (e.g., dependent on time), Two-dimensional (e.g., images dependent on two co-ordinates in a plane), Three-dimensional (e.g., describing an object in space), Or higher dimensional.

1.3 Pattern recognition
Pattern recognition is a field within the area of machine learning. Alternatively, it can be defined as "the act of taking in raw data and taking an action based on the category of the data" . As such, it is a collection of methods for supervised learning. Pattern recognition aims to classify data (patterns) based on either a priori knowledge or on statistical information extracted from the patterns. The patterns to be classified are usually groups of measurements or observations, defining points in an appropriate multidimensional space. Are to represent, for example, color images consisting of three component colors.

1.4 Image functions:
 

The image can be modeled by a continuous function of two or three variables; Arguments are co-ordinates x, y in a plane, while if images change in time a third variable t might be added.

 

The image function values correspond to the brightness at image points. The function value can express other physical quantities as well (temperature, pressure distribution, distance from the observer, etc.).

The brightness integrates different optical quantities - using brightness as a basic quantity allows us to avoid the description of the very complicated process of image formation.

The image on the human eye retina or on a TV camera sensor is intrinsically 2D. We shall call such a 2D image bearing information about brightness points an intensity image.

  

The real world, which surrounds us, is intrinsically 3D. The 2D intensity image is the result of a perspective projection of the 3D scene. When 3D objects are mapped into the camera plane by perspective projection a lot of information disappears as such a transformation is not one-to-one.

 

Recognizing or reconstructing objects in a 3D scene from one image is an ill-posed problem. Recovering information lost by perspective projection is only one, mainly geometric, problem of computer vision.

The second problem is how to understand image brightness. The only information available in an intensity image is brightness of the appropriate pixel, which is dependent on a number of independent factors such as
o

Object surface reflectance properties (given by the surface material, microstructure and marking),

o O

Illumination properties, And object surface orientation with respect to a viewer and light source.

CHAPTER 2 2. DIGITAL IMAGE FORENSICS
Today's technology allows digital media to be altered and manipulated in ways that were impossible twenty years ago. We are feeling the impact of this technology in nearly every corner of our lives, from the courts to the media, politics, business, and science. As this technology continues to evolve it will become increasingly more important for the science of digital forensics to keep pace. This presentation will describe state of the art techniques in digital image forensics. Digital watermarking has been proposed as a means by which an image can be authenticated. This approach works by inserting at the time of recording an imperceptible digital code (a watermark) into the image. With the assumption that tampering will alter a watermark, an image can be authenticated by verifying that the extracted watermark is the same as that which was inserted. The major drawback of this approach is that a watermark must be inserted at precisely the time of recording, which limits this approach to specially equipped digital cameras. In contrast, recent advances in digital forensics operate in the absence of any watermark or specialized hardware. With the assumption that tampering disturbs certain underlying statistical properties of an image, these forensic techniques can detect specific forms of tampering. Air-brushing or re-touching can be detected by measuring deviations of the underlying color filter array correlations. Specifically, virtually all digital cameras record only a subset of all the pixels needed for a full-resolution color image. Instead, only a subset of the pixels is recorded by a color filter array (CFA) placed atop the digital sensor. The most frequently used CFA, the Bayer array, employs three color filters: red, green, and blue. Since only a single color sample is recorded at each pixel location, the other two color samples must be estimated from the neighboring samples in order to obtain a three-channel color image. The estimation of the missing color samples is referred to as CFA interpolation or demosaicking. In its simplest form, the missing pixels are filled in by spatially averaging the recorded values. Since the CFA is arranged in a periodic pattern, a periodic set of pixels will be precisely correlated to their neighbors according to the CFA interpolation algorithm. When an image is re-touched, it is likely that these correlations will be destroyed. As such, the presence or lack of these correlations can be used to authenticate an image, or expose it as a forgery. A digital composite of two people can be detected by measuring differences in the direction to the illuminating light sources from their faces and body. By making some initial simplifying assumptions about the light and the surface being illuminated, we can mathematically express how much light a surface should receive as a function of its position relative to the light. A surface that is

. Duplication or cloning is a simple and powerful form of manipulation used to remove objects or people from an image. digital watermarks are only perceptible under certain conditions. the hidden information should. i. standard techniques can be used to determine the direction to the light source for any object or person in an image. It is prominently used for tracing copyright infringements and for banknote authentication. a fragile watermark would be applied. a region growing algorithm combines any significant number of neighboring blocks that are consistent with the cloning of an image region. the signal may be audio. "Watermarking" is the process of hiding digital information in a carrier signal. This form of tampering can be detected by first partitioning an image into small blocks. DIGITAL WATERMARKING A digital watermark is a kind of marker covertly embedded in a noise-tolerant signal such as audio or image data. The needed properties of a digital watermark depend on the use case in which it is applied. It is typically used to identify ownership of the copyright of such signal. but does not need to contain a relation to the carrier signal. Any inconsistencies in lighting can then be used as evidence of tampering. A signal may carry several different watermarks at the same time. it is of no use. For marking media files with copyright information. for example. Digital watermarks may be used to verify the authenticity or integrity of the carrier signal or to show the identity of its owners. video. Since it is statistically unlikely to find identical and spatially coherent regions in an image. Once expressed in this form. If a digital watermark distorts the carrier signal in a way that it becomes perceivable. Like traditional watermarks. will be brighter than a surface that is turned away from the light. if integrity has to be ensured. Traditional Watermarks may be applied to visible media (like images or video). 2. pictures.directly facing the light. a digital watermark has to be rather robust against modifications that can be applied to the carrier signal. texts or 3D models. after using some algorithm. their presence can then be used as evidence of tampering.e. whereas in digital watermarking. Unlike metadata that is added to the carrier signal. and imperceptible anytime else. The blocks are then re-ordered so that they are placed a distance to each other that is proportional to the differences in their pixel colors. a digital watermark does not change the size of the carrier signal.1. Instead. With identical and highly similar blocks neighboring each other in the re-ordered sequence.

Transparency requires a watermark to be imperceptible so that it does not affect the quality of the content. and therefore removal. If a copy of the work is found later. digital watermarking is a passive protection tool. and transparent.1. Watermarks of varying degrees of obtrusiveness are added to presentation media as a guarantee of authenticity. The effectiveness of a watermark is improved when the technique exploits known properties of the human visual system. . The signal. In particular. to authenticate the content. Digital watermarking is the process of inserting a digital signal or pattern (indicative of the owner of the content) into digital content. But whereas steganography aims for imperceptibility to human senses. known as a watermark.2. and to trace illegal copies of the work. including intentional attacks to remove the watermark. and makes detection. can be used later to identify the owner of the work. this method is useless in the digital world. These are known as perceptually based watermarking techniques.Both steganography and digital watermarking employ steganographic techniques to embed data covertly in noisy signals.1 Principle of digital watermarks A watermark on a bank note has a different transparency than the rest of the note when a light is shined on it. One application of digital watermarking is source tracking. digital watermarking tries to control the robustness as top priority. the class of image-adaptive watermarks proves most effective. depending on in which domain the watermark is constructed: the spatial domain (producing spatial watermarks) and the frequency domain (producing spectral watermarks). and source. To be effective in its purpose. it should be robust. It just marks data. Robustness requires that it be able to survive any alterations or distortions that the watermarked content may undergo. This technique reportedly has been used to detect the source of illegally copied movies. The media of focus in this paper is the still image. A watermark is embedded into a digital signal at each point of distribution. then the watermark may be retrieved from the copy and the source of the distribution is known. quality. Since a digital copy of data is the same as the original. falling into 2 main categories. . This is so that afterwards. Within this category. but does not degrade it nor controls access to the data. by pirates less possible. a watermark should adhere to a few requirements. There are a variety of image watermarking techniques. ownership. the owner can still be identified. and common signal processing alterations used to make the data more efficient to store and transmit. However.

Currently there are various techniques for embedding digital watermarks. This suggests that there are many applications that could require image watermarking. They are of the opinion that any encryption system can be broken. Furthermore. including writers. surveillance imaging. digital cameras.3 PURPOSES OF DIGITAL WATERMARKS Watermarks are a way of dealing with the problems mentioned above by providing a number of services:  They aim to mark digital data permanently and unalterably. which has resulted in pirating. the images must be watermarked as they are captured. However. Another scenario in which the enforcement of copyright is needed is in newsgathering. 2. medical imaging. musicians and artists. they all digitally write desired information directly onto images or audio data in such a manner that the images or audio data are not damaged. Copyright owners can incorporate . including Internet imaging. digital libraries. Digital watermarks are added to images or audio data in such a way that they are invisible or inaudible Ñ unidentifiable by human eye or ear. Digital watermarking is being recognized as a way for improving this situation. whether their music is released over the Internet or on DVD-Audio". they can be embedded in content with a variety of file formats. RIAA reports that "record labels see watermarking as a crucial piece of the copy protection system. and that digital watermarking is needed to indicate who the culprit is. video-ondemand systems. and satellite-delivered video. image's origin and content can be verified. An example of an area where copyright protection needs to be enforced is in the on-line music industry.1.2 IMPORTANCE OF DIGITAL WATERMARKS The Internet has provided worldwide publishing opportunities to creators of various works. image and video databases. and is therefore probable that duplication on the Internet occurs without the rightful owners' permission. sooner or later. Embedding a watermark should not result in a significant increase or reduction in the original data.1. photographers. When digital cameras are used to snapshot an event. 2. Basically. The Recording Industry Association of America (RIAA) says that the value of illegal copies of music that are distributed over the Internet could reach $2 billion a year. so that the source as well as the intended recipient of the digital work is known. It is easy to duplicate audio and visual files. these same opportunities provide ease of access to these works. Digital watermarking is the content protection method for the multimedia era. This is so that later.

1.4 ATTACKS ON WATERMARKS            Lossy Compression: Many compression schemes like JPEG and MPEG can potentially degrade the data’s quality through irretrievable loss of data. translation.  With a tracking service. scaling and cropping. That is. Common Signal Processing Operations: They include the followings.  Watermarks can be used to identify any changes that have been made to the watermarked data.  By indicating the owner of the work. The presence of a watermark in a work suspected of having been copied can prove that it has been copied. Requantization. A/D conversion Resampling.  Some more recent techniques are able to correct the alteration as well. Recompression Linear filtering such as high pass and low pass filtering.identifying information into their work. Addition of a constant offset to the pixel values Local exchange of pixels other intentional attacks: Printing and Rescanning Watermarking of watermarked image (rewatermarking) 2. In addition.1. watermarks are used in the protection of ownership. any unauthorized copies that s/he has distributed can be traced back to him/her. this section identifies digital watermarking applications and provides an overview of digital watermarking capabilities and useful benefits to customers. The various applications are:    Authentication Broadcast Monitoring Copy Prevention . Geometric Distortions: include such operations as rotation. 2. they demonstrate the quality and assure the authenticity of the work.5 DIGITAL WATERMARKING APPLICATIONS Digital watermarking is rapid evolving field. because each purchaser of the data has a unique watermark embedded in his/her copy. owners are able to find illegal copies of their work on the Internet. D/A conversion.

and must be implemented properly to be effective. video e. meaning that the signer cannot successfully claim they did not sign a message. Digital signatures can also provide nonrepudiation.2 DIGITAL SIGNATURE A digital signature or digital signature scheme is a mathematical scheme for demonstrating the authenticity of a digital message or document. Digimarc: For document verification. while also claiming their private key remains secret. Signum: Allow digital fingerprints to be embedded into grahics.c. and internet tracking. but properly implemented digital signatures are more difficult to forge than the handwritten type. Digital signatures are equivalent to traditional handwritten signatures in many respects. . Digital signatures are often used to implement electronic signatures. the signature is valid. For messages sent through a nonsecure channel. copyright protection. A valid digital signature gives a recipient reason to believe that the message was created by a known sender such that they cannot deny sending it (authentication and non-repudiation) and that the message was not altered in transit (integrity). so that even if the private key is exposed. Digital signatures employ a type of asymmetric cryptography.[4] and members of the European Union.  Forensic Tracking E-Commerce/Linking 2. audio. and in other cases where it is important to detect forgery or tampering. further. financial transactions. Digital signature schemes in the sense used here are cryptographically based. India. Stegnosign: For creating.6 WATERMARKING SOFTWARE&SREVICES      Alpha-Tec: watermarking software for copyright protection and infringement tracking. electronic signatures have legal significance. Digital signatures are commonly used for software distribution. partial encryption. embedding and detecting watermarks.[2][3] In some countries. including the United States. some non-repudiation schemes offer a time stamp for the digital signature. embedded messages and more. 2. a properly implemented digital signature gives the receiver reason to believe the message was sent by the claimed sender.[1] but not all electronic signatures use digital signatures.t. a broader term that refers to any electronic data that carries the intent of a signature.1. MediaSec: Provide software for various media types.

public key and a signature. identity. that information may not be accurate. Below are some common reasons for applying a digital signature to communications: 2.1 Authentication Although messages may often include information about the entity sending a message. either accepts or rejects the message's claim to authenticity.1 Uses of Digital Signature As organizations move away from paper documents with ink signatures or authenticity stamps. and Stanford are publishing electronic student transcripts with digital signatures. given a message. The importance of high confidence in sender authenticity is especially obvious in a financial context.  A signature verifying algorithm that. suppose a bank's branch office .2. it should be computationally infeasible to generate a valid signature for a party who does not possess the private key. produces a signature. a valid signature shows that the message was sent by that user.1. 2. For example. a signature generated from a fixed message and fixed private key should verify the authenticity of that message by using the corresponding public key. digital signatures can provide added assurances of the evidence to provenance. and congressional bills with digital signatures.  A signing algorithm that. University of Chicago.Digitally signed messages may be anything representable as a bitstring: examples include electronic mail. given a message and a private key. public and private laws. When ownership of a digital signature secret key is bound to a specific user. First. Two main properties are required. Digital signatures can be used to authenticate the source of messages. The algorithm outputs the private key and a corresponding public key. Universities including Penn State. The United States Government Printing Office (GPO) publishes electronic versions of the budget. or a message sent via some other cryptographic protocol. contracts. and status of an electronic document as well as acknowledging informed consent and approval by a signatory.2. A digital signature scheme typically consists of three algorithms:  A key generation algorithm that selects a private key uniformly at random from a set of possible private keys. Secondly.

2. any change in the message after signature will invalidate the signature.1. Similarly. If the central office is not convinced that such a message is truly sent from an authorized source. (Some encryption algorithms. acting on such a request could be a grave mistake. if a message is digitally signed. because this is still considered to be computationally infeasible by most cryptographic hash functions (see collision resistance). 2. By this property. is an important aspect of digital signatures. but others do not.sends instructions to the central office requesting a change in the balance of an account. access to the public key only does not enable a fraudulent party to fake a valid signature.3 Non-repudiation Non-repudiation.1.2 Integrity In many scenarios. or more specifically non-repudiation of origin. Although encryption hides the contents of a message. an entity that has signed some information cannot at a later time deny having signed it. The device signature may be in the form of  sensor pattern noise (SPN)  camera response function  Re sampling artifacts  Color filter array  Interpolation artifacts  JPEG compression  Lens aberration  sensor dust . there is no efficient way to modify a message and its signature to produce a new message with a valid signature. prevent this.) However. Furthermore.2. the sender and receiver of a message may have a need for confidence that the message has not been altered during transmission. known as nonmalleable ones. it may be possible to change an encrypted message without understanding it. 2.

each covering one pixel sensor. Each two-by-two submosaic contains 2 green. For example. 1 blue and 1 red filter. a color filter array (CFA). absolute color space. Color filter array The Bayer color filter mosaic. so a color translation is required to convert the tristimulus values into a common. The spectral transmittance of the CFA elements along with the demosaicing algorithm jointly determine the color rendition. and blue (RGB) wavelength regions. green. The raw image data captured by the image sensor is then converted to a full-color image (with intensities of all three primary colors represented at each pixel) by a demosaicing algorithm which is tailored for each type of color filter. The sensor's passbandquantum efficiency and span of the CFA's spectral responses are typically wider than the visible spectrum. or color filter mosaic (CFM). In photography.CHAPTER 3 COLOR FILTER ARRAY 3. thus all visible colors can be distinguished. the Bayer filter (shown to the right) gives information about the intensity of light in red. such that the separate filtered intensities include information about the color of light. The responses of the filters do not generally correspond to the CIEcolor matching functions. Since sensors are made of semiconductors they obey solid-state physics. The color filters filter the light by wavelength range. and therefore cannot separate color information. Color filters are needed because the typical photosensors detect light intensity with little or no wavelength specificity. is a mosaic of tiny color filters placed over the pixel sensors of an image sensor to capture color information. .

two yellow. one green. used in a few cameras. Traditional RGBW similar to Bayer and RGBE patterns.1 Manufacture of the CFA . This arrangement does not require a demosaicing algorithm because each pixel has information about each color. and one magenta. (See Bayer filter#Alternatives) 4×4 2×4 3. Bayer filter RGBE filter CYYM filter CYGM filter RGBW Bayer RGBW #1 2×2 2×2 2×2 2×2 RGBW #2 RGBW #3 Three example RGBW filters from Kodak. With one blue. with 50% white. One cyan. used in a few cameras of Kodak. and one magenta. and two green.The Foveon X3 sensor uses a different structure such that a pixel utilizes properties of multijunctions to stack blue.  List of color filter arrays Pattern (pixels) 2×2 size Image Name Description Very common RGB filter. One cyan. used in a few Sony cameras. Dick Merrill of Foveon distinguishes the approaches as "vertical color filter" for the Foveon X3 versus "lateral color filter" for the CFA. green. one red. and red sensors on top of each other. one yellow. Bayer-like with one of the green filters modified to "emerald".

Filter materials are manufacturer specific. to fine-tune the spectral responsivities. There is some interference between the dyes and the ultraviolet light needed to properly expose the polymer. but nowadays. attending optical properties. state "Several factors influence the CFA's design. 1978) and not as a hybrid. A few sources discuss other specific chemical substances. CMCR106G. at best. Pigment based CFAs have become the dominant option because they offer higher heat resistance and light resistance compared to dye based CFAs.. Ensuring that the dyes have the right mechanical properties—such as ease of application. while other sensors have the CFA manufactured directly on the surface of the imager." He provides a bibliography focusing on the number. . First. Theuwissen says "Previously.Diazonaphthoquinone (DNQ)-novolacphotoresist is one material used as the carrier for making color filters from color dyes. and subsequently dyed. and spatial frequencies of the absorptive filters. types. CMCR101G. all single-chip color cameras are provided with an imager which has a color filter on-chip processed (Dillon. Some sources indicate that the CFA can be manufactured separately and affixed after the sensor has been manufactured.[15] The gelatin is sectionalized. At least one early example of an on-chip design utilized gelatin filters (Aoki et al. Color photoresists sometimes used include those with chemical monikers CMCR101R. Adams et al. the color filter was fabricated on a separate glass plate and glued to the CCD (Ishikawa 1981). thicknesses ranging up to 1 micrometre are readily available. This makes it difficult. and resistance to humidity and other atmospheric stresses—is a challenging task. Theuwissen makes no mention of the materials utilized in CFA manufacture. CMCR101B. and optimal manufacturing processes of color filter arrays. and CMCR106B. In either case. aliasing effects.". Nakamura said that materials for on-chip color filter arrays fall into two categories: pigment and dye. For instance. moire patterns. via photolithography. with the G filter being an overlap of the Y and C filters. Aoki reveals that a CYWG arrangement was used. though solutions have been found for this problem. the individual CFA filters are usually layers of transmissive (absorptive) organic or pigment dyes. durability. CMCR106R. 1982).

Image noise can also originate in film grain and in the unavoidable shot noise of an ideal photon detector. and C. Carl Chiulli cites the use of 5 chemicals.I.# 4. ..3 IMAGE NOISE Image noise is random (not present in the object imaged) variation of brightness or color information in images.Given that the CFAs are deposited on the image sensor surface at the BEOL (back end of line.S. Solvent Yellow 88. organics would be preferred over glass. On the other hand. Solvent Blue 36. where a low-temperature regime must be rigidly observed (due to the low melting temperature of the aluminum metalized "wires" and the substrate mobility of the dopants implanted within the bulk silicon).808.P. #12715. three of which are C. some CVD silicon oxide processes are low temperature processes. and is usually an aspect of electronic noise.2 Some pigment and dye molecules used in CFAs In U. CAS Registry numbers.S. # 5.096. # 61551. but fail to provide chemical names.P. of Fuji Photo Film company. It can be produced by the sensor and circuitry of a scanner or digital camera. mainly azo dyes and pyrazolone-diazenyl. Ocean Optics has indicated that their patented dichroic filter CFA process (alternating thin films of ZnS and Cryolite) can be applied to spectroscopic CCDs. or Colour Index numbers. Gersteltec sells photoresists that possesses color filter properties. list some 150-200 chemical structures. 3.801 Koya et al. In U. Image noise is an undesirable by-product of image capture that adds spurious and extraneous information.501. the later stages of the integrated circuit manufacturing line). AKA Solvent Red 8.I. 3.

that is. to optical and radioastronomical images that are almost entirely noise. Amplifier noise is a major part of the "read noise" of an image sensor.Noise clearly visible in an image from a digital camera The original meaning of "noise" was and remains "unwanted signal".4. 3.1 Amplifier noise (Gaussian noise) The standard model of amplifier noise is additive. In color cameras where more amplification is used in the blue color channel than in the green or red channel." Image noise is. there can be more noise in the blue channel.4 Types o o o o o o o Amplifier noise (Gaussian noise) Salt-and-pepper noise Shot noise Dark current noise Quantization noise (uniform noise) Read noise Anisotropic noise 3. caused primarily by Johnson–Nyquist noise (thermal noise).2 Salt-and-pepper noise . of course. By analogy unwanted electrical fluctuations themselves came to be known as "noise. independent at each pixel and independent of the signal intensity. of the constant noise level in dark areas of the image. inaudible. The magnitude of image noise can range from almost imperceptible specks on a digital photograph taken in good light. unwanted electrical fluctuations in signals received by AM radios caused audible acoustic noise ("static"). from which a small amount of information can be derived by sophisticated processing (a noise level that would be totally unacceptable in a photograph since it would be impossible to determine even what the subject was). including that which comes from the reset noise of capacitors ("kTC noise"). 3.4. Gaussian.

that is. Dead pixels in an LCD monitor produce a similar. It can be mostly eliminated by using dark frame subtraction and interpolating around dark/bright pixels. This noise is known as photon shot noise. 3. If dark-frame subtraction is not done. display. and the noises at different pixels are independent of one another. An image containing salt-and-pepper noise will have dark pixels in bright regions and bright pixels in dark regions. In addition to photon shot noise. and hot pixels appear as salt-and-pepper noise. Shot noise follows a Poisson distribution. of the leakage. but non-random. The variable dark charge of normal and hot pixels can be subtracted off (using "dark frame subtraction"). bit errors in transmission. variation in the number of photons sensed at a given exposure level. there can be additional shot noise from the dark leakage current in the image sensor. . or if the exposure time is long enough that the hot pixel charge exceeds the linear charge capacity. Shot noise has a root-mean-square value proportional to the square root of the image intensity. Dark current is greatest at "hot pixels" within the image sensor.Image with salt and pepper noise Fat-tail distributed or "impulsive" noise is sometimes called salt-and-pepper noise or spike noise. the noise will be more than just shot noise. This type of noise can be caused by analog-to-digital converter errors. leaving only the shot noise. which is usually not very different from Gaussian. this noise is sometimes known as "dark shot noise" or "dark-current shot noise". etc.3 Shot noise The dominant noise in the lighter parts of an image from an image sensor is typically that caused by statistical quantum fluctuations. or random component.4.

adding to the signal measured in the pixel.4.5 In digital cameras . 3. the most effective way to reduce dark current is to cool the CCD. 3. For example.4 Dark current noise: Dark current is the result of imperfections or impurities in the depleted bulk silicon or at the siliconsilicon dioxide interface. image sensors are sometimes subject to row noise or column noise. These sites introduce electronic states in the forbidden gap which act as steps between the valence and conduction bands.4.4. since high read noise decreases the quality of the image. It has an approximately uniform distribution. Calibrating the read noise allows us know more about the quality of the CCD as well as the data distortion due to the reading of images. The read noise of a camera affects how well the image represents the actual data.4. or if dithering is explicitly applied. it will be signal independent if other noise sources are big enough to cause dithering.7 Anisotropic noise Some noise sources show up with a significant orientation in images. from which they are emitted into the conduction band. providing a path for valence electrons to sneak into the conduction band. and is present in all images taken and recorded by a camera. Though it can be signal dependent.[13] 3.3.5 Quantization noise (uniform noise) The noise caused by quantizing the pixels of a sensed image to a number of discrete levels is known as quantization noise. 3. The generation of dark current is a thermal process wherein electrons use thermal energy to hop to an intermediate state.6 Read noise Read noise is a property that is inherent to the CCD of digital cameras. robbing electrons of the thermal energy required to reach an intermediate state. with states near mid-band generating most of the dark current. The efficiency of a generation center depends on its energy level. For this reason.

higher gain (ISO sensitivity). where read noise (noise floor) is significant. In low light. is the largest determinant of signal levels that determine signal-to-noise ratio and hence apparent noise levels. The relative effect of both read noise and shot noise increase as the exposure is reduced. . longer shutter speeds lead to increased salt-and-pepper noise due to photodiodeleakage currents. assuming the aperture area is proportional to sensor area. or both. since fewer photons are counted (shot noise) and since more amplification of the signal is necessary. the pixel count makes little difference to perceptible noise levels – the noise depends primarily on sensor area. can be introduced through brightening shadows or through color-balance processing. Banding noise. On most cameras. this salt-and-pepper noise can be mostly eliminated by dark frame subtraction. not how this area is divided into pixels. or that the f-number or focal-plane illuminance is held constant. so larger sensors typically create lower noise images than smaller sensors. The image on the right has adequate lighting and 0. when the image is scaled to the same size on screen. In the case of images bright enough to be in the shot noise limited regime.1 second exposure. or printed at the same size. more pixels within a given sensor area will make the image noisier if the per pixel read noise is the same. the sensitivity of an imager scales roughly with the sensor area. similar to shadow noise. correct exposure requires the use of long shutter speeds. or effective light collection area per pixel sensor.Image on the left has exposure time of >10 seconds in low light.6 Effects of sensor size The size of the image sensor. 3. For images at lower signal levels (higher ISO settings). That is. At the cost of a doubling of read noise variance (41% increase in read noise standard deviation). for a constant f-number. corresponding to increased ISO sensitivity.

8 Image noise reduction Most algorithms for converting image sensor data to an image. This decision can be assisted by knowing the characteristics of the source image and of human vision. involve some form of noise reduction. But a definitive answer is not available. However. it may be more likely to be a true feature of the image.For instance. since there is little important fine chroma detail that one risks losing. Furthermore. so there is often a tradeoff made between noise removal and preservation of fine. it may be a defect in a group of pixels in the image-taking sensor (spurious and unwanted. Most noise reduction algorithms perform much more aggressive chroma noise reduction. With this in mind. There are many procedures for this. no algorithm can make this judgment perfectly.5" compact camera sensor (with roughly 1/16 the area) at ISO 100. many people find . but not strictly noise). and average out the former while attempting to preserve the latter. 3. An example shows a DSLR sensor at ISO 400 creating less noise than a point-and-shoot sensor at ISO 100.7 Sensor heat Temperature can also have an effect on the amount of noise produced by an image sensor due to leakage. 3. If this is a single pixel. the noise level produced by a Four Thirds sensor at ISO 800 is roughly equivalent to that produced by a full frame sensor (with roughly four times the area) at ISO 3200. which tend to use larger sensors than compacts. This ability to produce acceptable images at higher sensitivities is a major factor driving the adoption of DSLR cameras. Many cameras have settings to control the aggressiveness of the in-camera noise reduction. whether in-camera or on a computer. it is likely (but not certain) to be spurious and noise. it is known that DSLRs will produce more noise during summer than winter. if it covers a few pixels in an absolutely regular shape. low-contrast detail that may have characteristics similar to noise. but all attempt to determine whether the actual differences in pixel values constitute noise or real photographic detail. if it is irregular. and that produced by a 1/2. A simplified example of the impossibility of unambiguous noise reduction: an area of uniform red in an image might have a very small black part.

In cases of extreme noise. such as astronomical images of very distant objects. The high sensitivity image quality of a given camera (or RAW development workflow) may depend greatly on the quality of the algorithm used for noise reduction. . it is not so much a matter of noise reduction as of extracting a little information buried in a lot of noise. Since noise levels increase as ISO sensitivity is increased. most camera manufacturers increase the noise reduction aggressiveness automatically at higher sensitivities. seeking small regularities in massively random data.luminance noise less objectionable to the eye. since its textured appearance mimics the appearance of film grain. techniques are different. This leads to a breakdown of image quality at higher sensitivities in two ways: noise levels increase and fine detail is smoothed out by the more aggressive noise reduction.

CHAPTER 4 FIXED PATTERN NOISE Fixed pattern noise is the term given to a particular noise pattern on digital imaging sensors often noticeable during longer exposure shots where particular pixels are susceptible to giving brighter intensities above the general background noise. In practice. which describes the gain or ratio between optical power on a pixel versus the electrical signal output. occurring only at certain. The latter can be described as the local. it may vary with integration time. and fixed pixel locations. This problem arises from small differences in the individual responsibility of the sensor array (including any local post amplification stages) that might be caused by variations in the pixel size. etc. imager temperature. Sometimes pixel noise[2] as the average deviation from the array average under different illumination and temperature conditions is specified. integration time) but no external illumination and the PRNU (photo response nonuniformity). which is the offset from the average across the imaging array at a particular setting (temperature. The term "fixed pattern noise" usually refers to two parameters. imager gain and incident illumination. it is not expressed in a random (uncorrelated or changing) spatial distribution. Although FPN does not change appreciably across a series of captures. It is characterized by the same pattern of 'hot' (brighter) and cold (darker) pixels occurring with images taken under the same illumination conditions in an imaging array. material or interference with the local circuitry. degrading the image. . which might strongly deteriorate if additional electrical gain (and noise) is included. It might be affected by changes in the environment like different temperatures. a long exposure (integration time) emphasizes the inherent differences in pixel response so they may become a visible defect. exposure times. Fixed pattern noise (FPN) is a general term that identifies a temporally constant lateral nonuniformity (forming a constant pattern) in an imaging system with multiple detector or picture elements (pixels). Pixel noise therefore gives a number (commonly expressed in rms) that identifies FPN in all permitted imaging conditions.[1] One is the DSNU (dark signal non-uniformity). pixel dependent photo response nonlinearity (PRNL) and is often simplified as a single value measured at almost saturation level to permit a linear approximation of the non-linear pixel response.

we use the camera to take multiple images of a uniform scene. since the maximum value of the Opt linear device is around 4 candelas. And it is for CCD only. produced by the Optoliner. or PRNU for short.5.2 Methods To characterize the PRNU.1. Repeat the calculations for the different exposure times to compare the PRNU. PRNU is the pixel variation under illumination. However.1. since increasing the illumination level will enhance the difference in the photo-response of the pixels across the image and lead to a higher PRNU. The calculation of the PRNU is as follows:  Obtain the average image over the 100 images taken:   Subtract the DSNU image from this average image to eliminate the contribution from the DSNU. 4. Obtain the spatial variance of the pixel values over the entire CCD   Divide the spatial variance by the average image from (ii) to obtain the PRNU as a percentage of the actual pixel values. 1/4 and 1/2.1 Background Photo Response Non-Uniformity.1 PRNU (Photo Response Non-Uniformity) 4. while DSNU occurs as a variation in pixel responsively when the CCD is not illuminated. In our measurements. is one source of pattern noise in digital cameras. 4. We expect the PRNU to increase with increasing illumination.00 candelas since the brighter light is more easily detected by the camera. We took 100 exposures each for three exposure times: 1/10. One of the few engineering definitions for PRNU or "photoresponsenonuniformity" is in the photonics dictionary. We kept the illumination level fixed at 3. Like DSNU. it is seen as the variation in pixel responsively over the CCD. and increasing the illumination level increases the non-uniformity of the illumination . and we also checked to ensure that the camera is focused before taking the pictures.

a camera output model can be expressed as where I is the output image. However. As formulated in [11]. The dominating component of sensor pattern noise is photo response non-uniformity (PRNU). In Eq. the signal will inevitably be distorted when passing through each process and these distortions result in slight differences between the scene and the camera-captured image. stand for dark current. we chose to increase the exposure times to mimic the effect of increasing illumination levels. during the image acquisition process. (1) and keeping the first two terms of the expansion where  is the denoised image and is the ensemble of the noises. and . but for every pixel only therays of one colour component is passed through the CFA and subsequently converted into electronic signals by the sensor. This colour interpolation process is commonly known as demosaicking. .. (1).produced by the Opt linear. the PRNU can be contaminated by various types of noise introduced at different stages of the image acquisition process.s andr are random noise and is the fixed pattern noise (FPN) that is associated with every camera and can be removed by subtracting a dark frame from the image taken by the same camera.455) is the gamma correction factor. For most digital cameras. A colour photo is represented in three colour components (i. After the conversion. read-out noise and quantization (lossy Compression) noise. primarily the JPEG format. g is the colour channel gain. Figure 1 demonstrates the image acquisition process. Finally. shot noise. (1). Since is the dominating term in Eq. and is the input signal of the scene. The PRNU pattern noise K can then be formulated as . (= 0. The signals then undergo additional signal processing such as white balance. R. This colour filtering is determined by the CFA. including . respectively. In acquiring an image. the lenses let through the rays of the three colour components of the scene. and B). after applying Taylor expansion to Eq. gamma correction and image enhancement.e. K is the zero-mean multiplicative factor responsible for the PRNU. these signals are stored in the camera’s memory in a customized format. a colour interpolation function generates the electronic signals of the other two colour components for every pixel according to the colour intensities of the neighboring pixels. G.

is used to compare the noise Note in Eq. for each imaging device d. Since for most cameras.. the noise residual patterns are extracted using Eq. al.e. only one of the three colours of each pixel is physically captured by the sensor while the other two are artificially interpolated by the demosaicking process. Again the multiplication operation in Eq. the noise residual WI of image I under investigation is extracted using Eq.2 Use of PRNU in Device Identification The basic idea of using the PRNU noise pattern in device identification can be described as follows. it is important that the PRNU extracted is as close to the genuine pattern noise due to the sensor as possible. (5) and compared against the reference PRNU Kd of each device d available to the investigator in the hope that it will match one of the reference fingerprints. 4. this inevitably introduce noise with power stronger than that of the genuine . the discrete wavelet transform followed by a Wiener filtering operation).e. Although various denoising filters can be used. (5) from a number of low-contrast images taken by device d and then the PRNU is estimated using the ML estimation procedure adopted by Chen et. i. Given the PRNU-based approaches‟ potential in resolving device identification problem to the accuracy at individual device level. where S is the number of images involved in the calculation. as suggested in [11]. 2) Secondly.. we used . where is the mean function.is the s-th image taken by device d and Note the multiplication operation in Eq. has been reported as effective in producing good results. (6).. (6) is element-wise. the wavelet-based denoising process (i. thus identifying the source device that has taken the image under investigation. 1) First.is the noise residual obtained by applying a denoising filter on image I. is the gamma correction factor . The normalised cross-correlation is the noise residual extracted from . instead of using against the reference fingerprint . (5) is element-wise.

(3) and (4) that the accuracy of both PRNU K and noise residual W depends on the denoising operation applied to I in obtaining the most common method of obtaining . allowing the interpolation noise in the artificial components to contaminate the real PRNU in the physical components.e. as mentioned earlier that I is to apply the discrete wavelet transform followed by a Wiener filtering operation directly to the entire image I without differentiating physical components from artificial components and. such as chromatic aliases. rather than using the camera's built-in firmware. In this work. We can see from Eq. (2). as a result. The aim of a demosaicing algorithm is to reconstruct a full color image (i. in the rest of the work we will use the two terms. Eq.. 4.PRNU. (4)). interchangeably whenever there is no need to differentiate them. (3) and (5) as a function of the noise residual W (i. The algorithm should have the following traits:  Avoidance of the introduction of false color artifacts.3 DEMOSAICING A demosaicing (also de-mosaicing or demosaicking) algorithm is a digital image process used to reconstruct a full color image from the incomplete color samples output from an image sensor overlaid with a color filter array (CFA). Many modern digital cameras can save images in a raw format allowing the user to demosaic it using software. so demosaicing is part of the processing pipeline required to render these images into a viewable format. PRNU and noise residual. Because the PRNU is formulated in Eq. zippering (abrupt unnatural changes of intensity over a number of neighboring pixels) and purple fringing  Maximum preservation of the image resolution . we will look at the impact of demosaicking on PRNU fidelity in Section II and propose an improved formula for extracting PRNU in Section III.e. Addressing this shortcoming is the motivation of this work. It is also known as CFA interpolation or color reconstruction. we present some experiments on device identification and image content integrity verification to validate the proposed PRNU extractionformula. In Section IV. However. a full set of color triples) from the spatially under sampled color channels output from the CFA. Section V concludes this work. Most modern digital cameras acquire images using a single image sensor overlaid with a CFA.

e. which contains both artificial and physical components. Due to the fact that demosaicking is a key deterministic process that affects the quality of colour images taken by many digital devices. (4) and (5) based on the definition of the output image model in Eq. the colour variation between neighbouring pixels is greater. only one fourth of the pixels‟ red colour are physical and for each pixel with physical red colour all its 8 neighbours‟ red colours are artificial. the existing method for extracting PRNU as formulated in Eq. hence the accuracy of colour interpolation result is also content-dependent. Conversely. (4) and (5). each coefficient of the wavelet transform used in the noise residual extraction process involves multiple pixels and thus both artificial and physical components. (1) does not take this into account. For example. demosaicking has been rigorously investigated.4 DEMOSAICKING IMPACT ON PRNU FIDELITY In this work. Therefore it is desirable to devise a noise residual extraction method that can . However. a form of interpolation is needed to fill in the blanks. because of the low variation of the colour intensities of neighbouring pixels. edge-adaptive or non-adaptive. Most demosaicking approaches group the missing colours before applying an interpolation function. The mathematics here is subject to individual implementation. in the red colour component/plane of an image taken by a camera with a Bayer CFA. The grouping process is usually content-dependent. When wavelet transform is applied during the noise residual extraction process the interpolation noise residing in the artificial components propagates into the physical components. we call the colour components physically captured by the sensor as physical colours and the ones artificially interpolated by the demosaicking function as artificial colours.. in one noise residual extraction process. in a homogeneous area. However. To extract the PRNU using Eq. the discrete wavelet transform followed by a Wiener filtering operation is applied. 4. in inhomogeneous areas.g. (4) is that it involves the whole image plane. and is called demosaicing. The main problem inherent to Eq. thus the interpolation noise is also more significant. As a result the interpolation noise gets diffused from the artificial components into the physical ones. This indicates that the PRNU in physical colour components is more reliable than that in the artificial components. Low computational complexity for fast processing or efficient incamera hardware implementation  Amenability to analysis for accurate noise reduction To reconstruct a full color image from the data collected by the color filtering array. For example. the interpolation function can more accurately generate artificial components.

.prevent the artificial components from contaminating the reliable PRNU residing in the physical components with the interpolation noise.

e . F. According to Eq. without prior knowledge about the CFA. i. (7) is artificially applied to calculate the colour. the image output model of Eq. Although the 2×2 Bayer CFA is the most common CFA pattern. to make the proposed CD-PRNU versatile and applicable to cameras adopting different CFA patterns. we will discuss the formulation and extraction of CD-PRNU. For each colour component of a pixel . Otherwise.CHAPTER 5 CD-PRNU (Color Decoupled Photo Response Non-Uniformity) 5. First. That is to say that for each pixel .1 FORMULATION OF COLOUR DECOUPLED PRNU (CD-PRNU) In this section. (1) proposed in can be reformulated as . then no demosaicking is needed because c has been physically captured by the sensor. a mathematical model for the CD-PRNUis derived and then an extraction algorithm is proposed to extract the noise residual that is to be used for estimating the final CD-PRNU. only one of the three colour components takes a value physically captured by the sensor and this colour is determined by the colour configuration of the CFA pattern F. R (red). G (green) and B (blue) before colour interpolation.2 Mathematical Model of CD-PRNU A generic demosaicking process is to convolve an interpolation matrix with an image block of the same size centred at the pixel where the artificial colour is to be calculated. can be determined according to The first part of Eq. (7). The other two colour components are to be determined by the demosaicking process. Let be an interpolation matrix with 2N+1 × 2N+1 coefficients and be a X × Y-pixel input signal from the scene consisting of three colour components. except that it is a 2 × 2 square array. (7) means that if the colour component c is the same as the colour that the CFA pattern F allows to pass. 5. the second part of Eq. we makes no assumption about the CFA pattern.

the PRNU is actually the interpolation noise P while. That is where is a low-passed filtered version of the artificial components and is the corresponding ―sensor pattern noise‖. 5..3 CD-PRNU Extraction Algorithm . Eq. (9) that the physical components and artificial components have similar mathematical expression. P can be extracted in the same way as the sensor pattern noise K is extracted (i. which is actually the interpolation noise.Eq. in the physicalcomponents. We can also use the same ML estimate as in Eq. It can also be seen from Eq.e. (3)). (5) to extract the reference interpolation noise d from S low-variation images taken by d such that for a particular device where is the artificial colour components of the s-th low-contrast image taken by device d and . the PRNU remains unaffected by the interpolation noise. Hence if the physical and artificial colour components can be separated / decoupled.B. (9) suggests that in the artificial components. We will discuss how the physical and artificial is the interpolation noise extracted from colour components can be decoupled in simple manner without a priori knowledge about the CFA pattern in Section III.

several methods have been proposed to estimate the CFA. we perform a 2:1 down-sampling across both horizontal and . . these methods have to exhaust all of the possible CFA patterns in order to infer/estimate the „real‟/optimal CFA. we can prevent the artificial components from contaminating the physical components during the DWT process. Unfortunately. we can extract the sensor pattern noise and interpolation noise. (10) and (11). we do not know (actually we do not have to know) which pixels carry the colour captured physically by the hardware and which are not. . such that vertical dimensions to get four sub-images. we know that each of the four sub-images either contains only the physical colour or only the artificial colours. manufacturers usually do not provide information about the CFA used by their cameras. In this work. (4) is then used to obtain noise residual from each sub-images . and mod is the modulo operation. respectively. Most CFA patterns are of 2 × 2 elements and are periodically mapped to the sensors. The framework of the colour decoupled noise residual extraction process is shown in Figure 2 and the procedures are listed in Algorithm 1. only one of the three colour components is physical and the other two are artificial. Eq. For each colour channel. without knowing the CFA pattern used by the manufacturer. However. Eq. exhaustive search is by no means acceptable. to extract the CD-PRNU. . Therefore. We know that. for each pixel of I. we first separate the three colour channels of a colour image I of pixels. To estimate the CD-PRNU Pd of a particular device d and use it as the reference signature of d. Note that Algorithm 1 is for extracting the noise residual pattern W from an image I. However. so the second step is. By decoupling the physical and artificial colour components in this fashion before extracting the noise residual.According to Eq. Finally the CD-PRNU Wc of each colour channel c is formed by combining the four sub-noise residuals such that where. (11) is applied. But by decomposing into four sub-images. for each channel . from the physical and artificial components if the CFA is known.

.

we need clean PRNUs (which appear as high frequency bands of images) as device .5. Table 1. Source Camera Identification We have carried out source camera identification tests on 300 2048×1536-pixel photos of natural scenes taken by six cameras(C1 to C6). Cameras used in the experiments. For device identification purpose. (11).5 EXPERIMENTAL RESULTS In this section. The six cameras are listed in Table1. each responsible for 50. Noise residual extraction algorithm Input: original image I Output: colour decoupled noise residual W Noise residual extraction algorithm 5.e. The reference PRNU (i. 5.5.4 Algorithm 1.1. we carry out experiments on source camera identification and image content integrity verification to validate the feasibility of the proposed CD-PRNU in a comparative manner. ) of each camera Ci is generated by taking the weighted average of the PRNUs extracted from 30 photos of blue sky according to Eq.

such as rotation or scaling. PRNU performs equally well as CD-PRNU when the image size is 192 × 256 pixels and slightly outperforms CD-PRNU when the block size is 48 × 64 pixels.e. C4. Source camera identification requires similarity comparisons among PRNUs (CD-PRNUs) and therefore the feasibility of the chosen similarity metrics is important. Our empirical experience suggests that an average of 20 blue sky images is accurate enough. we also compare the performance of the proposed CD-PRNU against that of the traditional PRNU [11] when they are applied to blocks of 5 different sizes cropped from the centre of the fullsized PRNU (CD-PRNU). in the following experiments.0000). C1. so geometrical transformations will not be applied in order to prevent biased evaluation from happening. The reason blue-sky images are chosen in this work is because blue sky contains less scene details (high frequency signal). We suspect that the reason C2 does not perform as expected is because the CFA pattern is not a 2 × 2 square array as we have assumed. its reference PRNU (or CD-PRNU) is most similar to the PRNU (or CD-PRNU). the less data is available. normalised cross-correlation formulated as in Eq. except for a few cases when images are of full size (1536 × 2048 pixels) and the identification rates are close or equal to 100% (1. Because PRNU is often used in content integrity verification. a camera is identified as the source camera. C3. Table 2 lists the identification rates. . the normalised cross-correlation has to be greater than a specified threshold for a camera to be identified as the source camera. of the image I under investigation. if out of the six reference PRNUs (or CDPRNUs). However. thus giving better chance of extracting clean PRNU. Individually speaking. scenes without significant details) can be used instead. other images with low-variation scenes (i. where smaller image blocks have to be analysed. C5 and C6 perform significantly better when CD-PRNU is used in all cases. For C2. Fridrich suggested the use of the Peak to Correlation Energy (PCE) measure in [15]. Therefore. because the smaller the images. which has been proved to be a more stable detection statistics than normalised cross-correlation when applied to the scenarios in which the images of interest may have undergone geometrical manipulations. Actually. (6) will be used to measure the similarity between PRNUs (CD-PRNUs). The purpose of this experiment is to demonstrate the capability of the proposed CD-PRNU in dealing with the colour interpolation noise.. WI. Taking the average of the PRNUs from 30 blue sky images is to further reduce variation. Another reason is that. therefore identification results become less reliable. the key point is about demonstrating the different performance of the traditional PRNU and the proposed CD-PRNU.fingerprints for comparison against the PRNU extracted from individual images under investigation. Therefore. in this experiment. In practice. Generally speaking.

a ROC curve of the performance of PRNU and CD-PRNU are demonstrated. This has been graphically presented in Figure 3(a).when the statistics of the six cameras are pooled together. as listed in the Total column of Table 2. We can see that the CD-PRNU outperforms the PRNU because at all fixed False Positive rate the CDPRNU‟s True Positive rate are always higher than that of the PRNU. we can see that CD-PRNU still outperforms PRNU significantly. . Performance comparison of source camera identification a) Overall identification rates when CD-PRNU and PRNU are used as fingerprint In Figure 3(b). Figure 3.

. these two operations are trivial and only incur negligible increase of time complexity.567 seconds to calculate the similarity between a pair of CD-PRNUs of the same size. Performance comparison of source camera identification b) Overall ROC curve when CDPRNU and PRNU are used as fingerprint For a system with a Pentium Core II 1.3G CPU and 3 GB RAM. Although extracting CD-PRNU requires down-sampling and up-sampling. it takes 0.526 seconds to compute the similarity between the PRNUs of two images of 2048 × 1536 pixels and 0. The amount of data processed during the extraction of PRNU and CD-PRNU is the same.Figure 3.

we copied a 160 × 390-pixel area from Image I.2 Content Integrity Verification We also carried out the following three content integrity verification experiments on 640 × 480-pixel images.Table 2.3 as shown in Figure 4(c). 5.5. Source camera identification rates using traditional PRNU and proposed CD-PRNU. The images in Figure 4(a) and (b) are taken by Olympus C730.1 in Figure 4(a). . and pasted it at approximately the same location in Image I.  In the first experiment.2 in Figure 4(b) to create the forged Image I.

Figure 6. the windows are .2 and III.2 in Figure 5(b) to create the forged Image II.3  In the second experiment. Figure 5.2 (c) Forged Image I.1 (b) Original Image III.1 in Figure 5(a). we cropped an 80 × 100-pixel area from Image II. (a) Original Image I. which covers the face of the person. The original image. source image and forged images for the content verification experiments.2.2 in Figure 6(b). which is taken by Olympus C730. pasted it at the area where the face of another person is in Image II. we cropped a 60 × 80-pixel area from Image III. The original image. which covers the face of the person. II. (a) Original Image II.2 (c) Forged Image II. to create the forged Image III.2. source image and forged images for the content verification experiments. (a) Original Image III. The images in Figure 5(a) and (b) are also taken by the same camera.3 To detect the manipulated areas. In Chen’s method [11]. source image and forged images for the content verification experiments.3  In the third experiment.Figure 4.1 in Figure 6(a) taken by Canon Power Shot A400.3 in Figure 5(c).1 (b) Original Image II.2 (c) Forged Image III.3 in Figure 6(c). pasted it at the area where the face of another person is in Image III. we slid a 128 × 128-pixel window across the PRNU extracted from the image under investigation and another window of the same size across the reference PRNU of the cameras that have taken images I. The original image.1 (b) Original Image I.

By varying the value of t. and lower FP and FN indicate better performance. we use various thresholds defined as to analyze the performance of PRNU and CD-PRNU. (6).0 to 3. true positive (TP). Moreover. the cross-correlation of the PRNU patterns inside the two windows at the same location was calculated according to Eq. FP = |{B | M(B) = 0 and Md(B) = 1}| and FN = |{B | M(B) = 1 and Md(B) = 0}|. which incurs a high computational load. this method is not accurate at the pixel level [11]. TN and FN are defined as TP = |{B | M(B) = 1 and Md(B) = 1}|. Let B be an arbitrary block and M(B) and Md(B) be defined as TP. Table 3 lists the number of manipulated and non-manipulated blocks of 5 × 5 pixels in the forged images. If the cross-correlation is lower than a predetermined threshold t. false positive (FP).0 and use the four metrics. we will obtain lower TP and FP. and T(t) is the threshold. where and are the mean and standard deviation of the correlations distribution. Table 3. As discussed in [11]. in our experiment. therefore. the cross-follows the Generalized Gaussian (GG) distribution. while higher TN and FN. the block in the centre of the window is deemed as manipulated. To decide whether a block centered at the window superposed on the image has been manipulated or not. the sliding step/displacement is set to 5 pixels in order to reduce the computational load without sacrificing the accuracy of the integrity verification. . we can evaluate the integrity verification performance across a wide range of correlation thresholds T(t). TN = |{B | M(B) = 0 and Md(B) = 0}|. As t grows. Therefore. Number of manipulated and non-manipulated areas in each image (unit: block). FP. Higher TP and TN. true negative (TN) and false negative (FN) to measure the performance of integrity verifications based on PRNU and CD-PRNU. respectively. In the following experiments we will allow t to vary independently in the range from 0.moved a pixel at a time.

3 Figure 7 shows the performance of the PRNU and CD-PRNU in terms of TP. A lower correlation (similarity) allows the algorithm to detect more manipulated blocks. leading to higher TP. The ROC curves for the integrity verification experiments on image I. Thus. therefore the PRNU pattern noises in the two areas are almost the same. the ROC is formulated as At the same false positive rate .2. This is because the area we copied from the source image I. a low threshold also results in the situation where more authentic blocks are mistakenly detected as manipulated. FP and FN. FP and FN when authentication is carried out on image I. TN. the performance of the filtering / dilation operation strongly depends on parameter setting and hence many experiments must be run to obtain the best parameters for filtering. In order to simplify the comparison and to obtain a fair result. the experiments on III. As a result.According to Chen‟s predication. TN. However. However.5.3 demonstrate that CD-PRNU-based method significantly outperforms the PRNU-based method when the tampered area is about one quarter of the sliding window.3 across a range of correlation threshold T(t). As a result. It is clear that the ROC curve of the PRNU-based scheme mostly overlaps with that of Random Guess. giving rise to a higher FP.2. 5. which means the authentication result is generally as unreliable as that of a random guess. ―the block dimensions impose a lower bound on the size of tampered regions that our algorithm can identify. We can see from Figure 7(a) and 7(b) that CD-PRNU generally achieves higher TP and TN while maintaining lower FP and FN.1 Experiment on Image I. we remove all simply connected tampered regions from Z that contain less than 64×64 pixels (one quarter of the number of pixels in the block)‖. By . Therefore a ROCcurve of TP rate with respect to FP rate can be used to evaluate the overall performance of the PRNU and CD-PRNU. an algorithm with better performance will have a higher true positive rate (). the scheme cannot detect the manipulated area based on PRNU. we use the raw data without any filtering to calculate the TP. Chen applies erosion and dilation operations with a square kernel in order to filter small areas identified as tampered with.3 is illustrated as Figure 8. Let α be the number of manipulated blocks and β be the number of authentic blocks. The final authentication result is a image with the dilated areas highlighted as the tampered areas.1 is at approximately the same location as the original area in image I. which is marked vertically. which is marked along the horizontal axis of the ROC curve.

The ROC curve of Truth Positive Rate with respect to False Positive Rate of PRNU and CD-PRNU when authentication is performed on image I. 5.3: Integrity verification performance of the PRNU and CD-PRNU in terms of a) TP. Figure 8. which means that by using CD-PRNU manipulated blocks can be detected more reliably.5. across a range of correlation threshold T(t).0.0 to 3. with t varying from 0.3 . Authentication results on image I. the CD-PRNU-based scheme results in a curve much higher than the PRNU-based method. Figure 7.3.2.2 Experiment on Image II.contrast. b) TN.

Authentication results on image II.3.0 to 3. with t varying from 0. However. CD-PRNU‟s consistently higher TP and lower FN. as shown in Figure 9(a) and 9(d). Albeit their mixed performance in terms of TN and FP.When verifying the integrity of image II.3: Integrity verification performance of the PRNU and CD-PRNU in terms of a) TP. .0. Figure 10 also shows that the ROC curve of CD-PRNU is still slightly higher than that of PRNU. c) FP and d) FN across a range of correlation threshold T(t). Figure 9. indicating a slightly better performance of CD-PRNU. mixed performance in terms of TN and FP can be seen in Figure 9(b) and 9(c). again indicate its superiority to PRNU. both PRNU and CD-PRNU can effectively detect the manipulated blocks as their ROC curves have suggested in Figure 10. b) TN.

respectively. as demonstrated in Figure 11(a). CD-PRNU‟s significantly better performance in terms of TP and lower FN can still be seen again in Figure 11(a) and 11(d).3 Experiment on Image III. approximately one quarter of the window. the manipulated blocks can be effectively detected by the CD-PRNU-based scheme because the areas in question are from two images taken by different cameras and thus contain different interpolation noise.3.1.e. and therefore areas smaller than this should be filtered in order to remove the falsely identified noise.2.5. When the threshold t is higher than 1.3 When authenticating III. Since the tampered area is 60 × 80 pixels. 5. the PRNU cannot correctly detect any manipulated blocks (i. as can be seen in Figure 11(b) and 11(c). This poor performance is also reflected in the PRNU’s ROC curve in Figure 12 and is due to the fact that he manipulated area is too small (60 × 80 pixels).3. The ROC curve of Truth Positive Rate with respect to False Positive Rate of PRNU and CD-PRNU when authentication is performed on image II. As a result. the method based on PRNU can perform no better than a random guess. which is only about one quarter of the sliding window (128 × 128 pixels). the CD-PRNU-based method can identify smaller areas. . although the performance of PRNU and CD-PRNU in terms of TN and FP are mixed. The experiment result on III.3 conforms to Chen’s observation. Chen predicated in that one quarter of the sliding window is the lower bound on the size of tampered regions that our algorithm can identify. By contrast.Figure 10.

Figure 11. b) TN.0 to 3. with t varying from 0. Authentication results on image III.3: Integrity verification performance of the PRNU and CD-PRNU in terms of a) TP. . c) FP and d) FN across a range of correlation threshold T(t).0.

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.