Multimedia Forensics is not Computer Forensics

Rainer Bohme , Felix Freiling , Thomas Gloe , Matthias Kirchner

Technische Universitat Dresden Universitat Mannheim

International Workshop on Computational Forensics 2009 (IWCF09) The Hague 2009/8/14

Outline

1 2 3 4

Multimedia forensics and computer forensics Multimedia forensics is not computer forensics Counter-forensics And how does this all relate to practice?

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 2 of 24

Multimedia forensics
A science to assess the authenticity of digital media objects

manipulation detection and source device identication based on artifacts of processing operations
resampling copy & paste inconsistent lightning double compression

characteristics of the source device

e. g. digital camera

lter

lens

R G

G sensor B

color interpolation

post processing

scene lens distortion CFA layout hot pixels, sensor noise interpolation scheme quantization table

digital image

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 3 of 24

Multimedia forensics: Examples

digital camera identication based on sensor noise

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 4 of 24

Multimedia forensics: Examples

digital camera identication based on sensor noise

?
The Hague, 2009/8/14 Multimedia Forensics is not Computer Forensics slide 4 of 24

Multimedia forensics: Examples

digital camera identication based on sensor noise

?
The Hague, 2009/8/14 Multimedia Forensics is not Computer Forensics slide 4 of 24

Multimedia forensics: Examples

digital camera identication based on sensor noise

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 4 of 24

Multimedia forensics: Examples

digital camera identication based on sensor noise copy & paste detection

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 4 of 24

Multimedia forensics: Examples

digital camera identication based on sensor noise copy & paste detection

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 4 of 24

By the way, what is computer forensics?

Computer forensics

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 6 of 24

Computer forensics

01

11

1 1

10

11 0

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 6 of 24

Computer forensics
52 49 55 62 66 51 40 48 58 62 51 51 36 34 40 33 45 33 53 34 49 33 23 22 22

01

01

1 1

00

01 1

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 6 of 24

Computer forensics
52 49 55 62 66 51 40 48 58 62 51 51 36 34 40 33 45 33 53 34 49 33 23 22 22

01

10

1 0

10

00 1

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 6 of 24

Outline

1 2 3 4

Multimedia forensics and computer forensics

Multimedia forensics is not computer forensics

Counter-forensics And how does this all relate to practice?

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 7 of 24

Digital forensics: proposed ontology

forensics 0 0 0 0 0 0 0 1 0 1 1 1 0 1 1 0 0 0 1 0 0 0 1 1 1 0 1 0 0 0 0 1 digital forensics 1 1 0 0 0 0 1 0 0 1 1 1 1 0 0 0 0 0 1 1 1 0 0 1 0 0 1 0 computer 1 0 0 multimedia 0 1 1 0 1 0 0 forensics 0 1 0 forensics 0 1 0 0 0 1 0 1 1 0 1 0 1 0 0 1 0 1 1 digital evidence 0 1 1 0 1 1 0 0 1 1 1 0 1 0 1 1 1 1

analog forensics

physical evidence

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 8 of 24

Digital forensics: proposed ontology

forensics 1 1 1 0 0 1 0 0 1 0 1 1 1 1 1 0 0 0 0 1 1 0 1 1 1 0 0 0 0 1 1 1 digital forensics 0 0 0 0 1 0 0 0 1 0 0 1 0 0 1 1 1 1 0 1 0 0 0 1 1 1 1 1 computer 1 0 0 multimedia 0 1 0 0 0 1 1 forensics 0 1 0 forensics 0 0 0 1 0 0 1 0 0 1 0 1 1 0 0 1 1 1 1 digital evidence 1 1 0 0 1 0 0 0 1 0 0 1 1 0 0 1 0 1

analog forensics

physical evidence

nite sequence of discrete and perfectly observable symbols

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 8 of 24

WARNING! The following slides intentionally draw a very black-and-white picture

Computer forensics = Multimedia forensics

computer forensics
physical evidence

multimedia forensics
physical evidence

WWW

digital evidence 1001 1101

digital evidence 1001 1101

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 10 of 24

Computer forensics = Multimedia forensics

computer forensics
physical evidence

multimedia forensics
physical evidence

WWW

digital evidence
WWW

digital evidence 1101 1001 1101

1001

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 10 of 24

Computer forensics = Multimedia forensics

computer forensics
physical evidence

multimedia forensics
physical evidence

WWW

digital evidence
WWW

digital evidence 1101 1001 1101

1001

digital evidence is not linked to the outside world

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 10 of 24

Computer forensics = Multimedia forensics

computer forensics
physical evidence

multimedia forensics
physical evidence

WWW

digital evidence
WWW

digital evidence 1101 1001 1101

1001

digital evidence is not linked to the outside world

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 10 of 24

Computer forensics = Multimedia forensics

computer forensics
physical evidence

multimedia forensics
physical evidence

WWW

digital evidence
WWW

digital evidence 1101 1001 1101

1001

digital evidence is not linked to the outside world

digital evidence is linked to the outside world

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 10 of 24

Computer forensics: A closer look

processing

digital data

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 11 of 24

Computer forensics: A closer look

suspicious traces?

processing

digital data

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 11 of 24

Computer forensics: A closer look

suspicious traces?

processing

digital evidence is stored in the nite automaton each computer represents number of states in a closed system is nite

digital data

reality

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 11 of 24

Computer forensics: A closer look

suspicious traces?

processing

digital evidence is stored in the nite automaton each computer represents number of states in a closed system is nite

digital data

reality

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 11 of 24

Computer forensics: A closer look

suspicious traces?

processing

digital evidence is stored in the nite automaton each computer represents number of states in a closed system is nite

digital data

non-negligible chance that a computer is left in a state which perfectly erases all traces

reality

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 11 of 24

Multimedia forensics: A closer look

processing

digital media object

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 12 of 24

Multimedia forensics: A closer look

original? processing

digital media object

source (device) ?

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 12 of 24

Multimedia forensics: A closer look

original? processing

digital media object

sensors capture parts of the reality and transform them into digital representations

source (device) ?

sensor

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 12 of 24

Multimedia forensics: A closer look

original? processing

digital media object

sensors capture parts of the reality and transform them into digital representations reality is incognizable: ultimate knowledge whether a piece of digital media reects reality or not cannot exist

source (device) ?

sensor

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 12 of 24

Multimedia forensics: A closer look

original? processing

digital media object

sensors capture parts of the reality and transform them into digital representations reality is incognizable: ultimate knowledge whether a piece of digital media reects reality or not cannot exist multimedia forensics = empirical science

source (device) ?

sensor

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 12 of 24

Sensors: A source of uncertainty

projection of reality to discrete symbols means a dimensionality reduction

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 13 of 24

Sensors: A source of uncertainty

degrees of freedom projection of reality to discrete symbols means a dimensionality reduction multimedia forensics has to cope with an additional source of uncertainty

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 13 of 24

Sensors: A source of uncertainty

projection of reality to discrete symbols means a dimensionality reduction multimedia forensics has to cope with an additional source of uncertainty

what kind of common post-processing is legitimate / tolerable?

?
The Hague, 2009/8/14 Multimedia Forensics is not Computer Forensics slide 13 of 24

Models: Yet another dimensionality reduction

models make projection of reality to discrete symbols tractable with formal methods typical models in multimedia forensics:
sensor noise follows a Gaussian distribution connected regions of identical pixel values are unlikely to occur in original images

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 14 of 24

Models: Yet another dimensionality reduction

models make projection of reality to discrete symbols tractable with formal methods typical models in multimedia forensics:
sensor noise follows a Gaussian distribution connected regions of identical pixel values are unlikely to occur in original images

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 14 of 24

Models: Yet another dimensionality reduction

models make projection of reality to discrete symbols tractable with formal methods typical models in multimedia forensics:
sensor noise follows a Gaussian distribution connected regions of identical pixel values are unlikely to occur in original images

p projection to a 1-dimensional variable

models of reality function as yet another dimensionality reduction quality of forensic methods depends on the quality of the employed model!

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 14 of 24

Models: Yet another dimensionality reduction

models make projection of reality to discrete symbols tractable with formal methods typical models in multimedia forensics:
sensor noise follows a Gaussian distribution connected regions of identical pixel values are unlikely to occur in original images

p projection to a 1-dimensional variable

models of reality function as yet another dimensionality reduction quality of forensic methods depends on the quality of the employed model!

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 14 of 24

Outline

1 2 3 4

Multimedia forensics and computer forensics Multimedia forensics is not computer forensics

Counter-forensics
And how does this all relate to practice?

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 15 of 24

Digital forensics: proposed ontology

forensics 0 1 0 0 1 0 1 1 0 1 0 0 1 0 0 1 1 0 0 0 0 1 1 0 0 1 0 1 1 0 1 0 1 1 0 0 1 digital forensics 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1 0 0 0 1 0 0 1 0 computer 0 0 1 multimedia 0 0 1 0 0 0 1 forensics 1 1 0 forensics 1 1 0 1 1 0 1 1 0 0 0 1 0 1 1 0 0 1 1 1 1 0 0 1 digital evidence 0 1 0 1 0 1 1 0 1 0 1 0 1

analog forensics

physical evidence

forgeability

= b counter-forensics

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 16 of 24

Digital forensics: proposed ontology

forensics 0 1 0 0 1 0 1 0 0 0 1 1 0 0 1 1 1 1 1 0 1 1 0 0 0 0 1 1 1 1 0 1 digital forensics 0 1 0 0 1 0 1 0 1 0 1 0 1 1 1 0 0 0 0 0 1 0 0 0 0 0 0 1 computer 1 0 0 multimedia 0 1 1 1 0 0 0 forensics 0 1 1 forensics 1 1 1 0 1 0 1 1 0 1 1 0 0 0 1 0 1 0 0 digital evidence 1 0 0 0 0 1 1 0 0 0 0 1 1 0 0 0 1 0

analog forensics
physical evidence cannot be wrong, it cannot perjure itself, it cannot be wholly absent Kirk (1953)

physical evidence

forgeability

= b counter-forensics

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 16 of 24

Counter-forensics: Computer forensics

leave traces

valid state

invalid state

Counter-forensics: Computer forensics

leave traces

eliminate traces

valid state

invalid state

valid state

Counter-forensics: Computer forensics

leave traces

eliminate traces

valid state

invalid state

valid state

valid states are perfectly known or can be recorded before

Counter-forensics: Computer forensics

preemptively avoid traces

leave traces

eliminate traces

valid state

invalid state

valid state

valid states are perfectly known or can be recorded before

Counter-forensics: Computer forensics

virtualization in a larger system
preemptively avoid traces

leave traces

eliminate traces

valid state

invalid state

valid state

valid states are perfectly known or can be recorded before

Counter-forensics: Multimedia forensics

virtualization in a larger system
preemptively avoid traces

leave traces

eliminate traces

valid state

invalid state

valid state

invalidity depends on the model of reality

valid states are perfectly known or can be recorded before

Counter-forensics: Multimedia forensics

virtualization in a larger system is not possible
preemptively avoid traces

leave traces

eliminate traces

valid state

invalid state

valid state

invalidity depends on the model of reality

valid states are not perfectly known or can be recorded before and cannot be recorded before

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 18 of 24

Digital forensics: proposed ontology

forensics 0 0 0 1 1 0 1 1 0 0 1 1 1 0 0 1 0 0 1 0 1 0 1 0 0 0 1 0 0 1 digital forensics 1 1 1 1 0 0 0 0 0 1 0 0 1 0 0 0 1 1 0 1 0 0 0 1 1 1 0 1 0 computer 1 1 0 multimedia 1 0 1 1 1 forensics 1 1 0 forensics 1 1 0 1 0 1 0 1 0 1 0 1 1 1 1 0 0 1 1 0 for 0 1 1 crime 0 1compete0 0 perfect 1 1 1 possible 1 1 the 0 0 model 0 1 1 0 best 0 0
forgeability

analog forensics

perfect crime impossible

= b counter-forensics

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 19 of 24

Outline

1 2 3 4

Multimedia forensics and computer forensics Multimedia forensics is not computer forensics Counter-forensics

And how does this all relate to practice?

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 20 of 24

Computer forensics in a broader sense

computers interact with their environment
physical evidence

WWW

digital evidence
WWW

1001

1101

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 21 of 24

Computer forensics in a broader sense

computers interact with their environment
physical evidence

computers can be part of a network

WWW

WWW

WWW WWW

digital evidence
WWW

1001

1101

WWW WWW

WWW

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 21 of 24

Computer forensics in a broader sense

computers interact with their environment
physical evidence

computers can be part of a network computers can be sensors itself

WWW

WWW

WWW WWW

digital evidence
WWW

1001

1101

WWW WWW

WWW

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 21 of 24

Computer forensics in a broader sense

computers interact with their environment
physical evidence

computers can be part of a network computers can be sensors itself computers leave physical evidence
WWW

WWW

WWW WWW

digital evidence
WWW

1001

1101

WWW WWW

WWW

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 21 of 24

(Finally) A more practical view

IWCF 09

IWCF 09

IWCF 09

IWCF 09

IWCF 09

2 7

2A IWCF 09

3 8

3A IWCF 09

4 9

4A IWCF 09

5 10

5A IWCF 09

6 11

6A IWCF 09

7A

8A

9A

10

10A

11

11A

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 22 of 24

Concluding remarks
forensic examinations include techniques from a variety of forensic sciences important differences in the underlying assumptions between different methods are blurred by practice in particular: digital evidence = digital evidence (= physical evidence): digital evidence in computer forensics is not linked to the outside world whereas in multimedia forensics it is effects the reliability of forensic methods furture work: rigorous probabilistic modeling

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 23 of 24

Concluding remarks
forensic examinations include techniques from a variety of forensic sciences important differences in the underlying assumptions between different methods are blurred by practice in particular: digital evidence = digital evidence (= physical evidence): digital evidence in computer forensics is not linked to the outside world whereas in multimedia forensics it is effects the reliability of forensic methods furture work: rigorous probabilistic modeling

reality is ultimately incognizable, but

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 23 of 24

Concluding remarks
forensic examinations include techniques from a variety of forensic sciences important differences in the underlying assumptions between different methods are blurred by practice in particular: digital evidence = digital evidence (= physical evidence): digital evidence in computer forensics is not linked to the outside world whereas in multimedia forensics it is effects the reliability of forensic methods furture work: rigorous probabilistic modeling

reality is ultimately incognizable, but your comments will help to gain a more comprehensive view on it

The Hague, 2009/8/14

Multimedia Forensics is not Computer Forensics

slide 23 of 24

Thanks for your attention

Questions?
Rainer Bohme , Felix Freiling , Thomas Gloe , Matthias Kirchner

Technische Universitat Dresden Universitat Mannheim

Matthias Kirchner gratefully receives a doctorate scholarship from Deutsche Telekom Stiftung, Bonn, Germany.

Image sources

Iranian missile test (4) hard drive (6) oppy disk (11,17) core memory (11) multimedia (12,18) ngerprints (22) handcuffs (22)

http://www.spiegel.de

http://commons.wikimedia.org/wiki/File:Open_hard-drive.jpg http://commons.wikimedia.org/wiki/GNOME_Desktop_icons http://commons.wikimedia.org/wiki/File:KL_CoreMemory.jpg http://commons.wikimedia.org/wiki/GNOME_Desktop_icons http://www.lanl.gov/news/albums/chemistry/fingerprint.jpg http://commons.wikimedia.org/wiki/File:Handcuffs01_2003-06-02.jpg