CONCEPT OF

AMOUNT OF
INFORMATION AND
ITS PROPERTIES
AMOUNT OF INFORMATION
> >DEFINITION: Information generally refers to data that has been
processed or organized in a meaningful way to convey knowledge,
facts, or instructions. It can exist in various forms, including text,
numbers, images, sounds, and more. Information is the result of data
that has been given context, relevance, and purpose, making it
valuable and usable. Claude Shannon's information theory defines
information as the reduction of uncertainty or surprise when
receiving data. In everyday terms, it's what we gain when we learn
something new or receive meaningful content.
THIS CONCEPT IS FOR ANALOG OR
DIGITAL?
> The concept of the amount of information, as defined in information theory, is
applicable to both digital and analog data. It can be used to quantify and
measure the information content in a wide range of data types, regardless of
whether they are digital or analog.

> In the context of digital data, such as binary code, text, or digital images, the
amount of information is often measured in bits.

> In the context of analog data, such as continuous audio signals or analog
images, the amount of information can still be quantified using information
theory principles. However, it may be measured in units like nats or other
continuous scales, rather than discrete bits.
CONCEPT OF AMOUNT OF
INFORMATION
> Information theory is based on the idea that the amount of
information conveyed by an event or message is inversely
proportional to its probability. In other words, the less probable an
event is, the more information it carries.

> Let us consider the communication system transmits messages m₁,

m2, m3 with provability of occurance P1, P2, P3 ----
> The amount of Information transmitted through the message m, with
probability Pi is given as Amount of Information:
> Ik = log₂ [1/PK]
> Ik represents the amount of information in bits.
> log₂ is the logarithm base 2, which is commonly used in information
theory.
> 1/PK is the reciprocal of the probability PK of the event. In other words,
it's the probability's inverse.
> So, by taking the reciprocal of the probability and applying a base-2
logarithm, you can calculate the amount of information conveyed by an
event with a given probability. This formula tells you how many bits are
needed to represent or transmit that event efficiently.

> It's important to note that when PK is high (close to 1), the amount of
information is low, indicating that the event is not very surprising.
Conversely, when PK is low (close to 0), the amount of information is
high, indicating a more surprising or less expected event.
PROPERTIES OF AMOUNT OF
INFORMATION
> Additivity: The amount of information associated with independent
events or messages can be added together. If you have two independent
events, A and B, the total information is I(A and B) = I(A) + I(B).

> Inverse Relationship to Probability: The amount of information is

inversely proportional to the probability of an event. Less probable
events carry more information, while more probable events carry less.
> Maximum Information: The maximum amount of information is
associated with events that have a probability of 1 (certainty). In such
cases, the information content is zero because there is no
uncertainty.

> Minimum Information: The minimum amount of information is

associated with events that have a probability of 0 (impossibility). In
these cases, the information content is also zero because the event
cannot occur.
> Base Independence: The choice of logarithmic base (e.g., base 2,
base e) affects the unit of measurement (bits, nats) but doesn't
change the fundamental properties of information.

> These properties help us understand how information behaves and

how it can be quantified. They are fundamental to the field of
information theory and have applications in various areas, including
communication, data compression, and statistics.
 Zoho Show

Untitled Presentation.pdf
(This PDF has been generated using Zoho Show)

To create beautiful presentations, download Zoho Show from Play Store https://zoho.to/cy7