Professional Documents
Culture Documents
Information Theory
CHAPTER 9
Theorizing Information
for Information Science
Ian Cornelius
University College Dublin
Introduction
Does information science have a theory of information? There seems
to be a tendency within information science to seek a theory of informa-
tion, but the search is apparently unproductive (Hjorland, 1998;
Saracevic, 1999). This review brings together work from inside and out-
side the field of information science, showing that other perspectives on
information theory could be of assistance. Constructivist claims that
emphasize the uniqueness of the individual experience of information,
maintaining that there is no information independent of our social prac-
tices (Cornelius, 1996a), are also mentioned. Such a position would be
echoed in a symbolic interactionist approach.
Conventionally, the history of attempts to develop a theory of infor-
mation date from the publication of Claude Shannon’s work in 1948, and
his joint publication of that work with a n essay by Warren Weaver in
1949 (Shannon & Weaver, 1949/1963). Information science found itself
alongside many other disciplines attempting to develop a theory of infor-
mation (Machlup & Mansfield, 1983). From Weaver’s essay stems the
claim that the basic concepts of Shannon’s mathematical theory of com-
munication, which Shannon later referred to as a theory of information,
393
394 Annual Review of Information Science and Technology
This point, that sender and receiver must be in some form of social
relationship for there to be information, is reinforced by Gabor (1953),
who points out also that incomplete knowledge of the future is a require-
ment for information transfer. It is only if a message is not predictable
from data previously received that there can be information. So, follow-
ing Gabor, we can say that if there is perfect knowledge there can be no
information. This seems to remove any possibility of conflating the con-
cepts of knowledge and information, at least within information theory.
It also provides a basis for later formulations by Brookes (1980). Gabor
continues, “on the other hand, complete ignorance also precludes com-
munication; a common language is required, that is to say an agreement
between the transmitter and the receiver regarding the elements used
in the communication process” (Gabor, 1953, p. 2). Communication of
information presupposes a social system based on a shared language.
Except in the most artificially constructed of machine-machine environ-
ments, it seems that information must require a prenegotiated social
environment and language and an imperfect state of knowledge.
Mandelbrot (1953) extended the claims connected to information the-
ory by working on the statistical structure of language. This structure,
entirely independent of meaning, underlies meaningful written lan-
guages. Such a statistical structure, Mandelbrot hoped, would give a
quantitative justification to the assumption that the theory of commu-
nication (Shannon’s) applies also t o the brain. Moving information the-
ory away from its limitation to messages chosen from a finite set of
symbols would allow the extension of information theory to analysis of
messages in ordinary language.
Moving information theory away from a statistical basis to a seman-
tic basis was the aim of Bar-Hillel and Carnap (1953). They offered one
claim and one warning. First, the working premise that a semantic con-
cept of information will serve as a “better approximation for some future
explication of a psychological concept of information, than the statistical
concept of present day theory” (Bar-Hillel & Carnap, 1953, p. 503). The
limitations of their claim should be marked. Second, they observed that
404 Annual Review of Information Science and Technology
This reiterates MacKay’s (1969, p. 108) claim that all parties must be
in the same language game-substantially a similar point to one made by
Blair. Reviewing Blair’s work, Jacob and Shaw conclude that it shows
support for a sociocognitive perspective and for the socially constructed
nature of cognition itself (Jacob & Shaw, 1998, p. 146). Blair (1990, p.
134) demonstrates the difficulty in constructing any theory on the basis
of semiotics: “The problems with an ideational theory of signification
means that there can be no clear definition of the contentslsignifieds.”He
also demonstrates that Wittgenstein’s attribution of meanings of terms to
their common use is the most fruitful approach (Wittgenstein, 1953).
Measurement
The question of measurement, which is common to all approaches to
information theory, poses difficulties when we consider the different types
of measurement that can concern us. In the basic Shannon model of com-
munication, what is measured is either the amount of information, which
414 Annual Review of Information Science and Technology
Misinformation
The problem of misinformation can be represented at both philosoph-
ical and pragmatic levels. Godfrey-Smith (1989), tackling a slightly dif-
ferent problem, indicates that this problem of misinformation, or error,
may be unresolvable. With misinformation there are, transparently,
problems regarding measurement and meaning. Buckland (1991, pp.
111-112) introduces a complication by associating misinformation with
harm and distress, and therefore with some social and moral values. Fox
formulates misinformation as a falsehood, but maintains that misinfor-
mation remains a species of information (Fox, 1983, p. 193). Information
may not be true, but misinformation must be false. Dretske (1981, p. 45)
opposes this, claiming that, as information to be information must have
a truth value, then misinformation cannot be information. Losee (1997)
works through this case.
How can we measure this? The only sure method is within
Shannon’s original formulation of the process by which he accounts for
noise within a system (noise will distort a signal, thereby distorting its
message, possibly but not necessarily t o the point of falsehood). Levi’s
416 Annual Review of Information Science and Technology
(Dretske, 1981, pp. 81-82). This, too, is intuitively true, and corresponds
with all that has been said previously about senders and receivers being
in the same social practice and understanding the same language
games. If I interrogate an information system, I will reach a point where
I conclude that I have the answer to my question. How do I know when
that can be? It must be that, before I begin the interrogation, the for-
mulation of my enquiry reveals to me what would constitute an answer.
Information is a function of what I already know. In building his picture
of knowledge, Dretske claims that we know something by virtue of hav-
ing information about it, placing a condition on information that it be
true. He also, importantly, brings in the notion of learning. We are able
to build knowledge and to infer certain facts by the recursive nature of
information exchanges: We learn from previous requests for informa-
tion. The role of learning in building knowledge, whereby accretions of
information are worked upon in a learning process to produce inferences
that constitute knowledge, is also dealt with by Barnes (1981).
Barwise and Perry (1983) also tackle the problem of meaning. For
them, meaning arises out of recurring relations between situations.
Information can only be information about a structured reality, and is rel-
ative to the constraints holding between types of situations. Information
is provided only if the perceiving organism is attuned to those constraints
(Barwise & Perry, 1983, p. 120). The primary function of language, they
say, is to convey information; but that requires one to have some infor-
mation to convey. It is the meaning of expressions that allows them to
convey the information that they convey. Possessing information is like
knowing o r believing; and is therefore what they call an attitude. Having
information comes, rather as Dretske claims, before language; before any
expression of the information. The deep analysis of situations and atti-
tudes offers a means to investigate and understand the process of trans-
ferring information to another person-to give him or her certain
knowledge. This offers a more convincing approach to the elements in the
information models used in information science that deal with the con-
cept of transforming information offered to someone into knowledge for
that person-a process that will amend his or her knowledge structure.
This work has something to offer information science in its construction
of information theory. Green (1991) noted, from an analysis of informa-
tion science’s own linguistic usage, that the field lacked a coherent model
418 Annual Review of Information Science and Technology
this view leads to information, following the subject’s beliefs about the
world. Evans (1982, p. 23) prefers the notion of “being in an informa-
tional state with such-and-such a content,” t o explain the data that, for
any reasoning subject, are already imbued with objective significance
and have a propensity to influence our actions. The informational states
do not require us t o believe the truth of the information-we can listen
to a story without necessarily believing it to be true. This seems to be an
objection to the requirement that information be true.
These additional comments on information from outside the field of
information science show that there is significant work, relevant to some
of the operations that are a central concern to information science, that
could advance our understanding of information and help refine any the-
ory of information we might have. To relate theoretical work on infor-
mation to this external work would also link us more closely to general
academic enquiry. What might have hampered such development is the
practical orientation of information science, particularly in information
retrieval research. Our disinclination, as a field of study. to become
heavily involved with mathematics should not deter us from continued
use of logic. The practical work in information retrieval being done by,
for example, van Rijsbergen (19961, or van Rijsbergen and Lalmas
(1996), with its orientation to solving system problems, should not con-
ceal the fact that the foundation for such work is an analysis of infor-
mation informed by the best logical and epistemological commentary. We
should not be afraid to add to that work done in the social sciences that
emphasizes socially situated decision making, and the unique situation
of any recipient of information.
Conclusion
Theories are tools of enquiry. They are good as long as they are use-
ful. Their use is measured by the extent to which they can explain or
predict: A good theory will explain or predict more things for a longer
period. Theory development is part of the working apparatus ofa field of
study, and the facility to develop theory must itself be kept in good order.
Constructivists will argue that theorizing is part of any practice
(Cornelius, 199613). In this view, theories can be an implicit part of the
practice, rather than some explicit formulation external to the inevitable
420 Annual Review of Information Science and Technology
reflection of any practitioner. A discipline may feel the need for, or want
of, theory, both for behavioral and scholarly reasons. The scholarly or
scientific desire to have a theory that explains basic concepts in the field,
establishes conventional definitions, and addresses central problems is
understandable. The behavioral need for theory in order to establish a
field‘s claim to academic status, or to support the endeavors of its prac-
titioners, is equally understandable but less productive. Information sci-
ence should be clear regarding why it seeks a theory of information.
There is apparently little problem about information theory in infor-
mation science. The challenge to the dominance of the cognitive view-
point is essentially to seek a revision of that theory. Calls for more
radical departures, like the development of a hermeneutic theory of
information, have as yet no realization. An apparent tendency to con-
centrate on problems associated with the issue of representation
(Hjorland, 1997; Jacob & Shaw, 1998) merely mirrors the computer sci-
ence, or systems, tendency to concentrate on what can be realized within
systems. However, attempts to build information contexts into informa-
tion retrieval programs are disadvantaged by a limited view on what
constitutes context. The efforts to attach information theory to wider
social communication processes are hampered by a failure to show how
systems can be appropriately amended. The two main problems within
information science are structural. First, a failure to establish the
framework for information theory that will serve information science as
a whole: It is as if we do not really need a theory of information. Second,
a failure t o establish a convincing relationship between information,
knowledge, and models of learning: It seems as if we are concentrating
upon the technical and pragmatic tasks. It may be simply a feature of
the practice of information science that we need to express our regret at
the absence o f grand theory of information, but are content to operate at
the level of our own isolated technical problems.
This review shows a concentration in information science on modeling
operations, rather than constructing and analyzing grand theory. We can
build on the more vertically integrated analyses of data-information-
knowledge and use more sophisticated explanations of the “water into
wine” moment of information ingestion. If we build better representations
of the knowledge that we claim to organize and retrieve for all users, we
will improve our professional claim to resources from our society to do the
Theorizing Information for Information Science 421
Acknowledgments
The author wishes to thank the editors and anonymous reviewers for
their helpful and constructive comments, and to acknowledge the signif-
icant contribution they have made to this paper.
Bibliography
A g e , P. E. (1995). Institutional circuitry: Thinking about the forms and uses of
information. Information Technology and Libraries, 14, 225-230.
Artandi, S. (1975). Theories of information. In N. R. Stevens (Ed.), Essays for
Ralph Shaw (pp. 157-169). Metuchen, NJ: Scarecrow Press.
Baird, J . C. (1984). Information theory and information processing. Information
Processing & Management, 20, 373-381.
Bar-Hillel, Y., & Carnap, R. (1953). Semantic information. In W. Jackson (Ed.),
Communication theory: Papers read at a symposium on “Applications of
Communication Theory” held at the Institute of Electrical Engineers (pp.
503-512). London: Butterworths.
Barnes, B. (1981). On the conventional character of knowledge and cognition.
Philosophy of the Social Sciences, 11, 303-333.
Barwise, J., & Perry, J. (1983). Situations and attitudes. Cambridge, MA: MIT
Press.
Belkin, N. J. (1978). Information concepts for information science. Journal of
Documentation, 34, 55-85.
Belkin, N. J. (1990). The cognitive viewpoint in information science. Journal of
Information Science, 16, 11-15.
Blair, D. C. (1990). Language and representation i n information retrieval. New
York: Elsevier.
Boyce, B. R., & Kraft, D. H. (1985). Principles and theories in information sci-
ence. Annual Review of Information Science and Technology, 20, 153-178.
Brier, S. (1992). Information and consciousness: A critique of the mechanistic
concept of information. Cybernetics and H u m a n Knowing, 1(2/3), 71-94.
Brittan, A. (1998). Symbolic interactionism. In E. Craig (Ed.), Routledge
Encyclopedia of Philosophy, pp. 243-245. London: Routledge.
Broolres, B. C. (1980). The foundations of information science. Part I.
Philosophical aspects. Journal of Information Science, 2, 125-133.
Buckland, M. (1991). Information and information systems. New York: Praeger.
422 Annual Review of Information Science and Technology