Professional Documents
Culture Documents
net/publication/303548290
CITATION READS
1 44
2 authors:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Guy André Boy on 31 October 2016.
Proceedings of the European Cognitive Science Conference, Osnabrück, Germany, 11 September 2003
need to cooperate. For that matter, traceability of
decisions is crucial.
In this paper, we will present specific issues brought Dealing with complexity
by the design of safety-critical systems and human Even if the designers of safety-critical systems should
factors related to documentation generated and used in always have in mind to design for simplicity, what they
design processes. We will also focus on related current have to do is inherently complex. Systems are complex
design issues. The specificity of safety-critical design- and processes to design and develop these systems are
knowledge will be presented. Several KM management complex. In the design process, designers rely on
solutions will be discussed. The paper concludes with a knowledge that is available in the form of handbooks,
discussion on the difficulties and challenges of KM in lessons learned, and best practices. Designers have to
engineering. take into account the experiences with older systems, on
The paper is coming from several knowledge which the new system is usually building, making sure
management projects performed in cooperation with that incidents and accidents that have happened are no
groups of engineers in large aerospace and longer possible in the new design. Designs are verified
telecommunication companies. In particular, most and validated in extensive, well-defined processes. In
recent findings come from the European Research and the end of the design process, certification by different
Development project WISE (IST-2000-29280). In authorities and certification bodies can also play a large
WISE (Web-Enabled Information Services for role. In order to get a system certified, one has to be
Engineering) we study work-practices of engineers in able to justify the choices that were made, to prove, as
large manufacturing companies and we design practical far as possible, that all knowledge about problems with
methods to easily share and access essential knowledge similar systems has been taken care of, and that the
and information for their tasks. These methods are system will function safely in all kind of difficult and
supported by the development of an engineering even disastrous scenarios.
knowledge portal application. The industrial partners
involved in this project are Nokia and Airbus. Other Targeting completeness in an open world
partners are Cyberstream Interface SI,, PACE, Safety-critical systems require complete definition of
EURISCO International, Norwegian Computing Centre their (cognitive) functions that they involve in terms of
Helsinki University of Technology, and Technical role, context of validity and use, and appropriate
University of Berlin. resources that they need to accomplish their functions
(Boy, 1998). A cognitive function analysis is usually
Designing safety-critical systems mandatory when we need to demonstrate that the
Safety-critical systems have specific properties that system being designed satisfies a set of safety
directly affect the way knowledge management is requirements. Completeness does not apply only to the
carried out. Examples of safety-critical systems are mandatory kinds of functions but also to the situations
aircraft, power plants, medical equipment, and that end-users may encounter when they are using the
telecommunication systems. They are basically systems. Completeness is difficult and often impossible
complex, as complete as possible and described by to reach. This is why groups that design safety-critical
mature knowledge. Safety is not only a matter of end- systems use simulators in order to multiply the number
user emotion, attention and cognition, it is also a matter of situations and cover a broader spectrum. They
of organization and people involved in the whole life- incrementally accumulate and articulate related
cycle of related products. They involve experts that knowledge by categorizing relevant situations.
Proceedings of the European Cognitive Science Conference, Osnabrück, Germany, 11 September 2003
Maturity of knowledge and maturity in design Team-3 did a few years ago on the same topic or a
We claim that knowledge is constructed; let’s say similar one? How can we increase awareness? In some
designed. The design of knowledge is incremental. cases, it would be nice to have the appropriate
Safety-critical systems are designed over time. They are information pushed to the front so potential users are
tested, modified several times and certified. Their use is aware of its existence. In addition, efficient search
carefully observed and documented. The resulting mechanisms should provide the necessary means to pull
observation product, usually called experience appropriate information when needed. In both cases,
feedback, is provided to designers who use it to modify context-sensitive algorithms, that may take the form of
their current understanding of the artifacts they have software agents, are necessary.
designed. Knowledge about these artifacts becomes When designers know about a type of incident or
progressively mature through this incremental process. accident that involved a piece of equipment that they
There are short-loop design-knowledge validation are designing, they (at least try to) design artifacts in
processes that lead to official documents guiding the order to provide users with the necessary means to
design process. There are also long-loop design- handle related situations in the best possible way. They
knowledge validation processes that involve experience are expert in their field, i.e., design. People who are
feedback on already mature artifacts. In particular, likely to provide this “incident/accident” knowledge are
engineers involved in safety analysis have an everyday human factors specialists, end-users themselves or
need in using internal official documents. For example experiences laid down in appropriate databases and
for a system safety analyst, requirements, courses, knowledge bases. In any case, experts need to
applicable documents, lessons learned (In-Service communicate either in a live way, e.g., using computer-
experience), FAQ, list of experts, previous similar supported cooperative work environments, or a remote
deliverables, review results, validation and verification way, e.g., using knowledge bases. Space-time
checklists/action-lists, and system review action lists constraints usually impose choices in the way such
are crucial information that needs to be easily accessed. communication would happen.
Proceedings of the European Cognitive Science Conference, Osnabrück, Germany, 11 September 2003
clearly understood. Consequently, knowledge should support, experience feedback and a large variety of
come with contextual information that reinforces our official documents. It has been observed that people
understanding of its maturity and context if use. who are already in senior positions in an organization
In the study of Bonini, Jackson and McDonald know the benefit of good documentation, and tend to
(2001) three dimensions of trust were found of write more that younger employees who do not have as
importance: belief, dependence and experience. If you much experience. Document content should satisfy the
have to trust the information coming from others, you objectives, i.e., answering the question: why and for
have to be confident in the other and the information whom are we writing this document?
provided, you are dependent because you need the In addition, in international environments such as
information and you rely on the experience you have contemporary European multi-national companies,
with this person and the information. In design writing in English may be a difficult task for non-native
processes, the designer is regularly in a dependent English-speaking personnel. The result is that produced
position, because preliminary versions are shared English-written documents may be difficult to
between group members and designs of other, related understand
systems are often also in a not yet stable version
(participatory design). The time-for-writing issue
Especially in the design of safety critical systems Project deadlines are always very short and do not
one has to make sure that the knowledge that is shared allow for enough time for decent writing. In an
is correct and can be trusted. For this reason extensive engineering organization, the real job is design, not
validation and document version management is in writing. People are usually awarded on design
place in industries. One should avoid the risk to base performance issues, not on documentation issues.
one’s design on information that has not been verified Writing time should be clearly planned in a project
and designers should be aware what the latest version of schedule and given the same priority as other activities,
a document is in order to use it. so that when there is an extension in the duration of the
project, writing is not the last item on the agenda when
there is little time left to perform it, as it is often the
Design is writing and writing is design
case.
Knowledge management for safety-critical systems
mostly deals with documentation since everything What is obvious for someone (expert) is not
should be traceable and formally validated. necessarily for someone else
Consequently, the way things are written is crucial.
However, writing is not always perceived as a key issue There is no consensus whether writing has improved
in engineering and design. Engineers are not scientists over the years in the aeronautics domain for example.
who base their careers on the number and quality of the However, some people think that most aerospace
papers they produce. A technical document may be technical documents generated during the 1960s are
remarkably precise. They were not ambiguous. Work
generated the day before delivery just because it was
was very well done. People had time and resources to
planned to do so. Engineering culture is based on
write properly. Other people think that current
creativity and efficiency, based on very specific
languages, often in the form of drawings and engineers do not capitalized the necessary technical
schematics that cannot be understood by an outsider. background to produce appropriate and sufficiently
Engineers do not perceive the writing-for-all detailed technical documents. It is very important that a
philosophy as relevant. selected group of readers reviews all documents. If
someone does not understand a technical document then
Two separate worlds: engineering and it should be modified and improved towards a better
comprehension. We should apply to documents the
literature
same kind of usability testing and user-centered design
The distinct worlds of engineering and literature barely procedures as for systems. Human factors principles are
met during the last century. The human-computer very similar. Sometimes we say “writing is design, and
interaction (HCI) community has nicely introduced design is writing.”
design “into the picture” since user interfaces require a
subtle combination amount of technique and graphical Redefining prose rules using multimedia
art. There, science and arts met. In knowledge
This statement claims that the quality of technical
management, a deeper step is required. Designers need
documentation contributes to the quality of design. We
to step into literature. They need to write technical
usually write for potential readers. In the same way, we
documents describing requirements, specifications, job
design for potential users. Researchers know that
orders, evaluation rationale, training and performance
several persons must review papers before being
Proceedings of the European Cognitive Science Conference, Osnabrück, Germany, 11 September 2003
delivered outside. We also know that several persons Appropriate tools and organizational
must test artifacts before being delivered outside. The setups
reader of a multimedia document has become a user of
a software application. From this viewpoint, reading a In industries that develop safety-critical systems, a
physical note, report or book has evolved towards variety of knowledge management tools are available.
interacting with a computer. Writing has also evolved Also in R&D projects (including projects in the
towards design of interactive software. Writing words, European Frameworks), many KM tools have been
phrases, paragraphs and chapters has become designing developed. It is clear that tools cannot be designed and
objects and software agents. Static paper documents used without appropriate organizational setups. People
have become (inter)active documents. adapt to technology and groups, whether they are
The active part of a book (system) is the reader teams, organizations or communities. However,
(user). In addition, the organization of the book adaptation can be limited by the constraints imposed by
(system), the way phrases (objects) are written tools and socio-cultural habits of the people involved.
(designed), style and lexicon used suggest reader (user)
activity. Sometimes, the reader (user) hardly Active notifications of changes in design
understands what the author (designer) wanted to Designers of safety-critical systems are expected to be
express. Instead of mobilizing the cognition of the proactive people who manage information using
reader (user) on interaction problems, the most available tools in their organizational setups. However,
important part of the cognitive activity of the reader information technology is capable of augmenting their
(user) should be centered on the understanding and initial skills. Software agents may provide assistance in
interpretation of (active) document content. a variety of tasks that require routine, and usually
boring, actions. Safety-critical technology always
Toward simplicity incrementally changes due to accidents and incidents,
Design documents are not only outputs of design customer requirements and needs that continuously
processes but also inputs, i.e., formulating design evolve and refinement of the technology itself. There is
rationale contributes to improving the design itself. always a discrepancy between these effective changes
There are two issues of simplicity: documenting to of technology and its related operational
improve the simplicity of use of a system being documentation. People need to be notified about
developed; and reducing the difficulty of generating changes in order to operate such technology in a safe
technical documents, i.e., making it simpler. Simplest way. When such notification is timely, it is usually
systems are best used. In most cases, when systems are passive and left to the expertise or intuition of the user,
too complicated, they are not used at all. This is true it may not be noticed. This is why a system that would
both for the system being developed and for its provide proactive notification of changes would be
documentation. tremendously useful. In the WISE environment, people
can subscribe to documentation, indicating about which
Writing from bottom-up (annotations) vs. top- changes (updates, deletion, status changes etc.) they
down (requirements) want to be notified, by email or in the active work
environment.
People tend to write little notes either by using post-its,
personal notebooks, page marks, and so on. They Supporting the writing process
annotate what they do and use these notes in order to
Above we have emphasized the importance of writing
improve the capacities of their own short-term and
for the design process. Tools are available that can
long-term memories. If this kind of practice is very
support engineers in documenting their work, and
useful to people themselves, on a short term,
capture annotations during the design work, not just
interoperability becomes a problem when such
after the design is finished. An example of such a tool is
knowledge needs to be exchanged with others or reused
the Computer Integrated Documentation (CID) system
by the same person after a longer period of time.
developed at NASA (Boy, 1991). Another example can
Annotations can be considered as pragmatic knowledge
be found in the IMAT (Integrating Manuals and
that needs to be structured if it is to be used by others.
Training) system developed for designing learning
People cannot structure such knowledge in the first
material (de Hoog et al., 2002). Also in the WISE
place because it is intrinsically situated, i.e., it is
workspace the engineer is enabled to make annotations
captured in context to keep its full sense. This is why a
to all different kinds of knowledge objects and to
mechanism that would support annotation generation
choose whether to share them with team-members or
and structuring can be a powerful tool.
others.
Crisp and clearly understood design rationale is a
good indicator of maturity in design. Formalisms have
Proceedings of the European Cognitive Science Conference, Osnabrück, Germany, 11 September 2003
been developed to describe design rationale such as like the interaction blocks (Boy, 1998). A main
gIBIS (graphical Issue-Based Information System) advantage of using interaction blocks is to enable
(Conklin & Begeman, 1989) or QOC (Questions formal testing of interaction complexity, and expressing
Options Criteria) (MacLean et al., 1991). They support contexts and abnormal conditions of use explicitly.
the elicitation of design rationale and enable the A test user either follows IDs and produces an
documentation of design decisions, development plans activity by using appropriate related IOs, or interacts
and systems that are effectively developed. directly with IOs and verifies the validity of related IDs.
In both cases, he or she tests the links between IOs and
Organization of personal and team work-spaces Ids in context (i.e., in the context of the task being
In current communication and cooperation software, performed). The corresponding category is called
very efficient search engines are available; bottlenecks contextual links (CLs). This is where usefulness and
are elsewhere. They are in the way people categorize usability evaluations (sometimes annotations) are stored
incoming information with respect to what is already in the form of either free text or specific preformatted
available on their desktop. This categorization is a forms.
strong condition for further retrieval and traceability. Design rationale (DR) provides the reasons why the
People organize their workspace in order to perform
IOs and IDs of an artifact have been designed the way
their tasks efficiently and manage time and content
they are, and design alternatives that were not chosen.
accordingly. They use post-its, bookmarks, documents
DR is commonly implemented by using semi-formal
piles, proximity for urgent or frequent access, and so
on. In any case, people don’t stop to fine-tune their languages such as gIBIS or QOC already mentioned.
initial categorization to better fit their everyday needs. ADDs are tools that support not only
In the WISE project we have developed a environment communication and mediation, but also prototyping and
in which users have a personal workspace in which they evaluation. They enable their users to store design
can organize the knowledge they need for their task, as knowledge according to a concrete and systematic
well as a workspace for groups in which knowledge can formalism. Creation and maintenance of such ADDs
be pre-structured and shared. The environment consists enable an entire organization to maintain awareness of
of a portal that gives access to the companies their design processes and products.
documentations, databases and tools, including search
facilities on all knowledge objects thus available, of Interoperable documents and the Portal
whatever format or location. concept
Documents should be interoperable. This requirement
Active design documents induces two kinds of issues: standards and integrated
The concept of active design document (ADD) (Boy, environments. When people exchange documents
1997) was developed to support designers of safety- across teams, organizations and communities, they
critical systems in knowledge management. Active expect the others to be able to process what they
documentation may take various forms and involve provide. This is commonly a matter of standards. In a
different kinds of content. An ADD is defined by four closed world where an organization can cope with an
categories that organize designer’s workspace: interface integrated environment in the form of intranet for
objects, interaction descriptions, contextual links and example, people don’t have to worry about standards.
design rationale. Nevertheless, standards progressively emerge from the
Interface objects (IOs) provide appropriate, useful extensive use of specific types of documents.
and natural illusions of designed artifacts. IOs have Designers require KM environments that are user-
their own behavior reflecting the behavior of related centred (easy to use, and avoiding overload) and
artifacts. They enable users to test usefulness and integrated within their current tasks. They should have
usability of related artifacts. They provide concrete easy access to KM services at each design step. For
feeling and grasp of the use of an artifact, its learning example, in a safety assessment process, there should
requirements, its purpose hands-on, etc. Their be information provided for performing safety analysis
progressive integration leads to a series of prototypes and related documents. In other words, the designer
and, in the end, the final product. workspace should be (re)designed in such a way that he
Interaction descriptions (IDs) provide the or she has easy access to experience feedback (e.g., not
specification of user-artifact dialogue. IDs may be only a list of what is necessary to do and forbidden
expressed in either natural language, or a domain- (checklists), but providing deep knowledge to foster
specific technical language ranging from textual preventive design actions and avoid later corrective
descriptions in simplified English (operational actions) at any time. Having this knowledge available at
procedures for example) to a knowledge representation the designer’s desktop at all times can be realized by a
Proceedings of the European Cognitive Science Conference, Osnabrück, Germany, 11 September 2003
KM portal. A portal means that it provides access to References
knowledge, wherever it is located, but does not contain Bentley, R., Horstmann, T., and Trevor, J. (1997). The
this knowledge itself. In the KM portal developed in World Wide Web as enabling technology for CSCW:
WISE designers have access to knowledge available in The case of BSCW, in Computer-Supported
for example databases with experience feedback, Cooperative Work: Special issue on CSCW and the
lessons-learned and best-practices, to all kinds of Web, Vol. 6.
relevant documents, and to people who can bring Bonini, D., Jackson, A. & McDonald, N. (2001). Do I
interesting knowledge and experience. Access to all trust thee? An approach to understanding trust in the
these sources is provided in the same manner and with a domain of air traffic control. In Proceedings of IEE
single search facility. People in Control, UMIST, Manchester, UK.
Boy, G.A. (1999). Traceability. EURISCO-Airbus
report T-99-060-V3.
Conclusion and perspectives Boy, G.A., Cognitive function analysis. Ablex,
The way knowledge is exchanged during the design and Stamford, CT, 1998.
the further life-cycle of a safety-critical system induces Boy, G.A. (1997). Active design documents.
several factors related to systems (complexity, Proceedings of ACM DIS’97 Conference. ACM
completeness, maturity, traceability) and people Press, New York.
(expertise, writing, simplicity, drafts, information Boy, G.A., Indexing Hypertext Documents in Context.
credibility, uncertainty and awareness). Proceedings of the Hypertext'91 Conference, San
Several actual developments influence the design Antonio, Texas, December, 1991.
processes of safety-critical systems: more people from Conklin, E.J. & Begeman M.L. (1989). gIBIS: A tool
different organizations (within the company or for all reasons. Journal of the American Society for
Information Science, 40, pp. 200-213. This paper
(sub)contractors) get involved, more procedures are in
desribes a graphical support tool for IBIS.
place (such as certification procedures, involving
Hoog, R. de., Kabel, S., Barnard, Y., Boy, G., DeLuca,
human factors in particular), product development P., Desmoulins, C., Riemersma, J. & Verstegen, D.
needs to be faster than before. These evolutions have a (2002). Re-using technical manuals for instruction:
direct impact on the increase of both the number and creating instructional material with the tools of the
content of documents. Information technology provides IMAT project. In Y. Barnard, M. Grnadbastien, R. de
new means to generate, maintain and use such Hoog & C. Desmoulins. Ingrating Technical and
documents. A main issue is to improve the use of such Training Documentation, Proceedings of the ITS2002
means. workshop, p. 27-38. 6th International conference on
Important questions remain to be answered: does Intelligent Tutoring Systems, San Sebastian,
this technology change the job of engineers? Does it Spain/Biarritz, France.
MacLean A., Young R.M., Bellotti V.M.E. & Moran,
free up engineers from boring tasks? Or does it create
T.P. (1991). Questions, options and criteria: Elements
new ones? Answers to these questions are complex.
of design space analysis. Human-Computer
However, this paper contributes by providing categories Interaction, 6, pp. 201-250. This article describes the
of KM solutions such as the organization of personal background for designing space analysis and the
and team work-spaces, active design documents, and QOC notation. It explains how it can be used in the
knowledge portals. Usefulness and usability of such design process, and to study reasoning in design.
solutions need to be tested carefully in a real-world Other articles of interest on this topic also appear in
environment with a critical mass of people involved. this issue of the journal.
This is very difficult to do since experts and specialists Poltrock, S.E. & Grudin, J. (2001). Collaboration
(e.g., designers of safety critical systems) are always Technology in Teams, Organizations, and
occupied, busy and constrained into an already existing Communities. Tutorial. 8th IFIP TC13 IFIP
INTERACT Conference on Human-Computer
KM system, often very far from the solutions proposed.
Interaction, Tokyo, JAPAN July 9-13, 2001.
Transformations should be incremental, accepted by the
WISE: www.ist-wise.org
people involved. Implementing a new KM system is
also redefining a new philosophy of work, a new
culture. This is hard to do and hard to implement! This
is the main reason why the design of new KM systems
must be human-centred, i.e., team-centred,
organization-centred and community-centred. Each of
these types of group has its own motivations,
requirements and constraints.
Proceedings of the European Cognitive Science Conference, Osnabrück, Germany, 11 September 2003