JURNAL 2

communication, expression and flexibility are relationships of cognitive subsystems

that get information from the surrounding environment as well as provide input. while
additional information on extra-linguistic d makes an instrumental contribution to
overcoming difficulties in knowledge for example in the area of ambiguity or reference
resolution, which is said to be linguistically that will encourage listeners to receive
information from the relevant area. there will be closed feedback and can be productive if it is
relevant then there will be a visual effect on hearing. language is not only what is spoken but
must first think about its linguistic reference, then the expression to the entities in the world
and the relationship between each other. therefore from a technical perspective, such as using
sign language which will process natural language (Bloch, 2008).

2. SPEAKER INTENTION

language and vision will produce the best resolution. task intent detection, reference
resolution, cross modal information fusion and prediction are closely related to each other
rather than being handled separately by the system. While Gorniak and Roy (2005) system is
limited to order execution, the task becomes much more difficult in terms of collaborative
problem solving, where dynamic efforts from both parties are required. Collaborative
problem solving usually occurs in structurally rich visuals environment such as the one in
Figure 1, which contains several windows, cupboards, boxes, pills, magazines, bottles etc,
some of they are even (partially) closed from the viewer's point of view. Objects in such an
environment can be called enough different, sometimes less specific, and they are simply
numbers naturally create a much larger space for reference resolution.

3. RESOLUTION OF LINGUISTIC AMBIGUITIES

ambiguity is the most difficult language understanding because it is attached to both lexical
items and complex structures. Therefore, the development of the algorithm approach to
disambiguation is always the main concern when designing natural language comprehension
systems. This systems usually rely on combinatorial decision procedures combined with a
strong scoring mechanism as the basis for preferential reasons. However, in a limited domain,
number of linguistic expressions with some complete different meaning is quite low. Part-of-
speech ambiguity of words like open (adjective vs. verb) or book (noun vs. verb) is more
relevant in such a scenario. They may creates false interpretations in the process of
understanding and thereby expands the space of possible intermediate hypotheses. then it will
be the same as the problem processing is created by completely structural ambiguity as is the
case with the attachment of the famous prepositional phrase (“… close the box on the
table.”). The fourth kind of ambiguity appears from an internal-language reference, which
can be assigned to example through different pronouns or definite noun phrases (“...but
broken.”). For all types of ambiguous constructs, crossmodal evidence can help resolve
ambiguity by re-rating
interpretations that may or may even exclude some of them according to their possibilities in
the visual world. Especially when linguistic cues alone are not sufficient to determine the
actual intended interpretation, for example because they contradict both the frequency of use
and human preferences, cross-modal interaction will be indispensable to achieve effective
and timely disambiguation.

4. CROSS-CAPITAL REFERENCE RESOLUTION

expressions can be associated with entities in environment that may be confused with other
objects of a similar appearance. In addition, the visual object sometimes is partially closed
from the listener's point of view, or otherwise difficult to understand. Once again, the
resulting combinatorics can be well controlled if possible mapping space can be limited as
much as possible. It creates a very strong incentives for additional (and predictive) processing
modes, which facilitates the interaction between the two modalities as early as possible to
exchange disambiguation information in both directions. Even under ideal conditions, there is
no precise mapping between linguistically and visually contributed concepts perceived
information. The two modalities differ in the ability to accommodate various aspects and
details of conceptualization. Language, for example, is perfect for describes entities with a
very fine inventory of categories.

6. CROSSMODAL INTERACTION OF LANGUAGE AND VISION

Cross-modal integration can occur at different levels, from multi-sensory fusion (e.g., audio-
visual, audio-tactile or visio-tactile) to higher-level comprehension processes such as
language understand. Usually biased processing with one modality dominate others, for
example, evolutionarily acquired dominance of visual modality over one hearing for sound
source localization. However, this domination could be neutralized or even reversed when the
dominant modality is not reliable (Witten and Knudsen, 2005; King, 2009). While perceptual
phenomena have a real influence, integration of linguistic and visual information mostly
occurs at the conceptual level. The syntax-first approach assumes the privileged role of a
modularly applied grammar without external influence from other sources of information. In
the cross modal conflict case, the syntax-first approach assumes that only structures are
licensed by the selected grammar (For example, Frazier and Clifton, 2001, 2006). As a result,
this theory predicts a strictly temporal sequence of processing steps with syntactic constraints
are applied first and others later.

7. INCREMENTALITY
Incremental processing is particularly valuable for early reference resolution. Reliable
hypotheses about suitable candidates in the visual environment reduce the space of possible
alternative interpretations, which may save computational effort. Moreover, a closer
inspection of the candidate and its visual surrounding may contribute additional information
that supports the ongoing comprehension process,
From a technical point of view, incremental (online) processing can be distinguished from
batch mode (offline)processing. While a system in batch mode waits for the whole input
being available before attempting to analyze it, an incremental analysis integrates partial
input into a (coherent) processing result as soon as it becomes available. this type of
processing is not able to explain the dynamics of human language comprehension. One
problem of processing sentences incrementally is caused by possible dependencies between
incrementality and crossmodality, which have an impact on the system performance that is
hard to assess.

8. PREDICTION
Knoeferle et al. (2005) demonstrated the predictive nature of sentence comprehension
using German sentences with Subject-Object ambiguities that directly map to an ambiguous
AGENT/PATIENT assignment. Each visual scene to which a sentence refers depicts two
actions and three characters. Two different sentence patterns have been compared either in an
unmarked word order (Subject-Verb-Object) or in a marked word order (Object-Verb-
Subject).Even the objects that have no semantical relation to the spoken words are able to
attract attention to themselves. To study to which degree the aforementioned findings
(Knoeferle et al., 2005) are influenced by the visual complexity of the visual scene, Alaçam
et al. (2019) followed the same experimental paradigm and used the same sentence patterns
(unmarked and marked word orders).

9. HEURISTIC DECISION TAKING

Such kind of heuristic decision taking plays a crucial role in human cognition as
comprehensively discussed in Gigerenzer (2000) and Gigerenzer (2008). In complex enough
tasks, reaching a decision by considering all possible options is unrealistic due to temporal or
memory-related resource limitations. Thus, there must be a cognitive mechanism that is able
to abandon processing and to settle the issue as soon as a sufficiently high degree of
confidence has been reached.
10. CONCLUSIONS
It is commonplace that language comprehension takes advantage of the availability of
crossmodal information. Indeed, recent psycholinguistic research as well as the development
of computational language comprehension systems have confirmed this assumption and
contributed a number of valuable insights into how this added benefit comes about and what
its prerequisites are:
• Language comprehension benefits from being sensitive to extra-linguistic information about
the kind of entities in the surrounding world, their spatial relationships, the events they take
part in, and the general or episodic affordances they offer
• The interplay between linguistic and visual processing components seems to be based on
interaction rather than fusion.
• Language processing can profit most from the available visual information if it proceeds in
an incremental and predictive manner.
1) STRENGTH AND WEAKNESS JOURNAL 2 :
A. Strength :
1. The title used is appropriate and relates to the entire content of the research.
2. The abstract and introduction are good enough.
3. The material in this journal is also quite good with many expert opinions.
4.This journal also has a conclusion, and has a fairly clear reference

B. Weakness

Writing space regularly does not make it difficult for readers to know and does not
completely record the formulas written in the journal.