Professional Documents
Culture Documents
Visual Informatics
journal homepage: www.elsevier.com/locate/visinf
article info a b s t r a c t
Article history: Visual programming tools are widely applied in the game industry to assist game designers in
Received 7 September 2021 developing game artificial intelligence (game AI) and gameplay. However, testing multiple game
Received in revised form 4 November 2021 engines is a time-consuming operation, which degrades development efficiency. To provide an
Accepted 4 November 2021
asynchronous platform for game designers, this paper introduces Asyncflow, an open-source visual
Available online 19 November 2021
programming solution. It consists of a flowchart maker for game logic explanation and a runtime
Keywords: framework integrating an asynchronous mechanism based on an event-driven architecture. Asyncflow
Visual programming supports multiple programming languages and can be easily embedded in various game engines to
Flowchart run flowcharts created by game designers.
Game Artificial Intelligence © 2021 The Authors. Published by Elsevier B.V. on behalf of Zhejiang University and Zhejiang University
Press Co. Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction few seconds or longer, like walking from one place to another.
Game engines usually use a main loop to drive the game logic
Game AI is the intelligence behaviors of non-player characters frame by frame and the execution of durative actions will be split
(NPCs) in the game. Most games use rule-based AI written by into each frame. Visual programming tools need to simplify this
game designers. During the development of the game industry, operation so that game designers do not need to pay attention
multiple rule-based game AI programming architectures, such to the implementation details. Behaviac provides a special action
as finite-state machine (Wagner, 2006), hierarchical-state ma- node with running status to implement durative actions. In each
chine (Girault et al., 1999) and behavior tree (Sekhavat, 2017) frame, the action node will execute a part of the overall action
have been proposed by developers to handle complex rules of and be set to the running state, then other game logic can be
game AI. Meanwhile, numerous visual programming tools have executed in the same frame. In each frame, Behaviac will check
been developed to help game designers with limited program- whether these action nodes have finished running, which is a
ming skills create rule-based game AI and gameplay (the rules polling-based method. In addition, an event-driven method can
of interaction between player and game) through graphical user also implement these action nodes, and it will be more efficient.
interfaces. But there is no visual programming tool that uses an event-driven
However, the codes generated by existing tools can only be method for now.
used in game projects developed by their respective engines. For In this paper, we propose Asyncflow,3 a flowchart-based vi-
example, Unity Visual Scripting 1 is designed only for the game en- sual programming solution, which can support interactively de-
gine Unity. Behaviac 2 is a full-platform solution that supports C++ sign and inspect execution of game AI and gameplay very well.
and C# languages, but most game servers use script languages Asyncflow integrates a universal runtime framework and various
like Lua. Therefore, a full-platform visual programming tool that nodes are compatible in the flowchart. Especially, event nodes
supports more programming languages will be helpful for game allow asynchronous logic in game AI to avoid blocking of durative
development. game actions. In summary, the contributions of this paper include
Visual programming tools are used to combine game actions the following:
into game AI. Most game actions are durative and will last a • We develop Asyncflow, an interface that supports visual
programming for development of game AI and gameplay
∗ Corresponding author at: NetEase Inc., Hangzhou, China. through the flowchart maker.
E-mail addresses: zphu@corp.netease.com (Z. Hu), • We propose a runtime framework to manage the operation
fanchangjie@corp.netease.com (C. Fan), hzzhengqiwei@corp.netease.com
of game AI, which breaks limitations of game AI codes on
(Q. Zheng), wuwei02@corp.netease.com (W. Wu), hzliubai@corp.netease.com
(B. Liu). specific engines.
1 https://unity.com/cn/products/unity-visual-scripting.
2 https://github.com/Tencent/behaviac. 3 See https://github.com/fuxi-asyncflow/asyncflow/ for details.
https://doi.org/10.1016/j.visinf.2021.11.001
2468-502X/© 2021 The Authors. Published by Elsevier B.V. on behalf of Zhejiang University and Zhejiang University Press Co. Ltd. This is an open access article under the
CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Z. Hu, C. Fan, Q. Zheng et al. Visual Informatics 5 (2021) 20–25
• We use the runtime framework based on an event-driven optimization. Users can determine whether the modification is
architecture to enable the nodes of the flowchart to be necessary by comparing the outputs before and after code modi-
executed asynchronously. fication.
Game developers also have an urgent need to perform coding
2. Related work tasks by visual programming. Several visual programming tools
create and express complex AI logic through visual graphic ele-
2.1. Flowchart application and interactive construction ments for game AI design. After classifying and layering states, the
hierarchical-state machine can clearly represent the structure of
Flowchart is extensively used to describe neural networks massive states in game AI. Besides, the behavior tree manages the
(Kahng et al., 2017) and explain event sequences (Gotz and logic modules through a tree structure. When the state is more
Stavropoulos, 2014). TensorFlow Graph Visualizer (Wongsupha- complex, the behavior tree is more conducive to maintaining
sawat et al., 2018) generates clustered graphs to summarize data the behavior of roles than the traditional finite-state machine.
flow in neural networks. The compact flow representations sup- Topiary (Shoulson et al., 2011) support the flexible generation of
port users to explore complicated model structures interactively. behavior tree through parameters specification. Unreal Engine 4
Similarly, EventThread (Guo et al., 2017) extracts event flows by provides a visual programming tool Blueprints,6 which is widely
clustering event sequences, from which evolution patterns can be used in game development projects. It can implement all the
observed. functions that need to be written in C++ before. Similar tools
To support flowchart applications, related researchers and are widely used in other industries. For example, aflak (Boussejra
developers put emphasis on interactive flowchart construction. et al., 2019) is a visual programming tool for astronomy data.
Microsoft’s visio4 provides different styles and types of nodes for
users, who can use the mouse to adjust the position of the nodes 3. The flowchart maker
flexibly to achieve a good layout effect. Open-source software
Graphviz (Ellson et al., 2001) automatically draws a flowchart In order to help game designers easily convert their ideas into
by reading a file in DOT format. The style of the nodes in the executable codes, we provide a flowchart-based flowcharts maker
flowchart needs to be defined in the DOT file. The flowchart (as illustrated in Fig. 1). The conceptual model it uses is very con-
drawn by Graphviz is automatically laid out. Normally, it can sistent with the conceptual model in game AI development; thus,
provide a good layout result, but if the drawing effect is not the it greatly reduces the difficulty of game AI development. Through
same as expected, making adjustments is very troublesome. In special event nodes and timer nodes, the asynchronous opera-
the low-code platform Qingzhou5 of NetEase, the business process tion of the process is implemented in an event-driven method,
of an enterprise can be drawn with a standardized flowchart avoiding the blocking phenomenon in the game, which is where
model based on the Business Process Model and Notation 2.0, and Asyncflow gets its name. The nodes in our flowchart are divided
enterprise resource planning (ERP) applications can be quickly into four types of nodes, namely, normal node, event node, timer
customized. In recent years, several companies have begun to node, and control node. Details of these nodes and the lines
build platforms for running constructed flowcharts. For instance, between them are as follows.
Microsoft’s Power Automate (Pearson et al., 2020) corresponds
nodes in flowcharts to operations or services. 3.1. Normal node
2.2. Visual programming Normal nodes are the most widely used nodes in Asyncflow
flowcharts (as illustrated in Fig. 2). Their content is a piece of code
Visual programming was first used in programming teach- expression, which can play two roles. Firstly, it performs several
ing. Charntaweekhun and Wangsiripitak (2006) found that most calculation operations to determine the subsequent running child
students were more willing to accept flowcharts as the car- nodes (see a in Fig. 2). Secondly, it serves as an interface to control
rier of programming instead of writing codes directly. There- the behavior of the game object (see c in Fig. 2). Game objects
fore, they implemented a system that converts flowcharts into usually refer to some particular objects or elements in the game,
executable codes. The logical judgment and sequence in the like characters, weapons and ambient objects.
codes are visually represented by the flowchart. RAPTOR (Carlisle The operations of normal nodes are independent. If the results
et al., 2004) is a programming environment specially designed in one node need to be used in other nodes, then the internal
for students. Students can use RAPTOR to implement visual pro- variables of the flowchart need to be used. The declaration (see
gramming by drawing flowcharts and class diagrams of Unified b in Fig. 2) and use (see d in Fig. 2) of these variables are limited
Modeling Language. Yuan et al. (2008) identified the flowchart to the current flowchart.
sketch on the electronic whiteboard during programming teach- The syntax of the expressions in the node is similar to Lua,
ing and finally transformed it into a standard C language program. but the flowchart maker performs strict type checking and fi-
Massachusetts Institute of Technology’s public project Scratch nally generates the function codes of the programming language
(Resnick et al., 2009) can be applied to build programs like supported by Asyncflow.
building blocks.
To benefit from functions, like structure explanation and live 3.2. Line
programming, visual programming is used in other scenarios. To
facilitate data wrangling, Wrex (Drosos et al., 2020) provided The directed lines are used to indicate the sequence of node
users with visual examples during manual programming. Re- operation as well as to clarify the operating conditions of the
sult verification can benefit from intuitive data representation. nodes. Three types of lines are distinguished by red, green and
Moreover, juxtaposing source codes and corresponding output blue colors. The normal node in the flowchart returns a Boolean
can assist code evaluation (Wood et al., 2018). Vis-a-Vis (Bolte result after running. If the result is True, then the node connected
and Bruckner, 2020) can track output evolution during coding by the green line will be run; otherwise, the node connected
4 https://www.microsoft.com/en-us/microsoft-365/visio/flowchart-software. 6 https://docs.unrealengine.com/4.27/en-US/ProgrammingAndScripting/
5 https://sf.163.com/product/lcap. Blueprints/.
21
Z. Hu, C. Fan, Q. Zheng et al. Visual Informatics 5 (2021) 20–25
Fig. 1. The interface of flowchart maker. The main area is used to draw flowcharts. The left side is used to display the node types provided by Asyncflow, and the
lower left lists the game types that can be used in the flowchart maker. At the bottom is the output bar, which is used to display the output information. The right
side is used to display and edit some data information of the node.
Fig. 2. Examples of normal nodes. (a) Game function call. (b) Declaration of
variable. (c) Behavior of the game object. (d) Use of variable.
3.3. Event node The timer node only needs to set one parameter, which is the
waiting time of the timer. Similar to the event node, when the
An event node is a special node provided by the flowchart. timer node is executed, it enters the waiting state. It reactivates
Event nodes are used to implement durative actions in the game, and continues to run subsequent nodes only when the waiting
such as a game object walking to a specified location, or behav- time has elapsed. When the timer nodes and event nodes are
iors that require specific trigger conditions, such as attacking an running, they switch to other normal nodes to continue running.
enemy when the enemy enters the field of view. A polling-based Until their running ends, Asyncflow will return and run their
solution is to check the state of the game object in every frame of subsequent nodes. This behavior is very similar to the concept
the game (see a in Fig. 3), and then execute the subsequent code of asynchronous programming. Thus, it is called an asynchronous
when the condition is met. node in Asyncflow, and it is distinguished from normal nodes by
In Asyncflow, the use of event nodes in the flowchart can a different color.
clearly achieve these requirements (see b in Fig. 3). When the
flowchart runs to an event node, the node enters the waiting 3.5. Control node
state, and then advances to run other normal nodes. Once the
specified game object generates an event of the corresponding Asyncflow also provides several nodes that change the running
type, the event node is activated and then runs down, which is flow of the flowchart, called control nodes. Control nodes can be
more efficient than many checks and judgments in each frame.
further divided into:
The event type in Asyncflow can be freely defined by the user,
and each event includes a name and an arbitrary number of • StopNode can be used to cancel an asynchronous node in a
parameters. These parameters can be used to transfer some state waiting state.
22
Z. Hu, C. Fan, Q. Zheng et al. Visual Informatics 5 (2021) 20–25
• WaitAll node continues to run after all its parent nodes are
running.
• WaitAny node continues to run if at least one parent node
is running; it only runs once if multiple parent nodes are
running.
These nodes make the flow of the flowchart more flexible and
allow for easier implementation of complex game logic.
3.6. Layout
Table 1
Usage comparison among four visual programming tools.
Tools↓\Performance → Supported engines Supported languages Asynchronous node
Asyncflow Cross-platform Lua, C#, Python ✓
Blueprints Unreal C++ ✓
Behaviac Cross-platform C++, C# ✓
Behavior Designer Unity C# ✗
5. Experiments
Carlisle, M.C., Wilson, T.A., Humphries, J.W., Hadfield, S.M., 2004. Raptor: intro-
ducing programming to non-majors with flowcharts. J. Comput. Sci. Coll. 19
(4), 52–60.
Charntaweekhun, K., Wangsiripitak, S., 2006. Visual programming using
flowchart. In: 2006 International Symposium on Communications and
Information Technologies. IEEE, pp. 1062–1065.
Drosos, I., Barik, T., Guo, P.J., DeLine, R., Gulwani, S., 2020. Wrex: A unified
programming-by-example interaction for synthesizing readable code for data
scientists. In: Proceedings of the 2020 CHI Conference on Human Factors in
Computing Systems. pp. 1–12.
Ellson, J., Gansner, E., Koutsofios, L., North, S.C., Woodhull, G., 2001. Graphviz—
open source graph drawing tools. In: International Symposium on Graph
Drawing. Springer, pp. 483–484.
Girault, A., Lee, B., Lee, E.A., 1999. Hierarchical finite state machines with
multiple concurrency models. IEEE Trans. Comput.-Aided Des. Integr. Circuits
Syst. 18 (6), 742–760.
Fig. 7. Performance comparison among three tools. Gotz, D., Stavropoulos, H., 2014. Decisionflow: Visual analytics for high-
dimensional temporal event sequence data. IEEE Trans. Vis. Comput. Graphics
20 (12), 1783–1792.
Guo, S., Xu, K., Zhao, R., Gotz, D., Zha, H., Cao, N., 2017. Eventthread: Visual
CRediT authorship contribution statement summarization and stage analysis of event sequence data. IEEE Trans. Vis.
Comput. Graphics 24 (1), 56–65.
Zhipeng Hu: Conceptualization, Methodology, Writing – orig- Kahng, M., Andrews, P.Y., Kalro, A., Chau, D.H., 2017. Activis: Visual exploration
of industry-scale deep neural network models. IEEE Trans. Vis. Comput.
inal draft. Changjie Fan: Supervision, Writing – reviewing and
Graphics 24 (1), 88–97.
editing. Qiwei Zheng: Investigation, Software. Wei Wu: Visual- Pearson, M., Knight, B., Knight, D., Quintana, M., 2020. Introduction to power
ization, Validation. Bai Liu: Resources, Formal analysis. automate. In: Pro Microsoft Power Platform. Springer, pp. 73–78.
Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Bren-
Declaration of competing interest nan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., et al., 2009.
Scratch: programming for all. Commun. ACM 52 (11), 60–67.
Sekhavat, Y.A., 2017. Behavior trees for computer games. Int. J. Artif. Intell. Tools
The authors declare that they have no known competing finan-
26 (02), 1730001.
cial interests or personal relationships that could have appeared Shoulson, A., Garcia, F.M., Jones, M., Mead, R., Badler, N.I., 2011. Parameterizing
to influence the work reported in this paper. behavior trees. In: International Conference on Motion in Games. Springer,
pp. 144–155.
Ethical approval Sugiyama, K., Tagawa, S., Toda, M., 1981. Methods for visual understanding of
hierarchical system structures. IEEE Trans. Syst. Man Cybern. 11 (2), 109–125.
The study does not involve human subjects. All data used in Wagner, F., 2006. Modeling Software with Finite State Machines: A Practical
the study are taken from public databases that were published in Approach. Auerbach Publications.
the past Wongsuphasawat, K., Smilkov, D., Wexler, J., Wilson, J., Mane, D., Fritz, D.,
Krishnan, D., Viégas, F.B., Wattenberg, M., 2018. Visualizing dataflow graphs
of deep learning models in tensorflow. IEEE Trans. Vis. Comput. Graphics 24
References (1), 1–12.
Wood, J., Kachkaev, A., Dykes, J., 2018. Design exposition with literate
Bolte, F., Bruckner, S., 2020. Vis-a-vis: visual exploration of visualization source visualization. IEEE Trans. Vis. Comput. Graphics 25 (1), 759–768.
code evolution. IEEE Trans. Vis. Comput. Graphics. Yuan, Z., Pan, H., Zhang, L., 2008. A novel pen-based flowchart recognition system
Boussejra, M.O., Uchiki, R., Takeshima, Y., Matsubayashi, K., Takekawa, S.,
for programming teaching. In: Workshop on Blended Learning. Springer, pp.
Uemura, M., Fujishiro, I., 2019. aflak: Visual programming environment en-
55–64.
abling end-to-end provenance management for the analysis of astronomical
datasets. Vis. Inform. 3 (1), 1–8.
25