FLOW
ACADEMIC RESEARCH POSTER TEMPLATE
Filtering and LSTM-based Optimization for Web Browser
Subtitle for Academic Research Poster (36x48 inches) Interactions
Dongyijie Pan, Zhengnan Li, Guangzheng Fei From Communication University of China
Your names and the names of the people who contributed to this presentation
Long Ling from Tongji University
Introduction
Web browsers have become an indispensable tool for
human-computer interactions across various platforms.
However, interactive performance (e.g the immediacy of
interaction, the accuracy of interaction data) in browser-
based apps can be hindered by hardware limitations and
network delays, resulting in unwanted jitter and input signal
delays that negatively impact the user experience. To tackle
these challenges, we propose a novel approach that
leverages effective filtering algorithms and lightweight
machine learning models. By integrating the 1 euro filter and
Long Short-Term Memory (LSTM) algorithm, we successfully
optimize the accuracy of Mediapipe-based gesture
interactions and tangible table application within web
browsers. Our findings demonstrate that this combined
approach not only enhances input signal accuracy but also
ensures seamless and highly responsive web browser
interactions.
The Mediapipe hand landmark Fast movements result in pronounced
detection in the browser faces on-screen smearing in TUIO based
inaccuracy in many scenarios tangible interaction Web App. The
rendered position could not catch the
token at high speed.
Methods
We innovatively combined the 1 Euro filter with the LSTM
algorithm, and with the aid of Tensorflow.js, we implemented
a lightweight filtering plus RNN algorithm on the browser side.
This serves to optimize a variety of issues encountered in
browser-based interactions.
Compared to traditional Kalman filters, the 1 Euro filter is
more user-friendly for HCI researchers in terms of parameter
tuning. It only has two adjustable parameters and can
smooth interaction data in real-time with O(1) time
complexity during interactions. This makes it a small yet
elegant filtering algorithm in the field of Human-Computer
Interaction.
Although the 1 Euro filter significantly optimizes tailing, jitter, and
latency in interactions, this pre-machine learning era algorithm
can be substantially empowered today by incorporating some
neural network-based temporal trajectory prediction algorithms.
In temporal trajectory prediction, LSTM can be employed to
implement a ’Seq2Seq’ approach, where we use past trajectory
data to predict future trajectory data. We can certainly build
an ’N to1’ prediction model, which predicts the next coordinate
position based on the previous N steps, but such a sliding
window prediction method is too time-consuming under low
computational power. We recommend web developers to turn
the 1 euro filter for reducing significant jitter, and then perform
latency statistics on the filtered data. For example, under a fixed
sampling rate, we observe how many frames the filtered data
lags behind the ground truth. We found that using the maximum
lag value multiplied by 1.5 or 2 as input to the LSTM yields
significantly improved prediction accuracy.
For the challenge of determining an uncertain frame delay, we
innovatively use the alpha parameter in the filter to adaptively
select the predictive frames generated by the LSTM. By
applying a linear mapping on the prediction sequence using the
alpha parameter, this approach effectively addresses some
overfitting issues that arise when using a fixed frame as the
predictive frame.
Paradigm for Data Preprocessing and Training in FLOW
Application-Level Processing Paradigm of FLOW
Dongyijie Pan Zhengnan Li
From CUC From CUC
Email: Email:
primojaypan@cuc.edu.cn lzhengnan389@gmail.com
Experiments and Results
We conducted two experiments, focusing on Mediapipe
Hands across various devices. The standard for
comparison was set using the GPU-accelerated Python
version of Mediapipe. We compared the errors by
calculating the Mean Squared Error (MSE) of each key
node, post-'FLOW' algorithm processing, against the
standard answer. It was observed that after optimization
with the FLOW algorithm, the accuracy and performance
of the browser-based Mediapipe Hands improved
significantly compared to before the optimization.
For the application of interactive table tokens in
browsers, we divided the movement of the tokens into
three steps, based on the characteristics closely related
to errors occurring in physical interactions, such as
sudden acceleration and deceleration.
According to the theory of Jota et al. , we categorized the
collected data into three stages according to the
acceleration of the token in each frame. 1:initial reaction,
2:large ballistic motion, 3:feedback adjusted final
adjustments. Based on these three criteria, we have
organized the data into the following table: original data,
data after 1 Euro filter processing, data after prediction
using LSTM + mean latency frame, and data after 𝛼-
mapped frame and subsequent filtering
Conclusion
FLOW provides a feasible and experimentally implementable paradigm to
address jitter and latency in complex human-computer interaction
applications on the browser platform. The 1 euro filter serves as a
preprocessor before using neural networks, and it dynamically selects
predicted values and further smooths the output results after the neural
network's predictions. Moreover, our optimized and integrated algorithm
maintains a lightweight profile in terms of both time and space complexity.
In application scenarios where prompt human-computer interaction
feedback is crucial, our algorithm effectively addresses the two issues
encountered in the experiment. We believe it holds the potential to be
applied in various interactive applications on the browser platform.
In the coming years, we would have the opportunity to incorporate more
complex yet powerful machine learning models into browsers.At the same
time, the HCI researchers and their achievements, like $1 Unistroke
Recognizer, before the surge of machine learning algorithms also remind
us that apart from feeding interaction data into black-box-like hidden layers,
we humans still possess countless ingenious ideas to solve seemingly
challenging problems using concise code and remarkably low time
complexity. Together, we will expand the possibilities of the browser, the
most fundamental medium for communication between humans and
computers, and humans and the internet. The potential for the future is
immense.
Long Ling Guangzheng Fei
From:TJU From:CUC
Email: Email:
lucyling0224@gmail.com gzfei@cuc.edu.cn