Learning Physics-Consistent Particle Interactions

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.
1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

Learning physics-consistent particle
interactions
Zhichao Hana , David S. Kammera , Olga Finkb *
PT
a
Institute for Building Materials, ETH Zürich, 8093 Zürich,
RI
Switzerland
SC
b
Laboratory of Intelligent Maintenance and Operations
Systems, EPFL, 1015 Lausanne, Switzerland
U
N
November 17, 2022
A
M
Abstract
ED
Interacting particle systems play a key role in science and engineer-

ing. Access to the governing particle interaction law is fundamental
IT
for a complete understanding of such systems. However, the inher-

ent system complexity keeps the particle interaction hidden in many
ED
cases. Machine learning methods have the potential to learn the be-
havior of interacting particle systems by combining experiments with
data analysis methods. However, most existing algorithms focus on
N
learning the kinetics at the particle level. Learning pairwise interac-

tion, e.g., pairwise force or pairwise potential energy, remains an open
U
challenge. Here, we propose an algorithm that adapts the Graph Net-

works framework, which contains an edge part to learn the pairwise
L
interaction and a node part to model the dynamics at particle level.

A
Different from existing approaches that use neural networks in both

parts, we design a deterministic operator in the node part that allows
IN
to precisely infer the pairwise interactions that are consistent with

* To
G
whom correspondence should be addressed: Email: olga.fink@epfl.ch

RI
1
O
© The Author(s) 2022. Published by Oxford University Press on behalf of National Academy of Sciences. This is an Open
Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits
unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
underlying physical laws by only being trained to predict the particle
Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

acceleration. We test the proposed methodology on multiple datasets
and demonstrate that it achieves superior performance in inferring
correctly the pairwise interactions while also being consistent with
the underlying physics on all the datasets. While the previously pro-
posed approaches are able to be applied as simulators, they fail to infer
physically consistent particle interactions that satisfy Newton’s laws.
Moreover, the proposed PIG’N’PI also outperforms the other base-
PT
line models in terms of generalization ability to larger systems and
robustness to significant levels of noise. The developed methodology
can support a better understanding and discovery of the underlying
RI
particle interaction laws, and hence guide the design of materials with
targeted properties.
SC
Key words Interacting particle systems, Pairwise interaction, Graph
neural networks, Deterministic physics operator, Physics consistency.
U
N
Significance Statement
Understanding, modeling and predicting the kinetic behaviour of
A
interacting particle systems rely on knowing the governing interac-
M
tion laws between individual particles. However, in real applications,
the ground truth information on pairwise interactions remains un-
known. Here, we propose a Graph Neural Network framework that
ED
incorporates universal physical laws to infer pairwise force (or poten-

tial energy) for any particle system. The proposed method precisely
infers pairwise particle interactions that are consistent with underlying
IT
physical laws without any supervision by only being trained to predict

particle acceleration. The proposed methodology is a step forward in
ED
developing flexible and robust tools for the discovery of physical laws,
which will be the basis for various applications such as designing new
N
materials.
U
Interacting particle systems play a key role in nature and engineering as

they govern planetary motion [1], mass movement processes [2] such as land-
L
slides and debris flow, bulk material packaging [3], magnetic particle trans-
A
port for biomaterials [4], and many more. Since the macroscopic behavior of
such particle systems is the result of interactions between individual parti-
IN
cles, knowing the governing interaction law is required to better understand,

model and predict the kinetic behaviour of these systems. Particle interac-
G
tions are determined by a combination of various factors including contact,

RI
2
O
friction, electrostatic charge, gravity, and chemical interaction, which affect

the particles at various scales. The inherent complexity of particle systems in-
hibits the study of the underlying interaction law. Hence, they remain largely
unknown and particle systems are mostly studied in a stochastic framework
or with simulations based on simplistic laws.
Recent efforts on developing machine learning (ML) methods for the dis-
covery of particle interaction laws have shown great potential in overcoming
these challenges [5, 6, 7, 8, 9, 10, 11]. These ML methods, such as the Graph
PT
Network-based Simulators (GNS) [12] for simulating physical processes, Dy-
namics Extraction From cryo-em Map (DEFMap) [13] for learning atomic
RI
fluctuations in proteins, the SchNet [14, 15] which can learn the molecu-
lar energy and the neural relational inference model (NRI) [16] developed
SC
for inferring heterogeneous interactions, can be applied on various types of
interacting particle systems such as water particles, sand and plastically de-
formable particles. They allow implicit and explicit learning of the mechani-
U
cal behavior of particle systems without prior assumptions and simplifications
N
of the underlying mechanisms. A commonly applied approach is to predict
directly the kinetics of the particles without explicitly modeling the interac-
A
tions [14, 17, 18, 19, 20, 12, 21]. The neural networks, then, map directly the
M
input states to the particle acceleration, occasionally by virtue of macroscopic
potential energy [12, 14, 17, 18, 19]. While these approaches give an accurate
prediction of the particle system as it evolves, they do neither provide any
ED
knowledge about the fundamental laws governing the particle interactions

nor are they able to extract the particle interactions precisely.
IT
Recent work [22] proposed an explicit model for the topology of particle
systems, which imposes a strong inductive bias and, hence, provides access
ED
to the individual pairwise particle interactions. [22] demonstrated that their

Graph Network (GN) framework predicts well the kinetics of the particles
system. However, as we will show, the inferred particle interactions violate
N
Newtonian laws of motion, such as the action-reaction property, which states

U
that two particles exert the same but opposed force onto each other. There-
fore, the extracted pairwise particle interactions do not correspond to the
L
real underlying particle interaction force or potential, which are the funda-
A
mental properties of a physical system. The origin of these discrepancies

lies in the design of the GN approach, which does not sufficiently constrain
IN
the output space, and clearly demonstrates the need for a physics-consistent
Graph Neural Network framework for particle interactions.
G
Besides interacting particle systems, recent works applied Graph Network

RI
3
O
models to learn micro-level dynamics in other physical systems. For exam-

ple, the work of Moon et al. [23] integrated domain knowledge into the graph
neural network to learn drug–target binding affinities in a supervised way.
Ha and Jeong [24] trained the Graph Network model as a simulator for inter-
acting discrete physical systems other than interacting particles, including
cellular automata, the Vicsek model, the active Ornstein–Uhlenbeck particle
model and the movement of a flock of birds. Another follow-up research line
aimed to improve graph neural networks to learn particle dynamics in more
PT
complicated scenarios. For example, the work of Huang et al. [25] learns par-
ticle dynamics in interacting particle systems with geometrical constraints
RI
where some particles are connected by bounds. However, to the best of
our knowledge, no other previous work aimed to reveal physics-consistent
SC
particle interactions from observed particles trajectories without any direct
supervision.
Here, we propose a Graph Neural Network (GNN) framework that in-
U
corporates universal physical laws, specifically Newton’s second law of mo-
N
tion, to learn the interaction potential and force of any physical particle
system. The proposed algorithm, termed physics-induced graph network for
A
particle interaction (PIG’N’PI), combines the graph neural network method-
M
ology with deterministic physics operators to guarantee physics consistency
when indirectly inferring the particle interaction forces and potential energy
(Fig. 1). We will show that PIG’N’PI learns the pairwise particle potential
ED
and force by only being trained to predict the particle acceleration (without
providing any supervision on the particle interactions). Moreover, we will
IT
show that the inferred interactions by PIG’N’PI are physically consistent

(contrary to those inferred by purely data-driven approaches). We will fur-
ED
ther demonstrate that predictions provided by PIG’N’PI are more accurate,

generalize better to larger systems and are more robust to noise than those
provided by purely data-driven graph network approaches. Moreover, we
N
will demonstrate on a case study that is close to real applications that the
U
proposed algorithm is scalable to large systems and is applicable to any type

of particle interaction laws.
L
A
IN
G
RI
4
O
Time: * − 1 Time: * Time: * + 1
Ⓐ (r t
<latexit sha1_base64="I/GJvSnWJKeo458fs6hoLihOauA=">AAACSXicdVDdTtswGHXKz6AbUMblbiyqSZ0EUVKqtrlD7IZLJq20UpJVjuOCVdvJ7C+bqiivsKfZ7fYCe4I9BnfTruaUggCxI1k6Puf7/Pk7SS64Ac/77TTW1jc2X2xtN1++2tnda+2/vjRZoSkb0UxkepIQwwRXbAQcBJvkmhGZCDZO5u9rf/yFacMz9REWOYsluVJ8xikBK01bnU6UyFJXn0o49qsjXN+iNAMr3Wufj7B8N221PTcY9oeDLvbcXm8QDAJLgiA46XvYd70l2miFi+m+s20fooVkCqggxoS+l0NcEg2cClY1o8KwnNA5uWKhpYpIZuJyuVKF31olxbNM26MAL9WHHSWRxixkYislgWvz1KvF57ywgNkwLrnKC2CK3g6aFQJDhut8cMo1oyAWlhCquf0rptdEEwo2xUdTEml3UOwrzaQkKi0jkpgq9OMyEjZxwG0fR7pmVdOGd5cQ/j+57Lp+3z350Gufnq1i3EJv0CHqIB8N0Ck6RxdohCj6hr6jH+in88u5cf44f29LG86q5wA9QmPtHyQcsR0=</latexit>
1
, ṙ t 1
, q, m)
<latexit sha1_base64="yEptK72DwnOT2TTPOt3WuuK02MU=">AAACRXicdVDLTttAFB1DeYVXoMtuRg1IICHLDibgHaIblqnUAFJsovFkQkaZGZuZa1Bk+Qf4mm7bH+Ab+hHdoW7bcZJKgOBIIx2dcx9zT5IJbsDzfjlz8x8WFpeWV2qra+sbm/Wt7QuT5pqyDk1Fqq8SYpjginWAg2BXmWZEJoJdJqMvlX95x7ThqfoG44zFktwoPuCUgJV69Z29KJGFLq8LKA9wxaN+ClaYKbcHWO736g3PDU/CVusIe24QHgfNwJIwDAMvwL7rTdBAM7R7W86KHUNzyRRQQYzp+l4GcUE0cCpYWYtywzJCR+SGdS1VRDITF5NzSrxrlT4epNo+BXiiPu8oiDRmLBNbKQkMzWuvEt/yujkMTuKCqywHpuh00SAXGFJcZYP7XDMKYmwJoZrbv2I6JJpQsAm+2JJIe4Ni9zSVkqh+EZHElF0/LiJh0wbc8HGkK1bWbHj/E8Lvk4um67fcw69B4/RsFuMy+oQ+oz3ko2N0is5RG3UQRQ/oO/qBfjqPzm/nyfkzLZ1zZj0f0Qs4f/8BD3+xMA==</latexit>
(r , ṙ , q, m) t t
(r t+1 , ṙ t+1 , q, m)
<latexit sha1_base64="vgl/KTAQYBom+0MvE8WtbXeqQv8=">AAACSXicdVDRTtswFHUKjFLGaLfHvVhEkzqBogSylrxV42WPTFpbpCZUjuuChe1k9s2mKsov7Gv2Cj+wL+Az9jbtaU7pJkDsSJaOz7nX1/ekueAGfP/WaaytbzzbbG61tp/vvNhtd16OTFZoyoY0E5k+S4lhgis2BA6CneWaEZkKNk6vTmp//IVpwzP1CRY5SyS5UHzOKQErTdvdbpzKUlfnJewH1QGub/EsAyv90z4fYPl22nZ9LzqOer132PfCqB8ehpZEURT6IQ48fwkXrXA67Thb9iFaSKaACmLMJPBzSEqigVPBqlZcGJYTekUu2MRSRSQzSblcqcJvrDLD80zbowAv1fsdJZHGLGRqKyWBS/PYq8WnvEkB8+Ok5CovgCl6N2heCAwZrvPBM64ZBbGwhFDN7V8xvSSaULApPpiSSruDYl9pJiVRszImqakmQVLGwiYO2A1wrGtWtWx4fxPC/yejQy/oeUcfQ3fwfhVjE71Ge6iLAtRHA/QBnaIhougb+o6u0Y3zw/np/HJ+35U2nFXPK/QAjbU/D8mxEQ==</latexit>
(r t 1
, ṙ t 1
...
<latexit sha1_base64="DUpY/UN0Ni4bhxPbEUmS/G8upKo=">AAACSXicdVDLTuMwFHXKMECZGQos2VhTIXUkiGIepewQbFiCRAGpCZXjumBhO8G+AVVRfoGvYQs/wBfwGewQK5xSEKDhSJaOz7nX1/fEqRQWguDBq4z9GP85MTlVnf71+89MbXbu0CaZYbzNEpmY45haLoXmbRAg+XFqOFWx5Efx+U7pH11yY0WiD2CQ8kjRUy36glFwUrfWaISxyk1xksMyKZZweQt7CTjpXbtYwupft1YP/M1NskpaOPBbQdBsbjhCSGttvYmJHwxRRyPsdWe9KfcQyxTXwCS1tkOCFKKcGhBM8qIaZpanlJ3TU95xVFPFbZQPVyrwolN6uJ8YdzTgofqxI6fK2oGKXaWicGa/eqX4P6+TQb8V5UKnGXDNXgf1M4khwWU+uCcMZyAHjlBmhPsrZmfUUAYuxU9TYuV20PyKJUpR3ctDGtuiQ6I8lC5xwHWCQ1OyourCe0sIf08OV3zS9Ff31+pb26MYJ9EC+osaiKANtIV20R5qI4au0Q26RXfevffoPXnPr6UVb9Qzjz6hMvYC7Pqw/A==</latexit>
... , q, m)
(r t+1 , ṙ t+1 , q, m)
<latexit sha1_base64="XyJ2KweeZyKYxQA+smtfmpHb5BM=">AAACSXicdVDLThsxFPWEdygQ2iUbi6hSKtDIA0kZdgg2XVKpAaTMEHkch1jYnql9p1U0ml/o13QLP8AX8BnsKlZ4QkCA6JEsHZ9zr6/vSTIpLBBy69VmZufmFxaX6ssfVlbXGusfT2yaG8a7LJWpOUuo5VJo3gUBkp9lhlOVSH6aXB5V/ukvbqxI9Q8YZzxW9EKLoWAUnNRvtFpRogpTnhewFZTbuLpFgxSc9Kz93MbqS7/RJP5+uN/ZI5j47ZCEYduRYKcTkg4OfDJBE01x3F/3ltxDLFdcA5PU2l5AMogLakAwyct6lFueUXZJL3jPUU0Vt3ExWanEn50ywMPUuKMBT9SXHQVV1o5V4ioVhZF961Xie14vh2EYF0JnOXDNHgcNc4khxVU+eCAMZyDHjlBmhPsrZiNqKAOX4qspiXI7aP6bpUpRPSgimtiyF8RFJF3igJsBjkzFyroL7ykh/H9ysuMHX/3d7+3mweE0xkW0gTZRCwVoDx2gb+gYdRFDf9BfdIWuvRvvzvvn3T+W1rxpzyf0CrWZB/OPsQA=</latexit>
(r t , ṙ t , q, m)
<latexit sha1_base64="FRmSXHmIN6vlz7U1zdHF9KTJpLI=">AAACRXicdVDdSiMxGM34s2rVta6X3oStgoIME1u13oneeKmwVaEzlkya2mCSmU2+2aUM8wI+jbf6Aj6DD+GdeKuZWkFFDwQO53w/+U6cSmEhCO69sfGJyR9T0zOV2bn5nwvVxV8nNskM4y2WyMScxdRyKTRvgQDJz1LDqYolP40vD0r/9B83ViT6DwxSHil6oUVPMApO6lRX1sJY5aY4z6HYwCUPuwk4YaT83cBqvVOtBf5uc7e5RXDgb9UJ2Q4cIY1Gc6eOiR8MUUMjHHUWvRk3hmWKa2CSWtsmQQpRTg0IJnlRCTPLU8ou6QVvO6qp4jbKh+cUeNUpXdxLjHsa8FB935FTZe1Axa5SUejbz14pfuW1M+g1o1zoNAOu2euiXiYxJLjMBneF4QzkwBHKjHB/xaxPDWXgEvywJVbuBs3/s0Qpqrt5SGNbtEmUh9KlDbhGcGhKVlRceG8J4e/JyaZPtv36caO2tz+KcRoto99oDRG0g/bQITpCLcTQFbpGN+jWu/MevEfv6bV0zBv1LKEP8J5fAOXVsRc=</latexit>
(r t , ṙ t , q, m)
<latexit sha1_base64="vKPzow+/wzb5CDdCNTnggmZ/GLM=">AAACRXicdVDdTtswGHVgG6WwUeCSG2sFiUkoSmhJyx2CGy6ZtAJSEyrHdalV2wn2F1AV5QV4Gm63F+AZeAju0G43p+0kisaRLB2d8/34O3EquAHPe3IWFj98/LRUWa6urH7+slZb3zg3SaYp69BEJPoyJoYJrlgHOAh2mWpGZCzYRTw6Kf2LW6YNT9QPGKcskuRa8QGnBKzUq23vhrHMdXGVQ7GHSx72E7DCTLnZw/Jbr1b33MNDv+21see2g4PWQdMSvxE0ggD7rjdBHc1w1lt3lu0YmkmmgApiTNf3UohyooFTwYpqmBmWEjoi16xrqSKSmSifnFPgHav08SDR9inAE/V1R06kMWMZ20pJYGjeeqX4P6+bwaAd5VylGTBFp4sGmcCQ4DIb3OeaURBjSwjV3P4V0yHRhIJNcG5LLO0Nit3RREqi+nlIYlN0/SgPhU0bcN3HoS5ZUbXh/UsIv0/O910/cBvfm/Wj41mMFbSFvqJd5KMWOkKn6Ax1EEX36AH9RL+cR+fZeXF+T0sXnFnPJpqD8+cv+aqxIw==</latexit>
(r t 1
, ṙ t 1
<latexit sha1_base64="3zuM/ZmkG8GLMciYyu6LIURzVsg=">AAACSXicdVBBThsxFPWEtkCgJcCSjdUIKUh05IEEhh2iG5YgEUDKDJHHccDC9gz2H1A0mitwGrbtBThBj8EOscITAipVeZKl5/f+9/d/SSaFBUL+eLWpT5+/TM/M1ufmv35baCwuHds0N4x3WSpTc5pQy6XQvAsCJD/NDKcqkfwkufxZ+SfX3FiR6iMYZTxW9FyLoWAUnNRvtFpRogpTnhXwIyjXcXWLBik46U27Wsdqrd9oEn8n3OlsE0z8dkjCsO1IsNEJSQcHPhmjiSY46C96s+4hliuugUlqbS8gGcQFNSCY5GU9yi3PKLuk57znqKaK27gYr1TiVacM8DA17mjAY/XvjoIqa0cqcZWKwoX916vE/3m9HIZhXAid5cA1exk0zCWGFFf54IEwnIEcOUKZEe6vmF1QQxm4FN9NSZTbQfMblipF9aCIaGLLXhAXkXSJA24GODIVK+suvNeE8MfkeMMPtvzNw3Zzd28S4wxaQd9RCwVoG+2ifXSAuoihW3SHfqHf3r334D16Ty+lNW/Ss4zeoTb1DPrLsQQ=</latexit>
, q, m)
(r t+1 , ṙ t+1 , q, m)
<latexit sha1_base64="r9uhm7/gmf3DJvge/uAVQWypHn8=">AAACSXicdVDLTuMwFHXKMECZRxmWbKypkDoCRfEUQtkh2LAEiQJSEyrHdcHCdoJ9M6Mqyi/M17CFH+AL+Ax2iBVOKQjQcCRLx+fc6+t7kkwKC0Fw69WmPk1/npmdq89/+frte2Phx6FNc8N4l6UyNccJtVwKzbsgQPLjzHCqEsmPkvOdyj/6w40VqT6AUcZjRU+1GApGwUn9RqsVJaow5UkBK6RcxdUtGqTgpBftYhWrX/1GM/A3N0kn6ODA74TrG+trjpB22A5DTPxgjCaaYK+/4M25h1iuuAYmqbU9EmQQF9SAYJKX9Si3PKPsnJ7ynqOaKm7jYrxSiZedMsDD1LijAY/V1x0FVdaOVOIqFYUz+96rxP95vRyGnbgQOsuBa/Y0aJhLDCmu8sEDYTgDOXKEMiPcXzE7o4YycCm+mZIot4Pmf1mqFNWDIqKJLXskLiLpEgfcJDgyFSvrLrznhPDH5PC3T0K/vb/W3NqexDiLltBP1EIEbaAttIv2UBcx9A9doit07d14d9699/BUWvMmPYvoDWpTj/n0sQQ=</latexit>
Ⓑ Ⓒ
① ② ③ ④
PT
ĜE( ) ĜE( )
!" ĜE( ) !" ĜE ( )
RI
ĜE( )
ĜE( )
ĜE ( ) ĜE( )
SC
!$ !# ĜE( ) !$ !#
ĜE( )
ĜE( ) )
ĜE (
U
Edge NN Physics operator GV (·)
N
Edge Part: Learn Pairwise Interaction Law Node Part: Aggregate and Predict
Figure 1: Framework of the proposed model to learn pairwise force or pair- A

M
wise potential energy. (A) The interacting particle system contains three
particles that evolve over time. At every time step, each particle is described
ED
by multiple features, which include position, velocity, charge and mass (rep-
resented by the bar). Position and velocity evolve with time whereas other
properties remain constant. (B-C) The proposed method learns physics-
IT
consistent pairwise force or pairwise potential at every time step t. The

model has two components: the edge part B and the node part C. In the
ED
edge part (B), two nodes’ vectors are concatenated as edge feature (process
¬). An edge neural network ĜE (·; θE ) (θE represents the trainable parame-
ters) takes the edge feature as input (process ) and outputs a learnt vector
N
on that edge representing the pairwise force or potential energy. In the node
U
part (C), the output vectors by the edge neural network and the raw node
feature are aggregated on each node (process ®). We design the determin-
L
istic node operator GV (·) by incorporating physics knowledge to derive the

A
net acceleration on nodes (process ¯). By minimizing the loss on node-level

accelerations, the edge neural network ĜE (·; θE ) will output pairwise force
IN
or potential energy exactly.

G
RI
5
O
PIG’N’PI: Physics-induced Graph Network for

Particle Interaction
We propose a framework that is able to infer pairwise particle forces or poten-
tial energy by simply observing particle motion in time and space. In order
to provide physics-consistent results, a key requirement is that the learnt
particle interactions need to satisfy Newtonian dynamics. One of the main
challenges in developing such a learning algorithm is that only information
PT
on the particle position in time and space along with particle properties (e.g.,
charge and mass) can be used for training the algorithm and no ground truth
RI
information on the interactions is available since it is very difficult to measure
it in real systems.
SC
The proposed framework comprises the following distinctive elements
(Fig. 1): 1) a graph network with a strong inductive bias representing the
U
particles, their properties and their pairwise interactions; and 2) physics-
consistent kinetics imposed by a combination of a neural network for learn-
N
ing the edge function and a deterministic physics operator for computing
A
the node function within the graph network architecture. In addition, the
proposed framework consists of the two steps: 1) training the network to pre-
M
dict the particle motion in time and space; and 2) extraction of the pairwise
forces or the pairwise potential energy from the edge functions of the trained
ED
network.
Particle systems We consider particle systems that are moving in space
and time and are subject to Newtonian dynamics without any external forces.
IT
A particle system in this research is represented by the directed graph G =

(V, E), where nodes V = {v1 , v2 , . . . , v|V | } correspond to the particles and the
ED
directed edges E = {eij : vi , vj ∈ V, i 6= j} correspond to their interactions.

The graph is fully-connected if all particles interact with each other. Each
N
particle i, represented by a node vi , is characterized by its time-invariant

properties, such as charge qi and mass mi and time-dependent properties
U
such as its position rit and its velocity ṙit . We use ηit to denote the features of
particle i at time t, ηit = [rit , ṙit , qi , mi ]. We limit our evaluations to particle
L
systems comprising homogeneous particle types. This results in particles

A
exhibiting only one type of interaction with all its neighboring particles,
leading to |E| = |V |(|V | − 1). We further assume that the position rit of
IN
each particle i is observed at each time step t and that this information is
available during training. Based on the position information rit , velocity ṙit
G
RI
6
O
Nodes
Ⓐ Ⓑ

<latexit sha1_base64="aH61Y56JnFC7KvCQnzluMorYIis=">AAACU3icbVBNT9wwEHVSCuxS6NIee7G6QqJStUoAQS+VUHvpcSuxgLSJool3FixsJ7UnrVZR/kd/Ta/lD3Dgt3CpE/bAR0ey9ea9GY/n5aWSjqLoNghfrLxcXVvv9TdebW69Hmy/OXVFZQVORKEKe56DQyUNTkiSwvPSIuhc4Vl+9bXVz36idbIwJ7QoMdVwYeRcCiBPZYO9JNd1ggRNJvlnvtum1uOPvBNmBfm0y3+0l87kh2wwjEZRF/w5iJdgyJYxzraDnn9IVBoNCQXOTeOopLQGS1IobPpJ5bAEcQUXOPXQgEaX1t1yDd/xzIzPC+uPId6xDztq0M4tdO4rNdCle6q15P+0aUXzT2ktTVkRGnE/aF4pTgVvneIzaVGQWngAwkr/Vy4uwYIg7+ejKbn2Oxj8JQqtwczqBHLXTOO0TpT3nvgw5oltUdP35sVPrXoOTvdG8eFo//vB8PjL0sZ19o69Z7ssZkfsmH1jYzZhgv1mf9hfdh3cBHdhGK7cl4bBsuctexTh5j/Rq7Ld</latexit>
⌘i = (ri , ṙi , qi , mi )
Edges <latexit sha1_base64="aVPYhaHy/ZNTWfTe2AGNk+EPmNM=">AAAB/XicbVA9SwNBEJ3zM8avqKXNYhCs4p2KWgZtLCOYD0iOsLfZS9bs7h27e0I4Dv+CrfZ2YutvsfWXuEmuMIkPBh7vzTAzL4g508Z1v52l5ZXVtfXCRnFza3tnt7S339BRogitk4hHqhVgTTmTtG6Y4bQVK4pFwGkzGN6O/eYTVZpF8sGMYuoL3JcsZAQbKzU6gUhPs26p7FbcCdAi8XJShhy1bumn04tIIqg0hGOt254bGz/FyjDCaVbsJJrGmAxxn7YtlVhQ7aeTazN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8Nxb/89qJCa/9lMk4MVSS6aIw4chEaPw66jFFieEjSzBRzN6KyAArTIwNaGZLILKiDcWbj2CRNM4q3mXl/P6iXL3J4ynAIRzBCXhwBVW4gxrUgcAjvMArvDnPzrvz4XxOW5ecfOYAZuB8/QLB/ZW2</latexit>
/
<latexit sha1_base64="aVPYhaHy/ZNTWfTe2AGNk+EPmNM=">AAAB/XicbVA9SwNBEJ3zM8avqKXNYhCs4p2KWgZtLCOYD0iOsLfZS9bs7h27e0I4Dv+CrfZ2YutvsfWXuEmuMIkPBh7vzTAzL4g508Z1v52l5ZXVtfXCRnFza3tnt7S339BRogitk4hHqhVgTTmTtG6Y4bQVK4pFwGkzGN6O/eYTVZpF8sGMYuoL3JcsZAQbKzU6gUhPs26p7FbcCdAi8XJShhy1bumn04tIIqg0hGOt254bGz/FyjDCaVbsJJrGmAxxn7YtlVhQ7aeTazN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8Nxb/89qJCa/9lMk4MVSS6aIw4chEaPw66jFFieEjSzBRzN6KyAArTIwNaGZLILKiDcWbj2CRNM4q3mXl/P6iXL3J4ynAIRzBCXhwBVW4gxrUgcAjvMArvDnPzrvz4XxOW5ecfOYAZuB8/QLB/ZW2</latexit> <latexit sha1_base64="aVPYhaHy/ZNTWfTe2AGNk+EPmNM=">AAAB/XicbVA9SwNBEJ3zM8avqKXNYhCs4p2KWgZtLCOYD0iOsLfZS9bs7h27e0I4Dv+CrfZ2YutvsfWXuEmuMIkPBh7vzTAzL4g508Z1v52l5ZXVtfXCRnFza3tnt7S339BRogitk4hHqhVgTTmTtG6Y4bQVK4pFwGkzGN6O/eYTVZpF8sGMYuoL3JcsZAQbKzU6gUhPs26p7FbcCdAi8XJShhy1bumn04tIIqg0hGOt254bGz/FyjDCaVbsJJrGmAxxn7YtlVhQ7aeTazN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8Nxb/89qJCa/9lMk4MVSS6aIw4chEaPw66jFFieEjSzBRzN6KyAArTIwNaGZLILKiDcWbj2CRNM4q3mXl/P6iXL3J4ynAIRzBCXhwBVW4gxrUgcAjvMArvDnPzrvz4XxOW5ecfOYAZuB8/QLB/ZW2</latexit>
<latexit sha1_base64="1dWjzT8l7uj+K+aQKsWRhzvu0CY=">AAACLnicbVDLSgMxFM34tr6qLt0Ei6AgZcb3RhDduFSwKnTKkElvNTXJDMkdsQzzJf6Ev+BW94ILEVz5Gaa14PPAhZNz7uXmnjiVwqLvP3sDg0PDI6Nj46WJyanpmfLs3KlNMsOhxhOZmPOYWZBCQw0FSjhPDTAVSziLrw66/tk1GCsSfYKdFBqKXWjREpyhk6LyZhirHIooF+2C7tIQ4QZznmjnF8tdLwRkRSRW6dejvRKVK37V74H+JUGfVEgfR1H5LWwmPFOgkUtmbT3wU2zkzKDgEopSmFlIGb9iF1B3VDMFtpH3zivoklOatJUYVxppT/0+kTNlbUfFrlMxvLS/va74n1fPsLXTyIVOMwTNPxe1Mkkxod2saFMY4Cg7jjBuhPsr5ZfMMI4u0R9bYlWUXCjB7wj+ktO1arBVXT/eqOzt9+MZIwtkkSyTgGyTPXJIjkiNcHJL7skDefTuvCfvxXv9bB3w+jPz5Ae89w+m0qnL</latexit>
/ /
eij = concat(⌘i , ⌘j )
Edge NN
Edge Messages Mij
<latexit sha1_base64="UioYb2FoqYiAXTCMbfwWKnN8fBA=">AAACCXicbVC7SgNBFJ2Nr5j4WLW0GYyCVdhVUMugjY0QwTwgWZbZySQZMzO7zMwGwrJf4C/Yam8jYpuvsPUftHY2SWGiBy4czrmXezhBxKjSjvNh5ZaWV1bX8uuF4sbm1ra9s1tXYSwxqeGQhbIZIEUYFaSmqWakGUmCeMBIIxhcZX5jSKSiobjTo4h4HPUE7VKMtJF8225zpPsYseQm9RN6n/p2ySk7E8C/xJ2RUuXw63U8LH5Xffuz3QlxzInQmCGlWq4TaS9BUlPMSFpox4pECA9Qj7QMFYgT5SWT5Ck8MkoHdkNpRmg4UX9fJIgrNeKB2cxyqkUvE//zWrHuXngJFVGsicDTR92YQR3CrAbYoZJgzUaGICypyQpxH0mEtSlr7kvA04IpxV2s4C+pn5Tds/LprWnnEkyRB/vgABwDF5yDCrgGVVADGAzBI3gCz9aD9WK9We/T1Zw1u9kDc7DGP7LFnuM=</latexit>
Node part design: learning force

MAEef MAE
<latexit sha1_base64="o9AGBfyC27264MXrWkQCV+03RQU=">AAACH3icbVC7SgNBFJ2NrxhfUcHGZjAoVmFXQS2jItgICZhESEKYndzVwdkHM3fFsOwf+BN+gI2tloKd2Kax8EucbFL4OjDM4Zx7ufceN5JCo20PrNzE5NT0TH62MDe/sLhUXF5p6DBWHOo8lKG6cJkGKQKoo0AJF5EC5rsSmu718dBv3oDSIgzOsR9Bx2eXgfAEZ2ikbnGrjXCLCc0+7SVnhycjGruaKxFhAl6apt1iyS7bGehf4oxJqbJW+xAPRy/VbvGz3Qt57EOAXDKtW44dYSdhCgWXkBbasYaI8Wt2CS1DA+aD7iTZPSndNEqPeqEyL0Caqd87EuZr3fddU+kzvNK/vaH4n9eK0TvoJCKIYoSAjwZ5saQY0mE4tCcUcJR9Q5i53uxK+RVTjKOJ8McU108LJhTndwR/SWOn7OyVd2tOqXJERsiTdbJBtolD9kmFnJIqqRNO7sgjeSLP1r31ar1Z76PSnDXuWSU/YA2+AEWeqDQ=</latexit>
<latexit sha1_base64="KLmEujPX5mswcDHX0/JWzdwwGM8=">AAACH3icbZDJSgNBEIZ74hbjFvXoZTBRPIUZFfUYN/AiRDALJCH0dGqSJj0L3TViGOYNfAlfwavevYnXXH0SO8vBJP7Q8PNXFVX9OaHgCi1rYKQWFpeWV9KrmbX1jc2t7PZORQWRZFBmgQhkzaEKBPehjBwF1EIJ1HMEVJ3e9bBefQKpeOA/Yj+Epkc7Pnc5o6ijVvawgfCMyo3vL2/HNnIUkzzEON+4AYE0D2GStLI5q2CNZM4be2JyZKJSK/vTaAcs8sBHJqhSddsKsRlTiZwJSDKNSEFIWY92oK6tTz1QzXj0n8Q80EnbdAOpn4/mKP07EVNPqb7n6E6PYlfN1obhf7V6hO5FM+Z+GCH4bLzIjYSJgTmEY7a5BIairw3VEPStJutSSRlqhFNbHC/JaCj2LIJ5Uzku2GeFk4fTXPFqgidN9sg+OSI2OSdFckdKpEwYeSFv5J18GK/Gp/FlfI9bU8ZkZpdMyRj8Ai1Zo/U=</latexit>
Ⓒ
ep
Aggregator (sum)
P
MAEF
<latexit sha1_base64="svFG4AB89rmOgSBqbW2w2piPJZQ=">AAACPXicbVBNTxsxEPUCbUP6QQJHOFiNKvUU7RbU9hiokLggBYl8SNlt5HVmwYrtXdmzQLTaC7+mV/gD/A5+ALeqV644IUgE+iRLz+/NaGZenElh0fdvvaXllTdv31VWq+8/fPy0Vquvd22aGw4dnsrU9GNmQQoNHRQooZ8ZYCqW0IvHv6Z+7wyMFak+xkkGkWInWiSCM3TSsLYVIlygTYrD3f1y+PSxE6XK3+1hreE3/RnoaxLMSYPM0R7WvdVwlPJcgUYumbWDwM8wKphBwSWU1TC3kDE+ZicwcFQzBTYqZmeU9ItTRjRJjXsa6Ux93lEwZd1msatUDE/tS28q/s8b5Jj8jAqhsxxB88dBSS4ppnSaCR0JAxzlxBHGjXC7Un7KDOPokluYEit3g4ZznirF9KgIWWzLQRAVoXQpI20ENDRTVlZdeMHLqF6T7rdm8L25fbTTaO3NY6yQTfKZfCUB+UFa5IC0SYdwckn+kCty7d14d95f799j6ZI379kgC/DuHwCRA66S</latexit>
<latexit sha1_base64="kdgpRV9SmbJuH7sUMzimR5cjXVU=">AAACPXicbVDLSiQxFE35GLXVsdWlLsI0gqumyhnUpTooboQWbBW6yiaVvqXBJFUkt9SmqM18jVv9Ab/DD3Anbt2afgg+5kDg5Jx7ufeeOJPCou8/eiOjY+M/JianKtMzsz/nqvMLxzbNDYcmT2VqTmNmQQoNTRQo4TQzwFQs4SS+/NvzT67AWJHqI+xmECl2rkUiOEMntavLIcIN2qQ42N4t2+8f21WqPNtrV2t+3e+DfifBkNTIEI32vDcVdlKeK9DIJbO2FfgZRgUzKLiEshLmFjLGL9k5tBzVTIGNiv4ZJV1xSocmqXFPI+2rHzsKpqzbLHaViuGF/er1xP95rRyTzagQOssRNB8MSnJJMaW9TGhHGOAou44wboTblfILZhhHl9ynKbFyN2i45qlSTHeKkMW2bAVREUqXMtJaQEPTY2XFhRd8jeo7OV6rB+v134d/als7wxgnyRL5RVZJQDbIFtknDdIknPwjt+SO3HsP3pP37L0MSke8Yc8i+QTv9Q1/q66I</latexit>
symm MAEsymm + + +
<latexit sha1_base64="tiPPO38HztYuEKQvifcFA9QqTxA=">AAAB/XicbVBNSwMxEJ31s9avqkcvwSIIQtlVUY9FLx4r2A9ol5JNs21skl2SrFCWxb/gVe/exKu/xau/xLTdg219MPB4b4aZeUHMmTau++0sLa+srq0XNoqbW9s7u6W9/YaOEkVonUQ8Uq0Aa8qZpHXDDKetWFEsAk6bwfB27DefqNIskg9mFFNf4L5kISPYWKnRCUR6mnVLZbfiToAWiZeTMuSodUs/nV5EEkGlIRxr3fbc2PgpVoYRTrNiJ9E0xmSI+7RtqcSCaj+dXJuhY6v0UBgpW9Kgifp3IsVC65EIbKfAZqDnvbH4n9dOTHjtp0zGiaGSTBeFCUcmQuPXUY8pSgwfWYKJYvZWRAZYYWJsQDNbApEVbSjefASLpHFW8S4r5/cX5epNHk8BDuEITsCDK6jCHdSgDgQe4QVe4c15dt6dD+dz2rrk5DMHMAPn6xe7qZWy</latexit> <latexit sha1_base64="tiPPO38HztYuEKQvifcFA9QqTxA=">AAAB/XicbVBNSwMxEJ31s9avqkcvwSIIQtlVUY9FLx4r2A9ol5JNs21skl2SrFCWxb/gVe/exKu/xau/xLTdg219MPB4b4aZeUHMmTau++0sLa+srq0XNoqbW9s7u6W9/YaOEkVonUQ8Uq0Aa8qZpHXDDKetWFEsAk6bwfB27DefqNIskg9mFFNf4L5kISPYWKnRCUR6mnVLZbfiToAWiZeTMuSodUs/nV5EEkGlIRxr3fbc2PgpVoYRTrNiJ9E0xmSI+7RtqcSCaj+dXJuhY6v0UBgpW9Kgifp3IsVC65EIbKfAZqDnvbH4n9dOTHjtp0zGiaGSTBeFCUcmQuPXUY8pSgwfWYKJYvZWRAZYYWJsQDNbApEVbSjefASLpHFW8S4r5/cX5epNHk8BDuEITsCDK6jCHdSgDgQe4QVe4c15dt6dD+dz2rrk5DMHMAPn6xe7qZWy</latexit> <latexit sha1_base64="tiPPO38HztYuEKQvifcFA9QqTxA=">AAAB/XicbVBNSwMxEJ31s9avqkcvwSIIQtlVUY9FLx4r2A9ol5JNs21skl2SrFCWxb/gVe/exKu/xau/xLTdg219MPB4b4aZeUHMmTau++0sLa+srq0XNoqbW9s7u6W9/YaOEkVonUQ8Uq0Aa8qZpHXDDKetWFEsAk6bwfB27DefqNIskg9mFFNf4L5kISPYWKnRCUR6mnVLZbfiToAWiZeTMuSodUs/nV5EEkGlIRxr3fbc2PgpVoYRTrNiJ9E0xmSI+7RtqcSCaj+dXJuhY6v0UBgpW9Kgifp3IsVC65EIbKfAZqDnvbH4n9dOTHjtp0zGiaGSTBeFCUcmQuPXUY8pSgwfWYKJYvZWRAZYYWJsQDNbApEVbSjefASLpHFW8S4r5/cX5epNHk8BDuEITsCDK6jCHdSgDgQe4QVe4c15dt6dD+dz2rrk5DMHMAPn6xe7qZWy</latexit>
P
Nodes Mi = j Mij
<latexit sha1_base64="WIUEzdY8kZRw2jcJ1bCA/84RF7k=">AAACI3icbVC7SgNBFJ2Nrxhfq5Y2g0EQi7iroDaBoI2NEME8IFmW2ckkmWRmd5mZFcKyta2fYOMv2CqW2omNhY1f4mySwiQeuHA4517uvccLGZXKsr6MzNz8wuJSdjm3srq2vmFublVlEAlMKjhggah7SBJGfVJRVDFSDwVB3GOk5vUvUr92S4SkgX+jBiFxOOr4tE0xUlpyzYMmR6qLEYuvEjemCSzCpoy424OTRi+Brpm3CtYQcJbYY5IvHb4+3L21imXX/Gm2Ahxx4ivMkJQN2wqVEyOhKGYkyTUjSUKE+6hDGpr6iBPpxMOfErinlRZsB0KXr+BQ/TsRIy7lgHu6Mz1UTnup+J/XiFT7zImpH0aK+Hi0qB0xqAKYBgRbVBCs2EAThAXVt0LcRQJhpWOc2OLxJKdDsacjmCXVo4J9Uji+tvOlczBCFuyAXbAPbHAKSuASlEEFYHAPnsAzeDEejXfjw/gctWaM8cw2mIDx/QvEYqjU</latexit>
@Mi
<latexit sha1_base64="8aiRyHddOTQbhtwmxw0L33wyBwU=">AAACY3icbVHdShwxGM2O2uqqddXeiRBcBG9cZmxpvRFEofRGUHBV2AzLN9mMG0wyQ/JNZQnzPn2a3to+gO/RzLoX/n0QOJzz/Z5kpZIO4/hfK5qbX/jwcXGpvbyy+mmts75x5YrKctHnhSrsTQZOKGlEHyUqcVNaATpT4jq7O23061/COlmYS5yUItVwa2QuOWCghp0TlmnPxoD+R10P5dE+yy1wz0qwKEF5pgHHPICzoNb1MyHU2Sk37HTjXjwN+hYkM9AlszgfrreW2KjglRYGuQLnBklcYuqbzlyJus0qJ0rgd3ArBgEa0MKlfnpsTXcDM6J5YcMzSKfs8woP2rmJzkJms7t7rTXke9qgwvww9dKUFQrDnwbllaJY0MY5OpJWcFSTAIBbGXalfAzBLQz+vpiS6XCDEfe80BrMyDPIXD1IUs9U+Auk3YQy26C6HcxLXlv1Flwd9JJvvS8XX7vHJzMbF8kW2SF7JCHfyTH5Sc5Jn3Dym/whD+Rv6zFajjaiz0+pUWtWs0leRLT9H7JYu74=</latexit>
@ @
<latexit sha1_base64="h0q8f3Ak4ZEG1siommj8GqeGrTA=">AAACBHicbVDLSsNAFL2pr1pfVZdugkVwVRIVdVl047KCfUATymQ6aYfOTOLMRCghW3/Bre7diVv/w61f4qTNwrYeuHA45744Qcyo0o7zbZVWVtfWN8qbla3tnd296v5BW0WJxKSFIxbJboAUYVSQlqaakW4sCeIBI51gfJv7nSciFY3Eg57ExOdoKGhIMdJG8r2Ap16MpKaIZf1qzak7U9jLxC1IDQo0+9UfbxDhhBOhMUNK9Vwn1n6ar8OMZBUvUSRGeIyGpGeoQJwoP50+ndknRhnYYSRNCW1P1b8TKeJKTXhgOjnSI7Xo5eJ/Xi/R4bWfUhEnmgg8OxQmzNaRnSdgD6gkWLOJIQhLan618QhJhLXJae5KwLOKCcVdjGCZtM/q7mX9/P6i1rgp4inDERzDKbhwBQ24gya0AMMjvMArvFnP1rv1YX3OWktWMXMIc7C+fgHEf5kW</latexit>
@
MAEnf MAE
<latexit sha1_base64="R04/ViOlnOOggRdr0jQ35dG1VTM=">AAACF3icbZDLSsNAFIYnXmu9RV0KEiyCq5KoqMuqCG6ECvYCbSiT6aQdOjMJMydiCdn5Er6CW927E7cu3fokTi8L2/rDwMd/zuHM+YOYMw2u+23NzS8sLi3nVvKra+sbm/bWdlVHiSK0QiIeqXqANeVM0gow4LQeK4pFwGkt6F0N6rUHqjSL5D30Y+oL3JEsZASDsVr2XhPoI+gwvb24HmESaKJYDKkMs6xlF9yiO5QzC94YCmiscsv+abYjkggqgXCsdcNzY/BTrIARTrN8M9E0xqSHO7RhUGJBtZ8O78icA+O0nTBS5klwhu7fiRQLrfsiMJ0CQ1dP1wbmf7VGAuG5nzIZJ0AlGS0KE+5A5AxCcdpMUQK8bwCb481fHdLFChMw0U1sCUSWN6F40xHMQvWo6J0Wj+9OCqXLcTw5tIv20SHy0BkqoRtURhVE0BN6Qa/ozXq23q0P63PUOmeNZ3bQhKyvX4/RoRY=</latexit>
<latexit sha1_base64="madhP9KStZDeRCzFN7LxeEW4Ygw=">AAACH3icbZDJSgNBEIZ74hbjFvXoZTBRPIUZFfXoCl4EBbNAEkJPpyZp7OkZumvEMMwb+BK+gle9exOvufokdpaDJv7Q8PNXFVX9eZHgGh2nb2VmZufmF7KLuaXlldW1/PpGRYexYlBmoQhVzaMaBJdQRo4CapECGngCqt7DxaBefQSleSjvsRdBM6AdyX3OKJqold9tIDyh9pObs6uRjT3NFI8wKTYuQSAtyihNW/mCU3KGsqeNOzYFMtZtK//daIcsDkAiE1TruutE2EyoQs4EpLlGrCGi7IF2oG6spAHoZjL8T2rvmKRt+6EyT6I9TH9PJDTQuhd4pjOg2NWTtUH4X60eo3/STLiMYgTJRov8WNgY2gM4dpsrYCh6xlADwdxqsy5VlKFB+GeLF6Q5A8WdRDBtKvsl96h0cHdYOD0f48mSLbJN9ohLjskpuSa3pEwYeSav5I28Wy/Wh/VpfY1aM9Z4ZpP8kdX/ATuoo/4=</latexit>
np F̂i =
@ri
Node Part Operator
PT
Force Potential <latexit sha1_base64="aVPYhaHy/ZNTWfTe2AGNk+EPmNM=">AAAB/XicbVA9SwNBEJ3zM8avqKXNYhCs4p2KWgZtLCOYD0iOsLfZS9bs7h27e0I4Dv+CrfZ2YutvsfWXuEmuMIkPBh7vzTAzL4g508Z1v52l5ZXVtfXCRnFza3tnt7S339BRogitk4hHqhVgTTmTtG6Y4bQVK4pFwGkzGN6O/eYTVZpF8sGMYuoL3JcsZAQbKzU6gUhPs26p7FbcCdAi8XJShhy1bumn04tIIqg0hGOt254bGz/FyjDCaVbsJJrGmAxxn7YtlVhQ7aeTazN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8Nxb/89qJCa/9lMk4MVSS6aIw4chEaPw66jFFieEjSzBRzN6KyAArTIwNaGZLILKiDcWbj2CRNM4q3mXl/P6iXL3J4ynAIRzBCXhwBVW4gxrUgcAjvMArvDnPzrvz4XxOW5ecfOYAZuB8/QLB/ZW2</latexit>
/
<latexit sha1_base64="aVPYhaHy/ZNTWfTe2AGNk+EPmNM=">AAAB/XicbVA9SwNBEJ3zM8avqKXNYhCs4p2KWgZtLCOYD0iOsLfZS9bs7h27e0I4Dv+CrfZ2YutvsfWXuEmuMIkPBh7vzTAzL4g508Z1v52l5ZXVtfXCRnFza3tnt7S339BRogitk4hHqhVgTTmTtG6Y4bQVK4pFwGkzGN6O/eYTVZpF8sGMYuoL3JcsZAQbKzU6gUhPs26p7FbcCdAi8XJShhy1bumn04tIIqg0hGOt254bGz/FyjDCaVbsJJrGmAxxn7YtlVhQ7aeTazN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8Nxb/89qJCa/9lMk4MVSS6aIw4chEaPw66jFFieEjSzBRzN6KyAArTIwNaGZLILKiDcWbj2CRNM4q3mXl/P6iXL3J4ynAIRzBCXhwBVW4gxrUgcAjvMArvDnPzrvz4XxOW5ecfOYAZuB8/QLB/ZW2</latexit> <latexit sha1_base64="aVPYhaHy/ZNTWfTe2AGNk+EPmNM=">AAAB/XicbVA9SwNBEJ3zM8avqKXNYhCs4p2KWgZtLCOYD0iOsLfZS9bs7h27e0I4Dv+CrfZ2YutvsfWXuEmuMIkPBh7vzTAzL4g508Z1v52l5ZXVtfXCRnFza3tnt7S339BRogitk4hHqhVgTTmTtG6Y4bQVK4pFwGkzGN6O/eYTVZpF8sGMYuoL3JcsZAQbKzU6gUhPs26p7FbcCdAi8XJShhy1bumn04tIIqg0hGOt254bGz/FyjDCaVbsJJrGmAxxn7YtlVhQ7aeTazN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8Nxb/89qJCa/9lMk4MVSS6aIw4chEaPw66jFFieEjSzBRzN6KyAArTIwNaGZLILKiDcWbj2CRNM4q3mXl/P6iXL3J4ynAIRzBCXhwBVW4gxrUgcAjvMArvDnPzrvz4XxOW5ecfOYAZuB8/QLB/ZW2</latexit>
/ /
Nodes
MAEacc
<latexit sha1_base64="hfAAiV0WQKJySp6Qt52nsZpEUuw=">AAACIHicbVC7SgNBFJ2NrxhfUcHGZjCIVmFXQS1jRLAREjAPSEKYncwmQ2YfzNwVw7Kf4E/YW9lqZ2EnllpY+CVOsilM4oFhDufcy7332IHgCkzz00jNzS8sLqWXMyura+sb2c2tqvJDSVmF+sKXdZsoJrjHKsBBsHogGXFtwWp2/2Lo126ZVNz3bmAQsJZLuh53OCWgpXb2oAnsDiI8+pQTXZ9fJjS0FZU8gIhQGsdxO5sz8+YIeJZYY5Ir7JS/+WPxtdTO/jQ7Pg1d5gEVRKmGZQbQiogETgWLM81QsYDQPumyhqYecZlqRaODYryvlQ52fKmfB3ik/u2IiKvUwLV1pUugp6a9ofif1wjBOWtF3AtCYB5NBjmhwODjYTq4wyWjIAaaEH2+3hXTHpGEgs5wYortxhkdijUdwSypHuWtk/xx2coViihBGu2iPXSILHSKCugKlVAFUXSPntAzejEejDfj3fhISlPGuGcbTcD4+gULjaia</latexit>
Backpropagation to learn
the parameters ✓ in edge NN
RI
<latexit sha1_base64="mu9AyaUmwCOswfTGENJHyTvkk88=">AAAB/XicbVDLSgNBEJyNr7i+oh69DAbBU9hVUC9i0IvHCOYByRJmJ5NkzMzsMtMrhCX4C171Kt7Eq5/gJ4hXv8TJ42ASCxqKqm66u8JYcAOe9+1kFhaXlleyq+7a+sbmVm57p2KiRFNWppGIdC0khgmuWBk4CFaLNSMyFKwa9q6GfvWeacMjdQv9mAWSdBRvc0rASpUGdBmQZi7vFbwR8DzxJyR/8emexy9fbqmZ+2m0IppIpoAKYkzd92IIUqKBU8EGbiMxLCa0RzqsbqkikpkgHV07wAdWaeF2pG0pwCP170RKpDF9GdpOSaBrZr2h+J9XT6B9FqRcxQkwRceL2onAEOHh67jFNaMg+pYQqrm9FdMu0YSCDWhqSygHrg3Fn41gnlSOCv5J4fjGzxcv0RhZtIf20SHy0SkqomtUQmVE0R16RE/o2XlwXp03533cmnEmM7toCs7HL4jFmRQ=</latexit>
Node part design: learning potential
SC
Figure 2: Physics-induced graph network for particle interaction (PIG’N’PI).
(A) The workflow where the edge neural network ĜE (·; θE ) takes edge fea-
U
tures as input. The corresponding output message Mij is the predicted
pairwise force or potential energy, depending on the physics operator (B) or
N
(C) in the node part. Parameters θE in ĜE (·; θE ) are trained by minimizing
the loss on particle acceleration.
A
M
and acceleration r̈it are computed.
Proposed framework The proposed physics-induced graph network for
ED
particle interaction (PIG’N’PI) framework extends the general Graph Net-

work framework proposed by [26], which is a generalized form of message-
IT
passing graph neural networks. The architecture of the proposed framework

is illustrated in Fig. 2. We use a directed graph to represent the inter-
ED
acting particle system where nodes correspond to the particles and edges
correspond to their interactions. The framework imposes a strong inductive
bias and enables to learn the position-invariant interactions across the en-
N
tire particle system network. Given the particle graph structure, the input
is then defined by the node features ηi (representing particle’s characteris-
U
tics). The target output is the acceleration r̈it of each node at time step t.
The standard GNs block [26], typically, comprises two neural networks: an
L
edge neural network ĜE (·; θE ) and a node neural network ĜV (·; θV ), where
A
θE and θV are the trainable parameters. Here, we propose to substitute the

IN
node neural network ĜV (·; θV ) by a deterministic node operator GV (·) to en-
sure that the learned particle interactions are consistent with the underlying
physical laws. The main novelty compared to the standard GN framework
G
RI
7
O
is, that we impose known basic physical laws to ensure that the inferred

pairwise force or potential energy correspond to the real force or potential
energy while only being trained on predicting the acceleration of the par-
ticles. Whether to use the force-based PIG’N’PI (blue color in Fig. 2) or
the potential-based PIG’N’PI (purple color in Fig. 2) depends on different
demands or downstream applications in practice, i.e., we can use force-based
PIG’N’PI if we are only interested in the pairwise force without caring about
the pairwise potential energy; or we can use potential-based PIG’N’PI if we
PT
want to know the pairwise potential. Furthermore, note that the force-based
PIG’N’PI method is not limited to energy-conserving systems, but can also
RI
be extended to dissipative systems (e.g., friction).
It is important to emphasize that only information on particle positions
SC
is used for training the algorithm and the ground-truth information on the
forces and the potential energy is not available during training. For each edge,
the property vectors ηi of two nodes connected by an edge are concatenated
U
as the edge feature vector. The edge neural network ĜE (·; θE ) outputs a
N
message on every edge that corresponds to the pairwise force or potential
energy. The output dimension is set to be the same as the spatial dimension
A
d (two or three) if ĜE (·; θE ) is targeted to learn the pairwise force or one
M
to learn the pairwise potential energy. Edge messages are aggregated on
nodes and the node part operator computes the output corresponding to
the acceleration of nodes, imposing physics-consistency on edge messages.
ED
Trained to predict the node-level acceleration, once applied to a new particle

system, the GN predicts the particle motion at consecutive time steps. The
IT
pairwise forces or the pairwise potential energy can then be extracted from
the edge function for each time step.
ED
Contributions of the present work compared to precious research Here,

we propose a methodology to learn the pairwise force or pairwise potential
energy from the observed particle trajectories. This focus distinguishes our
N
work from many previous works such as [27, 28, 29, 14, 30, 31] that learn
U
the energy of the system and then derive the per-particle dynamics from
the global energy. Moreover, as outlined above, our proposed approach does
L
not have access to any ground truth information during training but rather
A
learns to infer the force and potential energy indirectly. This is contrary to
the previously proposed approaches that rely on such information [17, 21].
IN
While our proposed framework has several similarities with two previously
proposed frameworks that are also aiming to infer pairwise force and pairwise
G
potential energy using also only the particle accelerations for training [22,
RI
8
O
22], none of the previously proposed methods is able to infer the underlying

particle interactions that are consistent with the underlying physical laws.
We demonstrate in our experiments that the learnt particle interactions of
the previously proposed approaches are not consistent with the underlying
physical laws and do not correspond to the real forces or potential energy.
In fact, the proposed algorithm has also similarities to the Physics-informed
neural networks (PINNs) [32] which aim to solve partial differential equa-
tions. Both PINNs and PIG’N’PI integrate known physical laws. While
PT
PIG’N’PI integrates Newton’s second law, PINNs enforce the structure im-
posed by partial differential equation at a finite set of collocation points.
RI
Results and Discussion
SC
Performance evaluation metrics
U
We evaluate the performance of the proposed PIG’N’PI framework on syn-
N
thetic data generated from two- (d = 2) and three- (d = 3) dimensional nu-
A
merical simulations. The key distinctive property of the generated datasets
is the definition of the inter-particle potential energy P , which defines the
M
inter-particle pairwise force by F = −∂P/∂r. The selected cases, which have
also been used in prior work [22] and can be considered as a benchmark case
ED
study, cover a wide range of particle interaction features, including depen-

dence on particle properties, e.g., mass and charge, dependence on interaction
properties, e.g., stiffness, and varying degrees of smoothness (see Table 1 and
IT
Fig. S2 in SI Appendix J for visualization).

The method developed by [22], which applies multilayer perceptrons (MLPs
ED
[33]) in the edge and node part, serves as the baseline for comparison. We do
not change the architecture of the baseline except for changing the output di-
N
mension of its edge part MLPs when learning the pairwise force or potential
energy. The output dimension is a d-dimensional vector for learning the pair-
U
wise force and a one-dimensional scalar for the potential energy. Besides the
baseline, we compare the performance of PIG’N’PI to an alternative method
L
proposed by [34] that is also based on GN and was specifically designed to

A
infer pairwise forces. We denote this method as GN+ and the details are
introduced in Sec. .
IN
We split each dataset into training, validation and testing datasets and
use Ttrain , Tvalid and Ttest to indicate the corresponding simulation time steps
G
RI
9
O
Table 1: The force and potential energy equations for different datasets,

where Fij is the force from particle j to particle i , Pij is the potential
incurred by particle j on particle i, rij is the Euclidean distance between
particle i and particle j, nij is the unit vector pointing from particle i to
particle j, qi and mi are the electric charge and mass of particle i. k, L, c
and Θ are constants.
PT
Dataset Pairwise force (Fij ) Pairwise Potential (Pij )
1
Spring k(rij − L)nij 2
k(rij − L)2
RI
2
Charge −cqi qj nij /rij cqi qj /rij
Orbital mi mj nij /rij mi mj ln(rij )
SC
Discnt 0, if rij < Θ 0, if rij < Θ
(rij − 1)nij , otherwise 0.5(rij − 1)2 , otherwise
U
for these different splits. Details regarding the dataset generation are pro-
N
vided in Sec. . The baseline algorithm and PIG’N’PI are trained and eval-
A
uated on the same training and testing datasets from simulations with an
8-particle system. Further, the evaluation of the generalization ability uses
M
a 12-particle system.
It should be noted that [22] measure the quality of the learnt forces by
ED
quantifying the linear correlation between each dimension of the learnt edge
message and all dimensions of the ground-truth pairwise force. This is a nec-
essary but not sufficient condition to claim the correspondence of the learnt
IT
edge message with the pairwise interactions and to evaluate the performance
of the indirect inference of the pairwise interactions. Instead, we evaluate
ED
the proposed methodology with a focus on two key aspects: 1) supervised

learning performance, and 2) consistency with underlying physics. For all
N
the evaluations, the mean absolute error on the testing dataset of various
particle and interaction properties is used and is defined as follows
U
i6=j
1 1 X X
L
inter
MAE (φ̂, φ) = l1 (φ̂tij , φtij ) , (1)
|Ttest | |E| t∈T
A
i,j
test
IN
and
|V |
part 1 1 X X
MAE (φ̂, φ) = l1 (φ̂ti , φti ) (2)
G
|Ttest | |V | t∈T i=1

test
RI
10
O
respectively, where the superscript hat indicates the predicted values. Here,

φ̂ij and φij are the predicted and corresponding ground-truth, respectively,
of a physical
P quantity between
P particle j and particle i (e.g., pairwise force),
and φ̂i = j φ̂ij and φi = j φij are the aggregated prediction and the cor-
responding ground-truth, respectively, on particle i (e.g., net force). l1 (x, y)
computes thePsum of absolute differences between each element in x and
y, l1 (x, y) = i |xi − yi |, if x and y are vectors or the absolute difference,
l1 (x, y) = |x − y|, if x and y are scalars. Hence, MAEpart measures the av-
PT
eraged error of the physical quantity on particles over Ttest , and MAEinter is
the averaged error of the inter-particle physical quantity over Ttest .
RI
The supervised learning performance is evaluated on the prediction of
the acceleration MAEacc = MAEpart (ˆ r̈, r̈). The true acceleration values serve
SC
as target values during training. The physical consistency is evaluated on
two criteria. First, we evaluate the ability of the proposed framework to
infer the underlying physical quantities that were not used as target during
U
training (e.g., pairwise force), and second, we evaluate physical consistency
N
by verifying whether Newton’s action-reaction property is satisfied.
The following metrics are used to evaluate the consistency with the true
A
pairwise interaction. For pairwise force, we use MAEef = MAEinter (F̂ , F );
M
and for potential energy case, we evaluate the increment in potential energy
MAE∆ep = MAEinter (P̂ − P̂ 0 , P − P 0 ), where superscript 0 refers to the initial
configuration.
ED
For the second part of the evaluation of the physical consistency, we verify
whether Newton’s action-reaction property is satisfied. For that, we P evaluateP
IT
the symmetry in either inter-particle forces with MAEsymm = |Ttest | |E| t∈Ttest i6i,j=j l1 (F̂ijt , −F̂jit )
F 1 1
P Pi6=j
or inter-particle potential with MAEPsymm = |Ttest1 1 t t
i,j l1 (P̂ij , P̂ji ).
ED
| |E| t∈Ttest
Performance evaluation between PIG’N’PI, GN+ and

N
baseline for pairwise force

U
First, we analyze PIG’N’PI for application on particle systems with inter-

actions given by pairwise forces. We start with evaluating the supervised
L
learning performance by evaluating the prediction of the acceleration using

A
MAEacc . The results show that PIG’N’PI provides slightly better predic-
tions than the GN+ and the baseline model for both the spring dataset (see
IN
Fig. 3D) and all other datasets (see SI Appendix B).

To verify the physical consistency, we first evaluate if the implicitly in-
G
RI
11
O
(A) Ground-truth force field Predicted force field by baseline Predicted force field by PIG’N’PI

y dimension
y dimension
y dimension
PT
x dimension x dimension x dimension
(B) Magnitude error: Magnitude error: (C) Angle error: Angle error:
baseline and ground-truth PIG’N’PI and ground-truth baseline and ground-truth PIG’N’PI and ground-truth
RI
y dimension
y dimension
y dimension
y dimension
SC
U
x dimension x dimension x dimension x dimension
N
(D) MAEacc standard deviation MAEef standard deviation symm standard deviation
MAEF Generalization MAEef std
<latexit sha1_base64="/DcCJ7aInkkeonnE9gAvcAVMorY=">AAACSnicbZDNThsxEMe94Tt8BTj2YhGQ4BLtAgKOfKhVL5VAagApu0ReZzZY2N6VPQtE1j4DT9Nr+wJ9gb5Gb1UvdUIq8TWSpZ//M6OZ+aeFFBbD8FdQm5icmp6ZnavPLywuLTdWVi9sXhoObZ7L3FylzIIUGtooUMJVYYCpVMJlens6zF/egbEi119xUECiWF+LTHCGXuo2tmOEB0wztzECm7kvxx+r7v+PHShVXbtP1UbVbTTDVjgK+haiMTTJOM66K8Fc3Mt5qUAjl8zaThQWmDhmUHAJVT0uLRSM37I+dDxqpsAmbnRTRTe90qNZbvzTSEfq8w7HlPXbpb5SMbyxr3ND8b1cp8TsMHFCFyWC5k+DslJSzOnQINoTBjjKgQfGjfC7Un7DDOPobXwxJVX+Bg33PFeK6Z6LWWqrTpS4WHrLkTYjGpshVXVvXvTaqrdwsdOK9lu753vNo5OxjbPkA1knWyQiB+SIfCZnpE04eSTfyHfyI/gZ/A7+BH+fSmvBuGeNvIja5D/8qrMs</latexit>
<latexit sha1_base64="xOg6+IsBdBgvhc8AOXchqx8x6pY=">AAACRHicbZBLSgQxEIbTvh3funQTHAVXQ7eKuvSB4EZQcFSYboZ0plrDJOkmqVaHpg/gadzqBbyDd3AnbsXMOIKvgsBXf1VRqT/OpLDo+8/ewODQ8Mjo2HhlYnJqemZ2bv7MprnhUOepTM1FzCxIoaGOAiVcZAaYiiWcx+39bv38GowVqT7FTgaRYpdaJIIzdFJzthoi3GKcFMs9sElxtHtQNr8SSMrl0nX5Nb8X9C8EfaiSfhw357zxsJXyXIFGLpm1jcDPMCqYQcEllJUwt5Ax3maX0HComQIbFb1rSrrilBZNUuOeRtpTv08UTFnbUbHrVAyv7O9aV/yv1sgx2Y4KobMcQfPPRUkuKaa0aw1tCQMcZccB40a4v1J+xQzj6Az8sSVW7gYNNzxViulWEbLYlo0gKkLpzEZaDWhoulRWnHnBb6v+wtlaLdisrZ9sVHf2+jaOkUWyRFZJQLbIDjkkx6ROOLkj9+SBPHpP3ov36r19tg54/ZkF8iO89w9BvbFY</latexit>
<latexit sha1_base64="1lL5fyQfCaRazWqJLMhzjqd+vLo=">AAACSXicbZDPThRBEMZ7FlFYRRY8eum4MeG0mVECHFFi4sUEExdIdiab6t4a6NDdM+muUTedeQWehqu8AE/gY3gznuz9cxCwkk7/8lVVquoTtVae0vRn0ll5tPr4ydp69+mzjeebva3tE181TuJQVrpyZwI8amVxSIo0ntUOwQiNp+LyaJY//YrOq8p+oWmNhYFzq0olgaI07u3khN9JlGH++zJ8evdhgY3w0qmaAkjZtu24108H6Tz4Q8iW0GfLOB5vJev5pJKNQUtSg/ejLK2pCOBISY1tN2881iAv4RxHES0Y9EWYn9Ty11GZ8LJy8Vnic/XfjgDG+6kRsdIAXfj7uZn4v9yoofKgCMrWDaGVi0FlozlVfOYPnyiHkvQ0AsT7465cXoADSdHFO1OEiTdY/CYrY8BOQg7Ct6OsCLmOjhPvZzx3M2q70bzsvlUP4eTNINsbvP282z98v7Rxjb1kr9gOy9g+O2Qf2TEbMsmu2DX7wW6S2+RX8jv5syjtJMueF+xOdFb+Ag0Js0Y=</latexit>
Particle acceleration MAEacc
A
Pairwise force MAEFsymm
Pairwise force MAEef

MAEacc average MAEef average MAEF average Generalization MAEef average
symm
M
ED
2D 3D 2D 3D 2D 3D 2D 3D
IT
Figure 3: Case study: Quality of pairwise force prediction of PIG’N’PI and

ED
the baseline model on two-dimensional spring dataset. (A) The spring force
field around a given particle. Color indicates the force amplitude. From left
to right: ground-truth spring force field, predicted force field by the baseline
N
model, predicted force field by PIG’N’PI. (B) The magnitude error between
predicted force and the ground-truth force (|norm(F̂ ) − norm(F )|). Left is
U
the result of baseline model and right is the result of PIG’N’PI. (C) The angle
L
difference between predicted force and the ground-truth force (Angle(F̂ , F ),

in radian). Left is the result of baseline model and right is the result of
A
PIG’N’PI. (D) Comparison of the quality of PIG’N’PI, GN+ and baseline

IN
model on learning the pairwise force, where bottom is average result of five

experiments and top is the corresponding standard deviation. From left to

G
right (in logarithmic scale): acceleration error MAEacc , pairwise force error
MAEef , force symmetry error MAEFsymm and pairwise force error MAEef on
RI
generalization dataset. 12
O
ferred pairwise forces are consistent with the true physical quantity. PIG’N’PI

is able to infer the force field around a particle correctly, while the baseline
model fails to predict the force field (see Fig. 3A for the spring dataset). A
force field needs to be precise in both amplitude and direction. The error of
the magnitude (see Fig. 3B) and angle (see Fig. 3C) demonstrate unambigu-
ously the superior performance of PIG’N’PI compared to the baseline model.
We quantitatively summarize the performance of the pairwise force inference
with MAEef , which shows PIG’N’PI is able to infer the pairwise force cor-
PT
rectly, while both GN+ and the baseline fail to infer the pairwise force (2 − 3
orders of magnitude worse inference performance for the spring dataset (see
RI
Fig. 3D) and all other datasets (see Fig. 4A and SI Appendix B)).
Secondly, we verify the consistency of the implicitly inferred pairwise
SC
forces with Newton’s action-reaction law by evaluating the symmetry of the
inter-particle forces with MAEFsymm . Our results demonstrate that PIG’N’PI
satisfies the symmetry property. However, GN+ and the baseline model are
U
not able to satisfy the underlying Newton’s laws for the spring dataset (see
N
Fig. 3D) and the other datasets (see Fig. 4B and SI Appendix B).
Furthermore, we test the robustness of PIG’N’PI and the baseline model
A
to learn from noisy data. We impose noise to the measured positions and
M
then compute the noisy velocities (first-order derivative of position) and noisy
accelerations (second-order derivative of position). The noisy accelerations
serve then as the target values for the learning tasks of all the models. The
ED
performance of PIG’N’PI decreases with increasing noise level (see SI Ap-

pendix I; Table S11, Table S12 and Table S13). This is to be expected given
IT
that adding noise makes the training target (particle accelerations) less sim-
ilar to the uncorrupted target that is associated with particle interactions.
ED
However, PIG’N’PI can still learn reasonably well the particle interactions
despite the corrupted data. The performance of the baseline model fluctu-
ates, however, with different noise levels significantly. This is due to the fact
N
that the baseline model does not learn the particle interactions but rather
U
the particle kinematics and is, therefore, more sensitive to noise.

Finally, we note that the proposed algorithm is also able to generalize
L
well when trained on an eight-particle system and applied to a 12-particle

A
system for all datasets (see Fig. 3D, Fig. 4 and Table S4).
Overall, the results demonstrate that the proposed algorithm learns cor-
IN
rectly the pairwise force (that is consistent with the underlying physics)
without any direct supervision, i.e., without access to the pairwise force in
G
the first place, and that the inferred forces are consistent with the imposed
RI
13
O
MAEef standard deviation symm standard deviation
MAEF Generalization MAEef std

Pairwise force MAEFsymm

MAEef average MAEF

symm average Generalization MAEef average
PT
2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D
Charge Orbital Discnt Charge Orbital Discnt Charge Orbital Discnt
(A) (B) (C)
RI
Figure 4: Quality of pairwise force prediction of PIG’N’PI, the GN+ and the
baseline model. We report the average and standard deviation of different
SC
errors with five experiments, in logarithmic scale. (A) Pairwise force predic-
tion error MAEef . (B) Pairwise force symmetry error MAEFsymm . (C) Pairwise
U
force error MAEef on generalization dataset.
N
underlying physical laws.
A
M
Performance evaluation between PIG’N’PI and baseline
for pairwise potential energy
ED
Besides learning the pairwise force, the proposed methodology is extended

to learn the pairwise potential energy (see node part design for learning po-
IT
tential in Sec. ). In this case, the physics operator computes the pairwise
force via partial derivative. Since GN+ was solely designed for learning the
ED
pairwise force, it is not possible to apply it to infer the pairwise potential.

Therefore, for the task of pairwise potential energy inference, we only com-
pare PIG’N’PI with the baseline model. Our results show that PIG’N’PI
N
performs well in the supervised learning of the acceleration (MAEacc ). Here

U
again, its performance is considerably better compared to the baseline model

(see Fig. 5B). Moreover, the performance is similar to that in the force-based
L
version of the algorithm.

A
However, when comparing the performance of the baseline model on the

supervised learning task between the potential-based version and the force-
IN
based version of the model, the performance reduces significantly in the

potential-based implementation (compare to Fig. 3D). This drop of perfor-
G
RI
14
O
(A) Ground-truth potential field Predicted potential field by baseline Predicted potential field by PIG’N’PI
PT
y dimension
y dimension
y dimension
RI
SC
x dimension x dimension x dimension
U
(B) MAEacc standard deviation standard deviation MAEP standard deviation Generalization MAE std
<latexit sha1_base64="KPN+6OftgRs/NfUQH6UjwqGTCFc=">AAACSnicbZDBbhMxEIa9IdAkpTSFYy8WaSV6iXbbquUYQEi9VAoSSSpll8jrzLZWbe/KngUia5+Bp+EKL8AL9DV6q7jgpItEW0ay9PmfGc3MnxZSWAzDq6DxqPn4yVqr3Vl/uvFss7v1fGzz0nAY8Vzm5ixlFqTQMEKBEs4KA0ylEibp5btlfvIZjBW5/oiLAhLFzrXIBGfopVl3L0b4imnmdlZgM3f65n01+/uxC6WqT25Y7VSzbi/sh6ugDyGqoUfqGM62gnY8z3mpQCOXzNppFBaYOGZQcAlVJy4tFIxfsnOYetRMgU3c6qaK7nplTrPc+KeRrtR/OxxT1m+X+krF8MLezy3F/+WmJWavEyd0USJofjsoKyXFnC4NonNhgKNceGDcCL8r5RfMMI7exjtTUuVv0PCF50oxPXcxS201jRIXS2850l5EY7OkquPNi+5b9RDG+/3oqH/w4bA3eFvb2CLb5CV5RSJyTAbkhAzJiHDyjXwnP8jP4FdwHdwEv29LG0Hd84LciUbzDw4vszY=</latexit>
MAE
<latexit sha1_base64="SyHBxZct1IY25Z+/VniqzzjxBxw=">AAACUHicbZDJbhQxEIarhyXJhGUCx1wsJkichu6ACMewROISKUhMEmm6NbI91YkV292yq4GR1Y/B03CFF+CWN8mNeBYkspRk1a+/qlSuT9RaeUrT86Rz5+69+yura931Bw8fPe5tPDn0VeMkDmWlK3csuEetLA5Jkcbj2iE3QuOROPswqx99RedVZb/QtMbC8BOrSiU5RWvce5kTfidRhnn2Zdh/t7eQjfDSqZrCVv4RNfEtrNu2Hff66SCdB7spsqXowzIOxhvJWj6pZGPQktTc+1GW1lQE7khJjW03bzzWXJ7xExxFablBX4T5ZS17Hp0JKysXnyU2d/+fCNx4PzUidhpOp/56bWbeVhs1VL4tgrJ1Q2jlYlHZaEYVm2FiE+VQkp5GwSOG+FcmT7njkiLMK1uEiTdY/CYrY7idhJwL346yIuQ6gifWz1juZqrtRnjZdVQ3xeH2IHszePX5dX/3/RLjKmzCM3gBGezALnyCAxiChB/wE37B7+RPcpH87SSL1n8ZnsKV6HQvAUoptMk=</latexit>
ep symm ep
Particle acceleration MAEacc
ep
Pairwise potential MAEPsymm
ep
N
Pairwise potential MAE
MAEacc average average MAEP

symm average Generalization MAE average
MAE
ep ep
A
M
ED
2D 3D 2D 3D 2D 3D 2D 3D
Figure 5: Case study: Quality of pairwise potential prediction of PIG’N’PI

IT
and the baseline model on Spring dataset. (A) The spring potential field
around a given particle. Color indicates the potential amplitude. From left
ED
to right: ground-truth spring potential field, predicted potential field by the

baseline model, predicted potential field by PIG’N’PI. (B) Comparison of
the quality of PIG’N’PI and baseline model on learning the pairwise force.
N
From left to right (in logarithmic scale): acceleration error MAEacc , pairwise
U
potential error MAE∆ep , potential symmetry error MAEPsymm and pairwise

potential error MAE∆ep on generalization dataset.
L
A
IN
G
RI
15
O
ep standard deviation MAEP Generalization MAE ep std
MAE symm standard deviation

<latexit sha1_base64="SyHBxZct1IY25Z+/VniqzzjxBxw=">AAACUHicbZDJbhQxEIarhyXJhGUCx1wsJkichu6ACMewROISKUhMEmm6NbI91YkV292yq4GR1Y/B03CFF+CWN8mNeBYkspRk1a+/qlSuT9RaeUrT86Rz5+69+yura931Bw8fPe5tPDn0VeMkDmWlK3csuEetLA5Jkcbj2iE3QuOROPswqx99RedVZb/QtMbC8BOrSiU5RWvce5kTfidRhnn2Zdh/t7eQjfDSqZrCVv4RNfEtrNu2Hff66SCdB7spsqXowzIOxhvJWj6pZGPQktTc+1GW1lQE7khJjW03bzzWXJ7xExxFablBX4T5ZS17Hp0JKysXnyU2d/+fCNx4PzUidhpOp/56bWbeVhs1VL4tgrJ1Q2jlYlHZaEYVm2FiE+VQkp5GwSOG+FcmT7njkiLMK1uEiTdY/CYrY7idhJwL346yIuQ6gifWz1juZqrtRnjZdVQ3xeH2IHszePX5dX/3/RLjKmzCM3gBGezALnyCAxiChB/wE37B7+RPcpH87SSL1n8ZnsKV6HQvAUoptMk=</latexit> <latexit sha1_base64="SyHBxZct1IY25Z+/VniqzzjxBxw=">AAACUHicbZDJbhQxEIarhyXJhGUCx1wsJkichu6ACMewROISKUhMEmm6NbI91YkV292yq4GR1Y/B03CFF+CWN8mNeBYkspRk1a+/qlSuT9RaeUrT86Rz5+69+yura931Bw8fPe5tPDn0VeMkDmWlK3csuEetLA5Jkcbj2iE3QuOROPswqx99RedVZb/QtMbC8BOrSiU5RWvce5kTfidRhnn2Zdh/t7eQjfDSqZrCVv4RNfEtrNu2Hff66SCdB7spsqXowzIOxhvJWj6pZGPQktTc+1GW1lQE7khJjW03bzzWXJ7xExxFablBX4T5ZS17Hp0JKysXnyU2d/+fCNx4PzUidhpOp/56bWbeVhs1VL4tgrJ1Q2jlYlHZaEYVm2FiE+VQkp5GwSOG+FcmT7njkiLMK1uEiTdY/CYrY7idhJwL346yIuQ6gifWz1juZqrtRnjZdVQ3xeH2IHszePX5dX/3/RLjKmzCM3gBGezALnyCAxiChB/wE37B7+RPcpH87SSL1n8ZnsKV6HQvAUoptMk=</latexit>
ep
Pairwise potential MAEPsymm
ep

MAEP
MAE average symm average Generalization MAE average

<latexit sha1_base64="SyHBxZct1IY25Z+/VniqzzjxBxw=">AAACUHicbZDJbhQxEIarhyXJhGUCx1wsJkichu6ACMewROISKUhMEmm6NbI91YkV292yq4GR1Y/B03CFF+CWN8mNeBYkspRk1a+/qlSuT9RaeUrT86Rz5+69+yura931Bw8fPe5tPDn0VeMkDmWlK3csuEetLA5Jkcbj2iE3QuOROPswqx99RedVZb/QtMbC8BOrSiU5RWvce5kTfidRhnn2Zdh/t7eQjfDSqZrCVv4RNfEtrNu2Hff66SCdB7spsqXowzIOxhvJWj6pZGPQktTc+1GW1lQE7khJjW03bzzWXJ7xExxFablBX4T5ZS17Hp0JKysXnyU2d/+fCNx4PzUidhpOp/56bWbeVhs1VL4tgrJ1Q2jlYlHZaEYVm2FiE+VQkp5GwSOG+FcmT7njkiLMK1uEiTdY/CYrY7idhJwL346yIuQ6gifWz1juZqrtRnjZdVQ3xeH2IHszePX5dX/3/RLjKmzCM3gBGezALnyCAxiChB/wE37B7+RPcpH87SSL1n8ZnsKV6HQvAUoptMk=</latexit> <latexit sha1_base64="SyHBxZct1IY25Z+/VniqzzjxBxw=">AAACUHicbZDJbhQxEIarhyXJhGUCx1wsJkichu6ACMewROISKUhMEmm6NbI91YkV292yq4GR1Y/B03CFF+CWN8mNeBYkspRk1a+/qlSuT9RaeUrT86Rz5+69+yura931Bw8fPe5tPDn0VeMkDmWlK3csuEetLA5Jkcbj2iE3QuOROPswqx99RedVZb/QtMbC8BOrSiU5RWvce5kTfidRhnn2Zdh/t7eQjfDSqZrCVv4RNfEtrNu2Hff66SCdB7spsqXowzIOxhvJWj6pZGPQktTc+1GW1lQE7khJjW03bzzWXJ7xExxFablBX4T5ZS17Hp0JKysXnyU2d/+fCNx4PzUidhpOp/56bWbeVhs1VL4tgrJ1Q2jlYlHZaEYVm2FiE+VQkp5GwSOG+FcmT7njkiLMK1uEiTdY/CYrY7idhJwL346yIuQ6gifWz1juZqrtRnjZdVQ3xeH2IHszePX5dX/3/RLjKmzCM3gBGezALnyCAxiChB/wE37B7+RPcpH87SSL1n8ZnsKV6HQvAUoptMk=</latexit>
ep ep
PT
2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D 2D 3D
Charge Orbital Discnt Charge Orbital Discnt Charge Orbital Discnt
(A) (B) (C)
RI
Figure 6: Quality of pairwise potential prediction of PIG’N’PI and the base-
SC
line model. We report different errors in terms of consistency with the under-
lying physical laws, in logarithmic scale. (A) Pairwise potential incremental
error MAE∆ep . (B) Pairwise potential symmetry error MAEPsymm . (C) Pairwise
U
potential error incremental error MAE∆ep on generalization dataset.
N
mance is potentially explained by the adjustment of the output dimension
of the edge neural network in the baseline model to enable the extraction of
A
M
the potential energy.
Further, our results demonstrate a superior performance of the PIG’N’PI
ED
algorithm on consistency with underlying physics. Firstly, it infers well the

increment of the potential energy (see Fig. 5A). It clearly infers the inrement
of the potential field correctly. On the contrary, the baseline model fails to
IT
infer the potential energy. This can be quantitatively assessed with MAE∆ep .
ED
The results (see Fig. 5B and 6) show that PIG’N’PI is able to infer the
potential energy, while the performance of the baseline model indicates that
it is not able to learn the potential energy only from observing particles
N
movements.
It is important to note that our algorithm cannot predict the abso-
U
lute value of the potential energy; only the increment (see SI Appendix D;
Fig. S1). The reason is that the model is trained on the acceleration, which is
L
computed from the derivative of the potential energy (i.e., the force). Hence,
A
the model only constrains the derivative, and the constant of integration re-
IN
mains unknown. This limitation can be overcome by one of the two following
options: either the potential energy is constrained by a spatial boundary con-
dition or by an initial condition. In the former, we can impose a known value
G
RI
16
O
for a given value of rij , e.g., we could use the assumption that the poten-

tial energy for a charge interaction approaches zero with increasing particle
distance. The alternative (but less likely) approach consists in knowing the
potential energy at a given time, e.g., at the beginning of the observation
and add the inferred increment to the known initial value. Nevertheless,
knowing the absolute value of the potential energy is, in fact, not crucial as
only its derivative determines the dynamics of a particle system. This is also
confirmed in our experiments by the accurate prediction of the acceleration
PT
(see MAEacc ).
Secondly, similar to the pairwise force prediction, PIG’N’PI also provides
RI
a superior performance on satisfying Newton’s action-reaction property, while
again, the baseline model fails to satisfy the underlying Newton’s law. The
SC
performance is quantified by the symmetry of the inter-particle potential
energies (MAEPsymm ). (Fig. 5B, Fig. 6 and Table S3).
Finally, we test the generalization ability of the learning algorithms in a
U
similar way as in the pairwise force case study. We apply the models trained
N
on an eight-particles system to a new particle system comprising 12 particles.
The results (see SI Appendix C; Table S5) show that PIG’N’PI predicts
A
well the pairwise force and potential energy, and outperforms considerably
M
the baseline model (see Fig. 5B and Fig. 6). This demonstrates that the
PIG’N’PI model provides a general model for learning particle interactions.
ED
Case study under more realistic conditions: learning

pairwise interactions for a LJ-argon system
IT
To evaluate the performance of the proposed framework on a more realistic

ED
system with a larger particle interaction system (to evaluate the scalability),
we apply PIG’N’PI on a large Lennard-Jones system.
We adopt the dataset introduced in [35]. This dataset simulates the move-
N
ments of liguid argon atoms governed by the Lennard-Jones (LJ) potential.

U
The LJ potential, which is given by V (r) = 4{(σ/r)1 2 − (σ/r)6 }, is an ex-

tensively used governing law for two non-bonding atoms [36]. The simulation
L
contains 258 particles in a cubic box whose length is 27.27 Å. The simulation
A
is run at 100 K with periodic boundary conditions. The potential well depth
is set to 0.238 kilocalorie / mole, the van der Waals radius σ is set to 3.4
IN
Å, and the interaction cutoff radius for the argon atoms is set to 3σ. The
mass of argon atom is 39.9 dalton. The dataset is run for 10 independent
G
RI
17
O
simulations. Each simulation contains 1000 time steps with randomly ini-

tialized positions and velocities. The position, velocity and acceleration of
all particles are recorded at each time step.
Figure 7 summarizes the learning pipeline. Contrary to the previous case
study where for a small number of particles, a fully connected graph is con-
sidered, in this case study, we construct the graph of neighboring particles
at every time step (Fig. 7). We connect the particles within the defined
interaction cutoff radius while taking the periodic boundary conditions into
PT
consideration. Particles in the LJ-argon system are characterized by their
position, velocity and mass. The charge is not part of the particle proper-
RI
ties, which is different from the particle systems considered in the previous
case study (Sec. and Sec. ). Moreover, we compute the position difference
SC
under the periodic boundary condition and use it as an edge feature. This
edge feature is required because the distance between two particles in this
simulation does not correspond to the Euclidean distance in the real world
U
due to the periodic boundary conditions. The node features and the edge
N
features are then concatenated and are used as the input to the edge part
of PIG’N’PI (Fig. 7C) or the baseline model. Similarly to the previous case
A
study, the learning target is the accelerations of the particles. The pairwise
M
force and pairwise potential energy are then inferred from the intermediate
output of edge part.
We evaluate the performance of PIG’N’PI on inferring pairwise interac-
ED
tions of the LJ-argon particle system with the same performance metrics
as in the previous case study. The results are reported in Fig. 8 and Ta-
IT
ble S6. Because the particles in this dataset have the same mass, we also
test a variant of GN+ such that we assign all nodes with a unique learnable
ED
scalar. We denote this variant as GN+uni . The results confirm the very
good performance of PIG’N’PI as observed in the previous case study. Gen-
erally, GN+uni outperforms the GN+, but PIG’N’PI still surpasses GN+uni
N
and the baseline. On the one hand PIG’N’PI performs better than the base-
U
line, GN+, and GN+uni on the supervised prediction task of predicting the
acceleration (achieving less than half of the MAEacc compared to the base-
L
line and the GN+uni (Fig. 8A)). On the other hand, PIG’N’PI is also able
A
to infer the learn pairwise force correctly. Again, the baseline model is not
able to infer the pairwise force (PIG’N’PI outperforms the baseline by more
IN
than two orders of the magnitude on the MAEef ). Moreover, particle interac-
tions inferred by PIG’N’PI are consistent with Newton’s action-reaction law
G
(MAEFsymm ). The bad performance of the baseline model indicates that the
RI
18
O
Make ( C ) thin

PT
<latexit sha1_base64="E2QUyiOj5dAwJZVizJ7ok2wWxIg=">AAADoXicbZJbb9MwFMe9hssIl3XwyItFN6mToipOehsS0jQegAe0MtFuqKkqx3Fbr3YSJc66YuVj8WEQr/A9cLKC1q5HsvT3OT8fHx8fP+Yslbb9c6diPHj46PHuE/Pps+cv9qr7LwdplCWE9knEo+TSxynlLKR9ySSnl3FCsfA5vfDn74v4xTVNUhaFX+UypiOBpyGbMIKldo2rZ4demcSPbtRixiTN1YHnC+VRifPxFXwH68U20dqCZSCIpN6WezG+OjrIzXpwY8Fgqdf3o3G1Zjfs0uB9gVaiBlbWG+9X3uqUJBM0lITjNB0iO5YjhRPJCKe56WUpjTGZ4ykdahliQdORKovO4aH2BHASJXqFEpbeuycUFmm6FL4mBZazdDNWOLfFhpmcdEeKhXEmaUhuL5pkHMoIFm2EAUsokXypBSYJ07VCMsMJJlI3e+0WX+g3hHRBIiFwGCgP+2k+RCPlcf0xEtYQ9JJC5aZpegGd6J8s36E+L3ssnOfq/MNprpyWa0HU7FjQadm6HniNNulvlPNo8Z9vaRQhC3ZaJe5u4mcJDqf0Lo7abQu2nRJ3NoopR2mA1nDn2IJNN98GOv9AR6d0i9XZyrkrroWKfJpDXXsr2FyBXd0B1NHgMcpNPW1oc7bui4HTQO2G+6VZOzldzd0ueA3egDpAoANOwEfQA31AwA/wC/wGf4ya8cnoGee3aGVndeYVWDNj+BcK7R/Y</latexit>
(dx, dy, dz) PIG’N’PI

F̂ij or P̂ij
<latexit sha1_base64="mX0GK8oKgv0Cuxpl9TldFqBnwwo=">AAAD4XicbZJLbxMxEMedhkdZHm3h2ItFEimVVtV68ywSUlUE5YIIiKZF2Wjl9TqN231p12kbLH8ATiCufDI+CHdmtws0aSyN9J+Zn8dje7wkEJm0rF+Vteqdu/furz8wHj56/GRjc+vpMItnKeNHLA7i9MSjGQ9ExI+kkAE/SVJOQy/gx975qzx/fMHTTMTRJzlP+Dikp5GYCEYlhNzNbw2nKOLFV+pyKiTXqu54oXK4pNo9wy9xM3dT0CYuEn4swS380D3bqWujgZv+lYn9OdiXHXCdKZXqULvDpsMAL0L/a+aenIJ2XxtFOKffQEUlzrS7WbN2rWLh24KUoobKNXC31l5AS2wW8kiygGbZiFiJHCuaSsECrg1nlvGEsnN6ykcgIxrybKyKS2vcgIiPJ3EKFklcRG/uUDTMsnnoARlSOc2Wc3lwVW40k5P+WIkomUkeseuDJrMAyxjn34B9kXImgzkIylIBvWI2pSllEj5r4RQvhDtE/JLFYUgjXznUy/SIjJUTwMdKXCPYSXOlDcNwfD6BSSjuod7NByI61+rj4YFWdqdlYtLumdjuWNAPviDL9GceBPHlP74DKCEm7nUKvLWMv09pdMpv4qTbNXHXLnB7qZliFIdkAbf3TNxu6VWg/Re0oWQrt95KrlVyHZLXA470rZVguwT78AKkB+Ae0QZMG1merdtiaO+S7m7rQ7u2f1DO3TraRs9RExHUQ/voLRqgI8TQ78p2pV5pVFn1a/V79cc1ulYp9zxDC6v68w/ZrTU3</latexit> <latexit sha1_base64="EZrrP51TeKEQDsSvSAjymXZLQoo=">AAAD3HicbZJbb9MwFMfdlcsItw4kXnjAopvUSdEUJ71sSEjTEBoviIJYN9SUyHHc1m0uVeJsK5bfeALxynfjW/AROMkKbF0tHel/zvn5+Ng+/iwUmbSsX5W16o2bt26v3zHu3rv/4GFt41EvS/KU8SOWhEl64tOMhyLmR1LIkJ/MUk4jP+TH/vRVkT8+5WkmkvijnM/4IKKjWAwFoxJCXu3bllsW8ZNzdTYWkmu16fqRcrmk2pvgl7hRuCloE5eJIJHgln7kTbY3tbGFG8G5iYM52JdtcN0xlepQe72GywAvQ/9rFp4cg/ZeGyXZ1Z4SE+3V6taOVS58XZCFqKPF6nobay+gG5ZHPJYspFnWJ9ZMDhRNpWAh14abZ3xG2ZSOeB9kTCOeDVR5X423IBLgYZKCxRKX0cs7FI2ybB75QEZUjrPlXBFclevncrg7UCKe5ZLH7OKgYR5imeDiB3AgUs5kOAdBWSqgV8zGNKVMwj9dOcWP4A4xP2NJFNE4UC71M90nA+WG8KcS1wl200JpwzDcgA9hCMp7qLfzroinWn04PNDKbjkmJs2Oie2WBf3gU7JMf+JhmJz941uAEmLiTqvEnWX8XUrjEb+Mk3bbxG27xO2lZsop7JEruL1n4qajV4H2X9CGkk5hnZWcs+BapKgHHNm1VoLNBbgLL0A6AO4RbcC0keXZui569g5p7zjvm/X9g8XcraOn6DlqIII6aB+9QV10hBj6XXlSeVbB1c/Vr9Xv1R8X6FplsecxurKqP/8A/Ooy7A==</latexit>
{
{
RI
ĜE (⌘i,⌘j(dx,
<latexit sha1_base64="oFopvzutyBClJQUnuvO6gZe8PK4=">AAADrXicbZLdbtMwFMe9hY8Rvjq45Maiq9RJUYmTfg0JaRpC4wZREO2GmhI5jttmtZMocdYVKw/HY/AE3MIb4GQBrV0tWfqfc34+9jk+XsyCVJjmz51d7c7de/f3HugPHz1+8rS2/2yURllC6JBELErOPZxSFoR0KALB6HmcUMw9Rs+8xdsifnZJkzSIwi9iFdMJx7MwmAYEC+Vya+OGUybxoiu5nAeC5vLA8bh0qMC5ewHfwGZhJkobsAz4kVBmaXP34vAg1xuw6V8Z0F+p/f1Qd+ZYyNPcfefW6mbLLBe8LVAl6qBaA3d/97VKTzJOQ0EYTtMxMmMxkTgRAWE0150spTEmCzyjYyVDzGk6kWUBOWwojw+nUaJ2KGDpvXlCYp6mK+4pkmMxTzdjhXNbbJyJaX8igzDOBA3J9UXTjEERwaKl0A8SSgRbKYFJEqi3QjLHCSZCNX7tFo+rGkK6JBHnOPSlg700H6OJdJj6JAHrCDpJoXJd1x2fTtWvlnXID6tBEC5y+fn0JJdWxzYgavcMaHVM9R54iTbpr5SxaPmf7ygUIQP2OiVub+IfExzO6E0cdbsG7Folbm08phyrEVrDrSMDtu18G2j9Ay2V0i52bytnV1wHFfkUh/rmVrBdgX3VAdRT4BHKdTVtaHO2bouR1ULdlv2pXT8+qeZuD7wAL0ETINADx+A9GIAhIOAH+AV+gz/aK22oOdq3a3R3pzrzHKwtbfYXWmckGw==</latexit>
,(dx,,dy,,dz)
<latexit sha1_base64="EPdK3Fs5cQdsnz3hBCFe3bJmwIo=">AAAD0XicbZJZa9tAEMfXVo9U6eH0eOrLUsfggDBa+UyhEFJK+lLqHnZSLCNWq7WtWIeR1nHURbT0tZ+wH6DfoyNFbWPHCwP/mfnt7Ozu2AvPjYWu/yqVlVu379zduafu3n/w8FFl7/EwDpcR4wMWemF0ZtOYe27AB8IVHj9bRJz6tsdP7fnrLH96waPYDYPPIlnwsU+ngTtxGRUQsirfamZexA4v5WrmCp7KfdP2pckFTa1z/ArXMzcCreE84YQC3Nz3rfOD/VSt4bpzqWEnAft6AK45o0KepNawbjLA89D/mpknZqCtN2rdqlT1hp4vfFOQQlRRsfrWXvkltMCWPg8E82gcj4i+EGNJI+Eyj6equYz5grI5nfIRyID6PB7L/JIprkHEwZMwAgsEzqPXd0jqx3Hi20D6VMzizVwW3JYbLcWkN5ZusFgKHrCrgyZLD4sQZ8+OHTfiTHgJCMoiF3rFbEYjygR8ztoptg93CPiKhb5PA0ea1I7TERlL04OPFLhKsBllKlVV1XT4BH4+v4d8l/TdYJ7KjyfHqTTaTQ2TVlfDRluHfvAF2aS/cM8LV//4NqCEaLjbzvHmJv4+osGUX8dJp6PhjpHjxkYz+egNyRpuHGq41Uy3gcZf0ICSzcy6W7lmwbVJVg840tO3gq0C7MELkC6AhyRVYdrI5mzdFEOjQTqN5odW9ei4mLsd9By9QHVEUBcdobeojwaIod+l3dLT0jPlk5Io35UfV2i5VOx5gtaW8vMPCkctmw==</latexit> <latexit sha1_base64="rGQ8TQl0ClewfPOQZ6GlMyZdl6Q=">AAAD0XicbZJZa9tAEMfXUY9U6eH0eOrLUsdggzBayVcKhZBS0pdS97CTYhmxWq1txTqMtI6jLqKlr/2E/QD9Hh0pbhs7Xhj4z8xvZ2d3x5n7XiJ0/VdpR7l1+87d3Xvq3v0HDx+V9x8PkmgRM95nkR/FZw5NuO+FvC884fOzecxp4Pj81Jm9zvOnFzxOvCj8LNI5HwV0Enpjj1EBIbv8rWoVRZzoUi6nnuCZPLCcQFpc0Mw+x69wLXdj0BouEm4kwC38wD6vH2RqFdfcSw27KdjXOrjWlAp5ktmDmsUAL0L/a+aemIK236h1u1zRG3qx8E1BVqKCVqtn7++8hBbYIuChYD5NkiHR52IkaSw85vNMtRYJn1M2oxM+BBnSgCcjWVwyw1WIuHgcxWChwEX0+g5JgyRJAwfIgIppspnLg9tyw4UYd0fSC+cLwUN2ddB44WMR4fzZsevFnAk/BUFZ7EGvmE1pTJmAz1k7xQngDiFfsigIaOhKizpJNiQjafnwkQJXCLbiXGWqqlouH8PPF/eQ79KeF84y+fHkOJNGy9QwaXY0bLR06AdfkE36C/f9aPmPbwFKiIY7rQI3N/H3MQ0n/DpO2m0Nt40CNzaaKUZvQNZw41DDTTPbBhp/QQNKmrl1tnLmimuRvB5wpKtvBZsrsAsvQDoAHpJMhWkjm7N1UwyMBmk3zA/NytHxau520XP0AtUQQR10hN6iHuojhn6X9kpPS8+UT0qqfFd+XKE7pdWeJ2htKT//AA00LZw=</latexit>
; ✓E )
<latexit sha1_base64="yauD12/stVKerLKZ0e+dwcy7l2A=">AAADk3icbZLdbtMwFMe9hY8RvjoQV9xYdEhDiqo4adIWhDQKEtxMFES7oaaqHNdtrcZOlDgrlZXn4Wm4BfE2OFmZ1q6WLP3POT8fHx+fMIlYJm37796+cev2nbsH98z7Dx4+elw7fDLI4jwltE/iKE7PQ5zRiAnal0xG9DxJKeZhRM/CxfsyfnZB04zF4ptcJXTE8UywKSNYate49i6ocoTxD7WcM0kLdRSEXAVU4mLM4Ft4XJqp1hasApNYarOy+Zi9OirGtbrdcJw28tvQbviO1+zYEDXsal2JOliv3vhw/7XOQnJOhSQRzrIhshM5UjiVjES0MIM8owkmCzyjQy0F5jQbqarOAr7UngmcxqneQsLKe/2EwjzLVjzUJMdynm3HSueu2DCX0/ZIMZHkkgpyedE0j6CMYdk4OGEpJTJaaYFJynStkMxxionU7d24JeT6DYIuScw5FhMV4DArhmikgkh/hYR1BIO0VIVpmsGETvXfVe9Qp6seE4tCff3YLZTjuRZEzZYFHc/W9cALtE1/p1EUL694T6MIWbDlVbi7jX9OsZjR6zjyfQv6ToU7W8VUwzNAG7jTsWDTLXaBzn/Q0Sndcrd2cu6a81CZT3Oobe8Em2uwrTuAWhrsoMLU03Zjtm6KgdNAfsP90qyfdNdzdwCegxfgGCDQAifgE+iBPiDgJ/gFfoM/xjPjjdE1Plyi+3vrM0/BxjJO/wFSZxvP</latexit>
<latexit sha1_base64="E2QUyiOj5dAwJZVizJ7ok2wWxIg=">AAADoXicbZJbb9MwFMe9hssIl3XwyItFN6mToipOehsS0jQegAe0MtFuqKkqx3Fbr3YSJc66YuVj8WEQr/A9cLKC1q5HsvT3OT8fHx8fP+Yslbb9c6diPHj46PHuE/Pps+cv9qr7LwdplCWE9knEo+TSxynlLKR9ySSnl3FCsfA5vfDn74v4xTVNUhaFX+UypiOBpyGbMIKldo2rZ4demcSPbtRixiTN1YHnC+VRifPxFXwH68U20dqCZSCIpN6WezG+OjrIzXpwY8Fgqdf3o3G1Zjfs0uB9gVaiBlbWG+9X3uqUJBM0lITjNB0iO5YjhRPJCKe56WUpjTGZ4ykdahliQdORKovO4aH2BHASJXqFEpbeuycUFmm6FL4mBZazdDNWOLfFhpmcdEeKhXEmaUhuL5pkHMoIFm2EAUsokXypBSYJ07VCMsMJJlI3e+0WX+g3hHRBIiFwGCgP+2k+RCPlcf0xEtYQ9JJC5aZpegGd6J8s36E+L3ssnOfq/MNprpyWa0HU7FjQadm6HniNNulvlPNo8Z9vaRQhC3ZaJe5u4mcJDqf0Lo7abQu2nRJ3NoopR2mA1nDn2IJNN98GOv9AR6d0i9XZyrkrroWKfJpDXXsr2FyBXd0B1NHgMcpNPW1oc7bui4HTQO2G+6VZOzldzd0ueA3egDpAoANOwEfQA31AwA/wC/wGf4ya8cnoGee3aGVndeYVWDNj+BcK7R/Y</latexit> <latexit sha1_base64="E2QUyiOj5dAwJZVizJ7ok2wWxIg=">AAADoXicbZJbb9MwFMe9hssIl3XwyItFN6mToipOehsS0jQegAe0MtFuqKkqx3Fbr3YSJc66YuVj8WEQr/A9cLKC1q5HsvT3OT8fHx8fP+Yslbb9c6diPHj46PHuE/Pps+cv9qr7LwdplCWE9knEo+TSxynlLKR9ySSnl3FCsfA5vfDn74v4xTVNUhaFX+UypiOBpyGbMIKldo2rZ4demcSPbtRixiTN1YHnC+VRifPxFXwH68U20dqCZSCIpN6WezG+OjrIzXpwY8Fgqdf3o3G1Zjfs0uB9gVaiBlbWG+9X3uqUJBM0lITjNB0iO5YjhRPJCKe56WUpjTGZ4ykdahliQdORKovO4aH2BHASJXqFEpbeuycUFmm6FL4mBZazdDNWOLfFhpmcdEeKhXEmaUhuL5pkHMoIFm2EAUsokXypBSYJ07VCMsMJJlI3e+0WX+g3hHRBIiFwGCgP+2k+RCPlcf0xEtYQ9JJC5aZpegGd6J8s36E+L3ssnOfq/MNprpyWa0HU7FjQadm6HniNNulvlPNo8Z9vaRQhC3ZaJe5u4mcJDqf0Lo7abQu2nRJ3NoopR2mA1nDn2IJNN98GOv9AR6d0i9XZyrkrroWKfJpDXXsr2FyBXd0B1NHgMcpNPW1oc7bui4HTQO2G+6VZOzldzd0ueA3egDpAoANOwEfQA31AwA/wC/wGf4ya8cnoGee3aGVndeYVWDNj+BcK7R/Y</latexit>
(dx, dy, dz)

<latexit sha1_base64="E2QUyiOj5dAwJZVizJ7ok2wWxIg=">AAADoXicbZJbb9MwFMe9hssIl3XwyItFN6mToipOehsS0jQegAe0MtFuqKkqx3Fbr3YSJc66YuVj8WEQr/A9cLKC1q5HsvT3OT8fHx8fP+Yslbb9c6diPHj46PHuE/Pps+cv9qr7LwdplCWE9knEo+TSxynlLKR9ySSnl3FCsfA5vfDn74v4xTVNUhaFX+UypiOBpyGbMIKldo2rZ4demcSPbtRixiTN1YHnC+VRifPxFXwH68U20dqCZSCIpN6WezG+OjrIzXpwY8Fgqdf3o3G1Zjfs0uB9gVaiBlbWG+9X3uqUJBM0lITjNB0iO5YjhRPJCKe56WUpjTGZ4ykdahliQdORKovO4aH2BHASJXqFEpbeuycUFmm6FL4mBZazdDNWOLfFhpmcdEeKhXEmaUhuL5pkHMoIFm2EAUsokXypBSYJ07VCMsMJJlI3e+0WX+g3hHRBIiFwGCgP+2k+RCPlcf0xEtYQ9JJC5aZpegGd6J8s36E+L3ssnOfq/MNprpyWa0HU7FjQadm6HniNNulvlPNo8Z9vaRQhC3ZaJe5u4mcJDqf0Lo7abQu2nRJ3NoopR2mA1nDn2IJNN98GOv9AR6d0i9XZyrkrroWKfJpDXXsr2FyBXd0B1NHgMcpNPW1oc7bui4HTQO2G+6VZOzldzd0ueA3egDpAoANOwEfQA31AwA/wC/wGf4ya8cnoGee3aGVndeYVWDNj+BcK7R/Y</latexit>
⌘i = (ri , ṙi , mi ) dy, dz)

<latexit sha1_base64="KhRuBIbN5vvi7W5m/1txprVAc2k=">AAADxnicbZJta9swEMeVeA+t95RuL/dGLA2kYILlPHYwKB2lfTOWjSXtiIORZSVxY8tGlptmwrCvuX2aya63NWkEgv/d/XS6k86NAz8RpvmrUtUePX7ydG9ff/b8xctXtYPX4yRKOaEjEgURv3JxQgOf0ZHwRUCvYk5x6Ab00l1+zOOXN5QnfsS+iXVMpyGeM3/mEyyUy6mJhl0kcaNbuVr4gmby0HZDaVOBM+cafoDN3ORKG7AIeJFQZmGHzvXRYaY3YNO7NaC3VvvHkTLtBRbyPHPOcv0/mW6LhVLOmVOrmy2zWPChQKWog3INnYPqe3UvSUPKBAlwkkyQGYupxFz4JKCZbqcJjTFZ4jmdKMlwSJOpLDrLYEN5PDiLuNpMwMJ7/4TEYZKsQ1eRIRaLZDuWO3fFJqmYDabSZ3EqKCN3F83SAIoI5m8NPZ9TIoK1EphwX9UKyQJzTIT6kY1b3FD1wOiKRGGImSdt7CbZBE2lHajfE7COoM1zlem6bnt0pr676EN+Wg99tszk1/PTTFrdtgFRp29Aq2uqeuAN2qa/0yCIVv/4rkIRMmC/W+Dtbfwzx2xO7+Oo1zNgzypwa6uYYt7GaAO3jg3YaWe7QOsvaKmU7Xz3d3LtkuuiPJ/i0MDcCXZKcKBeAPUVeIwyXU0b2p6th2JstVCv1f7SqZ+clnO3B96Cd6AJEOiDE3ABhmAECPhdAZX9iq5daExLtdUdWq2UZ96AjaX9/AMqAypO</latexit>
<latexit sha1_base64="n5w0cc9KiyQ5e4Who5k9/1F6WzY=">AAADu3icdZJba9swFMfVepfOu7TdHvcilgZaMMGS4zgZFErH6F7GsrFeRhyCLCuJGvmCLbfNhD/jnvdB9rpNdtPRhExw4H/O+ekcXU6QCp5L2/65sWk8ePjo8dYT8+mz5y+2d3ZfnuVJkVF2ShORZBcByZngMTuVXAp2kWaMRIFg58HsXZU/v2JZzpP4q5ynbBiRSczHnBKpQ6Md3vTrIkFyo66nXLJS7flBpHwmSTm6hIdwv3IzrS1YJ8JEarf2o9HlwV5pNuF+eGPBcK7t+4F2/SmR6qQcvTfvlRrtNOyW0+04vR60W10Xtz1HC+xgjF2IWna9GmCx+qPdzbe6GS0iFksqSJ4PkJ3KoSKZ5FSw0vSLnKWEzsiEDbSMScTyoaqvU8KmjoRwnGTaYgnr6P0dikR5Po8CTUZETvPVXBVclxsUctwdKh6nhWQxvW00LgSUCaweGIY8Y1SKuRaEZlyfFdIpyQiV+huWugSRvkPMrmkSRSQOlU+CvBygofKF/jIJGwj6WaVK0zT9kI31H9f3UB/nfR7PSvXl5LhU2HUsiNqeBbFr6/PAK7RKf2NCJNf/eFejCFnQc2vcWcU/ZSSesPs46nQs2ME1jlcOUw/ZGVrCcc+CbadcB+I7EOuSTmXeWs5ZcC6q6mkOde21YHsBdvULIE+DPVSaetruRgr+X5zhFuq0nM/txtHxYu62wGvwBuwDBDxwBD6APjgFFPwAv8Bv8Mc4NKhxaYhbdHNjsecVWFpG8RdbgynL</latexit>
<latexit sha1_base64="v7WfzoJYIQipjoxLWuUuvbY/qiw=">AAADu3icdZLdb9MwEMC9lY8RvjZ45MWim9RJURUnTT+QJk1DaLwgBqLdUFNVjuO2Xp2kSpx1xcrfyDN/CK/AJetgrcpJJ93Hz3dn+/yZFKmyrB9b25V79x883HlkPH7y9Nnz3b0XvTTOEsa7LJZxcuHTlEsR8a4SSvKLWcJp6Et+7k/fFvnzK56kIo6+qMWMD0I6jsRIMKogNNwVB15ZxI+v9XwiFM/1vueH2uOK5sNLfIRrhZuAbeIyEcQK3NIPh5eH+7lxgGvBtYmDBei3Q3C9CVX6NB++M/6VEsPdqlUn7Y7ddLFVdxt2y7HA6IA4bUzqVilVtJSz4d72G2jGspBHikmapn1izdRA00QJJnlueFnKZ5RN6Zj3wYxoyNOBLq+T4wOIBHgUJ6CRwmX07glNwzRdhD6QIVWTdD1XBDfl+pkatQdaRLNM8YjdNBplEqsYFw+MA5FwpuQCDMoSAbNiNqEJZQq+YaWLH8IdIj5ncRjSKNAe9dO8Twbak/BlClcJ9pLCyg3D8AI+gj8u76E/LM5ENM3159OTXNuuY2LSaJnYdi2YB1+RdforlzKe/+VdQAkxccstcWcd/5jQaMzv4qTZNHHTLnF7bZhyyXpkBbc7Jm44+SbQvgVtKOkU2trIOUvOJUU94Ejb2gg2lmAbXoC0AOyQ3IBtu10p/H+jZ9dJs+58alSPT5Z7t4NeodeohghqoWP0Hp2hLmLoO/qJfqHflaMKq1xW5A26vbU88xKtSCX7A4UYKdo=</latexit>
<latexit sha1_base64="VaK83wsXI6rGVSQQPIVlq5+Q8K8=">AAAD0XicbZJbb9MwFMfdlcvIgHVcnnix6Cp1UlTFSW8DIU1DaLwgyqXdUFNFjuO2WXOpEmddsSIQr3xCPgDfg5MswNrV0pH+55yfj4/tY889Nxaa9qu0Vb51+87d7XvKzv0HD3cre48GcZhEjPdZ6IXRmU1j7rkB7wtXePxsHnHq2x4/tWevs/zpBY9iNww+i+Wcj3w6Cdyxy6iAkFX5VjPzInZ4KRdTV/BU7pu2L00uaGqd41e4nrkRaBXnCScU4Oa+b50f7KdKDdedSxU7S7CvB+CaUyrkSWoN6iYDPA/9r5l5YgraeqO8tCpVraHlC98UpBBVVKyetbf1Alpgic8DwTwax0OizcVI0ki4zOOpYiYxn1M2oxM+BBlQn8cjmV8yxTWIOHgcRmCBwHn0+g5J/The+jaQPhXTeD2XBTflhokYd0fSDeaJ4AG7OmiceFiEOHt27LgRZ8JbgqAscqFXzKY0okzA56ycYvtwh4AvWOj7NHCkSe04HZKRND34SIGrBJtRplJFUUyHj+Hn83vId8ueG8xS+fHkOJV6y1AxaXZUrLc06AdfkHX6C/e8cPGPbwFKiIo7rRw31vH3EQ0m/DpO2m0Vt/Uc19eayUdvQFZw/VDFTSPdBOp/QR1KGpl1NnJGwbVIVg840tU2gs0C7MILkA6AhyRVYNrI+mzdFAO9QdoN40OzenRczN02eoaeozoiqIOO0FvUQ33E0O/STulJ6Wn5U3lZ/l7+cYVulYo9j9HKKv/8A0HeLa4=</latexit>
<latexit sha1_base64="W2huJxsLzQo4LCeoPBxZ0TMVL24=">AAAEOXicnZLNb9MwGMbTlY9RvjaQuHCx2CYNKZTYafqBhDSNA1wmBmIfqKkqx3XXrLFT2c66yvjA38UfghAnTiCuiDtO2k1bVThgydLr1z8/j5080SiJpfK8L6Wl8pWr164v36jcvHX7zt2V1Xv7Ms0EoXskTVJxGGFJk5jTPRWrhB6OBMUsSuhBNHyR7x+cUCHjlL9TkxHtMHzE435MsLKt7sr3DRAWKlF6qseDWFGjQ5EltP3Eq8KAsI72qp7nIcaMrQLbMWAdhBHT4QArHfZ6qdLCGNM9BqG7Hip6qgrBczUV8wlwjan8txFVOJd/vpmvhC3daX9mna9Z9/jxv8y7K2te1W/W/VYLeNVmgGoN3xbIRwgFAObOdqxtPX0gP37++nu3u7r0zOqTjFGuSIKlbENvpDoaCxWThNrXZJKOMBniI9q2JceMyo4u3A3YsJ0e6KfCTq5A0b14QmMm5YRFlmRYDeT8Xt5ctNfOVL/Z0TEfZYpyMjXqZwlQKch/L+jFghKVTGyBiYjtXQEZYIGJsiG45BIx+wZOxyRlDPOeDnEkTRt2dJjYwCiwBkEo8spUKpWwR/s2YdNvuzPZjfnQ6Lcvt41Gge8CWGu4AAWevQ84gfP0e5ok6ficDywKoQsaQYH78/hrgfkRvYjDet0FdVTgaO4yRcT34SUctVxQ880iEJ2ByEr6+Wws5PwZF8Bcz3Kw6S0EazOwab8AbFiwBU3Fpu0sUuDvxT6qwnrVf2Njt+1Mx7Lz0HnkbDrQaThbzitn19lzSGmnJEsfSqb8qfyt/KP8c4oulWZn7juXRvnXH/I8XoY=</latexit>
<latexit sha1_base64="GG/3de7zd6HXYH4Wz+JXZ4W7uZg=">AAAD0XicbZJZa9tAEMfXVo9U6eH0eOrLUsfggDBa+UyhEFJK+lLqHnZSLCNWq7WtWIeR1nHURbT0tZ+wH6DfoyNFbWPHCwP/mfnt7Ozu2AvPjYWu/yqVlVu379zduafu3n/w8FFl7/EwDpcR4wMWemF0ZtOYe27AB8IVHj9bRJz6tsdP7fnrLH96waPYDYPPIlnwsU+ngTtxGRUQsirfamZexA4v5WrmCp7KfdP2pckFTa1z/ArXMzcCreE84YQC3Nz3rfOD/VSt4bpzqWEnAft6AK45o0KepNawbjLA89D/mpknZqCtN6pmVap6Q88XvilIIaqoWH1rr/wSWmBLnweCeTSOR0RfiLGkkXCZx1PVXMZ8QdmcTvkIZEB9Ho9lfskU1yDi4EkYgQUC59HrOyT14zjxbSB9KmbxZi4LbsuNlmLSG0s3WCwFD9jVQZOlh0WIs2fHjhtxJrwEBGWRC71iNqMRZQI+Z+0U24c7BHzFQt+ngSNNasfpiIyl6cFHClwl2Iwylaqqajp8Aj+f30O+S/puME/lx5PjVBrtpoZJq6tho61DP/iCbNJfuOeFq398G1BCNNxt53hzE38f0WDKr+Ok09Fwx8hxY6OZfPSGZA03DjXcaqbbQOMvaEDJZmbdrVyz4Nokqwcc6elbwVYB9uAFSBfAQ5KqMG1kc7ZuiqHRIJ1G80OrenRczN0Oeo5eoDoiqIuO0FvURwPE0O/Sbulp6ZnySUmU78qPK7RcKvY8QWtL+fkHFfstnw==</latexit> <latexit sha1_base64="GG/3de7zd6HXYH4Wz+JXZ4W7uZg=">AAAD0XicbZJZa9tAEMfXVo9U6eH0eOrLUsfggDBa+UyhEFJK+lLqHnZSLCNWq7WtWIeR1nHURbT0tZ+wH6DfoyNFbWPHCwP/mfnt7Ozu2AvPjYWu/yqVlVu379zduafu3n/w8FFl7/EwDpcR4wMWemF0ZtOYe27AB8IVHj9bRJz6tsdP7fnrLH96waPYDYPPIlnwsU+ngTtxGRUQsirfamZexA4v5WrmCp7KfdP2pckFTa1z/ArXMzcCreE84YQC3Nz3rfOD/VSt4bpzqWEnAft6AK45o0KepNawbjLA89D/mpknZqCtN6pmVap6Q88XvilIIaqoWH1rr/wSWmBLnweCeTSOR0RfiLGkkXCZx1PVXMZ8QdmcTvkIZEB9Ho9lfskU1yDi4EkYgQUC59HrOyT14zjxbSB9KmbxZi4LbsuNlmLSG0s3WCwFD9jVQZOlh0WIs2fHjhtxJrwEBGWRC71iNqMRZQI+Z+0U24c7BHzFQt+ngSNNasfpiIyl6cFHClwl2Iwylaqqajp8Aj+f30O+S/puME/lx5PjVBrtpoZJq6tho61DP/iCbNJfuOeFq398G1BCNNxt53hzE38f0WDKr+Ok09Fwx8hxY6OZfPSGZA03DjXcaqbbQOMvaEDJZmbdrVyz4Nokqwcc6elbwVYB9uAFSBfAQ5KqMG1kc7ZuiqHRIJ1G80OrenRczN0Oeo5eoDoiqIuO0FvURwPE0O/Sbulp6ZnySUmU78qPK7RcKvY8QWtL+fkHFfstnw==</latexit>
<latexit sha1_base64="GG/3de7zd6HXYH4Wz+JXZ4W7uZg=">AAAD0XicbZJZa9tAEMfXVo9U6eH0eOrLUsfggDBa+UyhEFJK+lLqHnZSLCNWq7WtWIeR1nHURbT0tZ+wH6DfoyNFbWPHCwP/mfnt7Ozu2AvPjYWu/yqVlVu379zduafu3n/w8FFl7/EwDpcR4wMWemF0ZtOYe27AB8IVHj9bRJz6tsdP7fnrLH96waPYDYPPIlnwsU+ngTtxGRUQsirfamZexA4v5WrmCp7KfdP2pckFTa1z/ArXMzcCreE84YQC3Nz3rfOD/VSt4bpzqWEnAft6AK45o0KepNawbjLA89D/mpknZqCtN6pmVap6Q88XvilIIaqoWH1rr/wSWmBLnweCeTSOR0RfiLGkkXCZx1PVXMZ8QdmcTvkIZEB9Ho9lfskU1yDi4EkYgQUC59HrOyT14zjxbSB9KmbxZi4LbsuNlmLSG0s3WCwFD9jVQZOlh0WIs2fHjhtxJrwEBGWRC71iNqMRZQI+Z+0U24c7BHzFQt+ngSNNasfpiIyl6cFHClwl2Iwylaqqajp8Aj+f30O+S/puME/lx5PjVBrtpoZJq6tho61DP/iCbNJfuOeFq398G1BCNNxt53hzE38f0WDKr+Ok09Fwx8hxY6OZfPSGZA03DjXcaqbbQOMvaEDJZmbdrVyz4Nokqwcc6elbwVYB9uAFSBfAQ5KqMG1kc7ZuiqHRIJ1G80OrenRczN0Oeo5eoDoiqIuO0FvURwPE0O/Sbulp6ZnySUmU78qPK7RcKvY8QWtL+fkHFfstnw==</latexit> <latexit sha1_base64="GG/3de7zd6HXYH4Wz+JXZ4W7uZg=">AAAD0XicbZJZa9tAEMfXVo9U6eH0eOrLUsfggDBa+UyhEFJK+lLqHnZSLCNWq7WtWIeR1nHURbT0tZ+wH6DfoyNFbWPHCwP/mfnt7Ozu2AvPjYWu/yqVlVu379zduafu3n/w8FFl7/EwDpcR4wMWemF0ZtOYe27AB8IVHj9bRJz6tsdP7fnrLH96waPYDYPPIlnwsU+ngTtxGRUQsirfamZexA4v5WrmCp7KfdP2pckFTa1z/ArXMzcCreE84YQC3Nz3rfOD/VSt4bpzqWEnAft6AK45o0KepNawbjLA89D/mpknZqCtN6pmVap6Q88XvilIIaqoWH1rr/wSWmBLnweCeTSOR0RfiLGkkXCZx1PVXMZ8QdmcTvkIZEB9Ho9lfskU1yDi4EkYgQUC59HrOyT14zjxbSB9KmbxZi4LbsuNlmLSG0s3WCwFD9jVQZOlh0WIs2fHjhtxJrwEBGWRC71iNqMRZQI+Z+0U24c7BHzFQt+ngSNNasfpiIyl6cFHClwl2Iwylaqqajp8Aj+f30O+S/puME/lx5PjVBrtpoZJq6tho61DP/iCbNJfuOeFq398G1BCNNxt53hzE38f0WDKr+Ok09Fwx8hxY6OZfPSGZA03DjXcaqbbQOMvaEDJZmbdrVyz4Nokqwcc6elbwVYB9uAFSBfAQ5KqMG1kc7ZuiqHRIJ1G80OrenRczN0Oeo5eoDoiqIuO0FvURwPE0O/Sbulp6ZnySUmU78qPK7RcKvY8QWtL+fkHFfstnw==</latexit>
⌘j = (rj , ṙj , mj ),
Edge part Implicitly learnt
SC
<latexit sha1_base64="lscdE1l965csPbs8xd9/zVKqMgk=">AAADzXicbZLfb9MwEMfdFdgIv1p45MWiq9RKURUn/TUkpGkIjRe0gmg31FSR47ht1vyoEmddMeGVv5FH/hMuWUBtV0uWvnf38fnOPnvpubHQtN+lg/KDh48Ojx4rT54+e/6iUn05isMkYnzIQi+Mrmwac88N+FC4wuNXy4hT3/b4pb14n8Uvb3gUu2HwVayXfOLTWeBOXUYFuKyKrJt5Eju8lau5K3gqj03blyYXNLWu8TvcyMwItIrzgBMKMHPbt66bx6lSxw3nVsXOGvb3pmLOqZDnqTVqmAzgJsQ3MmaWmIO2PliVmtbS8oXvC1KIGirWwKoevIXrWeLzQDCPxvGYaEsxkTQSLvN4qphJzJeULeiMj0EG1OfxROYNprgOHgdPwwh2IHDu3TwhqR/Ha98G0qdiHu/GMue+2DgR0/5EusEyETxgdxdNEw+LEGdPjh034kx4axCURS7UitmcRpQJ+JitW2wfegj4ioW+TwNHmtSO0zGZSNODTxS4RrAZZSpVFMV0+BR+Pe9DfloP3GCRyi/nZ6nUO4aKSbunYr2jQT34huzS37jnhav/fAdQQlTc6+S4sYtfRDSY8U2cdLsq7uo5ru8Uk4/diGzh+omK20a6D9T/gTqkNLLd28sZBdchWT7gSF/bC7YLsA8vQHoAnpBUgWkju7N1X4z0Fum2jM/t2ulZMXdH6DV6gxqIoB46RR/RAA0RQ39Kh6VKqVq+KCflH+Wfd+hBqTjzCm2t8q+/diks/A==</latexit>
<latexit sha1_base64="5IAfEv4JtOTE/C9TXetqJAWpkXI=">AAADunicdZJLbxMxEMfdhkcJrxYkLlwsWiQOS1h7s3kgDlU5wAVREH2gbBR5vU5jsvZGXm/TyPKBL8gVIb4GHEF4N23VRGEkS3/N/DwzHk88SXmuff/H2nrt2vUbNzdu1W/fuXvv/ubWg8M8KxRlBzRLM3Uck5ylXLIDzXXKjieKERGn7Cgevy7jR6dM5TyTn/RswvqCnEg+5JRo5xpsjqIqR5ydmemIa2ZNpIqU9Z77DRRS0Td+w/d9LIR1KnQeC3dgFAsTjYg2UZJk2ihr7eALjLydSLMzXSW8zKa5nEHPAZvbfiPotIJuF/qNToib7cAJHGCMQ4jKKs62d188yr9+//l7f7C1/jJKMloIJjVNSZ73kD/RfUOU5jRlth4VOZsQOiYnrOekJILlfVNVt/Cp8yRwmCl3pIaV9+oNQ0Sez0TsSEH0KF+Olc5VsV6hh52+4XJSaCbpvNCwSKHOYDlfmHDFqE5nThCquOsV0hFRhGr3CwtVYuHeINmUZkIQmZiIxLntob6JUvdjGm4jGKlS2Xq9HiVs6L54Ptt3s30ux9Z8fLNnDQ4DD6Jm24M49F0/8BQt059ZmmbTSz50KEIebIcVHizj7xWRJ+wqjlotD7ZwheOlZqodO0QLOO56sBnYVSC+ALFLGZSnvZILzrkQlfkchzr+SrB5DnbcBFDbgV1k627bLlYK/l8c4gZqNYIPbu32wNw2wGPwBDwDCLTBLngL9sEBoOAb+AX+gL+1V7W4xmvjObq+dn7nIViwmv4H9kkvvw==</latexit>
ˆ
<latexit sha1_base64="3yR/N24GaG0MNUEeBDUhbAUobnc=">AAAECHicdZLdbtMwFMfTlY8RPrbBJVxYdJM6KaripG0aJKSpCA0JEAWxbqipIsdxV6/5qBJ3W7H8AjwNVyBueQvehpOsg63qLFn6n3N+PufYPsE04rkwzT+Vteqt23furt/T7z94+Ghjc+txP09nGWUHNI3S7CggOYt4wg4EFxE7mmaMxEHEDoPJqyJ+eMqynKfJZzGfsmFMjhM+4pQIcPmbP3a8MkmQnsuzMRdMyW0viKXHBFH+CXqJ6oWZgTZQGQhTAWZpx/7J7rbSd1A9PDdQOIf9dRdMb0yE3Fd+v+5RwEvX/5yFJcag/deXbE/5kp8o/YZeCsQLF4WVz7eVv1kzG9h1XNtGZqPVdJpWB4Truu2OhXDDLFdNW6yev7X2Ajqns5glgkYkzwfYnIqhJJngNGJQepazKaETcswGIBMSs3woy34U2gFPiEZpBjsRqPRePSFJnOfzOAAyJmKcL8cK56rYYCZGnaHkyXQmWEIvCo1mERIpKn4LhTxjVERzEIRmHHpFdEwyQgX86bUqQQx3SNgZTeOYJKH0SJCrAR5KL4L/F6iGkZcVSum67oVsBANT3kO+n/d4MlHy035XSatlGwg3HQNZLRP6Qad4mf7Coig9+8e3AMXYQE6rxO1l/ENGkmN2FcfttoHaVolbS82UE9vH13DLNVDTVqtA6xK0IKVdbGclZy+4Fi7yAYc75kqwuQA78ALYAdDFSodpuxwpdLPoWw3cbtgfm7W97mLu1rWn2nOtrmHN0fa0N1pPO9Bo5VmlW3lbeVf9Vv1e/Vn9dYGuVRZnnmjXVvX3XyzTRWc=</latexit>
ĜV (·) r̈ i ˆ
r̈ j ,
Node part
U
Training target
(A) (B) (C) (D)
N
Figure 7: Pipeline of PIG’N’PI to learn pairwise force or potential for the LJ-
argon particle data. Solid-line arrows indicate the data processing path from A
M
input to the output. The dash-line arrow depicts the intermediate output of
every edge corresponding to the inferred pairwise force or potential energy.
ED
(A) Positions of 258 particles at a random time step. (B) Representation

of the constructed graph. Node features comprise position, velocity and
mass. Edge features comprise the relative position difference under periodic
IT
boundary conditions. (C) PIG’N’PI. Edge part takes the concatenation of

two nodes’ features and the edge feature as input and infers the pairwise force
ED
or potential. Node part aggregates the output on every edge and predicts
the acceleration. (D) The inferred pairwise force or potential by edge part
and the acceleration by node part.
N
U
L
A
IN
G
RI
19
O
MAEF
MAEacc standard deviation MAEef standard deviation symm standard deviation

<latexit sha1_base64="OKmNwkndxegj2L/43JC2O1kFEcA=">AAADf3icbZLdbtMwFMfdho8RPtYxiRtuLCoEF6WKk37uahQhuJnoEO2GmqpyXKez6jhR7GxUUZ6C5+ABuAVeghueBSctaM16JCt//c/P9vHJ8SLOpLKs35Wqcev2nbt798z7Dx4+2q8dPB7LMIkJHZGQh/G5hyXlTNCRYorT8yimOPA4PfOWb/L82SWNJQvFJ7WK6DTAC8F8RrDS1qz2KnUV/aKkv/56fnry+m2WzcouJiTTdq1uNa0i4E2BNqJ+/OT0D/s2+DWcHVSP3HlIkoAKRTiWcoKsSE1THCtGOM1MN5E0wmSJF3SipcABldO0eFcGn2tnDv0w1ksoWLjXd6Q4kHIVeJoMsLqQ5Vxu7spNEuX3pikTUaKoIOuL/IRDFcK8SXDOYkoUX2mBScx0rZBc4BgTpVu5dYsX6DcIekXCIMBinrrYk9kETVOX67YrWEfQjXOVmabpzqmv/1PxjvRkNWRimaUf3w2y1G47DYha3Qa025auB16iMv2Zch5e/efbGkWoAbvtAnfK+IcYiwW9jqNOpwE7doHbpWKKQRmjLdzuN2DLyXaB9j/Q1kc6+eru5JwN10b5eZpDPWsn2NqAPd0B1NVgH2WmnjZUnq2bYmw3UafpnOqxG4B17IGn4Bl4CRDogmPwHgzBCBDwFXwHP8BPo2K8MJqGtUarlc2eQ7AVRv8vv9oY1w==</latexit>
Pairwise force MAEFsymm (meV/Å)
Pairwise force MAEef (meV/Å)
Acceleration MAEacc (Å/ps2)
MAEacc average MAEef average MAEF

symm average
PT
PIG’N’PI GN+ GN+uni Baseline PIG’N’PI GN+ GN+uni Baseline PIG’N’PI GN+ GN+uni Baseline
(A) (B) (C)
RI
Figure 8: Performance of the algorithms on pairwise force predictions on the
LJ-argon system. We report the MAE on the acceleration prediction, which is
SC
the target for the learning task (A), the MAE on the pairwise force inference
(indirect inference task) MAEef (B) and the consistency with Newton’s action-
U
reaction property: the MAE on pairwise force symmetry MAEFsymm (C). The
N
average (plots at the bottom) on logarithmic scale and standard deviation
(plots in the top row) are computed from five experiments.
A
M
learnt interactions do not satisfy Newton’s law.
To summarize, similar to the cases discussed in Sec. , PIG’N’PI learns
ED
the pairwise force well without any direct supervision for this complex and
large system.
Besides, we test PIG’N’PI to learn the pairwise potential energy for this
IT
LJ system. Results are reported in Fig. 9 and Table S7. We first examine
the MAEacc that is the learning target. The MAEacc of PIG’N’PI is similar to
ED
that in the force-based version of the algorithm. PIG’N’PI performs signifi-

cantly better than the baseline model with more than 2 orders of magnitude
(Fig. 9A). And, similar to the cases in Sec. , we again observe the perfor-
N
mance drop of the baseline model in this potential-based version with the
U
force-based version. Then, we evaluate the MAE∆ep and MAEef that are the
two metrics for measuring the quality of the learnt pairwise potential energy.
L
Results show that PIG’N’PIprovides again superior performance on inferring

A
the increment of the potential energy MAE∆ep (Fig. 9B). The bad perfor-
mance of the baseline model clearly shows that it is not able to infer the
IN
potential energy correctly. Finally, we check Newton’s action-reaction prop-

erty in the potential energy by MAEPsymm . Here again, the potential energy
G
RI
20
O
MAEP
MAEacc standard deviation MAE standard deviation symm standard deviation

ep

Pairwise potential MAEPsymm (meV)
(meV)
Acceleration MAEacc (Å/ps2)
ep
MAEacc average MAE average symm average
MAEP
ep

PT
PIG’N’PI Baseline PIG’N’PI Baseline PIG’N’PI Baseline
(A) (B) (C)
RI
Figure 9: Quality of pairwise potential prediction on the LJ-argon data.
We report different errors in logarithmic scale. The average and standard
SC
deviation are computed from five experiments. (A) Acceleration prediction
error MAEacc . (B) Pairwise potential incremental error MAE∆ep . (C) Pairwise
U
potential symmetry error MAEPsymm .
N
inferred by PIG’N’PI follows Newton’s laws while the baseline model fails
to infer the underlying interaction laws correctly (Fig. 9D). All evaluations A
M
demonstrate that the predicted pairwise potential energy by PIG’N’PI is con-
sistent with the LJ potential used in the simulation, even though PIG’N’PI
does not access the ground truth information.
ED
The results on this case study demonstrate the scalability of PIG’N’PI

to larger systems and the applicability to more realistic case studies. More-
IT
over, the results confirm the results obtained in the previous case study that
PIG’N’PI is able to infer the underlying interaction laws correctly. While the
ED
other baseline methods are able to precisely predict the particle dynamics,
they fail to infer the underlying pairwise interaction laws correctly.
N
Comparison of PIG’N’PI to alternative hyperparameter

U
choices and an alternative regularized architecture

L
We compare the performance of the proposed approach first to alternative

A
hyperparameter choices, in particular to different activation functions, and,

second, to an alternative way of imposing physical consistency in the network
IN
architecture.
First, we evaluate different choices of activation functions following the
G
observations made in previous studies [21, 37, 18] that confirmed their sig-
RI
21
O
nificant influence on the performance of MLPs in approximating physical

quantities. The performance of PIG’N’PI with different activation functions
is reported in Table S8 in SI Appendix F and Table S9 in SI Appendix G.
The results demonstrate that PIG’N’PI with SiLU activation function (which
was in fact used in all case studies) performs consistently best on most test
datasets compared to PIG’N’PI with other commonly used activation func-
tions, such as ReLU or LeakyReLU. Based on this observation, the perfor-
mance of the baseline with the SiLU activation function was evaluated (SI
PT
Appendix B). The results show that the SiLU activation improves the learn-
ing performance of the baseline model to some degree (when only evaluating
RI
the prediction performance MAEacc ). However, it still performs consistently
worse than PIG’N’PI and, more importantly, the consistency with underlying
SC
physics in terms of the inferred force (or potential) and interaction symmetry
worsens even considerably.
Second, we compare the performance of PIG’N’PI to an alternative way
U
of imposing physical consistency: we add a regularization into the baseline
N
model to enforce the symmetry property onto the output messages of the
edge function. The goal of imposing the symmetry regularization term is
A
to ensure that the model satisfies the action-reaction physical consistency
M
requirement. It is expected that by satisfying this symmetry constraint,
the model performance on learning physics-consistent pairwise forces and
potential energy can be improved. We add a symmetry regularization term
ED
on the learnt pairwise corresponding messages to enforce the action-reaction

property. The details on this regularization term can be found in Sec. .
IT
While the performance is improved compared to the baseline model with-

out any regularization, the results demonstrate that PIG’N’PI still per-
ED
forms considerably better on inferring physical meaningful quantities for the

pairwise force and potential energy than the symmetry-regularized baseline
model (see SI Appendix H; Table S10 for detailed results and SI Appendix E
N
for the evaluation on the LJ-argon system).

U
Conclusions
L
A
In this paper, we propose the Physics-induced Graph Network for Particle

IN
Interaction algorithm to learn particle interactions that are consistent with

the underlying physical laws. The main novelty of the proposed algorithm is
in the design of the physics operator in the node part. The designed physics
G
RI
22
O
operator on nodes guides the edge neural network to learn the pairwise force

or pairwise potential energy exactly. This design also reduces the model
complexity for this machine learning algorithm by reducing the number of
tunable parameters.
While our method shows a similar performance on the supervised learn-
ing task of predicting accelerations compared to the other baseline models
(purely data-driven graph networks), it is also able to infer the particle in-
teractions correctly that follow the underlying physical laws. However, while
PT
the baseline models are able to be applied as simulators, they fail to infer
physically consistent particle interactions that satisfy Newton’s laws. More-
RI
over, the proposed PIG’N’PI also outperforms the other baseline models in
terms of generalization ability to larger systems and robustness to significant
SC
levels of noise.
The proposed methodology can generalize well to larger particle systems.
However, we have to point out that the trained model cannot extrapolate the
U
data arbitrarily far from the training distribution. In our experiments, we
N
found that the edge neural network converges to linear functions outside the
training input space. This observation matches the discussion in [38], which
is an inherent limitation of MLPs. A
M
The developed methodology will help to make a step forward in develop-
ing a flexible and robust tool for the discovery of physical laws in material
mechanics. Such tools will be able to support, for example, additive manu-
ED
facturing with heterogeneous materials that are particularly subject to highly

varying material properties, e.g., sustainable or recycled materials [39].
IT
Methods
ED
Notations and formal task description

N
We use a fully-connected directed graph G = (V, E) to represent the inter-

U
acting particle system, where nodes V = {v1 , v2 , . . . , v|V | } correspond to the

particles and the directed edges E = {eij : vi , vj ∈ V, i 6= j} correspond to
L
particle interactions. Under this notation, vi refers to the i-th particle, and
A
eij is the directed edge from vj to vi . We use {ηit }i,t to denote the observa-
tion of particle states at different time steps, where ηit is a vector describing
IN
the state of particle vi at time t. We note that ηit ∈ R2d+2 (d is the space
dimension) includes position rit ∈ Rd , velocity ṙit ∈ Rd , electric charge qi ∈ R
G
RI
23
O
and mass mi ∈ R. The velocity ṙit and acceleration r̈it at time t are computed

from the position series of particle vi . We use Mij to denote the message
from vj to vi learnt by the neural network ĜE (·; θE ) with parameters θE .
Our goal is to infer the pairwise force Fijt and the potential energy Pijt on
every edge eij at each time t given the observation of particle trajectories.
All notations are summarized in Table S1.
PIG’N’PI details
PT
PIG’N’PI contains an edge part to learn the pairwise interaction and a node
RI
part to aggregate the interactions to derive node accelerations (see Fig. 1).
In the edge part, we use MLPs as universal approximators [40, 41] to learn
SC
the pairwise force or pairwise potential energy. We denote this edge neural
network as ĜE (·; θE ). ĜE (·; θE ) takes the vectors ηi and ηj of two nodes
as input. The output Mij of ĜE (·; θE ) is the inferred pairwise force F̂ij or
U
potential energy P̂ij on edge eij , depending on the operator in the node part.
N
We design the physics operator GN (·) to aggregate the edge messages in the
t
node part and derive the acceleration ˆ
A
r̈ i for every particle vi at time t. We
optimize parameters θE by minimizing the mean absolute error between the
M
predicted acceleration and the true acceleration. The objective function is
given by:
ED
|V |
1 1 X X ˆt t
arg min L = l1 (r̈ i , r̈i ) (3)
|Ttrain | |V | t∈T
IT
θE
i=1
train
In the following, we explain the design of the edge neural network ĜE (·; θE )
ED
and the node part operator GN (·) in two cases: inferring the pairwise force
and inferring the pairwise potential energy.
N
Learning pairwise force

U
We use an MLP as the edge neural network ĜE (·; θE ) to learn the pairwise
L
force from vj to vi on each edge eij . The output dimension of ĜE (·; θE ) is
A
the same as the spatial dimension d. We first concatenate ηit and ηjt which
is the input of ĜE (·; θE ). We denote the corresponding output as Mtij ∈ Rd ,
IN
e.g.,
Mtij , ĜE (concat(ηit , ηjt ); θE ) (4)
G
RI
24
O
According to Newton’s Second law, the net acceleration of every particle

is equal to the net force divided by its mass. Hence, inPthe node part of
PIG’N’PI, we first sum up all incoming messages Mi = j6j =i Mij of every
particle vi , and then divide it by the mass of the particle mi . The output of
GN (·) is the predicted acceleration ˆr̈ i on particle vi :
t
ˆ
r̈ i = GN (ηit , Mti )
P
= GN (ηit , j6j =i Mtij )
PT
Pj6=i t (5)
j Mij
=
RI
mi
SC
We optimize the parameters θE in ĜE (·; θE ) by minimizing the objective
function Eq. (3). Through this process, the node part operator GN (·) guides
the edge neural network ĜE (·; θE ) to predict the pairwise force exactly, e.g.,
U
F̂ijt = Mtij
N
(6)
This is illustrated in Block (B) of Fig. 2.

A
M
Learning pairwise potential energy
ED
For the pairwise potential energy case, the edge neural network ĜE (·; θE ) is
designed to output the pairwise potential energy. Here, the output dimension
of ĜE (·; θE ) is one because the potential energy is a scalar. We still first
IT
concatenate ηit and ηjt as the input of ĜE (·; θE ) and use MLPs as ĜE (·; θE ).
The corresponding output Mtij ∈ R is denoted as:
ED
Mtij , ĜE (concat(ηit , ηjt ); θE ) (7)

N
We know that the net force of every particle equals to the negative partial
U
derivative of the potential energy with respect to its position.

Pj6=i Hence, in the
node part, we first sum up all incoming messages Mi = j Mij for every
L
particle i, then compute the negative derivative with respect to the input
A
position and finally divide it by the mass. The final output corresponds then
to the predicted acceleration. The node part operator GN (·) for the potential
IN
G
RI
25
O
energy case is given by:

t
ˆ
r̈ i = GN (ηit , Mti )
P
= GN (ηit , j6j =i Mtij )
P (8)
∂( j6j =i Mtij )/∂rit
=−
mi
PT
Analogously to the force-based case, we optimize for the parameters θE in
ĜE (·; θE ) by minimizing the acceleration loss (Eq. 3). The node part operator
GN (·) here guides the edge neural network ĜE (·; θE ) to learn the pairwise
RI
potential energy exactly. The learnt message on each edge corresponds to
the predicted pairwise potential energy, and the negative partial derivative
SC
is the predicted pairwise force, e.g.,
U
P̂ij = Mij
(9)
N
F̂ij = −∂ P̂ij /∂ri = −∂Mij /∂ri
This is illustrated in Block (C) of Fig. 2.

A
We note that the commonly used ReLU activation function is not suitable
M
as activation function in ĜE (·; θE ) for learning the potential energy. The rea-
son is that we compute the partial derivative of Mij = ĜE (concat(ηi , ηj ); θE )
ED
to derive the predicted accelerations for every particle. The derivative should
be continuous and even smooth considering physical forces. However, ReLU
approximates the underlying function by piece-wise linear hyper-planes with
IT
sharp boundaries. The first-order derivative is, thus, piece-wise constant that
ED
does not change with input [21]. Details on selecting the activation function
in ĜE (·; θE ) are explained in Sec. .
N
Details on imposing a symmetry regularization on the

U
baseline model
As mentioned in Sec. , to ensure that the model satisfies the action-reaction
L
physical consistency requirement, we also test an extension of the baseline

A
model by imposing a symmetry regularization on the corresponding pairwise

IN
messages in the baseline model. This can be considered as an alternative way

of imposing physical consistency. In details, let Mij be the message from vj
G
to vi which is the output of the edge neural network of the baseline model.
RI
26
O
In our experimental setup, the message Mij corresponds to the force from

vj to vi . We impose the symmetry regularization by adding a regularization
term on the learnt messages in the objective function (Eq. (3)). This results
in the following objective function:
|V |
1 X 1 X ˆt
arg min L = ( |r̈ − r̈it |
θE |Ttrain | t∈T |V | i=1 i
train
| {z }
PT
Acceleration loss on nodes
i6=j
(10)
1 X t
+ α |Mij + Mtji | )
RI
|E| i,j
| {z }
SC
Symmetry regularization loss on edges
where α is a weight parameter. The original baseline model can be considered
U
as the special case with α = 0 in Eq. (10). In our experiments, we evaluate
the impact of the regularization term with different weights (α = 0.1, 1.0,
N
10, 100). The results are reported in Table S10.
A
M
Details of the method to learn pairwise force introduced
by [34]
ED
[34] proposed a method that has a similar goal to the proposed PIG’N’PI
applied to pairwise force prediction. The authors also impose Newton’s sec-
ond law in the standard GN block by dividing the aggregated messages by
IT
the node property. We denote this method as GN+. The main difference
ED
between GN+ and PIG’N’PI is that GN+ treats the node property as a
learnable parameter. It assigns an individual learnable scalar wi for each
particle vi and predicts the acceleration of vi by dividing the aggregated in-
N
coming messages by 10wi . The learnable scalars on all nodes representing the
pairwise force are learnt together with all other parameters. It is important
U
to point out that GN+ was designed solely for learning the pairwise force
while PIG’N’PI can be applied both: to infer the pairwise forces and also
L
the pairwise potential energy. The detailed results of GN+ are reported in
A
Table S2, Table S4, Table S6, Figs 3D, 4 and 8.

IN
G
RI
27
O
Details about simulation and experiments

Here, we summarize the different force functions used in our simulation.
Please note that in this work, we used the same case studies as in previ-
ous work [22]. However, we adapted the parameters of the particle systems
slightly to make the learning more challenging.
Spring force We denote the spring constant as k and balance length

as L. The pairwise force from vi to vj is k(rij − L)nij and its potential
PT
energy is 0.5k(rij − L)2 , where rij = krj − ri k is the Euclidean distance
r −r
and nij = krjj −rii k is the unit vector pointing from vi to vj . We set
RI
k = 2.0 and L = 1.0 in our simulations.
SC
Charge force The electric charge force from vi to vj is −cqi qj nij /rij
2
and the potential energy is cqi qj /rij , where c is the charge constant,
and qi , qj are the electric charges. We set c = 1.0 in the simulation.
U
Furthermore, to prevent any zeros in the denominator, we add a small
N
number δ (δ = 0.01) when computing distances.
A
Orbital force The orbital force from vi to vj equals to mi mj nij /rij
and the potential energy is mi mj ln(rij ), where mi , mj are the masses of
M
vi and vj . We again add a small number δ (δ = 0.01) when computing
distances to prevent zeros in the denominator and logarithm.
ED
Discontinuous force We set threshold constant Θ = 2.0 such that the

pairwise force is 0 if the Euclidean distance rij is strictly smaller than
IT
this threshold and (rij −1)nij otherwise. The corresponding potential is

0 if rij is strictly smaller than this threshold and 0.5(rij −1)2 otherwise.
ED
We intentionally omit units for variables because the simulation data can
be at arbitrary scale. Moreover, the presented cases serve as proof of concept
N
to learn the input–output relation. Further, we note that mi is sampled from

U
the log-uniform distribution within the range [−1] (ln(mi ) ∼ U(−1, 1)). qi
is uniformly sampled from the range [−1, 1]. Initial location and velocity of
L
particles are both sampled from the normal Gaussian distribution N (0, 1).
A
Each simulation contains eight particles. Each particle is associated with the
corresponding features including position, velocity, mass and charge. The
IN
target for prediction is node accelerations. Every simulation contains 10,000

time steps with step size 0.01. We randomly split the simulation steps into
G
training dataset, validation dataset and testing dataset with the ratio 7 :
RI
28
O
1.5 : 1.5. We use Ttrain , Tvalid and Ttest to indicate the simulation time steps

corresponding to training split, validation split and testing split. We train
the model on the training dataset (optimizing the parameters θE in ĜE (·; θE ))
by optimizing Eq. (3), fine-tune hyperparameters and select the best trained
model on the validation dataset and report the performance of the selected
trained model on the testing dataset. For generalization tests, we re-run each
simulation on 12 particles with 1500 time steps (same size as original testing
dataset). The previously selected trained model with eight particles is tested
PT
on the new testing dataset.
We only fine-tune hyperparameters on the spring validation dataset and
RI
use the same hyperparameters in all experiments. We set the learning rate to
0.001, the number of hidden layers in the edge neural network to 4, the units
SC
of hidden layers to 300, max training epochs to 200. The dimension of the
output layer in the edge neural network is d to learn the force or one to learn
the potential energy. We use the Adam optimizer with the mini-batch size of
U
32 for the force case study and eight for the potential case study to train the
N
model. The SiLU activation function is used in all PIG’N’PI evaluations.
This project has been funded by the ETH Grant office. The contribution of
A
Olga Fink to this project was funded by the Swiss National Science Founda-
M
tion (SNSF) Grant no. PP00P2 176878.
ED
Supplementary Material
Supplementary material is available at PNAS Nexus online.
IT
ED
Competing Interests
The authors declare no competing interest.
N
U
Author Contributions Statement

L
DK and OF conceptualized the idea, ZH developed the methodology, ZH,

A
DK and OF conceived the experiments, ZH conducted the experiments, ZH,

IN
DK and OF analysed the results, and all authors wrote the manuscript.
G
RI
29
O
Code and Data Availability

The implementation of PIG’N’PI is based on PyTorch [42] and pytorch-
geometric [43] libraries. The source code is available on Gitlab: https://
gitlab.ethz.ch/cmbm-public/pignpi. The data used in the experiments
are generated by the numerical simulator. All data used for the experiments
are included in the associated Gitlab repository: https://gitlab.ethz.ch/
cmbm-public/pignpi/-/tree/main/simulation.
PT
References
RI
[1] Murray CD, Dermott SF. 1999. Solar system dynamics. Cambridge:
SC
Cambridge University Press.
U
[2] Dramis F. 2016. Mass movement processes and landforms. International
Encyclopedia of Geography: People, the Earth, Environment and Tech-
N
nology: People, the Earth, Environment and Technology. 1–9.
A
[3] Sawyer WG, Argibay N, Burris DL, Krick BA. 2014. Mechanistic studies
M
in friction and wear of bulk materials. Ann Rev Mat Res. 44:395–427.
[4] Furlani EP. 2010. Magnetic biotransport: Analysis and applications.

ED
Materials. 3(4):2412–2446.
[5] Radovic A, et al. 2018. Machine learning at the energy and intensity
IT
frontiers of particle physics. Nature. 560(7716):41–48.

ED
[6] Carleo G, et al. 2019. Machine learning and the physical sciences. Rev
Mod Phys. 91(4):045002.
N
[7] Zhou J, et al. 2020. Graph neural networks: A review of methods and
applications. AI Open. 1:57–81.
U
[8] Shlomi J, Battaglia P, Vlimant JR. 2020. Graph neural networks in

L
particle physics. Mach Learn Sci Tech. 2(2):021001.

A
[9] Atz K, Grisoni F, Schneider G. 2021. Geometric deep learning on molec-

IN
ular representations. Nat Mach Int. 3(12):1023–1032.

G
RI
30
O
[10] Méndez-Lucio O, Ahmad M, del Rio-Chanona EA, Wegner JK. 2021. A

geometric deep learning approach to predict binding conformations of
bioactive molecules. Nat Mach Int. 3(12):1033–1039.
[11] Park CW, et al. 2021. Accurate and scalable graph neural network force
field and molecular dynamics with direct force architecture. npj Comp
Mat. 7(1):73.
[12] Sanchez-Gonzalez A, et al. 2020. Learning to simulate complex physics
PT
with graph networks. In International Conference on Machine Learning
PMLR. p. 8459–8468.
RI
[13] Matsumoto S, et al. 2021. Extraction of protein dynamics information
SC
from cryo-EM maps using deep learning. Nat Mach Int. 3(2):153–160.
[14] Schütt KT, et al. 2017. SchNet: A Continuous-Filter Convolutional Neu-
U
ral Network for Modeling Quantum Interactions. In Advances in Neural
N
Information Processing Systems. p. 992–1002.
A
[15] Schütt KT, Sauceda HE, Kindermans PJ, Tkatchenko A, Müller KR.
2018. Schnet–a deep learning architecture for molecules and materials.
M
J Chem Phys. 148(24):241722.
ED
[16] Kipf T, Fetaya E, Wang KC, Welling M, Zemel R. 2018. Neural rela-
tional inference for interacting systems. In International Conference on
Machine Learning PMLR. p. 2688–2697.
IT
[17] Unke OT, Meuwly M. 2019. PhysNet: A neural network for predicting
ED
energies, forces, dipole moments, and partial charges. J Chem Theor

Comp. 15(6):3678–3693.
N
[18] Klicpera J, Groß J, Günnemann S. 2020. Directional message passing

for molecular graphs. In International Conference on Learning Repre-
U
sentations.
L
[19] Klicpera J, Giri S, Margraf JT, Günnemann S. 2020. Fast and

A
Uncertainty-Aware Directional Message Passing for Non-Equilibrium

Molecules. In Machine Learning for Molecules Workshop at NeurIPS.
IN
[20] Bapst V, et al. 2020. Unveiling the predictive power of static structure
G
in glassy systems. Nat Phys. 16(4):448–454.

RI
31
O
[21] Hu W, et al. 2021. ForceNet: A graph neural network for large-scale

quantum calculations. In SimDL Workshop at ICLR.
[22] Cranmer M, et al. 2020. Discovering Symbolic Models from Deep Learn-
ing with Inductive Biases. In Advances in Neural Information Processing
Systems. vol. 33. p. 17429–17442.
[23] Moon S, Zhung W, Yang S, Lim J, Kim WY. 2022. PIGNet: a physics-
informed deep learning model toward generalized drug–target interac-
PT
tion predictions. Chem Sci. 13(13):3661–3673.
RI
[24] Ha S, Jeong H. 2021. Unraveling hidden interactions in complex systems
with deep learning. Sci Rep. 11(1):1–13.
SC
[25] Huang W, et al. 2022. Equivariant graph mechanics networks with con-
straints. In International Conference on Learning Representations.
U
[26] Battaglia PW, et al. 2018. Relational inductive biases, deep learning,
N
and graph networks. arXiv preprint arXiv:180601261.
A
[27] Greydanus S, Dzamba M, Yosinski J. 2019. Hamiltonian neural net-
M
works. In Advances in Neural Information Processing Systems. vol. 32.
[28] Sanchez-Gonzalez A, Bapst V, Cranmer K, Battaglia P. 2019. Hamilto-

ED
nian graph networks with ode integrators. In Machine Learning and the
Physical Sciences Workshop at NeurIPS.
IT
[29] Lutter M, Ritter C, Peters J. 2019. Deep lagrangian networks: Using

physics as model prior for deep learning. In International Conference on
ED
Learning Representations.
[30] Wang J, et al. 2019. Machine learning of coarse-grained molecular dy-

N
namics force fields. ACS Central Sci. 5(5):755–767.

U
[31] Finzi M, Wang KA, Wilson AG. 2020. Simplifying hamiltonian and la-
grangian neural networks via explicit constraints. In Advances in Neural
L
Information Processing Systems. vol. 33. p. 13880–13889.

A
[32] Karniadakis GE, et al. 2021. Physics-informed machine learning. Nature

IN
Reviews Physics. 3(6):422–440.

G
[33] Goodfellow I, Bengio Y, Courville A. 2016. Deep learning. MIT Press.

RI
32
O
[34] Lemos P, Jeffrey N, Cranmer M, Battaglia P, Ho S. 2021. Rediscovering

Newton’s gravity and Solar System properties using deep learning and
inductive biases. In SimDL Workshop at ICLR.
[35] Li Z, Meidani K, Yadav P, Barati Farimani A. 2022. Graph neural net-

works accelerated molecular dynamics. J Chem Phys. 156(14):144103.
[36] Rapaport DC, Rapaport DCR. 2004. The art of molecular dynamics
simulation. Cambridge: Cambridge University Press.
PT
[37] Niaki SA, Haghighat E, Campbell T, Poursartip A, Vaziri R. 2021.
RI
Physics-informed neural network for modelling the thermochemical cur-
ing process of composite-tool systems during manufacture. Comp Meth
SC
Appl Mech Eng. 384:113959.
[38] Xu K, et al. 2021. How Neural Networks Extrapolate: From Feedforward
U
to Graph Neural Networks. In International Conference on Learning
N
Representations.
A
[39] Wang Q, Zhang L. 2021. Inverse design of glass structure with deep
graph neural networks. Nature Communications. 12(1):5359.
M
[40] Hornik K, Stinchcombe M, White H. 1989. Multilayer feedforward net-
ED
works are universal approximators. Neural Networks. 2(5):359–366.
[41] Hornik K. 1991. Approximation capabilities of multilayer feedforward

IT
networks. Neural Networks. 4(2):251–257.
[42] Paszke A, et al. 2017. Automatic differentiation in pytorch. In Workshop

ED
on Autodiff at NeurIPS.
N
[43] Fey M, Lenssen JE. 2019. Fast Graph Representation Learning with Py-
Torch Geometric. In Representation Learning on Graphs and Manifolds
U
Workshop at ICLR.
L
A
IN
G
RI
33
O

Learning Physics-Consistent Particle Interactions

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Learning Physics-Consistent Particle Interactions

Uploaded by

Copyright:

Available Formats

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.

1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

Interacting particle systems play a key role in science and engineer-

for a complete understanding of such systems. However, the inher-

learning the kinetics at the particle level. Learning pairwise interac-

challenge. Here, we propose an algorithm that adapts the Graph Net-

interaction and a node part to model the dynamics at particle level.

Different from existing approaches that use neural networks in both

to precisely infer the pairwise interactions that are consistent with

whom correspondence should be addressed: Email: olga.fink@epfl.ch

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

incorporates universal physical laws to infer pairwise force (or poten-

physical laws without any supervision by only being trained to predict

Interacting particle systems play a key role in nature and engineering as

cles, knowing the governing interaction law is required to better understand,

tions are determined by a combination of various factors including contact,

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

knowledge about the fundamental laws governing the particle interactions

to the individual pairwise particle interactions. [22] demonstrated that their

Newtonian laws of motion, such as the action-reaction property, which states

mental properties of a physical system. The origin of these discrepancies

Besides interacting particle systems, recent works applied Graph Network

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

show that the inferred interactions by PIG’N’PI are physically consistent

ther demonstrate that predictions provided by PIG’N’PI are more accurate,

proposed algorithm is scalable to large systems and is applicable to any type

Figure 1: Framework of the proposed model to learn pairwise force or pair- A

consistent pairwise force or pairwise potential at every time step t. The

istic node operator GV (·) by incorporating physics knowledge to derive the

net acceleration on nodes (process ¯). By minimizing the loss on node-level

or potential energy exactly.

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

A particle system in this research is represented by the directed graph G =

directed edges E = {eij : vi , vj ∈ V, i 6= j} correspond to their interactions.

particle i, represented by a node vi , is characterized by its time-invariant

systems comprising homogeneous particle types. This results in particles

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

Node part design: learning force

Node part design: learning potential

particle interaction (PIG’N’PI) framework extends the general Graph Net-

passing graph neural networks. The architecture of the proposed framework

θE and θV are the trainable parameters. Here, we propose to substitute the

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

Trained to predict the node-level acceleration, once applied to a new particle

Contributions of the present work compared to precious research Here,

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

study, cover a wide range of particle interaction features, including depen-

Fig. S2 in SI Appendix J for visualization).

proposed by [34] that is also based on GN and was specifically designed to

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

the proposed methodology with a focus on two key aspects: 1) supervised

|Ttest | |V | t∈T i=1

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

Performance evaluation between PIG’N’PI, GN+ and

baseline for pairwise force

First, we analyze PIG’N’PI for application on particle systems with inter-

learning performance by evaluating the prediction of the acceleration using

Fig. 3D) and all other datasets (see SI Appendix B).

Downloaded from https://academic.oup.com/pnasnexus/advance-article/doi/10.1093/pnasnexus/pgac264/6832788 by guest on 19 November 2022

Pairwise force MAEef

Figure 3: Case study: Quality of pairwise force prediction of PIG’N’PI and

difference between predicted force and the ground-truth force (Angle(F̂ , F ),

PIG’N’PI. (D) Comparison of the quality of PIG’N’PI, GN+ and baseline

The LJ potential, which is given by V (r) = 4{(σ/r)1 2 − (σ/r)6 }, is an ex-