ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72

Contents lists available at ScienceDirect

ISPRS Journal of Photogrammetry and Remote Sensing

journal homepage: www.elsevier.com/locate/isprsjprs

A review of non-rigid transformations and learning-based 3D point cloud

registration methods
Sara Monji-Azad a ,∗, Jürgen Hesser a,b,c,d , Nikolas Löw a
a Mannheim Institute for Intelligent Systems in Medicine (MIISM), Medical Faculty Mannheim, Heidelberg University, Mannheim 68167, Germany
b
Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Heidelberg, Germany
c
Central Institute for Computer Engineering (ZITI), Heidelberg University, Heidelberg, Germany
d
CZS Heidelberg Center for Model-Based AI, Heidelberg University, Mannheim, Germany

ARTICLE INFO ABSTRACT

Keywords: Point cloud registration is a research field where the spatial relationship between two or more sets of points in
Point cloud registration space is determined. Point clouds are found in multiple applications, such as laser scanning, 3D reconstruction,
Non-rigid transformation and time-of-flight imaging, to mention a few. This paper provides a thorough overview of recent advances in
Quantitative assessments metrics
learning-based 3D point cloud registration methods with an emphasis on non-rigid transformations. In this
Robustness
respect, the available studies should take various challenges like noise, outliers, different deformation levels,
Registration datasets
and data incompleteness into account. Therefore, a comparison study on the quantitative assessment metrics
and robustness of different approaches is discussed. Furthermore, a comparative study on available datasets is
reviewed. This information will help to understand the new range of possibilities and to inspire future research
directions.

1. Introduction any two points is not changed by transformation. In contrast, to model a

Point cloud registration is one of the most fundamental and chal-
lenging areas in computer vision. It has a wide range of applications,
such as 3D localization (Elbaz et al., 2017), 3D object recognition (Al-
hamzi et al., 2015), 3D reconstruction (Takimoto et al., 2016), aug-
mented/virtual reality (Mahmood and Han, 2019), generating free-
viewpoint videos (Zhang et al., 2021b), tracking (Wang et al., 2019e),
and medical imaging (Petricek and Svoboda, 2017) etc. The goal of
point cloud registration is to find a transformation between two or
more corresponding point sets, which minimizes the distance between
the transformed source point cloud and the target one (Feng et al.,
2019), (Ge, 2016). Moreover, the transformation can be defined as a
specific transformation model (Besl and McKay, 1992) or as a model-
free solution with a displacement vector field (DVF) (Myronenko et al.,
2006). Finding the model’s optimal parameters is the main task of
model-based methods while estimating the displacement field that
transfers the source point set to the target one is the main task of model-
free methods (Wang et al., 2019c). A sample of point cloud registration
both for 2D and 3D data is demonstrated in Fig. 1.
Traditionally, the registration approaches can be categorized into
rigid and non-rigid transformations. Rigid registration usually estimates
rotation and translation parameters, which means the distance between
non-rigid transformation, some typical traditional transformation mod-
els are considered, such as affine transformation, projection models,
and thin-plate spline (TPS) (Chui and Rangarajan, 2003), to mention
a few (Bellekens et al., 2015).
Still, in comparison with rigid approaches, the transformation func-
tion for non-rigid registration is more challenging. Finding the stable
corresponding points and using a proper deformation model can be
named as two main challenges of deformable registration (Castellani
and Bartoli, 2020).
Rigid and non-rigid point cloud registration, regardless of the trans-
formation function, can be further divided into non-learning as well as
learning-based registration methods. Non-learning registration methods
are mostly based on developing an iterative optimization algorithm to
estimate the rigid or non-rigid geometric transformation (Horn et al.,
2020). Nevertheless, learning-based methods learn the transformation
based on the geometric features of the point clouds.
Having said that, point cloud registration includes a wide range of
topics, which can be divided into different categories. However, this
area is suffering from different challenges such as different deformation
levels, different types of noise, different levels of the outlier, and

∗ Corresponding author.
E-mail addresses: sara.monjiazad@medma.uni-heidelberg.de (S. Monji-Azad), juergen.hesser@medma.uni-heidelberg.de (J. Hesser),
nikolas.loew@medma.uni-heidelberg.de (N. Löw).

https://doi.org/10.1016/j.isprsjprs.2022.12.023
Received 28 June 2022; Received in revised form 13 December 2022; Accepted 22 December 2022
Available online 3 January 2023
0924-2716/© 2023 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.
S. Monji-Azad et al.
Fig. 1. The output of a point cloud registration method for 2D and 3D point sets. The
inputs and outputs of a registration algorithm are shown in the first and second rows,
respectively.
data incompleteness (Wang et al., 2020c). Point cloud partial overlap-
ping (Zhao et al., 2021b) and cluttered scenes registration (Tazir et al.,
2018) are other challenges that should be handled through registration.
There are some existing survey articles on point cloud registration.
The available approaches have been studied from some specific points
of view in each of them. A review of point set registration based on
pairwise and groupwise registrations is provided in Zhu et al. (2019).
Pairwise registration methods use two point sets, while groupwise
registration methods take more than two point sets into account. Re-
view papers (Zhang et al., 2020; Tang et al., 2021) have provided an
overview of the learning-based point cloud registration methods. Some
non-learning methods are studied in Maiseli et al. (2017). In Huang
et al. (2021b), both learning and non-learning methods have been
studied for non-rigid point clouds from same-source and cross-source
domains. Same-source and cross-source refer to the point cloud acqui-
sition approach. In fact, when the point sets are captured using similar
sensors from different points of view or at various times, it is called
the same-source approach, while the cross-source methods mention the
ones which generate the point sets from different types of sensors.
In Saiti and Theoharis (2020), a review of 3D multi-modal registration
is presented. Multi-modal registration is an area of registration to find
correspondences between some different data, which are captured from
the same object in different modalities. A study on available learning-
based registration methods, which have used different inputs such
as 2D/3D mesh, images, voxel grid, and point clouds is presented
in Villena-Martinez et al. (2020). Furthermore, a review of point cloud
registration methods on mobile robotics is presented in Pomerleau et al.
(2015) and a comprehensive study on laser scanner point clouds is
presented in Dong et al. (2020), Cheng et al. (2018).
As discussed, although there are some existing reviews about reg-
istration, each of them presents some aspects of this problem. Some
reviews are suffering from not considering the recent advances in deep
learning. In another aspect, the reviews which cover novel learning-
based methods did not consider significant points, e.g., non-rigid trans-
formations. Focusing on some specific applications, such as medical
imaging or laser scanning to mention a few, is another motivation to
present a novel review of the point cloud registration topic. Another
point that has been studied but can be discussed more comprehensively
59
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
is how the point cloud registration methods can be evaluated and how
they can be robust to certain challenges. Additionally, reviewing the
available datasets and their characteristics is another significant matter
of the point cloud registration problem that can be addressed in more
detail.
Due to all mentioned points, this article focuses on learning ap-
proaches as well as non-rigid transformations. This review provides a
comprehensive look at the point cloud registration topic from various
points of view, which can be summarized below:
• Studying learning-based methods: a review of the most well-
known and qualified point cloud registration learning-based
methods.
• Non-rigid transformation: a study on the existing point cloud
registration methods based on non-rigid transformation.
• Review on datasets: a comparative study on available 2D and
3D datasets and their characteristics of point cloud registration.
• Evaluation performance and metrics: a study on evaluation
metrics and performance indicators for point cloud registration.
• Challenges and robustness: a study on the challenges of the
point cloud registration problem, how methods tackle the chal-
lenges, and how robust to challenges they are.
Although tremendous efforts have been made to present a clear view
of the point cloud registration problem, there are some aspects that
will not be considered in this review. Multi-modal registration methods,
partial registration techniques, and multi-view registration approaches
are some of them. Nevertheless, the methods that will be discussed in
this review can be generalized to the mentioned points.
The rest of this paper is organized as follows: the importance of
the study about point cloud registration is discussed as a systematic
review in Section 2. Section 3 provides the problem definition of non-
rigid point cloud registration. Section 4 outlines a general overview of
registration approaches, and Section 5 provides an overview of existing
methods. The experimental datasets for point cloud registration are
described in Section 6. The evaluation performance and metrics are
presented in Section 7. A discussion on point cloud registration and
its future outlook is provided in Section 8. Finally, Section 9 concludes
the paper.
2. Systematic review
Point cloud registration includes a wide range of research areas and
can be used in various applications. As it is shown in Fig. 2, the number
of publications in this research field has increased in recent years.
Before discussing the related articles and their approaches, a systematic
review will be presented in the following. There are different search
engines of research repositories which help to provide the statistics
of articles based on their topics and the years of publications. To
elaborate, Dimensions (Dimentions, 2022) is one of the popular sources
which collects the results from various datasets, e.g., Google Scholar,
IEEE Xplore digital library, ScienceDirect, arXiv, and so on. As a matter
of fact, these statistics have been collected using as many articles as
possible to show this research area’s importance. Based on a large
amount of data, missing some studies is inevitable.
A number of collected articles shown in Fig. 2 are based on both
learning and non-learning publications. Different keywords including
‘‘point cloud registration’’ as well as ‘‘point cloud alignment’’, ‘‘point
cloud matching’’, and so on are used. In this investigation, the pub-
lication type is limited to ‘‘Articles’’ and ‘‘Proceeding’’. Additionally,
the field of research is limited to ‘‘Information and computing science’’,
‘‘Artificial intelligence and image processing’’, and ‘‘Engineering’’.
In recent years, using learning approaches to solve computer vision
problems has rapidly become more popular. Particularly, using deep
learning methods to solve point cloud registration has been used more
compared to the years before 2017. The number of publications be-
tween 2017 and 2021 on learning methods based on deep architectures
S. Monji-Azad et al.
Fig. 2. The number of publications per year for both learning and non-learning
approaches is demonstrated, and it is shown how the interest in point cloud registration
is increasing over the years.
is shown in Fig. 3. The statistics are collected from Dimensions by using
different keywords such as: ‘‘point cloud registration’’, ‘‘point cloud
alignment’’ and ‘‘point cloud matching’’ as well as ‘‘learning’’, ‘‘machine
learning’’, ‘‘deep learning’’. Finally, a total of 124 articles were studied,
which are closely related to learning-based point cloud registration,
and then are classified according to their network architecture. As it is
shown in Fig. 3, the number of learning-based methods has increased
in recent years. Among the available architectures, the convolutional
neural network (CNN) and the Graph Neural Network (GNN) are the
most frequently used approaches.
Among all available articles, which are discussed in the systematic
review, the methods based on non-rigid transformation are selected to
be discussed more in detail. The selected methods can be considered
as baseline approaches which have more citations in comparison with
the rest of the studies as well as being published in journals based on
having a good reputation.
3. Problem definition
Given a source point set 𝑋 = {𝑥𝑖 |𝑥𝑖 ∈ R3 , 𝑖 = 1, 2, … , 𝑁} and a target
point set 𝑌 = {𝑦𝑖 |𝑦𝑖 ∈ R3 , 𝑖 = 1, 2, … , 𝑀}, where 𝑁, 𝑀 are the number
of points, the goal of non-rigid point cloud registration is to compute
the non-rigid transformation function 𝑓 ∶ R𝑁×3 → R𝑀×3 , such that the
deformed point set 𝑋̂ = 𝑓 (𝑋) is as close as possible to the target point
set Y, |𝑌 − 𝑋|
̂ → 0.
Recently, there are various studies on learning-based non-rigid point
cloud registration. Commonly, these methods train a model by using
training data to predict the non-rigid transformation 𝑓 with the source
and target point clouds 𝑋 and 𝑌 as inputs. Furthermore, there are
some other studies that train a network to learn the displacement field.
Moreover, there is the third studies group which relies on learning
correspondences for unseen data. Therefore, training the model can
be done by searching for model parameters to minimize the loss func-
tion or learning the best displacement field by minimizing a distance
function.
4. General overview of the existing methods
Considering available articles, most point cloud registration meth-
ods can be addressed in different points of view.
In one categorization, point cloud registration methods can be
classified into coarse and fine approaches. To this end, the coarse-
based registration methods compute an initial geometric alignment
while fine-based registration methods estimate a transformation to
solve registration problems as precisely as possible. Iterative Closest
Point (ICP) (Besl and McKay, 1992) and the CPD method (Myronenko
and Song, 2010) are two common fine approaches.
60
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
As the second categorization, feature-based registration methods
can be presented. The feature-based methods are mainly used for coarse
registration that provides prior estimates for fine methods like ICP.
Feature-based approaches can be considered in different phases which
are shown in Fig. 4. These methods extract the key points from the
point sets. After extracting feature descriptors, their dissimilarity is
calculated. By finding the corresponding points, the equation system
can be solved and transformation parameters can be calculated. There
are two additional steps, noise removal and outlier removal, which
can be optional in this model. Some approaches apply noise removal
methods as a pre-processing step to be robust to noise. However, to
be robust to the outlier, some methods use an outlier-robust approach
for finding corresponding points such as RANSAC (Fischler and Bolles,
1981) and its variants (Fotouhi et al., 2019; Li et al., 2020a, 2021a;
Quan and Yang, 2020).
In addition, it is worth mentioning that the schema which is shown
in Fig. 4 can be changed due to the fact that some approaches can solve
the registration problem without the feature extraction step. Beside
mentioned outlooks, point cloud registration methods can be discussed
from different points of view as well:
• Some studies show how robust the methods can be to the different
challenges in order to improve the results.
• Another approach is based on the registration area and the search
space. Global and local point cloud registration can be discussed
in this approach (Zhang et al., 2021a).
• There are several articles that focus on the loss function. Proba-
bilistic methods such as Expectation–Maximization (EM) (Chang
and Zwicker, 2009) and the approaches that utilize different
dis-similarity or distance metrics are common instances of this
group.
• Another group of studies concentrates on optimization issues.
Most feature-based approaches can be considered in this group.
The approach of these articles can be divided into two categories:
– Some studies use the pre-alignment techniques or an initial
guess to prevent the error function from getting stuck in
local minima. This can also speed up the convergence rate
of optimization, which leads to decreasing the running time.
– Another category consists of optimization techniques in gen-
eral. Levenberg–Marquardt (Fitzgibbon, 2003), or Graph-
cuts (Chang and Zwicker, 2008), to mention a few, are some
of the approaches, which focus on the optimization problem
under different constraints.
Nevertheless, it is worth mentioning that the studied methods,
which will be discussed in this review, can take place in more than
one of the mentioned categorizations. Due to this fact, the learning-
based methods will be categorized based on network architectures. The
summarization of studied approaches is presented in Table 1.
5. An overview of learning-based methods
There are some learning architectures that are mostly used in
learning-based registration methods, namely CNN, RNN, GNN, and
MLP. In the following, an overview of non-rigid registration methods
based on learning approaches will be discussed. Through this article, it
is tried to consider the most significant approaches which have more
citations in comparison with the rest of the studies as well as being
published in journals based on having a good reputation.
5.1. Convolutional neural network (CNN)
CNN is a class of neural networks that consists of one or more
convolutional layers. In other words, CNN is the regularized version
of a multi-layer perceptron. Various approaches have been proposed
to apply CNN for point cloud registration. However, using CNN for
S. Monji-Azad et al.
Fig. 3. An overview of the number of publications for learning-based point cloud registration approaches. The presented statistics includes both rigid and non-rigid transformations.
Fig. 4. An overview of feature-based approaches to solve point cloud registration problem.
point cloud registration has some challenges. To be more precise, unlike
images that are made of a regular grid, point clouds are a set of 3D
points without specific orders. Therefore, there are different approaches
to overcome these challenges such as converting the point cloud to a
2D image or 3D volumetric grids (Wu et al., 2019).
ProRegNet is a registration framework based on CNN which is
proposed in Fu et al. (2021). It applies the biomechanical constraints
to improve registration accuracy. ProRegNet has three main steps:
volumetric point cloud generation from input images, preparation of
training dataset based on the finite element (FE), and registration learn-
ing phase. In the first step, a CNN network is used for MR and TRUS
prostate segmentation based on Lei et al. (2019), Wang et al. (2019b).
Then, the segmented ROIs are meshed into tetrahedron elements to
extract the volumetric prostate point cloud. For the second step, which
is the training dataset generation, the finite element model is defined.
To this end, a surface point correspondence is established. Then, surface
point registration is done by predicting point cloud deformation based
on finite element modeling and biomechanical constraints.
A volume-to-surface registration network (V2S-Net) using a biome-
chanical model is proposed in Pfeiffer et al. (2020). In this way, CNN
has performed the registration step without searching for corresponding
points. To achieve this purpose, V2S-Net generates random meshes of
different organs synthetically to increase the network generalization.
To elaborate, V2S-Net registers the preoperative liver volume mesh on
the intraoperative point cloud of a partial liver surface. The meshes are
used as network input and are presented in the form of distance fields
on a regular grid. Learning surface representation makes it possible
61
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
to solves the registration problem without a requirement for a prior
alignment.
In another non-rigid registration study (Wang et al., 2019a), the
authors propose PR-Net for non-rigid registration based on the CNN and
learning the formulation of the correlation between source and target
point clouds. PR-Net learns the global and local features based on the
voxelized point cloud which is called the shape descriptor tensor in their
paper. In this paper, it is discussed that the defined shape descriptor
tensor is efficient and effective to extract geometric features of shapes.
In Wei et al. (2016), a non-rigid surface registration is proposed
which finds dense correspondences in the form of two depth maps
of surfaces. To achieve this purpose, the surface depth map and 204
points of extracted curves are extracted. They prove that using feature
descriptors on depth map pixels can lead to more accurate results for
registration.
DispVoxNets (Shimada et al., 2019) is a non-rigid registration ap-
proach that converts point clouds to voxel grid representations. The
approach has two main steps refinement and displacement estimation.
The method implements a U-Net-style architecture of CNN. In the first
step, the large global deformation is predicated. In the second step,
refinement of the displacement vector for resolving small displacement
is done.
PFCNN (Yang et al., 2020) is a customized convolutional network
that is proposed for different applications such as segmentation, classi-
fication as well as registration. PFCNN is a mesh grid-based convolution
framework. In this article, a new translation structure is proposed
which is similar to surface convolutions to learn features for mentioned
applications. For non-rigid registration, PFCNN classifies each input
S. Monji-Azad et al. ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72

Table 1
Overview of some learning-based registration methods.
Approach name (Ref) Year Network architecture Code availability Experimental data
ProRegNet (Fu et al., 2021 CNN Not yet Medical dataset
2021)
CPD-Net (Wang et al., 2019 PointNet - MLP https://github.com/nyummvc/CPD-Net Chui-Rangarajan (Chui and
2019c) Rangarajan, 2000), 3D face, 3D cat,
ShapeNet (Chair) (Chang et al.,
2015)
V2S-Net (Pfeiffer et al., 2020 CNN https://gitlab.com/ncttsopublic/Volume2SurfaceCNN Medical dataset
2020)
RMA-Net (Feng et al., 2021 RNN https://github.com/WanquanF/RMA-Net Human motion (Vlasic et al., 2008),
2021) TOSCA (Bronstein et al., 2008),
FaceWareHouse (Cao et al., 2013),
ModelNet (Wu et al., 2015),
SURREAL (Varol et al., 2017),
PR-Net (Wang et al., 2019 CNN https://github.com/Lingjing324/PR-Net Chui-Rangarajan (Chui and
2019a) Rangarajan, 2000), 3D face, 3D cat
(Hansen and Heinrich, 2021 DGCNN https://github.com/multimodallearning/deep-geo-reg Medical dataset
2021)
(Wei et al., 2016) 2016 CNN https://github.com/halimacc/DenseHumanBodyCorrespondences SCAPE (Anguelov et al., 2005),
Yobi3D (model search engine, 2022),
MIT (Tevs et al., 2012)
DispVoxNets (Shimada 2019 CNN https://vcai.mpi-inf.mpg.de/projects/DispVoxNets/ Thin plate (Golyanik et al., 2018),
et al., 2019) FLAME (Li et al., 2017), Dynamic
FAUST (DFAUST) (Bogo et al.,
2017), cloth (Bednarik et al., 2018).
GP-Aligner (Wang 2020 MLP Not yet ShapeNet (Chang et al., 2015)
et al., 2020b)
(Hansen et al., 2019) 2019 DGCNN Not yet Medical dataset
FPT (Baum et al., 2022 PointNet, MLP Not yet ModelNet (Wu et al., 2015)
2022)
SyNoRiM (Huang et al., 2022 CNN https://github.com/huangjh-pub/synorim Ma et al. (2020), Clothcap (Pons-Moll
2022a) et al., 2017), 4dcomplete (Li et al.,
2021c), Deepdeform (Bozic et al.,
2020), SAPIEN (Xiang et al., 2020)
(Trappolini et al., 2021 Transformer https://github.com/GiovanniTRA/transmatching SURREAL (Varol et al., 2017),
2021) FAUST (Bogo et al., 2014),
SHREC’19 (Melzi et al., 2019)
NDP (Li and Harada, 2022 MLP https://github.com/rabbityl/DeformationPyramid 4DMatch/4DLoMatch (Li and Harada,
2022b) 2022a)
NrtNet (Hu et al., 2022 DGCNN, Transformer Not yet SURREAL (Varol et al., 2017),
2022) SHREC’19 (Melzi et al., 2019), MIT
(Grosse et al., 2009)
PFCNN (Yang et al., 2020 CNN https://github.com/msraig/pfcnn FAUST (Bogo et al., 2014)
2020)
(Trimech et al., 2020) 2020 CNN Not yet ModelNet (Wu et al., 2015)
ResNet-LDDMM (Amor 2021 Residual networks Not yet SHREC’19 (Melzi et al., 2019)
et al., 2021)
(Netto and Oliveira, 2022 DGCNN Not yet ModelNet (Wu et al., 2015), TOSCA
2022) (Bronstein et al., 2008), Human
motion (Vlasic et al., 2008)
PR-GAN (Tang and 2021 GAN Not yet Chui-Rangarajan (Chui and
Zhao, 2021) Rangarajan, 2000)
(Shi et al., 2021) 2021 PointNet, MLP Not yet Medical dataset
FlowNet3D (Liu et al., 2019 PointNet++ https://github.com/xingyul/flownet3d FlyingThings3D (Mayer et al., 2016),
2019) KITTI (Geiger et al., 2012)
FlowNet3D++ (Wang 2020 PointNet++ Not yet FlyingThings3D (Mayer et al., 2016),
et al., 2020d) KITTI (Geiger et al., 2012),
KillingFusion (Slavcheva et al., 2017)
Lepard (Li and Harada, 2022 Transformer https://github.com/rabbityl/lepard 3DMatch (Zeng et al., 2017),
2022a) 3DLoMatch (Huang et al., 2021a)

mesh vertex into its corresponding vertex on the template mesh. This
approach is similar to the methods like (Fey et al., 2018; Boscaini
et al., 2016). In the second step, a pipeline is presented including
a sequence of convolutions for PFCNN as well as a two-level net-
work for MDGCNN (Poulenard and Ovsjanikov, 2018). MDGCNN is the
patch-based multi-directional geodesic.
A 3D facial expression recognition based on non-rigid point cloud
recognition is proposed in Trimech et al. (2020). The approach includes
a pre-processing step. To this end, 70 landmarks are extracted as well
as 204 points of extracted curves. Furthermore, they use SIFT for 3D
keypoint extraction. All mentioned features are used as the input of a
non-rigid CPD registration inspired by Trimech et al. (2016). Finally,

62
S. Monji-Azad et al.
Fig. 5. A taxonomy of graph convolutional networks from Zhang et al. (2019). This schema provides two points of view, GCNN in the application areas and GCNN based on
convolutional types.
the accuracy of the proposed method is reported for face expressions
recognition (FER) while there is no report for the accuracy of the
registration step.
5.2. Graph convolutional neural network (GCNN)
As discussed before, in the basic CNN, the main elements are
convolutional and pooling layers. These components in CNN can be
operated on structured data like images. In the literature, two impor-
tant characteristics are considered for structural data namely, Euclidean
and grid-like structures (Bronstein et al., 2017). Considering the non-
Euclidean characteristic of graphs, the convolutions and filtering on
graphs are not as simple as on images (Zhang et al., 2019). In Shuman
et al. (2013), an overview of signal processing on graphs is presented in-
cluding different approaches for graph analysis. To present an overview
of graph convolutional neural networks, a taxonomy of convolution
types and application areas is shown in Fig. 5. This taxonomy is taken
from Zhang et al. (2019). Regarding the mentioned taxonomy, several
aspects can be considered for GCNN. In this respect, the spectral-based
graph convolutions are defined based on graph Fourier transforma-
tion. Spatial-based graph convolutions focus on the nodes and their
neighborhood representations. GCNNs are a model of neural network
architectures which take the graph structure and neighboring nodes in a
convolutional manner into account. Recently, there are several studies
on using GCNNs for point cloud applications. Then, the main idea is
to learn geometric features which are extracted from point clouds with
neighborhood relations. This information is commonly defined on KNN
graphs, semantic labeling, and so on.
A key point-based method is presented in Hansen and Heinrich
(2021) for medical purposes. Point clouds are extracted from the input
scan pair. In the following, edge convolutions as a geometric feature
are combined with differentiable loopy belief propagation (LBP) (Ihler
et al., 2005). LBP is applied for the regularization of displacements
on a KNN graph. Therefore, they could solve 3D lung registration as
a geometric alignment of two point sets. The used GCNN structure
in Hansen and Heinrich (2021) is inspired by EdgeConv network ar-
chitecture (Wang et al., 2019). EdgeConv is one of the most applicable
methods which captures the graph’s local geometric structure. Edge-
Conv generates edge features in the manner of representing the relation
between the points and their neighbors. This method is robust to the
ordering of neighbors and permutation invariant.
In another study (Hansen et al., 2019) from the same authors
of Hansen and Heinrich (2021), another dynamic GCNN is proposed. In
the first step, the approach is using EdgeConv and similar architecture
to learn robust correspondences between the source and target point
sets. The second step is followed via probabilistic refinement. There-
fore, the learned features are applied to improve prior probabilities to
be used in the CPD algorithm.
SyNoRiM (Huang et al., 2022a) is a registration approach that uses
a point cloud graph as an input of a CNN network. In their problem
formulation, a fully connected graph is assumed. Then, the graph
vertices are input point clouds and the edges represent the graph con-
nectivity. SyNoRiM computes the pairwise correspondences which are
parameterized using functional maps. Although the SyNoRiM approach
could handle the occlusion problem by learning non-orthogonal basis
functions to regularize deformations, the approach still suffers from
scaling to large scenes (Huang et al., 2022b).
63
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
5.3. MLP and PointNet-based
Multilayer perceptron (MLP) is one of the network architectures
which is widely used in studies and is common for learning point cloud
features. However, the problem of processing unordered point sets is
a fundamental matter in this area. Hence, any network architecture
should take this problem into account. To this purpose, the PointNet
algorithm (Qi et al., 2017a) is a deep learning architecture based on
MLP that is able to directly consume the 3D points cloud. PointNet al-
gorithm can be used for classification and semantic segmentation tasks
by handling the unordered structure of its inputs. Considering the fact
that PointNet cannot present high accuracy to pinpoint local changes,
PointNet++ (Qi et al., 2017b) is presented to learn local features with
increasing contextual scales. PointNet (Qi et al., 2017a) and Point-
Net++ (Qi et al., 2017b) are used in many learning-based registration
methods as the feature descriptor for point clouds. These approaches
are pioneering methods for point cloud processing. Their results can be
used as the input of the point cloud registration methods. PointNet is
based on the multilayer perceptron (MLP). Then, in comparison with
CNN, the input is not challenging. After a while, another version of
PointNet called PointNet++ was proposed. Using hierarchical feature
aggregation for points was provided. Furthermore, PointNetLK (Aoki
et al., 2019) is another well-known method based on shape feature
learning and PointNet. PointNetLK is a rigid-based registration ap-
proach to learning the global feature representation of point sets. The
approach applies the Lucas–Kanade algorithm (LK) (Lucas et al., 1981)
for 3D point cloud registration in an iterative process with PointNet.
The authors improved the proposed method by using an analytical
Jacobian matrix and decomposing it to feature and warp components
in Li et al. (2021b). They use PointNet and the LK algorithm in one
recurrent neural network.
GP-Aligner (Wang et al., 2020b) is an approach to solve non-rigid
groupwise point cloud registration. More specifically, the first step of
GP-Aligner is an optimized Group Latent Descriptor (GLD) to charac-
terize the groupwise relation between a group of point sets. Therefore,
GLD is defined as a Gaussian distribution and in the following is con-
catenated with the coordinates of the points. Finally, after optimizing
the GLD results, an MLP network is used to learn coherent drifts as the
desired transformation.
Neural deformation pyramid (NDP) (Li and Harada, 2022b) is an-
other approach based on MLP architecture. In NDP, a pyramid archi-
tecture is proposed which each pyramid level contains a multilayer
perceptron. Each level of this hierarchical approach takes an input
of a sinusoidally encoded 3D point and generates the output of the
motion increments from the previous level. In this paper, it is proven
that the frequency of the sinusoidal function can represent non-rigidity.
Based on their definition, low frequencies represent rigid motion while
high frequencies produce more fluctuations. Then, it is suitable for
representing highly non-rigid motion.
In another study, an unsupervised learning method of the geometric
non-rigid transformation based on coherent point morph drift (CPD-
Net) is proposed in Wang et al. (2019c). CPD-Net has three main steps.
Learning the global shape descriptor is the first step based on the
PointNet network. For the second step, the method concatenates the
S. Monji-Azad et al.
achieved descriptors to the source point cloud. Finally, by using another
MLP architecture, the drifts of each source point are learned.
Another approach based on MLP and PointNet is proposed in Shi
et al. (2021). This method estimates points drift by learning the dis-
placement field function and omitting an additional iterative optimiza-
tion process. Furthermore, FPT (Baum et al., 2022) is a multi-modal
biomedical method based on two main steps. The first step is a global
feature extractor based on PointNet architecture and the second step is
a point transformer network based on Baum et al. (2021). This network
is to learn a unique displacement vector without any smoothness or
coherence constraints.
There is another registration method is called FlowNet3D (Liu et al.,
2019) to learn scene flow in 3D point clouds. In the FlowNet3D
article, two applications for scene flow output are presented, scan
registration and motion segmentation. In the first frame to indicate
the motion between two frames, FlowNet3D estimates a transnational
flow vector for every point, individually. To this aim, they propose
two new learning layers. One layer is called the flow embedding layer
which learns the correlation of two point sets. Another layer is called
the upconv layer which learns to propagate features from one point
set to the other one. However, in Guo et al. (2020) two problems
with the FlowNet3D approach are mentioned. One is that FlowNet3D
cannot predict all motion vectors correctly. Therefore, some motion
vectors have significant differences in comparison with ground truth in
their direction. The second problem is that FlowNet3D cannot handle
non-static scenes like deformable objects. Hence, FlowNet3D++ (Wang
et al., 2020d) is proposed to solve the mentioned problems by minimiz-
ing the angle between the predicted motion vector and the ground truth
ones. For solving the second mentioned problem, FlowNet3D++ uses
a point-to-plane distance loss to achieve better accuracy for dynamic
scenes.
5.4. Transformers
A transformer is a deep learning model which is introduced to
process sequential input data by a pioneering work (Vaswani et al.,
2017). The transformers mechanism is quite similar to the recurrent
neural network (RNN), however, in contrast to RNN, the whole input
is processed with transformers once. As an example, BERT (Devlin
et al., 2018) is one of the most important approaches which shows how
transformers can be used in the field of natural language processing.
For computer vision applications, Dosovitskiy et al. (2020) is a method
that shows transformers’ capability in comparison with CNN. The pio-
neering transformer approaches, which have used point clouds as input,
are presented in Zhao et al. (2021a), Guo et al. (2021), to mention
a few. Although there are different studies on using transformers for
classification or segmentation, the number of studies on point cloud
registration based on transformers is limited. One of the first methods
using transformers for rigid point cloud registration is deep closet point
(DCP) (Wang and Solomon, 2019a). DCP contains three steps: firstly,
a point cloud embedding network for feature extraction is suggested,
which is based on dynamic GCNN. In the second step, the method has
an attention-based module to show the relationship between source
and target point clouds. As the last step, to calculate rigid transforma-
tion, which includes rotation and translation, a differentiable singular
value decomposition is used. Furthermore, for computing the feature
representation, DCP applies a transformer network (Vaswani et al.,
2017) to provide global and local information. Due to the fact that the
focus of this article is on non-rigid registration, in the following, some
registration approaches which are based on transformer architecture
will be discussed.
A non-rigid registration approach is proposed in Trappolini et al.
(2021) which is based on an auto-encoder architecture and a trans-
former network. The transformer is applied as a geometric translator
between two point sets. Furthermore, the attention mechanism named
64
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
surface attention is modified to make it possible to be aware of the un-
derlying density of the geometry. Considering the refinement procedure
in the transformer network architecture, this approach is suffering from
long training and post-processing time as well.
NrtNet is another non-rigid registration method based on trans-
formers proposed in Hu et al. (2022). NrtNet uses a dynamic GCNN
architecture as a feature extractor and then learns correct correspon-
dences using a transformer-based framework. They claim that NrtNet
is able to learn correspondences between two dense point clouds. Then,
the last module calculates the relative drift of the point pairs to solve
the registration problem.
Lepard (Li and Harada, 2022a) is another registration method based
on transformers. Lepard is known as a position-aware feature-matching
method. The proposed method is built of a fully convolutional feature
extractor network, called KPFCN (Thomas et al., 2019). Furthermore,
for the matching step, a transformer architecture with self and cross-
attention is used, which is inspired by Vaswani et al. (2017). Finally,
the differentiable matching step has been done to solve the registration
problem using (Sarlin et al., 2020; Sun et al., 2021).
5.5. Other network architectures
As discussed, although there are several deep-learning architectures
that are most popular in recent studies, the existing approaches are
not limited to the mentioned ones. In the following, a few more
non-rigid registration methods based on different architectures are
discussed, namely, generative adversarial network (GAN), recurrent
neural network (RNN), and deep residual networks (ResNet).
GAN is presented for the first time in VGoodfellow et al. (2014).
The GAN architecture is formed of two neural networks which contest
with each other, namely a generator and a discriminator. This contest
is in the form of a zero-sum game. To this end, one agent’s gain is
another agent’s loss. The generator produces fake samples, while the
discriminator determines the generated faked samples from the real
data. This process will be continued as long as the discriminator cannot
distinguish the fake data. Another GAN version, called conditional
GAN, is proposed in Mirza and Osindero (2014). This architecture
allows adding some constraints to generated data (e.g., class labels),
regarding the model’s expected outputs.
In Tang and Zhao (2021), a conditional GAN for non-rigid point
set registration is proposed which is called PR-GAN. In the first step,
considering the irregular format of point clouds, the proposed method
uses an autoencoder to overcome this challenge. The output of the
autoencoder is used for PR-GAN architecture. The generator in PR-GAN
aims to generate the parameters of the geometric transformation while
the discriminator applies to force the generated parameters to register
two-point clouds accurately.
RNN is a type of neural network which is designed to interpret
sequential information. There are different variants of RNN, e.g., long
short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997) and
gated recurrent unit (GRU) (Cho et al., 2014). They are suitable to
process time series data. However, using RNN as an architecture for
point cloud processing carries some challenges. Firstly, RNN learns a
one-dimensional vector that is not suitable to represent the entire point
cloud. One solution is to make a flat representation of the point cloud.
However, this can change the data structure and cause the omission of
the local structure. Extending the one-vector representation to a two-
dimensional representation is another solution. Secondly, with regard
to the unordered structure of the point cloud, concatenation cannot be
applied. Then, one solution is to concatenate the point feature, the state
of the neighbor, and the displacement from the neighbor to the point.
This can be done for each neighbor separately (Fan and Yang, 2019).
Estimating a non-rigid transformation by a combination of 𝑁 rigid
transformation based on the recurrent network is proposed in Feng
et al. (2021) (RMA-Net). RMA-Net defines a skinning weight for each
point to show the influence of the rigid transformation. Therefore, the
S. Monji-Azad et al. ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72

Table 2
Some available benchmark for the point cloud registration.
Dataset name (Ref) Year Synthetic/Real Included data URL
3DLoMatch (Huang 2021 Real A collection of 62 scenes, with overlap https://overlappredator.github.io/
et al., 2021a) ratios between 10 percent to 30 percent
Faust (Bogo et al., 2014 Real 300 triangulated meshes of 10 various http://faust.is.tue.mpg.de/overview
2014) subjects, five male and five female. Each
scan is in 30 different poses
SCAPE (Anguelov et al., 2005 Synthetic 71 registered meshes of a particular person http://ai.stanford.edu/~drago/Projects/scape/scape.html
2005) in different poses
4DMatch/4DLoMatch 2022 Synthetic Randomly selected 1761 sequences from https://github.com/rabbityl/lepard
(Li and Harada, 2022a) DeformingThings4D, 4DLoMatch has low
overlap in comparison with 4D match

skinning weight and a rigid transformation are estimated by RNN at
each iteration. In this paper, it is proven that the proposed method is
appropriate for both rigid and non-rigid transformation.
The residual network (ResNet) (He et al., 2016) is an architecture
based on a new technique called skip connection. The skip connection
is a name for an operation of skipping some layers to connect a
layer to the further one, which makes a residual block. This process
can solve two problems, namely vanishing gradients and degradation
(accuracy saturation) problems. For non-rigid point cloud registra-
tion, an approach called ResNet-LDDMM is proposed in Amor et al.
(2021). ResNet-LDDMM is a ResNet-based diffeomorphic registration
approach to register 3D shapes and 3D point clouds. This approach
solves the non-stationary ordinary differential equation (ODE)(flow
equation) based on Euler’s discretization scheme. Considering the fact
that ResNet speeds the learning process by reducing the impact of
vanishing gradients, the ResNet-LDDMM method takes the advantage
of this fact to reduce the computation cost.
6. Datasets
To show the accuracy of point cloud registration methods and to
study how robust they are to the existing challenges, several datasets
with different characteristics have been published. These characteristics
can provide some information, such as color and depth of point clouds,
or some 3D geometric information such as pose, alignments, parts,
and key points, or symmetry of point clouds. Some datasets provide
language-related or semantic annotations of point sets and some others
provide segmentation data. It is of great importance to mention that
this information is not available for all datasets at the same time. De-
pending on which challenges are most important for a method to solve,
the dataset can be selected. In addition, the specific characteristics of
the datasets have led them to be able to be used for other applications
as well as point cloud registration such as reconstruction, segmentation,
and recognition.
However, despite all these useful details, the lack of some informa-
tion, e.g., the models’ complexities or deformation level makes a fair
comparison between the state-of-the-art methods difficult. In this way,
the articles have customized the available benchmarks. Depending on
which challenges are going to be solved, a new customized dataset will
be generated. For instance, for reporting how robust to noise or outlier
an algorithm is, some noise or outlier will be added to the dataset.
As another example, the studied articles generate their own dataset in
different deformation levels.
As it is shown in Table 1, the studied approaches are using different
datasets. It is worth mentioning that only a few benchmark datasets
are available for point cloud registration. However, there are multiple
datasets that are generated for other purposes like reconstruction or
matching, which can be used for registration purposes as well.
The most frequently used datasets of the studied methods, in Sec-
tion 5, will be discussed in the following. The datasets can be catego-
rized into two main groups: real-world datasets and synthetic ones. The
characteristics of the most used datasets as well as the benchmark ones
are provided in Table 2.
6.1. Real-world datasets
The datasets that represent some objects in the real scene and
provide some knowledge about them accordingly are called real-world
datasets. Usually, they are captured by using navigating robots
equipped with cameras, LiDAR cameras, RGB-D cameras, or some other
depth cameras (Li et al., 2019). A real-world dataset can include large
or small-scale areas or indoor/outdoor environments (Handa et al.,
2016). In the following, some real-world datasets, which are common
for point cloud registration in the studied articles, will be discussed.
Cloth (Bednarik et al., 2018), is one of the used datasets which con-
tains different states of deformable objects including paper, sweaters,
hoodies, cloth, and T-shirts. It presents non-linear deformations be-
tween the source and target surfaces. Meshes, normal vectors, and
depth maps are available for the Cloth dataset. Deepdeform (Bozic
et al., 2020) is another dataset that includes partial views captured with
the real RGB-D camera. KITTI (Geiger et al., 2012) is an outdoor real-
world dataset that is captured by using a Velodyne HDL64 3D LiDAR
scanner in Karlsruhe, Germany. There are 11 sequences which include
the grayscale dataset, color dataset, Velodyne laser data, calibration
files, and ground truth poses. The available information makes this
dataset usable for 3D object detection and 3D tracking, too. Further-
more, KillingFusion (Slavcheva et al., 2017) is another dataset that is
used for registration, but this dataset is a benchmark for non-rigid 3D
reconstruction without correspondences.
SHREC is the name of two different datasets, namely SHREC’19
(Melzi et al., 2019) and SHREC’20 (Dyke et al., 2020), which are widely
used both for matching and registration problems. SHREC datasets
are generated for shape correspondences purpose of isometric and
non-isometric deformations. SHREC’19 (Melzi et al., 2019) includes
humans’ real scans under various deformations, namely articulating,
bending, stretching, and topologically changing as well as the ground
truth of correspondences. In comparison with SHREC’19 (Melzi et al.,
2019), SHREC’20 (Dyke et al., 2020) includes a set of synthetic models
and real-world scans, e.g., four-legged animals.
In the following, some datasets which are known as the benchmark
for registration are presented.
3DMatch (Zeng et al., 2017) is a dataset of a real-world scene.
It is provided in two categories: keypoint matching benchmark and
geometric registration benchmark. The keypoint matching benchmark
contains both 2D and 3D RGB-D patches of some base-lined correspon-
dences. Depth and color information is available for this dataset which
includes 62 RGB-D scenes. The geometric registration benchmark also
includes real-world RGB-D data scanned by Microsoft Kinect and Intel
RealSense. It includes eight sets of scene fragments. Each fragment is
a 3D point cloud of a surface. It is made of 50-depth frames. Beside
registration, this dataset can be used for reconstruction, model align-
ment, and surface correspondence. 3DLoMatch (Huang et al., 2021a) is
another real-world dataset for the registration of 3D point clouds with
overlap ratios between 10 percent to 30 percent.
Faust (Bogo et al., 2014) is another real-world scene dataset. It
contains 300 triangulated meshes of 10 various subjects, five males and

65
S. Monji-Azad et al.
five females. Each scan is in 30 different poses. This dataset represents
the real human meshes with ground-truth correspondences. A full-body
3D stereo capture system composed of 22 modular scanning units is
used to produce the Faust dataset. The number of vertices is about
6890. Dynamic FAUST (DFAUST) (Bogo et al., 2017) is a benchmark for
registering human bodies in motion. DFAUST includes scanned meshes
of humans in different positions.
6.2. Synthetic datasets
The synthetic dataset is the information artificially generated using
simulations or computer algorithms. Synthetic datasets can be gen-
erated as ground truth. Regarding this, they can be ideal for those
applications that their real-world datasets can be difficult to gen-
erate. For instance, in surgical applications, generating a real-world
dataset can be difficult. Then, synthetic datasets can be an appropriate
replacement approach.
Considering the fact that there are some challenges in real-world
datasets such as noise, camera distortion, etc., synthetic datasets are
generated to provide a benchmark for point cloud registration methods.
To this aim, there are several datasets generated for different 3D topics,
like reconstruction, segmentation, or matching purposes.
One of the oldest popular datasets for deformable object registration
is Chui–Rangarajan (Chui and Rangarajan, 2000). Chui–Rangarajan is
a synthetic dataset that includes two different point clouds: Chinese
characters with 105 points and fish shapes with 98 points.
TOSCA (Bronstein et al., 2008) is another synthetic dataset that
collects 3D non-rigid shapes in different poses. It contains 80 objects
and the number of vertexes for each object is about 50000. The dataset
includes some shapes of different animals like cats, dogs, wolves,
horses, centaurs, gorillas, and some shapes of the human figure like
female and two different male figures with different poses.
MIT (Tevs et al., 2012) is a synthetic dataset that contains three var-
ious characters from different animation sequences. The ground truth of
correspondence points is available, which can be used in the matching
problem. Furthermore, Thin plate (Golyanik et al., 2018) is one dataset
that contains various synthetic isometric surfaces. Also, there is a free
3D model search engine called Yobi3D (model search engine, 2022)
which is used in Wei et al. (2016) paper. FlyingThings3D (Mayer et al.,
2016) is a large dataset for disparity, optical flow, and scene flow
estimation. This dataset is generated based on learning scene flow in
3D point cloud, which is used in some methods like (Liu et al., 2019)
and (Wang et al., 2020d). Although the main goal of these methods is
learning scene flow estimation, non-rigid registration is the main step
in these algorithms.
One of the most popular topics for registration is about the human
body, human dress, or human facial expressions. Considering the fact
that the mentioned areas need to take deformation difficulties into
account, the related datasets are appropriate to show the accuracy
of the non-rigid registration methods. According to the mentioned
points, Human motion (Vlasic et al., 2008) is the name of a dataset
that provides articulated mesh animation from multi-view silhouettes.
Considering that captured meshes are in full correspondence, make it
usable for matching problems as well as matching-based registration
methods. Some information like texture and deformation transforma-
tion are also available. Furthermore, FaceWareHouse (Cao et al., 2013)
is a dataset that includes 150 different face shapes with 47 various
expressions. Also, FLAME (Li et al., 2017) is another one that consists of
different captured human facial expressions, including 10k face meshes.
Clothcap (Pons-Moll et al., 2017) is a synthetic dataset that consists
of clothed humans scanned by a 3dMD scanner. 4dcomplete (Li et al.,
2021c) is a dataset for non-rigid motion estimation which is also used
for registration purposes as well.
SURREAL (Varol et al., 2017) is one of the available benchmarks for
matching problems with proving the corresponding ground truth. To
this end, the photo-realistic synthetic images and their corresponding
66
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
ground truth are available, which can be used for learning pixel-wise
classification as well as matching and non-rigid registration problems
like (Trappolini et al., 2021; Hu et al., 2022). Human part segmentation
and depth estimation are other applications for this dataset.
SCAPE (Anguelov et al., 2005) is a benchmark for the matching
problem and non-rigid registration by providing the full objects’ cor-
respondences. 71 registered meshes of a particular person in different
poses are available in this dataset.
4DMatch/4DLoMatch (Li and Harada, 2022a) are two benchmarks
for registration and matching problems both for being used in rigid and
deformable scenes. 4DMatch is a partial point cloud benchmark while
its low overlap version is called 4DLoMatch. 4Dmatch is captured by
using the sequence from Li et al. (2021c) dataset. The ground truth
dense correspondence is available as well. The availability of time-
varying geometry in both datasets makes it possible to show more
challenges in matching and registration applications
7. Evaluation performance
As discussed before, finding a transformation between two or more
point clouds and considering that how the challenges are overcome
is the main goal of the point cloud registration methods. Some of
these challenges have been mentioned in the previous sections. In this
section, the challenges will be discussed in detail and it will show how
methods can evaluate their robustness to the challenges. Performance
evaluation of point cloud registration methods will be reported through
different criteria. Quantitative assessment metrics and robustness are
the most common criteria. These will be explained in detail in the
following. Moreover, the summarization of the performance evaluation
of studied papers is presented in Table 3.
7.1. Quantitative assessment metrics
The quantitative analysis of the algorithm and the assessment of
its performance are the most significant measurements to evaluate a
method. As it is discussed in the problem definition in Section 3, the
loss function and distance metric should be minimized. The available
point cloud registration methods use different quantitative assessment
metrics to show the performance of their methods. Some of them are
based on distance metrics meanwhile others are based on transforma-
tion errors. Beside reporting distance and loss function, some articles
report matching errors to show how robust the methods are to the
outlier and noise. These types of error metrics will be studied in the
robustness section.
The quantitative assessment metrics as well as robustness to the
common challenges for learning-based and non-rigid 3D point cloud
registration are shown in Fig. 6.
Quantitative assessment metrics based on distance: some meth-
ods calculate the distance between the transformed source point clouds
and target ones and then report it as a quantitative evaluation (Peterlík
et al., 2018). Chamfer distance (CD) is one of the metrics that is widely
used in papers. The mean neighbor distance, the Hausdorff distance
(HD), the Euclidean distance, the chordal distance, the Earth-Mover dis-
tance (EMD), and the Mahalanobis distance are some common distance
measurements in the articles. In Fu et al. (2021), the Hausdorff distance
(HD) is used to calculate the distance between two input surfaces. They
also use mean surface distance (MSD) to calculate the distance between
two input surfaces. In Liang et al. (2018), a distance metric based on
a combination of the distance term and the energy term is used. The
registration loss is calculated based on misaligned points between the
source and the transformed point clouds in Pais et al. (2020). In Liu
et al. (2019) and Wang et al. (2020d), the L2 distance between the
estimated flow vector to the ground truth flow vector is considered
to report the accuracy. Furthermore, according to the similarity of the
source and target point clouds after deformation, the distance loss is
defined as 𝐿𝑑𝑖𝑠 in Hu et al. (2022). There is another distance metric
S. Monji-Azad et al. ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72

Table 3
Evaluation performance and metrics - some learning approaches.
Approach name (Ref) Year Evaluation metrics Robustness
ProRegNet (Fu et al., 2021 Hausdorff distance, MSD, Mean, STD Outliers, Noise
2021)
CPD-Net (Wang et al., 2019 Chamfer distance Deformation levels, Noise, Outliers, Incompleteness
2019c)
V2S-Net (Pfeiffer et al., 2020 MDE, MTD Noise
2020)
RMA-Net (Feng et al., 2021 MSE, RMSE, MAE, Chamfer Distance, Deformation level, Outliers, Incompleteness
2021) EMD
(Wei et al., 2016) 2016 Mean Partial overlapping, Outlier, Data density variation,
Deformation level
DispVoxNets (Shimada 2019 RMSE, STD Noise, Deformation level, Outlier, Incompleteness
et al., 2019)
GP-Aligner (Wang 2020 Chamfer distance Noise, outlier, Incompleteness, Deformation level
et al., 2020b)
(Hansen et al., 2019) 2019 TRE Noise, Outlier
FPT (Baum et al., 2022 RMSE Partial overlapping
2022)
SyNoRiM (Huang et al., 2022 L2 distance, MAE, EPE, Acc Noise, Incompleteness, Outlier, Data density
2022a) variation
(Trappolini et al., 2021 Mean, MGO, Chamfer distance, Max Noise, Data density variation
2021) EU, Mean EU
NDP (Li and Harada, 2022 EPE, Acc, Outlier, Partial overlapping, Data Density variation
2022b)
NrtNet (Hu et al., 2022 Percentage of correct correspondence, Deformation levels
2022) Chamfer distance, 𝐿𝑑𝑖𝑠 , 𝐿𝑚𝑎𝑡
PFCNN (Yang et al., 2020 Geodesic error, mIoU, mA, oA Data density variation
2020)
ResNet-LDDMM (Amor 2021 Geodesic error Noise, Incompleteness, Partial overlapping,
et al., 2021) Articulated deformations, Near-isometric
deformations, Non-isometric deformations,
Topological and geometric changes
(Netto and Oliveira, 2022 Precision, recall, EPE, MAE, MIE Noise, Outlier, Incompleteness, Partial overlapping,
2022) Data density variation, Deformation levels
PR-GAN (Tang and 2021 MSE, Chamfer distance Noise, Outlier, Deformation levels
Zhao, 2021)
(Shi et al., 2021) 2021 Chamfer distance Deformation levels
FlowNet3D (Liu et al., 2019 EPE, Acc, L2 distance Partial overlapping, Noise, Outlier, Data density
2019) variation, Different number of sampled points
FlowNet3D++ (Wang 2020 EPE, Acc, L2 distance, ADE, Mean Noise, Outlier, Data density variation
et al., 2020d)
Lepard (Li and Harada, 2022 IR, FMR, Recall, RRE, RTE Outlier, Data density variation, Partial overlapping
2022a)

which is called geodesic distance. A geodesic is a curve representing the
shortest path between two points on a surface. Some articles like (Yang
et al., 2020; Amor et al., 2021) represent the geodesic distance to report
their methods’ accuracies.
The root-mean-squared error (RMSE) is another common approach
to report the error of the registration method. The mean squared error
(MSE) and the mean absolute error (MAE) are two other metrics which
can be used to estimate the algorithm’s error. The root-mean-squared
distance (RMSD) is a distance metric proposed in Hirose (2020) for
evaluating the method. Mean displacement error (MDE) and mean
target displacement (MTD) are two other measurements that are in-
troduced in Pfeiffer et al. (2020). Likewise, in Hansen et al. (2019)
target registration error (TRE) is proposed to evaluate registration
accuracy. This error is called End-point-error (EPE) in some articles
like FlowNet (Liu et al., 2019) and FlowNet++(Wang et al., 2020d).
ResNet (Amor et al., 2021) reports the mIoU which is the class mean
intersection over the union as well as the mA which is the class
mean accuracy. Furthermore, ResNet presents the oA to report the
overall accuracy. The mean isotropic error (MIE) is another error mea-
surement that is reported in Netto and Oliveira (2022). Furthermore,
to present the average of angle deviation error, ADE is reported in
FlowNet++(Wang et al., 2020d).
As discussed before, some articles find correspondences before the
registration step. Then, usually they report the corresponding error as
well as transformation and distance errors. Some registration methods
like Le et al. (2019), Chaudhury (2020), and Ma et al. (2017) consider
the number of matches to report the accuracy of their methods. This
measurement is called inlier ratio (IR) in Li and Harada (2022a). In
SyNoRiM (Huang et al., 2022a) and NDP (Li and Harada, 2022b), two
accuracy measurements are defined, namely Accuracy Strict (AccS) and
Accuracy Relaxed (AccR). The Accuracy Strict (AccS) is the percentage
of points whose relative error is less than 5% or greater than 2 cm.
The Accuracy Relaxed (AccR) is the percentage of points whose relative
error is less than 10% or greater than 5 cm. In He et al. (2017), the
matching error is calculated based on the corresponding point and the
number of iterations. The mean and standard deviation (SD) are two
other factors to show the accuracy of a registration algorithm.
Another way to compute the accuracy of a point cloud registration
method is using precision and recall measurements. In fact, by consid-
ering different distance threshold values, the correct match for each
point correspondence can be defined. Then, by using this information,
precision and recall will be calculated (Chaudhury, 2020). In Li and
Harada (2022a), the same concept is called feature matching recall
(FMR).

67
S. Monji-Azad et al.
Fig. 6. (a) Quantitative assessment metrics (b) Robustness to the common challenges.
Quantitative assessment metrics based on transformation er-
ror: as discussed before, to solve the point cloud registration problem,
some methods calculate the transformation error between source and
target point sets. For instance, some articles report rotation and trans-
lation errors, like Li and Harada (2022a). This paper defines relative
rotation error (RRE) and relative translation error (RTE). Some others,
like Eckart et al. (2018), report average Euler angular deviation from
the ground truth. The convergence rate is calculated as the algorithm
accuracy in Ahmed et al. (2021). The converge rate definition in this
article is based on translation and rotation error. To be more precise,
if the mentioned error is less than 5 degrees for rotation and 5 cm for
translation, then the convergence rate can be defined as 100 percent.
The isotropic error for both rotation and translation (ISO) (Bauer et al.,
2021), geometric loss (Huang et al., 2020), the absolute trajectory error
(ATE) (Ding and Feng, 2019), and area under the curve (AUC) (Sarode
et al., 2019) are some other error evaluation metrics that are used in
the mentioned articles. Furthermore, in Hu et al. (2022), the matrix
optimization loss formula is used which is called 𝐿𝑚𝑎𝑡 .
As the research has demonstrated, the quantitative assessment met-
ric based on distance is the most important and common measurement
to evaluate the point cloud registration problem. To this end, the
distance between the transformed source point cloud and the target one
is calculated. Chamfer distance (CD) and Hausdorff distance (HD) are
two widely used distance functions. In other studies, the registration
methods are based on finding the corresponding points. Therefore,
calculating the percentage of correct corresponding points beside the
registration error, e.g. distance metric, is a significant measurement to
report. Furthermore, reporting the percentage of correct corresponding
points can be helpful to study how robust the methods are to noise
and outliers. Despite using the two mentioned measurements, when
the approaches calculate the transformation error, the geometric loss
between the predicted transformation matrix and ground truth one can
be used as a measurement metric.
7.2. Robustness
In the second criterium of evaluating the available approaches, the
robustness to the challenges will be studied. Different challenges that
various methods should be robust to them are shown in Fig. 6, (b).
For instance, the data captured using LiDAR, scanners, or real-world
cameras suffer from various types of noise. Therefore, some papers
provide their achievements when the dataset has a different level of
68
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
noise to show how robust they are to noise. Another important factor is
how robust the method is to outliers. In some articles like Shimada et al.
(2019), Wang et al. (2020a) methods’ robustness to outliers is studied.
Robustness to occlusion is shown in Li et al. (2020b) meanwhile
robustness to incompleteness is evaluated in Bernreiter et al. (2021).
For another example, in some applications, the occlusion between the
objects in the scene can be a challenge. Partial registration is another
difficulty, which happens when from one point of view just some parts
of the object appear. Regarding these aspects, some measurements are
used to show how robust a registration method is to various challenges.
Some approaches such as Yang et al. (2016), Ge (2016), and Billings
et al. (2015) report the failure or success rate of the correspondences.
This can show how robust to outlier and noise the method is. Although
partial point cloud registration methods are not considered as the main
topic in this survey, some articles, e.g., (Chen et al., 2020; Huang
et al., 2020; Min et al., 2021; Wu et al., 2021), show how robust their
methods can be to partial registration as well. Some methods like Li
et al. (2020b) and Yang et al. (2018) increase the level of deformation
to show that their method can overcome large deformation as well.
Robustness to data density variation or the size of data is studied
in Ahmed et al. (2021), Min et al. (2021), Ao et al. (2021).
8. Discussion
Based on the studies shown in this article, some points can be
concluded. In the following, first, the existing challenges are discussed.
It is shown how different approaches can be robust to them. Then, an
overview of datasets challenges is provided. Finally, different learning
architectures of point cloud registration are going to be discussed.
As discussed before, data incompleteness, partial overlapping, noise,
and outlier are discussed more in the articles. To this end, finding the
transformation, when some correspondences are missed is challenging.
To overcome this challenge, the available approaches are categorized
into correspondence-free or correspondence-based methods. However,
both mentioned approaches suffer from some issues. The differences
between global features of the two point sets can be the main challenge
of correspondence-free methods while dealing with the lack of some
correspondences is the primary problem of correspondence-based meth-
ods. Detecting keypoint correspondences can be a solution to overcome
this challenge (Wang and Solomon, 2019b). Additionally, using the
deep neural network to find the alignment of incomplete shapes can
be another approach (Hanocka et al., 2018). Although there are some
S. Monji-Azad et al.
suggestions to overcome these challenges, data incompleteness, and
partial overlapping are still open issues for the point cloud registration
problem. Robustness to noise and outliers, as two other challenges,
are still studied to evaluate different registration methods. The first
approach is to overcome these challenges by improving the feature
extraction phase or using different methods to find the corresponding
points. For instance, some methods use the feature extractor, which is
robust to noise such as RANSAC variants. Therefore, they can improve
the registration method’s accuracy by finding the correct corresponding
points. Nevertheless, the second group of studies such as ICP and its
variants can solve the registration problem without feature extraction
and different descriptors. Both mentioned aspects have their own ad-
vantages and disadvantages. Therefore, selecting one of the mentioned
approaches depends on the circumstances of the problem, which must
be solved.
Despite the fact that there are different datasets used for evaluating
the point cloud registration methods, a number of challenges exist, such
as deformation level, noise level, etc., which are not presented in the
available datasets. Therefore, the lack of a benchmark that includes
all mentioned challenges can be discussed. In this fashion, the articles
generate their customized datasets with different challenges to evalu-
ate proposed registration methods. For example, different deformation
levels, different noise levels, or different outlier levels are generated
in each study based on the authors’ opinions. Consequently, without
considering how the dataset is generated, it can be impractical to
compare the results of the articles just by using the reported numbers.
As another challenge, it can be said that the most available datasets do
not provide the transformation ground truth between source and target
point clouds. In the upcoming work, generating a dataset that presents
all challenges, the train, as well as the test data categorizations can be
considered as future work.
Regarding various studies solving point cloud registration based
on the learning approach, a discussion between different architectures
can be presented. As it is discussed, CNN could achieve some success
to improve the generalization in image applications by applying the
convolutional filters on the whole image and reducing the number of
parameters. However, using CNN for unordered 3D data, such as point
clouds, has some challenges that can be solved if it converts to a 2D
image or 3D volumetric grids. Another group of studies is based on
PointNet. Then, in comparison with CNN, the input is not challenging.
However, it suffers from handling the local registration. As a solution,
there are some methods that segment the point cloud into different
regions and estimate the transformation for each region separately (Mei
et al., 2022; Truong et al., 2019). Although this approach can handle
local registration, it is suffering from segmentation challenges as well as
finding the appropriate transformation model for each region. Having
said that, graph neural network (GNN) has attracted considerable
research interest in recent years. GNN could achieve good accuracy
by learning the neighborhood information of points and presenting
the local features. For the future of deep learning methods, regarding
all mentioned advantages of GNN, it is not unexpected to see more
novel graph-based approaches. Using transformer-based architectures is
another recent trend in learning-based registration methods. However,
they still suffer from the generalization problem. In this way, there are
some approaches like Trappolini et al. (2021), which improve the gen-
eralization of the network but still suffer from being time-consuming.
Furthermore, in some recent studies, it is shown that using different
representations of the point cloud can be an efficient solution for some
mentioned challenges of the registration problem. For instance, Peng
et al. (2021) as a learning-based method considers a grid representation
like mesh to tackle the problem of finding correspondences between
two surfaces. In addition, in some datasets, additional features like
color for each point are available. Then, using texture and color infor-
mation as well as point set coordinates can be other upcoming solutions
for the learning-based registration methods.
69
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
9. Conclusion
Point cloud registration is one of the fundamental topics that can be
used in many applications, such as 3D reconstruction, pose estimation,
augmented/virtual reality, etc. Traditionally, finding the spatial trans-
formation between two corresponding point sets is the goal of point
cloud registration methods. The spatial transformation was defined into
rigid and non-rigid transformation categories. In this review paper,
non-rigid transformations and learning-based 3D point cloud registra-
tion methods were studied. Statistics of the recent developments and a
comprehensive review of the most important approaches were provided
in this article. Some of the existing challenges and their robustness
were reviewed. Additionally, the most used datasets and a review of
existing benchmarks which are common in the articles were studied
and compared. By the end of the paper, a discussion on challenges
and some of their existing solutions, datasets, and different learning
architectures was presented.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared to
influence the work reported in this paper.
Acknowledgments
The authors of this paper would like to thank Dr. Mehran Fotouhi
for his valuable comments and suggestions to improve the work.
The authors gratefully acknowledge the data storage service
SDS@hd supported by the Ministry of Science, Research and the Arts
Baden-Württemberg (MWK) and the German Research Foundation
(DFG) through grant INST 35/1314-1 FUGG and INST 35/1503-1
FUGG.
This work was partially funded by Zentrales Innovationsprogramm
Mittelstand (ZIM), Germany under grant KK5044704CS0.
References
Ahmed, M.T., Ziauddin, S., Marshall, J.A., Greenspan, M., 2021. Point cloud registration
using virtual interest points from Macaulay’s resultant of quadric surfaces. J. Math.
Imaging Vision 63 (4), 457–471. http://dx.doi.org/10.1007/s10851-020-01013-z.
Alhamzi, K., Elmogy, M., Barakat, S., 2015. 3D object recognition based on local and
global features using point cloud library. Int. J. Adv. Comput. Technol. 7 (3), 43.
Amor, B.B., Arguillère, S., Shao, L., 2021. ResNet-LDDMM: Advancing the LDDMM
framework using deep residual networks. arXiv preprint arXiv:2102.07951.
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J., 2005. SCAPE:
Shape completion and animation of people. ACM Trans. Graph. (ToG) (ISSN:
0730-0301) 24 (3), 408–416. http://dx.doi.org/10.1145/1073204.1073207.
Ao, S., Hu, Q., Yang, B., Markham, A., Guo, Y., 2021. Spinnet: Learning a general
surface descriptor for 3D point cloud registration. In: Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition. pp. 11753–11762.
Aoki, Y., Goforth, H., Srivatsan, R.A., Lucey, S., 2019. Pointnetlk: Robust & efficient
point cloud registration using pointnet. In: Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition. pp. 7163–7172. http://dx.doi.org/10.
1109/cvpr.2019.00733.
Bauer, D., Patten, T., Vincze, M., 2021. Reagent: Point cloud registration using
imitation and reinforcement learning. In: Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition. pp. 14586–14594.
Baum, Z.M., Hu, Y., Barratt, D.C., 2021. Real-time multimodal image registration with
partial intraoperative point-set data. Med. Image Anal. 74, 102231.
Baum, Z., Ungi, T., Schlenger, C., Hu, Y., Barratt, D.C., 2022. Learning generalized
non-rigid multimodal biomedical image registration from generic point set data. In:
International Workshop on Advances in Simplifying Medical Ultrasound. Springer,
pp. 141–151.
Bednarik, J., Fua, P., Salzmann, M., 2018. Learning to reconstruct texture-less de-
formable surfaces from a single view. In: 2018 International Conference on 3D
Vision. 3dV, IEEE, pp. 606–615.
Bellekens, B., Spruyt, V., Berkvens, R., Penne, R., Weyn, M., 2015. A benchmark survey
of rigid 3D point cloud registration algorithms. Int. J. Adv. Intell. Syst. 8, 118–127.
Bernreiter, L., Ott, L., Nieto, J., Siegwart, R., Cadena, C., 2021. PHASER: A robust and
correspondence-free global pointcloud registration. IEEE Robot. Autom. Lett. 6 (2),
855–862. http://dx.doi.org/10.1109/LRA.2021.3052418.
S. Monji-Azad et al.
Besl, P.J., McKay, N.D., 1992. Method for registration of 3-D shapes. In: Sensor Fusion
IV: Control Paradigms and Data Structures, Vol. 1611. Spie, pp. 586–606.
Billings, S.D., Boctor, E.M., Taylor, R.H., 2015. Iterative most-likely point registration
(IMLP): A robust algorithm for computing optimal shape alignment. PLoS One 10
(3), e0117688. http://dx.doi.org/10.1371/journal.pone.0117688.
Bogo, F., Romero, J., Loper, M., Black, M.J., 2014. FAUST: Dataset and evaluation for
3D mesh registration. In: Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition. pp. 3794–3801.
Bogo, F., Romero, J., Pons-Moll, G., Black, M.J., 2017. Dynamic FAUST: Registering
human bodies in motion. In: Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition. pp. 6233–6242.
Boscaini, D., Masci, J., Rodolà, E., Bronstein, M., 2016. Learning shape correspondence
with anisotropic convolutional neural networks. Adv. Neural Inf. Process. Syst. 29.
Bozic, A., Zollhofer, M., Theobalt, C., Nießner, M., 2020. Deepdeform: Learning
non-rigid rgb-d reconstruction with semi-supervised data. In: Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7002–7012.
Bronstein, A.M., Bronstein, M.M., Kimmel, R., 2008. Numerical Geometry of Non-Rigid
Shapes. Springer Science & Business Media.
Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P., 2017. Geometric
deep learning: Going beyond euclidean data. IEEE Signal Process. Mag. 34 (4),
18–42.
Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K., 2013. Facewarehouse: A 3D facial
expression database for visual computing. IEEE Trans. Vis. Comput. Graphics 20
(3), 413–425.
Castellani, U., Bartoli, A., 2020. 3D shape registration. In: 3D Imaging, Analysis and
Applications. Springer, pp. 353–411.
Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S.,
Savva, M., Song, S., Su, H., et al., 2015. Shapenet: An information-rich 3D model
repository. arXiv preprint arXiv:1512.03012.
Chang, W., Zwicker, M., 2008. Automatic registration for articulated shapes. Comput.
Graph. Forum 27 (5), 1459–1468.
Chang, W., Zwicker, M., 2009. Range scan registration using reduced deformable
models. Comput. Graph. Forum 28 (2), 447–456.
Chaudhury, A., 2020. Multilevel optimization for registration of deformable point
clouds. IEEE Trans. Image Proc. Publ. IEEE Signal Proc. Soc. PP, http://dx.doi.
org/10.1109/TIP.2020.3019649.
Chen, S., Nan, L., Xia, R., Zhao, J., Wonka, P., 2020. PLADE: A plane-based descriptor
for point cloud registration with small overlap. IEEE Trans. Geosci. Remote
Sens. (ISSN: 0196-2892) 58 (4), 2530–2540. http://dx.doi.org/10.1109/TGRS.2019.
2952086.
Cheng, L., Chen, S., Liu, X., Xu, H., Wu, Y., Li, M., Chen, Y., 2018. Registration of
laser scanning point clouds: A review. Sensors 18 (5), 1641.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H.,
Bengio, Y., 2014. Learning phrase representations using RNN encoder-decoder for
statistical machine translation. arXiv preprint arXiv:1406.1078.
Chui, H., Rangarajan, A., 2000. A new algorithm for non-rigid point matching. In:
Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR
2000 (Cat. No. PR00662), Vol. 2. IEEE, pp. 44–51.
Chui, H., Rangarajan, A., 2003. A new point matching algorithm for non-rigid
registration. Comput. Vis. Image Underst. 89 (2–3), 114–141.
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep
bidirectional transformers for language understanding. arXiv preprint arXiv:1810.
04805.
Dimentions, 2022. https://app.dimensions.ai/discover/publication (Last Access: 12 May
2022).
Ding, L., Feng, C., 2019. DeepMapping: Unsupervised map estimation from multiple
point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition. pp. 8650–8659.
Dong, Z., Liang, F., Yang, B., Xu, Y., Zang, Y., Li, J., Wang, Y., Dai, W., Fan, H.,
Hyyppä, J., et al., 2020. Registration of large-scale terrestrial laser scanner point
clouds: A review and benchmark. ISPRS J. Photogramm. Remote Sens. 163,
327–342.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T.,
Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al., 2020. An image is
worth 16x16 words: Transformers for image recognition at scale. arXiv preprint
arXiv:2010.11929.
Dyke, R.M., Lai, Y.-K., Rosin, P.L., Zappalà, S., Dykes, S., Guo, D., Li, K., Marin, R.,
Melzi, S., Yang, J., 2020. Shrec’20: Shape correspondence with non-isometric
deformations. Comput. Graph. 92, 28–43.
Eckart, B., Kim, K., Kautz, J., 2018. Fast and accurate point cloud registration using
trees of gaussian mixtures. arXiv preprint arXiv:1807.02587.
Elbaz, G., Avraham, T., Fischer, A., 2017. 3D point cloud registration for localization
using a deep neural network auto-encoder. In: Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition. pp. 4631–4640.
Fan, H., Yang, Y., 2019. PointRNN: Point recurrent neural network for moving point
cloud processing. arXiv preprint arXiv:1910.08287.
Feng, M., Hu, S., Ang, M.H., Lee, G.H., 2019. 2D3D-matchnet: Learning to match
keypoints across 2D image and 3D point cloud. In: 2019 International Conference
on Robotics and Automation. ICRA, IEEE, http://dx.doi.org/10.1109/icra.2019.
8794415.
70
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
Feng, W., Zhang, J., Cai, H., Xu, H., Hou, J., Bao, H., 2021. Recurrent multi-
view alignment network for unsupervised surface registration. In: Proceedings
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp.
10297–10307.
Fey, M., Lenssen, J.E., Weichert, F., Müller, H., 2018. Splinecnn: Fast geometric deep
learning with continuous b-spline kernels. In: Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition. pp. 869–877.
Fischler, M.A., Bolles, R.C., 1981. Random sample consensus: A paradigm for model
fitting with applications to image analysis and automated cartography. Commun.
ACM 24 (6), 381–395.
Fitzgibbon, A.W., 2003. Robust registration of 2D and 3D point sets. Image Vis. Comput.
21 (13–14), 1145–1153.
Fotouhi, M., Hekmatian, H., Kashani-Nezhad, M.A., Kasaei, S., 2019. SC-RANSAC:
Spatial consistency on RANSAC. Multimedia Tools Appl. 78 (7), 9429–9461.
Fu, Y., Lei, Y., Wang, T., Patel, P., Jani, A.B., Mao, H., Curran, W.J., Liu, T., Yang, X.,
2021. Biomechanically constrained non-rigid MR-TRUS prostate registration using
deep learning based 3D point cloud matching. Med. Image Anal. 67, 101845.
http://dx.doi.org/10.1016/j.media.2020.101845.
Ge, X., 2016. Non-rigid registration of 3D point clouds under isometric deformation.
ISPRS J. Photogramm. Remote Sens. (ISSN: 0924-2716) 121, 192–202. http://dx.
doi.org/10.1016/j.isprsjprs.2016.09.009.
Geiger, A., Lenz, P., Urtasun, R., 2012. Are we ready for autonomous driving? the kitti
vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern
Recognition. IEEE, pp. 3354–3361.
Golyanik, V., Shimada, S., Varanasi, K., Stricker, D., 2018. Hdm-net: Monocular
non-rigid 3D reconstruction with learned deformation model. In: International
Conference on Virtual Reality and Augmented Reality. Springer, pp. 51–72.
Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T., 2009. Ground truth dataset
and baseline evaluations for intrinsic image algorithms. In: 2009 IEEE 12th
International Conference on Computer Vision. IEEE, pp. 2335–2342.
Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R.R., Hu, S.-M., 2021. Pct: Point
cloud transformer. Comput. Vis. Media 7 (2), 187–199.
Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., Bennamoun, M., 2020. Deep learning for 3D
point clouds: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 43 (12), 4338–4364.
Handa, A., Patraucean, V., Badrinarayanan, V., Stent, S., Cipolla, R., 2016. Under-
standing real world indoor scenes with synthetic data. In: Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition. pp. 4077–4085.
Hanocka, R., Fish, N., Wang, Z., Giryes, R., Fleishman, S., Cohen-Or, D., 2018. Alignet:
Partial-shape agnostic alignment via unsupervised learning. ACM Trans. Graph. 38
(1), 1–14.
Hansen, L., Dittmer, D., Heinrich, M.P., 2019. Learning deformable point set registration
with regularized dynamic graph cnns for large lung motion in copd patients.
In: International Workshop on Graph Learning in Medical Imaging. Springer, pp.
53–61.
Hansen, L., Heinrich, M.P., 2021. Deep learning based geometric registration for
medical images: How accurate can we get without visual features? In: International
Conference on Information Processing in Medical Imaging. Springer, pp. 18–30.
He, Y., Liang, B., Yang, J., Li, S., He, J., 2017. An iterative closest points algorithm
for registration of 3D laser scanner point clouds with geometric features. Sensors
17 (8), 1862.
He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recog-
nition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition. pp. 770–778.
Hirose, O., 2020. Acceleration of non-rigid point set registration with downsampling
and Gaussian process regression. IEEE Trans. Pattern Anal. Mach. Intell. 43 (8),
2858–2865.
Hochreiter, S., Schmidhuber, J., 1997. Long short-term memory. Neural Comput. 9 (8),
1735–1780.
Horn, M., Engel, N., Belagiannis, V., Buchholz, M., Dietmayer, K., 2020. DeepCLR:
Correspondence-less architecture for deep end-to-end point cloud registration.
In: 2020 IEEE 23rd International Conference on Intelligent Transportation Sys-
tems. ITSC, IEEE, ISBN: 978-1-7281-4149-7, pp. 1–7. http://dx.doi.org/10.1109/
ITSC45102.2020.9294279.
Hu, X., Zhang, D., Chen, J., Wu, Y., Chen, Y., 2022. NrtNet: An unsupervised method
for 3D non-rigid point cloud registration based on transformer. Sensors 22 (14),
5128.
Huang, J., Birdal, T., Gojcic, Z., Guibas, L.J., Hu, S.-M., 2022a. Multiway non-rigid
point cloud registration via learned functional map synchronization. IEEE Trans.
Pattern Anal. Mach. Intell..
Huang, S., Gojcic, Z., Huang, J., Wieser, A., Schindler, K., 2022b. Dynamic 3D scene
analysis by point cloud accumulation. arXiv preprint arXiv:2207.12394.
Huang, S., Gojcic, Z., Usvyatsov, M., Wieser, A., Schindler, K., 2021a. Predator:
Registration of 3D point clouds with low overlap. In: Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition. pp. 4267–4276.
Huang, X., Mei, G., Zhang, J., 2020. Feature-metric registration: A fast semi-
supervised approach for robust point cloud registration without correspondences.
In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
CVPR, IEEE, ISBN: 978-1-7281-7168-5, pp. 11363–11371. http://dx.doi.org/10.
1109/CVPR42600.2020.01138.
S. Monji-Azad et al.
Huang, X., Mei, G., Zhang, J., Abbas, R., 2021b. A comprehensive survey on point
cloud registration. arXiv preprint arXiv:2103.02690.
Ihler, A.T., Fisher III, J.W., Willsky, A.S., Chickering, D.M., 2005. Loopy belief
propagation: Convergence and effects of message errors. J. Mach. Learn. Res. 6
(5).
Le, H.M., Do, T.-T., Hoang, T., Cheung, N.-M., 2019. SDRSAC: Semidefinite-based
randomized approach for robust point cloud registration without correspondences.
In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR,
IEEE, ISBN: 978-1-7281-3293-8, pp. 124–133. http://dx.doi.org/10.1109/CVPR.
2019.00021.
Lei, Y., Tian, S., He, X., Wang, T., Wang, B., Patel, P., Jani, A.B., Mao, H., Curran, W.J.,
Liu, T., et al., 2019. Ultrasound prostate segmentation based on multidirectional
deeply supervised V-Net. Med. Phys. 46 (7), 3194–3206.
Li, T., Bolkart, T., Black, M.J., Li, H., Romero, J., 2017. Learning a model of facial
shape and expression from 4D scans. ACM Trans. Graph. 36 (6), 194–1.
Li, Y., Harada, T., 2022a. Lepard: Learning partial point cloud matching in rigid and
deformable scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition. pp. 5554–5564.
Li, Y., Harada, T., 2022b. Non-rigid point cloud registration with neural deformation
Pyramid. arXiv preprint arXiv:2205.12796.
Li, J., Hu, Q., Ai, M., 2020a. GESAC: Robust graph enhanced sample consensus for
point cloud registration. ISPRS J. Photogramm. Remote Sens. 167, 363–374.
Li, J., Hu, Q., Ai, M., 2021a. Point cloud registration based on one-point Ransac and
scale-annealing biweight estimation. IEEE Trans. Geosci. Remote Sens. 59 (11),
9716–9729.
Li, X., Pontes, J.K., Lucey, S., 2021b. Pointnetlk revisited. In: Proceedings of
the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp.
12763–12772.
Li, Y., Takehara, H., Taketomi, T., Zheng, B., Nießner, M., 2021c. 4Dcomplete: Non-
rigid motion estimation beyond the observable surface. In: Proceedings of the
IEEE/CVF International Conference on Computer Vision. pp. 12706–12716.
Li, M., Xu, R.Y., Xin, J., Zhang, K., Jing, J., 2020b. Fast non-rigid points registration
with cluster correspondences projection. Signal Process. (ISSN: 01651684) 170,
107425. http://dx.doi.org/10.1016/j.sigpro.2019.107425.
Li, J., Yang, B., Chen, C., Habib, A., 2019. NRLI-UAV: Non-rigid registration of
sequential raw laser scans and images for low-cost UAV LiDAR point cloud
quality improvement. ISPRS J. Photogramm. Remote Sens. (ISSN: 0924-2716) 158,
123–145. http://dx.doi.org/10.1016/j.isprsjprs.2019.10.009.
Liang, L., Wei, M., Szymczak, A., Petrella, A., Xie, H., Qin, J., Wang, J., Wang, F.L.,
2018. Nonrigid iterative closest points for registration of 3D biomedical surfaces.
Opt. Lasers Eng. (ISSN: 01438166) 100, 141–154. http://dx.doi.org/10.1016/j.
optlaseng.2017.08.005.
Liu, X., Qi, C.R., Guibas, L.J., 2019. Flownet3D: Learning scene flow in 3D point clouds.
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition. pp. 529–537.
Lucas, B.D., Kanade, T., et al., 1981. An iterative image registration technique with an
application to stereo vision. IJCAI.
Ma, Q., Yang, J., Ranjan, A., Pujades, S., Pons-Moll, G., Tang, S., Black, M.J., 2020.
Learning to dress 3D people in generative clothing. In: Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition. pp. 6469–6478.
Ma, J., Zhao, J., Jiang, J., Zhou, H., 2017. Non-rigid point set registration with
robust transformation estimation under manifold regularization. In: Thirty-First
AAAI Conference on Artificial Intelligence.
Mahmood, B., Han, S., 2019. 3D registration of indoor point clouds for augmented real-
ity. In: Computing in Civil Engineering 2019: Visualization, Information Modeling,
and Simulation. American Society of Civil Engineers Reston, VA, pp. 1–8.
Maiseli, B., Gu, Y., Gao, H., 2017. Recent developments and trends in point set
registration methods. J. Vis. Commun. Image Represent. 46, 95–106.
Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., Brox, T., 2016.
A large dataset to train convolutional networks for disparity, optical flow, and
scene flow estimation. In: Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition. pp. 4040–4048.
Mei, G., Huang, X., Zhang, J., Wu, Q., 2022. Partial point cloud registration via soft
segmentation. In: 2022 IEEE International Conference on Image Processing. ICIP,
IEEE, pp. 681–685.
Melzi, S., Marin, R., Rodolà, E., Castellani, U., Ren, J., Poulenard, A., Wonka, P.,
Ovsjanikov, M., 2019. Shrec 2019: Matching humans with different connectivity.
In: Eurographics Workshop on 3D Object Retrieval, Vol. 7. p. 3.
Min, T., Kim, E., Shim, I., 2021. Geometry guided network for point cloud registration.
IEEE Robot. Autom. Lett. 6 (4), 7270–7277. http://dx.doi.org/10.1109/LRA.2021.
3097268.
Mirza, M., Osindero, S., 2014. Conditional generative adversarial nets. arXiv preprint
arXiv:1411.1784.
model search engine, F.D., 2022. https://youbi3d.com (Last Access: 29 Oct 2022).
Myronenko, A., Song, X., 2010. Point set registration: Coherent point drift. IEEE Trans.
Pattern Anal. Mach. Intell. 32 (12), 2262–2275.
Myronenko, A., Song, X., Carreira-Perpinan, M., 2006. Non-rigid point set registration:
Coherent point drift. Adv. Neural Inf. Process. Syst. 19.
Netto, G.M., Oliveira, M.M., 2022. Robust point-cloud registration based on dense point
matching and probabilistic modeling. Vis. Comput. 38 (9), 3217–3230.
71
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
Pais, G.D., Ramalingam, S., Govindu, V.M., Nascimento, J.C., Chellappa, R., Miraldo, P.,
2020. 3DRegNet: A deep neural network for 3D point registration. In: 2020
IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR, IEEE,
ISBN: 978-1-7281-7168-5, pp. 7191–7201. http://dx.doi.org/10.1109/CVPR42600.
2020.00722.
Peng, S., Jiang, C., Liao, Y., Niemeyer, M., Pollefeys, M., Geiger, A., 2021. Shape as
points: A differentiable poisson solver. Adv. Neural Inf. Process. Syst. 34.
Peterlík, I., Courtecuisse, H., Rohling, R., Abolmaesumi, P., Nguan, C., Cotin, S., Sal-
cudean, S., 2018. Fast elastic registration of soft tissues under large deformations.
Med. Image Anal. 45, 24–40. http://dx.doi.org/10.1016/j.media.2017.12.006.
Petricek, T., Svoboda, T., 2017. Point cloud registration from local feature
correspondences-evaluation on challenging datasets. PLoS One 12 (11), e0187943.
http://dx.doi.org/10.1371/journal.pone.0187943.
Pfeiffer, M., Riediger, C., Leger, S., Kühn, J.-P., Seppelt, D., Hoffmann, R.-T., Weitz, J.,
Speidel, S., 2020. Non-rigid volume to surface registration using a data-driven
biomechanical model. In: International Conference on Medical Image Computing
and Computer-Assisted Intervention. Springer, pp. 724–734.
Pomerleau, F., Colas, F., Siegwart, R., et al., 2015. A review of point cloud registration
algorithms for mobile robotics. Found. Trends® Robot. 4 (1), 1–104.
Pons-Moll, G., Pujades, S., Hu, S., Black, M.J., 2017. ClothCap: Seamless 4D clothing
capture and retargeting. ACM Trans. Graph. (ToG) 36 (4), 1–15.
Poulenard, A., Ovsjanikov, M., 2018. Multi-directional geodesic neural networks via
equivariant convolution. ACM Trans. Graph. 37 (6), 1–14.
Qi, C.R., Su, H., Mo, K., Guibas, L.J., 2017a. Pointnet: Deep learning on point sets
for 3D classification and segmentation. In: Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition. pp. 652–660.
Qi, C.R., Yi, L., Su, H., Guibas, L.J., 2017b. Pointnet++: Deep hierarchical feature
learning on point sets in a metric space. Adv. Neural Inf. Process. Syst. 30.
Quan, S., Yang, J., 2020. Compatibility-guided sampling consensus for 3-d point cloud
registration. IEEE Trans. Geosci. Remote Sens. 58 (10), 7380–7392.
Saiti, E., Theoharis, T., 2020. An application independent review of multimodal 3D
registration methods. Comput. Graph. 91, 153–178.
Sarlin, P.-E., DeTone, D., Malisiewicz, T., Rabinovich, A., 2020. Superglue: Learning
feature matching with graph neural networks. In: Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition. pp. 4938–4947.
Sarode, V., Li, X., Goforth, H., Aoki, Y., Srivatsan, R.A., Lucey, S., Choset, H., 2019.
Pcrnet: Point cloud registration network using pointnet encoding. arXiv preprint
arXiv:1908.07906.
Shi, J., Wan, P., Chen, F., 2021. An unsupervised non-rigid registration network for
fast medical shape alignment. In: 2021 43rd Annual International Conference of
the IEEE Engineering in Medicine & Biology Society. EMBC, IEEE, pp. 1887–1890.
Shimada, S., Golyanik, V., Tretschk, E., Stricker, D., Theobalt, C., 2019. Dis-
pvoxnets: Non-rigid point set alignment with supervised learning proxies. In: 2019
International Conference on 3D Vision. 3DV, IEEE, pp. 27–36.
Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P., 2013. The
emerging field of signal processing on graphs: Extending high-dimensional data
analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30
(3), 83–98.
Slavcheva, M., Baust, M., Cremers, D., Ilic, S., 2017. Killingfusion: Non-rigid 3D
reconstruction without correspondences. In: Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition. pp. 1386–1395.
Sun, J., Shen, Z., Wang, Y., Bao, H., Zhou, X., 2021. LoFTR: Detector-free local
feature matching with transformers. In: Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition. pp. 8922–8931.
Takimoto, R.Y., Tsuzuki, M.d.S.G., Vogelaar, R., de Castro Martins, T., Sato, A.K.,
Iwao, Y., Gotoh, T., Kagei, S., 2016. 3D reconstruction and multiple point cloud
registration using a low precision RGB-D sensor. Mechatronics 35, 11–22.
Tang, H., Zhao, Y., 2021. A conditional generative adversarial network for non-rigid
point set registration. In: 2021 IEEE Asia-Pacific Conference on Computer Science
and Data Engineering. CSDE, IEEE, pp. 1–6.
Tang, W., Zou, D., Li, P., 2021. Learning-based point cloud registration: A short review
and evaluation. In: 2021 2nd International Conference on Artificial Intelligence
in Electronics Engineering. ACM, New York, NY, USA, http://dx.doi.org/10.1145/
3460268.3460273.
Tazir, M.L., Gokhool, T., Checchin, P., Malaterre, L., Trassoudaine, L., 2018. CICP:
Cluster iterative closest point for sparse–dense point cloud registration. Robot.
Auton. Syst. 108, 66–86.
Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., Seidel, H.-P., 2012.
Animation cartography—intrinsic reconstruction of shape and motion. ACM Trans.
Graph. 31 (2), 1–15.
Thomas, H., Qi, C.R., Deschaud, J.-E., Marcotegui, B., Goulette, F., Guibas, L.J., 2019.
Kpconv: Flexible and deformable convolution for point clouds. In: Proceedings of
the IEEE/CVF International Conference on Computer Vision. pp. 6411–6420.
Trappolini, G., Cosmo, L., Moschella, L., Marin, R., Melzi, S., Rodolà, E., 2021. Shape
registration in the time of transformers. Adv. Neural Inf. Process. Syst. 34.
Trimech, I.H., Maalej, A., Amara, N.E.B., 2016. 3D facial expression recognition
using nonrigid CPD registration method. In: 2016 7th International Conference
on Sciences of Electronics, Technologies of Information and Telecommunications.
SETIT, IEEE, pp. 478–481.
S. Monji-Azad et al.
Trimech, I.H., Maalej, A., Amara, N.E.B., 2020. Point-based deep neural network for
3D facial expression recognition. In: 2020 International Conference on Cyberworlds.
CW, IEEE, pp. 164–171.
Truong, G., Gilani, S.Z., Islam, S.M.S., Suter, D., 2019. Fast point cloud registration
using semantic segmentation. In: 2019 Digital Image Computing: Techniques and
Applications. DICTA, IEEE, pp. 1–8.
Varol, G., Romero, J., Martin, X., Mahmood, N., Black, M.J., Laptev, I., Schmid, C.,
2017. Learning from synthetic humans. In: Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition. pp. 109–117.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł.,
Polosukhin, I., 2017. Attention is all you need. Adv. Neural Inf. Process. Syst. 30.
VGoodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S.,
Courville, A., Bengio, Y., 2014. Generative adversarial nets. In: Proceedings of the
International Conference on Neural Information Processing Systems. NIPS.
Villena-Martinez, V., Oprea, S., Saval-Calvo, M., Azorin-Lopez, J., Fuster-Guillo, A.,
Fisher, R.B., 2020. When deep learning meets data alignment: A review on deep
registration networks (DRNs). Appl. Sci. 10 (21), 7524. http://dx.doi.org/10.3390/
app10217524.
Vlasic, D., Baran, I., Matusik, W., Popović, J., 2008. Articulated mesh animation from
multi-view silhouettes. ACM SIGGRAPH 2008 Papers. ACM, pp. 1–9.
Wang, L., Chen, J., Li, X., Fang, Y., 2019a. Non-rigid point set registration networks.
arXiv preprint arXiv:1904.01428.
Wang, B., Lei, Y., Tian, S., Wang, T., Liu, Y., Patel, P., Jani, A.B., Mao, H., Curran, W.J.,
Liu, T., et al., 2019b. Deeply supervised 3D fully convolutional networks with group
dilated convolution for automatic MRI prostate segmentation. Med. Phys. 46 (4),
1707–1718.
Wang, L., Li, X., Chen, J., Fang, Y., 2019c. Coherent point drift networks: Unsupervised
learning of non-rigid point set registration. arXiv preprint arXiv:1906.03039.
Wang, L., Li, X., Fang, Y., 2020a. GP-Aligner: Unsupervised non-rigid groupwise point
set registration based on optimized group latent descriptor. arXiv preprint arXiv:
2007.12979.
Wang, L., Li, X., Fang, Y., 2020b. GP-Aligner: Unsupervised non-rigid groupwise
point set registration based on optimized group latent descriptor. arXiv preprint
arXiv:2007.12979.
Wang, L., Li, X., Fang, Y., 2020c. Unsupervised learning of 3D point set registration.
arXiv preprint arXiv:2006.06200.
Wang, Z., Li, S., Howard-Jenkins, H., Prisacariu, V., Chen, M., 2020d. Flownet3D++:
Geometric losses for deep scene flow estimation. In: Proceedings of the IEEE/CVF
Winter Conference on Applications of Computer Vision. pp. 91–98.
Wang, Y., Solomon, J., 2019a. Deep closest point: Learning representations for point
cloud registration. In: 2019 IEEE/CVF International Conference on Computer
Vision. ICCV, IEEE, ISBN: 978-1-7281-4803-8, pp. 3522–3531. http://dx.doi.org/
10.1109/ICCV.2019.00362.
Wang, Y., Solomon, J.M., 2019b. Prnet: Self-supervised learning for partial-to-partial
registration. Adv. Neural Inf. Process. Syst. 32.
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M., 2019. Dynamic
graph cnn for learning on point clouds. ACM Trans. Graph. (ToG) 38 (5), 1–12.
Wang, Y., Zhang, S., Bai, X., 2019e. A 3D tracking and registration method based on
point cloud and visual features for augmented reality aided assembly system. Xibei
Gongye Daxue Xuebao/J. Northwestern Polytech. Univ. 37 (1), 143–151.
72
ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023) 58–72
Wei, L., Huang, Q., Ceylan, D., Vouga, E., Li, H., 2016. Dense human body correspon-
dences using convolutional networks. In: Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition. pp. 1544–1553.
Wu, B., Ma, J., Chen, G., An, P., 2021. Feature interactive representation for point
cloud registration. In: Proceedings of the IEEE/CVF International Conference on
Computer Vision. pp. 5530–5539.
Wu, W., Qi, Z., Fuxin, L., 2019. Pointconv: Deep convolutional networks on 3D point
clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition. pp. 9621–9630.
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J., 2015. 3D shapenets: A
deep representation for volumetric shapes. In: Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition. pp. 1912–1920.
Xiang, F., Qin, Y., Mo, K., Xia, Y., Zhu, H., Liu, F., Liu, M., Jiang, H., Yuan, Y.,
Wang, H., et al., 2020. Sapien: A simulated part-based interactive environment.
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition. pp. 11097–11107.
Yang, J., Cao, Z., Zhang, Q., 2016. A fast and robust local descriptor for 3D point cloud
registration. Inform. Sci. (ISSN: 00200255) 346–347, 163–179. http://dx.doi.org/
10.1016/j.ins.2016.01.095.
Yang, C., Liu, Y., Jiang, X., Zhang, Z., Wei, L., Lai, T., Chen, R., 2018. Non-rigid
point set registration via adaptive weighted objective function. IEEE Access 6,
75947–75960. http://dx.doi.org/10.1109/ACCESS.2018.2883689.
Yang, Y., Liu, S., Pan, H., Liu, Y., Tong, X., 2020. PFCNN: Convolutional neural
networks on 3D surfaces using parallel frames. In: Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition. pp. 13578–13587.
Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J., Funkhouser, T., 2017. 3Dmatch:
Learning local geometric descriptors from rgb-d reconstructions. In: Proceedings of
the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1802–1811.
Zhang, Z., Chen, G., Wang, X., Shu, M., 2021a. DDRNet: Fast point cloud registra-
tion network for large-scale scenes. ISPRS J. Photogramm. Remote Sens. (ISSN:
0924-2716) 175, 184–198. http://dx.doi.org/10.1016/j.isprsjprs.2021.03.003.
Zhang, Z., Dai, Y., Sun, J., 2020. Deep learning based point cloud registration: An
overview. Virtual Real. Intell. Hardw. (ISSN: 2096-5796) 2 (3), 222–246. http:
//dx.doi.org/10.1016/j.vrih.2020.05.002.
Zhang, Z., Sun, J., Dai, Y., Zhou, D., Song, X., He, M., 2021b. A representation separa-
tion perspective to correspondences-free unsupervised 3D point cloud registration.
IEEE Geosci. Remote Sens. Lett..
Zhang, S., Tong, H., Xu, J., Maciejewski, R., 2019. Graph convolutional networks: A
comprehensive review. Comput. Soc. Netw. 6 (1), 1–23.
Zhao, H., Jiang, L., Jia, J., Torr, P.H., Koltun, V., 2021a. Point transformer. In:
Proceedings of the IEEE/CVF International Conference on Computer Vision. pp.
16259–16268.
Zhao, H., Liang, Z., Wang, C., Yang, M., 2021b. CentroidReg: A global-to-local
framework for partial point cloud registration. IEEE Robot. Autom. Lett. 6 (2),
2533–2540. http://dx.doi.org/10.1109/lra.2021.3061369.
Zhu, H., Guo, B., Zou, K., Li, Y., Yuen, K.-V., Mihaylova, L., Leung, H., 2019. A review of
point set registration: From pairwise registration to groupwise registration. Sensors
(Basel, Switzerland) 19 (5), http://dx.doi.org/10.3390/s19051191.