Professional Documents
Culture Documents
org
Linda-Sophie Schneider1,2 , Mareike Thies1 , Richard Schielein2 , Christopher Syben2 , Mathias Unberath3 , Andreas Maier1,2
1
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
2
Fraunhofer EZRT, Flugplatzstraße 75, 90768 Fürth, Germany
3
Laboratory for Computational Sensing + Robotics, Johns Hopkins University, Baltimore, MD, USA
e-mail: linda-sophie.schneider@fau.de
Abstract
Non-circular twin robot computed tomography (CT) systems provide a high degree of flexibility in positioning the X-ray source
More info about this article: https://www.ndt.net/?id=27748
and detector. This makes the design of optimal CT trajectories particularly important, as it can potentially allow for a reduction in
the number of projections required for image acquisition. Therefore, the optimization of CT trajectories is a key area of research
in this field. This paper presents a method for optimising the trajectory of a non-circular twin robot computed tomography system,
using a so-called detectability index to guide the optimization process. The proposed method involves simulating possible X-ray
projections that are reachable by the robotic CT system and using a regressive ResNet-34 to predict the detectability index of each
projection. An integer optimization problem is then used to generate an optimised CT trajectory based on this prediction, with
the haversine distance between projections serving as an additional constraint. The performance of the ResNet-34 is evaluated on
unseen projections. It effectively predicted the detectability index for a spherical sampling space, allowing for the optimization
of arbitrary CT trajectories. In addition, the method was able to identify valuable projections according to the detectability index,
enabling the reconstruction of a test object with only 30 out of 800 of the previously acquired projections without significant loss
of information in the reconstruction domain.
Keywords: computed tomography, robot-based CT, deep learning, task-specific acquisition trajectory, detectability index
1 Introduction
Industrial X-ray computed tomography (CT) is usually performed with fixed X-ray source and detector positions and a speci-
men placed on a turntable between the X-ray source and detector. In order to create a flexible CT system, the X-ray source
and detector are connected to the end-effectors of two articulated arm robots (RoboCT). This increased flexibility allows us to
perform arbitrary CT trajectories that can be optimised for specific applications [7]. Hence, the process of selecting a certain
subset of projections that leads to a higher-quality reconstruction compared to other subsets is called CT trajectory optimization.
CT trajectory optimization and planning in non-destructive testing is usually performed before the acquisition process. CT is
often used for quality control in production, where equivalent test specimens are scanned and analysed for defects. Thus, for
such applications, the same optimised CT trajectory can be used for multiple CT scans. In addition, manufacturers often define
in advance where critical defects should be detected in the test specimen. With the help of this information, a complete scan of
the object can be avoided, thus reducing scan time and costs. However, it is important to ensure that critical areas can still be
reconstructed with a resolution that allows critical defects to be identified. In addition, missing information creates artefacts that
can obscure defects in the test specimen. Therefore, a trade-off between evaluability and scan time must be considered when
optimising CT trajectories. However, defining such an optimal CT trajectory for a given application is difficult since it leads to
an NP-hard optimization problem with a huge solution space.
This topic is also of great interest in the field of medical CT. In medical CT, the goal is not necessarily to reduce the cost of the
scan but to reduce the radiation dose to the patient. Therefore, there is also an interest in using as few projections as possible
while still being able to answer the medical question. The approach which serves as a reference for this work uses a so-called
detectability index as a task-based metric to optimize CT trajectories for C-arm systems with respect to a predefined task [1–3].
The detectability index is a projection-based quality metric to assess the contribution of a single projection in a predefined region
of interest (ROI) in the reconstruction domain. To make use of the detectability index without a priori knowledge about the 3D
structure of the object being scanned, a VGG-19-based network was trained in [2] to predict the detectability index for the next
possible projections of an ongoing scan. To increase data completeness, a fixed increment in the in-plane angle is enforced. The
next best out-of-plane angle is selected based on the network predictions in a greedy approach [1, 2].
In this work, we extend the former discussed approach to the less restricted optimization setting in RoboCT. Instead of optimising
over one angular direction only, all possible X-ray projections with a constant source-to-detector distance are considered. To pre-
dict the detectability index, a regressive ResNet-34 is trained. The predicted detectability index is used as a metric to evaluate an
optimised CT trajectory for detecting a small defect inserted into all test specimens. We apply the haversine distance as a con-
straint to avoid using neighbouring projections. This allows us to define a minimal great circle distance between two projections
in order for both to be part of the optimal CT trajectory. Under this constraint, an integer optimization problem is solved that
maximises the predicted detectability index. The optimised CT trajectory can achieve good results in the reconstruction domain,
requiring only 4% of the projections needed by the benchmark reconstruction.
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.
https://doi.org/10.58286/27748
12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), www.ict2023.org
2 Methods
Defining an optimal CT trajectory for a RoboCT is a problem with many degrees of freedom. For this work, we stick to a fixed
source-object distance and source-detector distance, thus limiting the possible scan positions to a sphere, to evaluate the general
feasibility of the approach. This enables us to follow the ideas in [4] and parametrize the problem in terms of an azimuth angle
ϕ and an elevation angle θ . Each CT trajectory consists of sets of pairs (ϕt , θt ), t ∈ {0, . . . , N}, where N is the total number of
projection images. The trajectory planning pipeline we propose is shown in 1. At first, N source-detector-coordinates are sampled
on a sphere. A discrete sampling of the azimuth and elevation angles is chosen such that the surface of the sphere is sampled as
uniformly as possible. Therefore, we use a Fibonacci-based sampling as shown in [8]. We simulate a polychromatic projection
with a 60 keV spectrum and with additional noise for each sampled source-detector-coordinate pt = (ϕt , θt ). Subsequently, the
detectability index (see Section 2.1) for the respective projection is predicted by a ResNet-34. To obtain our optimal CT trajectory,
T < N projection positions are used that maximize the overall detectability index. The haversine distance constraint regularises
the optimization (see Section 2.4), which helps to avoid neighbouring projection positions. Avoiding neighbouring projections is
of high interest because selecting too similar viewing angles incorporates redundant information, while other valuable projections
an their information content might be missing instead. This constraint has to be added artificially to our optimization approach
because the detectability index is computed for each projection independently and has no notion of data completeness.
dpred
1
dpred
2
dpred max Predicted detectability
3
subject to:
dpred
4. Haversine distance constraint
.
. Maximum T projections
pred
dN −1
dpred
N
Fibonacci-based Sampling Simulated X-Ray Projections Trained ResNet-34 Predicted Detectability Execute Integer Program Optimized CT Trajectory
Figure 1: High-level overview of the task-based CT trajectory optimization workflow for a RoboCT assuming a trained network.
Projections are simulated using the CAD model of the object of interest. Subsequently, a ResNet-34 predicts the detectability
index for all generated projections. Based on the predictions, a CT trajectory is constructed using an integer optimization
approach incorporating the haversine distance maximizing the overall detectability.
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.
12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), www.ict2023.org
The haversine distance is a "great circle" distance, which is the shortest distance between two points on the surface of a unit
sphere. To extend this to spheres with arbitrary radius, we have to multiply the calculated distance by the radius r of the CT
acquisition sphere:
Dr (pi , p j ) = Dunit (pi , p j ) ∗ r . (3)
By calculating the haversine distance Dr (pi , p j ) between any two projection positions, we can now formulate a constraint (see
Eq. (4) ) to ensure that the selection of neighbouring projections is omitted. This has to be ensured because we assume that the
information content of the signal is distributed in angular space. Therefore, it is important, especially in a setting with a reduced
number of projections, to avoid neighbouring projections.
where Dmean
r is the minimum distance between any two projections on the CT acquisition sphere, T < N is the number of source-
detector-coordinates we use for reconstruction and χ is our decision variable, which shall be 1 in case the source-detector-
coordinate belongs to our optimal subset of projections T . By solving this optimization problem, we ensure a maximized
detection of the signal without using neighbouring viewing angles. This is ensured by the haversine distance constraint, which
forces each chosen projection to have at least γ times the mean distance to each other chosen projection in T . The multiplication
factor γ is a hyperparameter generated by parameter search and has to be adjusted in case of different parameter settings.
We further propose an alternative formulation of the integer optimization problem in Eq. (4), which is computationally more
efficient. In the above formulation, we need N(N − 1) inequality constraints to represent the haversine constraint. To achieve a
better constraint for the haversine distance, we first reformulate the distance matrix Dr (p, q). Instead of capturing the spherical
distance between p and q, we can also express the possibility of being in the same set, denoted by the N × N matrix D{0,1} . If
D{0,1} (p, q) = 0, both projections p, q can be part of the optimal trajectory according to the haversine constraint, for D{0,1} (p, q) =
1 they can not. To compute D{0,1} (p, q), we use the constant term γ · Dmeanr :
(
1 , i f Dr (pi , p j ) < γ · Dmean
r
D{0,1} (pi , p j ) = ∀ i, j ∈ {0, . . . , N} . (5)
0 , i f Dr (pi , p j ) ≥ γ · Dmean
r
Using the matrix D{0,1} for formulating the haversine constraint, the integer program can be written as follows:
The formulation of our integer program in Eq. (6) significantly reduces problem size because the haversine constraint now only
consists of N single constraints instead of N(N − 1). Using integers for the constraints only leads to an acceleration of calculating
the optimal CT trajectory.
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.
12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), www.ict2023.org
Figure 2: Two examples of the generated test specimen, together with the visualisation of the inserted defect. Each of those has
been created by sampling 100 vertex locations and calculating the convex hull of the generated points.
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.
12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), www.ict2023.org
Test Specimen 1
Test Specimen 2
Figure 3: The result of the optimised CT trajectory calculated by our approach for two of the test specimen. For each test
specimen, we see a clustering at the view angle closest to the inserted defect. Thanks to the haversine constraint, we also have a
distribution of the chosen projections to achieve good results in the reconstruction domain.
In Figure 3, we displayed the selected projections for T ∈ [30, 50, 80]. We observed that our optimisation approach resulted
in a clustering of projections closest to the defect inserted in the specimen. Compared to the distribution of the projection of
an optimised CT trajectory without using the haversine constraint, a larger distribution of the chosen projections was shown,
improving the reconstruction of both the object and the inserted defect in particular. During the optimization of CT trajectories,
we observed that the occurrence of some projections was consistently across all optimised trajectories, while the inclusion of
additional projections resulted in the removal of other projections from the set of optimal projections. Using the integer program
instead of a greedy approach, we optimised the CT trajectory as a function of all projections.
Looking at the reconstructions of the optimised CT trajectories for the different numbers of projections shown in Figure 5, it was
found that the introduced defect was visible in all cases. In addition, it could be seen that, especially with only 30 projections,
the edges of the object could no longer be reconstructed as sharply as compared to the reconstruction from 80 projections. We
expected this behaviour because the task function Wtask that the neural network learned was to find projections of high importance
for the representation of the inserted defect. For better visualisation of the differences between the same test specimen but with
different number of projections, a line plot is draw. There we showed the development of the grey values through the test
specimen following the red line. The main difference between the plots for the same objects can be seen in the defect area and at
the edges. For the edges, the growing blur with less projection can be observed. Looking at the region of the defect, we noticed
similar grey value sequences. The main difference occurred at 30 Projections, where a rise of the grey values in the defect area
can be observed.
Additionally, we investigated if the reconstruction quality could be improved compared to a random choice of projections on a
sphere. The result for 80 projections is displayed in Figure 4. Our method shows better results for the same number of projections
chosen. A major difference could be seen for test specimen 2. Both the edges and the inserted defect could not be represented
as sharply with the choice of random projections as in comparison with our presented method. This was also clearly visible
looking at the corresponding line plots. In contrast, test specimen 4 shows that a very good set of projections could be found
even with random selection. This showed that our method worked in general, but the setup was not optimal for investigation of
its advantages. Generally, the results were promising, which is why a further analysis of this approach will be worthwhile.
Calculating the SSIM between the reference volume and the volumes reconstructed from the optimised CT trajectories at the
region of interest, it can be concluded that the reduction of projections does not significantly impact the SSIM result, as can
be seen in Table 1. In general, the SSIM was relatively low. This behaviour was expected since we added artificial noise
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.
12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), www.ict2023.org
80 Projections
80 Projections
Line Plot
80 Random
Projections
80 Random Projections
Line Plot
Figure 4: We compared 80 optimised projections with 80 randomly chosen projections. Except of test specimen 4, we can see
a better result for our presented method. The line plots mostly showed the same structure, but the grey values in the area of
the defect are higher for the random choice compared to the optimised ones. This shows that our optimised approach is able to
improve the reconstruction quality of the defect area (ROI).
to the projection images used for the reconstructions. We could observe a slight decrease in the SSIM when the number of
projections has been reduced. This is reasonable because there is less information in the projection domain that could be used
for reconstruction. Comparison of our method with the choice of random projections always showed an improvement. Further
research is necessary to generalize the results presented in this section to other test specimens and different kind of inserted
defects.
We consider this work as a very fist proof of concept, therefore we kept the sample definition at a quite artificial level. This
enables us to test and understand the method itself. The samples do not yet represent the underlying task for real industrial parts.
The latter will be reported in future work.
Table 1: SSIM between the ideal reconstruction of the full projection stack and the projections chosen by the presented optimiz-
ation approach at the region of interest Wtask . The value of the SSIM was expected to be low because of the noise added to the
projection images. There is a decrease in the SSIM if fewer projections have been used for reconstruction. This is reasonable
because fewer projections should equal less information about the test specimen.
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.
12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), www.ict2023.org
Instead, we can train a neural network to predict the image quality of a projection with respect to a specific task. Although this new
approach can find a good representation of the domain of the task Wtask , we lose some information and sharpness in other parts of
our test object. Compared to the previous work in [1–3], we replace the greedy approach with an integer optimisation problem.
This is a great advantage since our result for the optimal CT trajectory does not depend on the first projection chosen. However,
we still need to consider a constraint to avoid similar projections in our final CT trajectory. The choice of the haversine distance
is very convenient for our spherical setup but only necessary because the assessment of the image quality of the projections is
based on each projection independently. Hence, high data completeness of the full set of projections needs to be encouraged
additionally via the presented distance constraint. Additionally, we have to investigate our choice of the hyperparameter γ in
more detail. By now, its value was found by parameter search, but its expression as dependency of radius, spherical sampling
and number of projection should be examined.
Furthermore, we only tested this approach for simulated data and artificial objects. This objects are well-chosen for our task, as
they are convex and the inserted defect was relatively large. These simplification were made to get a proof of concept and should
be tested for a more realistic setup. For these data, we obtained satisfactory results, which should be verified on real data. Another
topic that should be investigated in more detail is the neural network’s generalisation for predicting the detectability index for
different test objects. So far, the generated objects are very similar and the artificial defect is the same for all test objects, not
only in shape but also in placement. Therefore, further variations of these aspects should be investigated in future work.
Acknowledgements
This research was financed by the „SmartCT – Methoden der Künstlichen Intelligenz für ein autonomes Roboter-CT System“
project (project nr. DIK-2004-0009).
References
[1] Jan-Nico Zäch, Cong Gao, Bastian Bier, Russell Taylor, Andreas Maier, Nassir Navab, Mathias Unberath, Learning to
Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories, Miccai2019 (2019) 11-19
[2] Mareike Thies, Jan-Nico Zäch, Cong Gao, Russell Taylor, Nassir Navab, Andreas Maier, Mathias Unberath, A
learning-based method for online adjustment of C-arm Cone-beam CT source trajectories for artifact avoidance,
International Journal of Computer Assisted Radiology and Surgery, 15 (2020) 1787-1796
[3] J. Webster Stayman, Sarah Capostagno, Grace J. Gang, Jeffrey H. Siewerdsen, Task-driven source–detector trajectories in
cone-beam computed tomography: I. Theory and methods, Journal of Medical Imaging, 6 (2019)
[4] J. Webster Stayman, Jeffrey H. Siewerdsen, Task-based trajectories in iteratively reconstructed interventional cone-beam
CT , Proceedings 12th international meeting on fully three-dimensional image reconstruction in radiology and nuclear
medicine, (2013) 257–260
[5] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Deep Residual Learning for Image Recognition, Proceedings of
the IEEE conference on computer vision and pattern recognition, (2016) 770-778
[6] Grace J. Gang, J. Webster Stayman, Wojciech Zbijewski, Jeffrey H. Siewerdsen, Task-based detectability in CT image
reconstruction by filtered backprojection and penalized likelihood estimation, Med Phys. (2014) , doi:
10.1118/1.4883816
[7] Gabriel Herl, Jochen Hiller, Andreas Maier, Scanning trajectory optimisation using a quantitative Tuybased local quality
estimation for robot-based X-ray computed tomography, Nondestructive Testing and Evaluation, 35 (2020) 287-303
[8] Á. González, Measurement of Areas on a Sphere Using Fibonacci and Latitude–Longitude Lattices, Math Geosci 42, 49
(2010)
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.
12th Conference on Industrial Computed Tomography, Fürth, Germany (iCT 2023), www.ict2023.org
Reference Volume
Reference Volume
Line Plot
80 Projections
80 Projections
Line Plot
50 Projections
50 Projections
Line Plot
30 Projections
30 Projections
Line Plot
Figure 5: Qualitative results of the reconstruction of our presented approach. The inserted defect is visible in each of the
reconstructed slices. The major difference can be observed at the edges of the objects. Additionally, we see artefacts occurring
in the background when using an iterative reconstruction approach with a relatively low number of projections.
Copyright 2022 - by the Authors. Licensed under a Creative Commons Attribution 4.0 International License.