Bundle Adjustment in a Large: A DSR Critique

Kseniya Cherenkova
PhD school of Informatics
TUWien, Austria

June 6, 2011

1 Introduction
This paper presents the critique of the work [1] from the point of view of Design Science
Research (DSR). The main idea behind the current paper is to demonstrate the main
objectives of DSR paradigm on the example of analyzing the paper [1] in the context of
DSR.
The DSR consists of two basic activities namely building and evaluating. Building is
concerned with the constructing an artifact for a specific purpose; evaluation is aimed to
prove how well the artifact performs. So, we are to say that any paper that is referred
as DSR paper should contain these two blocks.
The [1] presents a perfect example of DSR oriented paper. It builds the solution to
Bundle Adjustment problem for the large datasets of photographs (further referred as
BAL). This problem has gained a lot of attention from Computer Vision community quite
recently as the part of fundamental problem of 3D scene reconstruction from multiple
images. The specific of the multi-view reconstruction for Community Photo Collections
(CPC) is the extremely large amount of images which are unstructured, uncalibrated
and differently illuminated.
In the scope of the paper the BAL algorithm will be described as the building step for
solving the Structure from Motion (SfM) problem. There will be demonstrated how
well [1] corresponds to the structure of DSR paper and how well the guidelines are
followed. The main attributes of DSR research like building artifact and evaluation are
also discussed.
The rest of the paper is organized as following: Section 2 gives the insights into SfM
technology and the basics of the BA procedure; Section 3 describes the solution to
BAL problem proposed in [1]; in Section 4 gives the critique of the paper [1]; Section 5
concludes the paper.

2 Bundle Adjustment
Before moving to the critique of the paper it is necessary to locate BAL in the current
state of the art. Further I introduce the field of studies of the work [1] and give a rather

1
brief description of BAL problem. The BA research has found its origin in recent years
with the development and popularization of the web Internet services for Community
Photo Collections (CPC). Such services as Flickr or Picasa Google nowadays contain the
abundance of photographs related to the same scene, piece of architectural heritage or
(well)known landscape. CPC has a very varying nature: all these different landmarks
and cities have been photographed many different times, both from the ground and
from the air, from almost every viewing position and angle, different times of day and
night, changes in season, weather and during different events. So, the specific of CPC on
the Internet includes unorganized, uncalibrated photos taken with variable illumination,
resolution and quality. How can we approach this enormous amount of information? As
you may reasonably assume, having several views of the same object, it is possible to
reconstruct the three-dimensional shape of it. The problem of estimating the locations
of 3D points from multiple images is formulated as Structure from Motion(SfM).
The SfM pipeline consists of the following steps presented in Figure 1:

Figure 1: MVS pipeline: a) input images b) feature detection, c) feature matching d)

SfM

The SfM computing uses only on a set of correspondences between image features. The
process involves simultaneously estimating both 3D geometry (structure) and camera
pose (motion). In this case, the pipeline can be split into stages as displayed on Figure
1. During the feature detection (extraction) stage, each image is searched for features
- locations that are likely to match well in other images. The feature matching stage
efficiently searches for likely matching candidates in other images. The final step in SfM
requires to reconstruct the 3D points cloud using one of the SfM methods. This stage is
the focus of the [1].
The basic mathematics of the BA problem are well understood [4]. According to it, BA
is concerned with the finding 3D point positions and camera parameters using non-linear
least squares algorithms to minimize reprojection error. This minimization problem is
usually solved with algorithms such as Levenberg-Marquardt (LM). The LM algorithm is
augmented to use so called Schur complement trick to solve the problem for the reduced
camera matrix S. This method is called Sparse Bundle Adjustment (SBA)[2]. There is a
freely available high-quality implementation of SBA [3]. The method of choice for solving

2
symmetric positive definite systems exactly is via the Cholesky factorization. There are,
in general, two options. The first is explicit-direct factorization and the second option is
sparse direct methods. For precise mathematical details, please refer to the text of the
paper and its list of references. Let’s only say, that sparse direct methods, depending
on the exact sparsity structure of the Schur complement, allow BA algorithms to signif
However, it is not enough for CPC. First of all, SBA has space complexity that is
quadratic and time complexity that is cubic in the number of photos. While this works
well for problems with a few hundred photos, for problems involving tens of thousands of
photos from CPC, it is prohibitively expensive. Secondly, the development of largescale
BA algorithms until now have been suited for video and structured survey datasets such
as street-level and aerial imagery. For these datasets, the connectivity graph(the graph
in which each photo is a node, and two photos are connected if they have correspondent
features) is extremely sparse, and has a mostly band-diagonal structure with a large
diameter. Meanwhile, the connectivity graph for CPC tends to be less structured and
have a significantly smaller diameter.

3 Proposed solution to BAL

The [1] addresses the described challenges for BAL problem and proposes the method to
deal with them. The factorization methods described above are based on computing an
exact solution of minimization problem. But it can be noticed that the exact solution
is not absolutely necessary at each step of the LM algorithm. Indeed, it is possible to
construct non-linear optimization algorithms in which the linearized problem is solved
approximately. These algorithms are known as truncated Newton methods.
For approximately solving systems of linear equations an preconditioned Conjugate Gra-
dients method is used. As for the preconditioners the three simple were used: explicit-
jacobi, implicit-jacobi (applied to reduced camera matrix) and explicit-ssor. These pro-
posed alternative methods do not depend on the construction, storage, and factorization
of reduced matrix and yet give good performance on large problems.

4 Critique
The fact how well the paper follows DSR in IS helps the judgment of researches, reviewers
and readers to understand the content and relevance of the paper as well as to estimate
the effectiveness of the proposed solution. I alredy mentioned that [1] was choosen
as a well example of DSR paper. Thus further I demonstrate how well it fulfills the
DSR structure. The research [1] clearly addresses all the guidelines. The structure of
the chapters of it fully corresponds to the Design Research Guidelines. The following
paragraphs give the comments on each of the guidline in a relation to [1].

1)Design as an Artifact : The resulting artifacts of the BAL project are the design and
implementation of a new algorithm for solving BAL problem for thousands of
images. The fact that the code, test problems and detailed performance data are

3
 freely available at [5] makes this project extremly useful for the Computer Vision
community.
2)Problem Relevance : The problem of BAL has been never solved before for CPC. This
fact is clearly stated in the research.
3)Design Evaluation : In the scope of the BAL project 6 BA algorithms had been evalu-
ated, among them there are explicit-direct, explicit-sparse, normal-jacobi, explicit-
jacobi, implicit-jacobi and implicit-ssor. The evaluation was performed for several
datasets from two source: Ladybug camera mounted on moving vehicle and Flickr
images. This choice of the testbed is explained by the necessity to compare the
possibilities of the algorithms
The results of the evaluation are discussed in terms of performance, memory us-
age and reconstruction error. The experiments show that proposed methods, even
when paired with relatively simple preconditioners, offer state of the art perfor-
mance for BAL. Moreover, based on the evaluation and analysis the authors give
recommendation concerning which algorithm is more suitable for which type of
data. This brings another considerable contribution to the knowledge base. Fi-
nally, it is observed that for large scale problems, the iterative methods are a
significant memory and time win over Cholesky factorization-based methods. The
thorough evalutation allowed to prove the usefulness and good quality of the tran-
cated Newton method when applied to BAL.
4)Research Contributions : Contribution to the knowledge base is due to introducing
new methods for the specific study domain. Also the experiments were carried out
for different combinations of datasets and algorithms. This allowed to defined the
best performing combination and draw conclusions on the applicability of each of
the method. The outcome algorithm improves performance of BAL solving and
extends theory to apply it to BA problem.
5)Research Rigor : The paper gives an extremely rigor, simple but at the same time
sufficient for understanding description of the exploited mathematical methods.
The rigor justification is based on the correctness of the applied mathematical
methods. The authors don’t go into the hard details of the mathematical theory
behind the solving non-linear minimization problem, however the structure has a
logic that can be quite easily followed when posessing basic knowledge in the field.
6)Design as a Search Process : The only weak point of the paper is that it lacks the
implementation details. It states only that all six algorithms were implemented
as part of a single C++ code base using GotoBLAS2 for dense linear algebra and
CHOLMOD for sparse Cholesky factorization. However, the code is availabale [5]
to anyone who wishes to dig deeper into the implementation.
7)Communication of Research : The paper is really new, it was published in 2010, so
the number of citations is not high yet. But objectively it has a high potential due
to its novely and particular relevance.

4
Place in DSR Knowledge Contribution Framework As it was mentioned before, the prob-
lem of has become popular in 5 last years. It was never solved before in the conditioners
of CPC data. That allows to address it as a completely new problem. The BA prob-
lem itself then reformulated as a purely mathematical that makes it possible to directly
adopt the methods from nonlinear algebra and system numerical solution. This reasoning
places the BAL in the exaptation quadrant of DSR knowledge contribution framework.

Future research. The future research possibilities as they are discussed by the authors
are the following. The preconditioners considered in this paper are relatively simple. So,
the one possible future work may be concerned with the discovering of the using much
more sophisticated preconditioners developed for solving nonlinear systems. Another
task is to explain the numerical instability of the implicit-jacobi algorithms and to de-
velop method that accounts for the numerical stability of evaluating the matrix-vector
products using the explicit and the implicit-schemes.

In summary, I’d like to say that the paper [1] can be seen as quite brilliant example
of the DSR paper. The concepts of DSR are clearly demonstrated in the text. Moreover,
[1] gives a highly positive impression. It presents technical details on a good level and
still able to keep interest of less prepared audience and provide a sufficient understanding
of the importance of the problem and its novelty as well as the utility of the proposed
method.

5 Conclusions
The paper talks about BAL research [1] with respect to the concept of Design Science.
Considering the main ideas behind DSR the research The results of the evaluation and its
analysis demonstrate that the proposed method improves the state-of-the-art technology
and also contribute to the knowledge base in BA for the CPC.

References
[1] Sameer Agarwal, Noah Snavely, Steven M. Seitz and Richard Szeliski, Bundle Ad-
justment in the Large, ECCV, 2010

[2] M. Lourakis and A. Argyros, SBA: A Software Package for Generic Sparse Bundle
Adjustment, TOMS 36, 2009

[3] http://www.ics.forth.gr/lourakis/sba/

[4] Triggs, B., McLauchlan, P., R.I, H., Fitzgibbon, A.: Bundle Adjustment - A modern
synthesis. Vision Algorithms, pp. 298-372, 1999

[5] http://grail.cs.washington.edu/projects/bal