Professional Documents
Culture Documents
Document
Series
M&CT-TECH-02-004
March 2002
John T. Betts
Joerg M. Gablonsky
M&CT-TECH-02-004
March 2002
Phantom Works
Mathematics and Computing Technology
P.O. Box 3707, M/S 7L-21
Seattle, WA 98124-2207
Jorg M. Gablonsky
Abstract
This paper compares four dierent methods to solve sparse Nonlinear
Programming Problems (NLP)s arising out of the discretization of optimal control problems. The four methods compared are SNOPT, SPRNLP,
LOQO and BARNLP. Of these the rst two are SQP methods, whereas
the other two are interior point methods. A set of 173 test problems
varying both in size and diculty is used for this comparison.
Introduction
Test problems
A collection of nonlinear programming test problems derived from optimal control applications is given in Reference [7]. These problems have been collected
over more than a decade from practical applications in aerospace trajectory optimization and optimal control. The suite contains problems of dierent size,
nonlinearity, and diculty and have been used in previous algorithmic comparisons [5]. The suite was chosen because it is very representative of the target
applications for the nonlinear optimization software.
Solvers
This section provides a short overview of the four dierent methods that are
used in our comparison. Our intent is only to provide a brief overview of the
dierences in the methods, and we will not attempt a detailed description. We
refer the reader to [14, 12, 11, 10, 13, 9, 3, 6, 4] for more information. In the
following we rst standardize the notation of the NLP.
All four algorithms solve the N onlinear P rogramming P roblem (NLP). In
this problem we want to minimize the scalar objective function
f (x)
(1)
(2)
xL x xU ,
(3)
where x Rn [3].
4000
max(20000, 50m)
max(1500, 10m)
max(1000, 2m)
1.D-8
1.D-9
1.D-10
1.D-6
0
3.1
Sequential quadratic programming (SQP) methods solve a nonlinear programming problem (NLP) by solving a sequence of Quadratic Programming (QP)
problems.
The two (SQP) methods we use dier mainly in how they
approximate the Hessian,
choose the step length in the line search, and
nd the solution of the (QP).
3.1.1
SNOPT
For the purpose of these computations, we used SNOPT version 5.3-4. SNOPT
uses a BFGS quasi-Newton update with safeguards to maintain a positive denite Hessian approximation, see [10]. This means that SNOPT only uses rstorder derivative information and cannot exploit sparsity information in the Hessian. This inability to exploit sparsity information is a major disadvantage for
SNOPT when applied to large problems. Even though SNOPT uses a dense
quasi-Newton Hessian, in contrast to the other algorithms which exploit a full
sparse Hessian, it is included in our tests because of its widespread use.
Table 1 shows the settings we used for SNOPT which dier from the standard
settings. Computations were done using both forward sparse dierencing and
centered sparse dierencing to approximate the rst derivatives. The latter
provides more accuracy for the cost of extra function evaluations, and is what
we used in the reported results. The higher accuracy allowed us to solve more
of the test problems.
SNOPT gives the user the option to dene linear parts both of the objective
function, as well as the constraints. Since our NLPs were created by SOCS ,
we were not able to supply this information. Therefore, by necessity, the whole
objective function and all constraints were treated as nonlinear.
In addition, SNOPT allows the user to instruct the optimizer that an error
occurred during the function or constraint evaluation. In spite of this feature
3
SPRNLP
SPRNLP uses rst and second order information to create the quadratic approximation. If the Hessian is not positive denite, it is modied using a Levenberg
parameter. Furthermore SPRNLP incorporates four general strategies. These
are:
M
Minimize. Optimize from the start.
FM
Feasible Minimize. Find a feasible point and then minimize the
function.
FME Feasible Minimize Equality. Find a feasible point and then
minimize the function subject to equalities.
F
Feasible. Find a feasible point.
The rst strategy is the most aggressive one, since it assumes that the starting point is already close to the solution. This is very often the case when grid
renement is done, and the solution from a courser grid provides a good starting
point for a ner grid. The most conservative of the three optimization strategies
is the third one, which in most cases is the slowest. A good mixture of the two
other strategies is the second, which is also the default for SPRNLP. It combines
robustness and good performance. All numerical results in this report used the
second strategy.
Finally the fourth strategy is used if no optimization is required and some
problems only require nding a feasible point.
To solve the underlying systems of linear equations both Boeing codes use
the ecient multifrontal algorithm [1].
3.2
LOQO
itnlim
honor bnds init
bndpush
sf req
inftol
900
0
.1
9
1.0e-08
Numerical results
We start this section by summarizing the overall results, and then proceed with
a more detailed comparison of the dierent methods in Sections 4.2 and 4.3.
5
Converged
Warning
164
162
137
104
1
4
16
1
Local
solution
8
5
6
4
Failure
0
2
14
64
Total
time
6271
12546
1509327
65513
4.1
Table 3 shows the overall performance of the methods. The column labeled
converged lists how many times each method converged to the best solution
found by all four methods. The next column, labeled warning, shows how
many times each method found the best solution (or a feasible solution with a
function value very close to the best found solution), but reported a warning.
For example, the method might have stopped because of small step size, thus
not satisfying its convergence criteria. Nevertheless, the best point found was
feasible and the value of the function at this point was close to the minimum
value found.
The fourth column lists the number of times each method converged to a
local solution, that is a feasible solution whose function value is larger then the
best function value found by more then 1.0e-4.
Finally the last column lists the number of times each method failed to nd
a solution. This could mean that the method could not nd a feasible point, or
did not converge.
This table shows that the two Boeing methods are very robust, and can
solve a very high percentage of all problems. SNOPT is also able to solve a high
percentage of the problems, although it creates more warning messages then all
other methods. Finally, LOQO fails on nearly one-third of the problems. Later
on we will go into more details and show that for certain classes of problems
LOQO is a very ecient solver.
In Table 4 we show both the total time needed for all 173 problems (this
includes failures to converge), and the time needed to solve a subset of 97 problems where all methods found similar solutions. These timings were done on a
4.2
In this section we take a more detailed look at the results. Before we do this,
we explain how we compare dierent solutions. While some of the comparisons
are one to one, in other cases, we compare all four methods with each other.
We used the results calculated by SPRNLP as the reference solution, even if
another algorithm produced a better solution, that is a feasible point whose
function value is lower by more than 1.e-4 than the lowest (feasible) function
value found by SPRNLP.
When comparing function values, we give the order of magnitude of the relative dierences. That is, when comparing f 1 and f 2 , using f 2 as the reference
value, we look at
|f 1 f 2 |
rf = log10
.
(4)
max{|f 2 |, 1}
Similarly, when comparing two points x1 and x2 , using x2 as a reference
value, we look at
x1 x2
rx = log10
.
(5)
max{x2 , 1}
We used the innity norm to measure the maximum discrepancy in the coordinates.
4.2.1
log(|f f*|/max(|f*|,1))
4
6
8
10
10
12
12
14
0
10
20
30
40
50
60
% of problems
70
80
90
100
16
4
6
8
10
10
12
12
14
0
10
20
30
40
50
60
% of problems
70
80
90
100
16
log(|f f*|/max(|f*|,1))
18
16
2
Relative time
12
10
4
6
log(|f f*|/max(|f*|,1))
14
8
6
10
4
12
2
10
20
30
40
50
60
% of problems
70
80
90
100
16
accuracy. Problems are grouped such that for all problems in the rst group,
the value of rf was less than or equal to -12. In the second group the value of
rf lies between -12 and -10, and so on. The white block on the right represents
the problems were LOQO did not nd a point that satises either the nonlinear
constraints or the bounds on the variables to within convergence tolerance. In
this case approximately 36% of the problems were not solved by LOQO.
In Figure 2 we interchange rf and rx . That means now the y-axis measures
the agreement in the coordinates, whereas the darkness measures the agreement
in the objective function value.
Finally, Figure 3 compares the relative solution times for the two dierent
methods. Note that for nearly 20 % of the problems LOQO is at least as fast
or faster than SPRNLP.
Note again that we use the solution from SPRNLP as our basis for this
comparison, even if SPRNLP only found a local solution. As seen in Table 3,
this happens only eight times, a low percentage of all problems. This means
that some of the times when the solution of SPRNLP and LOQO diers greatly,
LOQOs solution is actually better.
The large percentage of failures prevent LOQO from being really competitive
with the other methods. In a recent study [2] the authors of LOQO report that
for equality constraint NLPs the default initialization parameters for LOQO
often fail, and further research in this area is needed. For use in an industrial
setting LOQO needs to include methods to handle function errors. When LOQO
solves a problem, it often does so in an ecient way.
6
4
log(|f f*|/max(|f*|,1))
8
10
10
12
12
14
0
4.2.2
10
20
30
40
50
60
% of problems
70
80
90
100
16
Figures 4 through 6 compare the barrier code BARNLP and SPRNLP, using
the same series of graphs as we did for LOQO. From Figure 4 it is clear that
the optimal function values from these two methods agree to a high precision
for nearly 80% of all problems. But, as is emphasized in Figure 5, this does not
mean that the two methods nd the same solution point. Instead, the solution
can dier considerably, even though the function value is nearly the same. One
reason for this could be that the optimal solution is very insensitive to changes,
in other words, the optimum is in a at valley. Finally, Figure 6 shows that the
barrier method is as fast, or faster than the SPRNLP method for about 20%
of the problems. And for about 50% of the problems BARNLP needs, at most,
twice as long to obtain a solution. However for nearly half the test set BARNLP
is considerably slower than SPRNLP. In Section 4.3.2 we come back to this and
show that, as expected, certain kinds of problems seem to be better suited for
interior point methods while others are better suited for SQP methods.
4.2.3
Finally Figures 7 through 9 compare the SNOPT code and SPRNLP, using
the same series of graphs as before. Figure 7 and 8 show that the function
value of the SNOPT solution agrees to a high precision with the function value
of the SPRNLP solution, although the coordinates of the solution may dier
considerably. As pointed out earlier, SNOPT only uses rst order information,
10
4
6
log(|f f*|/max(|f*|,1))
8
10
10
12
12
14
0
10
20
30
40
50
60
% of problems
70
80
90
100
16
18
16
2
Relative time
12
10
4
6
8
6
10
4
12
2
10
20
30
40
50
60
% of problems
11
70
80
90
100
16
log(|f f*|/max(|f*|,1))
14
Figure 7: Comparison between SNOPT and SPRNLP, focus on objective function value.
SNOPT versus SPRNLP, SPRNLP value as reference
2
log(|f f*|/max(|f*|,1))
4
6
8
10
10
12
12
14
0
10
20
30
40
50
60
% of problems
70
80
90
100
16
4
6
8
10
10
12
12
14
0
10
20
30
40
50
60
% of problems
12
70
80
90
100
16
log(|f f*|/max(|f*|,1))
18
16
2
Relative time
12
10
4
6
log(|f f*|/max(|f*|,1))
14
8
6
10
4
12
2
10
20
30
40
50
60
% of problems
70
80
90
100
16
and does not exploit Hessian sparsity. Therefore it is not surprising that the
run time for many problems are considerably larger, and only for about 10 %
of the problems does SNOPT demonstrate a run time advantage. The strength
of SNOPT lies in its dependability.
4.3
To look at the results in more detail, we divided the problems up into classes,
and report how the dierent methods performed on these classes. The rst
classication is based on the size of the problem, whereas the other classication
is based on the diculty for an SQP method to solve a problem.
4.3.1
Classification by size
13
Size
0 < N 100,
100 < N 500,
500 < N 1000,
1000 < N 2000,
2000 < N,
Number of Problems
51
50
39
26
7
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
50
46
45
43
1
3
4
0
0
1
0
3
0
1
2
5
Total
time
50
48
36
144
Time to solve
43 problems
all methods
could solve
46
34
33
47
Classication by size does not take into account the computational complexity
of the problems. In particular, for problems with many inequality constraints,
one might expect a performance degradation caused by the computational complexity of an active set algorithm such as the SQP codes SNOPT and SPRNLP.
Method
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
Total
time
46
48
43
29
0
0
4
0
4
2
2
1
0
0
1
20
342
562
2497
6070
Time to solve
27 problems
all methods
could solve
163
362
1710
311
Table 7: Performance of the methods for problems of class S2 (100 < N 500).
14
Method
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
Total
time
37
39
29
17
0
0
4
0
2
0
4
0
0
0
2
22
894
1912
12694
19842
Time to solve
15 problems
all methods
could solve
265
782
5131
583
Table 8: Performance of the methods for problems of class S3 (500 < N 1000).
Method
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
24
22
16
10
0
1
3
0
2
2
0
0
0
1
7
16
Total
time
3227
7374
197417
33550
Time to solve
8 problems
all methods
could solve
213
407
80908
354
Method
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
7
7
4
5
0
0
1
1
0
0
0
0
0
0
2
1
Total
time
1758
2650
296684
5906
Time to solve
4 problems
all methods
could solve
1241
711
1290643
2981
Table 10: Performance of the methods for problems of class S5 (2000 < N ).
15
Complexity
1 hSQP 3,
3 < hSQP 5,
5 < hSQP 10,
10 < hSQP 20,
20 < hSQP ,
Number of Problems
23
60
52
25
13
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
Total
time
23
22
18
15
0
1
2
0
0
0
1
1
0
0
2
7
497
2333
8700
3174
Time to solve
13 problems
all methods
could solve
70
325
3446
360
Table 12: Performance of the methods for problems of complexity class H1.
This should be particularly apparent when a comparison is made with an interior point algorithm. Therefore we introduce another classication of the
problems, which is based on the following observation: a problem is hard for an
SQP method if the active set changes a lot from one nonlinear iteration to the
next. We can measure this by looking at the number of QP iterations q and the
number of gradient calls ng . A measurement of how hard a problem is for an
SQP method is then given by
hSQP =
q
.
ng
(6)
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
58
58
53
38
0
2
4
0
2
0
1
0
0
0
2
22
Total
time
953
1979
1289917
14690
Time to solve
36 problems
all methods
could solve
527
1109
1286919
1133
Table 13: Performance of the methods for problems of complexity class H2.
Method
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
Total
time
49
47
40
30
1
1
6
0
2
3
1
2
0
1
5
20
954
1292
24064
15617
Time to solve
27 problems
all methods
could solve
44
78
477
82
Table 14: Performance of the methods for problems of complexity class H3.
Method
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
Total
time
24
23
16
14
0
0
4
0
1
1
1
1
0
1
4
10
2093
5979
75437
22611
Time to solve
14 problems
all methods
could solve
129
299
2823
235
Table 15: Performance of the methods for problems of complexity class H4.
Method
SPRNLP
BARNLP
SNOPT
LOQO
Converged
Warning
Local
sol.
Failed
10
12
10
7
0
0
0
1
3
1
2
0
0
0
1
5
Total
time
1773
963
111210
9420
Time to solve
7 problems
all methods
could solve
1158
486
84761
2466
Table 16: Performance of the methods for problems of complexity class H5.
17
Conclusions
We compared four dierent methods to solve NLPs. Two of these were SQP
methods, and two were interior point methods. Overall the two Boeing implementations proved to be both most ecient and most successful. In a direct
comparison between the two Boeing methods, overall the SQP method has an
advantage, but for certain classes of problems the interior point method performs better. SNOPT is a very dependable method, since it can solve a large
percentage of the test problems. However since it is a quasi-Newton method,
and especially because it does not exploit Hessian sparsity, it needs much more
time to solve the problems. This becomes very apparent once the problems grow
in size. Finally, LOQO also eciently solves the problems which are hard for the
SQP methods, but overall is not as dependable as the other implementations.
Acknowledgments
The authors are grateful to C.T. Kelley, R.J. Vanderbei and Paul D. Frank for
their help in preparing this report and their comments.
References
[1] Cleve Ashcraft, Roger G. Grimes, and John G. Lewis. Accurate Symmetric Indenite Linear Equation Solvers. SIAM Journal of Matrix Analysis,
20:513561, 1999.
[2] H. Y. Benson, D. F. Shanno, and R. J. Vanderbei. A Comparative Study of
Large-Scale nonlinear Optimization Algorithms. Technical Report ORFE01-04, Princeton University, 2001.
[3] John T. Betts. Practical Methods for Optimal Control using Nonlinear Programming. Society for Industrial and Applied Mathematics, Philadelphia,
PA., 2001.
[4] John T. Betts, Samuel K. Eldersveld, Paul D. Frank, and John G. Lewis.
An Interior-Point Nonlinear Programming Algorithm for Very Large Scale
Optimization. Mathematics and Computing Technology Report M&CTTECH-003, Mathematics and Computing Technology, The Boeing Company, PO Box 3707, Seattle, WA 98124-2207, April 2000.
[5] John T. Betts, Samuel K. Eldersveld, and William P. Human. A Performance Comparison of Nonlinear Programming Algorithms for Large Sparse
18
In the following pages we give detailed results for all four methods on the 173 test
problems. We used three dierent discretizations, H stands for Hermite Cubic
19
IERNLP
-133
101
106
107
108
110
IERNLP
-160
104
105
108
116
IERNLP
-4242
-1005
-1004
-1002
20
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
1
alpr01
29
1
H
166
121
0
9.15
-10
-15
-10
574
11.04
0
13.64
-11
-2.41e-08
-4
-6
-14
-15
-14
105
60.84
-3
-3.54e-01
0
0
-12
-15
-12
0
14.85
-11
-2.75e-08
-4
-6
-14
-15
-14
2
aotv01
15
1
T
176
171
0
8.45
-10
-15
-10
816
20.92
0
7.32
-3
-1.85e-03
0
0
-10
-15
-10
105
22.59
-2
1.79e-02
0
0
-8
-15
-8
-1002
104.82
0
1.73e+00
0
0
0
-2
0
21
3
aqua01
37
1
T
140
90
0
2.20
-12
-15
-12
384
24.00
0
0.73
-10
9.03e-09
-8
-9
-13
-15
-13
0
2.00
-13
4.04e-12
-6
-7
-12
-15
-12
0
1.42
-10
-1.01e-08
-7
-8
-10
-11
-10
4
arao01
25
1
T
50
24
0
0.15
-11
-15
-11
39
2.79
0
0.16
-14
4.55e-12
-7
-7
-13
-15
-13
105
0.22
-14
5.46e-12
-7
-7
-13
-15
-13
0
0.11
-10
-1.96e-07
-7
-7
-6
-13
-6
5
arao02
25
2
T
75
48
0
23.10
-10
-15
-10
3963
3.00
0
0.09
-10
1.69e-10
-6
-6
-12
-15
-12
0
2.94
-10
1.68e-10
-6
-6
-12
-15
-12
0
4.38
-9
-9.52e-10
-6
-6
-9
-15
-9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
6
ashr01
17
1
T
20
18
0
0.01
-14
-15
-14
3
3.00
0
0.01
-14
0.00e+00
-15
-14
-14
-15
-14
0
0.00
-14
0.00e+00
-12
-12
-12
-15
-12
0
0.02
-14
0.00e+00
-10
-10
-9
-11
-9
7
ashr02
17
2
T
20
18
0
0.01
-13
-15
-13
3
3.00
0
0.01
-14
0.00e+00
-13
-13
-14
-15
-14
0
0.00
-14
0.00e+00
-12
-12
-12
-15
-12
0
0.03
-14
0.00e+00
-7
-7
-10
-11
-10
22
8
ashr03
17
3
T
20
18
0
0.01
-14
-15
-14
3
3.00
0
0.01
-14
0.00e+00
-14
-14
-15
-15
-15
0
0.03
-14
0.00e+00
-11
-11
-12
-15
-12
0
0.02
-14
0.00e+00
-10
-10
-10
-11
-10
9
ashr04
17
4
T
20
18
0
0.01
-14
-15
-14
3
3.00
0
0.02
-14
0.00e+00
-14
-14
-15
-15
-15
0
0.00
-14
0.00e+00
-12
-12
-12
-15
-12
0
0.02
-14
0.00e+00
-12
-12
-10
-13
-10
10
ashr05
17
5
T
80
78
0
1.39
-8
-15
-8
144
3.69
101
5.15
-14
0.00e+00
-1
-1
-14
-15
-14
108
0.78
-14
0.00e+00
0
0
-1
-15
-1
-1002
19.36
0
1.91e+00
0
0
308
-4
308
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
11
bang01
22
1
T
31
18
0
0.14
-10
-15
-10
58
5.80
0
0.10
-7
8.47e-08
-6
-6
-11
-15
-11
0
0.00
-9
-8.07e-10
-8
-8
-8
-9
-8
0
0.04
-8
1.05e-09
-7
-7
-9
-10
-9
12
brac01
10
1
T
41
27
0
0.35
-8
-15
-8
137
8.06
0
0.85
-8
1.05e-09
-3
-3
-8
-15
-8
0
0.28
-8
1.04e-09
-6
-6
-12
-15
-12
0
0.11
-8
1.06e-09
-3
-3
-7
-13
-7
23
13
brac02
10
2
T
41
37
0
0.36
-9
-15
-9
140
8.24
0
0.39
-8
3.60e-09
-3
-3
-7
-15
-7
105
0.31
-10
9.78e-11
-5
-5
-12
-15
-12
0
0.20
-10
8.71e-11
-4
-4
-10
-15
-10
14
brgr01
5
1
T
100
98
0
1.02
-14
-15
-14
121
7.12
108
5.20
-14
0.00e+00
0
0
0
-15
0
108
0.42
-14
0.00e+00
0
0
0
-15
0
-1002
22.99
-14
0.00e+00
0
0
0
-3
0
15
brn201
28
1
T
307
257
0
22.65
-11
-15
-11
570
22.80
0
14.16
-11
5.07e-12
-3
-3
-11
-15
-11
0
152.38
-10
5.92e-11
-2
-2
-12
-15
-12
-1005
517.50
0
1.38e+00
1
1
8
-15
8
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
16
brn202
28
1
T
307
257
0
21.87
-9
-15
-9
396
18.00
0
11.56
-9
7.23e-10
-2
-2
-12
-15
-12
0
176.22
-9
7.36e-10
-3
-3
-12
-15
-12
-1004
209.86
0
1.38e+00
1
1
8
-15
8
17
brn203
28
2
T
271
257
0
12.42
-12
-15
-12
79
4.39
0
12.95
-10
-3.20e-11
-3
-3
-10
-15
-10
0
50.91
-10
1.39e-11
-2
-2
-12
-15
-12
-1004
201.01
0
1.38e+00
1
1
15
-15
15
24
18
brn204
28
2
T
271
257
0
19.53
-10
-15
-10
130
5.65
0
8.51
-10
1.31e-11
-3
-3
-10
-15
-10
0
37.28
-10
5.04e-11
-3
-3
-12
-15
-12
-1004
215.06
0
1.38e+00
1
1
15
-15
15
19
capt01
8
1
T
587
574
0
4.33
-10
-15
-10
24
2.67
0
36.88
-9
1.35e-09
-5
-5
-9
-15
-9
0
11.11
-10
1.45e-11
-4
-4
-9
-15
-9
-1002
141.90
0
2.38e+00
0
0
-1
-1
-1
20
capt02
8
1
T
587
574
0
4.37
-9
-15
-9
21
2.33
0
31.29
-8
1.67e-09
-5
-5
-11
-15
-11
0
19.88
-10
8.64e-11
-4
-4
-10
-15
-10
-1002
167.15
0
2.38e+00
0
0
-1
-2
-1
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
21
capt03
8
2
T
587
574
0
7.00
-10
-15
-10
32
3.20
0
33.92
-8
2.59e-09
-5
-5
-11
-15
-11
0
9.61
-11
3.02e-12
-4
-4
-9
-15
-9
-1002
139.44
0
1.70e+00
0
0
-1
-2
-1
22
capt04
8
2
T
587
574
0
7.12
-9
-15
-9
30
3.00
0
31.25
-8
3.24e-09
-5
-5
-10
-15
-10
0
19.52
-10
4.75e-11
-5
-5
-11
-15
-11
-1002
139.95
0
1.70e+00
0
0
-1
-2
-1
25
23
capt05
8
3
T
587
574
0
4.33
-10
-15
-10
24
2.67
0
34.20
-9
1.36e-09
-5
-5
-9
-15
-9
0
12.16
-10
4.14e-11
-5
-5
-11
-15
-11
-1002
226.36
0
-9.10e-01
0
0
-1
-2
-1
24
capt06
8
3
T
587
574
0
4.37
-9
-15
-9
21
2.33
0
31.40
-9
1.74e-09
-5
-5
-12
-15
-12
105
27.53
-10
1.60e-10
-4
-4
-10
-15
-10
0
27.35
-10
1.43e-10
-5
-5
-9
-13
-9
25
capt07
8
4
T
813
877
0
31.96
-9
-15
-9
90
3.21
0
94.92
-5
7.74e-06
-1
-1
-8
-15
-8
0
111.25
-5
-2.72e-06
-1
-1
-10
-15
-10
-1002
171.49
0
3.30e+00
0
0
4
-2
4
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
26
capt08
8
4
T
813
877
0
16.70
-11
-15
-11
145
8.53
0
42.37
-8
1.60e-09
-5
-5
-10
-15
-10
0
51.58
-11
-6.90e-12
-4
-4
-9
-15
-9
-1002
183.65
0
3.30e+00
0
0
4
-2
4
27
capt09
8
5
T
1064
1063
0
46.31
-8
-15
-8
118
4.21
0
74.75
-7
5.88e-08
-1
-1
-8
-15
-8
0
86.44
-9
-1.06e-09
-1
-1
-10
-15
-10
-1002
258.97
0
3.26e+00
0
0
1
-2
1
26
28
capt10
8
5
T
1064
1063
0
9.48
-10
-15
-10
36
3.60
0
77.85
-10
1.13e-10
-3
-3
-8
-15
-8
0
60.84
-10
-2.35e-11
-4
-4
-9
-15
-9
-1002
257.94
0
3.26e+00
0
0
1
-1
1
29
capt11
8
6
T
1290
1366
0
63.49
-9
-15
-9
133
3.91
0
218.12
-5
6.39e-06
-1
-1
-8
-15
-8
0
305.97
-5
-2.71e-06
-1
-1
-9
-15
-9
-1002
312.18
0
3.26e+00
0
0
4
-2
4
30
capt12
8
6
T
1290
1366
0
26.91
-11
-15
-11
70
4.12
0
85.49
-11
1.20e-11
-5
-5
-10
-15
-10
0
143.20
-10
3.97e-11
-3
-3
-9
-15
-9
-1002
307.37
0
3.26e+00
0
0
4
-2
4
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
31
capt13
8
7
T
753
802
0
49.03
-12
-15
-12
523
14.94
0
48.91
-4
-3.59e-05
0
0
-13
-15
-13
105
147.19
-7
-5.28e-08
0
0
-11
-15
-11
-1002
173.17
0
1.31e+00
0
0
0
-2
0
32
capt14
8
8
T
812
876
0
8.73
-9
-15
-9
36
2.77
0
41.70
-11
6.40e-12
-5
-5
-10
-15
-10
0
80.03
-11
7.73e-12
-4
-4
-9
-15
-9
-1002
189.23
0
3.29e+00
0
0
0
-2
0
27
33
chmr01
20
1
T
30
18
0
0.16
-9
-15
-9
62
5.64
0
0.12
-8
2.40e-09
-6
-6
-9
-15
-9
0
0.09
-9
7.34e-10
-7
-7
-12
-15
-12
0
0.61
-9
7.34e-10
-6
-6
-15
-15
-15
34
chmr02
20
2
T
30
18
0
0.20
-10
-15
-10
90
6.92
0
0.13
-8
1.03e-09
-6
-6
-10
-15
-10
0
0.09
-10
4.46e-11
-6
-6
-12
-15
-12
0
0.40
-10
4.55e-11
-6
-6
-15
-15
-15
35
chmr03
20
3
T
30
18
0
0.19
-13
-15
-13
93
8.45
0
0.18
-7
1.00e-08
-7
-7
-12
-15
-12
0
0.03
-13
6.52e-14
-13
-13
-15
-15
-15
0
0.14
-10
1.70e-11
-9
-9
-10
-13
-10
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
36
chmr04
20
4
T
30
18
0
0.15
-12
-15
-12
70
7.78
0
0.10
-7
1.00e-08
-7
-7
-10
-15
-10
0
0.03
-12
2.43e-13
-12
-12
-15
-15
-15
0
0.08
-10
3.19e-11
-9
-9
-11
-13
-11
37
chmr05
20
5
T
30
18
0
0.19
-9
-15
-9
86
6.62
0
0.12
-8
7.61e-09
-6
-6
-10
-15
-10
0
0.09
-9
7.21e-10
-6
-6
-12
-15
-12
0
0.32
-9
7.21e-10
-6
-6
-15
-15
-15
28
38
chmr06
20
6
T
30
18
0
0.14
-11
-15
-11
53
5.30
0
0.14
-8
4.61e-09
-3
-3
-9
-15
-9
0
0.06
-10
1.84e-11
-6
-6
-12
-15
-12
-1005
0.38
-2
5.41e-03
-4
-4
-15
-10
-10
39
chmr07
20
7
T
60
38
0
0.44
-11
-15
-11
106
6.24
0
0.25
-8
1.04e-09
-7
-7
-11
-15
-11
0
0.78
-10
5.59e-11
-9
-9
-12
-15
-12
0
0.59
-10
5.40e-11
-8
-8
-12
-10
-10
40
chmr08
20
8
T
120
78
0
0.84
-9
-15
-9
144
9.00
0
0.60
-8
1.23e-09
-7
-7
-10
-15
-10
0
3.16
-9
3.51e-10
-8
-8
-12
-15
-12
0
1.43
-9
3.44e-10
-8
-8
-12
-10
-10
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
41
chmr09
20
9
T
150
98
0
1.49
-9
-15
-9
258
15.18
0
1.02
-8
3.52e-09
-6
-6
-10
-15
-10
0
4.50
-8
2.63e-09
-7
-7
-12
-15
-12
-1002
32.33
0
1.48e-01
-7
-7
0
-15
0
42
chmr10
20
10
T
30
18
0
0.20
-10
-15
-10
80
5.33
0
0.15
-8
8.96e-09
-5
-5
-9
-15
-9
0
0.06
-4
-5.55e-05
0
0
-13
-15
-13
0
0.15
-11
8.04e-12
-7
-7
-10
-14
-10
29
43
clym01
7
1
R
180
133
0
8.26
-10
-15
-10
98
4.26
0
19.42
-10
1.57e-11
-8
-8
-11
-15
-11
0
15.28
-10
1.50e-11
-6
-6
-12
-15
-12
0
11.68
-9
-3.56e-10
-8
-8
-9
-11
-9
44
clym02
7
1
R
450
343
0
13.83
-8
-15
-8
60
3.75
0
20.19
-8
7.81e-09
-6
-6
-8
-15
-8
0
53.69
-8
3.32e-09
-6
-6
-12
-15
-12
0
37.07
-8
3.31e-09
-6
-6
-10
-13
-10
45
clym03
7
1
R
900
693
0
33.76
-10
-15
-10
78
4.33
0
48.78
-8
2.73e-09
-4
-4
-8
-15
-8
0
169.84
-11
5.96e-12
-5
-5
-12
-15
-12
-1002
5251.89
-4
-9.87e-05
0
0
4
-2
4
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
46
clym04
7
1
T
81
63
0
0.85
-11
-15
-11
56
3.29
0
1.44
-12
5.73e-13
-10
-10
-13
-15
-13
0
0.81
-12
5.62e-13
-8
-8
-11
-15
-11
0
1.36
-9
-4.22e-10
-9
-9
-9
-12
-9
47
clym05
7
1
T
401
343
0
4.34
-11
-15
-11
58
3.62
0
5.84
-12
1.36e-13
-8
-8
-12
-15
-12
0
10.38
-12
1.22e-13
-6
-6
-12
-15
-12
-1002
305.51
-1
5.06e-02
0
0
4
-3
4
30
48
clym06
7
1
T
801
693
0
10.67
-10
-15
-10
81
4.50
0
16.06
-9
3.83e-10
-5
-5
-9
-15
-9
0
44.09
-9
2.13e-10
-6
-6
-12
-15
-12
0
13.12
-9
1.79e-10
-6
-6
-10
-13
-10
49
clym07
7
1
H
90
63
0
2.73
-8
-15
-8
57
3.35
0
4.97
-8
1.92e-09
-7
-7
-11
-15
-11
0
5.31
-8
1.92e-09
-7
-7
-12
-15
-12
0
5.47
-8
1.49e-09
-7
-7
-9
-11
-9
50
clym08
7
1
H
450
343
0
15.54
-12
-15
-12
61
3.59
0
22.70
-8
4.00e-09
-6
-6
-11
-15
-11
0
35.42
-12
1.03e-13
-6
-6
-12
-15
-12
0
31.49
-10
-1.63e-11
-8
-8
-10
-12
-10
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
51
clym09
7
1
H
900
693
0
35.35
-11
-15
-11
76
4.22
0
57.45
-8
6.00e-09
-5
-5
-10
-15
-10
0
137.39
-11
1.96e-12
-5
-5
-12
-15
-12
0
64.46
-9
-1.29e-10
-6
-6
-10
-13
-10
52
clym10
7
2
R
140
95
0
5.62
-10
-15
-10
98
4.26
0
13.07
-10
1.62e-11
-8
-8
-11
-15
-11
0
11.83
-10
1.56e-11
-6
-6
-12
-15
-12
0
7.50
-10
-2.44e-11
-8
-8
-10
-12
-10
31
53
clym11
7
2
R
350
245
0
9.25
-8
-15
-8
60
3.75
0
13.56
-8
7.81e-09
-6
-6
-8
-15
-8
0
41.91
-8
3.32e-09
-6
-6
-12
-15
-12
0
23.15
-8
2.85e-09
-6
-6
-9
-12
-9
54
clym12
7
2
R
700
495
0
22.81
-10
-15
-10
78
4.33
0
32.71
-8
2.75e-09
-4
-4
-8
-15
-8
0
138.34
-10
1.03e-11
-5
-5
-12
-15
-12
0
41.36
-10
-6.66e-11
-6
-6
-10
-13
-10
55
clym13
7
2
T
61
45
0
0.66
-11
-15
-11
56
3.29
0
1.11
-12
5.92e-13
-10
-10
-13
-15
-13
0
0.73
-12
5.80e-13
-8
-8
-11
-15
-11
0
0.76
-9
-4.20e-10
-9
-9
-9
-11
-9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
56
clym14
7
2
T
301
245
0
3.16
-10
-15
-10
58
3.62
0
4.37
-10
1.42e-11
-8
-8
-12
-15
-12
0
9.27
-10
1.42e-11
-6
-6
-11
-15
-11
-1002
157.59
-1
5.06e-02
0
0
4
-2
4
57
clym15
7
2
T
601
495
0
7.67
-10
-15
-10
81
4.50
0
17.14
-9
2.66e-10
-5
-5
-9
-15
-9
0
36.86
-9
1.49e-10
-6
-6
-12
-15
-12
0
10.48
-10
8.04e-11
-6
-6
-10
-13
-10
32
58
clym16
7
2
H
70
45
0
1.62
-8
-15
-8
57
3.35
0
3.23
-8
1.93e-09
-7
-7
-11
-15
-11
0
3.52
-8
1.93e-09
-7
-7
-12
-15
-12
0
2.86
-8
1.32e-09
-7
-7
-9
-11
-9
59
clym17
7
2
H
350
245
0
8.85
-11
-15
-11
61
3.59
0
12.84
-8
3.99e-09
-6
-6
-11
-15
-11
0
26.44
-11
-7.30e-12
-6
-6
-12
-15
-12
0
17.29
-10
-7.04e-11
-8
-8
-10
-13
-10
60
clym18
7
2
H
700
495
0
19.83
-11
-15
-11
76
4.22
0
34.25
-8
4.03e-09
-4
-4
-8
-15
-8
0
111.16
-11
-9.69e-12
-5
-5
-12
-15
-12
0
37.03
-10
-4.92e-11
-7
-7
-10
-13
-10
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
61
clym19
7
3
T
81
63
0
2.39
-12
-15
-12
136
4.25
101
3.71
-12
-2.84e-13
-6
-6
-13
-15
-13
0
0.81
-12
-2.98e-13
-6
-6
-11
-15
-11
0
0.82
-10
-5.03e-11
-6
-6
-10
-12
-10
62
clym20
7
4
T
81
63
110
2.92
-8
-15
-8
142
5.26
101
3.27
-6
-1.05e-07
-3
-3
-13
-15
-13
0
0.81
-6
-1.05e-07
-3
-3
-11
-15
-11
0
1.07
-6
-1.05e-07
-3
-3
-10
-13
-10
33
63
dlay01
33
1
T
40
38
0
0.05
-12
-15
-12
9
3.00
0
0.04
-14
0.00e+00
-11
-12
-12
-15
-12
0
0.05
-14
0.00e+00
-10
-11
-11
-15
-11
0
0.08
-14
0.00e+00
-9
-10
-10
-13
-10
64
gdrd01
4
1
R
50
27
0
0.22
-14
-15
-14
72
9.00
0
0.25
-7
1.90e-08
-6
-6
-13
-15
-13
0
0.06
-10
4.64e-11
-9
-9
-10
-15
-10
0
0.21
-9
2.88e-10
-8
-8
-10
-11
-10
65
gdrd02
4
1
T
41
27
0
0.19
-13
-15
-13
54
6.00
0
0.19
-7
1.00e-08
-7
-7
-14
-15
-14
0
0.05
-9
2.11e-10
-9
-9
-9
-15
-9
0
0.09
-9
7.28e-10
-7
-7
-9
-9
-9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
66
gdrd03
4
1
H
50
27
0
0.26
-14
-15
-14
74
9.25
0
0.28
-7
1.90e-08
-6
-6
-13
-15
-13
0
0.06
-10
5.21e-11
-9
-9
-10
-15
-10
0
0.26
-9
3.91e-10
-8
-8
-10
-11
-10
67
gdrd04
4
2
R
250
147
0
3.13
-10
-15
-10
276
19.71
0
1.97
-7
7.82e-08
-4
-5
-13
-15
-13
0
3.23
-10
1.61e-10
-4
-5
-12
-15
-12
0
1.61
-9
3.81e-10
-4
-5
-9
-10
-9
34
68
gdrd05
4
2
T
201
147
0
1.45
-9
-15
-9
157
12.08
0
1.10
-7
4.02e-08
-5
-6
-12
-15
-12
0
1.56
-10
2.31e-10
-6
-6
-12
-15
-12
0
0.79
-9
-5.16e-10
-6
-6
-9
-10
-9
69
gdrd06
4
2
H
250
147
0
7.51
-9
-15
-9
569
21.88
0
3.62
-7
9.61e-08
-5
-5
-10
-15
-10
0
3.70
-4
1.29e-04
0
0
-12
-15
-12
0
5.16
-4
-1.28e-04
0
0
-9
-12
-9
70
gsoc01
24
1
T
825
602
0
55.21
-12
-15
-12
547
10.94
0
116.39
-4
-1.46e-05
0
0
-8
-15
-8
105
427.75
-4
7.08e-05
0
0
-10
-15
-10
-1002
1086.19
0
-1.15e-01
0
0
0
-2
0
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
71
gydn01
9
1
T
181
153
0
2.63
-9
-15
-9
87
4.35
0
4.15
-8
4.11e-09
-5
-5
-10
-15
-10
0
4.97
-10
2.41e-11
-5
-5
-11
-15
-11
0
2.46
-13
9.67e-14
-7
-7
-9
-14
-9
72
gydn02
9
2
T
161
134
0
2.03
-11
-15
-11
75
4.41
0
2.64
-8
4.13e-09
-5
-5
-10
-15
-10
0
4.36
-11
-4.31e-12
-6
-6
-12
-15
-12
-1004
23.24
-10
-3.11e-11
0
0
0
-15
0
35
73
hang01
14
1
T
126
96
0
1.17
-12
-15
-12
95
4.13
0
1.24
-3
-1.13e-03
0
0
-11
-15
-11
0
1.72
-3
-1.13e-03
0
0
-11
-15
-11
0
1.03
-3
-1.13e-03
0
0
-7
-14
-7
74
hang02
14
1
H
150
96
0
2.01
-11
-15
-11
134
5.36
0
1.33
-7
1.80e-08
-5
-5
-11
-15
-11
0
3.72
-12
-3.86e-13
-5
-5
-11
-15
-11
0
1.39
-3
3.92e-04
0
0
-6
-12
-6
75
heat01
23
1
T
140
119
0
0.30
-13
-15
-13
29
5.80
0
0.50
-8
8.09e-09
-4
-4
-13
-15
-13
0
0.38
-14
7.60e-16
-7
-7
-11
-15
-11
0
0.37
-9
7.84e-10
-4
-5
-11
-13
-11
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
76
heat02
23
2
T
2650
2550
0
16.79
-9
-15
-9
15
2.50
0
88.03
-5
-2.09e-06
0
0
-8
-15
-8
0
75.41
-5
-2.09e-06
0
0
-7
-15
-7
0
157.97
-5
-2.09e-06
0
0
-10
-15
-10
77
imp201
26
1
A
8
5
0
0.19
-7
-15
-7
123
12.30
0
0.13
-9
2.21e-10
-6
-7
-9
-15
-9
0
0.19
-9
2.33e-10
-6
-7
-12
-15
-12
0
0.27
-9
2.08e-10
-6
-7
-10
-15
-10
36
78
imp202
26
-1
A
10
7
0
0.25
-11
-15
-11
126
11.45
0
0.17
-10
-3.17e-11
-7
-7
-9
-15
-9
0
0.25
-12
6.30e-13
-9
-9
-12
-15
-12
0
0.35
-11
-4.03e-12
-9
-9
-9
-15
-9
79
imp203
26
2
A
61
52
0
0.45
-10
-15
-10
75
6.82
0
0.34
-11
3.62e-12
-7
-7
-11
-15
-11
0
0.41
-10
-1.54e-11
-7
-7
-12
-15
-12
0
6.01
-11
3.38e-12
-7
-7
-11
-15
-11
80
imp204
26
-2
A
63
54
0
0.28
-9
-15
-9
24
2.67
0
0.60
-10
2.10e-11
-8
-8
-11
-15
-11
0
0.41
-10
2.12e-11
-8
-9
-11
-15
-11
0
8.89
-10
2.26e-11
-9
-9
-10
-15
-10
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
81
jmp201
27
1
T
213
168
0
4.67
-9
-15
-9
54
3.60
0
7.65
-8
2.13e-09
-3
-3
-11
-15
-11
0
27.03
-8
2.14e-09
-3
-4
-11
-15
-11
-1002
983.01
0
2.51e+00
2
1
3
-2
3
82
jmp202
27
2
T
177
168
0
4.98
-10
-15
-10
65
4.33
0
5.95
-11
5.24e-12
-4
-5
-11
-15
-11
0
9.56
-11
-8.45e-12
-5
-5
-11
-15
-11
-1002
1118.15
0
2.50e+00
0
0
0
-4
0
37
83
kplr01
18
1
T
10
10
0
0.03
-12
-15
-12
18
3.00
0
0.03
-14
0.00e+00
-11
-11
-11
-15
-11
0
0.00
-14
0.00e+00
-8
-8
-8
-15
-8
0
0.02
-14
0.00e+00
-9
-9
-9
-15
-9
84
lnts02
2
2
R
60
36
0
0.34
-7
-15
-7
76
5.07
0
0.25
-9
9.39e-10
-4
-5
-12
-15
-12
0
1.60
-9
9.39e-10
-4
-5
-12
-15
-12
0
1.26
-9
6.44e-10
-4
-5
-8
-15
-8
85
lnts03
2
2
R
120
76
0
0.88
-11
-15
-11
134
7.88
0
0.39
-10
-1.06e-11
-6
-7
-9
-15
-9
0
6.90
-14
2.89e-15
-6
-7
-12
-15
-12
0
1.66
-10
-8.25e-11
-8
-9
-9
-15
-9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
86
lnts04
2
2
R
1200
796
0
40.56
-11
-15
-11
1068
62.82
0
7.62
-11
1.55e-12
-6
-7
-12
-15
-12
0
383.22
-12
8.64e-13
-5
-6
-11
-15
-11
0
9.04
-9
-2.62e-10
-7
-8
-9
-11
-9
87
lnts05
2
2
T
51
36
0
0.29
-8
-15
-8
74
3.89
0
0.14
-9
9.77e-10
-4
-5
-13
-15
-13
0
0.48
-9
9.77e-10
-4
-5
-12
-15
-12
0
0.22
-9
8.57e-10
-4
-5
-8
-14
-8
38
88
lnts06
2
2
T
101
76
0
0.64
-9
-15
-9
113
5.65
0
0.24
-9
1.98e-10
-4
-5
-11
-15
-11
0
1.82
-9
1.99e-10
-4
-5
-12
-15
-12
0
2.77
-11
-9.50e-12
-4
-5
-8
-15
-8
89
lnts07
2
2
T
1001
796
0
11.45
-12
-15
-12
338
19.88
0
3.86
-11
-2.83e-12
-6
-7
-10
-15
-10
0
215.88
-11
1.73e-12
-4
-6
-12
-15
-12
0
2.72
-10
4.75e-11
-7
-8
-9
-11
-9
90
lnts08
2
2
T
2001
1596
0
39.43
-12
-15
-12
610
35.88
0
8.47
-10
-5.31e-11
-6
-7
-8
-15
-8
0
989.85
-10
1.15e-11
-4
-5
-11
-15
-11
0
6.51
-9
-2.91e-10
-8
-9
-9
-12
-9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
91
lnts09
2
2
T
10001
7996
0
975.00
-12
-15
-12
2868
168.71
0
100.40
-12
-8.47e-13
-5
-6
-10
-15
-10
0
82191.57
-10
6.03e-11
-3
-4
-9
-15
-9
-1002
2331.73
-5
2.67e-06
1
0
-15
-7
-7
92
lnts10
2
2
H
60
36
0
0.30
-8
-15
-8
76
5.07
0
0.20
-9
9.06e-10
-4
-5
-12
-15
-12
0
1.23
-9
9.06e-10
-4
-5
-12
-15
-12
0
0.23
-9
5.21e-10
-4
-5
-8
-11
-8
39
93
lnts11
2
2
H
120
76
0
0.79
-11
-15
-11
134
7.88
0
0.33
-10
-1.06e-11
-6
-7
-9
-15
-9
0
3.84
-13
1.27e-14
-6
-7
-11
-15
-11
0
1.26
-9
-6.96e-10
-8
-9
-8
-14
-8
94
lnts12
2
2
H
1200
796
0
39.11
-12
-15
-12
1068
62.82
0
6.00
-12
3.89e-13
-6
-7
-12
-15
-12
0
383.90
-12
4.44e-13
-5
-6
-11
-15
-11
0
5.46
-9
-2.85e-10
-8
-9
-9
-11
-9
95
lowt01
19
1
T
50
36
0
0.42
-9
-15
-9
99
5.50
0
0.32
-8
2.71e-09
-5
-5
-15
-15
-15
0
0.22
-8
2.71e-09
-5
-5
-12
-15
-12
-1002
14.39
-1
6.96e-02
0
0
13
-6
13
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
96
lwbr01
36
1
T
1176
1071
0
147.04
-9
-15
-9
1857
16.01
106
316.41
-3
-7.11e-04
0
0
-2
-15
-2
104
3408.76
-3
1.69e-04
0
0
-5
-15
-5
-1005
183.68
0
-1.21e+00
0
0
1
-15
1
97
mirv01
30
1
T
1259
1155
0
26.41
-9
-15
-9
204
9.27
0
38.14
-7
-1.61e-08
-7
-7
-11
-15
-11
0
3749.50
-7
-1.61e-08
-4
-4
-8
-15
-8
-1005
353.94
3
3.34e+03
0
0
1
-15
1
40
98
nlqr01
6
1
T
41
30
0
0.78
-10
-15
-10
45
3.21
0
0.79
-9
-3.73e-10
-7
-7
-8
-15
-8
0
0.72
-11
3.35e-12
-7
-7
-9
-15
-9
0
1.62
-10
-3.79e-11
-10
-10
-10
-15
-10
99
nlqr02
6
2
T
41
45
0
0.31
-9
-15
-9
21
2.62
0
0.72
-12
8.93e-13
-6
-6
-9
-15
-9
0
0.97
-12
-5.89e-13
-6
-6
-12
-15
-12
0
1.10
-12
-2.21e-13
-6
-6
-10
-15
-10
100
nlqr03
6
2
T
161
180
0
2.88
-8
-15
-8
37
3.36
0
4.45
-10
-1.10e-11
-6
-6
-9
-15
-9
0
7.06
-10
-1.23e-11
-6
-6
-11
-15
-11
0
6.64
-12
1.23e-13
-7
-7
-8
-15
-8
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
101
nlqr04
6
2
T
401
450
0
15.13
-7
-15
-7
66
3.30
0
14.96
-5
-1.71e-06
-1
-1
-9
-15
-9
0
25.86
-5
-1.71e-06
-1
-1
-12
-15
-12
0
20.34
-5
-1.71e-06
-1
-1
-8
-15
-8
102
orbe02
13
1
T
502
397
0
18.56
-7
-15
-7
327
6.81
0
24.36
-11
3.82e-12
-7
-7
-11
-15
-11
105
600.92
-11
4.13e-12
-5
-5
-11
-15
-11
-1002
382.33
0
2.45e-01
0
0
0
-6
0
41
103
orbe03
13
1
T
502
397
0
55.66
-13
-15
-13
283
5.78
0
107.33
-13
5.98e-14
-8
-8
-12
-15
-12
0
4504.05
-12
-2.99e-13
-6
-6
-12
-15
-12
-1002
2494.64
0
2.44e-01
0
0
0
-15
0
104
orbe04
13
1
T
502
397
0
12.49
-11
-15
-11
116
4.83
0
11.22
-11
-3.36e-12
-6
-6
-8
-15
-8
0
36.67
-12
-6.72e-13
-6
-6
-12
-15
-12
-1005
77.08
0
2.45e-01
0
0
0
-15
0
105
orbe05
13
2
T
502
397
0
36.94
-12
-15
-12
797
14.49
0
13.04
-11
-6.61e-12
-7
-7
-9
-15
-9
116
38.95
0
1.84e-01
0
0
-8
-15
-8
-1002
363.89
0
2.04e-01
0
0
0
-3
0
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
106
orbe06
13
2
T
502
397
0
74.38
-11
-15
-11
558
9.79
0
144.95
-1
-2.62e-02
0
0
-8
-15
-8
116
64.44
0
1.36e-01
0
0
-7
-15
-7
-1002
3042.46
0
2.04e-01
0
0
0
-3
0
107
orbe07
13
2
T
502
397
0
13.65
-12
-15
-12
105
3.89
0
17.39
-3
-1.04e-04
0
0
-10
-15
-10
105
42.94
-12
7.22e-13
-5
-5
-11
-15
-11
-1005
66.35
0
2.30e-01
0
0
0
-15
0
42
108
orbe08
13
3
T
1002
797
0
205.75
-8
-15
-8
2422
21.43
0
93.35
-8
7.57e-09
-4
-4
-10
-15
-10
104
5912.52
-1
-1.78e-02
0
0
-2
-15
-2
-1002
796.00
0
2.06e-01
0
0
9
-4
9
109
orbe09
13
3
T
1002
797
0
367.13
-11
-15
-11
2449
22.47
0
356.46
-1
-1.85e-02
0
0
-12
-15
-12
0
20353.39
-1
-1.85e-02
0
0
-12
-15
-12
-1002
5418.24
0
2.05e-01
0
0
9
-14
9
110
orbe10
13
3
T
1002
797
0
23.76
-8
-15
-8
206
8.58
0
32.15
-11
-2.57e-12
-5
-5
-9
-15
-9
105
148.42
-11
6.59e-12
-5
-5
-11
-15
-11
-1002
633.48
0
2.24e-01
0
0
9
-4
9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
111
orbe11
13
4
T
1502
1197
0
465.15
-10
-15
-10
3108
15.86
0
262.95
-12
3.44e-13
-5
-5
-11
-15
-11
104
16113.62
-1
1.39e-02
0
0
-3
-15
-3
-1002
1104.13
0
2.21e-01
0
0
7
-3
7
112
orbe12
13
4
T
1502
1197
0
932.63
-9
-15
-9
3208
16.37
0
1045.76
-11
5.66e-12
-4
-4
-11
-15
-11
104
47607.67
-2
2.99e-03
0
0
-2
-15
-2
-1002
6190.04
0
2.20e-01
0
0
7
-3
7
43
113
orbe13
13
4
T
1502
1197
0
111.12
-8
-15
-8
386
6.77
0
128.65
-10
-2.89e-11
-7
-7
-8
-15
-8
105
4223.84
-11
1.65e-12
-5
-5
-9
-15
-9
-1002
1098.12
0
2.21e-01
0
0
7
-3
7
114
orbt01
12
1
T
1661
1610
0
64.30
-10
-15
-10
63
3.15
0
86.17
-8
-9.05e-09
0
-1
-13
-15
-13
0
1741.78
-8
-8.93e-09
0
-1
-12
-15
-12
-1005
2432.14
0
5.68e-01
1
0
6
-15
6
115
orbt02
12
2
T
1661
1610
0
148.34
-10
-15
-10
258
6.45
0
193.91
-5
-4.50e-06
1
0
-10
-15
-10
105
6021.56
-3
-2.07e-04
3
2
-12
-15
-12
-1005
2755.48
0
5.13e-01
1
0
6
-15
6
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
116
orbt03
12
5
T
1016
913
0
90.96
-10
-15
-10
270
10.00
0
124.69
-8
6.77e-09
-5
-5
-9
-15
-9
104
1931.98
0
-4.54e-01
0
0
0
-15
0
-1004
609.27
0
3.28e-01
0
0
3
-15
3
117
plnt01
16
2
T
1470
1418
0
198.16
-8
-15
-8
301
11.15
0
3745.45
-11
7.90e-12
-5
-5
-9
-15
-9
116
4479.06
-3
-5.19e-04
0
0
0
-15
0
-1002
10375.80
0
-3.44e-01
0
0
7
-4
7
44
118
pndl01
38
1
T
60
46
0
1.19
-9
-15
-9
217
4.62
0
1.03
-11
8.10e-11
-9
-9
-14
-15
-14
0
0.53
-11
1.00e-10
-6
-7
-12
-15
-12
0
1.16
-8
-1.91e-08
-8
-8
-8
-11
-8
119
putt01
21
2
T
61
68
0
2.21
-12
-15
-12
185
4.02
0
6.17
-7
2.69e-08
-6
-7
-12
-15
-12
105
1.84
-10
6.45e-11
-6
-7
-12
-15
-12
0
2.98
-11
7.54e-12
-6
-7
-8
-15
-8
120
qlin01
1
1
T
90
54
0
0.11
-11
-15
-11
22
4.40
0
0.14
-12
-3.16e-13
-10
-11
-15
-15
-15
105
2.02
-12
-3.41e-13
-8
-8
-12
-15
-12
0
0.16
-9
-9.93e-10
-8
-8
-8
-10
-8
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
121
qlin02
1
2
T
100
63
0
0.30
-9
-15
-9
41
3.42
0
0.60
-9
9.16e-10
-4
-4
-11
-15
-11
0
2.01
-9
9.26e-10
-4
-4
-12
-15
-12
0
0.31
-9
-3.01e-10
-4
-4
-9
-11
-9
122
qlin03
1
3
T
90
64
0
0.11
-11
-15
-11
22
4.40
0
0.16
-12
-3.16e-13
-10
-11
-15
-15
-15
0
2.12
-12
-2.79e-13
-7
-7
-12
-15
-12
0
0.17
-9
-7.65e-10
-8
-8
-9
-10
-9
45
123
qlin04
1
4
T
10
0
0
0.01
-15
-15
-15
8
2.67
0
0.03
-14
-7.35e-17
-8
-8
-15
-15
-15
0
0.00
-14
-9.78e-17
-8
-8
-15
-15
-15
-4242
0.00
0
5.77e-01
0
0
-15
-15
-15
124
robo01
31
1
T
100
63
0
0.67
-9
-15
-9
131
10.92
0
1.36
-8
4.37e-09
-5
-5
-10
-15
-10
0
0.62
-9
3.24e-10
-5
-5
-12
-15
-12
0
1.02
-9
-1.67e-10
-5
-5
-10
-11
-10
125
robo02
31
2
T
90
54
0
0.47
-12
-15
-12
99
11.00
0
0.47
-9
3.97e-09
-8
-8
-15
-15
-15
0
0.72
-12
-1.69e-11
-6
-6
-12
-15
-12
0
1.27
-10
-1.73e-09
-10
-10
-12
-12
-11
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
126
robo03
31
3
T
91
54
0
0.99
-8
-15
-8
97
6.06
0
2.14
-9
5.52e-10
-4
-4
-8
-15
-8
0
1.09
-11
9.28e-12
-4
-4
-9
-15
-9
-1005
39.85
1
2.02e+01
5
5
1
-15
1
127
robo04
31
4
T
287
277
0
0.74
-10
-15
-10
19
4.75
0
0.60
-14
0.00e+00
-11
-11
-10
-15
-10
0
1.09
-14
0.00e+00
-11
-11
-12
-15
-12
0
2.37
-14
0.00e+00
-9
-9
-9
-11
-9
46
128
skwz01
32
1
T
1053
1039
0
10.21
-10
-15
-10
39
3.00
101
175.92
-14
0.00e+00
-10
-10
-11
-15
-11
0
18.35
-14
0.00e+00
-9
-9
-10
-15
-10
0
126.02
-14
0.00e+00
0
0
-8
-12
-8
129
skwz02
32
2
T
1092
1039
0
47.68
-10
-15
-10
212
7.57
0
53.03
-1
2.65e-02
0
0
-13
-15
-13
0
87.92
-10
-4.65e-11
-7
-7
-11
-15
-11
0
67.50
-9
4.51e-10
0
0
-10
-14
-10
130
skwz03
32
3
T
1093
1039
0
32.73
-9
-15
-9
185
10.88
0
34.51
-7
3.90e-08
-4
-4
-10
-15
-10
0
45.70
-13
-2.98e-14
-9
-9
-9
-15
-9
0
50.66
-9
1.11e-10
0
0
-7
-11
-7
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
131
skwz04
32
4
T
924
877
0
20.67
-8
-15
-8
143
8.41
0
16.14
-9
2.15e-10
-7
-8
-13
-15
-13
0
58.02
-9
2.08e-10
-6
-6
0
-15
0
0
37.45
-9
-6.59e-10
0
0
0
-12
0
132
traj01
3
1
R
233
185
0
10.58
-12
-15
-12
132
4.55
0
14.05
-11
-7.32e-12
-7
-7
-11
-15
-11
0
13.95
-12
-4.66e-13
-6
-6
-12
-15
-12
-1002
638.53
0
1.65e+00
2
2
13
-1
13
47
133
traj02
3
1
R
923
755
0
47.81
-11
-15
-11
303
10.82
0
104.75
-10
-3.46e-11
-7
-7
-11
-15
-11
0
150.30
-11
1.12e-12
-5
-5
-12
-15
-12
-1002
2642.78
0
1.65e+00
1
1
-15
-5
-5
134
traj03
3
1
T
217
185
0
3.36
-12
-15
-12
113
5.14
0
3.40
-8
1.00e-09
-7
-7
-12
-15
-12
0
3.80
-12
-3.34e-13
-5
-5
-11
-15
-11
-1002
134.61
0
1.63e+00
1
1
0
-2
0
135
traj04
3
1
T
847
755
0
17.66
-11
-15
-11
150
6.00
0
29.50
-11
4.42e-12
-10
-10
-14
-15
-14
0
60.22
-11
4.35e-12
-7
-7
-11
-15
-11
-1002
605.82
0
1.65e+00
1
1
-15
-3
-3
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
136
traj05
3
1
H
233
185
0
9.82
-11
-15
-11
129
4.96
0
10.90
-8
1.01e-09
-7
-7
-12
-15
-12
0
12.43
-11
6.27e-12
-6
-6
-12
-15
-12
-1002
563.58
0
1.65e+00
1
1
0
-3
0
137
traj06
3
1
H
923
755
0
50.14
-10
-15
-10
222
7.66
0
50.70
-10
3.33e-11
-9
-9
-13
-15
-13
0
168.73
-10
3.33e-11
-6
-6
-11
-15
-11
-1002
1508.49
0
1.65e+00
2
2
0
-2
0
48
138
traj07
3
2
R
306
200
0
8.83
-9
-15
-9
109
3.52
0
12.39
-9
2.26e-10
-8
-8
-11
-15
-11
0
30.77
-9
2.19e-10
-6
-6
-11
-15
-11
-1002
75.61
0
2.00e+00
0
0
13
-3
13
139
traj08
3
2
R
606
400
0
39.96
-11
-15
-11
244
4.44
0
20.28
-10
1.07e-11
-9
-9
-11
-15
-11
0
83.16
-11
3.33e-12
-5
-5
-10
-15
-10
-1002
128.57
0
2.00e+00
0
0
2
-3
2
140
traj09
3
2
T
256
200
0
2.65
-12
-15
-12
80
3.33
0
3.24
-11
-3.66e-12
-8
-8
-13
-15
-13
0
6.77
-10
-1.30e-11
-6
-6
-11
-15
-11
-1005
40.23
0
2.00e+00
0
0
-5
-3
-3
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
141
traj10
3
2
T
506
400
0
11.42
-11
-15
-11
180
4.86
0
10.25
-12
8.21e-13
-9
-9
-12
-15
-12
0
28.88
-11
-5.80e-12
-5
-5
-11
-15
-11
0
30.90
-9
-2.15e-10
-9
-9
-10
-15
-10
142
traj11
3
2
T
1006
800
0
30.18
-11
-15
-11
235
4.90
0
18.51
-11
-7.08e-12
-8
-8
-11
-15
-11
0
353.24
-10
-2.19e-11
-5
-5
-11
-15
-11
0
15.27
-9
-7.93e-10
-8
-8
-9
-12
-9
49
143
traj12
3
2
T
2006
1600
0
74.44
-10
-15
-10
314
6.83
0
54.26
-9
-6.16e-10
-6
-6
-10
-15
-10
0
710.96
-9
-4.94e-10
-4
-4
-10
-15
-10
-1002
994.14
-4
5.26e-05
0
0
4
0
4
144
traj13
3
2
H
306
200
0
4.92
-12
-15
-12
88
3.67
0
10.49
-12
2.10e-13
-7
-7
-12
-15
-12
0
17.73
-12
7.01e-13
-5
-5
-11
-15
-11
0
11.15
-9
-6.37e-10
-8
-8
-7
-13
-7
145
traj14
3
2
H
606
400
0
11.03
-9
-15
-9
100
3.70
0
14.85
-11
9.25e-12
-6
-6
-10
-15
-10
116
28.73
0
-3.56e-01
0
0
0
-15
0
0
26.59
-8
-1.08e-09
-6
-6
-9
-15
-9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
146
traj15
3
2
H
1206
800
0
38.91
-10
-15
-10
185
5.29
0
33.84
-11
-6.15e-12
-6
-6
-10
-15
-10
116
132.34
0
-1.64e-01
0
0
-1
-15
-1
0
41.44
-8
-2.01e-09
-8
-8
-8
-12
-8
147
traj16
3
2
H
2406
1600
0
59.35
-10
-15
-10
172
6.37
0
94.67
-10
-1.69e-11
-8
-8
-10
-15
-10
116
897.94
-1
-4.21e-02
0
0
-1
-15
-1
0
140.61
-9
-5.82e-10
-7
-7
-8
-15
-8
50
148
traj17
3
3
R
458
250
0
7.20
-9
-15
-9
47
2.94
0
27.89
-11
8.58e-12
-5
-5
-8
-15
-8
116
13.29
-1
-2.51e-02
0
0
-1
-15
-1
-1002
146.50
0
-8.08e-01
0
0
13
-3
13
149
traj18
3
3
R
458
250
0
7.42
-9
-15
-9
108
6.75
0
24.29
-10
-1.86e-11
-5
-5
-8
-15
-8
0
127.24
-11
-5.36e-12
-4
-4
-11
-15
-11
-1002
157.05
0
-8.08e-01
0
0
13
-3
13
150
traj19
3
3
R
908
500
0
16.20
-11
-15
-11
54
3.18
0
41.02
-11
8.53e-12
-3
-3
-9
-15
-9
0
1165.84
-10
-1.09e-11
-5
-5
-12
-15
-12
0
47.29
-10
-3.71e-11
-4
-4
-8
-14
-8
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
151
traj20
3
3
R
908
500
0
18.83
-9
-15
-9
182
10.71
0
36.50
-7
3.00e-08
-3
-3
-8
-15
-8
0
575.05
-10
1.51e-11
-3
-3
-11
-15
-11
0
60.33
-10
3.21e-11
-4
-4
-8
-15
-8
152
traj21
3
3
T
358
250
0
3.54
-9
-15
-9
61
3.21
0
7.06
-9
6.30e-10
-5
-5
-9
-15
-9
105
66.66
-9
6.32e-10
-4
-4
-11
-15
-11
0
5.12
-9
6.15e-10
-5
-5
-9
-13
-9
51
153
traj22
3
3
T
358
250
0
3.67
-11
-15
-11
111
6.17
0
14.42
-10
1.74e-11
-6
-6
-10
-15
-10
105
53.15
-12
-2.13e-13
-4
-4
-12
-15
-12
0
5.82
-10
3.33e-11
-5
-5
-9
-14
-9
154
traj23
3
3
T
358
250
0
12.79
-9
-15
-9
334
6.07
0
13.73
-9
3.58e-10
-6
-6
-10
-15
-10
0
38.75
-9
3.28e-10
-5
-5
-12
-15
-12
0
18.68
-9
3.31e-10
-7
-7
-11
-15
-11
155
traj24
3
3
T
708
500
0
6.93
-10
-15
-10
53
2.94
0
15.07
-10
1.14e-11
-4
-4
-10
-15
-10
0
665.18
-3
5.03e-04
0
0
-11
-15
-11
0
12.80
-11
-3.85e-12
-5
-5
-8
-15
-8
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
156
traj25
3
3
T
708
500
0
8.19
-8
-15
-8
145
8.53
0
16.47
-9
-4.45e-10
-4
-4
-8
-15
-8
0
174.23
-10
3.55e-11
-4
-4
-12
-15
-12
0
11.98
-8
-2.64e-09
-5
-5
-8
-13
-8
157
traj26
3
3
T
1408
1000
0
14.73
-9
-15
-9
53
2.94
0
28.34
-9
2.83e-10
-4
-4
-9
-15
-9
0
3320.56
-9
2.79e-10
-4
-4
-12
-15
-12
0
34.30
-9
-2.00e-10
-4
-4
-8
-14
-8
52
158
traj27
3
3
T
10011
7145
0
209.48
-8
-15
-8
50
3.12
0
514.27
-8
0.26e-08
-4
-4
-8
-15
-8
105
1207386.34
-9
0.82e-09
-2
-2
-13
-15
-13
0
484.97
-8
0.25e-08
-4
-4
-9
-15
-9
159
traj28
3
3
H
458
250
0
7.65
-10
-15
-10
54
3.18
0
25.25
-10
1.91e-11
-3
-3
-9
-15
-9
0
146.81
-11
9.24e-12
-4
-4
-11
-15
-11
0
23.96
-10
-2.79e-11
-5
-5
-8
-14
-8
160
traj29
3
3
H
458
250
0
9.05
-8
-15
-8
128
7.11
0
24.12
-7
1.50e-08
-3
-3
-9
-15
-9
0
188.72
-10
2.85e-11
-4
-4
-12
-15
-12
0
26.55
-10
2.11e-11
-5
-5
-9
-15
-9
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
161
traj30
3
3
H
908
500
0
17.11
-9
-15
-9
57
3.17
0
42.65
-9
1.28e-10
-3
-3
-9
-15
-9
0
854.19
-9
1.24e-10
-4
-4
-12
-15
-12
0
59.52
-10
7.42e-11
-4
-4
-8
-15
-8
162
traj31
3
3
H
908
500
0
20.36
-9
-15
-9
222
12.33
0
41.10
-7
3.00e-08
-3
-3
-8
-15
-8
0
866.59
-11
1.46e-12
-4
-4
-12
-15
-12
0
55.52
-11
6.73e-12
-5
-5
-8
-15
-8
53
163
traj32
3
3
H
1808
1000
0
34.12
-9
-15
-9
77
4.53
0
132.22
-9
7.82e-10
-4
-4
-9
-15
-9
0
76186.88
-9
7.83e-10
-4
-4
-12
-15
-12
0
110.95
-9
7.56e-10
-4
-4
-8
-15
-8
164
traj33
3
3
H
12869
7145
0
383.61
-8
-15
-8
46
2.88
0
1789.76
-8
5.06e-09
-3
-3
-7
-15
-7
-160
4432.02
-3
7.93e-04
0
0
-2
-15
-2
0
1789.92
-8
4.91e-09
-4
-4
-8
-15
-8
165
traj34
3
4
R
458
351
0
12.27
-12
-15
-12
278
15.44
0
50.96
-10
-4.84e-11
-4
-4
-8
-15
-8
104
803.50
-10
3.06e-11
-4
-4
-12
-15
-12
0
22.30
-9
-1.23e-10
-4
-4
-8
-15
-8
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
166
traj35
3
4
R
908
701
0
30.60
-8
-15
-8
502
27.89
0
201.62
-9
3.59e-10
-5
-5
-9
-15
-9
0
398.81
-9
3.61e-10
-4
-4
-11
-15
-11
0
59.86
-9
3.85e-10
-4
-4
-8
-13
-8
167
traj36
3
4
T
358
301
0
3.79
-9
-15
-9
161
11.50
0
13.07
-9
5.38e-10
-5
-5
-10
-15
-10
0
42.38
-9
5.89e-10
-6
-6
-12
-15
-12
0
5.28
-9
-2.11e-10
-4
-4
-8
-13
-8
54
168
traj37
3
4
T
708
601
0
11.21
-9
-15
-9
290
17.06
0
42.29
-9
3.29e-10
-6
-6
-9
-15
-9
0
161.00
-9
3.16e-10
-4
-4
-12
-15
-12
0
11.18
-9
2.50e-10
-5
-5
-8
-14
-8
169
traj38
3
4
H
458
351
0
11.93
-9
-15
-9
299
17.59
0
71.26
-9
2.21e-10
-5
-5
-9
-15
-9
0
105.84
-9
2.20e-10
-5
-5
-12
-15
-12
0
21.28
-10
3.34e-11
-4
-4
-7
-13
-7
170
traj39
3
4
H
908
701
0
30.75
-8
-15
-8
499
27.72
0
160.98
-9
9.75e-10
-5
-5
-8
-15
-8
0
411.34
-9
9.88e-10
-3
-3
-11
-15
-11
0
52.15
-9
9.92e-10
-5
-5
-8
-14
-8
SPRNLP
BARNLP
Case
Problem
Ode
Class
Discr.
N
M
IERNLP
Total time
CONERR
BDERR
SUM
QP Iterations
Factor
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
SNOPT
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
LOQO
CONERR
BDERR
SUM
IERNLP
Total time
|f f |
|f |
f f
x x
xx
x
CONERR
BDERR
SUM
171
tran01
39
1
T
40
18
0
0.26
-13
-15
-13
90
6.43
0
0.21
0
1.13e+00
0
0
-14
-15
-14
0
0.17
-12
8.23e-13
-9
-10
-12
-15
-12
0
0.20
0
1.13e+00
0
0
-10
-12
-10
55
172
wind01
11
1
T
241
235
0
4.30
-9
-15
-9
480
43.64
0
2.10
-7
1.80e-08
0
0
-8
-15
-8
0
4.34
-10
1.75e-11
0
0
-12
-15
-12
-1002
112.20
0
4.90e-01
0
0
3
-3
3
173
zrml01
40
1
T
31
18
0
0.17
-8
-15
-8
113
8.69
0
0.07
-7
1.22e-08
-6
-6
-10
-15
-10
0
0.17
-7
1.23e-08
-7
-7
-12
-15
-12
0
0.05
-7
1.15e-08
-7
-7
-9
-10
-9