See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/269401434

Understanding Systems of Linear Equations and Programming through lpSolve

and R Language

Technical Report · December 2014

DOI: 10.13140/2.1.2592.9280

CITATION READS
1 4,937

1 author:

Kamakshaiah Musunuru
BS Enterprises
17 PUBLICATIONS   65 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

HR Analytics View project

All content following this page was uploaded by Kamakshaiah Musunuru on 13 June 2015.

The user has requested enhancement of the downloaded file.

 Understanding Systems of Linear Equations and
Programming through lpSolve and R Language
Kamakshaiah Musunuru
kamakshaiah.m@gmail.com

Contents
1 Introduction 2

2 lpSolve 2
2.1 What is lp solve? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

3 Methods in lp solve 3
3.1 Basic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.1.1 The matrix form of the model . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.1.2 Bounds & Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2.1 Integer & binary variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2.2 Semi-continuous variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.3 Special ordered sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3.1 Absolute values on constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.4 Sensivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.4.1 Dualities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.5 Integer programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.6 Presolve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.7 Infeasible solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4 How to solve equations in R 8

4.1 What is R? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2 How do we implement linear programming in R? . . . . . . . . . . . . . . . . . . . . 8
4.3 How to install lpSolve? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.4.1 Integer programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 The other free or open source packages for LP problems 12

1
1 Introduction
R has good stamina to solve systems of linear equations and also rich in optimization algorithms.
There are numerous alternatives to take care of problems of optimization, viz. optimx, Rcgmin,
Rvmmin, ROI etc.. However, lpSolve, lpSolveAPI appears to be strait and simple to deal with
systems of linear equations. These packages are not available as part of R-base, rather as open (or
free) projects at their respective sites. This paper describes as how to solve LP problems by using
these packages. 1
Linear programming (LP; also called linear optimization) is a method to achieve the best out-
come (such as maximum profit or lowest cost) in a mathematical model whose requirements are
represented by linear relationships. Linear programming is a special case of mathematical program-
ming (mathematical optimization).
More formally, linear programming is a technique for the optimization of a linear objective
function, subject to linear equality and linear inequality constraints. Its feasible region is a convex
polytope, which is a set defined as the intersection of finitely many half spaces, each of which is
defined by a linear inequality. Its objective function is a real-valued affine function defined on this
polyhedron. A linear programming algorithm finds a point in the polyhedron where this function
has the smallest (or largest) value if such a point exists. [1]

2 lpSolve
The following information was taken from lp solve project page at sourceforge.net. Source-
force.net is a community repository renowned for free and open source projects and software.
Right now (at the time of this manual) there are 430, 000 projects; hosting over 3.7 million
registered users. Serving 41.8 million customers with all of these open source projects with
more than 4, 800, 000 downloads a day.[2]

2.1 What is lp solve?

What is lp solve and what is it not? The simple answer is, lp solve is a Mixed Integer Linear
Programming (MILP) solver.
lp solve is a free linear (integer) programming solver based on the revised simplex method and
the Branch-and-bound method for the integers. It contains full source, examples and manuals.
lp solve solves pure linear, (mixed) integer/binary, semi-continuous and special ordered sets
(SOS) models. Note the word linear. This means that equations must be of the first order.
5 ∗ x − 3 ∗ y is an example. However x ∗ y is not linear and cannot be handled by lp solve. Both
the objective function and the constraints have this restriction.
Via the Branch-and-bound algorithm, it can handle integer variables, semi-continuous variables
and Special Ordered Sets(SOS).
lp solve has no limit on model size and accepts standard both lp or mps input files, but even
that can be extended. Note however that some models could give lp solve a hard time and will
1 There are other proprietary solutions. Namely, IBM-Decision Optimization a sibling to SPSS. SAS has optimiza-

tion algorithms through OR module. But the pricing mechanisms are excruciating. SAS Analytics Pro cost around
Rs. 5, 30, 700 for an individual in India. You may read the pangs of acquiring SAS from here

2
even fail to solve. The larger the model the likely the chance for that. But even commercial solvers
have problems with that.
It can also be called as a library from different languages like C, VB, .NET, Delphi, Excel,
Java, ... It can also be called from AMPL, MATLAB, O-Matrix, Scilab, Octave, R via a driver
program. lp solve is written in ANSI C and can be compiled on many different platforms like linux
and WINDOWS.
lp solve has its own community via the Yahoo group http://groups.yahoo.com/group/lp_
solve/. There you can find the latest sources, executables for the common platforms, examples,
manuals and a message board where people can share their thoughts on lp solve.
lp solve was originally developed by Michel Berkelaar at Eindhoven University of Technology.
The work of Jeroen Dirks made the transition from the basic version 1.5 to the full version 2.0
possible. He contributed the procedural interface, a built-in MPS reader, and many fixes and
enhancements to the code. At that point there was also a Java port of lp solve 2.0. This was not a
Java native interface (JNI) to the C library, but rather a translation of the algorithm (ver 2.0) from
C into Java. It was also very limited This meant that it did not keep up with lp solve as it evolved.
Starting at version 3.0, lp solve is released under the LGPL license. Before the code could only
be used for non-commercial purposes. Many other people also contributed to lp solve, but there
was no track of them. Sorry if you are not mentioned. Development was stagnated for some time
at version 3.2, but now it is again alive and well via the new developers Kjell Eikland and Peter
Notebaert. But also other people help to improve the product. For example the new Java interface
to lp solve was made by Juergen Ebert. It is a JNI interface that supports the full functionality
of lp solve. He did a great job. We encourage other people to participate in the development of
lp solve.
First a version 4 was introduced that had already several enhancements and improvements and
now with version 5 this enhancement continues resulting is faster solving times, more stability and
able to solver larger models and new functionality. See Changes from version 4 to version 5.1 for
the changes done to version 5 and Changes from version 5.1 to version 5.5 for the changes done to
version 5.5.
Basically, lp solve is a library, a set of routines, called the API that can be called from almost
any programming language to solve MILP problems. There are several ways to pass the data to
the library. For instance:

• Via the API

• Via input files

• Via an IDE

[3]

3 Methods in lp solve
3.1 Basic model
• A linear function to be maximized or minimized.
e.g.
max:c1 x1 + c2 x2 (linear objective function)

3
 • Problem constraints.
e.g.

a_11x_1+a_12x_2;

a_21x_1+a_22x_2;

a_31x_1+a_32x_2

• Bounds (default lower bounds of zero on variables))

3.1.1 The matrix form of the model

max: cT x
subject to : Ax <= B
x >= 0 (implicit lower bound)

3.1.2 Bounds & Ranges

• Bounds (lower bound & upper bound)
– implicit (lower) bound
– explicit bound
• Ranges
– Restrictions (less than/more than)
– equalities are more convenient compared to inequalities.

3.2 Variables
3.2.1 Integer & binary variables
• By default all variables are real sometimes it requires one/more variables must be integers.
• Problems with integer variables are called integer/discrete programming problems (IP).
• If all variables are integers it is called a pure integer programming problem, else Mixed Integer Programming
Problems (MIP).

• The disadvantages with integer programming:

– Reduced processing speed.
– Inaccurate sensitivity analysis.

3.2.2 Semi-continuous variables

• The variables that must take values between crtain minimum & maximum limits.

4
3.3 Special ordered sets
• A special order set of degree N is a collection of variables where at most N variables may be
non-zero.
• The non-zero variables must be contiguous (neighbors) sorted by the ascending value of their
respective unique neighbors.
• lps olvemethodologyunderSOSengenderthreeequations, viz.

• function rows
• reference rows
• convexity rows

e.g.
min: -x1 − x2 − 3.x3 − 2x4 − 2.x5

c1 : −x1 − x2 − x3 − x4 <= 30;

c2 : −x1 − x2 + x3 + x4 <= 30;
x1 <= 40;

x2 <= 1;
x3 <= 1

sos
sos: x1 , x2 , x3 , x4 , x5 ≤ 4

The above system is defined with SOS4, which means any 4 consecutive variables from x1 to
x5 might be non-zero.

3.3.1 Absolute values on constraints

• linear equations of the type
a1 x1 + a2 x2 + a3 x3 · · · <= max
a1 x1 + a2 x2 + a3 x3 · · · >= min are easy to solve but in case below constraint;

−a1 x1 − a2 x2 − a3 x3 · · · <= max

We may use the command “abs” to convert the direction (i.e. ¡=) as below:
abs(−a1 x1 − a2 x2 − a3 x3 · · ·) >= min
Notice the constant is now “min” in stead of “max”. Where “max” & “min” are constants.

5
3.4 Sensivity
The idea of “sensivity” in linear programming is beyond the scope of this manual. More description
is required to cover this topic. In short, sensitivity can be understood as the investigation that
deals with changes in the optimal solution due to changes in the parameters (aij , bi , and cj ). In
addition to this, the following points might help us understand sensitivity which is also known as
post-optimality analysis.
• The linear output will appear in four parts.
– values of objective function
– actual values of the constraints
– objective function limits (sensitivity)
– dual values with from-to limits (dualities).
• Sensitivities answer as what is the impact of constraints on the value of objective function

3.4.1 Dualities
One way of understanding duality is exchange of objectives in case of two attributes are under
study. For instance, if a dealer has to take decision on two different attributes of any given product
namely, price and performance. The linear problem might help us find optimal solutions for both
having price as objective function with performance as constants for constraints and vice versa.
The concept of duality might offer multiple solutions as by branching the main problem. Every
linear problem is associated with another linear programming problem. The original problem is
called the “primal”, and the other is called “dual”.

• Dualities answer as what is the impact of variables on the value of objective function.

For more information and examples you may refer this page.

3.5 Integer programming

At times it is not possible for us to workout solution in fractions. For example, 2.5 persons, 0.5
boilers, 32.2 trucks might make no sense. Hence, we may have to solve problems with different
methods. Simplex methods might not be relevant to this type of cases. For instance, we might
round the value either floor or by ceiling but that doesn’t yield expected results. At times it is
hazardous for such solutions might ignore importance of constraints. So, we might require special
methods of solving equations. There are two important methods that might prove handy for such
cases. They are
• I-Gomory’s cutting plane method
• Branch-and-bound technique

In lp solve there are wonderful methods to take care of integer case. All those methods are
simple yet efficient. Refer R code explained in subsequent sections of this manual.

6
3.6 Presolve
• It looks the ways to simplify lp.
• It detects unused variables and restrictions

• Lessening the variables and constraints is taken care by this step in lp solve.
e.g.
max: x1 + x2 + x3 ;

r1 : x2 = 2;
r2 : x1 + x2 <= 6;
r3 : x1 − 2x3 <= 2;
r4 : x1 + 3x3 <= 5;
• In the above eq. row1 is a constraint on one variable. Presolve can detect this and comment
it to a bound on that variable as below.
objective function
max: x1 + x2 + x3 ;

constraints
r2 : x1 + x2 <= 6;
r3 : x1 − 2x3 <= 2;
r4 : x1 + 3x3 <= 5;

Variable bound
x == 2

3.7 Infeasible solutions

• A linear program is infeasible if there exist no solution that satisfies all of the constraints.
• The source of infeasibility often difficult to track down. It might stem from an error in
specifying source of the constraints in the model or it might arise out of wrong data. It can
also be the result of a combination of factors.
e.g.
min : x + y;

c1 : x >= 6;
c2 : y >= 6;
c3 : x + y >= 11;

This model is clearly infeasible. Now we might introduce extra variables to locate the infeasibility.
i.e.

7
min : x + y + 1000e1 + 1000e2 + 1000e3 ;

c1 : x + e1 >= 6;
c2 : y + e2 >= 6;
c3 : x + y − e3 >= 11;

4 How to solve equations in R

4.1 What is R?
The following information is taken from http://www.r-project.org/; the place where the main
and official R development takes place. There are other places where there is really plenty of
information. R is not only renowned but also respected by experts of various domains of knowl-
edge. For instance, for general information you may visit: http://en.wikipedia.org/wiki/R_
%28programming_language%29; [4] for instructions on download and installation you may visit:
http://cran.r-project.org/bin/windows/base/. [5] There is another important place without
which it is not possible to operate R, that is CRAN. You may visit this at http://cran.rstudio.
com/. 2 This is the place where almost all packages are available. As on today (08 − December −
2014), there are approximately 4, 500 packages available from CRAN. All of them are not only free
but also rich in methods and execution when compared to their respective counter alternatives.
R is a free software environment for statistical computing and graphics. It compiles and runs on a
wide variety of UNIX platforms, Windows and MacOS. It is a GNU project [6] which is similar to
the S language and was developed at Bell Laboratories (formerly ATT, now Lucent Technologies)
by John Chambers and colleagues. R can be considered as a different implementation of S. There
are some important differences, but much code written for S runs unaltered under R. R provides
a wide variety of statistical (linear and nonlinear modeling, classical statistical tests, time-series
analysis, classification, clustering, ...) and graphical techniques, and is highly extensible. The S
language is often the vehicle of choice for research in statistical methodology, and R provides an
Open Source route to participation in that activity.
One of R’s strengths is the ease with which well-designed publication-quality plots can be produced,
including mathematical symbols and formula where needed. Great care has been taken over the
defaults for the minor design choices in graphics, but the user retains full control.
R is available as Free Software under the terms of the Free Software Foundation’s GNU General
Public License in source code form. It compiles and runs on a wide variety of UNIX platforms and
similar systems (including FreeBSD and Linux), Windows and MacOS.

4.2 How do we implement linear programming in R?

As it was stated earlier there are wounderful solutions in R for linear programming or optimization
techniques. For more information you may visit CRAN task views. [8] As on today, there are
33 views available for both mathematical, statistical, and other programming. You may find a
2 There are many mirrors to reach CRAN. But the url mentioned here is a mirror maintained by RStudio.

8
view under the category optimization which exclusively deal with optimization techniques. There
are also instructions as how to install these packages at the same web-page (mentioned before).
There are two packages namely; lpSolve under the category: Mathematical Programming Solvers;
and lpSolveAPI under the category: Interfaces to Open Source Optimizers. We need to install
these packages to solve the problems that fall under the category “systems of linear equations and
programming”.

4.3 How to install lpSolve?

You may do as mentioned below to install this package (lpSolve) in R editor.

> install.packages("lpSolve")

After installing you may invoke respective libraries by:

> library("lpSolve")

Now all those routines (software code) as in the form of objects (most importantly functions) are
available for our calculations.

4.4 Example
Let us see the below given set of equations through lpSolve in R.

maximize: x1 + 9x2 + x3 ;

subject to:

x1 + 2x2 + 3x3 <= 9

3x1 + 2x2 + 2x3 <= 15

∀xi > 0

The following procedure shows as how to solve the above equations.

# let us input constants (or coefficients) of objective function

> obj = c(1,9,1)
# let us create a data matrix in which all the constants (or coefficients) in constraints
are given in rows.
> con = matrix(c(1,2,3,3,2,2), nrow=2, byrow=TRUE)
# let us define relationships (i.e. directions) given in constraints
> rel = c("<=", "<=")
# let us form a vector of constants in the RHS of constraint equations
> rhs = c(9,15)

9
Now we finished the basic requirements to compute lp model. The following are statements which
we need to execute to set the lp model.

# for solution
# we compute the solution and parse it to an object with a name ’my.lp’
> my.lp = lp("max", obj, con, rel, rhs)
# now the solution is ready, we might get the solution with the below command
> my.lp$solution
# for rest of the attribute we might execute the following command
> names(my.lp)
[1] "direction" "x.count" "objective"
[4] "const.count" "constraints" "int.count"
[7] "int.vec" "bin.count" "binary.vec"
[10] "num.bin.solns" "objval" "solution"
[13] "presolve" "compute.sens" "sens.coef.from"
[16] "sens.coef.to" "duals" "duals.from"
[19] "duals.to" "scale" "use.dense"
[22] "dense.col" "dense.val" "dense.const.nrow"
[25] "dense.ctr" "use.rw" "tmp"
[28] "status"

From the above list of attributes most important attributes are objval and solution. We can
check our values with the help of following commands.

> my.lp$objval
[1] 40.5
> my.lp$solution
[1] 0.0 4.5 0.0

From the above output it is clear that our objective function has a value 40.5 for variables x1 =
0; x2 = 4.5, x3 = 0. Very nice solution. This solution is of type SOS1. We can achieve the
objective (the value mentioned before) with only one variable, it shows that rest of the variables
are redundant. As discussed earlier in lp solve methodology. We may enquire further by exercising
“sensitivities” and “dualities” in order to know the nature of variables and constants and their
influence on value of the objective function.

> my.lp.sense = lp("max", obj, con, rel, rhs, compute.sens = TRUE)

> names(my.lp.sense)
[1] "direction" "x.count" "objective"
[4] "const.count" "constraints" "int.count"
[7] "int.vec" "bin.count" "binary.vec"
[10] "num.bin.solns" "objval" "solution"
[13] "presolve" "compute.sens" "sens.coef.from"
[16] "sens.coef.to" "duals" "duals.from"
[19] "duals.to" "scale" "use.dense"
[22] "dense.col" "dense.val" "dense.const.nrow"

10
 [25] "dense.ctr" "use.rw" "tmp"
[28] "status

From the above output the most important attributes are “sens.coef.from” and “sens.coef.to”;
based on these we might judge the impact of coefficients in objective function on the value of
objective function.

> my.lp.sense$sens.coef.from
[1] -1e+30 2e+00 -1e+30
> my.lp.sense$sens.coef.to
[1] 4.50e+00 1.00e+30 1.35e+01

From the above output it is clear that the x1 coefficient in objective function i.e. 1 need to be
between −1e + 30 and 4.50e + 00 in order to achieve the maximum value 40.5 for said variables. x2
value as a matter of feasibility is 4.5, it should be at least 2e + 00 to keep the value as best as 40.5.
Now let us look at dualities by executing following set of commands.

> my.lp.sense$duals
[1] 4.5 0.0 -3.5 0.0 -12.5
> my.lp.sense$duals.from
[1] 0e+00 -1e+30 -1e+30 -1e+30 -6e+00
> my.lp.sense$duals.to
[1] 1.5e+01 1.0e+30 3.0e+00 1.0e+30 3.0e+00

From the above output it seems that there are three non-zero variables (known as non-basic vari-
ables). Constraint one is active in the solution. So, it might be possible to solve the system with
first constraint alone. The constant for first constraint is 9 and from the input it is clear that it
might range from 0 to 15. These numbers explain as how the value of objective function is subjected
to change for a unit change in the value of RHS i.e. <= 9. For instance, the value of objective
function change to the extent (40.5) + 1 ∗ (4..5) for every unit change in constant i.e. <= 9. The
strain on the value of objective function is going to be The second constraint appears to be very
much constricted and also the differences appears to be significant from the actual values. They
doesn’t appear to be useful to construct the solution.

4.4.1 Integer programming

In the same fashion we might solve these equations through integer programming as well. The only
requirement is that we need to supply an additional argument in lp model. For instance:

> my.lp.sense$duals.to
[1] 1.5e+01 1.0e+30 3.0e+00 1.0e+30 3.0e+00
> my.int.lp=lp("max", obj, con, rel, rhs, int.vec = 1:3)
> my.int.lp$solution
[1] 1 4 0
> my.int.lp$objval
[1] 37
>

11
The integer solution for integers 1, 2, 3 for x1 , x2 and x3 is going to be 1, 4, 0 for maximum value 37.
There are two non-zero values for variables. The simplex solution appears to be more feasible as
compared to the second.

5 The other free or open source packages for LP problems

As mentioned in introduction lpSolve is not unique solution for LP problems. One of the advantages
of using R is that there are millions of individuals working incessantly on and around R as by being
community members. There are thousands of packages with not less than a million functions to
take care of all most all (analytical) problems of different domains of knowledge. There are umpteen
number of packages on CRAN to take care of optimization techniques. The following are only few
that might be best but rest are not despicable. Each package is great in its way of solving the
problem in a very unique way.
These packages are available under CRAN Task View: Optimization and Mathematical Program-
ming. The following is the list with description.

– optimx: helps to proper specify the (nonlinear) optimization problem including objective
function, gradient function, and scaling. This package supports the (local) optimization
of smooth, nonlinear functions with at most box constraints (bounds).
– Rcgmin and Rvmmin: ”pure R” implementations of conjugate gradient minimization
and variable metric nonlinear function minimization algorithms, respectively.
– The R Optimization Infrastructure (ROI): provides a framework for handling optimiza-
tion problems in R. It uses an object oriented approach to define and solve various opti-
mization tasks in R which can be from different problem classes (e.g., linear, quadratic,
non-linear programming problems).
– stats: offers several general purpose optimization routines. First, function optim()
provides an implementation of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method,
bounded BFGS, conjugate gradient, Nelder-Mead, and simulated annealing (SANN) op-
timization methods. It utilizes gradients, if provided, for faster convergence. Typically it
is used for unconstrained optimization but includes an option for box-constrained opti-
mization. Additionally, for minimizing a function subject to linear inequality constraints
stats contains the routine constrOptim(). For one-dimensional unconstrained function
optimization there is optimize() which searches an interval for a minimum or maxi-
mum. Then there is nlm which is used for solving nonlinear unconstrained minimization
problems. Eventually, nlminb() offers unconstrained and constrained optimization using
PORT routines.
– For other packages you may visit http://cran.r-project.org/web/views/Optimization.
html.

References
[1] Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Linear_programming

12
 [2] Retrieved from http://sourceforge.net/about
[3] Retrieved from http://lpsolve.sourceforge.net/5.5/
[4] Retrieved from http://en.wikipedia.org/wiki/R_%28programming_language%29
[5] Retrieved from http://cran.r-project.org/bin/windows/base/
[6] Retrieved from http://www.gnome.org/
[7] Retrieved from http://en.wikipedia.org/wiki/S_%28programming_language%29
[8] Retrieved from http://cran.r-project.org/web/views/

13

View publication stats