EE 686 Optimization Theory

Fall Semester 2014

Professor:

Dong-Jo Park

Teaching Assistants:

Junghoon Ko

Tel: (042) 350-3438

Office: 5th Story Section Room 2223

E-mail: djpark@ee.kaist.ac.kr
Cellular phone: 010-5407-8381
http://armi.kaist.ac.kr

Tel: (042) 350-5438

Office: 5th Story Section Room 2232

E-mail: junghoon_ko@kaist.ac.kr
Cellular phone: 010-5183-1107

Hyeong Geun Yu Tel: (042) 350-5438

Office: 5th Story Section Room 2232

E-mail: elloss@kaist.ac.kr
Cellular phone: 010-6434-0189

1. Course Contents

The primary objective of this course is to present fundamental concepts and basic techniques of
optimization which are essential for researches in circuit design, communications, signal processing,
control and industrial engineering and to deal with their application areas. We study linear vector
spaces and linear operators, linear estimation and filtering, and optimization techniques in function
spaces using geometric principles. Using duality, we consider dual problems in the dual spaces also
deal with topics on existence of a Lagrangean multiplier in the constrained optimization problems.
Other topics covered in this course are linear programming, nonlinear programming, dynamic
programming, genetic programming (evolutionary computation), network traffic control problems, and
neural networks.
A brief course description is as follows:
1) Linear programming
2) Nonsimplex method
3) Nonlinear programming
4) Linear vector spaces and Hilbert space
5) Least square estimation & Kalman filtering
6) Dual spaces
7) Linear operators and adjoints
8) Optimization of functionals
9) Constrained optimization
10) Multiobjective optimization
11) Optimal control
12) Dynamic programming
13) Genetic algorithm and variants
14) Neural networks
15) Convex optimization
16) Advanced topics

2. Lectures

Lectures are better if they are interactive. Ask plenty of questions, in class or out of class. There are
no foolish questions. If you do not understand something I am explaining, please feel free to interrupt
me and ask for clarification. You may be doing a favor to rest of the class. Lecture notes are available
on http://armi.kaist.ac.kr

3. Textbooks

1) An Introduction to Optimization - Fourth ed. -- Edwin K. P. Chong and Stanislaw H. Zak, WileyInterscience, A John Wiley & Sons, Inc., Publication 2013
2) Linear and Nonlinear Programming - Third ed. -- David G. Luenberger and Yinyu Ye
Springer Science + Business Media LLC 2008
3) Optimization by Vector Space Methods -- David G. Luenberger,
John Wiley & Sons, Inc., 1969

4. Homework

Homework will be assigned on every Thursday and will be due on next Thursday 24:00. Late
homeworks will have their penalty by - 20 % per each delayed week. The homework box to submit
your homework is located in front of the entrance door of room 2232.

5. Examination

There will be two midterm exams and one final exam. Date will be announced.

6. Grades

Grade = Homework and Term Project (20%) + Midterm Exams (30%) + Final Exam (40%)
+ Attendance and Quizzes (10%)

7. Office Hour

Tuesday, Thursday 13:30 - 15:00, or Catch me, or Call me for appointment.

8. References

1) Convex Optimization -- Stephen Boyd and Lieven Vandenverghe, Cambridge Unversity Press
2004
2) Mathematical Methods and Algorithms for Signal Processing -- Todd K. Moon and
Wynn C. Stirling, Prentice-Hall, Inc., 2000
3) Optimization Concepts and Applications in Engineering -- Ashok. D. Belegundu and
Tirupati. R. Chandrupatla, Prentice-Hall, Inc., 1999
4) Optimization in Operations Research -- Ronald L. Rardin, Prentice-Hall, Inc., 1998
5) Introduction to Mathematical Programming -- 4th Edition -- Wayne L. Winston and
Munirpallam Venkataramanan, Brooks/Cole-Thomson Learning Inc., 2003
6) Introduction to Operation Research -- 7th Edition -- Frederick S. Hillier and
Gerald J. Lieberman, McGraw-Hill Companies, Inc. International Edition, 2001
7) Optimization: Theory and Applications -- S. S. Rao, John Wiley & Sons, Inc., 1979
8) Nonlinear Programming: Theory and Algorithms -- 4th Edition -- Mokthar S. Bazaraa,
Hanif D. Sherali & C. M. Shetty, John Wiley & Sons, Inc., 1993
9) Genetic Algorithms in Search, Optimization & Machine Learning -- David E. Goldberg,
Addison Wesley Publishing Company, Inc., 1989
10) Evolutionary Computation -- David B. Fogel, IEEE Press, 1995
11) Neural Networks for Optimization and Signal Processing -- Andrzej Cichocki and
R. Unbehauen, John Wiley and Sons, Inc., 1993
12) Linear Operator Theory in Engineering and Science -- Arch W. Naylor and
George R. Sell, Springer-Verlag, 1982