Advances in solving optimal control problems

In 2013 the Advanced Systems Analysis (ASA) Program made several new contributions to the development of optimal control theory for managing socio-environmental systems within a modeling framework.

Puzzle © Orla| iStock


Analytical methods for solving optimal control problems (OCPs) allow closed-form solutions to be obtained in low-dimensional conceptual models. A useful tool for studying qualitative phenomena, these reveal general patterns in the underlying processes of complex interactions in society, in the environment, and between them.

The widely used Pontryagin maximum principle (MP) supplies a set of necessary (and in some cases sufficient) conditions for optimality. In addition to state and control variables, such conditions have a so-called adjoint variable. This was originally seen as an auxiliary parameter. However, the properties of the adjoint variable play an important role when the MP is applied to solving OCPs, often allowing a single optimal process to be selected from a set of candidates derived from the core conditions of the MP. In economic applications the adjoint variable is used to represent the notion of the shadow price.

For many years the ASA program has contributed to advances in optimal control theory by developing special versions of the MP for OCPs on the infinite time horizon. These are the most common methodological tools for many applications, for example, economic growth theory.

In 2013 ASA made several new contributions. A class of infinite time-horizon OCPs with a dominating discount that appear in economic growth theory was studied [1]. An explicit formula for the adjoint variable was derived and it was shown that as well as satisfying core MP conditions, it also satisfied the additional stationarity condition for the Hamiltonian. The results obtained extend to a general case of non-differentiable value function.

There was consideration of how to interpret a solution to an OCP posed in terms of achieving a long-term goal from a perspective of short-term goals [2]. Using a stylized model of an enterprise as an illustrative example plus the MP as a solution tool, it was shown that by properly evaluating current shadow prices, a decision maker can reduce, or even eliminate, a conflict between long- and short-term interests in enterprise management. This effectively makes current short-term-optimal actions identical to long-term optimal actions.

Some properties of attainability sets of nonlinear dynamic systems were explored [3]; the results are useful for analyzing control systems with phase constraints.

In view of the growing complexity of socio-environmental problems considered by the scientific community, it is also important to develop numerical methods for solving OCPs that properly approximate an original solution and are computationally efficient.

In 2013 ASA continued its work on developing numerical approaches to solving OCPs on the infinite time horizon.  It is difficult to apply standard numerical algorithms to solve infinite time-horizon optimal control problems because the boundary condition on the right-hand side of the Hamiltonian system, which relates to the special properties of the adjoint variable, extends to infinity. 

There was a presentation and demonstration of the applicability of the new numerical method to localizing the search for optimal trajectories on the phase plane in the vicinity of the steady state of the Hamiltonian system by building a non-linear stabilizer [4] [5]. The non-linear stabilizer approach to developing a numerical algorithm for solving an infinite time-horizon optimal control problem with phase constraints was applied [6]. Scientists also addressed the problem of controlling a system affected by disturbing inputs and suggested an on-line state-reconstruction algorithm to build a feedback controller [7]. 


[1] Aseev SM (2013). On some properties of adjoint variables in relations of the Pontryagin maximum principle for optimal economic growth problems. Proceedings of the Institute of Mathematics and Mechanics of the Ural Branch of Russian Academy of Sciences, 19 (4), 15—24 [In Russian].
[2] Kryazhimskiy A (2013). Long-term and short-term targets: conflict and reconciliation. Innovation and Supply Chain Management, 6 (3), 1—5.
[3] Davydov AA, Mann HD (2013). Singularities of the attainable set on an orientable surface with boundary. Journal of Mathematical Sciences, 188(3), 185—196.
[4] Tarasyev A, Usova A (2013). Application of a nonlinear stabilizer for localizing search of optimal trajectories in control problems with infinite horizon. Numerical Algebra, Control and Optimization, 3 (3), 389—406.
[5] Tarasyev A, Usova A (2013). Algorithms for construction of optimal trajectories based on control approximations in problems on infinite time horizon.Proceedings of the Scientific Conference on Informatics Problems (SPISOK 2013), St. Petersburg, Russia, 346—353.[in Russian].
[6] Tarasyev A, Usova A (2013). Nonlinear stabilizers in optimal control problems with infinite time horizon. In: D. Hoemberg, F. Troeltzsch (Eds.). System Modeling and Optimization, Springer, Berlin, Germany, 391, 286-295.
[7] Kryazhimskiy A, Maksimov V (2013). On combination of the processes of reconstruction and guaranteeing control. Automation and Remote Control, 74(8), 1235-1248 [Original Russian text published in Avtomatika i Telemekhanika, 2013, 8, 5-21].


IIASA's Russian National Member Organization has always strongly supported the methodological research at IIASA.
ASA’s main collaborators in the field for solving optimal control problems include: K.O. Besov, Research Scholar, Steklov Mathematical Institute, Moscow, Russia; V.I. Maksimov, Department Head, Institute of Mathematics and Mechanics, Urals Branch, Russian Academy of Sciences, Russia; V. Veliov, Professor, Vienna University of Technology, Austria.

Print this page

Last edited: 21 May 2014


Elena Rovenskaya

Program Director:Acting Program Director

Advanced Systems Analysis

Evolution and Ecology

T +43(0) 2236 807 608

Further information



International Institute for Applied Systems Analysis (IIASA)
Schlossplatz 1, A-2361 Laxenburg, Austria
Phone: (+43 2236) 807 0 Fax:(+43 2236) 71 313