Preface XI
1 Function Root-Finding 1
1.1 Introduction 1
1.2 Substitution Algorithms 4
1.3 Bolzano’s Algorithm 7
1.4 Function Approximation 8
1.4.1 Newton’s Method 9
1.4.2 The Secant Method 10
1.4.3 Regula Falsi Method 11
1.4.4 Muller’s Method or Parabolic Interpolation 12
1.4.5 Hyperbolic Interpolation Method 13
1.4.6 Inverse Polynomial Interpolation Method 13
1.4.7 Inverse Rational Interpolation Method 14
1.5 Use of a Multiprocessor Machine with a Known Interval of Uncertainty 16
1.6 Search for an Interval of Uncertainty 17
1.7 Stop Criteria 17
1.8 Classes for Function Root-Finding 21
1.9 Case Studies 26
1.9.1 Calculation of the Volume of a Nonideal Gas 26
1.9.2 Calculation of the Bubble Point of Vapor–Liquid Equilibrium 28
1.9.3 Zero-Crossing Problem 30
1.9.4 Stationary Condition in a Gravity-Flow Tank 33
1.10 Tests for BzzFunctionRoot and BzzFunctionRootMP Classes 35
1.11 Some Caveats 39
2 One-Dimensional Optimization 41
2.1 Introduction 41
2.2 Measuring the Efficiency of the Search for the Minimum 45
2.3 Comparison Methods 46
2.4 Parabolic Interpolation 58
2.5 Cubic Interpolation 59
2.6 Gradient-Based Methods 59
2.7 Combination of Algorithms in a General Program 60
2.8 Parallel Computations 60
2.9 Search for the Interval of Uncertainty 61
2.10 Stop Criteria 61
2.11 Classes for One-Dimensional Minimization 62
2.12 Case Studies 69
2.12.1 Optimization of Unimodal Functions 69
2.12.2 Optimization of a Batch Reactor 71
2.12.3 Maximum Level in a Gravity-Flow Tank in Transient Conditions 74
2.13 Tests 77
3 Unconstrained Optimization 79
3.1 Introduction 79
3.1.1 Necessary and Sufficient Conditions 80
3.1.2 Quadratic Functions 81
3.1.3 Directions of Function Decrease 83
3.1.4 Comparison with the One-Dimensional Case 84
3.1.5 Classification of Methods 85
3.2 Heuristic Methods 86
3.2.1 Modified Hooke–Jeeves Method 88
3.2.2 The Rosenbrock Method 89
3.2.3 The Nelder–Mead Simplex Method 92
3.2.4 Robust Optnov Method Combined with the Simplex Method 95
3.3 Gradient-Based Methods 98
3.4 Conjugate Direction Methods 100
3.5 Newton’s Method 105
3.6 Modified Newton Methods 109
3.6.1 Singular or Nonpositive Definite Hessian Matrix 110
3.6.2 Convergence Problems 116
3.6.3 One-Dimensional Search 118
3.6.4 Trust Region Methods 121
3.6.5 Use of Alternative Methods 124
3.7 Quasi-Newton Methods 126
3.8 Narrow Valley Effect 131
3.9 Stop Criteria 133
3.10 BzzMath Classes for Unconstrained Multidimensional Minimization 135
3.11 Case Study 141
3.11.1 Optimization of a Batch Reactor 142
3.11.2 Optimal Adiabatic Bed Reactors for Sulfur Dioxide with Cold Shot Cooling 144
3.11.3 Global Optimization 148
3.12 Tests 150
4 Large-Scale Unconstrained Optimization 153
4.1 Introduction 153
4.2 Collecting a Sparse Symmetric Matrix 154
4.3 Ordering the Hessian Rows and Columns 156
4.4 Quadratic Functions 161
4.5 Hessian Evaluation 171
4.6 Newton’s Method 173
4.7 Inexact Newton Methods 173
4.8 Practical Preconditioners 180
4.9 openMP Parallelization 180
4.10 Class for Large-Scale Unconstrained Minimization 180
5 Robust Unconstrained Minimization 185
5.1 Introduction 185
5.2 One-Dimensional Minimization 186
5.3 Classes for One-Dimensional Robust Minimization 187
5.4 Examples in One-Dimensional Space 190
5.5 Examples in Multidimensional Space 199
5.6 Two-Dimensional Minimization 202
5.7 Classes for Robust Two-Dimensional Minimization 205
5.8 Examples for BzzMinimizationTwoVeryRobust Class 206
5.9 Multidimensional Robust Minimization 216
5.9.1 Outer Optimizer 217
5.9.2 Inner Optimizer 218
5.10 Class for Robust Multidimensional Minimization 219
6 Robust Function Root-Finding 225
6.1 Introduction 225
6.2 Class and Examples 225
7 Nonlinear Systems 235
7.1 Introduction 235
7.2 Comparing Nonlinear Systems to Other Iterative Problems 236
7.2.1 Comparison with Function Root-Finding 236
7.2.2 Comparison with the Multidimensional Optimization 238
7.3 Convergence Test 240
7.4 Substitution Methods 244
7.5 Minimization Methods 244
7.6 Jacobian Evaluation 245
7.7 Newton’s Method 245
7.8 Gauss–Newton Method 248
7.9 Modified Newton Methods 250
7.9.1 Singular or Ill-Conditioned Jacobian 250
7.9.2 Convergence Problem 253
7.10 Newton’s Method and Parallel Computations 256
7.11 Quasi-Newton Methods 257
7.12 Quasi-Newton Methods and Parallel Computing 260
7.13 Stop Criteria 261
7.13.1 Bounds, Constraints, and Discontinuities 261
7.14 Classes for Nonlinear System Solution with Dense Matrices 262
7.15 Tests for the BzzNonLinearSystem Class 268
7.16 Sparse and Large-Scale Systems 272
7.17 Large Linear System Solution with Iterative Methods 278
7.18 Classes for Nonlinear System Solution with Sparse Matrices 279
7.19 Continuation Methods 281
7.20 Solution of Certain Equations with Respect to Certain Variables 284
7.21 Case Studies 287
7.21.1 Heat Exchange in a Thermal Furnace 287
7.21.2 Calculation of Chemical Equilibria 289
7.21.3 Multiple Solutions in a CSTR 294
7.21.4 Critical Size of a Nuclear Reactor 296
7.21.5 Stationary Conditions for a Nonisothermal Continuous Stirred Tank Reactor 298
7.21.6 Vapor–Liquid Equilibrium: The Flash Separator 300
7.21.7 Boundary Value Problems 302
7.22 Special Cases 302
7.22.1 Vapor–Liquid Equilibrium: Distillation Column 302
7.22.2 Kinetic Postprocessor 303
7.23 Some Caveats 306
8 Underdimensioned Nonlinear Systems 313
8.1 Introduction 313
8.2 Underdimensioned Linear Systems 314
8.2.1 Null Space 314
8.2.2 Determination of One Solution 316
8.2.3 Projection Methods 327
8.2.4 Stable Gauss Factorization 331
8.3 Class for Underdimensioned Nonlinear System Solution 339
9 Constrained Minimization 343
9.1 Introduction 343
9.2 Equality Constraints 344
9.3 Equality and Inequality Constraints 346
9.4 Lagrangian Dual Problem 350
10 Linear Programming 355
10.1 Introduction 355
10.2 Basic Attic Method Concepts 357
10.3 Attic Method 358
10.3.1 Certain Important Peculiarities of the Attic Method 359
10.3.2 What Happens in a Generic Iteration 360
10.3.3 Selecting the New Feasible Point and the New Inequality Constraint 361
10.3.4 Selection of the Search Direction 361
10.4 Differences between the Attic Method and Traditional Approaches 363
10.4.1 A Simple Application of the Attic Method 367
10.4.2 Certain Advantages to the Attic Method 368
10.5 Explosion in the Number of Iterations 370
10.5.1 Selecting Constraints Rather Than Vertices 371
10.5.2 The Tile Effect 377
10.5.3 Wall Constraints and Roof Constraints 378
10.5.4 When a Constraint with l>0 Should also be Removed 380
10.6 Degeneracy 382
10.7 Duality 385
10.8 General Considerations 387
11 Quadratic Programming 389
11.1 Introduction 389
11.2 KKT Conditions for a QP Problem 392
11.3 Equality-Constrained QP 393
11.3.1 Solving the Full KKT System 394
11.3.2 Shur-Complement Method 397
11.3.3 Null Space Methods 399
11.4 Equality- and Inequality-Constrained Problems 404
11.5 Class for QP 406
11.6 Projection or Reduced Direction Search Methods for Bound-Constrained Problems 407
11.7 Equality, Inequality, and Bound Constraints 412
11.8 Tests 416
12 Constrained Minimization: Penalty and Barrier Functions 419
12.1 Introduction 419
12.2 Penalty Function Methods 419
12.2.1 Quadratic Penalty Function 421
12.2.2 Nonsmooth Exact Penalty Function 423
12.2.3 The Maratos Effect 426
12.2.4 Augmented Lagrangian Penalty Function 430
12.2.5 Bound-Constrained Formulation for Lagrangian Penalty Function 434
12.3 Barrier Function Methods 434
12.4 Mixed Penalty–Barrier Function Methods 437
13 Constrained Minimization: Active Set Methods 439
13.1 Introduction 439
13.2 Class for Constrained Minimization 441
13.3 Successive Linear Programming 455
13.4 Projection Methods 457
13.5 Reduced Direction Search Methods 461
13.6 Projection or Reduced Direction Search Methods for Bound-Constrained Problems 463
13.7 Successive Quadratic Programming or Projected Lagrangian Method 464
13.7.1 Selection of the Merit Function 467
13.7.2 Updating the Jacobian of the System 470
13.8 Narrow Valley Effect 471
13.9 The Nonlinear Constraints Effect 472
13.10 Tests 473
14 Parametric Continuation in Optimization and Process Control 477
14.1 Introduction 477
14.2 Algebraic Constraints 478
14.2.1 Distillation Column 478
References 481
Appendix A: Copyrights 487
Index 489
