Chapter 9: MM Algorithms

DOI: https://doi.org/10.1137/1.9781611976175.ch9

2020-01-01

Abstract:Other Titles in Applied Mathematics, Page 105-120. <br/> 9.1. Introduction Every MM algorithm is motivated by the desire to convert a hard optimization problem into a sequence of simpler ones [88, 111]. In minimization the MM principle majorizes the objective function f(x) by a surrogate function g(x | xn) anchored at the current iterate xn. Majorization combines two conditions: the tangency condition g(xn | xn) = f(xn) and the domination condition g(x | xn) ≥ f(x) for all x. Figure 9.1 depicts two majorizations of the function cos x. The next iterate of the MM algorithm by definition minimizes the function x → g(x | xn). Because the MM iterates generate a descent algorithm driving the objective function downhill. Strictly speaking, the descent property depends only on decreasing g(x | xn), not on minimizing it. Constraint satisfaction is automatically built into the solution xn + 1. Under appropriate regularity conditions, an MM algorithm is guaranteed to converge to a stationary point of the objective function. If the objective function is strictly convex, then all stationary points collapse to the global minimum. In maximization, one first minorizes and then maximizes. Minorization reverses the domination inequality and produces an ascent algorithm. Thus, the acronym MM can be interpreted as either majorize-minimize or minorize-maximize.