Orthogonal projection non-negative matrix factorization using alternating direction method of multipliers
Huabin Wang,Cheng Lu,Jian Zhou,Liang Tao,Hanqin Shi
DOI: https://doi.org/10.11834/jig.20170406
2017-01-01
Journal of Image and Graphics
Abstract:Objective The multiplicative update rule is generally used in non-negative matrix factorization (NMF).Such rule exhibits simple implementation characteristics,and thus,the linear complexity for each iteration is frequently applied because it is more stable than Newton's method.However,the multiplicative update rule also has disadvantages,such as slow convergence,asymptotic convergence to zero,and poor local optima.In particular,when the size of the data to be processed is large,the multiplicative update rule exhibits extremely low timeliness,and thus,it cannot be applied to several real-time applications,such as online object tracking.To address these limitations,an orthogonal projection NMF method based on alternating direction method of multipliers is proposed.The traditional NMF algorithm does not guarantee the availability of good partial-based representation;hence,a non-negative projection matrix and an orthogonal constraint are introduced.An orthogonal projection NMF (OPNMF),which can reduce computation complexity,is applied.Simultaneously,the bases exhibit high orthogonality due to the orthogonal and sparse characteristics of OPNMF.Therefore,obtaining an excellent part-based representation of the object becomes possible.Method As a new development of the Lagrange augmentation algorithm,the alternative direction method of multipliers (ADMM) is a simple and effective approach to solve the separable convex programming problem (particularly large-scale problems).One of the advantages of ADMM is that it can separate the objective function of the original problem into several sub-problems,which are easier to find local solutions via the separability of original function,and thus,an optimal solution is obtained.Compared with the NMF based multiplicative update rule,OPNMF based on the alternative direction method of multipliers does not only converge to the global solution,but also requires less convergence time.The derivation of the proposed algorithm is presented in this study.First,the solution of the original objective function is decomposed into the alternative optimization of different sub-problems based on the orthogonality and sparsity of OPNMF.The augmented Lagrangian equation of the original objective function is established to equally represent the sub-problems of the original problem by introducing auxiliary variables.Then,the main and dual variables of the transformed equation are optimized alternately.In particular,the partial derivatives of these variables are used for each equation to find the current optimal solution.Finally,the corresponding update rules are derived and the iterative process of each variable is updated alternately to obtain the optimal solution.Result Four matrices with different sizes are selected for the simulation to compare the performance of the ADMM algorithm with the multiplicative update rule in OPNMF.Experimental results show that the proposed method clearly outperforms the traditional approach in terms of convergence speed and accuracy.In particular,as the size of the matrix increases,the convergence rate of the algorithm becomes considerably higher than that of the multiplicative update rule.In the second experiment,we apply the proposed method to object tracking,which is a classical study area in computer vision.The observation model of the moving object is represented by the bases of OPNMF,i.e.,the candidate object in each frame is represented by the linear combination of the basis vectors.During the tracking stage,the observation model is updated on time to avoid tracking drift due to the continuous appearance changes of the target object.A new template-updating strategy based on ADMM,which combines OPNMF part-based representation and ADMM update speed,is also proposed.Compared with the multiplicative update rule and three other state-of-the-art object tracking algorithms,the experimental results show that the proposed method is effective with OPNMF based on the multiplicative updating rule.The tracking speed is approximately 3.8 times that based on the multiplicative updating rule.The overlap rate of the proposed method is approximately 0.73,which is better than those of the other three object tracking algorithms.Conclusion The proposed method is considerably superior when the size of the matrix is large.The ADMM method can achieve faster convergence rate and higher convergence precision when ρ is appropriately selected.Moreover,object tracking based on OPNMF,which benefits from the sparseness and orthogonality of this algorithm,can address certain interference situations,such as occlusion,scale change,and illumination change,thereby achieving robust tracking.