Towards Robust Discriminative Projections Learning via Non-Greedy <inline-formula><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xl
Feiping Nie,Zheng Wang,Rong Wang,Zhen Wang,Xuelong Li
DOI: https://doi.org/10.1109/TPAMI.2019.2961877
IF: 23.6
2021-01-01
IEEE Transactions on Pattern Analysis and Machine Intelligence
Abstract:Linear Discriminant Analysis (LDA) is one of the most successful supervised dimensionality reduction methods and has been widely used in many real-world applications. However, <inline-formula><tex-math notation="LaTeX">$\ell _2$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq2-2961877.gif"/></alternatives></inline-formula>-norm is employed as the distance metric in the objective of LDA, which is sensitive to outliers. Many previous works improve the robustness of LDA by using <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq3-2961877.gif"/></alternatives></inline-formula>-norm distance. However, the robustness against outliers is limited and the solver of <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq4-2961877.gif"/></alternatives></inline-formula>-norm is mostly based on the greedy search strategy, which is time-consuming and easy to get stuck in a local optimum. In this paper, we propose a novel robust LDA measured by <inline-formula><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq5-2961877.gif"/></alternatives></inline-formula>-norm to learn robust discriminative projections. The proposed model is challenging to solve since it needs to minimize and maximize (minmax) <inline-formula><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq6-2961877.gif"/></alternatives></inline-formula>-norm terms simultaneously. As a result, we first systematically derive an efficient iterative optimization algorithm to solve a general ratio minimization problem, and then rigorously prove its convergence. More importantly, an alternately non-greedy iterative re-weighted optimization algorithm is developed based on the preceding approach for solving proposed <inline-formula><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq7-2961877.gif"/></alternatives></inline-formula>-norm minmax problem. Besides, an optimal weighted mean mechanism is driven according to the designed objective and solver, which can be applied to other approaches for robustness improvement. Experimental results on several real-world datasets show the effectiveness of proposed method.