62 43 v 2 [ cs . L G ] 30 J an 2 01 5 Structure Regularization for Structured Prediction : Theories and Experiments

Xu Sun

2018-01-01

Abstract:While there are many studies on weight regularization, the s udy on structure regularization is rare. Many existing systems on structured pr ediction focus on increasing the level of structural dependencies within the mo del. However, this trend could have been misdirected, because our study suggests tha t complex structures are actually harmful to generalization ability in structur ed prediction. To control structure-based overfitting, we propose a structure regula rization framework via structure decomposition , which decomposes training samples into mini-samples with simpler structures, deriving a model with better gener alization power. We show both theoretically and empirically that structure reg ularization can effectively control overfitting risk and lead to better accuracy. As a by-product, the proposed method can also substantially accelerate the trai ning speed. The method and the theoretical results can apply to general graphical m odels with arbitrary structures. Experiments on well-known tasks demonstrate t hat our method can easily beat the benchmark systems on those highly-competit ive tasks, achieving state-of-the-art accuracies yet with substantially faste r training speed.