PO-DARTS: Post-optimizing the Architectures Searched by Differentiable Architecture Search Algorithms

Debei Hao,Songwei Pei

DOI: https://doi.org/10.1007/978-3-031-44201-8_15

2023-01-01

Abstract:In neural architecture search, differentiable architecture search algorithm has become one of the mainstream methods. However, no matter in the search or evaluation stage, the architecture is repeatedly stacked by two kinds of Cells, namely Normal Cell and Reduction Cell, respectively. This undoubtedly limits the performance of the evaluation architecture to a large extent due to the architecture restriction, resulting in sub-optimal performance. In order to alleviate the impact of architecture restriction on network performance, this paper proposes to post-optimize the architecture searched by differentiable architecture search algorithms by freezing the architecture parameters of partial Cells and further searching other Cells to bring more diversity into the stacked Cells. The proposed post-optimizing methods consist of the global post-optimizing search method and the local post-optimizing search method, respectively. The performance of the evaluation architecture can benefit from the diverse stacked Cells with less architecture restriction. In the experiments, the proposed post-optimizing method is applied to the mainstream differentiable architecture search algorithms such as DARTS and P-DARTS, and superior results have been achieved on CIFAR-10 and CIFAR-100 datasets. Moreover, the proposed method can obtain the post-optimized architecture with limited computing resources.