Dynamic Routing with Path Diversity and Consistency for Compact Network Learning.

Huanyu Wang,Zequn Qin,Songyuan Li,Xi Li
2020-01-01
Abstract:Dynamic routing networks, aimed at finding the best routing paths in thenetworks, have achieved significant improvements to neural networks in terms ofaccuracy and efficiency. In this paper, we see dynamic routing networks in afresh light, formulating a routing method as a mapping from a sample space to arouting space. From the perspective of space mapping, prevalent methods ofdynamic routing didn't consider how inference paths would be distributed in therouting space. Thus, we propose a novel method, termed CoDiNet, to model therelationship between a sample space and a routing space by regularizing thedistribution of routing paths with the properties of consistency and diversity.Specifically, samples with similar semantics should be mapped into the samearea in routing space, while those with dissimilar semantics should be mappedinto different areas. Moreover, we design a customizable dynamic routingmodule, which can strike a balance between accuracy and efficiency. Whendeployed upon ResNet models, our method achieves higher performance andeffectively reduces average computational cost on four widely used datasets.
What problem does this paper attempt to address?