A Novel Neuron Connection Model Mimicking Human Beings.
Jiaxu Leng,Zhenyu Cui,Chao Xiang,Tianlin Zhang,Pei Quan
DOI: https://doi.org/10.1109/bibm47256.2019.8983419
2019-01-01
Abstract:Neural Networks have achieved great success in many computer vision tasks, especially in image recognition. However, as neural networks grow deeper and deeper, to some extend, we've found them becoming difficult to train, and requiring samples in large scale dramatically, even with the help of Dropout and Dropconnect methods, which do improve the accuracy a bit but burdens the training process as a sacrifice. To overcome this, we propose a novel neuron connection model to generate dynamic graphs of computation. As synapses have two kinds: excitatory and inhibitory ones, our model also has two kinds of connections for neurons. In addition, we propose a training algorithm that deals with non-differentiable because the equations of the connections and activation function of neurons in our model are not really differentiable. To evaluate the effectiveness the proposed method, we apply it to the image recognition task, and the results show that our proposed model achieves state-of-the-art performance on three public datasets: MNIST, CIFAR-10, and CIFAR-100.