Improving Image Similarity Learning by Adding External Memory

Xinjian Gao,Tingting Mu,John Y. Goulermas,Jingkuan Song,Meng Wang

DOI: https://doi.org/10.1109/tkde.2020.3047104

IF: 9.235

2020-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:The type of neural networks widely used in artificial intelligence applications mixes its computation and memory modules in neuron weights and activities. The previously learned information are stored in network weights. When dealing with complex data, e.g., those possessing diverse content or containing long-sequences, some information stored in the weights can be altered drastically or wiped as the training goes, but they are not necessarily unimportant. External memory is a recent technique proposed to prevent from forgetting significant previously learned information. In this work, we aim at taking advantage of this recent technique to advance the similarity learning task that is critical in many real-world artificial intelligence applications. We propose suitable external memory design supported by extended attention mechanism. Two different kinds of memory modules are proposed so that the similarity learning process can dynamically shift focus over a wide range of diverse content contained by the training data. Effectiveness of the proposed method is demonstrated through evaluations based on different image retrieval tasks and compared against various state-of-the-art algorithms in the field.