A remark on a paper of Krotov and Hopfield [<a class="link-https" data-arxiv-id="2008.06996" href="https://arxiv.org/abs/2008.06996">arXiv:2008.06996</a>]

Fei Tang,Michael Kopp

DOI: https://doi.org/10.48550/arXiv.2105.15034

2021-06-03

Abstract:In their recent paper titled "Large Associative Memory Problem in Neurobiology and Machine Learning" [<a class="link-https" data-arxiv-id="2008.06996" href="https://arxiv.org/abs/2008.06996">arXiv:2008.06996</a>] the authors gave a biologically plausible microscopic theory from which one can recover many dense associative memory models discussed in the literature. We show that the layers of the recent "MLP-mixer" [<a class="link-https" data-arxiv-id="2105.01601" href="https://arxiv.org/abs/2105.01601">arXiv:2105.01601</a>] as well as the essentially equivalent model in [<a class="link-https" data-arxiv-id="2105.02723" href="https://arxiv.org/abs/2105.02723">arXiv:2105.02723</a>] are amongst them.

Neurons and Cognition,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning