Abstract:The k-nearest neighbor (k-NN) decision rule is the basis of a well-established, high-performance pattern-recognition technique but its sequential implementation is inherently slow. More recently, feedforward neural networks trained on error backpropagation have been widely used to solve a variety of pattern-recognition problems. However, it is arguably unnecessary to learn such a computationally intensive solution when one (i.e., the k-NN rule) is effectively available a priori, especially given the well-known pitfalls of backpropagation. Accordingly, there is some interest in the literature in network implementations of this rule, so as to combine its known, good performance with the speed of a massively parallel realization. In this paper, we present a novel neural-network architecture which implements the L-NN rule and whose distinctive feature relative to earlier work is its synchronous (i.e., clocked) nature. Essentially, it has a layered, feedforward structure but, in its basic form, also incorporates feedback to control sequential selection of the k neighbors. The principal advantages of this new scheme are the avoidance of the stability problems which can arise with alternative asynchronous feedback (lateral-inhibition) circuits, the restriction of analog weights to the first hidden layer and the fact that network design uses noniterative weight calculations rather than iterative backpropagation. Analysis of the network shows that it will converge to the desired solution (faithfully classifying the input pattern according to the k-NN rule) within (2k - 1) clock cycles. Apart from minor changes which can be effected externally, the same design serves for any value of X. The space complexity of the ('brute-force'' network implementation is O(N-2) units, where N is the number of training patterns, and it has O(N(2)d) analog weights where d is the dimensionality of these patterns. Thus, some modifications to reduce the required number of units (and, thereby, weighted connections) are considered. Overall, this paper affords for high-speed, parallel implementations of proven pattern-classification techniques.

A Neural-Network Dimension Reduction Method for Large-Set Pattern Classification

Deep Manifold Transformation for Dimension Reduction

Kernel Generalized Nonlinear Discriminant Analysis Algorithm For Pattern Recognition

Application on Classification of One Dimension Sequence Using Nonlinear Model Based on Olfactory System

Image Classification Method Rationally Utilizing Spatial Information of the Image

Pattern Classification with Ordered Features Using Mrmr and Neural Networks

Image Classification by Feature Dimension Reduction and Graph based Ranking

A Kernel Fractional-Step Nonlinear Discriminant Analysis for Pattern Recognition

On Neural-Network Implementations of K-Nearest Neighbor Pattern Classifiers

Fusion of effective dimension reduction and discriminative dictionary learning for high-dimensional classification

A Comparison Study on Nonlinear Dimension Reduction Methods with Kernel Variations: Visualization, Optimization and Classification.

A Sphere Neighborhood Covering-Structuring Approach Based on Neural Network and Its Application

Belted and Ensembled Neural Network for Linear and Nonlinear Sufficient Dimension Reduction

Pattern recognition of multivariate information based on non-statistical techniques

A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks

Golden Ratio-Based Sufficient Dimension Reduction

Feature dimensionality reduction: a review

Rank-Adaptive Non-Negative Matrix Factorization

A Learning Algorithm Detecting the Similar Chinese Characters' Boundary Based on Unequal-Contraction of Dimension

Dimensionality reduction for evolving neural network

Classifying Unbalanced Pattern Groups by Training Neural Network