Bilaterally Symmetrical Encoding in the Evolution of Artificial Neural Networks for Symmetry Detection
Derek James,P. Dallas,A. Maida
Abstract:In order for neuroevolutionary techniques to produce increasingly complex and sophisticated topologies, new methods need to be developed which effectively exploit reuse and modularity. Bilateral symmetry is an important form of reuse and a key feature of complex biological central nervous systems. We present a method for encoding bilateral symmetry within the context of an existing neuroevolutionary framework, NEAT (NeuroEvolution of Augmenting Topologies). We then present a model of symmetry detection that relies on the symmetry in the structure of the neural system to make symmetry judgments. We demonstrate that this model performs better than an asymmetrical representation on a symmetry discrimination task in which the axis of symmetry is given. On a second task, the networks must first find the axis of symmetry before making the symmetry judgment. In this task the symmetrical encoding performs worse than the asymmetrical one. We discuss some possible explanations for these results. 1.INTRODUCTION The specific class of techniques that apply evolutionary algorithms to exploration of both the connection weights and topologies of artificial neural networks can be broadly classified in two ways. As Yao [18] described these classes of techniques: "At one extreme, all the details, i.e., every connection and node of an architecture, can be specified by the chromosome. This kind of representation scheme is called direct encoding." This type of encoding specifies a 1-to-1 mapping between a gene and a topological feature (such as a neuron or connection), which is both impractical from a scalability aspect and not biologically plausible. Conversely, indirect encoding refers to the class of techniques in which there is some aspect of development used in deriving the network from the genetic representation so that there is not simply a direct 1-to-1 mapping of gene to feature. For example, a gene in an indirect encoding scheme may encode for a modular substructure that may be reused many times in the course of building the phenotype. Stanley and Mikkulainen [17] refer to the development phase in such a computational paradigms as artificial embryogeny (AE). They point out the importance of genetic reuse and lay out the conceptual framework for classifying these different approaches. Modular reuse in neuroevolution has been investigated with a variety of different methods [1][8][12][14]. Eggenberger [5] explicitly selected for the emergence of bilateral symmetry in the evolution of artificial 3D morphologies and studied the effects. However, to our knowledge, the encoding of bilateral symmetry in artificial neural networks has not been investigated. Bilateral symmetry is a key feature of complex biological nervous systems, and is realized very early in the development of body plans of many biological embryos. Since most of the sensory input for complex organisms is collected from bilaterally-symmetrical organs, and many motor tasks (such as locomotion) involve symmetrical appendages, it is reasonable to hypothesize that there is a correlation between the bilateral symmetry inherent in the organism's nervous system and the symmetry inherent in its sensory input and motor output. It is also possible that the symmetrical structure of mammalian brains provides an explanation for other cognitive functions. In Braitenberg's Vehicles [2], he proposes a simple bilateral symmetry detector, an array of elements symmetrically spaced from each other with respect to a midline. An image projected onto such an array would exhibit strong activation for symmetrical images, in which input is balanced on both sides of the input array with respect to the midline, but weak activation for asymmetrical imagery, in which input is not balanced. Other researchers [4], including Braitenberg [3] have proposed that the symmetrical organization of the human brain may be responsible for the innate preference for vertical symmetry [5] and its increased salience over other orientations. Palmer and Hemenway [13] proposed a two-stage model of symmetry detection in humans, the first stage involving the location of the axis of symmetry, and the second the actual symmetry judgment. In recent years, strong models of symmetry detection have been developed [7][9]. However, these cognitive models of symmetry detection do not draw a direct link between the symmetry inherent in the task and the underlying symmetry of the neural structure. Neural network models of symmetry detection [11][15] have found connection weights trained via backpropagation tend to converge on similar values for symmetrical connections, though they have been criticized for either being too small to scale realistically, or taking too many epochs to train. However, the authors of [10] assert that increased preference and performance in vertical symmetry is a result of more early exposure to stimuli with vertical axes of symmetry. We introduce a new method of indirect encoding for the evolution of artificial neural networks that allows for the flexible exploration of topologies with varying degrees of bilateral symmetry. Using this encoding scheme, we introduce a model for symmetry detection with a basis on the underlying structure of neural systems. Copyright is held by the author/owner(s). GECCO’06, July 8–12, 2006, Seattle, WA, USA. ACM 1-59593-186-4/06/0007. 2.BILATERAL SYMMETRY ENCODING The new encoding system was developed as a modification to an existing neuroevolutionary algorithm, NEAT (NeuroEvolution of Augmenting Topologies) [16]. In its original implementation, NEAT is a direct encoding algorithm. There are only two types of genes specified: neuron genes and connection genes. Each gene encodes for exactly one topological feature, i.e., one neuron gene encodes for exactly one neuron in the resulting neural network. In the new encoding, the chromosome provides all information necessary for deterministically constructing the network. These instructions are in the form of additional attributes for each gene that define sidedness and connectivity. Unlike previous implementations of NEAT, the resulting network has spatial structure: a midline, and right and left sides. Connections either cross the midline when connecting neurons (contralateral) or do not cross the midline (ipsilateral). All genes are defined by a symmetry attribute with three values: LEFT, RIGHT, or BOTH. Neuron genes with the LEFT or RIGHT value are transcribed into neurons on that side of the resulting network. Neuron genes with the BOTH attribute are transcribed into mirrored copies on both sides of the resulting network (Fig. 1). Figure 1. Neuron genes 1 and 4 have a symmetry attribute of LEFT and neuron gene 2 has a symmetry attribute of RIGHT. Neuron genes 0 and 3 have a symmetry attribute of BOTH, so they encode for mirrored copies (a and b). In addition to the symmetry attribute, connection genes also have a lateralization attribute, with one of two values: IPSILATERAL (does not cross the midline) and CONTRALATERAL (crosses the midline). There are three cases of connectivity (where A denotes asymmetric neurons and S denotes symmetric neurons): 1) A to A 2) A to S and S to A 3) S to S For A to A connectivity, symmetry and lateralization attributes are ignored and transcription proceeds as in previous implementations of NEAT (Figure 2). Figure 2. Connection gene 5 encodes for a connection between LEFT neuron 4 and RIGHT neuron. Figure 3 shows the three possible cases of connectivity for A to S. In these cases, the symmetry attribute (LEFT, RIGHT, or BOTH) indicates which copies of the symmetrical neuron the connection will connect either to or from. The lateralization attribute is ignored. Figure 3. Three cases for connectivity between A to S neurons: left (top), right (middle), and both (bottom). These attributes encode for S to A connectivity in the same way. Figure 4 shows the six possible cases of connectivity between pairs of symmetrical neurons. In these cases, the symmetry attribute defines the connectivity from the source neuron(s), so a connection gene with a LEFT value would only connect from the left copy of the source neuron. The lateralization attribute determines whether the transcribed connection(s) cross the midline or not. For the initial topology, per the original NEAT paradigm, only input and output neurons are defined, which now may be either symmetrical or asymmetrical. NEAT also begins minimally, with no hidden units and only feedforward connections. Initially, all connections have a symmetry value of BOTH and a lateralization value of IPSILATERAL. This choice was made under the assumption that connectivity in biological brains is predominantly interhemispheric. This is one aspect of the implementation that is could be parameterized and explored in future work. NEAT has three mutation operators: 1) Perturb existing connection weights, 2) Add new connection (including recurrent ones), and 3) Add new neuron. These mutations act on the genes, and work as before with the following exceptions.