The daisy project will

• quantitatively characterize the daisy architecture
• identify the style of processing supported by the daisy architecture
• explore the role of the daisy architecture for the assignment of meaning (semantics) to cortical processing
• develop and test practical soft- and hardware implementations of cortical processing based on the daisy architecture

Quantitative characterization of the daisy architecture

Local circuits consisting of patchy lateral projections (daisies), are a ubiquitous feature of the superficial layers of neocortex and are integrated into the network of long-range feedback and feedforward inter-areal connections. However, the description in the literature of the long-range and local connectivity and their interaction is largely qualitative. Therefore, our first objective will be to obtain a firm quantitative, functional, and computational understanding of this network by attaining the following sub-objectives:

Obtain a quantitative description of the feedforward and feedback connections, and their relationship to the local daisy architecture

There is a well established body of evidence on the connectivity and physiology of intracortical connections but, other than the observation of patchiness itself, we have little idea how and why (functionally) the local lateral connections should express them selves as patches; what the computational significance of this arrangement may be; and how and why the local organization system relates structurally to inter-areal connectivity. Understanding the relationship between these systems will give insight into the integration of local and global signals. More specifically, we will obtain a better understanding of the nature of feedforward and feedback pathways and their linking of cortical areas in the cortical hierarchy, and whether the parent neurons of these pathways are distinct from the parent neurons of the intra-cortical connectivity.

Determine whether individual cortical areas exhibit a specific and unique connectivity profile and if the pattern of connections between a single area and its collaborators is homogenous across the target area, or varies in a manner suggestive of context dependent processing

While the cortico-cortical connectivity of early visual areas has been worked out in some detail, we do not yet know if a given area receives the same or different proportions of connections from each afferent area. We refer to this distribution of proportions as the ‘connectivity profile’. If the connectivity profile is characteristic for each area it suggests that the computations carried out by an area are constrained by its connectivity profile. In the visual system the physiology of cortex subserving the peripheral visual field is markedly different from that in the central visual field. Our working hypothesis is that, more generally, the connectivity profiles of visual areas change with eccentricity, indicating that visual processing within and between areas is context sensitive across the visual field.

Describe quantitatively the hierarchical architecture of cortex, and explore the role of static and dynamic hierarchies in models of inter-areal computation

There is a large body of evidence that cortical areas are organized statically in a hierarchical fashion, and that information flows between lower and higher areas by feedforward and feedback pathways. We will attack the important open question of whether there is a single static hierarchy, or whether the hierarchy depends in a context-dependent way on the regions of the cortical areas in which they are measured. We will use these data to explore how the static hierarchy relates to dynamic hierarchies of processing.

Map quantitatively the dimensions and patterns of the individual axonal arbors and bouton clusters of layer 3 pyramidal cells in visual cortex

The lateral connections of the superficial layer pyramidal cells provide for two crucial functions: one is to distribute their output to other individual or clusters of pyramidal cells, the other is to provide a convergent input to an individual pyramidal cell or cluster of pyramidal cells. A feature of these lateral connections is that they form clusters that have a certain dimension and spacing. However, there are many unaswered questions about the relation between the axon of any individual pyramidal cell and the daisy. For example, is the size of the cluster fomed by an individual pyramidal cell equivalent to the size of a daisy cluster, or is it a fixed fraction of the diasy size? Are individual axons mapped within the daisy cluster, or do they just connect any where within the daisy? Answering these questions will give us important information as to the components that structure the daisy and the contribution of individual pyramidal cells to the daisy. We will use quantitative data from single and bulk studies to establish the relationship between individual neurons and the daisy to which they contribute.

Determine whether the neuronal network created by the horizontal connections is degenerate (all connect to all) or selective (some connect to some)

In developing the graph theoretic models of the daisy architecture, one key structural question needs to be answered: how do individual neurons connect within the daisy? One possibility is that the pyramidal cells are selective as to which clusters they connect. Another is that they simply connect to all possible clusters. As yet we do not know the answers. Within the inter- and intraareal systems it is not known whether each neuron projects to only a single target region, or whether individual neurons project to a large fraction of its possible targets via axonal collaterals. These organizations imply very different computational architectures: shared output corresponds to a degenerate or redundant architecture, whereas segregated output suggests selective routing of information, which could support selective, context-dependent, event-based processing. This objective will determine the relative contribution of redundant and segregated connections in the intra- and interareal systems and with respect to feedback and feedforward processing.

Estimate the weighting of the connections from individual neurons to target neurons located in local and distant clusters

Each cluster of the daisy contains a certain number of target neurons and each cluster is supplied by a certain number of afferent axons, arising in other clusters, which form synapses with the target neurons. However, within a cluster there are also local connections between pyramidal cells. A certain fraction of the local and intercluster synapses are also formed with inhibitory neurons. The form and degree of interaction between neurons in different patches is only approximately known (Ts'o et al., 1986; Kisvarday et al., 1997; Fitzpatrick, 2000). In order to understand the nature of processing in the daisy architecture, these interactions must be comprehensively measured at both the anatomical and physiological level. At the structural level we need to measure the number of synapses being supplied to the target neurons by their neighbours, by other clusters, and by interareal connections. These data will provide a quatitative description of the synaptic input to the average pyramidal neuron in a daisy, and thus form the basis for interpreting the physiological data and essential constraints for the theoretical modeling.

Identify the style of processing supported by the daisy architecture

The style of computation in nervous systems is fundamentally different from that used in present day computers. Nervous systems emphasize distributed, collective, self-organizing, event-based processing. These processing modes are poorly understood and generally not exploited in conventional computers. Therefore, we propose to combine characterization of the architecture (OBJ-1) with theoretical and physiological exploration of the general style of processing that the architecture could support. Our sub-objectives here are:

Demonstrate how constraints on neuronal processing and neuronal morphology relate to the observed daisy-like structure

We will measure spatial patterns of subthreshold and spiking activity by imaging voltage sensitive dyes and two-photon microscopy, and interpret these functional data in the light of quantitative neuroanatomy. We will explore theoretically whether and how the constraints imposed by simple axonal growth rules, circuit operating stability, and spike time dependent plasticity (STDP) learning could lead to the emergence of the daisy architecture in the superficial layers of cortex. We will establish whether complex temporal patterns of source activity, such as partial synchrony of sub-populations can lead to the formation of circuit primitives, such as the partitioning of a target pool to respond selectively to multiple inputs; and the factorization of lightly coupled input pools into their sources.

Explore the possible relationship between cortical computation and novel models of computation, such as graphical models, and dynamic link architectures

The existence of daisy architecture raises the possibility that the cortex may be using the principle of graphical models such as Bayesian Belief Networks and Factor Graphs. We will use graphical modelling to analyze and model the relationship between architecture and processing in biological circuits, and conversely, use the cortical circuits to guide the construction of useful, richly interconnected graphical models. In particular, we will explore the novel possibility that, unlike current models of graphical processing that emphasize static loop-free graphs, the daisy architecture supports dynamic, process-dependent, re-configuration of looped graphs. Dynamic link architectures (DLA) also provide a promising method for encoding and recognizing objects by dynamic composition of a subset of primitive attributes. Therefore, we will explore whether the daisy architecture offers a substrate for DLA, and consider how DLA can be seen in relation to graphical models.

Investigate the nature of asynchronous, event-encoded cortical processing by physiological experiment, simulation, and emulation in hybrid analog digital hardware

Unlike conventional computers, communication and computation in neuronal networks depends on asynchronous pulse communication, and stochastic computational elements with very large fan-in and fan-out and probabilistic connections. We will explore this mode of computation by direct physiological experiment, and by implementation in simulation and hardware. We will use novel imaging methods with high temporal resolution to characterize the spike output and interaction of large numbers of neurons during co-ordinated activity. We will examine how daisy-like networks of integrate and fire neurons with plastic synapses can establish and maintain the general operations required for graphical and dynamic-link processing. We will consider whether and how stochastic and event driven processing can contribute to the effectiveness of graphical and dynamic-link processing. In all of these studies we will be concerned to evaluate and test how event based computation can best be abstracted into electronic circuits and systems.

Explore the role of the daisy architecture for the assignment of meaning (semantics) to cortical processing

In conventional computers world data are semantically mapped onto a set of explicit, programmer-defined symbols that are manipulated by a well-defined algorithm. In brains, this semantic mapping occurs spontaneously without the aid of a programmer, and is far richer. It appears that neural patterns are interpreted, and acquire semantic significance, by being reproducibly linked with other patterns through learning and natural selection. These associative links are meaningful in representing causal or otherwise regularities in the external world, and make it possible for the brain to generate economically relevant motor patterns in response to perceived or remembered world-states; as when manipulating an object to be eaten. We propose to approach the problem of meaning by direct observation of fields of cortical activity as they respond to stimuli chosen for their perceptual significance, and by simulating complex meaningful tasks such as face recognition, using the biological plausible circuits of the DA.

Determine the relationship between the activation of neurons by visual stimuli, their relationship to the daisy architecture, and the structure of the sensory images that drive them

We will use voltage sensitive dyes and two-photon microscopy to image the sub-threshold and spiking activities of a field of cortical neurons. We will explore the relationship between the structure of the image, and the spatial and temporal dynamics of activation of neurons in relation to the daisy architecture. We will use elaborate stimuli, to explore the ability of the cortex to detect relevant long-range organization in an image. We will compare the form and strength of the activation states in response to simple semantically ordered (recognizable) and disordered (scrambled) stimuli. We will use simulation methods to reconcile our physiological measurements with the measured neuroanatomical characteristics of the daisy architecture, and the computational characteristics of graphical and dynamic link processing.

Develop a mechanism, based on the daisy architecture, that extracts tokens sensitive in an invariant manner to meaningful classes of stimuli

The responses of neurons in higher visual areas are sensitive to structure, but relatively insensitive to position, and appearance. We will explore conjunctive mechanisms for representing complex structure, and disjunctive mechanisms for expressing their invariance. We will develop methods for automatic extraction invariant tokens suitable for incorporation in Bayesian networks, and mechanisms for encoding such tokens also in DLA. We will assess whether and how these mechanisms can be implemented in the daisy architecture, and what their signature might be in fields of neurons observed by voltage-sensitive dye or two-photon microscopy methods.

Implement the topology constraint of invariant pattern matching using daisy connections

We propose that cortical columns are not isolated units but are integrated with their neighbors via the DA so that they can function together as topologically coherent sheets. This continuity expresses itself in a number of ways, for instance in implementing the Gestalt rules, such as the continuity of contours. We will demonstrate the use of the topology constraint during pattern matching. This constraint requires that neighboring points in one pattern link to neighboring points of the target pattern. We will implement this constraint using daisy connections between neurons in neighboring cortical columns (OBJ-3.2). We will establish how the satisfaction of the constraint can be learned.

Explore practical neuronal switching mechanisms that support assembly of tokens by rapid re-configuration within the daisy architecture

One solution to the object 'binding problem' is to dynamically assemble a token for that object composed of elements that encode its features. It is possible that these tokens could be composed of individual neurons, but the necessary strong synaptic connections between neurons cannot be changed on the short time-scale of perception. We propose that the daisy architecture is able to encode objects using dynamic linkage between its patches. We will explore the possibility that the linkages that form the object token do not depend on the synapses between individual neurons, but rather on the rapid selection or de-selection of entire groups of neurons communicating between participating patches. We will explore how the assembly of these tokens could be guided by the context provided by the inter-areal connections, by using for example a ‘pointer-map’ architecture (Hahnloser et al, 1999).

Develop and test practical implementations of cortical processing relevant to daisy architectures

We will develop and test practical ‘proof-of-concept’ implementations of cortical processing based on understanding gained via objectives (OBJ-1 to3) of this Project. Our sub-objectives here are:

Develop a factor graph-like processor to create a classifier generator

We will develop algorithms for automatically learning and constructing novel classifiers based only on training examples. Rather than constructing a specific classifier for a pre-defined class, we will develop algorithms that accept image examples as input and produce new classifiers as output. The input will consist of labeled images from several new classes of objects, unknown to the system. Based on the factor graph cortical interpretation model, the algorithms will automatically construct classifiers, which can assign novel images to any of the novel classes.

Develop a system for invariant object recognition based on dynamic link processing

We will demonstrate a system for invariant object recognition, in particular, human face recognition. In distinction to its more computer-adapted forerunners (Wiskott et al., 1997), our system will be formulated as a realistic model of neural dynamics and of cortical organization, particularly of its columnar structure and the DA.

Develop a system of custom neuromorphic VLSI electronic circuits, which will demonstrate how the principles of cortical computation can be implemented in hardware

We will demonstrate a hardware system of neurons that will perform context dependent recognition of simple objects. The hardware will consist of asynchronous, event encoding processors, fabricated as custom hybrid analog/digital CMOS VLSI circuits, in which the local interpretation of data by a sub-population depends on syntactical constraints between subpopulations.