SEMESTER/MASTER PROJECT: Retina AI project

A fundamental challenge in neuroengineering is finding the proper input to a sensory or motor system that yields a desired functional output. For example, sensory prostheses (like visual prostheses) are typically limited in the number of input channels (e.g. electrodes), and this number is orders of magnitude lower than natural visual inputs. Then, an open question is how to determine the simplest spatiotemporal pattern of electrical stimuli able to induce the desired sensation in a patient implanted with a retinal prosthesis. In this case, finding the proper input of the system is an ill-posed problem. In a linear system, the problem of finding the input that produces a desired output can be determined by monitoring its response to a series of arbitrary inputs and then computing the inverse of the system’s transmission matrix (a mapping from inputs to outputs). This entails measuring the responses of the system fully. For a given sensory system, this means measuring the activity of all neurons in the system at all times (which is technically not possible in mammals). In practice, the neural system can only be partially measured and is nonlinear. Even though the forward paths of the system could be fully characterized, obtaining its inverse is computationally intensive if not entirely intractable for large scale systems involving millions of variables. Therefore, a computational method to estimate the forward and backward paths of this severely ill-posed (non)linear system, given the limited set of examples obtained in the forward path, is highly desirable.

Neural networks perform well in solving ill-posed inverse problems for various applications. We have recently developed a data-driven approach based on the synergistic tandem combination of neural networks for controlling the output of a time-varying nonlinear medium consisting of an optical multimode fiber [1]. We have selected this medium as it provides a time-varying system with hundreds of thousands of degrees of freedom to control. With partial intensity measurements, the problem of choosing the proper input that would result in a desired output of the fiber becomes a nonlinear and severely ill-posed problem. But despite partial measurements, the approach has proved remarkably successful in achieving a performance on par with that of the gold standard methods that require full characterization of the system.

Encouraged by the success of the proposed algorithm, we propose to develop neural network-based algorithms for control applications in neuroscience. In particular, as a study case, we will engineer the stimuli of the system to produce desired responses of the network of neurons in the mouse visual system. The system we will consider, consists of an interconnected network of cells starting from light detection at the photoreceptors all the way to the visual cortex of the brain. This system is extremely complex and nonlinear. When an image is formed onto the photoreceptors, the neural activity is processed by several layers of neurons within the retina and a train of neural spikes is sent to the brain via the optic nerve. Within the brain, several areas process the information until it finally reaches the primary visual cortex.  We propose to use our computational frame work to first model the forward path from the stimuli (optical) to neural activity (electrical spikes) of the ganglion cells in the retina and then compute the appropriate input images (optical), that when projected onto the photoreceptors provides a desired temporal activity of the ganglion neural cells.

Responsibilities: The student working on the project would be provided with the dataset and is expected to apply learning algorithms such as recurrent networks and/or reinforcement learning. It is not required for student to have a know-how of these learning frame-works beforehand but he/she should be motivated enough to be able to replicate/modify currently available network architectures in the domain to our dataset. The student will be provided with a starter code and will work with the supervisor closely. A basic (not advanced) know-how of deep learning packages (tensorflow or pytorch) is required.

contact: [email protected] ; [email protected]