Brain-based algorithms make for better networks

Brain-based algorithms make for better networks

Salk and Carnegie Mellon researchers uncover how the brain prunes back synapses in development

LA JOLLA–When it comes to developing efficient, robust networks, the brain may often know best.
Researchers from the Salk Institute for Biological Studies and Carnegie Mellon University have, for the first time, determined the rate at which the developing brain eliminates unneeded connections between neurons during early childhood.

Though engineers use a dramatically different approach to build distributed networks of computers and sensors, the research team of computer scientists discovered that their newfound insights could be used to improve the robustness and efficiency of distributed computational networks. The findings were published in PLOS Computational Biology on July 16, 2015.

“By thinking computationally about how the brain develops, we questioned how rates of synapse pruning may affect network topology and function,” says Saket Navlakha, assistant professor at the Salk Institute’s Center for Integrative Biology and a former postdoctoral researcher in Carnegie Mellon’s Machine Learning Department. “We have used the resulting insights to develop new algorithms for constructing adaptive and robust networks in other domains.”

Network structure is an important topic for both biologists and computer scientists. In biology, understanding how the network of neurons in the brain organizes to form its adult structure is key to understanding how the brain learns and functions. In computer science, understanding how to optimize network organization is essential to producing efficient interconnected systems.

Salk and Carnegie Mellon researchers developed a new model for building efficient networks by studying the rate at which the brain prunes back some of its connections during development. In this model, nodes (such as neurons or sensors) make too many connections (left) before pruning back to connections that are most relevant (right). The team applied their synaptic pruning-based algorithm to air flight patterns and found it was able to create routes to allow passengers to reach their destinations efficiently.

Click here for a high-resolution image.

Image: Courtesy of the Salk Institute for Biological Studies

But the processes the brain and network engineers use to learn the optimal network structure are very different.
Neurons create networks through a process called pruning. At birth and throughout early childhood, the brain’s neurons make a vast number of connections–more than the brain needs. As the brain matures and learns, it begins to quickly cut away connections that aren’t being used. When the brain reaches adulthood, it has about 50 to 60 percent less synaptic connections than it had at its peak in childhood.

In sharp contrast, computer science and engineering networks are often optimized using the opposite approach. These networks initially contain a small number of connections and then add more connections as needed.

“Engineered networks are built by adding connections rather than removing them. You would think that developing a network using a pruning process would be wasteful,” says Ziv Bar-Joseph, associate professor in Carnegie Mellon’s Machine Learning and Computational Biology departments. “But as we showed, there are cases where such a process can prove beneficial for engineering as well.”

The researchers first determined key aspects of the pruning process by counting the number of synapses present in a mouse model’s somatosensory cortex over time. After counting synapses in more than 10,000 electron microscopy images, they found that synapses were rapidly pruned early in development, and then as time progressed, the pruning rate slowed.

The results of these experiments allowed the team to develop an algorithm for designing computational networks based on the brain pruning approach. Using simulations and theoretical analysis they found that the neuroscience-based algorithm produced networks that were much more efficient than the current engineering methods.

In the networks created with pruning, the flow of information was more direct, and provided multiple paths for information to reach the same endpoint, which minimized the risk of network failure.

“We took this high-level algorithm that explains how neural structures are built during development and used that to inspire an algorithm for an engineered network,” says Alison Barth, professor in Carnegie Mellon’s Department of Biological Sciences and member of the university’s BrainHubSM initiative. “It turns out that this neuroscience-based approach could offer something new for computer scientists and engineers to think about as they build networks.”

As a test of how the algorithm could be used outside of neuroscience, Navlakha applied the algorithm to flight data from the U.S. Department of Transportation. He found that the synaptic pruning-based algorithm created the most effective routes to allow passengers to reach their destinations.

“We realize that it wouldn’t be cost effective to apply this to networks that require significant infrastructure, like railways or pipelines,” Navlakha said. “But for those that don’t, like wireless networks and sensor networks, this could be a valuable adaptive method to guide the formation of networks.”

In addition, the researchers say the work has implications for neuroscience and believe that the change in pruning rates from adolescence to adulthood could indicate that there are different biochemical mechanisms that underlie pruning.

“Algorithmic neuroscience is an approach to identify and use the rules that structure brain function,” Barth said. “There’s a lot that the brain can teach us about computing, and a lot that computer science can do to help us understand how neural networks function.”

About the Salk Institute for Biological Studies:
The Salk Institute for Biological Studies (https://www.salk.edu/) is one of the world’s preeminent basic research institutions, where internationally renowned faculty probe fundamental life science questions in a unique, collaborative and creative environment. Focused both on discovery and on mentoring future generations of researchers, Salk scientists make groundbreaking contributions to our understanding of cancer, aging, Alzheimer’s, diabetes and infectious diseases by studying neuroscience, genetics, cell and plant biology, and related disciplines.