What machines can
teach us…by learning

Machine learning is an artificial intelligence approach that lets a computer learn from new data, updating its own results as it receives new information—allowing it to predict what might happen next. Los Alamos National Laboratory researchers are using machine learning in everything from forecasting earthquakes to exploring astrophysics and geoscience.



AI’s Big Challenge

AI's Big Challenge in Scientific AmericanTo maintain U.S. supremacy in AI, the best way forward is to adopt a strategy hewing more closely to the way humans learn, putting us on the best path to the benefits promised by full-fledged artificial intelligence.

AI's Big Challenge

To make it truly intelligent, researchers need to rethink they way they approach the technology

By Garrett Kenyon

The recently signed executive order establishing the American AI Initiative correctly identifies artificial intelligence as central to American competitiveness and national defense. However, it is unclear if AI has accomplished anywhere near as much as many have claimed. Indeed, current technology exhibits no convincing demonstration of anything remotely approaching “intelligence.”

To maintain U.S. supremacy in AI, the best way forward is to adopt a strategy hewing more closely to the way humans learn, which will put us on the best path to the economic growth and widespread social benefits promised by full-fledged artificial intelligence.

Here’s the challenge with most deep learning neural networks, which reflect the prevailing approach to AI: calling them both deep and intelligent assumes they achieve ever more abstract and meaningful representations of the data at deeper and deeper levels of the network. It further assumes that at some point they transcend rote memorization to achieve actual cognition, or intelligence. But they do not.

Consider computer vision, where deep neural networks have achieved stunning performance improvements on benchmark image-categorization tasks. Say we task our computer vision algorithm with correctly labeling images as either cats or dogs. If the algorithm correctly labels the images, we might conclude that the underlying deep neural network has learned to distinguish cats and dogs.

Now suppose all of the dogs are wearing shiny metallic dog tags and none of the cats are wearing cat tags. Most likely, the deep neural network didn’t learn to see cats and dogs at all but simply learned to detect shiny metallic tags. Recent work has shown that something like this actually underpins the performance of deep neural networks on computer vision tasks. The explanation may not be as obvious as shiny metallic tags, but most academic data sets contain analogous unintentional cues that deep learning algorithms exploit.

Using adversarial examples, which are designed to foil neural networks, adds even more disturbing evidence that deep neural networks might not be “seeing” at all but merely detecting superficial image features. In a nutshell, adversarial examples are created by running in reverse the same computational tools used to train a deep neural network. Researchers have found that adding very slight modifications to an image—imperceptible to humans—can trick a deep neural network into incorrectly classifying an image, often radically.

The problem, it turns out, is one of computational misdirection. Adding or deleting just a few pixels can eliminate a particular cue that the deep neural network has learned to depend on. More fundamentally, this error demonstrates that deep neural networks rely on superficial image features that typically lack meaning, at least to humans.

That creates an opportunity for serious mischief by bad actors using targeted adversarial examples. If you’re counting on consistent image recognition for self-driving cars designed to recognize road signs, for example, or security systems that recognize fingerprints … you’re in trouble.

This flaw is built into the architecture. Recent research in Israel led by Naftali Tishby has found that a deep neural network selectively drops non-essential information at each layer. A fully trained deep neural network has thrown away so much information and has become so dependent on just a few key superficial features—“shiny metal tags”—that it has lost all semblance of intelligence. Deep learning is more accurately described as deep forgetting.

Even more damning, deep neural networks exhibit no capacity to learn by analogy, the basis of all intelligence. For example, humans and other animals use analogy to learn that the world consists of objects that possess common attributes. Whether it’s a rock, an apple or a baseball, all such objects fall to the ground because they obey the laws of an intuitive physics learned during the development of intelligence.

Researchers at Brown University recently tested whether deep neural networks could learn by analogy. The team found that neural networks failed to learn the concept of sameness. Instead of learning by analogy the underlying concept linking the examples of similar images in a training set of images, deep neural networks simply memorized a set of templates for correctly labeling the images in the training set. The networks gained no capacity to generalize outside the training data.

It is difficult to imagine a more searing indictment of deep learning than the inability to learn by analogy. Essentially all cognitive development rests on learning and abstracting the principles underlying a set of concrete examples. The failure, thus far, of deep learning to do so reveals the emptiness behind the facade of intelligence presented by current A.I. systems.

By jumping over the long, slow process of cognitive development and instead focusing on solving specific tasks with high commercial or marketing value, we have robbed AI of any ability to process information in an intelligent manner.

This story first appeared in Scientific American.

Computers learn to imagine the future

Machine learning to predict future outcomes | Los Alamos National LaboratoryResearchers are simulating biological neural networks on supercomputers, enabling machines to learn about their surroundings, interpret data and make predictions.

Discover logoComputers learn to imagine the future

by Garrett Kenyon

In many ways, the human brain is still the best computer around. For one, it’s highly efficient. Our largest supercomputers require millions of watts, enough to power a small town, but the human brain uses approximately the same energy as a 20-watt bulb. While teenagers may seem to take forever to learn what their parents regard as basic life skills, humans and other animals are also capable of learning very quickly. Most of all, the brain is truly great at sorting through torrents of data to find the relevant information to act on.

At an early age, humans can reliably perform feats such as distinguishing an ostrich from a school bus, for instance – an achievement that seems simple, but illustrates the kind a task that even our most powerful computer vision systems can get wrong. We can also tell a moving car from the static background and predict where the car will be in the next half-second. Challenges like these, and far more complex ones, expose the limitations in our ability to make computers think like people do. But recent research at Los Alamos National Laboratory is changing all that.

Brain neuroscientists and computer scientists call this field neuromimetic computing – building computers inspired by how the cerebral cortex works. The cerebral cortex relies on billions of small biological “processors” called neurons. They store and process information in densely interconnected circuits called neural networks. In Los Alamos, researchers are simulating biological neural networks on supercomputers, enabling machines to learn about their surroundings, interpret data and make predictions much the way humans do.

This kind of machine learning is easy to grasp in principle, but hard to implement in a computer. Teaching neuromimetic machines to take on huge tasks like predicting weather and simulating nuclear physics is an enterprise requiring the latest in high-performance computing resources.

Los Alamos has developed codes that run efficiently on supercomputers with millions of processing cores to crunch vast amounts of data and perform a mind-boggling number of calculations (over 10 quadrillion!) every second. Until recently, however, researchers attempting to simulate neural processing at anything close to the scale and complexity of the brain’s cortical circuits have been stymied by limitations on computer memory and computational power.

All that has changed with the new Trinity supercomputer at Los Alamos, which became fully operational in mid-2017. The fastest computer in the United States, Trinity has unique capabilities designed for the National Nuclear Security Administration’s stockpile stewardship mission, which includes highly complex nuclear simulations in the absence of testing nuclear weapons. All this capability means Trinity allows a fundamentally different approach to large-scale cortical simulations, enabling an unprecedented leap in the ability to model neural processing.

To test that capability on a limited-scale problem, computer scientists and neuroscientists at Los Alamos created a “sparse prediction machine” that executes a neural network on Trinity. A sparse prediction machine is designed to work like the brain: researchers expose it to data – in this case, thousands of video clips, each depicting a particular object, such as a horse running across a field or a car driving down a road.

Cognitive psychologists tell us that by the age of six to nine months, human infants can distinguish objects from background. Apparently, human infants learn about the visual world by training their neural networks on what they see while being toted around by their parents, well before the child can walk or talk.

Similarly, the neurons in a sparse prediction machine learn about the visual world simply by watching thousands of video sequences without using any of the associated human-provided labels – a major difference from other machine-learning approaches. A sparse prediction machine is simply exposed to a wide variety of video clips much the way a child accumulates visual experience.

When the sparse prediction machine on Trinity was exposed to thousands of eight-frame video sequences, each neuron eventually learned to represent a particular visual pattern. Whereas a human infant can have only a single visual experience at any given moment, the scale of Trinity meant it could train on 400 video clips simultaneously, greatly accelerating the learning process. The sparse prediction machine then uses the representations learned by the individual neurons, while at the same time developing the ability to predict the eighth frame from the preceding seven frames, for example, predicting how a car moves against a static background.

The Los Alamos sparse prediction machine consists of two neural networks executed in parallel, one called the Oracle, which can see the future, and the other called the Muggle, which learns to imitate the Oracle’s representations of future video frames it can’t see directly. With Trinity’s power, the Los Alamos team more accurately simulates the way a brain handles information by using only the fewest neurons at any given moment to explain the information at hand. That’s the “sparse” part, and it makes the brain very efficient and very powerful at making inferences about the world – and, hopefully, a computer more efficient and powerful, too.

After being trained in this way, the sparse prediction machine was able to create a new video frame that would naturally follow from the previous, real-world video frames. It saw seven video frames and predicted the eighth. In one example, it was able to continue the motion of car against a static background. The computer could imagine the future.

This ability to predict video frames based on machine learning is a meaningful achievement in neuromimetic computing, but the field still has a long way to go. As one of the principal scientific grand challenges of this century, understanding the computational capability of the human brain will transform such wide-ranging research and practical applications as weather forecasting and fusion energy research, cancer diagnosis and the advanced numerical simulations that support the stockpile stewardship program in lieu of real-world testing.

To support all those efforts, Los Alamos will continue experimenting with sparse prediction machines in neuromorphic computing, learning more about both the brain and computing, along with as-yet undiscovered applications on the wide, largely unexplored frontiers of quantum computing. We can’t predict where that exploration will lead, but like that made-up eighth video frame of the car, it’s bound to be the logical next step.

Garrett Kenyon is a computer scientist specializing in neurally inspired computing in the Information Sciences group at Los Alamos National Laboratory, where he studies the brain and models of neural networks on the Lab’s high-performance computers. Other members of the sparse prediction machine project were Boram Yoon of the Applied Computer Science group and Peter Schultz of the New Mexico Consortium.

This story first appeared in Discover.

Machine-learning in lab shows promise

Machine-learning earthquake prediction in lab shows promise | Los Alamos National LaboratoryBy listening to the acoustic signal emitted by a laboratory-created earthquake, a computer science approach using machine learning can predict the time before the fault fails.

Machine-learning earthquake prediction in lab shows promise

Listening to faultline’s grumbling gives countdown to future quakes

LOS ALAMOS, N.M., Aug. 30, 2017—By listening to the acoustic signal emitted by a laboratory-created earthquake, a computer science approach using machine learning can predict the time remaining before the fault fails.

“At any given instant, the noise coming from the lab fault zone provides quantitative information on when the fault will slip,” said Paul Johnson, a Los Alamos National Laboratory fellow and lead investigator on the research, which was published today in Geophysical Research Letters.

“The novelty of our work is the use of machine learning to discover and understand new physics of failure, through examination of the recorded auditory signal from the experimental setup. I think the future of earthquake physics will rely heavily on machine learning to process massive amounts of raw seismic data. Our work represents an important step in this direction,” he said.

Not only does the work have potential significance to earthquake forecasting, Johnson said, but the approach is far-reaching, applicable to potentially all failure scenarios including nondestructive testing of industrial materials brittle failure of all kinds, avalanches and other events.

Machine learning is an artificial intelligence approach to allowing the computer to learn from new data, updating its own results to reflect the implications of new information. The machine learning technique used in this project also identifies new signals, previously thought to be low-amplitude noise, that provide forecasting information throughout the earthquake cycle.

“These signals resemble Earth tremor that occurs in association with slow earthquakes on tectonic faults in the lower crust,” Johnson said. “There is reason to expect such signals from Earth faults in the seismogenic zone for slowly slipping faults.”

Machine learning algorithms can predict failure times of laboratory quakes with remarkable accuracy. The acoustic emission (AE) signal, which characterizes the instantaneous physical state of the system, reliably predicts failure far into the future. This is a surprise, Johnson pointed out, as all prior work had assumed that only the catalog of large events is relevant, and that small fluctuations in the AE signal could be neglected.

To study the phenomena, the team analyzed data from a laboratory fault system that contains fault gouge, the ground-up material created by the stone blocks sliding past one another. An accelerometer recorded the acoustic emission emanating from the shearing layers.

Following a frictional failure in the labquake, the shearing block moves or displaces, while the gouge material simultaneously dilates and strengthens, as shown by measurably increasing shear stress and friction. “As the material approaches failure, it begins to show the characteristics of a critical stress regime, including many small shear failures that emit impulsive acoustic emissions,” Johnson described.

“This unstable state concludes with an actual labquake, in which the shearing block rapidly displaces, the friction and shear stress decrease precipitously, and the gouge layers simultaneously compact,” he said. Under a broad range of conditions, the apparatus slide-slips fairly regularly for hundreds of stress cycles during a single experiment. And importantly, the signal (due to the gouge grinding and creaking that ultimately leads to the impulsive precursors) allows prediction in the laboratory, and we hope will lead to advances in prediction in Earth, Johnson said.

The paper: "Machine learning predicts laboratory earthquakes" Geophysical Research Letters.

The funding: Los Alamos National Laboratory Directed Research and Development (LDRD).

How flounders (yes, the fish) can help national security 

How studying a flounder can help with national security | Los Alamos National LaboratoryLos Alamos National Laboratory researchers have developed an algorithmic framework to study camouflage in nature to learn how to identify things trying to disguise themselves.

HuffPost logo

How flounders (yes, the fish) can help national security

By Lakshman Prasad

When the American painter Abbott H. Thayer published his book Concealing-Coloration in the Animal Kingdom in 1909, he put forth the hypothesis that animals’ colors served one function and one function only: to camouflage. While that theory has since been disproven (animal colors also play a role in threatening predators and attracting mates), his work made a significant impact on our understanding of camouflage and how it could be used in war. During World War I, both the French and the German militaries relied on his book to develop designs for camouflaging their soldiers, and it became required reading for the U.S. Army’s newly launched unit of camoufleurs. Thayer’s work noted how nature “obliterates” contrast by both blending into its environment and disrupting it by using arbitrary patterns to break up outlines.

Thayer was right. Nature uses both blending and patterns to disguise itself. And it is exceedingly good at it. If you have any doubts, just watch this video of an octopus seamlessly blending in to its surroundings.

Over the last hundred-plus years, humans have looked to nature to improve our ability to camouflage ourselves. We’ve come a long way.

Of course, it’s not just the United States and our allies that have benefited from advances in camouflage. So have our adversaries. And it’s not just soldiers that use camouflage to blend into their surroundings. Facilities can also be camouflaged, as can the movement of people and equipment. These can present significant challenges to the military. How can we see what doesn’t want to be seen?

At Los Alamos National Laboratory, we study camouflage in nature to learn how we can identify things trying to disguise themselves. We do that by looking at marine organisms that are exceptionally good at the art of blending in: flounders, skates, cuttlefish, and octopi.

Take, for example, flounders. They’re not completely flat, but they appear flat—with two eyes on top of their heads. Similar to the octopus, they are able to change both the color and texture of their skin to imitate those found on the ocean floor. Identifying them is no easy task. Although animal and human vision have evolved to efficiently perceive a complex visual world by relying on cues such as coherent edges, color contrast, and texture differences, natural camouflage has evolved to frustrate these perceptions to escape detection. In the past, researchers have tried to unravel this conundrum by studying the workings of vision. In our research into the failings of vision when it comes to detecting camouflage, we’re taking a different approach by searching for clues about how visual perception works.

Working with the Woods Hole Oceanographic Institution (WHOI) and National Oceanic and Atmospheric Administration (NOAA), we’re using their autonomous underwater vehicles (AUVs) to obtain datasets representing different kinds of camouflage. The AUVs take millions of images of the ocean floor. We’ve already developed an algorithmic framework for image segmentation and shape analysis based on geometric modeling of principles of perceptual organization. The goal of this work was to develop efficient automated methods for detecting and analyzing features in remote sensing imagery for national security and intelligence applications.

This algorithmic framework has since been used in follow-on projects, beyond remote sensing, for analyzing radiographs, biomedical imagery, and marine imagery, which involved characterizing the structure and texture of marine organisms and their habitats from images obtained by WHOI. For instance, we have developed the first successful method for rapidly detecting and counting sea scallops. In working with marine imagery, we were surprised by the sensitivity of our methods in detecting subtle features, and even certain camouflage, in the presence of high clutter—a trait not shared by other image segmentation methods, which rely heavily on spectral cues. That’s promising and inspired our quest for cracking the camouflage code.

Yet still, most camouflage defeats our current detection capabilities. Our work now hopes to improve that. Rather than search for clues in the confounding world of blending and disruptive colors, we look for telltale structural cues. Indeed, color-blind people are often better at detecting camouflaged objects. This is perhaps because they rely less on colors and more on form and texture to discern the world around them.

In particular, we observe that the economy of animals’ physical forms, due to ease of motility and heat conservation, yields structural cues, such as a smoother edge or a slightly different texture from the ambient background, that hint at their possible location. These cues, which are unlikely to be just accidental, give us strong reason to look closer at the localized regions where they are found, with the help of other powerful techniques that are too expensive or slow to be applied all over a large image. Such initial cuing can also help automate the generation of large training sets required by machine-learning algorithms that can be taught to recognize camouflage.

Locating marine organisms in their natural habitats can help us not only better detect nefarious activities that could threaten national security, but better understand ocean biodiversity as well. We can use the technology to monitor fish populations and mitigate overfishing, a growing environmental concern. It can also give us clues as to how rising ocean temperatures are affecting fish populations.

When Abbott Thayer wrote his book more than a hundred years ago about how animals use their color to survive, I doubt he could have imagined all the technological advances that have made camouflage as sophisticated as it is today—and how good we’ve gotten at detecting it. Camouflage is nature’s best approximation of invisibility. Our job is to beat it at its own game.

Lakshman Prasad is a data scientist in the Intelligence and Space Research division of Los Alamos National Laboratory. His most recent paper on this topic can be found in the journal Methods of Oceanography.

This post originally appeared on Discover magazine’s blog The Crux.

Using machine learning to scan the sky

RAPTOR telescope | Los Alamos National LaboratoryLos Alamos National Laboratory’s robotic thinking telescope system, RAPTOR, checks gamma ray sources and photographs blazars as it roams the skies.

HuffPost logo

Using machine-learning to scan the sky

By Spencer Johnson

If you watch the night sky for a while, you’ll start to notice changes. Meteors streak by, the International Space Station glides over in silence, an airplane blinks overhead. Among these celestial transients, less noticeable but far more powerful objects called blazars flash on and off, in brilliant gamma-ray outbursts and flashes of visible light that can last for hours, days, or even weeks.

These flashing beacons telegraph the mind-bending processes inside some of the most energetic events in the universe, as matter swirls in the extreme conditions surrounding a supermassive black hole that is vacuuming up a galaxy’s core. Blazars, a kind of quasar, generate magnetic fields and shock waves that spew out jets of particles from their poles with super-high energies. These particles, traveling at nearly the speed of light, spray radiation across the spectrum, from radio waves to visible light to gamma rays, all aimed directly at Earth. A blazar’s gamma-ray bursts and visible light might take a few billion years to reach us, but they have a story to tell.

The information from these gamma rays and visible light are key to understanding the underlying causes of these spectacular radiation emissions and the tremendously powerful explosions that produce them. That’s what makes these astrophysical events interesting to physicists at Los Alamos National Laboratory—and why, every night in a remote clearing called Fenton Hill high in the Jemez Mountains of central New Mexico, a bank of robotically controlled telescopes tilt their lenses to the sky for another round of observations through digital imaging.

Because of blazars’ remoteness from Earth and the challenges of observing them, they remain an obscure, poorly understood, and therefore tantalizing phenomenon for astrophysicists to study. How do you decipher the inner workings of a cosmic cataclysm 3 billion light years away and 3 billion years in the past?

You start by looking at pictures from Fenton Hill. Lots of pictures—about 10,000 each week.

At this tranquil mountain meadow, far from the glare of city lights, the array of telescopes called the Rapid Telescopes for Optical Response, or RAPTOR, has for more than a decade kept an unblinking eye on the night sky, tracking and photographing the visible-light flares from blazars, with powerful computers using machine learning to process the images for analysis.

As RAPTOR makes its rounds across the sky to check on known gamma ray sources and respond to the occasional interesting transient, it has free time every night to photograph blazars. The Lab team processes the images and uses the data in conjunction with gamma ray observations of the same events made by other observatories to further refine the physics models of these fascinating active galaxies.

One task involves sifting through all the images, tossing out the ones that aren’t relevant or usable optically (some are compromised by clouds, wind gusts, focus problems, and so on), keeping the good ones, and determining their light intensity, or magnitude. It provides key information about the processes inside those blazar jets of plasma. Los Alamos has been a leader for years in developing machine-learning computer software to automate those tasks—making it possible to quickly process truly big data. As a graduate student at the Lab, my job has been to develop a new algorithm to streamline that sifting process—and for good reason.

Blazars provide a rich laboratory for studying the interactions among subatomic particles, radiation, and magnetic fields that cannot be conjured here on Earth. Because supermassive black holes produce huge quantities of highly energetic radiation, understanding their behavior as astronomical events helps physicists here at Los Alamos and elsewhere validate computer models coupling radiation and matter. These codes and physical experiments help assure the safety, security, and effectiveness of the U.S. nuclear deterrent, which is the Lab’s core mission.

When the far-reaches of the universe is your laboratory, the limits of learning are boundless.

Spencer Johnson is a post-master’s student in astrophysics working with mentor and Lab scientist Tom Vestrand in the Space and Remote Sensing group at Los Alamos National Laboratory. He plans to enter the doctoral program in high energy physics at the University of Illinois in fall 2017.

This story first appeared in Huffington Post.


Trinity Supercomputer Now Fully Operational
Designed to provide increased computational capability, the Trinity supercomputer performs 41.5 million billion calculations per second.

Eye on the Sky
Every night, a bank of robotically controlled telescopes tilt their lenses to the sky for another round of observation through digital imaging.

What's That Noise?
By listening to the acoustic signal emitted by a laboratory-created earthquake, machine learning can predict the time remaining before the fault fails.

A Biologically Realistic Computer Network
Researchers are developing computers that simulate neurons and their interconnections. Then computers can learn, interpret data, and make predictions.


    Tom Vestrand with RAPTOR-T

    Black download iconTom Vestrand poses with RAPTOR-T, four coaligned telescopes with insertable color filters.

    Black download icon Geoscientists are training computers to learn from a wide range of geologic data.


    Garrett Kenyon | Los Alamos National Laboratory

    Garrett Kenyon

    Kenyon is a computer scientist specializing in neurally inspired computing in the Information Sciences group at Los Alamos, where he studies the brain and models of neural networks on the Lab’s high-performance computers.

    Paul Johnson | Los Alamos National Laboratory

    Paul Johnson

    Johnson is a geophysicist, Los Alamos National Laboratory Fellow, a Fellow of the Acoustical Society of America, and an American Geophysical Union Fellow in the Laboratory’s Geophysics group. He applies machine learning to earthquake source mechanics.

    Lakshman Prasad | Los Alamos National Laboratory

    Lakshman Prasad

    Prasad is a data scientist in the Intelligence and Space Research division of Los Alamos National Laboratory. Using machine learning, he studies camouflage in nature to learn how we can identify things trying to disguise themselves.

    Tom Vestrand | Los Alamos National Laboratory

    Tom Vestrand

    Vestrand is in Los Alamos’ Space and Remote Sensing group. He works to develop fully autonomous “thinking telescopes” that utilize machine learning to catch gamma-ray bursts.


    Charles Poling, (505) 257-8006, cpoling@lanl.gov
    Nick Njegomir, (505) 665 9394,