Freiburg, Berlin – July 23, 2021

Tracking the traces of light in the short-lived memory of the eyes

The traces light leaves on the retina close gaps in visual perception and provide continuity in knowing what is where as we move.

Bild: M. Rolfs

Bernstein member involved: Martin Rolfs

/BN/ What we see leaves traces. In rapid movements, so-called saccades, our gaze jumps from one location to the next to project what is happening in front of our eyes as sharply as possible onto the central, high-acuity part of the retina, the fovea. With less than 50 milliseconds duration, saccades are the fastest movements a human being is capable of. As we look over to an object, it shifts from the corner of the eye to the center of our field of view, leaving a short-lived trace of light on the retina in the process. This is similar to the motion blur that occurs when panning a film camera.

Traces on the retina

For a long time, scientists have assumed that we are temporarily blind during rapid eye movements. A new study by Berlin researchers Richard Schweitzer and Martin Rolfs now proved this long-standing assumption wrong. Instead, information is processed throughout the rapid movement of the eyes, giving rise to a continuity of objects. This can be compared to following the traces of several hikers digitally vs analogously. If their coordinates are collected digitally, that is, only at discrete moments, the hikers’ paths cannot be distinguished from each other, quickly creating ambiguity about who has moved where and how fast. However, if we follow each individual’s tracks continuously, they can be tracked without fail.

Such continuous traces also emerge in the eye when we change the direction of our gaze. The light that is reflected from objects flits across the retina. Schweitzer and Rolfs have studied these light traces systematically. They reconstructed how objects move across the retina over the short duration of a saccade and then altered the traces that these movements leave in their experiments. They wanted to understand whether these traces help us connect successive impressions of the visual world. That is, whether it is possible to track objects continuously throughout eye movements, to recognize and act upon them more quickly when the eyes land.

Their new paper in Science Advances reveals that visual information processing is continuously taking place during the saccade; it is not interrupted and resumed once the movement is over, as previously assumed, which would require the matching of two images. Schweizer and Rolfs’ results show that the visual system actively uses the fleeting traces of light created by eye movements to track object positions through time.

To prove this, the two researchers conducted experiments in which study participants were asked to look at a predefined target in an array of objects. Using high-speed projection — which only few laboratories around the world are equipped to do — they were able to continuously change the scenery as the participant’s eyes were in flight towards the target object (i.e., in the few milliseconds of a saccade). By accurately measuring the movements of the eyes at the same time, they could translate the objects’ motion on the screen into the visual traces they must have left on the retina. And indeed, these traces influenced how fast and how often the test persons turned their gaze to the objects that created them.

New Research Approaches – New Opportunities

Schweitzer and Rolfs’ work refutes the long-standing research paradigm that assumed that visual processing is interrupted with periods of blindness during each saccade. Their findings provide a new approach to the nature of human visual perception and simplifies common theories which assume that gaps in visual perception must be filled by complex predictive processes.

Their findings can also be translated into clinical applications, especially with respect to the connection between visual perception and motor function, and psychological impairments that result from the disruption of this connection.

In industrial applications, the results may also foster further development of computer vision algorithms, such as the “foveated rendering” in VR glasses, where only those parts of the image are displayed at high resolution that a user is currently looking at. The results of Schweitzer and Rolfs allow specific predictions about which changes of the rendering during eye movements are suitable to enable uninterrupted perception.