Bernstein member involved: Sen Cheng

Do you remember your eighteenth birthday? What food was served? Which song did you dance to? Who was invited? And — are you completely sure? Research findings repeatedly show that our memories are not as reliable as we often assume. Perhaps your recollections of that birthday are partly based on stories you’ve heard or photographs you’ve seen. And maybe, just that year, your favorite apple cake wasn’t served.

How do we manage to remember past experiences? And how reliable is our memory? These questions have fascinated humanity since antiquity, and they are also an active area of research at Ruhr University Bochum. Since 2010, computational neuroscientist Professor Sen Cheng and philosopher Professor Markus Werning have been collaborating across disciplines to uncover the mechanisms underlying human memory.

“Since antiquity, memory has often been conceived of as a kind of storage system that encodes experiences the moment they occur and allows them to be retrieved later,” explains Sen Cheng. Over time, this storage metaphor has evolved alongside technological developments: “Ancient thinkers imagined memory as a wax tablet, into which information — memories — was inscribed with a stylus,” Cheng says. Today, many assume that the brain functions like a hard drive. “However, from a cognitive psychology perspective, this storage model — the idea that the brain stores files like a computer — is incorrect,” Cheng notes. Our brain —more specifically, what both researchers refer to as episodic memory — is neither precise nor reliable. “Even though we tend to believe it is, and many would strongly swear that it is,” Cheng adds.

The inadequacy of the storage model has been demonstrated repeatedly; for example, in studies on so-called flashbulb memories. These are memories formed during highly emotional events, such as the attacks on the World Trade Center or the Challenger disaster. “At the time, it was assumed that such events are imprinted on memory like a flash of lightning. However, numerous studies have shown that this is not the case,” says Cheng. In one example, participants were interviewed shortly after September 11 and again one year later. The result: their recollections had already changed significantly within that year.

What do we consciously or unconsciously add to our memories? Where does memory end and imagination begin? These questions also concern philosophers such as Markus Werning. In philosophy, memory research is a relatively young field, originating from a thought experiment proposed in 1966 by Charles Martin and Max Deutscher — the oft-cited Painter Case. “Imagine a painter who is commissioned to paint a garden,” Werning explains. The painter believes he is inventing a fictional garden, but it later turns out that the painting closely resembles a garden he saw as a child. “This phenomenon appears frequently in art history,” says Werning.

Did van Gogh borrow from Hokusai?

Art historians are currently debating whether, in 1889, Vincent van Gogh, while painting his famous work The Starry Night, was consciously or unconsciously inspired by Katsushika Hokusai’s iconic woodblock print The Great Wave off Kanagawa (1830–1836). Both works share striking similarities in their swirling forms and blue color palette. “It is known that around 1800, Japanese artists imported the pigment Prussian blue from Europe for woodblock printing,” Werning explains. Van Gogh later became an admirer of Japanese art and was familiar with Hokusai’s work. But did he know this specific print? Did he consciously appropriate the idea? Or did the memory of it unconsciously find its way into his mind?

“We cannot know whether this was a conscious or unconscious appropriation. What matters is that it is entirely possible that he did not consciously ‘borrow’ the idea. Memory research supports this,” says Cheng. “Our memory continuously incorporates new elements. We do many things based on past experiences without being consciously aware of them,” adds Werning. “Things we have only heard, or actions we have merely observed — later we may remember them as things we ourselves said or did. It is therefore not far-fetched that someone paints a picture believing it to be entirely original,” Cheng concludes.

Experienced or only read aloud?

Cheng has experienced this phenomenon himself. When his daughters were young, he often read them a story about a bear taking its cub to the dentist. Years later, he recounted the story as if it were his own lived experience. His daughters corrected him: “You didn’t experience that, Papa — you read it to us.” “I had appropriated the story as my own,” he admits with a laugh. A typical example of a false memory. “At the moment I told it, I was absolutely convinced that it had really happened.”

Cheng and Werning are working together on an explanation for this and other kinds of false memories. Their theory — the scenario construction model — stands in stark contrast to the storage model described earlier, as well as to the classical causal theory proposed by Martin and Deutscher, which assumes that a causal chain , the memory trace, links a stored experience to a later memory recall.

“Our idea is that an episodic memory always involves the construction of a scenario,” Cheng explains. “This scenario is built from the episodic memory trace — that is, details of a specific experience — as well as semantic information.” By semantic information, the researchers mean generic information, statistical regularities, and world knowledge. “We argue that the brain does not store faithful representations of reality. Instead, the hippocampus retains only very small fragments of information, neural traces, which are then combined with semantic information stored in the neocortex,” adds Werning.

But how can we tell whether a memory is based on an actual experience or on semantic information and probabilities? How can false memories be identified? To investigate this, Cheng joined a collaboration led by Prof. Dr. Oliver Wolf to conduct virtual reality experiments in which participants explore a virtual apartment consisting of three rooms: a bathroom, a living room, and a kitchen. In the bathroom, there is a toaster. After exploring the environment, participants are questioned about what they saw. In around 35 percent of cases, participants recalled the toaster in the bathroom — based on episodic memory rather than semantic knowledge or guessing, according to Cheng.

How can he be so certain? “Participants who made errors were roughly equally likely to place the toaster in the kitchen — the location where one would typically expect to find it—and only rarely in the living room,” he explains.

Cheng and Werning plan to use this virtual paradigm to address further research questions. For example: how does memory differ between younger and older individuals? Who is more susceptible to the pitfalls of memory? How does stress affect memory performance?

“Studies suggest that the process of scenario construction is indeed influenced by age and stress,” says Werning. “Our hypothesis is that older adults rely more on semantic information when remembering events or making decisions, whereas younger individuals recall episodic details more effectively and base their decisions on them,” Cheng summarizes. Stress also likely has a strong impact on memory. “We assume that stress leads people to rely more on statistical patterns and probabilities rather than on specific, personally experienced details,” Werning explains. “Stressful situations may limit memory retrieval, leading to more schematic processing. This is something we want to investigate further,” adds Cheng.

These findings could have far-reaching implications for people who must make decisions or provide statements under pressure, such as physicians, pilots, or eyewitnesses in court. The researchers emphasize that caution is particularly warranted when evaluating eyewitness testimony. Previous studies have shown that witnesses under stress are more likely to form inaccurate memories. “In one case in the United States, a witness reported that a robber was carrying a gun. In reality, surveillance footage showed a baseball bat. A gun simply seemed more plausible to the witness — a clear case of semantic construction,” Werning explains. It has also been disproven that highly detailed accounts are necessarily more reliable or truthful. “Even someone relying on schematic knowledge can provide very detailed descriptions. These are not necessarily less detailed than genuine episodic memories,” says Cheng. “The brain draws on vast resources,” Werning adds.

In their view, remembering is — and remains — a constructive process. The Bochum researchers continue to investigate it.

Translated into English by Elena Reiriz Martínez/BCOS, with the help of the research team.