imagine spending a Weekend afternoons at the art museum with friends: nodding with arms crossed, desperately searching for insightful words to say. The vast majority of paintings you walk by are instantly forgotten, but some stay in your mind. As it turns out, there’s a good chance that the painting you remember is the same one that other people drew.
There’s a scientific term for it: image memory. “Essentially, there are some internal patterns that make some content more memorable than others,” says Camilo Fosco, a computer science doctoral student at MIT and chief technology officer at Memorable AI, a startup that uses machine learning to test how attractive content is to advertisers and creators. In other words, certain works of art have “something unspeakable”—and now, a team of scientists is using artificial intelligence to figure out what it is.
In a study published earlier this month Proceedings of the National Academy of Sciences, University of Chicago researchers Trent Davis and Wilma Bainbridge have shown that the memorability of artwork is not only consistent across people, but also predictable by artificial intelligence. In an online experiment, they pulled about 4,000 paintings from the Art Institute of Chicago’s database, excluding any paintings the school marked as “enhanced” or particularly famous. Hundreds of images were viewed by more than 3,200 people, approximately 40 people per painting. Volunteers were then shown paintings they had seen and paintings they hadn’t seen and asked if they remembered them. People are really consistent — everyone tends to remember (or forget) the same images.
Using a deep learning neural network called ResMem (designed by data scientist Coen Needell as part of his master’s thesis in the Bainbridge Psychology Laboratory), the research team was able to predict the likelihood of each painting being memorable. ResMem roughly mimics how the human visual system transfers information from the retina to the cortex, first processing basic information such as edges, textures, and patterns, and then expanding to more abstract information such as object meaning. Its memory scores correlated well with scores given by people in online experiments—even though the AI knew nothing about each artwork’s cultural background, popularity, or significance.
Counterintuitively, these findings suggest that our memory for art has less to do with the subjective experience of beauty and personal meaning, and more to do with the artwork itself—which could have major implications for artists, advertisers, educators, and anyone else who wishes to keep their content in your brain. “You might think that art is a very subjective thing,” Bainbridge said, “but people are surprisingly consistent about what they remember and what they forget.”
While the online experiment was an interesting start, she continued, “It would be even more interesting if we could predict memories in the real world.” So Bainbridge, along with Davis, then an undergraduate double majoring in neuroscience and visual arts, recruited 19 more people and had them actually wander through the museum’s American Art section, like exploring with a friend. The only requirement is that they see each piece at least once. “Especially as an artist myself, I want to apply the results to the real world,” says Davis, now the lab manager. “We wanted it to be a natural and enjoyable museum experience.”
