In an award-winning study, researchers at Sheffield Hallam University and the University of Milano-Bicocca have found that facial expressions from artworks can be used as a valuable scientific resource alongside existing photographic databases, as they allow researchers to not only study clear-cut emotional expressions but also more complex emotional states.
Lead author of the paper, Zoe Thurtle from Sheffield Hallam, said: “For decades, psychologists have relied on photographic databases of facial expressions to study how people recognise emotion. While these resources have been highly influential, they are largely limited to posed expressions of a small number of basic emotions. Artworks, by contrast, have long depicted a far wider and more nuanced range of emotional and psychological states.
“Artists have spent centuries exploring how emotion is communicated visually through facial expression, gesture, and representation. By bringing together perspectives from fine art and psychology, our research takes an important first step towards developing more sensitive scientific approaches to understanding how people recognise and interpret complex emotional expressions and psychological states.”
The study introduced a systematic way of analysing how people interpret facial expressions in artworks. Participants viewed 116 cropped portraits and were asked to identify the emotion shown in each face, categorising them into one of six basic emotion categories; anger, disgust, fear, happiness, sadness, or surprise.
Included in the 116 images were portraits from paintings by Leonardo da Vinci, Vermeer, Caravaggio, Rembrandt, Velázquez, Rubens, Raphael, Michelangelo, and Jacques-Louis David, as well as female artists including Elisabeth Vigée Le Brun and Judith Leyster, alongside many other artists from the fifteenth to nineteenth centuries.
The results revealed a clear pattern: some faces produced strong agreement across viewers, while others generated a wide range of interpretations.
The approach developed in this study represents an early step towards building a new tool for studying facial expressions and how people interpret them. Creating a new database will be useful for further research, and of interest to other areas such as AI development, clinical psychology, psychological assessments and potentially for use as a diagnostic tool.
Lead author and final year PhD student at Sheffield Hallam University, Zoe Thurtle, received the Early Career Researcher Best Paper Prize from the journal i-Perception for the research.
Zoe said: “This work highlights the value of bringing art and science into dialogue, opening new perspectives and approaches for investigating emotion, perception, and the ways we make meaning from visual experiences. Receiving this recognition from i-Perception, a journal that values interdisciplinary research, is especially meaningful.”
The research forms part of a long-term international collaboration between Sheffield Hallam University and the University of Milano-Bicocca, which brings together expertise in cognitive psychology, visual perception, and psychology of art from both institutions (Sheffield Hallam University: Zoe Thurtle, Olga Daneyko, Anna Maria Di Betta, and Jane Morgan; The University of Milano-Bicocca: Daniele Zavagno).
