Digital Provenance Research: A computer-assisted image search in auction catalogues

The project

The German Sales database is essential for provenance research, providing access to thousands of auction and sales catalogues. These offer insights into the 20th century art market, make object sales transparent and sometimes also contain information on prices, consignors and buyers. The catalogues can currently be searched in full text. Although this access provides significant added value, changing object titles or artist attributions can hinder searches. Against this background, the project presented here was launched. In a collaboration between the Department of Digital Humanities and Social Studies and the Pattern Recognition Lab at Friedrich-Alexander Universität (FAU) Erlangen-Nürnberg, tests were conducted to determine the extent to which image recognition methods can be used to find images in catalogues using images as search queries. This image-based search aims to provide an additional, complementary access to the source material.

Gemälde Ludwig von Zumbusch, 1896 - verschiedene Titel: Römische Ideallandschaft, Landschaft mit Birken und Pappeln, Einsames Land
Fig. 1 illustrates the problem of changing titles, which motivated the project.

First, a neural network for object detection is used to recognise and cut out the images in the catalogues. Figure 2 illustrates this recognition process using an exemplary catalogue page. The visual features of these image sections are then extracted and stored in a database for later use in image searches (see Fig. 3). Features are also extracted from the selected search image and compared with those stored in the database. Finally, the images whose features are most similar to those of the search image are displayed. This search method is extremely valuable for provenance research as it provides an additional, quick access to source material, independent of changing titles or artist attributions. Furthermore, the method can be extended to other sources, such as exhibition catalogues, catalogues raisonnés, art magazines or primary market sources. As well as reconstructing provenance, the method can help to answer other questions concerning, for example, the focus of individual auction houses, contemporary tastes, price developments, object movements and image similarities.

Reproduktion vierer Gemälde auf der Seite eines Auktionskatalogs
Abb. 2: Images detected by the detection network..
Flussdiagramm
Fig. 3: The current workflow and its individual steps. Not shown here is the retrieval based on the extracted features (search image and image collection).

Project status

A database of 111 catalogues is currently being used, containing more than 3,500 recognised and extracted images. The images were detected using a YOLOv8 recognition network, which was trained using 148 annotated catalogue pages. Feature extraction was performed using a ResNet50 architecture trained on ImageNet. Further information on the method can be found in our article presented at the Digital Humanities Conference in German-speaking countries in 2025.

Future work

We are currently continuing to work on the project, focusing primarily on expanding the dataset, hosting the search interface, and searching for 3D objects and multimodal searches. In addition, steps one and two of the workflow, namely the acquisition of the catalogues, will be carried out via the Heidelberg University Library interface.

The project team

Sabine Lang is a research assistant in the Department of Digital Humanities and Social Studies (DHSS) at FAU Erlangen-Nürnberg. She specialises in the application of digital methods in art history and provenance research, with a particular focus on image and object recognition, as well as the documentation and representation of provenance gaps in the digital realm. Prior to joining the DHSS, Sabine completed a PhD in art history, undertook a traineeship at an auction house, and worked as a freelance provenance researcher.

Mathias Zinnen is a research assistant at the Chair of Pattern Recognition at FAU Erlangen-Nürnberg. He studied philosophy, history and computer science in Mainz, Berlin and Erlangen. From 2021 to 2024, he was part of the image recognition team of the Odeuropa project. In this role, he focused on the automatic extraction of odour references from historical paintings using computer vision. As part of the SODa project, he currently advises collection managers on the application of machine vision methods in research with collection data. His research focuses on object recognition, pose estimation and text recognition.

Publications

Presentations and lectures

Link to our demonstration

We welcome your feedback! If you are working on a similar project, we would love to hear from you: sab.lang@fau.de or mathias.zinnen@fau.de.