Tracking Transmission of Details in Paintings

In previous articles (di Lenardo et al, 2016; Seguin et al, 2016), we explored how efficient visual search engines operating not on the basis of textual metadata but directly through visual queries, could fundamen- tally change the navigation in large databases of work of arts. In the present work, we extended our search engine in order to be able to search not only for global similarity between paintings, but also for matching de- tails. This feature is of crucial importance for retriev- ing the visual genealogy of a painting, as it is often the case that one composition simply reuses a few ele- ments of other works. For instance, some workshops of the 16th century had repertoires of specific charac- ters (a peasant smoking a pipe, a couple of dancing, etc.) and anatomical parts (head poses, hands, etc.) ,that they reused in many compositions (van den Brink, 2001; Tagliaferro et al, 2009). In some cases it is possible to track the circulation of these visual pat- terns over long spatial and temporal migrations, as they are progressively copied by several generations of painters. Identifying these links permits to recon- struct the production context of a painting, and the connections between workshops and artists. In addi- tion, it permits a fine-grained study of taste evolution in the history of collections, following specific motives successfully reused in a large number of paintings. Tracking these graphical replicators is challenging as they can vary in texture and medium. For instance, a particular character or a head pose of a painting may have been copied from a drawing, an engraving or a tapestry. It is therefore important that the search for matching details still detects visual reuse even across such different media and styles. In the rest of the pa- per, we describe the matching method and discuss some results obtained using this approach.