Abstract
Context
- Image copy detection and retrieval from large databases leverage two components. First, a neural network maps an image to a vector representation, that is relatively robust to various transformations of the image. Second, an efficient but approximate similarity search algorithm trades scalability (size and speed) against quality of the search, thereby introducing a source of error.
- We improve the robustness of copy detection on an image by modifying it in an imperceptible manner before its release (like in watermarking). The goal is to push the image’s representation deep into its indexing partition.
How?
- We back-propagate a loss from the deep neural network back to the image, under perceptual constraints.
- Our experiments show that the retrieval and copy detection of activated images is significantly improved. For instance, activation improves by +40% the Recall1@1 on various image transformations, and for several popular indexing structures based on product quantization or locality sensitivity hashing.
When?
- The method could apply whenever Alice distributes images and wants to moderate versions edited and shared by Bobs over the web. For example, stock photo banks (like Shutterstock, Getty, DALL·E, etc.) that want to check that their pictures are rightfully credited. They would just distribute images activated for the index and feature extractor they use for indexing.
- Compared to watermarking, the method leverages the access to the original images, which makes it better than blind watermarking for the task of copy detection.
Poster
Slides