Microsoft Research Webinar Series
Available on-demand. Register now.
Discovering hidden connections in art with deep, interpretable visual analogies
Image retrieval systems allow individuals to find images that are semantically similar to a query image. This serves as the backbone of reverse image search engines and many product recommendation engines. Restricting an image retrieval system to particular subsets of images can yield new insights into relationships in the visual world.
In this webinar, Microsoft Research Development Engineer Mark Hamilton presents a novel method for specializing image retrieval systems called conditional image retrieval. When applied over large art datasets in particular, conditional image retrieval provides visual analogies that bring to light hidden connections among different artists, cultures, and media. This aims to encourage a new level of engagement with creative artifacts and inspire people to imagine new works of art. Hamilton will demonstrate how conditional image retrieval systems can efficiently find shared semantics between works of vastly different media and cultural origin. He’ll also demonstrate how this approach can improve generative adversarial networks (GANs), algorithms that create novel images from scratch, by identifying where GANs fail to model the true data distribution. Finally, Hamilton will introduce a generalization of Additive Shapley Values (SHAP) for better understanding why neural networks view images as similar.
Together, you’ll learn how to:
- Build a reverse image search engine
- Generalize these engines to handle extra requirements on the retrieved images, such as a filter for a particular artistic culture or time period
- Find shared inspiration and semantics in the visual arts
- Interpret and explain a reverse image search engine
- Use conditional image retrieval to discover blind spots in GANs
Mark Hamilton is a Research Development Engineer on the Azure Cognitive Services Research team and a machine learning PhD student at the MIT Computer Science & Artificial Intelligence Laboratory. Hamilton’s research interests include information theory, computer vision, and distributed systems. Recently, Hamilton has worked on creating new types of image retrieval engines to explore the visual arts and new types of networks that are aware of their own uncertainties. He leads Microsoft ML for Apache Spark, a distributed machine learning and microservice orchestration library, and has applied this work to problems in wildlife conservation, accessibility, and art museum outreach.
*This on-demand webinar features a previously recorded Q&A session and open captioning.