Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Syntharch: interactive image search with attribute-conditioned synthesis

Yu, Zac (2019) Syntharch: interactive image search with attribute-conditioned synthesis. Undergraduate Thesis, University of Pittsburgh. (Unpublished)

[img]
Preview
PDF
Download (2MB) | Preview

Abstract

The use of interactive systems has been proposed and found to be a promising approach for content-based image retrieval, the task of retrieving a specific image from a database based on its content. These systems allow the user to refine the set of results iteratively until the target is reached. In order to proceed with the search efficiently, conventional methods rely on some shared knowledge between the user and the system, such as semantic visual attributes of the images. Those approaches demand the images to be semantically labeled and introduce a new semantic gap between the two parties’ understanding. In my thesis, I explore an alternative approach to interactive image search where feedback is elicited exclusively in visual forms, therefore eliminating the semantic gap and allowing for a generalized version of the method to operate on unlabeled databases.
Thanks to the recent advancements in generative adversarial networks, we can now generate realistic images of certain controlled characteristics and use a multidimensional attribute space learned from an image database to condition image synthesis. I present Syntharch, a novel interactive image search approach which uses synthesized images as options instead of textual questions to gain information on the relative attribute values of the target image. For each iteration of the search, rather than asking the user to make an attribute-value comparison in words, Syntharch generates a pair of options (synthesized images) which varies only in one attribute and let the user select the option that is more visually similar to the target.
I then demonstrate that using synthesized images rather than real images retrieved from the database as feedback options, Syntharch causes less confusion to the user. Further, I establish that the specific search method I propose performs similarly or better in comparison to the conventional approach.
Overall, my thesis presents a new approach of interactive image search, proposes a specific implementation following that approach, and validates the hypotheses that guided the search approach as well as the implementation choices.


Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: University of Pittsburgh ETD
Status: Unpublished
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Yu, Zaczac.yu@pitt.eduZHY460000-0002-2452-5034
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairKovashka, Adrianakovashka@cs.pitt.eduKOVASHKA0000-0003-1901-9660
Committee MemberHe, Daqingdah44@pitt.eduDAH440000-0002-4645-8696
Committee MemberWalker, Erineawalker@pitt.eduEAWALKER
Committee MemberGoel, Mayankmayankgoel@cmu.edu0000-0002-7606-7282
Date: 28 August 2019
Date Type: Publication
Defense Date: 28 May 2019
Approval Date: 28 August 2019
Submission Date: 6 June 2019
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Number of Pages: 52
Institution: University of Pittsburgh
Schools and Programs: University Honors College
Dietrich School of Arts and Sciences > Computer Science
Degree: BPhil - Bachelor of Philosophy
Thesis Type: Undergraduate Thesis
Refereed: Yes
Uncontrolled Keywords: Content-Based Image Retrieval, CBIR, Interactive Image Search, Generative Adversarial Network, GAN, Image Synthesis, Image Editing, Computer Vision, Human-Computer Interaction
Date Deposited: 28 Aug 2019 19:17
Last Modified: 28 Aug 2019 19:17
URI: http://d-scholarship.pitt.edu/id/eprint/36898

Metrics

Monthly Views for the past 3 years

Plum Analytics


Actions (login required)

View Item View Item