Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Modeling visual rhetorics for persuasive media through self-supervised learning

Guo, Meiqi (2023) Modeling visual rhetorics for persuasive media through self-supervised learning. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

Download (15MB) | Preview


This dissertation addresses the challenging task of modeling and interpreting visual rhetorics in persuasive media using computational models. The focus is on self-supervised learning methods that leverage general data without specific annotations related to persuasion.
The research begins by modeling three fundamental modes of persuasion (ethos, pathos, logos) in multimodal media, incorporating both text and images. Traditional visual recognition models struggle to predict the applied persuasion modes in images beyond their literal content. Self-supervised learning methods prove to be more effective in modeling these modes. The detection of persuasive atypicality in ad images and the interpretation of symbolism are explored as common visual rhetorics for capturing viewers’ attention and creating lasting impressions. The hypothesis that atypicality detection relies on contextual compatibility and understanding common-sense spatial relations of objects is validated through the development of self-supervised attention-based techniques. To assess the feasibility of automatically interpreting symbolism, an evaluative framework is developed. It compares the performance of language models and multi-modality models pretrained on large-scale web data. Furthermore, a re-ranking strategy is introduced to mitigate pre-training bias and significantly enhance model performance, bringing it on par with human performance in certain cases.
Overall, this dissertation presents a range of techniques that enable computational intelligence to detect, understand, and explain the underlying messages in rhetorical media. These methods leverage self-supervised learning and process large volumes of data, providing unprecedented depth and insight into the analysis of persuasive visual communication.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Guo, Meiqimeiqi.guo@pitt.eduMEG1680009-0007-2339-2704
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairHwa,
Committee MemberKovashka,
Committee MemberLitman,
Committee MemberHe,
Date: 18 September 2023
Date Type: Publication
Defense Date: 28 June 2023
Approval Date: 18 September 2023
Submission Date: 1 August 2023
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Number of Pages: 138
Institution: University of Pittsburgh
Schools and Programs: School of Computing and Information > Computer Science
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: Visual Rhetoric, Persuasion Mode, Persuasive Atypicality, Symbolism, Persuasive Media, Social Media, Advertisement Understanding, Self-supervised Learning
Date Deposited: 18 Sep 2023 14:16
Last Modified: 18 Sep 2023 14:16


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item