Guo, Meiqi
(2023)
Modeling visual rhetorics for persuasive media through self-supervised learning.
Doctoral Dissertation, University of Pittsburgh.
(Unpublished)
Abstract
This dissertation addresses the challenging task of modeling and interpreting visual rhetorics in persuasive media using computational models. The focus is on self-supervised learning methods that leverage general data without specific annotations related to persuasion.
The research begins by modeling three fundamental modes of persuasion (ethos, pathos, logos) in multimodal media, incorporating both text and images. Traditional visual recognition models struggle to predict the applied persuasion modes in images beyond their literal content. Self-supervised learning methods prove to be more effective in modeling these modes. The detection of persuasive atypicality in ad images and the interpretation of symbolism are explored as common visual rhetorics for capturing viewers’ attention and creating lasting impressions. The hypothesis that atypicality detection relies on contextual compatibility and understanding common-sense spatial relations of objects is validated through the development of self-supervised attention-based techniques. To assess the feasibility of automatically interpreting symbolism, an evaluative framework is developed. It compares the performance of language models and multi-modality models pretrained on large-scale web data. Furthermore, a re-ranking strategy is introduced to mitigate pre-training bias and significantly enhance model performance, bringing it on par with human performance in certain cases.
Overall, this dissertation presents a range of techniques that enable computational intelligence to detect, understand, and explain the underlying messages in rhetorical media. These methods leverage self-supervised learning and process large volumes of data, providing unprecedented depth and insight into the analysis of persuasive visual communication.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
|
| Date: |
18 September 2023 |
| Date Type: |
Publication |
| Defense Date: |
28 June 2023 |
| Approval Date: |
18 September 2023 |
| Submission Date: |
1 August 2023 |
| Access Restriction: |
No restriction; Release the ETD for access worldwide immediately. |
| Number of Pages: |
138 |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
School of Computing and Information > Computer Science |
| Degree: |
PhD - Doctor of Philosophy |
| Thesis Type: |
Doctoral Dissertation |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
Visual Rhetoric, Persuasion Mode, Persuasive Atypicality, Symbolism, Persuasive Media, Social Media, Advertisement Understanding, Self-supervised Learning |
| Date Deposited: |
18 Sep 2023 14:16 |
| Last Modified: |
18 Sep 2023 14:16 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/45148 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}