Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Measuring cognitive, socio-affective, and behavioral engagement and evaluating their influence on the intervention outcomes in a provider-guided digital health intervention

Wang, Yan (2023) Measuring cognitive, socio-affective, and behavioral engagement and evaluating their influence on the intervention outcomes in a provider-guided digital health intervention. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

[img] PDF
Restricted to University of Pittsburgh users only until 11 August 2025.

Download (5MB) | Request a Copy


Background: Understanding intervention efficacy requires measuring multi-dimensional engagement, but this concept is rarely studied comprehensively.
Purpose: This study aimed to enhance our understanding of engagement in socio-affective, cognitive, and behavioral dimensions, explore its impact on intervention outcome patient symptom controllability and assess the feasibility of predicting their socio-affective and cognitive engagement.
Methods: Data from 68 patients randomized to Nurse WRITE, an effective nurse-guided digital health intervention (DHI) were analyzed. Content analysis of message boards summarized the total count of socio-affective and cognitive engagement classes. Additionally, socio-affective engagement was measured by total word count, while cognitive engagement was assessed through average response rate to nurses’ queries. The behavioral engagement was evaluated by counting symptom care plans and plan reviews. K-means clustering categorized patients based on six engagement measures. Regression analysis explored the relationship between patient factors, engagement, and symptom controllability. Machine learning models predicted engagement classes, and the LIME technique analyzed model challenges.
Results: Details of socio-affective and cognitive engagement classes are in Appendix A Manuscript #1. Participants were clustered into high (n = 13), moderate (n = 17), and low engagers (n = 38). Details are in Appendix B Manuscript #2. Education was a significant predictor of higher engagement overall and in each dimension, which, in turn, was significantly associated with improved symptom controllability. More specifically, improved control was significantly correlated with engagement classes Intervention information, Acknowledgment, Cancer-related experience, Vocatives, and Appreciation. Our fine-tuned transformer models reliably predicted engagement, achieving mean F1 scores of 78.8 and 78.5 for socio-affective and cognitive classes, respectively. Both models had three main types of errors: context error, inaccurate weighting, and practical use of words. The socio-affective model tended to make mistakes related to practical word usage, while the cognitive model tended to make context errors.
Conclusion: Engagement measures can provide insights for behavioral scientists to evaluate patient engagement in provider-guided DHIs. Higher education predicted increased communication, emotional connection, content/process coordination, and intervention activity completion, which in turn, enhanced symptom control. Future DHIs should consider patient factors and strategies to support multi-dimensional engagement. Reliable prediction of engagement can help interventionists adjust communication styles and tailor strategies to enhance engagement.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Wang, Yanyaw75@pitt.eduyaw750000-0002-3136-3109
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairDonovan, Heidi A.S.donovanh@pitt.edudonovanh0000-0002-4889-8644
Committee MemberAlikhani, MaliheMALIHE@pitt.eduMALIHE0000-0002-1315-2228
Committee MemberCampbell, Gracecampbellg2@duq.educampbellg20000-0003-3942-0662
Committee MemberDeVito Dabbs, Annetteajdst42@pitt.eduajdst420000-0003-3325-435X
Committee MemberThomas, Teresa H.t.thomas@pitt.edut.thomas0000-0001-7403-7314
Date: 11 August 2023
Date Type: Publication
Defense Date: 19 July 2023
Approval Date: 11 August 2023
Submission Date: 9 August 2023
Access Restriction: 2 year -- Restrict access to University of Pittsburgh for a period of 2 years.
Number of Pages: 128
Institution: University of Pittsburgh
Schools and Programs: School of Nursing > Nursing
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: Engagement; Digital Health Intervention; Symptom Management; Gynecological Cancer; mHealth; eHealth
Related URLs:
Date Deposited: 11 Aug 2023 17:36
Last Modified: 11 Aug 2023 17:36


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item