Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

An Exploration of Transformer and Convolution Layers in Medical Image Segmentation

Fu, Xiyao (2023) An Exploration of Transformer and Convolution Layers in Medical Image Segmentation. Master's Thesis, University of Pittsburgh. (Unpublished)

Download (2MB) | Preview


Deep convolutional neural networks (DCNNs) are a popular deep learning technique that has been widely used in segmentation tasks and has received positive feedback. However, DCNN-based frameworks are known to be inadequate in dealing with global relations within imaging features when it comes to segmentation tasks. While several techniques have been proposed to enhance the global reasoning of DCNN, these models are either unable to achieve satisfactory performance compared to traditional fully-convolutional structures or unable to utilize the fundamental advantages of CNN-based networks, namely the ability of local reasoning. In this study, we designed a novel attention mechanism for 3D computation and used it to fully extract the self-attention ability. We proposed a new segmentation framework (called 3DTU) for three-dimensional medical image segmentation tasks, which processes images in an end-to-end manner and performs 3D computation on both the encoder side (with a 3D transformer) and the decoder side (based on a 3D DCNN). In comparison to existing attempts to combine FCNs and global reasoning methods, our framework outperforms several state-of-the-art segmentation methods on two independent datasets consisting of 3D MRI and CT images, as evidenced by experimental results.


Social Networking:
Share |


Item Type: University of Pittsburgh ETD
Status: Unpublished
CreatorsEmailPitt UsernameORCID
Fu, Xiyaoxy_fu@pitt.eduxif39
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Thesis AdvisorZhan,
Committee MemberMao, Zhi-Hongmaozh@engr.pitt.eduzhm4
Committee MemberDallal, Ahmedahd12@pitt.eduahd12
Date: 14 September 2023
Date Type: Publication
Defense Date: 27 April 2023
Approval Date: 14 September 2023
Submission Date: 27 July 2023
Access Restriction: No restriction; Release the ETD for access worldwide immediately.
Number of Pages: 35
Institution: University of Pittsburgh
Schools and Programs: Swanson School of Engineering > Electrical and Computer Engineering
Degree: MS - Master of Science
Thesis Type: Master's Thesis
Refereed: Yes
Uncontrolled Keywords: Semantic Segmentation; COVID; Lung; Placenta; Transformer; 3D UNet; CT; MRI
Date Deposited: 14 Sep 2023 13:44
Last Modified: 14 Sep 2023 13:44


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item