Towards On-device Machine Learning: Efficient Inference, Independent Learning, and Collaborative Unsupervised LearningWu, Yawen (2023) Towards On-device Machine Learning: Efficient Inference, Independent Learning, and Collaborative Unsupervised Learning. Doctoral Dissertation, University of Pittsburgh. (Unpublished)
AbstractWith the increasing ubiquity of edge devices, such as the Internet of Things (IoT) and mobile devices, deploying machine learning models, particularly deep neural networks (DNNs), on these devices to extract information from sensed data can enable the democratization of artificial intelligence (AI). On-device AI has the potential to support various tasks, including wildlife monitoring, augmented reality, and autonomous driving. To enable on-device AI, both on-device inference and training need to be achieved. On-device inference enables edge devices to make predictions from collected data. On-device training enables the devices to adapt to dynamic environments by learning from the environments and updating the model in situ. By applying on-device training to distributed devices, collaborative learning uses multiple devices to learn a shared model while keeping the data on personal devices for privacy. However, it is challenging to achieve on-device inference and training. First, edge devices have limited computation capabilities and limited memory size, but DNNs are demanding in computation and memory. Therefore, DNNs need to be effectively compressed before deploying to edge devices. Second, there is a large gap between the high computation and energy demand of on-device training and the limited computing resources and battery capacity on edge devices. Third, during on-device training, each device can only collect a limited amount of data. However, model training needs a large amount of data to achieve a high generalization performance. To address these challenges, this dissertation proposes several techniques to enable inference and training on single devices, and collaborative learning with multiple devices. First, a model compression framework is proposed to compress multi-exit neural networks by pruning and quantization. After compression, computation cost and model size are reduced while the accuracy is preserved. Second, an efficient training method is proposed to reduce the computation cost of training by skipping unnecessary training data and pruning the gradient computation. To further learn with as few labels as possible, a data selection approach to select the representative data for training without using labels is proposed. Third, a collaborative unsupervised learning framework for distributed devices to learn a shared model from decentralized unlabeled data is proposed. Share
Details
MetricsMonthly Views for the past 3 yearsPlum AnalyticsActions (login required)
|
![[feed]](/images/feed-icon-32x32.png){kind=icon}