Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Transparency and Explanation in Deep Reinforcement Learning Neural Networks

Iyer, Rahul and Li, Yuezhang and Li, Huao and Lewis, Michael and Sundar, R. and Sycara, Katia P. (2018) Transparency and Explanation in Deep Reinforcement Learning Neural Networks. In: AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, 2-3 Feb 2018, New Orleans, LA.

Accepted Version

Download (479kB) | Preview


Autonomous AI systems will be entering human society in the near future to provide services and work alongside humans. For those systems to be accepted and trusted, the users should be able to understand the reasoning process of the system, i.e. the system should be transparent. System transparency enables humans to form coherent explanations of the systems decisions and actions. Transparency is important not only for user trust, but also for software debugging and certification. In recent years, Deep Neural Networks have made great advances in multiple application areas. However, deep neural networks are opaque. In this paper, we report on work in transparency in Deep Reinforcement Learning Networks (DRLN). Such networks have been extremely successful in accurately learning action control in image input domains, such as Atari games. In this paper, we propose a novel and general method that (a) incorporates explicit object recognition processing into deep reinforcement learning models, (b) forms the basis for the development of object saliency maps, to provide visualization of internal states of DRLNs, thus enabling the formation of explanations and (c) can be incorporated in any existing deep reinforcement learning framework. We present computational results and human experiments to evaluate our approach.


Social Networking:
Share |


Item Type: Conference or Workshop Item (Paper)
Status: Published
CreatorsEmailPitt UsernameORCID
Iyer, Rahul
Li, Yuezhang
Li, Huao
Lewis, Michaelml@sis.pitt.educmlewis0000-0002-1013-9482
Sundar, R.
Sycara, Katia P.
Date: February 2018
Date Type: Publication
Journal or Publication Title: Proceedings of AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society
Publisher: AAAI/ACM
Event Title: AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society
Event Dates: 2-3 Feb 2018
Event Type: Conference
Schools and Programs: School of Computing and Information > Information Science
Refereed: Yes
Date Deposited: 06 Jul 2018 16:05
Last Modified: 26 Jul 2022 19:03


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item