Iyer, Rahul and Li, Yuezhang and Li, Huao and Lewis, Michael and Sundar, R. and Sycara, Katia P.
(2018)
Transparency and Explanation in Deep Reinforcement Learning Neural Networks.
In: AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, 2-3 Feb 2018, New Orleans, LA.
Abstract
Autonomous AI systems will be entering human society in the near future to provide services and work alongside humans. For those systems to be accepted and trusted, the users should be able to understand the reasoning process of the system, i.e. the system should be transparent. System transparency enables humans to form coherent explanations of the systems decisions and actions. Transparency is important not only for user trust, but also for software debugging and certification. In recent years, Deep Neural Networks have made great advances in multiple application areas. However, deep neural networks are opaque. In this paper, we report on work in transparency in Deep Reinforcement Learning Networks (DRLN). Such networks have been extremely successful in accurately learning action control in image input domains, such as Atari games. In this paper, we propose a novel and general method that (a) incorporates explicit object recognition processing into deep reinforcement learning models, (b) forms the basis for the development of object saliency maps, to provide visualization of internal states of DRLNs, thus enabling the formation of explanations and (c) can be incorporated in any existing deep reinforcement learning framework. We present computational results and human experiments to evaluate our approach.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
Conference or Workshop Item
(Paper)
|
| Status: |
Published |
| Creators/Authors: |
|
| Date: |
February 2018 |
| Date Type: |
Publication |
| Journal or Publication Title: |
Proceedings of AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society |
| Publisher: |
AAAI/ACM |
| Event Title: |
AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society |
| Event Dates: |
2-3 Feb 2018 |
| Event Type: |
Conference |
| Schools and Programs: |
School of Computing and Information > Information Science |
| Refereed: |
Yes |
| Date Deposited: |
06 Jul 2018 16:05 |
| Last Modified: |
26 Jul 2022 19:03 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/34581 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}