Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Blending of brain-machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping

Downey, John E. and Weiss, Jeffrey M. and Muelling, Katharina and Venkatraman, Arun and Valois, Jean-Sebastien and Hebert, Martial and Bagnell, J. Andrew and Schwartz, Andrew B. and Collinger, Jennifer L. (2016) Blending of brain-machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping. Journal of NeuroEngineering and Rehabilitation, 13 (1). ISSN 1743-0003

[img] PDF
Published Version

Download (1MB)


Recent studies have shown that brain-machine interfaces (BMIs) offer great potential for restoring upper limb function. However, grasping objects is a complicated task and the signals extracted from the brain may not always be capable of driving these movements reliably. Vision-guided robotic assistance is one possible way to improve BMI performance. We describe a method of shared control where the user controls a prosthetic arm using a BMI and receives assistance with positioning the hand when it approaches an object.
Two human subjects with tetraplegia used a robotic arm to complete object transport tasks with and without shared control. The shared control system was designed to provide a balance between BMI-derived intention and computer assistance. An autonomous robotic grasping system identified and tracked objects and defined stable grasp positions for these objects. The system identified when the user intended to interact with an object based on the BMI-controlled movements of the robotic arm. Using shared control, BMI controlled movements and autonomous grasping commands were blended to ensure secure grasps.
Both subjects were more successful on object transfer tasks when using shared control compared to BMI control alone. Movements made using shared control were more accurate, more efficient, and less difficult. One participant attempted a task with multiple objects and successfully lifted one of two closely spaced objects in 92 % of trials, demonstrating the potential for users to accurately execute their intention while using shared control.
Integration of BMI control with vision-guided robotic assistance led to improved performance on object transfer tasks. Providing assistance while maintaining generalizability will make BMI systems more attractive to potential users.


Social Networking:
Share |


Item Type: Article
Status: Published
CreatorsEmailPitt UsernameORCID
Downey, John E.
Weiss, Jeffrey
Muelling, Katharina
Venkatraman, Arun
Valois, Jean-Sebastien
Hebert, Martial
Bagnell, J. Andrew
Schwartz, Andrew
Collinger, Jennifer
Date: 18 March 2016
Date Type: Publication
Journal or Publication Title: Journal of NeuroEngineering and Rehabilitation
Volume: 13
Number: 1
Publisher: BMC ( part of Springer)
DOI or Unique Handle: 10.1186/s12984-016-0134-9
Schools and Programs: School of Medicine > Physical Medicine and Rehabilitation
Refereed: No
ISSN: 1743-0003
Official URL:
Article Type: Research Article
Date Deposited: 13 May 2020 15:23
Last Modified: 13 May 2020 15:23


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item