Rehabilitation of Musculoskeletal Models Using Deep Reinforcement Learning

Abstract

Neural rehabilitation is a long and complex process that patients undergo after suffering a nervous system injury, such as stroke. These kinds of injuries usually result in brain cells death and partial loss of mobility and coordination. During rehabilitation, the patient performs a series of movements and physical exercises that promote neural plasticity, the brain’s mechanism to regenerate and make new pathways that substitute the damaged connections. Unfortunately, full recovery is almost impossible. The rehabilitation process is tailored to the patient based on the physician’s expertise, and it evolves with the patient’s needs and recovery. However, few computational models for rehabilitation have been developed. For instance, Lee et al. [1] trained a musculoskeletal model of a healthy subject using deep reinforcement learning, and then a prosthetic leg was added to simulate an injury. Results showed how the artificial neural network that controlled muscle contraction was able to adapt and learn to move with the prosthetic leg. Here we show how deep reinforcement learning can be used to control a musculoskeletal model. The algorithm is able to learn new and stable motions by maximizing the so-called reward function. The nervous system is modelled with an artificial neural network, and the deep deterministic policy gradient (DDPG) algorithm is used to train the model in a simulated environment.