Current Directions in Biomedical Engineering 2017; 3(2): 161–165
Christina Salchow*, Andreas Dorn, Markus Valtin and Thomas Schauer
Intention recognition for FES in a grasp-and-
release task using volitional EMG and inertial
sensors
Abstract: Functional Electrical Stimulation (FES) facilitates
the motor recovery of the hand function after stroke. The
integration of biofeedback and other strategies to actively in-
volve a patient in the therapy is important for the rehabili-
tation progress. We introduce a combined control approach
for a FES-driven neuroprosthesis using volitional
electromyo-graphy (vEMG) and motion capturing via a novel
inertial sensor network for patients that still possess a
residual activity in the paralyzed muscles. A real-time vEMG
measurement and signal processing in between stimulation
pulses has been realized during active FES. Experiments
showed that our system allows for quick adaption to
individual users.
Keywords: Functional electrical stimulation, volitional
EMG, inertial sensors, hand neuroprosthesis, grasping
https://doi.org/10.1515/cdbme-2017-0034
1 Introduction
Stroke is a major cause for disability in adulthood in
developed countries. According to the World Heart
Federation (2016), nearly 5 million stroke survivors world-
wide are left perma-nently disabled or paralyzed. Many
patients still possess a residual activity in the paralyzed
muscles. Functional Electrical Stimulation (FES) is a
beneficial treatment modality used in therapy to facilitate the
motor recovery of disabled limbs after stroke. The
application at the forearm with surface electrode arrays
allows for generating complex hand movements such as
grasping of objects or pointing [1]. Studies revealed that a
synchronized biofeedback maximizes the benefits of FES
therapy [2].
Several methods have been introduced for detecting the
patient’s intention in FES applications. They range from
ordinary triggering of the FES via push button, complex
motion capturing via optical systems or inertial sensors, to
brain computer interfaces [3,4]. Simple methods lack
sufficient involvement and instinctiveness, whereas physio-
logical approaches usually require high costs regarding
adaption and training with the individual patient. A popular
physiological approach is the detection and enhancement of
the surface electromyogram (EMG) of the remaining muscle
activity [5].
Schauer et al. [6] measured the volitional EMG (vEMG)
before and during electrical stimulation and realized a
vEMG-proportional control of the stimulation intensity for a
wrist extension. The restricted control of the remaining
volitional muscle activity led to oscillations in the
stimulation. Salbert et al. [7] developed an EMG-triggered
state machine, which enabled the user to control the motion
sequence of hand opening and closing via vEMG-
measurements of the hand extensors and flexors. However,
first experiments with patients revealed that the assumption
of a higher vEMG in the hand extensors compared to flexors
during the attempt of hand opening and vice versa did not
always hold due to a strong co-activation of muscle groups.
Besides, different electrode placement, shifting contact
resistances, as well as measure-ment noise led to a varying
vEMG quality in each trial.
We present a novel method for the control of a grasp-
and-release task with a hand neuroprosthesis. The vEMG of
the stimulated muscles and motion capturing via inertial
sensors were combined for intention recognition. Patients,
who still possess a residual activity in the paralyzed arm,
should be able to control the stimulation on- and offset of
three stimulation channels. Our goals were (1) to gain a high
robustness of intention detection for patients, and (2) to
provide a setup with a short adaption time, which can be used
in clinical practice. We achieved this by implementing a state
machine with adjustable conditions for the transition events
that rely on adaptive thresholds for the decision making. In
______
*Corresponding author: Christina Salchow: Control Systems
Group, TU Berlin, D-10587 Berlin, Germany, e-mail:
salchow@control.tu-berlin.de
Andreas Dorn, Markus Valtin, Thomas Schauer: Control
Systems Group, TU Berlin, D-10587 Berlin, Germany, e-mail:
dorn-/valtin-/schauer@control.tu-berlin.de
162
this paper, we introduce the concept of our method and show
preliminary results with healthy volunteers and stroke
patients.
2 Methods
2.1 Experimental setup
Central elements of our hand neuroprosthesis are the recently
introduced RehaMovePro stimulator (Hasomed GmbH,
Germany) with science adapter and de-multiplexer [8], the
StiMyo II EMG measurement unit (TU Berlin, Control Sys-
tems Group), as well as a lately presented hand sensor system
[9]. We applied FES via two customized electrode arrays (see
Figure 1): one with 35 elements placed above the wrist and
finger extensor muscles (array E), and one with 24 elements
placed above the finger flexors (array F). A single hydro-gel
layer (AG702, Axelgaard Manufacturing Co., Ltd., USA)
was used to attach the array electrodes. Bi-phasic pulses with
constant pulse width were applied at 25 Hz using the current
amplitude as adjustable stimulation intensity.
We measured EMG of the extensors (EMGE) and flexors
(EMGF) by the two channels of the StiMyo II unit via
separate EMG electrodes (cf. Figure 1) at 4 kHz. The
synchronization between the StiMyo II and the
RehaMovePro allows the blanking of the stimulation pulse
artefacts in the EMG signal. To extract the vEMG (vEMGE,
vEMGF) from the measurement in real-time, the EMG signal
was digitally high-pass filtered at 200 Hz and smoothened by
a moving average filter with a cut-off frequency of ≈ 2.2 Hz.
The hand sensor system consists of a wireless IMU loca-
ted on the dorsal side of the forearm, a base unit, which is
attached to the back of the hand, and two sensor stripes on
the index and middle finger. Each sensor stripe contains three
single IMUs, one placed on each finger segment (cf. Figure
1) [9]. In total, the system measures ten joint angles at 100
Hz. The extension (negative) and flexion (positive) angle of
the metacarpal-phalangael (MCP) joint of the index (α1) and
middle (α2) finger, as well as the wrist extension/flexion
angle (β) proved to offer the best information regarding
intention.
2.2 Control of a grasp-and-release task
In general, the extraction of the vEMG via the two EMG-
channels allows the distinction of hand closing (grasping)
and hand opening (release). However, in stroke patients, high
co-activation levels between the extensor and flexor muscle
groups might obstruct a classification, e.g. due to the
presence of spasticity. Therefore, we combined the vEMG
with the hand joint angles to classify the patient’s intention.
A grasp-and-release task can be interpreted as a pre-set
sequence of movements. Similar to [7], we designed an
event-triggered state machine (see Figure 2). It consists of
four states: (1) rest, (2) hand opening, (3) grasp, and (4)
release. During state (1), the stimulation is turned off. In state
(2) and (4), the hand and finger extensor muscles are
stimulated above motor threshold (stimE,1), whereas in state
(3) FES above motor threshold is applied to the finger flexors
(stimF) and the extensors for stabilizing the wrist (stimE,2).
Via vEMG, wrist, and finger movements, the patient controls
the onset and the duration of each state and thereby the
Figure 2: vEMG and motion triggered state machine for a grasp-
and-release task.
Figure 1: Experimental setup on the left forearm.
timing of the stimulation. The stimulation channels are
always at least stimulated below motor threshold during state
(2) – (4), so that both EMG channels are effected equally by
the active electrical stimulation. The stimulation intensity
profile as well as the array element configuration for the three
stimulation positions (hand opening, wrist stabilization, and
grasp) must be preselected by the therapist.
Table 1: Possible conditions for the events of the state machine.
2.3 Adaptive classification
To provide a flexible system, which can be easily adapted to
the patient’s capabilities, we utilized adaptive thresholds and
developed a various number of conditions for the state
machine as listed in Table 1. By linking the conditions
regarding vEMG and angular motion with either ‘AND’ or
‘OR’, as well as by deleting conditions, the events and
thereby the behaviour of the state machine can be changed.
The event ‘relax’ can be deactivated completely, which might
be necessary for patients with spasticity, as they might not
able to relax their forearm muscles completely. This flexible
framework was realized to enable an adaption to individual
requirements for a patient.
The angular velocities of finger and wrist joints were
utilized to detect intention in hand motion. In comparison
with joint angles, angular velocities offer the benefit to apply
thresholds that are independent of the current hand posture.
We defined three constant but adjustable thresholds
,
,
and
for the signed angular velocities 1, 2 and . Default
values were obtained heuristically and were set to 0.7 °/ms
for
and
, and 1°/ms for
.
Three adaptive thresholds for the vEMG were used to
classify whether the patient wants to perform a movement or
not (relax). sE is designated for the extensors (vEMGE) and sF
holds for the flexors (vEMGF), respectively. Additionally, a
difference signal was introduced to distinguish between the
intention of hand opening and grasping. The difference is
calculated according to eq. 1, where is the
vEMG normalized to the maximum recorded volitional mus-
cular activity of each person and channel. To obtain those
maxima, the voluntary activity during three hand openings
and grasps are recorded initially. In this way, is usually
negative when closing the hand and positive when opening
the hand. The threshold sD holds for during active
stimulation.
(1)
(2)
All thresholds are set automatically when initializing the
system. The patient has to relax for 3 s, while the thresholds
are calculated according to eq. 2, which holds analogically
for sF and sD with ‘std’ as standard deviation. The thresholds
are continuously updated when the corresponding signal is
below its threshold for at least 3 s. To increase the robustness
of the detection, time constraints are applied to each
condition. A condition is classified as fulfilled, if it holds for
50 % of the values in the considered time window. A window
length of 300 ms showed to be appropriate for vEMG,
whereas for the angular velocities a shorter time window of
150 ms was applied. To trigger the ‘relax’ event, 100% of the
values need to fulfil condition C1 (see Table 1).
When FES is applied, the measured vEMG of the
stimulated muscle group is slightly disturbed by the FES.
This effect is crucial when using vEMGF to estimate the
patient’s intention. For this reason, sF adapts proportional to
the applied stimulation intensity of the extensors (stimE,1).
Furthermore, different events were established for utilizing
the vEMG condition (C1) during rest (cf. open(r)/close(r))
and during active stimulation (cf. open(a)/close(a) in
Table 1).
3 Results
The described setup and method were evaluated in three
healthy volunteers and one chronic stroke patient (male, age
54, paralysis of the left arm). Individual stimulation sides in
the electrode arrays were defined using the approach
presented in [10]. The stimulation intensities were manually
adjusted.
For each participant, we started the intention recognition
by using the state machine with all conditions of Table 1
Event
Conditions
C1
C2
C3
open (r)
1
2
open (a)
1
2
close (r)
1
2
close (a)
1
2
relax
164
‘OR’-linked. The participant was invited to try to reach each
state and stay for a while in it. Figure 3 shows the results of
one healthy volunteer of the ‘OR’-linked state machine. The
participant approached the four states consecutively and
complete grasp-and-release motions were generated.
If a participant was unable to remain in a desired state,
we removed misleading conditions or we ‘AND’-linked con-
ditions to increase the robustness of the intention detection.
Thereby, we were able to adapt to the specific behaviour of
each participant and a successful detection was possible in
each case. This procedure took less than 3 min.
The stroke patient appeared to have a good control of his
wrist extensors but not of his finger extensors. By removing
condition C3 from the events, the patient could perform three
complete grasp-and-release motions after another.
Afterwards, the stimulation intensity for the extensors needed
to be increased manually to gain sufficient hand opening.
4 Discussion and conclusion
We introduced a flexible framework for intention recognition
utilizing vEMG and motion capture. A state machine was
chosen to control the stimulation on- and offset in a grasp-
and-release task using adaptive vEMG thresholds. The
possibility of adjusting the transition event conditions
individually for each person respond to the specific
requirements and capabilities of each patient.
Our hand neuroprosthesis utilizes electrode arrays, which
complicates the EMG measurement due to shortage of space
on the forearm. Only the sum of the muscle groups, extensors
and flexors, can be measured. This and the problems of
vEMG mentioned previously render a standalone vEMG
intention recognition difficult. Our results revealed that the
combination of vEMG and inertial motion tracking is a
promising approach for an intention detection, because it
covers patients with an abnormal activation of hand extensors
and flexors.
The stimulation strategy in our approach did not take
into account the patient’s habituation to the applied FES. We
consider to replace the pre-set stimulation intensity profiles
by feedback control of unused finger joint angles. Additional
experiments with a larger group of stroke patients are
necessary to evaluate our framework. Based on those future
results, we plan to develop an automatic, logic-based
adaption procedure to configure the state machine.
Acknowledgment: We thank Axelgaard Manufacturing Co.
(Ltd., USA) for contributing the electrodes and gel layers.
Author’s Statement
Research funding: As part of the research project BeMobil,
this work was supported by the German Federal Ministry of
Education and Research (FKZ16SV7069K). Conflict of
interest: Authors state no conflict of interest. Informed
consent: Informed consent has been obtained from all
individuals included in this study. Ethical approval: The
research related to human use complies with all the relevant
national regulations, institutional policies and was performed
in accordance with the tenets of the Helsinki Declaration, and
has been approved by the Berlin Chamber of Physicians.
References
[1] Malešević NM, Popović-Maneski LZ, IIić V, et al. A multipad
electrode based functional electrical stimulation system for
restoration of grasp. J Neuroeng Rehabil 2012; 9(66).
[2] Burridge JH, Ladoceur M. Clinical and therapeutic
applications of neuromuscular stimulation: a review of current
use and speculation into future developments.
Neuromodulation 2001; 4(4): 147–154.
[3] Štrbac M, Kočović S, Marković M, Popović DB. Microsoft
Kinect-based artificial perception system for control of
functional electrical stimulation assisted grasping. BioMed
Research International, 2014; 740469.
[4] Rupp R, Rohm M, Schneiders M, Kreilinger A, Müller-Putz
GR. Functional rehabilitation of paralyzed upper extremity
after spinal cord injury by noninvasive hybrid neuro-
prostheses. Proc. IEEE 2015; 103(6): 954–968.
[5] Saxena S, Nikolić S, Popović DB. An EMG-controlled
grasping system for tetraplegics. J Rehab Res Dev 1995;
32(1): 17–24.
[6] Schauer T, Hossaini D, Hesse S, Raisch J. EMG-controlled
electrical stimulation of the paretic wrist and finger extensors
in stroke patients. In Proc. of the European Symp. Technical
Aids for Rehabilitation (TAR), Berlin, Germany, 2005.
Figure 3: Two successful iterations of the state machine of one
healthy volunteer. The states are highlighted as grey bars.
[7] Salbert RC, Schauer T, Schmidt S, Raisch J. Funktionelles
Handöffnen und -schließen mittels EMG-gesteuerter
elektrischer Stimulation. In Proc. of the 4th Symposium on
Automatic Control, Wismar, Germany, 2005.
[8] Valtin M, Kociemba K, Behling C, Kuberski B, Becker S,
Schauer T. RehaMovePro: A versatile mobile stimulation
system for transcutaneous FES applications. Eur J Transl
Myol 2016; 26(3): 203–208.
[9] Valtin M, Salchow C, Seel T, Laidig D, Schauer T. Modular
Finger and Hand Motion Capturing System Based on Inertial
and Magnetic Sensors. Current Directions in Biomedical
Engineering 2017, 3(1): 19–23.
[10] Salchow C, Valtin M, Seel T, Schauer T. A new semi-
automatic approach to find suitable virtual electrodes in
arrays using an interpolation strategy. Eur J Transl Myol
2016; 26(3): 6029.
