Toward a domestic system to assist people with Parkinson's

Freezing of gait (FoG) is a disabling symptom commonly occurring in the latter stages of Parkinson's disease. It is characterized by brief episodes of inability to step, or by extremely short steps that typically occur on gait initiation or on turning while walking.¹ FoG heightens the risk of falls, which leads to loss of independence.² Aggravated mobility and difficulties in activities of daily living have a direct effect on the quality of life of patients,³ demanding attention from a carer or causing affected people to move into specialized institutions. In the absence of any completely effective pharmacological treatment for FoG, technology-based solutions to alleviate the symptom and prolong people's ability to live independently are eagerly being sought.

Such technological solutions exploit ‘sensory cueing’. External stimuli, such as lines on the floor or rhythmic sounds, can focus the attention of a person experiencing an FoG episode and help him or her initiate gait.⁴ Researchers are currently developing wearable gait assistance systems, which comprise wearable devices with inertial sensors to detect FoG episodes and activate auditory rhythmic cueing over earphones.^{5, 6} However, it is difficult to detect FoG in a timely way under real-world conditions; the best detection result reported in laboratory trials to date is 83% sensitivity.⁷ Even higher detection accuracy is necessary for everyday domestic use. We have been developing a distributed home system based on Microsoft Kinect sensors that can use contextual information from the environment as an additional cue.

FoG depends on the person's walking situation. It often occurs on walking transitions: turns, setting off, arrival, and in open spaces.⁸ It can also occur when people approach narrow spaces, such as doors, and in crowded places. Recognizing such situations is a very powerful cue for correct detection. Three pieces of information are essential: the person's location, their surroundings, and a contextual model of the environmental influence on FoG. We decided to model the environmental influence on FoG, taking into account direct geometric relations,⁹ and we chose the Kinect camera to capture 3D data as the system sensor. We have been developing a system that uses this type of sensor to track multiple people inside a home.¹⁰

We use Kinect cameras only on the most problematic locations for FoG in people's homes; it is likely that camera coverage will be incomplete. The optimum mounting position for sensors is 2.2–2.4m above floor level, angled downward. We have developed a Kinect application that can track more than one person up to 6m from the camera. The application uses multiple particle filter trackers, one for each person in the scene.¹¹ Our implemented algorithm can still give accurate results with a low, 320×240 image resolution, making it suitable for use with cheap, low-end computing nodes. Laboratory experiments have shown a consistent position estimation error <0.2m.

Formulating an appropriate context model for FoG is crucial for the success of our proposed approach. Behavioral data on affected people is essential for this. In the project's final stage, we will collect the necessary behavioral data through one-day recording sessions in people's homes. This requires our system to be portable and fast to set up on site. During setup, we enter details of the scene the camera will be observing, such as ground level and interest zones (see Figure 1).

Figure 1. Simulation scenario for tracking two individuals. One person, affected by freezing of gait (FoG), is standing and the other person, a carer, is sitting. The camera scene is described by its tracking boundaries (green) and interest zones (blue). pid: Person identification.

Our model is run independently for each camera in a distributed system, both for scene setup and people tracking. Data on the appearance of people being tracked is maintained in one central processing node and shared to every camera on request. For re-identification between cameras, we use an appearance model based on the person's color histogram and height along with the known average movement times between cameras. We are currently testing this re-identification approach in a laboratory setup and we hope to report the results soon. The system under operation is shown in a short video that is available online.¹²

Our indoor position tracking system is just the first step toward the final goal of FoG detection based on context. For daily use it is important that the system can automatically recognize the patient. Future work will focus on identifying the patient based on combined data from the camera and the wearable sensor. In addition, we will use the system to record behavioral data for context modeling in patients' homes.

This work was supported in part by the doctoral program fellowship offered by the Erasmus Mundus Joint Doctorate in Interactive and Cognitive Environments.