Visual Perceptive Computing
The ever increasing capabilities of consumer sensors are reshaping the way we interact with computing devices, making it more natural, intuitive and immersive. Devices are able to perceive actions through gestures, finger articulation, body- or face tracking and more. The field of capturing and interpreting these extensive data streams, coined perceptive computing, is poised to profoundly alter medical and health applications and allow quantitative everywhere-assessment of clinical symptoms previously restricted to the physician’s subjective expertise.
At the core of these advances are sensors that three-dimensionally scan their environment and are able to accurately measure shapes and movements in the space in front of them. With current 3D cameras like the Microsoft® KinectTM or Intel® RealSenseTM body parts can be tracked with an accuracy in the millimetre range - without requiring the user to wear any markers or special clothing.
Motion Analysis Technology from Motognosis
We utilize visual perceptive computing to identify motor dysfunction in neurologic disorders.
To achieve this we are developing specific assessements optimized for different neurologic conditions. Depending on the assessment and setting, the algorithmic techniques required to identify and compute diagnostic parameters vary widely. Generally speaking, we utilize motion analysis, the field of capturing and processing motion in videos, leveraging volumetric data and combining it with other sensor streams, such as audio or RGB video. A typical workflow is as follows:
- The subject to be tested is performing a sequence of movements that was developed to uncover specific motor dysfunction.
- During the test all 3D data is recorded and then further analyzed. One integral part of the Motognosis analysis toolset is skeletal tracking, where raw video data is algorithmically mapped to a model of the body. For assessments where generic tracking algorithms fail, we have developed specifically trained segmentation algorithms, extracting exactly the movement signature of diagnostic value.
- From the change of position and orientation of body parts, we derive specific movement signals, which in turn are processed using signal analysis techniques. The resulting, assessment specific outcome parameters are robust quantifications of the important aspects of motor impairment. We additionally provide analysis modules directly quantifying depth streams and using computer vision approaches to generate aggregate visualizations of depth videos, of which motion profiles are one example, that can be used for validation of recordings or exploratory evaluation of movement patterns.
We currently use the Microsoft Kinect 2 sensor for our running systems. Our modular technology can be adapted to different hardware and software platforms. If you are interested in using our technology in your product,
Technology from Motognosis and the Motognosis Labs software have been used and are currently being used in several projects.
- Behrens JR, Mertens S, Krüger T, Grobelny A, Otte K, Mansow-Model S, Schmitz-Hübsch T, et al. Validity of visual perceptive computing for static posturography in patients with multiple sclerosis. Mult Scler J. 2016 January 26
- Behrens JR, Gusho E, Mertens S, Otte K, Mansow-Model S, Paul F, et al. Postural control analysis in multiple sclerosis with perceptive computing based on Microsoft’s Kinect. Mult Scler J. 2014 Sep;20:61–61.
- Behrens J, Pfüller C, Mansow-Model S, Otte K, Paul F, Brandt AU. Using perceptive computing in multiple sclerosis - the Short Maximum Speed Walk test. Journal f NeuroEngineering and Rehabilitation. 2014 May 27;11(1):89
- Behrens JR, Otte K, Mansow-Model S, Brandt AU, Paul F. Microsoft Kinect-based gait analysis in multiple sclerosis patients. Mult Scler J. 2013 Oct;19(11):263–263.
- Pfueller C, Otte K, Mansow-Model S, Paul F, Brandt AU. Kinect-Based Analysis of Posture, Gait and Coordination in Multiple Sclerosis Patients. Neurology. 2013 Feb 12;80.