Smart Sensing and Embodied Intelligence

Using intelligent data fusion with different sensors, and developing new methods for recognising abnormal data patterns can provide person-specific information to personalise interventions and ensure person-centred care and support. Our research is focussed on creating verifiable and robust machine learning algorithms and Artificial Intelligence approaches to analyse physiological and behavioural data collected over time, identifying causes for concern, providing early warnings for the patient themselves, their informal carers and healthcare professionals, as well as empowering people to self-manage their long-term conditions and rehabilitation. 

Research Topics

• Intelligent sensing: Understanding which sensors work best and how can these be integrated into robot platforms, wearables and the environment. Verifying different AI methods for reliable inferring from previously learnt behaviour and patterns.  Developing new approaches to transmit this information to ensure privacy, as well as safe and contextually appropriate operation of cyber-physical health and assistive robotics technologies 

• Adapting Assistive Robot Behaviour Using Intelligent Data Analysis of User Activity Patterns to support Interactive Rehabilitation Support: 

Assistive technologies, such as smart home environments, integrated sensors and service robotics are recognised as emerging tools in helping people with long-term conditions improve their quality of life and live independently for longer. A key aspect of the research into assistive robotics for assisted living is developing contextual and social intelligence for the robot to interact appropriately, safely, and reliably in real-time. Research  is required into developing assistive robot behaviour which incorporates both environmental and user characteristics, and behaviour, as part of an overall intelligent control system architecture. In addition to having a ‘memory’ of previous interactions and situations, assistive robots need access to information that is current and one that provides a dynamic world view of the user (including their emotional state) so that they can provide information and responses that are contextually appropriate. Typical activities for which support can be provided is support with rehabilitation, medication management, cognitive and social stimulation, nutrition management etc. 

• Temporal Learning: Construction of  Digital Twins to model and predict system operation in a range of scenarios. Designing and evaluating algorithms to realise real-time tracking and recognition of dynamic human and robot action and activity. 

• Knowledge Representation: How best to annotate and categorise sensor data to enable learning for autonomous behaviour. Using  non-invasive sensing techniques to improve contextual awareness

• Embodied Intelligence: Understanding the relationship between how a robot behaves and senses, and people’s expectations/acceptance of the robot. Designing and evaluating different embodiments which are best suited for exploring and integrating into different contexts.

Related Research Projects

• The autonomous analysis of movement in infants for the early detection of movement disorders

Neurodevelopmental delay, often seen following preterm birth can be significantly life limiting. However, the earlier the diagnosis, the better opportunity there is to improve outcomes. This gap in early diagnosis calls for a solution that's not only accurate but also accessible, cost-effective, and non-invasive.

Traditionally, detecting neurodevelopmental delay can be a complex task, primarily due to the subtlety of early signs and the high costs associated with in-person testing. We envisage a future where monitoring of neurological disorders can be conducted within the comfort of a patient's home, with data collected over an extended period. This approach not only increases data availability, boosting the confidence in diagnoses, but also eliminates geographical constraints and cuts down on healthcare costs.

In our most recent work, we recruited twelve parent-infant pairs (with infants aged between 3 and 12 months) and recorded 2D video clips of the infants engaging with toys. Our work combines deep learning algorithms and 2D pose estimation techniques to analyse these recordings. The goal is to classify the infants' movements in terms of dexterity and position as they interact with toys. This analysis is not just about identifying movements; it's about capturing the complexity of these movements and the postures adopted by the infants to help provide healthcare practitioners with reliable information about the infants’ development.

Project Lead: Alex Turner

• Low-cost automatic ambient assisted living (AAL) system in homes of older adults - human activity recognition (HAR ) 

Project Lead : Armaghan Moemeni

• Digital Twins for Human - Assistive Robot Teams 

The Digital Twins for Human-Assistive Robot Teams project and a related PhD are investigating approaches for developing and using digital twins which incorporate co-dependent and co-evolved models representing patients and assistive robots.  This could accelerate the development and deployment of assistive robots in health and social care. We will develop a framework for human – assistive robot digital twin teams through the development of a demonstrator. This will enable us to simulate, verify, and validate adaptation of assistive robots to support users with complex physical needs. This research will also contribute to increased trust, understanding and development of new standards for assistive robotics.  

Project Leads: Praminda Caleb-Solly and Dominic Price

Related Past Research Projects with CHART team member involvement

• Smart Technology Innovations in Retirement Villages - Knowledge Transfer Partnership with Extracare Charitable Trust (ECCT)

This was a two-year Knowledge Transfer Partnership (KTP) led by Prof Caleb-Solly when she worked at UWE that enabled the development, testing and implementation of intelligent healthcare solutions, incorporating Smart Home and Robotics Technology, in real-world settings. The aim was to enable better lives for older people and to create sustainable communities that provide homes older people want and lifestyles they can enjoy. This KTP was an opportunity to pioneer the integration of technologies into ECCT retirement villages in order to increase quality of life and prolong independent living. ExtraCare Charitable Trust runs over 15 retirement villages in the UK. By undertaking the KTP  ECCT acquired new knowledge and expertise to help understand how to make technology applications available and affordable in its retirement villages for the benefit of residents. 

https://eandt.theiet.org/content/articles/2023/03/smart-homes-offer-solutions-for-seniors-aging-in-place/ 

Project Lead: Praminda Caleb-Solly

Related Research Publications

• Kumar, P; Leake, J; Brodie, S; Molton, J; O'Reilly, R; Pearce, A; Steele, J; Caleb-Solly, P. Accelerometers-Embedded Lycra Sleeves to Test Wear Compliance and Upper-Limb Activity in People with Stroke: A Feasibility Study. Journal of Prosthetics and Orthotics: February 1, 2022  https://journals.lww.com/jpojournal/Abstract/9000/Accelerometers_Embedded_Lycra_Sleeves_to_Test_Wear.99770.aspx 

•  Gupta, P., McClatchey, R. and Caleb-Solly, P., 2020. Tracking changes in user activity from unlabelled smart home sensor data using unsupervised learning methods. Neural Computing and Applications, 32(16), pp.12351-12362. https://link.springer.com/article/10.1007/s00521-020-04737-6

• Turner, A. and Hayes, S., 2019. The classification of minor gait alterations using wearable sensors and deep learning. IEEE Transactions on Biomedical Engineering, 66(11), pp.3136-3145. https://ieeexplore.ieee.org/abstract/document/8648216 

• Turner, A.; Hayes, S.; Sharkey, D. The Classification of Movement in Infants for the Autonomous Monitoring of Neurological Development. Sensors 2023, 23, 4800. https://www.mdpi.com/1424-8220/23/10/4800

• P Caleb-Solly, J Leake, R Baines, S Gaertner, N Evans, K Sinclair, S Battle, T Adlam, 2019, Low-Cost Sensing and Data Analytics for Understanding Usage Patterns of Early Years Powered Mobility Devices, 31st Annual Meeting of the European Academy of Childhood Disability (EACD) http://edu.eacd.org/node/647 

• Gupta, P. and Caleb-Solly, P., 2018, September. A framework for semi-supervised adaptive learning for activity recognition in healthcare applications. In International Conference on Engineering Applications of Neural Networks (pp. 3-15). Springer, Cham. https://link.springer.com/chapter/10.1007/978-3-319-98204-5_1 

• Chance, G., Jevtić, A., Caleb-Solly, P., Alenya, G., Torras, C. and Dogramadzi, S., 2018. “elbows out”—predictive tracking of partially occluded pose for robot-assisted dressing. IEEE Robotics and Automation Letters, 3(4), pp.3598-3605. https://ieeexplore.ieee.org/abstract/document/8410029 

• Fiorini, L., Cavallo, F., Dario, P., Eavis, A. and Caleb-Solly, P., 2017. Unsupervised machine learning for developing personalised behaviour models using activity data. Sensors, 17(5), p.1034. https://www.mdpi.com/1424-8220/17/5/1034 

• Fiorini, L., Caleb-Solly, P., Tsanaka, A., Cavallo, F., Dario, P. and Melhuish, C., 2015, November. The efficacy of “Busyness” as a measure for behaviour pattern analysis using unlabelled sensor data: a case study. In IET International Conference on Technologies for Active and Assisted Living (TechAAL) (pp. 1-6). IET. https://ieeexplore.ieee.org/abstract/document/7389236 

• Malekmohamadi, H., Moemeni,A. ,Orun, A. , Purohit,J.K., 2019. Low-cost Automatic Ambient Assisted Living System, In IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)(pp. 693-697) https://ieeexplore.ieee.org/abstract/document/8480390