EMERITI - Project status and current research

Current project context 

This document describes the current status of the EMERITI project as of 01/10/2019 and the ongoing and planned developments for the next 5 months of work. It also details the lines of research that are being followed and the first advances of the project. 

The project has 5 work packages: 

- WP1: Voice assistants and their use by older people 

- WP2: Investigation of the integration of elements in the senior's home for use with voice assistants 

- WP3: Case Study 

- WP4: Project coordination 

- WP5: Dissemination of results 

The state of the art of the voice assistants currently available on the market and their technical characteristics have been elaborated in work package 1. This has allowed to have an initial overview of the research possibilities of each of these devices and to select the best platforms for the research work and the case study. At the same time, we have studied the different domotic elements with which these assistants can interact and which of them could be especially useful in the life of the elderly. 

This work package includes one of the tangible results of the project which is a guide to the use of voice assistants and home automation devices in the home of the elderly. We are currently working on its development and hope that it will be of great use to any elderly person or family member who wishes to include these devices in the home. 

Work package 2 includes the development and research tasks associated with the lines of research that will be presented later in this document. 

Work package 3 will be developed in the final 4 months of the project in which the necessary data for the elaboration of scientific publications in the lines of research detailed later on can be taken. Finally, work packages 4 and 5 will be described in the final report of the project including the publications produced as a result of the research project. 

It is worth mentioning that the research project officially starts in May 2019 but it has not been possible to start its implementation and formalise its signature until June 2019, thus losing one month of the project's available time. Despite this setback, the work is proceeding normally and following the planning set out in the initial project report.

Guide to the use of voice assistants and home automation in the home of the elderly 

The main objectives of the EMERITI project are:

- The improvement of the accessibility in the use of new technologies and the incursion of these in their daily life. 

- The promotion of communication with their families and peers using these technologies in a simple way. 

- The assistance in daily tasks that become challenges for these people with the aging process such as the handling of devices at home like raising and lowering blinds, shopping, remembering to take medication, etc. 

To respond to these objectives, one of the deliverables of the project is a didactic guide on how to easily introduce voice assistants in the daily life of the elderly. 

Firstly, this guide describes in a very accessible way for people without technical knowledge what a virtual assistant is, how it works and what they need to do their tasks correctly. It also presents the different alternatives available on the market. 

Secondly, the technical requirements that the elder's household must meet in order to include a voice assistant are presented. At the same time the main device use cases are described. 

Thirdly, and related to the previous point, a review is made of the home automation that can be included in the elderly person's home and how the devices that can be found on the market work in general, highlighting that it must be taken into account when acquiring these devices and their compatibility for use with voice assistants. Technologies and operation modes are always described in the most accessible way possible, without too much technicality and that the elderly person or a relative can understand it easily. 

Finally, this guide presents a series of case studies carried out in the home of an elderly person that can facilitate daily tasks, communication with family members and familiarization of the elderly person with new technologies through the use of voice assistants and home automation devices. 

Research 

This section details the main lines of research that are currently being followed within the EMERITI project. These lines of research include the use of voice assistants by the elderly and how to develop new services and algorithms to facilitate their daily life and communication with their families. Google Assistant and Alexa have been chosen as development platforms since, after reviewing the state of the art, they have been nominated as the most widely used and best software agent development platforms. As these are technologies with very recent APIs and under constant change, it is necessary to use other platforms that allow the development of certain routines that currently platforms such as Google Assistant or Amazon have not allowed. The main lines of research that are being followed to respond to work packages 2 and 3 are described below. 

Use of voice assistants by the elderly

The aim is to be able to evaluate in a quantitative and qualitative way the use of voice assistants by the elderly once they have been provided with one. In works such as that of the Front Porch Center for Innovation and Wellbeing (FPCIW) [1] which carried out a pilot study in which it distributed devices with Alexa [2]. They measured user satisfaction through surveys after a period of time. 

lustration 1 Transparent connection Home Assistant Google Assistant / Alexa 

On the contrary, in the EMERITI project we would like to quantify how much use is given to these devices by using a parallel platform like Home Assistant that allows us to extract all the actions that are carried out in each of the major, both by the assistant and by the home automation in order to quantify reliably the use and usefulness of these devices. 

In this way, one of the lines being explored in the project is the joint use of Home Assistant with Alexa and Google Assistant, so that this open platform serves as a proxy that allows us to effectively measure the events that occur in the home of the mayor without interfering with the normal use of these assistants.

New services for voice assistants useful for seniors 

Another research objective is the creation of new services for voice assistants that will be useful to the elderly. To do this, we seek to hold meetings and breastormings with different elderly people and their families to identify possible useful services that can be implemented using a skill from Alexa or Action from Google Assistant. 

Some of the skills ideas that have emerged in these brainstormings are the following: 

- Skill/Action of monitoring and promoting physical activity in the elderly: It consists of the development of a skill that allows sending messages of encouragement to the elderly to achieve certain challenges throughout the day, such as walking a certain number of steps. The user can set a goal by voice and include an activity bracelet to monitor the user's activity. 

- Skill/Action to place purchase orders to a nearby supermarket in the village: The idea is to develop a skill that allows you to send purchase orders from home to the village supermarket so that these small shops can attend to the larger one at home. 

- Skil/Action to collect suggestions or needs from the town council: Skill to send messages to a municipal mailbox to attend to the demands and suggestions of the town's senior citizens. 

- Skill /Action to control the daily medication: skill to remember the daily medication based on a previously established calendar.

Illustration 2 Skills / Google Assistant Brainstorming 

We are currently working and exploring how to develop some of these skills to offer new services to the wholesale market through virtual assistants. Their practical application and potential use in daily life is being investigated. In addition, we are studying how to improve the proactivity of these voice assistants since they fundamentally function under the explicit interaction of the user, notably limiting the capacity to act on their own without the need for a "Google Ok" or "Alexa". 

Communication of the elderly with family and friends via voice assistants 

It is in this line of work that the most difficulties are being encountered, since until relatively recently (12 September 2019) call and video call services through assistants were very limited to applications offered by Google or Amazon. In this line of work, the option of developing its own video call channel using technologies such as WebRTC has been considered, but it is subject to the limitations of these platforms. 

Integration of new devices in the senior's home with voice assistants 

In this line of research we are working on including the handling of new devices with the Google and Alexa assistant. This development is planned with open platforms such as Home Assistant [3] and IFTTT [4] which provide greater flexibility in the field of research and prototyping as they represent a first step towards a final market product. 

In this line of research, work is being done on several alternatives aimed at creating and integrating new devices that will improve some aspect of the life of the elderly. 

Two possible case studies are currently being considered and the viability of each one is being explored: 

Intelligent heating for the elderly: This case study proposes the integration of voice assistants with the existing heating in the elderly person's home. This will allow the desired temperature in the house to be controlled at all times using the assistant as an interface. In addition to this, the heating system will learn from the user in such a way that it locates the user in the home and adapts to his habits while trying to save energy. 

2. Intelligent video door entry system for the elderly: This case study is aimed at developing a video door entry system that enables the analysis of the people who come to the elderly person's home so that he or she knows who is at the door before going to open it. The aim is to integrate different automatic learning techniques for face detection, classification by age and sex and for this information to be notified to the user by the voice assistant. 

References 

[1] Front Porch Center for Innovation and Wellbeing, “Front Porch Center for Innovation and Wellbeing |.” [Online]. Available: https://fpciw.org/. [Accessed: 08-Jul-2019]. 

[2] FPCIW, “Front Porch Center for Innovation and Wellbeing Releases Results of Amazon Alexa Pilot Project in San Diego Retirement Community - Front Porch Center for Innovation and Wellbeing.” [Online]. Available: https://fpciw.org/news/front-porch-center-for- innovation-and-wellbeing-releases-results-of-amazon-alexa-pilot-project-in-san-diego- retirement-community/. [Accessed: 08-Jul-2019]. 

[3] “Home assistant Care-O-bot,” Environment, 1998. [Online]. Available: https://www.home- assistant.io/. [Accessed: 13-Sep-2019]. 

[4] “Every thing works better together - IFTTT.” [Online]. Available: https://ifttt.com/. [Accessed: 13-Sep-2019].