Repositorio Institucional de Documentos

2013 111 Rubén Blasco Marín Smart kitchen for Ambient Assisted Living Departamento Director/es Ingeniería Electrónica y Comunicaciones Casas Nebra, Roberto Marco Marco, Álvaro Director/es Tesis Doctoral Autor Repositorio de la Universidad de Zaragoza – Zaguan http://zaguan.unizar.es UNIVERSIDAD DE ZARAGOZA Departamento Director/es Autor Rubén Blasco Marín SMART KITCHEN FOR AMBIENT ASSISTED LIVING Ingeniería Electrónica y Comunicaciones Director/es Casas Nebra, Roberto Marco Marco, Álvaro Tesis Doctoral 2013 Repositorio de la Universidad de Zaragoza – Zaguan http://zaguan.unizar.es UNIVERSIDAD DE ZARAGOZA Departamento Director/es Director/es Tesis Doctoral Autor Repositorio de la Universidad de Zaragoza – Zaguan http://zaguan.unizar.es UNIVERSIDAD DE ZARAGOZA PhD THESIS Smart kitchen for Ambient Assisted Living RUBÉN BLASCO MARÍN ADVISORS: DR. ROBERTO CASAS NEBRA DR. ÁLVARO MARCO MARCO Zaragoza, May 2013 To Pilar, for her love, support and understanding I would like to thanks a lot of people and institutions for their collaboration in the development of this thesis: My advisors, Roberto and Álvaro, for their suggestions and corrections, for their encouragements to start, to continuous and to finally end this thesis. Their guide, problem vision and opinions have been essential in this work. Armando Roy, for his leadership and interest in the Easy Line Plus project, origin of this thesis as well as for his confidence in my work. My workmates in HOWLab and TECNODISCAP: Ángel, Diego and Teresa as well as to my workmates in TECNODISCAP Alejandro, Armando, David, Hector, Joaquin, Miguel and Victorian for their company, support and friendship. Also to the Easy Line Plus research team: BSH, IDENT, NEWI, Motive, G2V and CLab people. Vic Grout and Rich Picking and their team, especially Armando, for welcoming me for two months at the Glyndŵr University in the development of the Easy Line Project. Paul Lukowicz and their team, especially Gerald, to give me the opportunity to work together at the University of Passau and making very nice my stay. Last but not least, my friends and my family. -5INDEX Index........................................................................................................................................................................ 5 List of tables ............................................................................................................................................................ 9 List of figures ......................................................................................................................................................... 11 Chapter 1: Introduction ........................................................................................................................................ 13 1.1. Motivation ........................................................................................................................................... 13 1.2. Objectives ............................................................................................................................................ 14 1.3. Thesis framework ................................................................................................................................ 14 1.4. Methodology ....................................................................................................................................... 16 1.5. Thesis structure ................................................................................................................................... 17 Chapter 2: State of the art .................................................................................................................................... 18 2.1. Ambient Intelligence ........................................................................................................................... 18 2.1.1. Context interaction ..................................................................................................................... 20 2.1.2. Reasoning .................................................................................................................................... 24 2.1.3. Context modeling and system architecture ................................................................................ 25 2.2. The kitchen scenario ............................................................................................................................ 25 Chapter 3: Study of the needs of the target population in the kitchen ................................................................ 28 3.1. Introduction ......................................................................................................................................... 28 3.2. Related work ........................................................................................................................................ 29 3.2.1. User requirements for specific users groups .............................................................................. 29 3.2.2. International classification of Functioning (ICF) ......................................................................... 29 3.2.3. Human Device Interactions models ............................................................................................ 30 3.3. Users’ needs identification methodology ............................................................................................ 32 3.3.1. Research Approach ..................................................................................................................... 32 3.4. Looking for needs in the design of An AAL kitchen ............................................................................. 37 3.4.1. Users’ characterization ............................................................................................................... 37 3.4.2. User device interaction characterization .................................................................................... 42 -63.4.3. Identification of users’ needs ..................................................................................................... 51 3.5. Other information sources .................................................................................................................. 57 3.5.1. User surveys ................................................................................................................................ 57 3.5.2. Co-design sesions ........................................................................................................................ 58 3.6. Conclusions .......................................................................................................................................... 59 Chapter 4: Architecture design ............................................................................................................................. 61 4.1. System description .............................................................................................................................. 61 4.1.1. Context interaction ..................................................................................................................... 62 4.1.2. User interaction .......................................................................................................................... 63 4.1.3. Intelligence (E-Servant) ............................................................................................................... 63 4.1.4. Remote services .......................................................................................................................... 64 4.2. Software architecture of the AMI kitchen ........................................................................................... 64 4.2.1. Block diagram ............................................................................................................................. 64 4.2.2. Software platform ....................................................................................................................... 68 4.3. Conclusions .......................................................................................................................................... 68 Chapter 5: Context Interaction ............................................................................................................................. 69 5.1. Sensor-actuator infrastructure ............................................................................................................ 69 5.1.1. ZigBee sensors and actuators ..................................................................................................... 70 5.1.2. RFID readers ................................................................................................................................ 72 5.1.3. Household appliances ................................................................................................................. 74 5.2. Data processing ................................................................................................................................... 75 5.2.1. Fall detection .............................................................................................................................. 78 5.2.2. Indoor location ............................................................................................................................ 79 5.3. Context Manager Implementation ...................................................................................................... 81 5.3.1. Driver layer ................................................................................................................................. 81 5.3.2. Device layer ................................................................................................................................. 82 5.3.3. Device manager layer ................................................................................................................. 86 5.4. Conclusions .......................................................................................................................................... 87 Chapter 6: Reasoning ............................................................................................................................................ 88 Universidad de Zaragoza -13CHAPTER 1: INTRODUCTION This chapter describes the motivation that has led to the development of this thesis and defines their objectives, framework and work plan. It also exposes the methodology followed across the different phases of the work and finally presents the structure of the thesis. 1.1. MOTIVATION The ageing of our populations is a well-known problem in developed countries. European Union population projections are alarming; the ratio of people aged 65 years or over will increase from 17.1% to 30.0% in 2060 (from 84.6 million in 2008 to 151.5 million people in 2060) (Giannakouris 2008). Similar figures have been found in the USA, where elderly people will represent 20.2% of the population in 2050, or in Japan, with 35.7% for the same year (U.S. Census bureau 2012, NIPSSR 2002). Elderly people suffer several physical and/or cognitive impairments which increase with the passing of years. According to the Spanish National Statistics Institute (INE 2012), 13.9% of the population between 65 to 69 years old have some disability; this ratio increases to 29.4% for the age group of 75 to 79; growing exponentially with age. Old age affects sensing, information processing capability, reduces speed and increases timing of precise movements, etc. All these issues increase difficulties of comprehension of complex scenarios which may require multi-tasking or a heightened attention over long periods of time. As a consequence, elderly people progressively lose the capacity to perform autonomously their daily activities. Thus, household appliances, instead of fostering independent living, become a burden that adds to ageing limitations. Older people are one of the population groups most vulnerable to accidents, particularly at home (Angermann, Bauer et al. 2007). Most domestic injuries are related to works done in the kitchen; most dangerous utensils are knives and kitchen tools, cutlery and tableware followed by the household appliances. As a consequence of these accidents, elderly people lose confidence in their capabilities, decrease their self-esteem and, in many cases consequently decide to move to a nursing home. According to UK statistics, one in eight people attended by domestic accident in a hospital is over 65 years. The low mobility, coupled with low sensitivity to the smell of burning and smoke makes older people more likely to suffer burns or scalds (four to five times higher than the rest of the population). These demographic changes will lead to strong challenges in the health and pension system and thereby, impacting on workforce. However, this situation will also provide business opportunities for suppliers of innovative technology in the field of ICT applied to the domestic environment or AAL (Ambient Assisted Living). These companies link new technologies and the social environment providing products and services that aim to improve people´s quality of life at their own homes (Van Den Broek, Cavallo et al. 2010). The smart home concept raises a cost-effective manner of improving care for the elderly and persons with disabilities in a non-intrusive way, improving their independence, health and preventing social isolation (Chan, Campo et al. 2009). It poses an alternative to hospitalization or institutionalization, allowing the elderly and/or disabled to be cared at their own environment thanks to technology. This thesis aims to go one step further in this direction, emphasizing the use of Ambient Intelligence (AmI) in the kitchen, where the elderly need more support for doing their everyday activities, and creating an AAL ready to be setup at their home. Chapter 1: Introduction -141.2. OBJECTIVES The main objective of this thesis is the creation of an AAL environment centred in the kitchen, which supports elderly and/or disabled people in the daily activities more easily and safely. The system will help the person based on their capabilities and context. This raises the following sub-objectives: 1. Studying the needs of the target population in the kitchen This study defines the requirements of the smart kitchen taking into consideration all the stakeholders involved. This information is used to define the system functionalities. 2. Designing and implementing the system architecture according to the identified needs. After identifying the needs of the target population, the system architecture is designed. This architecture must be modular, flexible and able to interconnect all the devices on the AAL, allowing its subsequent deployment. 3. Designing and implementing the context awareness infrastructure A sensor network is designed and integrated in the environment in order to extract the relevant information from the context. Commercial sensors will be preferably used in order to reduce the system cost. Also household appliances must be connected to the system. 4. Designing and developing the system intelligence. Various techniques of analysis of information from the context and system are studied. Since it is necessary to interpret a multitude of data for decision-making, different techniques have been raised, for example, decision trees, neural networks, etc. The intelligence of the system is responsible for processing information from the environment (sensors, appliances and users) and to respond in every situation. Its main functionality is to assist the user in their daily tasks in the kitchen, simplifying and helping when needed. In addition, it must ensure and improve safety in the kitchen, being able to respond in emergency situations. On the other hand, the system should adapt to the user's capabilities, detecting changes in habits that may indicate a deterioration of their abilities. Available information of the context is analysed in order to identify which parameters could be relevant for this purpose. 5. Designing and analysing the system evaluation with real users and experts. The system is validated with real users and experts from social-health areas. The assessment evaluates the system in three areas: accessibility, functionality and usability. From the conclusions, proposals for future improvement emerge. 1.3. THESIS FRAMEWORK This thesis has been developed within the frame of the research project Easy Line Plus (Easy Line + 2010). The project, funded by the European Commission, has as main objective to develop near to market prototypes of advanced white goods in order to support elderly persons with or without disabilities to carry out a longer independent life which will compensate their loss of physical and/or cognitive abilities. The consortium of this project is composed for the next companies and institutions:  BSH Electrodomésticos España, S.A. (Spain): Manufacturer of electrical appliances. It is integrated into the European leader group BSH Bosch und Siemens Hausgeräte GmbH, which markets in Spain Bosch, Siemens, Gaggenau, Neff, Ufesa and Balay (BSH 2012). Universidad de Zaragoza -15-  Gis Gera Ident-Systeme GmbH (Germany): GERA-IDENT is an auto-ID company with its main focus on RFID technology (Gera-Ident 2012).  Motive Technology (formerly ADSS) (U.K.): It is a company on bespoke software development; creating integrated systems, applications and complete end-to-end solutions (Motive Technology 2012).  Grupo de Empresas G2V. ISDE Aragón, S.L. (Spain): It is a service company with its main focus on domotics setup (G2V 2012).  C-LAB (Germany): It is a joint research and development laboratory operated by Atos and the University of Paderborn. It collaborates in the project as accessibility expert (C-Lab 2012).  Glyndŵr University (U.K.): It contributes to the project as expert in new technologies focus on Human Interfaces and software development as well as technological evaluation (Glyndŵr University 2012)  Zaragoza University (Spain): Project coordinator and technical coordinator of the project, the University of Zaragoza contributes to the project as expert in new technologies focused on the software, hardware and firmware as well as technological evaluation (Universidad de Zaragoza 2012). The system developed in the project is based on appliances that provide functionality for the elderly, with particular emphasis on safety and reliability. It must be "intelligent enough” to interpret the context and to communicate with other devices in the environment such as distributed sensors and user interfaces. In addition, these appliances have to be friendly offering the user an easy interface to understand and operate and some of them should implement specific functionalities; for example, the refrigerator reporting which products has inside or warning when one of them is expired. The project has been developed from 2007 to 2011 with the effort of the said multidisciplinary teams. Consequently not all the work done in the project falls within the thesis scope. For example, all the work related with the Human Machine Interfaces (HMI) as well as its associated database are excluded of this thesis because they have been performed by Glyndwr University and Motive Technology teams 1 . All the contributions reflected in this thesis have been done by the author under the supervision of his advisors. When the work of third parties is mentioned in order to ease comprehension of the thesis, these contributions are always attributed to their original authors. In every case, this is used to explain my work, remarking the author(s) in text and citing them. This policy has been also followed with the work done by other researchers in my team at the University of Zaragoza. 1 I had the pleasure to work side by side with these teams for two months in Glyndwr University defining and deploying the communication between the human machine interfaces and the system intelligence. Also, during the integration sessions and in several work sessions in Germany and Spain, I have worked together the GeraIdent team in order to define the specification of the ZigBee modules communication (protocol, schematic, PCB, etc.) that finally has been used by the RFID readers in the system. Chapter 1: Introduction -161.4. METHODOLOGY According to the objectives, there are three thematic areas that required different methodologies: Study of the needs of the target population in the kitchen This thesis starts with the study of the needs of the target population. This study has been done based on statistical data and using the International Classification of Functionalities Disability and health (ICF) to describe the target population and their interaction with the kitchen appliances. The revision and systematization of this process has finished with the development of a new design methodology which permits the extraction of a specific population’s needs. The information obtained using it, has been completed with the stakeholders’ opinion using a combination of quantitative (surveys) and qualitative (co-design sessions with experts and discussions) tools. AAL design and deployment Design of technical developments has been always done following a top-down approach; from the whole system to the finest details. System architecture has been divided into several functional blocks defining the communication interfaces between them. This way, each block is designed and developed independently assuring its integration in the final solution. An important part of this thesis has been the design of the system architecture considering hardware, communications and software developments. The environmental context has been modelled following the OSGi4AmI (Marco, Casas et al. 2009) ontology and its complement in firmware design in order to assure the interoperability between devices of several manufacturers. In the case of hardware and firmware, the design process started with the definition of its specifications and with the identification of its block diagrams and corresponding schematics. In the same way, functionalities of the firmware have been identified, defined the flow diagram that characterizes its behaviour, and programing it in a modular strategy. Several prototypes have been manufactured and tested, entering a redesign process that culminated when the device worked correctly. Since it is necessary to process a multitude of data for reasoning and decision-making, different techniques has been raised. These analysis blocks have been developed and validated, determining which technique is the best in each situation. A gradual process of implementation has been followed, starting with the most basic features until achieving the objectives. Software implementation starts from the use cases and block diagram of the architecture. Each module has a flow chart to model its behaviour and UML (Unified Modelling Language) diagrams have been used for the design of different building classes. After programming each module, an iterative process for error debugging has been followed. Assessment with real user A final assessment of the system with real users and social-health experts has been performed to determine system’s strengths and weaknesses and also to indicate which parts of the system should be redesigned. The assessment mainly targeted accessibility, functionality and usability indicators. Again it has been chosen a mixed combination of quantitative (survey) and qualitative (objective observation, interviews, etc.) tools. Universidad de Zaragoza -17Every part of the thesis followed the classical research methodology, analysing the state of the art and studying the advances in the related field. 1.5. THESIS STRUCTURE This thesis is structured in the following chapters: Chapter 1: Introduction describes the motivation that has led to the development of this thesis, defining their objectives, framework and work plan. It also exposes the methodology followed during the different phases of the work to finally present the structure of the thesis. Chapter 2: State of the art. This chapter describes the key technical work related to this thesis. Starting from the concept of Ambient Intelligence that later introduces the Ambient Assisted Living to then delve into the related disciplines of the work presented. Finally, main advances in the kitchen scenario are commented. Specific state of the art in non-technical disciplines, i.e. study of needs and evaluation, is discussed in chapter 3 and 7 respectively. Chapter 3: Study of the needs of the target population in the kitchen presents the methodology designed to detect the needs of a specific collective and their application to the design, in this case, of the Smart Kitchen. This study is completed with the analysis of other data sources (statistics, user surveys and workshops with professional caregivers, family and engineers). Chapter 4: Architecture design. Derived from the study of the needs in order to support elderly people in their daily tasks, it presents the AmI architecture of the smart kitchen; ergo an Ambient Assisted Living in the Kitchen. First, system is described, explaining its interaction with the user and the environment. Next, software architecture is detailed, describing the functionality of its main blocks. Chapter 5: Context Interaction. This chapter contains the contributions made in the context interaction implementation of the system; i.e. the physical deployment of the sensors and actuators network as well as the modifications accomplished in the kitchen appliances to enhance their capabilities. Chapter 6: Reasoning presents the contributions in the design and implementation of the intelligence of the system describing the rationale of the logical rules and software implementation which drive the system operation. Also, contributions in the quality of life evaluation system and in the user modelling are included. Chapter 7: Smart kitchen assessment describes the methodology and the tools developed for the evaluation of the system. This evaluation was conducted simultaneously in two countries: UK and Spain, by multidisciplinary teams from the Glyndŵr University and the University of Zaragoza. Chapter 8: Conclusions and future work. The last chapter presents the conclusions of the work outlined in previous chapters highlighting the most important contributions of this thesis. Likewise, it proposes future lines of research in different related fields. Chapter 2: State of the art -18CHAPTER 2: STATE OF THE ART This chapter describes the key work related to this thesis. Ambient Intelligence and Ambient Assisted Living concepts are introduced to then delve into the related disciplines with the work presented. Finally, main advances in the kitchen scenario are commented. The state of the art related to the work presented in chapters 3 and 7, due to dealing with non-technical issues, it was decided to include them within the same chapters, making them self-contained. 2.1. AMBIENT INTELLIGENCE In 2007, when I started working on this thesis, it was quite common for people to have available two computers: a desktop and a laptop. The technological developments in consumer electronics has caused that, day by day, smaller and cheaper devices with advanced computing capabilities reach the mass market. Today, in 2012, Smart Phones are usually added to desktop and laptop equipment and, in many cases, laptops are replaced by tablets. As Cook et al. say (Cook, Augusto et al. 2009), this has been the evolution from the 80’s, where a PC was used by several people, to the current situation where it is quite normal that a person uses multiple devices with high capacities (GPS, Smart Phones, laptops, Tablets, etc.) This situation has shifted to other housing appliances such as television or video games, and even kitchen appliances. Today, it seems that, technologically, we are close to the situation posed by Weiser in the paradigm of ubiquitous computing, where computers are embedded and distributed in the environment and people use them naturally (Weiser 1991). As an evolution of this thought, the concept of Ambient Intelligence (AmI) is born in the late twentieth century as an overview of progress in consumer electronics, telecommunications and computing for the period between 2010 and 2020. Today, AmI is used to refer a multidisciplinary area of knowledge that encompasses several fields of engineering and computing (Aarts, Encarnação 2006). It is necessary to emphasize the role played by the Information Society and Technologies Advisory Group (ISTAG) in the consolidation of the AmI concept. In its twentieth report of 2001, the ISTAG gives expression to the AmI, which to this moment was just a "vision of the future" (Ducatel, Bogdanowicz et al. 2001). It reinforces the concept and use giving a formal definition of what was considered an AmI and identifies which technologies are relevant to its development (Cook, Augusto et al. 2009, Aarts, Encarnação 2006, Augusto, Nakashima et al. 2010). But, what is an AmI? An AmI can be defined as a “sensitive and adaptive electronic environments that respond to the actions of the persons and objects and cater for their needs” (Aarts, Wichert 2009). That is an intelligent system, customizable, aware of the context, adaptive and anticipatory. This approach includes the entire environment, taking into account each individual object and associating its interaction with humans. Many times, AmI and AAL terms appear in the literature interchangeably to refer to very similar technological solutions. As already mentioned, AmI is a sensitive environment capable of interacting with the person, ranging from ubiquitous computing to smart interfaces with certain social skills. The fundamental difference lies in their purpose: AAL attempts to prolong the time that a person can live in dignity at home, improving their autonomy and self-confidence, simplifying their daily tasks, monitoring and caring for the elderly, sick or disabled, improving safety and saving resources (Steg, Strese et al. 2006). Universidad de Zaragoza -19That is, AAL uses the AmI as a fundamental tool in each specific context. In fact, some authors (Aarts, Wichert 2009) consider the AAL as an area within the AmI focused on providing an integral solution for supporting the person in the independent living, home care and residential nursing houses. The European Commission has heavily invested in this direction, creating the "Ambient Assisted Living program”. This research and development program aims to improve the quality of life of older people through the use of the Information and Communication Technologies (ICTs). With a performance period from the 2008 to the 2013, AAL plans to mobilize funds around 600 M€ (AAL ASSOCIATION 2013). Currently, AAL is being extended to others environments such as the car or the workplace. The AAliance (The European Ambient Assisted Living Innovation Alliance) proposes three macro scenarios for the AAL development (Van Den Broek, Cavallo et al. 2010): AAL4persons: This domain is focused on person centric applications and it is divided in two subdomains: @home and @mobile. Both with the same purpose “Ageing well for the person” AAL in the community: This domain is focused on applications which improve the social inclusion of the elderly people, their communications and their participation in the community. AAL@work: This domain is focused on application which supporting elderly and people with disabilities at work. Next diagram summarizes the main field of the AAL domains: Figure 1 AAL domains Therefore, as O'Grady et al. pose (O’Grady, Muldoon et al. 2010), an “Ambient Assisted Living (AAL) is advocated as technological solutions that will enable the elderly population maintain their independence for a longer time than would otherwise be the case”. To achieve this goal, the AAL should be aware of context, including in this the person, providing help when needed, detecting abnormal situations and acting accordingly (Andrushevich, Kistler et al. 2009). To respond to an AAL, it is necessary to involve different fields of knowledge such as Artificial Intelligence (AI), robotics, sensor networks, wireless communications, natural interfaces, among others (Ramos, Augusto et al. 2008). The AAL roadmap (Van Den Broek, Cavallo et al. 2010) clusters these technologies in several fields: related to sensing, reasoning, action, interaction and communication. The following sections focus the state of the art on the fields related to the work presented in this thesis: context interaction, reasoning and system architecture. AAL4persons @home @mobile Health, rehabilitation, care Coping with impairments and disabilities Personal activity management and monitoring Activities of the Daily Life oriented support Shopping, eat and drink, Social interaction @community @work Social inclusion Entertainment and leisure Cultural and experience exchanges Mobility Background Needs of older workers in the workplace Access to working space Assuring enviromental working conditions Support for working Prevention of diseases and injuries Safety and health regulations Chapter 2: State of the art -202.1.1. CONTEXT INTERACTION Context awareness is very important because it provides information about the people, places, devices and things present in the environment (Hong, Suh et al. 2009). However, there is not a common definition of the context awareness, being possible to find different definitions of context depending on the application domain (Zainol, Nakata 2010). One of the most broadly accepted (Hong, Suh et al. 2009, Zainol, Nakata 2010, Liu, Li et al. 2011) is the definition proposed by Dey et al. (Dey, Abowd et al. 2001): “any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and application themselves.” Most of authors agree in segregating the User or People context from the Environmental or Physical context, proposing more fields conditioned by the application domain. One of the most complete approaches is proposed by Feng et al. (Feng, Apers et al. 2004) that categorizes the world in two different contexts that interact with the AmI system: Environmental context: There are physical environment (e.g. time, location, temperature, noise, etc.), social environment (e.g. traffic jam, surrounding people, etc.) and computational environment (e.g. surrounding devices, communication resources, etc. Person-centric context: The personal context includes background (e.g. interest, habit, preference, etc.), dynamic behaviour (e.g. task, activity, intention, etc.), physiological state (e.g. body temperature, heart rate, etc.) and emotional state (e.g. happiness, sadness, calm, etc.). Although most of the authors agree on the items present in the context, it is possible to find different clustering of them, producing several categorizations. For example Gu (Gu 2009) proposes, from a user-centric point of view, to divide the context in five areas: the computing, user, physical, time and the social contexts. Same way, Liu (Liu, Li et al. 2011) categorizes the context in three areas: User, Physical and Network contexts. Context Interaction domain includes the technologies related with extraction of information from the context (sensors, human interfaces, etc.) and acting over it (actuators as devices that can produce changes in the context). Next subsections go over these technologies talking about wireless sensor networks, interoperability and kitchen appliances. Note that Human Interfaces has not been included in this revision of the state of the art because they are not object of this thesis. Wireless Sensor Networks Distributed sensors are a fundamental part of any AmI. In this field, Wireless Sensor Networks (WSNs) are one of the most powerful tools, thanks to their capacity to access devices embedded in the environment. These platforms enable monitoring of the environment sending information to a base station or to an access point of a fixed infrastructure (as Internet) (Yick, Mukherjee et al. 2008). WSNs are increasingly incorporating smarter devices capable to make certain decisions. These nodes, in many cases, use microcontrollers with embedded operating systems (Bhatti, Carlson et al. 2005) in order to easier the development of new capacities as FreeRTOS (FreeRTOS 2012), Contiki OS (Contiki OS 2012), MICROSAR OS (MICROSAR OS 2012) or Tiny OS (Tiny OS 2012). Universidad de Zaragoza -21As Yick et al. described (Yick, Mukherjee et al. 2008) “Smart Sensor nodes are low power devices equipped with one or more sensors, a processor, memory, a power supply, a radio and an actuator 2 .” Normally, smart sensor nodes use specific sensor-oriented protocols (low transfer rate and very low power consumption) in order to increase the battery life. Standards are essential to provide interoperability among devices, being ZigBee (ZigBee Alliance 2012), Bluetooth (Bluetooth 2012) and EnOcean standard of ultra-low-power (Enocean 2012) the technologies most used. Due to increasing importance of the Internet of Things concept, use of IPenabled standards such as WiFi (WiFi Alliance 2012) and 6lowPAN (Shelby, Bormann 2009) are also becoming relevant in the field. WSNs can have a few sensors or hundreds of devices grouped in different topologies depending on the application and communication protocol used. These are typically based on WSN protocols that use routing and MAC layers of specialized transportation for sensors, being the self-organization one of the key features in these communication stacks. This feature allows the nodes to initialize by discovering their neighbours and building local area neighbour tables (Stankovic 2008). Several examples of the use of sensors in an AAL can be found: Villacorta et al (Villacorta, Jiménez et al. 2011) present a configurable sensor network with sound sensors (that are based in the radar philosophy tracking objects and detect presence), IP video cameras (to monitor detection and fall detection) and RFID (Radio Frequency IDentification) module (that is used to help the person to identify objects that have been previously labelled with passive RFID labels). Currently, this system is being studied in two scenarios: a nursing home and assistive home. Vacher et al. (Vacher, Portet et al. 2010) show the possibilities that audio processing could have in an AAL. They propose a scenario with 8 microphones distributed in the house. They raise two tests: in the first one they try to identify the speech, showing the detection distress in this scenario; in the second one, they try to identify daily activities as sleeping, resting, going to the toilets, etc. showing better results. However the main conclusion is that a real scenario includes problems as the discrimination of sounds (several events happen at the same time) introducing a high complexity to the analysis. Lombardi et al. (Lombardi, Ferri et al. 2009) present a wearable wireless sensor for fall detection based in a MEMS tri-axial accelerometer, ZigBee communication and a FPGA. The device processes sensor information in real time and when a fall is detected sends an alarm through a ZigBee network to the gateway, which is connected to internet. In the same line, Selvabala et al. (Selvabala, Ganesh 2012) propose a wireless sensor with an accelerometer which is complemented with a Passive Infrared sensor (PIR) in order to detect falls. Jara et al. (Jara, Zamora et al. 2011) propose a system for diabetes therapy. This system is based on a glucometer that uses 6LoWPAN protocol and a RFID reader in order to identify the user. This information is used to adjust the dosage of insulin in a personalized way. Park and Kautz (Park, Kautz 2008) propose a system that detects several activities as walking or preparing cereals. It is based on a RFID bracelet, which enables the detection of several labelled items, and cameras. Ibarz et al. (Ibarz, Bauer et al. 2008) propose a smart wireless sound sensor that placed in the sink is used to quantify the water flow and to identify the activity related with its consumption. 2 Actuator in this context is understated as a device that can be used to control the different components of the sensor node (sensor parameters, etc.) Chapter 2: State of the art -22Interoperability Interoperability between devices implies capacity to exchange data and understand the information embedded. Communication standards ensure stack layer’s interoperability between devices in the same network sharing the same protocol; i.e. network management and maintenance, security, data exchange, etc. Nevertheless, if application layer is not defined, devices will not understand among them unless they were previously agreed between developers; i.e. information understanding. This is the case of 6lowPAN, WIFI, RFID or 3G. Bluetooth and ZigBee go one step further in interoperability, defining profiles and device objects within application layer. Bluetooth defines profiles (hands‐free, health device, human‐interface device, etc.) corresponding to vertical applications. For example, a monitoring infrastructure with Bluetooth microphones streaming audio could be build according to hands‐free profile; no matter their manufacturer, any certified host compliant with the profile will play the audio gateway role without any additional programming. ZigBee not only defines vertical profiles (home automation, smart energy, healthcare, etc), also define horizontal clusters to specify how the devices must exchange application data attending their functionality. For example, every ZigBee compliant temperature sensor must implement “measurement & sensing cluster” and any other device in the network (e.g. a thermostat) would be able to get temperature information as defined in the cluster specification. Additionally, the standard defines how to create virtual bindings among devices allowing instantiation of intelligence in the network; e.g. program automatic triggering of actions between nodes. ZigBee Application Profiles Bluetooth profiles Released specifications ZigBee Home Automation ZigBee Smart Energy 1.0 ZigBee Telecommunication Services ZigBee Health Care ZigBee RF4CE - Remote Control Etc. Advanced Audio Distribution Profile (A2DP) Basic Printing Profile (BPP) File Transfer Profile (FTP) Health Device Profile (HDP) Hands-Free Profile (HFP) Human Interface Device Profile (HID) Headset Profile (HSP) SIM Access Profile (SAP, SIM, rSAP) Synchronization Profile (SYNCH) Video Distribution Profile (VDP) Etc. Table 1 Example of ZigBee and Bluetooth profiles IEEE1451 (IEEE 1451 2011) is a network‐independent specification that uses Transducer Electronic Data Sheet (TEDS) to describe a set of communication interfaces for connecting transducers (sensors or actuators) to microprocessors, instrumentation systems and control/field networks. TEDS is kept in sensor’s memory storing its relevant data (identification, calibration, measurement range, etc.). IEEE1451 provide a set of interfaces that enable that a wired or wireless sensor could be accessed. Thus, theoretically is the best suited option to assure interoperability, because it provides independence from the communication protocol. Nevertheless, its penetration in real applications is very limited, with little compatible hardware available and mainly relegating it to the academic and electronic instrumentation field. Kitchen appliances The kitchen appliances are a cornerstone of the environmental context of the kitchen: person needs to interact with them to carry out any routine activity. Therefore, they could be considered as devices embedding sensors, actuators and simple interfaces inside the kitchen context. When I started working on this thesis, the Universidad de Zaragoza -293.2. RELATED WORK This approach builds on the ICF and the models of the human-machine interaction. This section provides a little background information as well as a summary of the related researches. 3.2.1. USER REQUIREMENTS FOR SPECIFIC USERS GROUPS As mentioned above, knowing the users and understanding their needs and capabilities are fundamental in any design process. User models are a common technique which eases this process. A model is any representation of the potential user, created by or available to the designer, to assist him/her in making predictions about the actual user (Hasdoǧan 1996). These models are built on end user data which could be provided from different sources (statistical, capabilities databases, inquiries, surveys, etc.) (Van Isacker, Goranova-Valkova et al. 2008). It is relevant how the work done in the generation of capabilities databases in the last years is encouraging the development of inclusive design tools (Johnson, Clarkson et al. 2010, Gyi, Sims et al. 2004). For example, HADRIAN (Porter, Case et al. 2004) is a CAD design tool which enables automatic evaluation of the use of a product or service by person of their database; the Exclusion calculator which estimates the number of people who would be excluded from using a particular product (Clarkson, Coleman et al. 2007). Also, USERfit provides a methodology and toolkit for collating design material. USERfit has been designed for improving the design of assistive products, being applicable to the inclusive design (Poulson, Richardson 1998, EDeAN 1999). Other common design methodologies are applied to understand the capacities of specific users groups as the contextual inquiry (Beyer, Holtzblatt 1997), survey (Mikkonen, Väyrynen et al. 2002, Beecher, Paquet 2005), tasks analysis (Sangelkar, Cowen et al. 2012), focus group (Morgan 1997) or Delphi technique (Martin, Norris et al. 2008). 3.2.2. INTERNATIONAL CLASSIFICATION OF FUNCTIONING (ICF) According to the WHO, the ICF is a framework for measuring health and disability at both individual and population levels that allows the assessment of functioning at the level of the whole human being, in day-today life (World Health Organization 2001). The ICF is promoted and officially endorsed by all 191 WHO Member States as the international standard to describe and measure health and disability. This multipurpose tool is used in sectors as diverse as education, health, social policy and general legislation development, statistics or economics. ICF provides a universal model and taxonomy to describe different levels of functioning and disability, improving the communication across disciplines. By using a universal language, ICF enables comparison of results between, for example, people from different countries, different health systems or studies between population samples. It is a fact that two people with the same level of health (e.g. same Alzheimer stage) may have different functions and disabilities in a specific situation (e.g. operating the television). This evidence prevents the determination of the degree of disability of a person only from the knowledge of the condition being treated. ICF provides a classification of function and disability itemized into three lists called “Body Functions”, “Body Structures” and “Activity and Participation”. Furthermore, to classify contextual factors, ICF offers another two lists called “Environmental Factors” and “Personal Factors”. Each domain is structured into chapters where each item can have up to four levels of depth. Thus, the ICF offers about 1500 descriptors in its taxonomy. Chapter 3: Study of the needs of the target population in the kitchen -30ICF Functioning and Disability Contextual Factors Body function and structures Activities and participation Environmental factors Personal Factors b. Body functions s. Body structures d. Activities and participation e. Environmental factors b1. Chapter 1. Mental functions b2. Chapter 2. Sensory functions b210-b229 Seeing and related functions b2100 Visual acuity of distant vision b21999 Binocular acuity of distant vision ... b230-b249 Hearing and vestibular functions ... b3. Chapter 3. Voice and speech functions ... Figure 3 ICF structure This classification is so exhaustive that its daily use usually becomes too complex; thus, many professionals only use a subset of the ICF. Being aware of this fact, WHO has developed several tools to facilitate its use such as the ICF Checklist (World Health Organization 2003). However, it was found that these tools still are too general in some cases. This need has motivated the creation of the ICF Research Branch that develops, evaluates, and disseminates tools and models of functioning and health for different groups of patients and settings (Fayed, Cieza et al. 2011). Besides its main use in the health and rehabilitation sector (Kiltz, van der Heijde et al. 2011, Xu, Kohler et al. 2011), ICF taxonomy has been used for many different uses within Assistive Technology (AT); such as outcome research (Lenker, Paquet 2003), to describe the user activities related to consumer products (Sangelkar, Cowen et al. 2012) or for modelling the selection of AT (Scherer, Jutai et al. 2007). 3.2.3. HUMAN DEVICE INTERACTIONS MODELS Ergonomic (or human factors) is defined by the International Ergonomic Association as the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance (International Ergonomics Association 2011). This discipline is strongly related with other younger disciplines as the Human Computer Interaction (HCI) which born in the 1980’s as a generalization of the Human Machine Interaction (HMI) due to the increase of research works in the different aspects of the interaction between human and computer (Dix, Finlay et al. 2004). The interaction models used in HCI can be applied in a general way to the study of the interaction between human and devices, understanding device as a product, system or service. Several authors have modelled this interaction being the Norman’s action-circle (Norman 1988) one of the most influents (Dumas, Lalanne et al. 2009, Dix, Finlay et al. 2004). The action-circle defines the interaction in two steps: the execution and the evaluation. The execution involves doing something and the evaluation is the comparison between what really happened and what we wanted to happen in the world by performing the action (our goal). Norman defines seven stages in the interaction: 1. Establishing the goal. 2. Forming the intention. Universidad de Zaragoza -313. Specifying the action. 4. Executing the action. 5. Perceiving the world state. 6. Interpreting the state of the world. 7. Evaluating the outcome (i.e. the system state with respect to the goals and intentions) The Interaction Framework proposed by Abowd and Beale (Abowd, Beale et al. 1991) goes one step further in this direction including the system in the model and proposing a more realistic approach. They define four main components in the interaction: the user, the system, the inputs and the outputs. Each one has its own language which is used to express its purpose in the interaction. For Abowd and Beale Input and Output, together, compose the system interface. As Figure 4 shows, there are four steps in the interaction: articulation, observation, performance and presentation. Abowd describes this interactive cycle in his PhD Thesis (Abowd 1991) of this way: “User interactive cycle began with the formulation of a goal and task to achieve that goal. The only way the user can manipulate the machine is through the Input, and so the task must be articulated within the input language. The input language is translated into the core language as operations to be performed by the System. The System then transforms itself as described by the operation translated from the Input; the execution phase of the cycle is complete and the evaluation phase now begins. The System is in a new State, which must now be communicated to the User. The current values of system attributes are rendered as concepts or features of the Output. It is then up to the User to observe the Output and assess the results of the interaction relative to the original goal, ending the evaluation phase and, hence, interactive cycle.” O I Output Input U S core task presentation observation performance articulation Figure 4. PhD. Abowd extract (Abowd 1991) This concept has been generalized for multimodal interfaces. For example, Dumas, Lalanne et al. propose a model which takes as starting point the Norman’s action circle considering multimodal inputs and outputs (Dumas, Lalanne et al. 2009). Next figure summarize this model: Chapter 3: Study of the needs of the target population in the kitchen -32Figure 5 Multimodal interface interaction example (Dumas, Lalanne et al. 2009) Considering the Interaction Framework as starting point, next sections present a methodology which combines the user modelling and tasks study with the ICF lexicon. It provides a new tool to analyse the interaction between user and device which detects the key aspects in order to include the needs of a specific population in the design process. 3.3. USERS’ NEEDS IDENTIFICATION METHODOLOGY 3.3.1. RESEARCH APPROACH This methodology offers a systematic way to identify needs in the design of a product or system for a specific collective as elderly or disabled people. It is based in the study of the potential limitations these users have performing the tasks that will be supported by the new product as currently built, with current technology. To achieve this objective the interaction is studied following the model proposed by Abowd and Beal, and using the ICF language in order to describe human capacities and actions. As this model shows, both Input and Output, compose the system interface. Focusing in the user point of view of the interaction, we find that, as first step, the user formulates a goal and the tasks to achieve that goal. This implies that the user has been able to understand the stimuli that perceives, to process the related information and to produce a response according to the context and the interaction objectives. Then, the only way that the user has to interact with the system is using one of its inputs (microphone, keypad, touch screen, gesture, etc.). This step implies that the user has been able to produce a response that the system can perceive (voice, movement, pulsation, etc.). Once the system responds, the user must be able to perceive and interpret the outputs and assess the results of the interaction versus the original goal; same way than in the first step when the user needed to understand and process the stimuli and to produce a response according to the context and the interaction objectives. Thus, it is possible to conclude that the critical user’s capacities for the interaction are related with: cognition, sensory, physical (movement, pulsation, etc.) and voice. The methodology here presented defines three sequential phases with clearly differentiated objectives: Universidad de Zaragoza -33i. Users’ characterization: aims to identify which person's capabilities could be typically affected as consequence of belonging to a specific collective. Besides generic aspects, this focus in the critical user’s capacities for the interaction. ii. User-device interaction characterization: aims to describe the interaction of the user performing the tasks that will be covered with the new product or system as now done. iii. Identification of users’ needs: aims to extract the user needs derived from each task involved in the user-device interaction characterization taking into consideration the users’ characterization. 3.3.1.1. USERS’ CHARACTERIZATION Users’ characterization is performed in two phases; first, research about the specific capacities and limitations in specialized literature and statistics, provides a general overview of the target population. Many sources are available such as the Survey of Health, Ageing and Retirement in Europe (SHARE), EUROSTAT or any other source (Van Isacker, Goranova-Valkova et al. 2008, Tenneti, Johnson et al. 2012). This research is focused in the skills needed to perform a proper user-device interaction according to framework interaction model: how the person can receive information from the machine through the system output (senses), how the person can command the machine through the system input (hands, voice, gaze, etc.) and how the person can understand the information perceived and reason accordingly (cognitive capacities needed to propose a goal and to evaluate the results of the actions performed). In the second phase, this research is then embodied using the ICF taxonomy descriptors from the epigraph b. Body functions classification. The body functions that influence human-machine interaction and which could be affected as consequence of the elderliness, disease or disability the person might, are described by the ICF the chapters: b1. Mental functions, which is about the functions of the brain: both global mental functions, such as consciousness, energy and drive, and specific mental functions, such as memory, language and calculation mental functions. b2. Sensory functions and pain, which is about the functions of the senses, seeing, hearing, tasting and so on, as well as the sensation of pain. b3. Voice and speech functions, which is about the functions of producing sounds and speech. b7. Neuromusculoskeletal and movement-related functions, which is about the functions of movement and mobility, including functions of joints, bones, reflexes and muscles. A set of four tables (one per ICF chapter) indicating cognitive, sensorial, speech and movement-related functions is the outcome of this phase. Each table has two columns: first describing the problems, limitations, capacities of the target population according to literature and second listing the ICF descriptors associated to the items in first column. These descriptors are a set (named C, from characterization) which characterize the body functions which typically could affect to a person included in the target population i.e. { } being with a descriptor from the Body functions classification of the lists previously commented. Users’ characterization should be performed by a multidisciplinary group involving social workers, health professionals and other personnel related (caregivers, etc.) who could provide additional information about this characterization. For example, C = { b2100 Visual acuity functions, b2102 Quality of vision,…, b140 Attention functions, b710 Mobility of joint functions, b720 Mobility of bone functions, b730 Muscle power functions } Chapter 3: Study of the needs of the target population in the kitchen -343.3.1.2. USER-DEVICE INTERACTION CHARACTERIZATION This characterization is done studying how a person currently performs the tasks which will be supported by the new device or system. In this scope a task is understood as a set of actions to reach a goal. These tasks have been analysed considering the human capacities needed to perform them using the ICF epigraph d. Activity and participation. Tasks identification can be done through direct observation, by the study of the process, using flow diagrams or any other systematic technique. However, note that the more rigorous this process is, the better needs identification will be. As we said earlier, ICF taxonomy is very detailed and not all the chapters within the epigraphs are always applicable. According to the specific objectives of the methodology and the interaction model, we find this relevant: - d1. Learning and Applying Knowledge, which is about cognitive processes required for learning, applying the knowledge that is learned, thinking, solving problems, and making decisions. - d2. General tasks and demands, which is about general aspects of carrying out coordinated actions related to a task; i.e. initiating a task, organizing time, space and materials for a task, pacing task performance, etc. - d3. Communication and d4.Mobility, which are about how the person implements interaction with the device; receiving and producing messages, changing body position, carrying, moving or manipulating objects, etc. As result of this phase we obtain a set of tasks (named T, from tasks): { } being with the independent tasks which describe the usual interaction between user and device. Additionally, each one of these tasks is in turn described in ICF language, as a set of activities from the d. Activity and participation list i.e. { } being with a descriptor from the lists previously commented. Sometimes, it could be useful to break down the task in simple subtasks in order to ease the descriptor identification. As an example, we will apply this step to the design of a can opener. In this case, the only one task done by a can opener is to “open a can”. This process has been broken down in subtasks, taking as starter point the Activity-diagram proposed by Sangelkar (Sangelkar, Cowen et al. 2012) and related with the capabilities required according to ICF: Subtask involved Activity descriptor set Pickup the can opener and hold it in one hand Twist the handle with the other hand Import and position the can to be opened Engage the can opener with the can Remove the lid d110 Watching d160 Focusing attention d2100 Undertaking a simple task d440 Fine hand use d445 Hand and arm use d449 Carrying, moving and handling objects, other specified and unspecified Table 2 Can opener interaction Universidad de Zaragoza -35Sometimes, when the study is focused on the interaction, the environmental factors could be omitted considering an “ideal” environment which has not influence over it (this is the case of the previous example). However, when the new product or system will work in known and different environments which may condition the interaction, this study should be done in each scenario. In order to assure coherence with the rest of the methodology, these environmental situations should be described using the descriptors provided by the ICF’s e. Environmental factors list. Each situation is described by a set of environmental descriptors ( { } being with a descriptor from the lists previously commented). For example, the environmental factors vary if there is not enough light in the room to see. This phase should be performed by a multidisciplinary group involving technicians (product designer, ergonomics, etc.), social worker and other personnel related (caregivers, etc.) who could provide additional information. 3.3.1.3. IDENTIFICATION OF USERS’ NEEDS This phase combines the previous phases to thoroughly extract the user needs derived from each task involved in the user-device interaction characterization (3.3.1.2) and taking into consideration the users’ characterization (3.3.1.1). The needs identification is an iterative process which takes as starting point the set of tasks obtained in the previous step. For each of these tasks, the next process follows: - The task is described as { }, being the task and { } the set of ICF descriptors. - Our target population is described by a characterization set ( { }). - Then, if there is a relation between a task descriptor and one or more components of the set, it indicates that a need could exist (i.e. if ). If environmental factors are considered, i.e. the tasks set is composed by only one task T={Opening a can} and it could be described according ICF as a set of activities: Opening a can = {d110 Watching, d160 Focusing attention, d2100 Undertaking a simple task, d440 Fine hand use, d445 Hand and arm use, d449 Carrying, moving and handling objects, other specified and unspecified} Thus, a user who can perform these activities will be able to perform the task. e.g. as it has been commented, a can opener has an only one task characterized by the next descriptors: Opening a can = {d110 Watching, d160 Focusing attention, d2100 Undertaking a simple task, d440 Fine hand use, d445 Hand and arm use, d449 Carrying, moving and handling objects, other specified and unspecified} e.g. C = {b2100 Visual acuity functions, b2102 Quality of vision,…, b140 Attention functions, b710 Mobility of joint functions, b720 Mobility of bone functions, b730 Muscle power functions} This means that a user included in the target population could have affected any of these functions. Chapter 3: Study of the needs of the target population in the kitchen -36they must be also related with the task descriptors for each situation (i.e. if )) This is called indicator and indicates that the user could have problems to perform this action (d110 Watching) and the need or needs detected will be conditioned by the task. - Each indicator must be studied in order to detect the need. When an indicator ( ) is included in several tasks, although the indicator is the same because the relation is independently of the task ( ), the needs detected could be different for each task. Following with the example, each time that a task includes the descriptor d110 Watching with the current population objective, an indicator will appear due to the relationship between d110 Watching and the descriptors b2100 Visual acuity functions and b2102 Quality of vision. However the need detected is different for the task “open a can” that, for example, “looking for a specific can”. As result, we have a set of indicators (Iti) and needs (Nti) associated to each tasks (ti) which have been represented in a tabular format. Each table, one per task, has three columns indicating: a) list of descriptor of the activities (according to ICF); b) indicators (Iti, relation between activities and body functions implicated according to ICF); c) list of needs detected (Nti). These tables provide an extensive and reasoned perspective about the needs to be tackled in the design of the product for the target population of study. They also provide valuable information about the indicators to be used in latter product evaluation. e.g. d110 Watching is related with b2100 Visual acuity functions, b2102 Quality of vision. i.e.: d110 Watching = f(b2100 Visual acuity functions,… , b2102 Quality of vision) e.g. Need detected: People with visual impairments may have troubles positioning the can and the opener and to determine when the can is already open. Following with the study of the can opener for the example target population we would obtain the following table: Task Opening a can Capabilities required Indicators Needs detected d110 Watching d110 related to b2100, b2102 Although people with visual impairments could use it, they may have troubles positioning the can and the opener and to determine when the can is already open. d160 Focusing attention d2100 Undertaking a simple task d160, d2100 related to b140 People with cognitive problems may have troubles following the process (understand the process, injury, etc.) and may require support to do the same. d440 Fine hand use d445 Hand and arm use d449 Carrying, moving and handling objects, other specified and unspecified d440, d445, d449 related to b720, b730 Heavy cans can be a problem hindering the use of opener. The realization of precise movements and performing hand turns can be a problem. Table 3 Identification of users' needs for the task: "Opening a can" Universidad de Zaragoza -37This methodology formalizes and systematizes the process of looking for needs’ identifiers and must be performed by an expert team in order to set the need; ideally this team should be composed by the professionals in the first and second phases of the process. dp Indicator d3 = f(b3,b4,e2,e3) b1b2 b3b4 bn Enviromental Situation j (Esj) C, target population characterization Task i (ti) ANALYSIS Multidisciplinary team NEEDS e2e3 e4 en e1 d3 d2 d1 Figure 6 Example of needs identification for one task (ti) in a specific situation scenario (Esi) 3.4. LOOKING FOR NEEDS IN THE DESIGN OF AN AAL KITCHEN 3.4.1. USERS’ CHARACTERIZATION Characterization of elderly people, target of current study, takes as statistical source the report of the IMSERSO (Agency of the Government of Spain for the management of programs and services for elderly and handicapped people) (IMSERSO 2009). This report presents a thorough analysis of the situation of the elderly people in Spain. It also includes detailed information about the health status of the elderly people, main illness and percentages of people affected. As said earlier, this characterization is represented in four tables following the epigraphs of the ICF. First column specifies the main illnesses which can affect the elderly people together with the incidence rate over the total of the population. Second column identifies the ICF descriptors that could be affected by the illness. This results in a characterization of the elderly people based in the standard language proposed by the ICF and showing the body functions which could be affected by the ageing. Chapter 3: Study of the needs of the target population in the kitchen -38Diseases and conditions common in elderly people Related ICF descriptors b1.Mental functions Around 20% of the population over 65 in Spain has some cognitive disorder (IMSERSO 2009, Bartrés-Faz, Clemente et al. 1999, Wikipedia 2013, Casas, Blasco et al. 2009): The aging brain is characterized by some degree of natural decline of cognitive functions like memory, visual-spatial skills and speed of information processing. Senile dementia is the progressive loss of cognitive functions, because of damage or brain disorders. Typically, this cognitive impairment causes inability to perform activities of daily living. Cognitive deficits can affect any of brain functions, particularly the areas of memory, language (aphasia), attention, visualconstructive skills, the praxis and executive functions as troubleshooting or response inhibition. Alzheimer is the most common form of dementia. In the early stages, the most common symptom is difficulty in remembering recent events. As the disease advances, symptoms can include confusion, irritability and aggression, mood swings, trouble with language, and long-term memory loss. As the sufferers decline they often withdraw from family and society. Gradually, bodily functions are lost, ultimately leading to death. Parkinson's disease is a chronic neurodegenerative disorder that leads eventually to a progressive disability, produced as a result of destruction of the pigmented neurons of the substantia nigra. Parkinson's disease, as well as movement disorder also triggers alterations in cognitive function in the expression of emotions, speech and autonomic function. b114 Orientation functions: General mental functions of knowing and ascertaining one's relation to self, to others, to time and to one's surroundings. b117 Intellectual functions: General mental functions, required to understand and constructively integrate the various mental functions, including all cognitive functions and their development over the life span. b140 Attention functions: Specific mental functions of focusing on an external stimulus or internal experience for the required period of time. b144 Memory functions: Specific mental functions of registering and storing information and retrieving it as needed. b147 Psychomotor functions: Specific mental functions of control over both motor and psychological events at the body level. b156 Perceptual functions: Specific mental functions of recognizing and interpreting sensory stimuli: b1560 (Auditory perception), b1561 (Visual perception), b1562 (Olfactory perception), b1563 (Gustatory perception), b1564 (Tactile perception), b1565 (Visuospatial perception) b160 Thought functions: Specific mental functions related to the ideational component of the mind. b164 Higher-level cognitive functions: Specific mental functions especially dependent on the frontal lobes of the brain, including complex goal-directed behaviours such as decision-making, abstract thinking, planning and carrying out plans, mental flexibility, and deciding which behaviours are appropriate under what circumstances; often called executive functions. b167 Mental functions of language: Specific mental functions of recognizing and using signs, symbols and other components of a language b176 Mental function of sequencing complex movements: Specific mental functions of sequencing and coordinating complex, purposeful movements. Table 4. Cognitive diseases and conditions common in elderly people Universidad de Zaragoza -453.4.2.2. COOKING To study the interaction, next appliances have been chosen: The oven has three retractable rotary control knobs. The first one, placed on the top-left, is a timer with switch off function, which enables turn off the oven after a maximum of 120 min. The next one is a function selector which sets the type of heating (Top/bottom heating, large grill area, small grill area, bottom heating and oven light). The last one is a temperature selector (50270°C) or grill power selector (grill power I, II, III, depending on the function selector option). The door has a big handle placed on the top (see Figure 8). Also, it has a front glass that together with the interior light permits to monitor the food state. It has three racks which enable to put the tray or shelf in three different heights. Also the oven could be mounted at different heights in order to simply the access. The hob has four ceramics fires which are controlled through a touch control (Figure 9). Each fire has associated two buttons and a display which permit to select the power level from 0 (off) to 9 with 11 levels 1, 1., 2, 2., ..., 96. The top left fire has two diameters selected by a specific button. Also the hob has a turn on/off button which handles the four fires. When the hob is off and a fire is hot, the level indicator display an “H” or “h” depending on the temperature (H= high temperature, h=moderately high temperature). Also, it has a block function in order to prevent its use by children and a water warning which turns off the hob when the touch control detects water over it. The microwave has a power of 800 W with an interface composed by 2 rotary buttons that enable to select the power and the time, and a push button to open the door. Buttons are located in column at the right as the picture shows (Figure 10). Focusing in the study of the interaction, it is evident that the preparation of a specific dish can be very complex and different from another. However, leaving aside the use of kitchen tools (knives, bowls, dishes, etc.) to prepare the food to be cooked (remove packaging, slicing, cutting, etc.) and focusing in the interaction between users and white goods, the number of actions the user does is limited (place a recipient, select the fire level, monitor the process, etc.). Also, tasks related with the food management have not been considered due to they are explained in the previous section. Therefore, the user must be able to perform the next tasks in order to interact with the hob, oven and microwave:  Tasks associated with the food management (see previous section)  Preparation of food to be cooked  Placing/removing the recipient (over the hotplate or into the oven)  Opening/closing door (oven and microwave)  Configuring/Programing and monitoring (oven, hob and microwave)  Maintenance and cleaning Each one of these tasks (excluding transversal which are analysed in the section 3.4.2.4) has been described in ICF language, as a set of activities from the d. Activity and participation: Figure 8 Oven selected Figure 10 Microwave selected Figure 9 Hob selected Chapter 3: Study of the needs of the target population in the kitchen -46Preparation of food to be cooked Task: Preparation of food to be cooked This task includes the different subtasks related with the food preparation in order to be cooked. Note that only has been considered basic subtasks so, in order to elaborate complex recipes, it could be needed to add new ones, increasing consequently the user’s capabilities required. Subtask involved Capabilities required (according to ICF) Removing packaging Adding ingredients to the recipient Cutting Slicing Mixing d110 Watching d160 Focusing attention d175 Solving problems d177 Making decisions d2202 Undertaking multiple tasks independently d440 Fine hand use d445 Hand and arm use d4154 Maintaining a standing position Table 12 Preparation of food to be cooked characterization Placing/removing the recipient (over the hotplate or into the oven) Task: Placing/removing the recipient This task includes the tasks related with the access inside the appliance (oven and microwave) and to put/leave the recipient (hob, oven and microwave) over the hotplate. Door opening and close has been studied separately. Subtask involved Capabilities required (according to ICF) To access to inside of the appliance To decide where/how to put/leave the recipient Placing/removing the recipient into the oven / on the fire d110 Watching d177 Making decisions d4101 Squatting d4105 Bending d4300 Lifting d4301 Carrying in the hands d445 Hand and arm use Table 13 Placing/removing the recipient characterization Opening/closing door (oven and microwave) (See section 3.4.2.4) Configuring/Programing and monitoring (oven, hob and microwave) (See section 3.4.2.4) Maintenance and cleaning (See section 3.4.2.4) Universidad de Zaragoza -473.4.2.3. WASHING The processes of washing clothes and dishes, as discussed below, have several similarities. There are many tasks common for both processes. However, doing the laundry is much more complex from a cognitive point of view due to the large number of possibilities presented in the different steps required for using the washing machine (difficulty in separating the clothes, more programs and settings, etc.) In addition, misuse of the machine can cause the destruction of the clothing being washed while in the dishwasher this situation is unusual. A frontal load washer has been selected in order to study the user interaction. It has a detergent drawer on the top left, a rotary control knob to select the washing program, three status indicators (start, wash, rinse, spin, end) a flot button which enables to postpone the spin process, a button to select the spin speed (600rpm/1000rpm) and a start button. The door is placed in the middle of the frontal and has a handle placed on the right (see Figure 11). The dishwasher has an interface placed on the top of the door (see Figure 11). It is composed (from left to the right) for an on/off button, a handle (to open the door) a rotary control to program select with a start button placed in the middle and a set of indicators for salt refill, water supply and rinse refill, dry display and cleaning display. Figure 11 Dishwasher and washing machine selected Studying the process for washing of the clothes, the user has to organize and classify the laundry, taking into account the type of fabric, colour and recommended washing program. Then, user has to open the washing machine, load the selected clothes inside and close the door. Once the machine is ready, the user has to add detergent and softener; selecting and start the appropriate program. Occasionally, the user can monitor the process to see if the washer has finished. When it finished, user has to open the door and unload it. For the dishwashing, the user has to prepare (rinse food leftovers), organize and classify the crockery taking in to account that some pieces are not suitable for the dishwasher. Then, the user has to open the door, to load the crockery, add the detergent and close its door. Once the dishwasher is ready, the user has to select a program to start the washing process. Occasionally, the user can monitor the process to see if the washer has finished. When the dishwasher has finished, the user has to open the door and unload the crockery. Therefore, the user must be able to perform the next tasks in order to interact with the washer and dish washer:  Organization and classification of clothes or crockery  Opening/closing door Chapter 3: Study of the needs of the target population in the kitchen -48-  Loading and Unloading clothes or crockery  Measuring and adding detergent, softener and others  Configuring/Programing and monitor  Maintenance and cleaning Each one of these tasks (excluding transversal which are analysed in the section 3.4.2.4) has been described in ICF language, as a set of activities from the d. Activity and participation: Organization and classification of clothes or crockery: Task: Organization and classification of clothes or crockery This task includes the different subtasks related with the organization and classification of clothes (grouping, classify, unbutton, etc.) and crockery (rinse dishes, etc.) and their handling in order to use the washer and dish washer. Subtask involved Capabilities required (according to ICF) Grouping of clothing/crockery Classifying clothing/ crockery Execution of tasks prior to loading washing machine/ dishwasher (unbutton, turning trousers inside out, rinse dishes, etc.) d110 Watching d160 Focusing attention d1750 Solving simple problems d177 Making decisions d2100 Undertaking a simple task d4300 Lifting d4301 Carrying in the hands d440 Fine hand use Table 14 Organization and classification of clothes or crockery characterization Opening/closing door: (See section 3.4.2.4) Loading and Unloading clothes or crockery: Task: Loading and Unloading clothes or crockery This task is focused exclusively on the load and unload of clothes or crockery in the corresponding appliances Subtask involved Capabilities required (according to ICF) To access inside of the appliance Putting items into the appliance Taking items out of the appliance d430 Lifting and carrying objects d4105 Bending d4154 Maintaining a squatting position d445 Hand and arm use Table 15 Loading and Unloading clothes or crockery characterization Universidad de Zaragoza -49Measuring and adding detergent, softener and others: Task: Measuring and adding detergent, softener and others This task covers all the process of measuring and adding detergent and others products including the physical, cognitive and sensorial processes. Subtask involved Capabilities required (according to ICF) Opening the tray/detergent dispenser Picking up detergent or other products Measuring and pouring product into tray/detergent dispenser Closing tray/detergent dispenser d110 Watching d160 Focusing attention d1750 Solving simple problems d177 Making decisions d2100 Undertaking a simple task d430 Lifting and carrying objects d440 Fine hand use d445 Use of hand and arm Table 16 : Measuring and adding detergent, softener and others characterization Configuring/Programing and monitoring (oven, hob and microwave) (See section 3.4.2.4) Maintenance and cleaning (See section 3.4.2.4) Chapter 3: Study of the needs of the target population in the kitchen -503.4.2.4. TRANSVERSAL TASKS This section includes the analysis of the transversal tasks: Opening/closing door: Task: Opening/Closing door This task studies the opening and closing users’ capabilities required for all the appliances studied which have a door (refrigerator, oven, microwave, washer and dish washer). Subtask involved Capabilities required (according to ICF) To access to the door Grasping and opening the door Closing the door d1750 Solving simple problems d440 Fine hand use d445 Use of hand and arm Table 17 Door opening and closing characterization Configuring/Programing and monitoring: Task: Configuring/Programing and monitoring This task studies the configuring and programing users’ capabilities required for handle refrigerator, hob, oven, microwave, washer and dish washer. Subtask involved Capabilities required (according to ICF) Selecting a programme Initiating the programme Monitoring the programme Stopping the programme d110 Watching d160 Focusing attention d175 Solving problems d177 Making decisions d2202 Undertaking multiple tasks independently d440 Fine hand use d445 Hand and arm use d4154 Maintaining a standing position Table 18 Configuring/Programing and monitor characterization Maintenance and cleaning: Task: Maintenance and cleaning This task include maintenance and cleaning of all the appliances studied Subtask involved Capabilities required (according to ICF) d110 Watching d115 Listening d120 Other purposeful sensing d160 Focusing attention d175 Solving problems d177 Making decisions d2202 Undertaking multiple tasks independently d440 Fine hand use d445 Hand and arm use d4154 Maintaining a standing position Universidad de Zaragoza -51Table 19 Maintenance and cleaning characterization 3.4.3. IDENTIFICATION OF USERS’ NEEDS The needs identification process has been iteratively applied to each task obtained in the user-device interaction characterization following the process described in the section 3.3.1.3. The result has been summarized in the next set of tables, one by task: Food Classification and organization: Capabilities required (according to ICF) Indicators Needs detected d110 Watching d110 related to b210 People with visual problems may require help to recognize the different products or to read information about them. d160 Focusing attention d1750 Solving simple problems d177 Making decisions d2100 Undertaking a simple task d160, d1750, d177, d2100 related to b114, b117, b140, b144, b156, b160, b167 People with cognitive problems could require help to follow the steps of the process. Additional information about the products may be required to classifying them correctly. d4300 Lifting d4301 Carrying in the hands d440 Fine hand use d4300, d4301, d440 related to b147, b176, b710, b720, b730, b755, b760, b765, b770 Removing the package of some products may require a fine use of the hand. Also, sometimes it is needed to move heavy loads. In both cases, people with mobility problems may require help to carry out the task. Table 20 Needs identification in the ‘food classification and organization’ task Placing and removing items from the fridge or freezer: Capabilities required (according to ICF) Indicators Needs detected d110 Watching d110 related to b210 People with cognitive or visual problems may require help to find the items and to identify them. d160 Focusing attention d1750 Solving simple problems d177 Making decisions d2100 Undertaking a simple task d160, d1750, d177, d2100 related to b114, b117, b140, b144, b156, b160, b167 People with cognitive problems could need help in order to know where is stored each item. d4101 Squatting d4105 Bending d445 Hand and arm use d4101, d4105, d445 related to b147, b176, b710, b720, b730, b755, b760, b765, b770 The highest and lowest locations, both refrigerator and freezer, may present access problems for people with mobility problems. Small doors could increase the problem. Picking heavy load up also is problematic. Table 21 Needs identification in the ‘placing and removing items from the fridge or freezer’ task Chapter 3: Study of the needs of the target population in the kitchen -52Analysing the state of the food: Capabilities required (according to ICF) Indicators Needs detected d110 Watching d120 Other purposeful sensing d110 related to b210 d120 related to b230, b250, b255, b265 In general, aging causes degradation in the senses that may affect a person's ability to detect if the food is in good condition (presbyopia, presbyacusia, etc.) Also people with visual disabilities may require help to know the expired date of the products. d160 Focusing attention d177 Making decisions d160, d177, related to b114, b117, b140, b144, b156, b160, b167 Similarly, people with cognitive problems may have troubles to discern safe food to the expired food (b117, b144, b156). Table 22 Needs identification in the ‘analysing the state of food’ task Preparation of food to be cooked: Capabilities required (according to ICF) Indicators Needs detected d110 Watching d110 related to b210 People with visual disability may require help to handle and to identify some tools and aliments. d160 Focusing attention d175 Solving problems d177 Making decisions d2202 Undertaking multiple tasks independently d160, d175, d177, d2202 related to b114, b117, b140, b144, b156, b160, b164, b167 Usually this task is done concurrently with others and its complexity can change strongly depending on the dish. People with cognitive disability may require help to follow the correct steps, to remember it and to relate this task with the other. Reduce the number of steps and choices may help the users d440 Fine hand use d445 Hand and arm use d4154 Maintaining a standing position d440, d445, d4154 related to b147, b176, b710,b720,b730, b755, b760, b765, b770 Food preparation must not involve stressful motions, heavy weights or unnatural twisting motions affecting joints. However, people with mobility disability may require help to handle some kitchen tools or to maintain a standing position. Table 23 Needs identification in the ‘preparation of food to be cooked’ task Universidad de Zaragoza -53Placing/removing the recipient (over the hotplate or into the oven) Capabilities required (according to ICF) Indicators Needs detected d110 Watching d110 related to b210 People with visual disabilities could also have identification problems. This situation could also be dangerous for the user. d177 Making decisions d177 related to b114, b117, b140, b144, b156, b164 People with cognitive disabilities may require help to understand that hotplates are turned on in the hob. d4101 Squatting d4105 Bending d4300 Lifting d4301 Carrying in the hands d445 Hand and arm use d4101, d4105, d4300, d4301, d445 related to b147, b176, b710, b720, b730, b755, b760, b765, b770 Minimisation of handling and moving weight as well as to place the oven at certain height could facilitate its accessibility. Also a sliding tray could be useful for some people. Table24 Needs identification in the ‘placing/removing the recipient (over the fire or into the oven)’ task Organization and classification of clothes or crockery: Capabilities required (according to ICF) Indicators Needs detected d110 Watching d110 related to b210 People with visual problems may require help to differentiate clothes and crockery. d160 Focusing attention d1750 Solving simple problems d177 Making decisions d2100 Undertaking a simple task d160, d1750, d177, d2100 related to b114, b117, b140, b144, b156, b160, b167 People with cognitive problems may require help to follow the steps of the process. Additional information about the clothes and crockery may be required to classifying them correctly d4300 Lifting d4301 Carrying in the hands d440 Fine hand use d4300, d4301, d440 related to b147, b176, b710, b720, b730, b755, b760, b765, b770 Tasks as unbutton, unzip or rinse dishes, etc. may require a fine use of the hand. Also, to move the laundry in the case of the washing machine, or heavy recipients in the case of the dishwasher, sometimes could be heavy loads. In both cases, people with mobility problems may require help to carry out the task. Table 25 Needs identification in the ‘organization and classification of clothes or crockery’ task Chapter 3: Study of the needs of the target population in the kitchen -54Loading and Unloading clothes or crockery: Capabilities required (according to ICF) Indicators Needs detected d430 Lifting and carrying objects d4105 Bending d4154 Maintaining a squatting position d445 Hand and arm use d430, d4105, d4154,d445 related to b147, b176, b710, b720, b730, b755, b760, b765, b770 Minimisation of handling and moving weight could aid to the people with mobility problems. Also, minimizing the bending or squatting movements and times in these positions may help to these users. Door should be sufficiently large in order to facilitate the easy access to the appliance. Also, guarantee that the edges of appliance are rounded could prevent knocks. Table 26 Needs identification in the ‘loading and unloading clothes or crockery’ task Measuring and adding detergent, softener and others: Capabilities required (according to ICF) Indicators Needs detected d110 Watching d160 Focusing attention d1750 Solving simple problems d177 Making decisions d2100 Undertaking a simple task d110 related to b210 d160, d1750, d177, d2100 related to b114, b117, b140, b144, b156, b160, b167 People with cognitive and visual disabilities may need help to identify the product, to measure it and to know where must be placed. d430 Lifting and carrying objects d440 Fine hand use d445 Use of hand and arm d430, d440, d445 related to b147, b176, b710,b720,b730, b755, b760, b765, b770 People with mobility disabilities may need aid to open and to close the detergent/softener/etc. tray or the recipient where the product is stored. Also these people may have problems with heavy recipients. Minimizing the number of times it is needed to add detergent/softener/etc. will lead to all these problems decreasing. Table 27 Needs identification in the ‘measuring and adding detergent, softener and others’ task Universidad de Zaragoza -61CHAPTER 4: ARCHITECTURE DESIGN Chapter 4 presents a deployment proposal for an Ambient Intelligence in the kitchen with the objective of fulfilling the elderly people needs identified in previous chapter; namely an Ambient Assisted Living environment. First, the proposed system is described explaining its interaction with the user and context. Next, software architecture is detailed describing the functionality of its main blocks 5 . 4.1. SYSTEM DESCRIPTION The analysis of the interaction between elderly people and kitchen appliances led to determine that proposed AmI kitchen should provide three new functionalities:  To facilitate the use of household appliances and to provide useful information and warnings about the use of household appliances.  To detect emergency situations and to take corrective actions when needed.  To analyse all the data gathered to extract relevant information that could be useful for the user’s carers and/or relatives in order to evaluate the person’s quality of life. The e-servant interfaces with the use by means of familiar elements such as the TV and smart phones, adapted interface devices Broadband connection allows remote monitoring and manage of the eservant. Data for QoLE system is also reported QoLE Information about the context is retrieved by means of sensors and RFID readers, which communicate with the eservant e-servant INTERNET PLC WiFi Blueooth ZigBee Home appliances communicate with the e-servant though the mains. They integrate also RFID readers for product tracking Figure 12 AmI kitchen system 5 System description (section 4.1.) is included to provide a global vision of the complete system; it is a summary of the deliverable “D.3.2. Design, architecture, development and test of e-servant.”(Casas, Blasco et al. 2009). Besides my personal work, this description includes contributions from other project partners. Software architecture (section 4.2.) has been entirely designed within the framework of this PhD thesis work. Chapter 4: Architecture design -62In order to cover these functionalities, it is evident that “intelligence” and user interfaces have to be introduced in the kitchen; nevertheless, this doesn’t mean that each white good has to be smarter or incorporate new adapted interfaces. Considering the current market and state of the art in kitchen appliances, this would increase their unitary price, complicate their installation (adapting the functionality to the user’s particular case requires configuration) and consequently hinder the market penetration. Maybe, in a close future, capacities of the regular smart appliances will enable to develop specific functions without increasing their cost. Thus, instead of having smart appliances with embedded accessible interfaces, a central intelligence entity has been developed: which has been called “e-Servant”. This way, any electrical appliance, user interface or smart device with communication capability can be integrated in the system. As a result, the development and stability of the appliances eases (they don´t change their current way of functioning, they just need to add communication to inform about their status and execute actions). Also, similar to building automation sensors and actuators, a new market sector which reuses the same hardware is identified. Figure 12 shows the proposed system. 4.1.1. CONTEXT INTERACTION In any AmI application, the information from the physical context is essential because it constitutes the input for the logical rules and decision processes that build services. According to Feng, Apers et al. context categorization (Feng, Apers et al. 2004), the world is divided in two different contexts that interact with the AmI system: environment (environmental context) and people (person-centric context). From this ample taxonomy, white goods and different sensors are considered as the main context information sources within the kitchen scenario. As already mentioned, electrical appliances must be able to report their status and be remotely operated by the system or by the user through the built-in interface. Thus, any appliance with communication capabilities can be integrated in the system. The specific implementation uses BSH’s Serve@Home solution to control and monitor the appliances over PLC (Power Line Communications). RFID technology 6 is increasingly more integrated in our daily lives. It is expected that many goods will store information about their expiry date (food), washing instructions (garments) or dosage (medicines). As conventional white goods do not provide means to retrieve all the information needed, RFID readers with ZigBee communication are integrated to enhance the capacities of the fridge and washing machine. Also, as food is not just stored in the fridge and it is not feasible to put readers in every cupboard, a stand-alone RFID reader to gather information about any specific item is also developed. Standard security sensors commonly used inside kitchens (gas, fire, smoke, flooding) are also included to detect emergency situations. Additionally, other sensors not so commonly used in this scenario are integrated; for example, magnetic sensors —to detect when the user opens/closes a cupboard or drawer—; light sensors —to detect when the user forgets the lights on—; presence sensors —to detect when the user enters the kitchen—. All these sensors as well as RFID readers use ZigBee as the wireless standard for home control and automation that easily allows adding new devices to the system. 6 RFID technology has been provided by Gis Gera Ident-Systeme GmbH Universidad de Zaragoza -634.1.2. USER INTERACTION The purple frame in Figure 12 represents the human-machine interface (HMI) that manages user interaction 7 . HMI devices must be usable and accessible for any kind of user, having the capability of adapting their interfaces according to the user profile. They must have a standard communications interface not needing a powerful processor or large storage capacity. User interfaces are web-based in order to provide interoperability with any IP-enabled device having a browser. There are three different types of devices that can be used to manage the system: IP-enabled mobile devices which allow the user to carry them around the house; we choose smart phones and tablet-PCs as the most suitable according to the user’s capabilities and preferences. Fixed devices which act as centralized control; we focused on the digital TV plus infrared remote controller as elderly people accept TV and understand how to use the remote control to send user commands. Embedded devices which may be control panels attached to each of the current appliances. Also digital photo frames are considered as ubiquitous interfaces because they have a large market penetration, they are cheap, they can reproduce images, audio and video, and have wireless communication (Bluetooth or WiFi). 4.1.3. INTELLIGENCE (E-SERVANT) In the architecture proposed, the intelligence is centralized in the e-Servant. It runs in an embedded computer that centralizes the communication with the kitchen appliances, sensors and interfaces as well as acts as a gateway to the outside world. The e-Servant is defined as the central hub of the whole system being the coordinator with whom all the other subsystems communicate over different protocols: PLC, ZigBee, WiFi, Ethernet, Bluetooth, etc. It is aware of the context and user, it enhances the intelligence of the white goods, it is also a learning system able to detect and compensate the behaviour, habit changes and loss of abilities of the user. The e-Servant helps the user in each interaction with the appliances following operational rules which take into consideration the user capabilities and environmental context. For example, if a person without known disabilities sets the washing machine, the e-Servant would only check that the washing parameters are correct for the detected clothes, providing a warning if they are not. However, if the person has memory problems, the e-Servant would guide the process step by step. The e-Servant is continuously checking the status of the kitchen appliances, providing warnings through its interfaces if there is any problem or event requiring attention (e.g. the fridge door is open or the cooking has finished). In the same line, e-Servant detects emergency situations in the kitchen (merging the information provided by the sensors previously described) and takes corrective actions if the user does not respond. For example, if the e-Servant detects fire in the kitchen, it could automatically turn off the hob, the oven and provide a warning. Then, if the warning is not attended or no movement is detected, it could send a warning to an emergency service. 7 HMI has been developed by Glyndŵr University and Motive Technology Chapter 4: Architecture design -64The e-Servant manages records with the relevant events that have occurred in the kitchen gathered from the context (sensors, kitchen appliances) and user interaction. This data is processed and analysed using artificial intelligence methods in order to extract findings about the cognitive level of the person that could be useful to the carers and/or relatives. For example, some people are absent–minded and usually forget about closing the fridge’s door. This doesn’t indicate low cognitive level, but if a person that formerly never forgets about it starts to forget closing the door, he/she might be starting to experience cognitive problems. This information is forwarded via e-mail to the user’s carers or relatives to report about the user habits in four areas of daily living in the kitchen: food management, cooking, doing laundry and other activities; same areas than those identified in the user needs analysis in chapter 3. This report is also aimed for the carers to suggest changes in the system’s user profile; user profile defines the level of support that the system provides the user and its update is always done with human supervision (described in detail in section 6.3). 4.1.4. REMOTE SERVICES The blue region in Figure 12 shows the services that the e-Servant provides through Internet and PSTN connectivity. The system is prepared to send information to carers and relatives about the user condition, establishing a connection to the call centre or the emergency centre when needed, allowing remote maintenance of the system, etc. Also, due to the flexible software architecture implemented, it would easily allow the addition of new services to the system such as on-line shopping if it was needed. 4.2. SOFTWARE ARCHITECTURE OF THE AMI KITCHEN 4.2.1. BLOCK DIAGRAM The software architecture of the system is made up by different blocks that interoperate among them to provide the required functionality; following figure pictures them: eServant Context Manager White goods Sensors Actuators RFiD readers Logic Unit Context Database (MySQL) QoLE System (Neural Networks) DEVICES PLC Gateway ZigBee Gateway (ZB Coordinator) Interface Database (MySQL) User Interface Controller TV Remote control Event Handler Scenarios Handler Core Device Manager Database Logger Events Trigger Actions Driver Alternative User Interaction Information for Quality of Life Evaluation: q User forgettings q Changes in habits q ... User Interfaces: q TV + remote controller q Appliances q Portable devices User Profiles Carer User USB Bluetooth headset Speech Recognition server Ethernet USB PLC Driver ZigBee Driver InfraredBluetooth UDP TCP USB USB USB USB Ethernet (Web Services) IR Driver IR command transceiver Audio Driver Speech Recognition client Figure 13 E-Servant software architecture Universidad de Zaragoza -65Context Manager: The Context Manager (CM) is the interface between the virtual and the real world. The information about the status of the appliances, product inventory, user actions or any other event is gathered by the CM and sent to the Logic Unit which will decide whichever operation must be performed (control the appliances, generate remote alarm calls, etc.). Context manager is the agent responsible for retrieving that information, processing and presenting it in a structured way. It is organized in three levels: o First level (Drivers Layer) manages the communication with the physical devices carrying out the tasks related with physical channel establishment, device enumeration, network support, device instantiation and messaging service. o Second level (Device Layer) contains a virtual representation of the physical devices connected to the e-Servant. o Third layer (Device Manager) is responsible of manipulating and aggregating information from the devices and effectively offering awareness of the context to the upper layers. Further detail about the different parts that make up the CM can be found in Chapter 5. Logic Unit (LU) is the “brain” of the e-Servant, responsible of three main duties performed by three different services: to process all the information provided by the context manager (Event Handler), to reason through that information and to decide actions in order to support the user (Core), and to cooperate with the User Interface Controller in order to manage the interaction with the user from a logical perspective (Scenarios Handler). Detailed explanation of the different parts that make up the LU can be found in Chapter 6. The LU continuously analyses the information coming from the CM to detect what is happening in the kitchen. When it detects a situation where the user might need guidance, help or information, it starts a communication procedure with the user; this interaction is called a user-scenario. The LU starts and stops completed user-scenarios through the interface database. Execution of the user-scenario is done by the User Interface Controller that manages the different user interfaces, notifying the result of the interaction back to the LU through the interface database. For example, if the LU receives a smoke warning form the CM, it will launch the corresponding userscenario to warn the user. If the user does not answer, the User Controller Interface will notify the LU that will consequently turn off the hob and/or the oven to prevent a possible fire, launch a reminder user-scenario and call the emergency centre to notify the problem. The problems, oversights or mistakes the user might have as well as the relevant information from the context is recorded in the Context Database to be analysed and forwarded to carers or relatives by the QoLE Systems. User Interface Controller (UIC) periodically polls the database refreshing the information served to the user interfaces that will play the user-scenarios. User-scenarios guide and help the user in his/her interaction within the smart ambient. Each user-scenario embeds the visual and aural information necessary to maintain an interaction with the user. When the interaction with the user ends, the UIC writes the user’s answers in the interfaces database to be parsed back by the Scenarios Handler in the LU. Chapter 4: Architecture design -66The UIC serves the information to the user interfaces through an embedded http server. Each user interface employs a small Adobe Flash application that retrieves the information from the UIC and displays the graphic interface as seen in Figure 14. Figure 14 Screenshot of the user interface developed by Glyndwr University. Alternative User Interaction (AUI). This block groups the alternative user interfaces: Speech commands and infrared (IR) commands. Speech is one of the most natural ways of commanding user interfaces. Implementation entails the problem of limitations and interoperability of the platform used. The client side –the user interfacesimply needs a small application called the Speech Recognition Client (SRC) in order to support voice control over the system. This client gets the audio stream from the embedded microphone, packetizes and forwards it through a socket to the Speech Recognition Server (SRS). The Speech Recognition Server has been designed to allow these devices to use a speech recognition engine. The server analyses the audio stream and replies to the client with the identified command. As a proof-of-concept the speech recognition engine from Microsoft installed where the e-Servant is running has been used with good results. It would be simple to change the service to use any other engine’s API such as Dragon Naturally Speaking, Google Voice, etc. Also, the TV remote control is one of the interfaces most extended and elderly people are usually familiarized with it. A commercial IR to USB transceiver (USB-UIRT 2009) has been used in order to integrate this technology in the e-Servant. IR commands are parsed by the IR command transceiver module and converted in understandable stimuli for the e-Servant interface. QoLE System is a service that periodically analyses the context database looking for changes in the user washing, shopping and cooking habits which could be relevant in order to detect a loss of physical, cognitive or sensorial capabilities. For example, if the user starts going to the fridge at night or if s/he is doing the laundry less and less often. The QoLE System performs an indirect evaluation of the quality of life of the user through the measurement of its capabilities and habits in his/her daily tasks in the kitchen. It produces an easy to understand report which aims to provide objective information about the everyday life of the user. Designed for the use of non-technical people, it is intended as a tool for social workers to complement the information they typically use (surveys and personal interviews) to assess the user’s quality of life. Universidad de Zaragoza -67To illustrate the interaction of the various blocks of the architecture, the following use case shows what would happen in the event of smoke detection. Use case: smoke sensors notify the system that there is smoke in the kitchen, oven and hob are on but nobody is in the kitchen. The ZigBee smoke sensor (1) warns to the CM (2) that there is smoke in the kitchen. LU (3) is notified and decides to launch a user-scenario to warn to the user. UIC (4) commands the interfaces (5) in order to warn the user about the situation. After a timeout, the interfaces (6) notify to the UIC (7) that the user does interact with them and the LU (3) decides to turn off the PLC hob and the oven (10) through the CM (9). eServant Context Manager Oven and hob Smoke sensor RFiD readers Logic Unit Context Database (MySQL) QoLE System (Neural Networks) DEVICES PLC Gateway ZigBee Gateway (ZB Coordinator) Interface Database (MySQL) User Interface Controller TV Remote control Event Handler Scenarios Handler Core Device Manager Database Logger Events Trigger Actions Driver User Interaction Information for Quality of Life Evaluation: q User forgettings q Changes in habits q ... User Interfaces: q TV + remote controller q Appliances q Portable devices User Profiles Carer User USB Bluetooth headset Speech Recognition server Ethernet USB PLC Driver ZigBee Driver InfraredBluetooth UDP TCP USB USB USB USB Ethernet (Web Services) IR Driver IR command transceiver Audio Driver Speech Recognition client eServant Context Manager Oven and hob Smoke sensor RFiD readers Logic Unit Context Database (MySQL) QoLE System (Neural Networks) DEVICES PLC Gateway ZigBee Gateway (ZB Coordinator) Interface Database (MySQL) User Interface Controller TV Remote control Event Handler Scenarios Handler Core Device Manager Database Logger Events Trigger Actions Driver User Interaction Information for Quality of Life Evaluation: q User forgettings q Changes in habits q ... User Interfaces: q TV + remote controller q Appliances q Portable devices User Profiles Carer User USB Bluetooth headset Speech Recognition server Ethernet USB PLC Driver ZigBee Driver InfraredBluetooth UDP TCP USB USB USB USB Ethernet (Web Services) IR Driver IR command transceiver Audio Driver Speech Recognition client 1 2 3 4 5 6 7 8 9 10 Table 32 Use case: Smoke in the kitchen. Example the interaction between the architecture blocks Chapter 4: Architecture design -684.2.2. SOFTWARE PLATFORM Software platform must be able to integrate several technologies as PCL, ZigBee or RFID and products from different manufacturers. Also, It should permit adding new devices (appliances, sensors, etc.) easily. A ServiceOriented Architecture (SOA) is the most appropriated choice as it enables a modular design, simplifying the programming process and easing the integration of new devices or services. In this sense, the Open Service Gateway initiative (OSGi), provides a working framework which satisfies the technical needs of the system. In this framework, pieces of code are organized into bundles that can be managed dynamically. OSGi bundles are agents which might be dedicated to specialized tasks, such as handling a serial port, providing a command line interface, collecting, aggregating and analysing data, etc. These bundles communicate and interact with each other by means of services which are published within the framework, and each bundle can acquire and use them (OSGi Alliance 2010). The main strength of OSGi is that the framework manages these bundles dynamically, allowing them to be upgraded without terminating the full application, as well as enabling the availability of the services to other bundles depending on the situation. A special case of those services are object entities usually related to physical devices. These devices are published by bundles which handle the communication channel between the physical devices and their virtual representation, and have an independent identity in the framework. These features make OSGi one of most powerful tools to implement an ontology that represents the context having bundles as virtual representation of devices. Moreover, as it was noted in Section 2.1.3, most of the standard ontologies choose OSGi as working framework. Therefore, OSGi has been chosen as the backbone of the e-Servant in order to enhance its capabilities and minimize the installation and maintenance cost in future commercial products (easy update of software packets, addition of new sensors or appliances, etc.). 4.3. CONCLUSIONS This chapter, besides providing a global vision of the system, describes the architecture proposed their different building blocks. Its design has been addressed modularly, identifying the functionalities of each part of the system and its relation. From the beginning, the need to design a system close to the market has spotted essential aspects of the architecture such as the interoperability, flexibility and easy deployment, maintenance and update. The architecture promotes the interoperability between devices, technologies and manufacturers supporting the integration of devices with any communication technology (we integrated ZigBee and PLC), the interaction with any web based interface, infrared and Bluetooth remote control. Thanks to this modular and flexible design, it is easy to add new appliances or sensors with little changes in the Device Manager (as it is shown next chapter). Also, the OSGi framework chosen to develop the system permits dynamically the remote installation, deployment and update of the services which compose the system. Universidad de Zaragoza -69CHAPTER 5: CONTEXT INTERACTION Context interaction is crucial to any AmI because it provides the data upon which the system will make decisions and act accordingly. This is even more essential when we are designing an AAL environment, where the user might have limited capacities performing actions or be challenged understanding how to manage an appliance or interacting with the system. This chapter contains contributions made in the context interaction implementation of the system. It describes the physical deployment of the sensors and actuators network as well as the modifications accomplished in the kitchen appliances to enhance their capabilities. Also, it details data processing performed to develop devices with advanced features: fall detection and location. And finally, it explains the architecture and implementation of the Context Manager, which models the physical devices to provide upper layers a virtual representation of appliances, sensors and actuators in the environment. 5.1. SENSOR-ACTUATOR INFRASTRUCTURE When considering the interaction between elderly people and kitchen appliances, it is evident that the more information we can extract from the person, the appliance and the environment, the better we can support the user. Regardless this evidence, to build a usable and marketable system, it is necessary to find a compromise between the parameters that must be monitored and those that are technologically feasible in order to have the maximum information. It has been a PhD requirement to be close to the market, with an actual deployment feasible and acceptable by the user at home. This premise has been taken as a starting point to decide which sensors could be installed in the kitchen assessing their potential impact on the services developed versus the cost, intrusiveness and acceptation issues. However, it was also considered that sensor cost, size or user rejection, which today may suppose a barrier, could change in the future. Thus, the architecture has been designed to allow easily adding new devices or technologies. A key issue in the deployment of an interactive context in AmI is interoperability. Communication protocols are essential to provide adequate functioning of devices (coverage, data rate, user acceptance, etc.) and use of standards and coexistence of media channels (radio, cable, power line, optical) is also very mandatory in order to provide interoperability and flexibility. Thus, according to the nature of each device and also as PhD requirement, it has been decided to integrate several communication protocols in the deployment of sensors and actuators. Figure 15 shows the connections that currently supports e-Servant with the various elements of the AAL:  ZigBee as wireless sensor network standard to connect all the sensors distributed in the environment.  Bluetooth as transport channel for audio streaming in voice recognition and messaging.  WiFi/Ethernet for IP data exchange between user interfaces, e-Servant, database, etc.  Infrared (IR) to integrate existing remote controls in the system.  Power Line Communications as de-facto standard for communication between white goods. Chapter 5: Context interaction -70Tv RouterWiFi e-Servant Call centrer QoL reports Appliances Ethernet HDMi Tv remote Control USB IR USB ZigBee Dongle Bluetooth Dongle Touch Screens and Smart Phones Ethernet / PLC IR WiFi PLC Internet USB User Tag and telecare system Fire Smoke Flood sensors Presence Light sensors RFID readers Door sensors Headset USB ZigBee mesh network Appliances Remote services Interfaces Additional sensors and actuators Color code: E-Servant Figure 15 e-Servant physical connexions This subsection focuses on those devices whose interaction is handled by the Context Manager: appliances, sensors and actuators. CM models them, isolating their functionalities from their respective communications protocol, allowing their use from a high level of abstraction. Devices acting as user interfaces are excluded because they communicate directly with the web server (see section 4.2.1) or through the HDMI connection. 5.1.1. ZIGBEE SENSORS AND ACTUATORS Several ZigBee devices have been developed ad-hoc to be used in the Smart Kitchen to cover specific features or to ease the integration of commercial sensor in the system. The devices described below (except the Multisensor) have been fully designed and implemented (component selection, schematic design, PCB design, firmware programming and debug) in the framework of this PhD work. ZigBee Tag: Safety at home is one of the biggest concerns of the elderly and their relatives. User habits, location and detection of critical situations such as falls are important sources of information for AAL systems. Telecare systems based in a panic button are one of the electronic aids most widely used. Tag goes one step further: it aims to identify the user and his/her location and process movement data to infer relevant situations such as fall events and level of movement (see section 5.2). This device, designed to be worn as a neck pendant, is powered by a rechargeable 3.6V Ion-Lithium battery, includes a triaxial accelerometer (Freescale MMA7260Q), two buttons and a ZigBee transceiver (Figure 16. Left). Universidad de Zaragoza -77Figure 22 Three layers Neural Network. Source: (Demuth, Beale 2003) There are different learning techniques for Neural Networks. These techniques define how the weights of the neurons which compose the network are tuned to have the desired response. Learning techniques enable to classify the neural networks in four groups:  Supervised learning: Neural Network is trained from known examples, i.e. using couples of inputs and desired output.  Unsupervised learning (self-organized): In this case the Neural Network is trained using only the inputs.  Hybrid learning: These techniques combine supervised and unsupervised training in different layers of the neural network.  Reinforcement learning: These techniques of learning are based on reward-punishment, this way the Neural Networks learn trying to maximize the number or rewards. In the next studies, two kind of neural networks have been utilized: The Multilayer Perceptron (MLP) is a supervised feedforward (neurons model without feedback) Neural Network. The universal approximation capabilities of the multilayer perceptron (Hornik, Stinchcombe et al. 1989) make it a popular choice for modelling nonlinear systems, classification (discriminant analysis) and functional regression. It is composed by three or more neuron layers: The first one provides the input data, the intermediate layers, known as hidden layers, using a non-lineal activation function to process the information, and the output layer, which combines the outputs of the hidden layers proposing a result. It is trained using the backpropagation (BP) algorithm or one of its variants (LevenbergMardquardt, Resilent-Backpropagation, Scaled Conjugate Gradient, etc.). BP is a gradient descent algorithm that tries to find the best NN weights in order to minimize the error function between the output and the objective. The Seft-Organizing Features Map (SOFM), usually known as SOM, is an unsupervised Neural Network structured in two layers: inputs and map (1, 2 or 3 dimensions). SOM are a good tool for the visualization of high-dimensional data. They can convert n-dimensional data into a simple geometric, for example 2dimensional. They can be used also to produce some kind of abstractions (Kohonen 1990) and they are really useful grouping data with similar characteristics. Its training is based on a competitive method that choses as winner the neuron which has the minimum Euclidian distance between its weight vector and the input vector. The winner neuron and its neighbours update their weight in order to be closer to the input vector. Iteratively, groups of SOM neurons are specialized in a kind of inputs. Chapter 5: Context interaction -785.2.1. FALL DETECTION ZigBee Tag could work as fall detector thanks to the tri-axial accelerometer integrated. Following it is shown how Neural Networks enhances fall detection system in contrast with the traditional threshold-based methods (Chen, Kwong et al. 2006) increasing immunity false falls; events not with similar inertial patterns (e.g. sitting in a sofa abruptly). Ten people of different ages, weight, height and sex, with the ZigBee Tag hanged around the neck, have imitated the movements of elderly people to create a database of falls. Volunteers were asked to simulate true and false falls situations. In the true fall situations every volunteer falls down 10 times on a straw mat. The fall intensity changed (rough and soft) and the way of falling down too (side, front, backwards), hitting the ground with their back, hip, knees, etc. For the false-fall situations every volunteer flings himself down 5 times on the centre and 5 times on the side of a sofa, stumbles and hits a wall without falling down 5 times and walks around for 2 minutes doing normal movements like sitting up and down in chairs, picking up things, etc. During the test, the ZigBee Tag continuously samples the three acceleration axes each 32 ms sending them to a PC working as a data logger. In the end, we get a file with all the acceleration samples in axis X, Y and Z for every volunteer. The resulting database consists of 99 samples of true falls and 150 of false falls. This database has been analysed using an acceleration threshold, which was experimentally determined to 2 g in order to detect all the true fall events of the database. This election motivates that some normal movements are above the threshold; being detected as falls, increasing the number of false fall events to 241. Also, based on the stored data, it has been observed that of 800 ms defined the length of any event: time that includes all relevant information about the fall. This time has been increased 160 ms in order to include some preamble data before the acceleration crosses the threshold. Figure 23 represents a fall event. Figure 23 Fall event The “window time” (tw=t1+t2) includes 30 samples to be analysed by the NN. With this window time selected, the number of inputs to the neuronal network is set to 90; 3 axes per 30 samples. In order to reduce the number of network entries —and consequently the network size— a PCA (Principal Component Analysis) has been used. Applying PCA analysis with the 340 events detected with this threshold (99 falls plus 241 false-falls), the number of inputs was reduced from 90 to 55, keeping the 95% of the covariance of the original data. t 2 t 1 tw threshold Universidad de Zaragoza -79Different Multi-Layer Perceptron (MLP) architectures 55xMx1 have been trained (being M the number of neurons in the hidden layer, 5≤M≤35) using the 80% of the events (randomly selected) for training and 20% for validating. That is to say, from the whole 340 events (99 falls plus 241 false-falls), the validation group had 20 true falls and 48 events that could be confused with falls. In the end, a neural net with 22 hidden neurons was able to classify falls correctly. The final results using MLP neural networks for fall detection have been quite satisfactory. The application classifies correctly 92% of the validation group falls, achieving a better performance than other detection methods: 80% in (Chen, Kwong et al. 2006). Moreover, the number of false alarms is drastically reduced to 1%, which leads to enhance users trust on the fall detector. Nevertheless, a more extensive study with more users, being also elderly, needs to be conducted to gather more data and confirm the results in order to develop a new service in the AAL. 5.2.2. INDOOR LOCATION The goal to be achieved with this study was to determine whether it is possible to locate, at room level, with a ZigBee network which has not specifically designed for this purpose. This way, with an existing ZigBee infrastructure that provides communication service in the AAL, with the only additional cost of the tag bearing the user, it could be possible to offer new services. To validate this idea it was defined a test scenario Figure 24; different ZigBee routers (orange triangles in the figure) are distributed in the test environment with the only criteria of ensuring data coverage throughout the test area. Then a person wearing a ZigBee Tag as a pendant around the neck, moved to several positions inside each of the localization area; white circles in figure. For each of these positions, measures were collected with a different user orientation (north, west, south, and east). Each measurement consisted on the Received Signal Strength Indicator (RSSI) with which the tag “sees” its surrounding neighbours. This sequence was repeated in 32 locations, generating a pattern database of RSSI measurements. Information about the location of the beacons was not stored, as only RSSI was used to estimate the location of the Tag. Figure 24 Scenario map. Fixed devices are denoted by arrow caps, and tag reference locations are represented by circles. These data were analysed using a MLP (MultiLayer Perceptron) and SOM (Self Organizing Map), achieving a positioning error of 20% and 18,7% respectively, pointing out that the NN (Neural Network) does not have enough information to locate. Thinking about the physical nature of the problem, at the time this work has been done, localization systems based on ZigBee RSSI, locate targets with accuracy among 3 m and 5 m depending on the density of beacons (or fixed nodes) (ZigBee Alliance 2007). In this case, where a network with Chapter 5: Context interaction -80a minimum number of beacons has been forced to ensure coverage, it would be logical to think that the accuracy in meters should be 5 meters or more. Also, in the transition of localization areas without physical separation (doors or walls) the RSSI was very similar. Therefore, if this system provided a location in meters it would not be difficult to provide coordinates and a radius of confidence. When this situation takes place, each time one user is placed less than 5 meters away from the contiguous area, the system will detect the user as being placed in one of the areas located in the radius of confidence within the two areas. How can this idea be extrapolated to a SOM? If the activation of neurons is studied in detail, it will be observed that there are areas in the map which are activated mainly by stimulus of two areas, i.e. “zone L4.09” (7 activations) and “zone L4.08” (2 activations). If the wining label is considered, in this case “zone L4.09”, when points are set in the “zone L4.08”, which are very close to this area, and the map activates the neuron, the situation is considered as an error. Nevertheless when using any other method, if it is within the radius of confidence, it would be considered as correct. Sample Groups Test error Validation error Group 1: points where the tag detects at least one beacon. 17.20% 24.00% Group 2: points where the tag detects at least two beacons. 12.65% 19.59% Group 3: points where the tag detects at least three beacons. 11.11% 18.66% Table 35 Classification error with SOM using only one label. (a) (b) Figure 25 SOM map for one label (a) and two labels (b) methods Universidad de Zaragoza -81Therefore, if two labels are considered in those neurons responding to two specific boundaries, it won’t be known whether the user is located in the “zone L4.09” or in “zone L4.08”, but the area in which the user is located will be known, and probably even the boundary among them. Obviously, it can occur that the neuron is activated by stimulus in the two areas separated by 5 meters, being this unusual situation. In this case only one label will be considered. Taking into account this situation, the error in the localization is cut down to 7.1% for the samples which can see at least 3 beacons. In contrast, in some situations the network feedback is “zone A”- “zone B” instead of one only area (of course, they are adjacent zones). However, this resolution could be enough for many home services. 5.3. CONTEXT MANAGER IMPLEMENTATION Context Manager (CM) is the e-Servant block responsible for managing communications with other devices in the kitchen (sensors, appliances, etc.). It is the most flexible part of the e-Servant, since it must update the context’s virtual representation each time a new sensor or appliance is added or removed from the system. In this situation, benefits of using a modular architecture as OSGi against a monolithic application are considerable; incorporating a new range of kitchen appliances or a new sensor technology just involves updating one of the bundles that constructs the application. The Context Manager is structured in three layers: Driver Layer, which manages the communication network with physical devices of the system; Device Layer, which contains a logical representation of the physical devices and Device Manager, which manages the events of these devices with the rest of the e-Servant. Each driver has been implemented in an OSGi bundle, this way new drivers can be added without making changes to other services. Device Layer and Device manager have been implemented in a unique bundle 8 . The following sections describe in detail each of these layers. 5.3.1. DRIVER LAYER The lowest layer of the context manager is the driver layer, where communication with physical devices and transport services are developed. The tasks carried out by this layer are: Physical channel establishment. The driver layer has the responsibility of creating and opening communication ports with the network gateways. Automatic identification of ports is also performed if possible (i.e. serial port scan for the connected gateway in the ZigBee Network sensors set). Device enumeration and network support. Once the communication channel is up, sensor network management is carried out by the network driver. It deals with network support operations and performs enumeration and registration of the physical devices presenting the network infrastructure. Device instantiation and messaging service. Devices recognized by the driver in the network are instantiated by the driver layer and presented to the context manager, allowing exchange of information between them and their software representation. Two drivers have been implemented into the context manager architecture for communicating with physical devices: 8 Note that the Driver layer implementation should not be considered as part of this PhD because it was not part of my personal work within the project. Anyway a short description is included for sake of understanding the Context Manager operation. Chapter 5: Context interaction -82PLC driver to communicate with kitchen appliances. The PLC driver is connected to a PLC gateway through Ethernet, using web services. The driver checks that the gateway is running, and registers the devices (appliances). For each device, a PLC Device is instantiated and presented to the context manager. ZigBee driver to communicate with sensors, actuators, RFID readers and care-phone. The ZigBee driver establishes the connection with the ZigBee gateway, which also acts as the coordinator of the network, through a USB (serial) port. The driver checks if the network is created and, if so, gets the devices connected in the network. For each device in the network, a ZigBee Device is instantiated and presented to the context manager. 5.3.2. DEVICE LAYER Each of the above described drivers instantiate object devices related to the physical devices connected, or more precisely, controllable by means of the respective driver. These objects correspond to OSGi services, which have an independent identity within the framework. Device representation allows separating device functionality from the underlying technology and the transport layer. Each one holds data members, properties and methods which model the behaviour of the physical devices they represent. Activation or access to these methods implies the communication with the physical devices through the base driver that has instantiated the device (therefore, the base driver must be active to enable that communication). The representation of devices start from the OSGi4AmI implementation represented in the Open Source Project embodied in the same name (Marco, Cirujano et al. 2012) and published in (Marco, Casas et al. 2009) as book chapter. This taxonomy proposes three basic devices that are part of any AmI: Sensors are devices which are able to sense physical magnitudes (like temperature, presence, acceleration, etc.). Actuators are devices which are able to change any of the environment characteristics (like switch on/off a plug, dimming a light, close/open a door/window, etc.). Simple HMI (Human-Machine Interface) are devices which are used to input simple data into the system (remote controllers, level controllers, etc.) or are used by the system to provide information to the user in a simple way (LED controllers, buzzer, etc.). This taxonomy proposes the identification of the common properties inherent to the each device nature. These properties, clustered by functionalities, define the interface of the device. As Figure 26 shows, every device interface extends the base device, and therefore all devices have some common properties. The BasicDevice Interface, which is mandatory for all devices, includes the minimum information inherent to the device (Category, description, serial number, provider, etc.) that is needed to recognize it. Every device has an energy power supply (battery, main power, etc) and a location in the space. However, the virtual device could be aware or not about them and consequently leaving the power cluster and the location cluster as optional. As a proof of concept, location service presented in section 5.2.2 has been implemented at this level demonstrating its feasibility. The same philosophy has been applied to the specific devices like sensors and actuators, which may be able to provide different levels of information. Each device category includes mandatory (labelled in orange in the Figure 26) and optional clusters (labelled in violet Figure 26). For example, inherent to its nature, every sensor will be able to provide its measurements. For this reason, the method to retrieve measurements from a sensor Universidad de Zaragoza -83belongs to base sensor cluster; mandatory for every sensor device. Some sensors might be smart to implement measurement analysis and provide alarms when thresholds are exceeded; as this won’t be implemented by all sensors, such methods belong to optional threshold sensor cluster. This way, devices with the same functionality, for example two temperature sensors from different manufacturers, should implement the same interface with the same mandatory clusters and corresponding to their capacities optional clusters. Regardless to their implementation, one is a proprietary RF sensor while the other is a ZigBee sensor, both could be used by a “thermostat service” which needs temperature sensors devices (thanks to the BasicDevice) and both can be requested in the same way to provide the temperature value (thanks to the BasicSensor); i.e. implementation of the sensor is decoupled from its logical abstraction. Additionally each level of abstraction, from the device to the actuator, sensor or simpleHMI, defines a listener interface in order to raise events to registered services. LocationCluster Device EventSensorCluster Sensor ThresholdSensorCluster AutoRefreshSensorCluster MinMaxSensorCluster StreamingSensorCluster LevelActuatorCluster Actuator OnOffActuatorCluster GetStatusActuatorCluster OutputSimpleHMICluster SimpleHMI InputSimpleHMICluster BaseDevice BaseSensor BaseActuator BaseSimpleHMI SensorListener ActuatorListener DeviceListener SimpleHMIListener UpDownActuatorCluster PowerCluster Figure 26 Device categories. Source: (Marco, Casas et al. 2009) However, although this taxonomy covers the basic elements typical of an AmI, it can be extended to specific contexts. Thus, following the same philosophy, it has been enlarged to include the kitchen appliances. The interface shown in Figure 27, extends from device to enable control of the appliances present in this scenario. As shown in this figure, five appliance clusters that allow access to different levels of control are defined: - BaseApplianceCluster combines the basic features associated to the appliance as updating their status or setting its refresh time for automatic update. - GetStatusCluster allows getting the value of the appliance characteristic state variables. - SetStatusCluster allows setting the value of the appliance characteristic state variables. - OnOffApplianceCluster allows turning on/off the appliance. Appliance BaseApplianceCluster ApplianceListener GetStatusApplianceCluster SetStatusApplianceCluster OnOffApplianceCluster SmartApplianceCluster Figure 27 Appliance interface Chapter 5: Context interaction -84- - SmartApplianceCluster allows executing a concrete program, to pause it, checking its state or scheduling its execution. Besides publishing a representation of the basic devices present in the AAL in the OSGi framework, this layer groups some basic devices to build smarter devices that create a virtual representation of the whole; which is called superdevice. For example, in the case of refrigerators, the Device Manager publishes a superdevice called Fridge that is composed by the combination of Refrigerator (PLC), RFID reader (ZigBee) and door sensor (ZigBee). Figure 28 shows an example of how the Device Layer works. There are two bundles, PLC device discoverer and ZigBee device discoverer, looking for new devices and publishing them as services. Also, the SuperDevice Aggregator bundle is looking for devices which form part of a superdevice, in this case the Fridge. DEVICE LAYERDRIVER LAYER PLC Driver ZigBee Driver Device (Fridge) Device (Door sensor) Device (Hob) Device (RFID Sensor) Appliance (Hob) PLC device discoverer ZigBee device discoverer Appliance (Fridge) Device (Carephone) Sensor (Door sensor) Sensor (Stand alone RFID reader) Actuator (Carephone) SuperDevice Aggregator SuperDevice (Fridge) Appliance (Fridge) Sensor (RFID Sensor) Sensor (Door sensor) Sensor (RFID reader) Figure 28. Example of the composition of a superdevice along with the different layers making up the CM These superdevices have its own identity within the OSGi framework performing Action Driving and Event triggering functions. In addition, they are also independent devices that may require event management or control by the upper layers. Thus, upper bundles work isolated from the technology involved in the sensor, actuator, appliance or Simple HMI. Currently, the Device layer offers the following devices and superdevices: Device / Superdevice Composed by Functionalities Washing machine ZigBee door sensor This sensor triggers an event when the door of the washing machine is opened or closed. Also, it can be requested to know the current state of the door. ZigBee RFID reader This device provides information about the content of the washing machine PLC washing machine This device provides properties related to the washing programme being performed and its current phase, the water temperature (target), or the estimated finish time, as well as the operating state and some malfunction and alarm warnings. This device allows also remotely activating or deactivating the programme. Legend Bundle Service Universidad de Zaragoza -85Fridge ZigBee door sensor This sensor triggers an event when the door of the fridge is opened or closed. Also, it can be requested to know the current state of the door. ZigBee RFID reader This device provides information about the content of the fridge PLC Fridge Fridge Device exposes properties related to the current and target temperature of both compartments (fridge and freezer), as well as if they are active or not, and SuperFreezing or EcoMode functionality. Oven ZigBee door sensor This sensor triggers an event when the door of the oven is opened or closed. Also, it can be requested to know the current state of the door. PLC Oven Properties accessible by the e-Servant are its state (active or not, failure…), heating mode, current and target temperature, programme duration and absolute end time, child lock state. It is also possible to remotely operate the oven. Hob PLC Hob This device only provides monitoring of the state of each of the cooking zones in the hob (typically four) and the child lock, because legally it is not allowed to remotely operate this device, and it is only possible to stop the device if an emergency occurs. Door sensor ZigBee door sensor This device indicates the opening of a door in a cupboard or appliance in the application scenario. Presence sensor ZigBee Presence sensor This device indicates whether someone is in the sensor area or not. Temperature sensor ZigBee Temperature sensor Indicates the temperature of the location where the sensor is (maybe outdoors) Light sensor ZigBee light sensor Indicates if the light is turned on or off, which together with the presence sensor can easily indicate activity or set an alarm. Smoke sensor ZigBee smoke sensor Indicates the presence/absence of smoke. Fire sensor ZigBee fire sensor Indicates the presence/absence of fire Flood sensor ZigBee flood sensor Indicates the presence/absence of water in the area covered by the sensor RFID reader ZigBee RFID reader Products inventory and tracking will be performed by means of RFiD readers, which provide RFiD code label of devices in range by request. ZigBee Carephone ZigBee Care Phone This device allows control a commercial telecare system (as the Carephone of the Caretech(c)) generating up to four different alarm codes that can be identified in call centre. It is also capable of establishing a voice call with it. Light ON/OFF actuator ZigBee actuator This device allows turning on and off the light of the kitchen. ZigBee Tag Base device Include the basic information about the device (Category, description, end points, clusters supported, etc.) Accelerometer Tri-axial accelerometer used to detect falls Two Buttons Simple HMI used to generate alarms Table 36.Devices supported by the e-Servant Chapter 5: Context interaction -865.3.3. DEVICE MANAGER LAYER Lower layers of the Context Manager gather existing physical devices and present them in a structured way. Device Manager is responsible for manipulating and aggregating information from these devices and effectively offering context awareness to upper layers. Main tasks performed by Device Manager are: Context logger. The information collected about sensors together with the information derived from the context aggregation is logged into the Context Database, which will be used by the Logic Unit (LU) and Quality of Life Evaluation (QoLE) services. Action driver. Through this module, the Context Manager allows the LU to act over the physical devices. Event trigger. The information received from the sensors is stored in the context database, but some external events may require an immediate response by the e-Servant. This module notifies the LU that something is happening. Figure 29 shows the three bundles that integrate the Device Manager Layer. This layer groups the information of the context and isolates upper layers of the devices management. Legend DEVICE MANAGER LAYER DEVICE LAYERDRIVER LAYER PLC Driver ZigBee Driver Device (Fridge) Device (Door sensor) Device (Hob) Device (RFID Sensor) Appliance (Hob) PLC Device discoverer ZigBee device discoverer Appliance (Fridge) Device (Carephone) Sensor (Door sensor) Sensor (Stand alone RFID reader) Actuator (Carephone) SuperDevice Aggregator SuperDevice (Fridge) Appliance (Fridge) Sensor (RFID Sensor) Sensor (Door sensor) Sensor (RFID reader) Context LoggerEvent trigger Action Driver Context events Context actions Context Database (MySQL) Bundle Database Service Software packages Figure 29 Example of the Context Manager working Legend Bundle Database Service Universidad de Zaragoza -936.2.4. STAND-ALONE RFID READER In addition to appliances, rules have also been designed for the stand-alone RFID reader. This device, located in the kitchen, has as main function to provide additional information on those elements carrying an RFID tag. Two rules have been defined: R_r1: Display information Whenever a labelled object is close to the reader, the information of the object is displayed in the interfaces of the e-Servant. Currently there are two possibilities: - Food: it is displayed the name of the product and the expiration date (additional information could be added) - Clothes: it is displayed the name of the product, the colour and the suggested washing mode. R_r2: Automatic configuration of the oven Additionally, when the tagged item is detected as prepared meal, the interface may suggest an oven configuration and even configuring it if the user desires. 6.3. USER’S PROFILES Each person is represented in the system by his/her user profile. Decisions made by the Logical Unit are conditioned by this profile which mainly determines the person’s capabilities at different levels of interaction with the system. To design the user’s profiles, interaction between e-Servant and user has been studied using the “persona” concept. This idea, developed by Alan Copper in his book “The inmates are running the asylum” defines personas as “Personas are not real people, but they represent them throughout the design process. They are hypothetical archetypes of actual users” (Copper 2004). Led by the Glyndŵr University, this idea was applied to the user modelling to define the interface and the user’s profile (Casas, Blasco et al. 2008). Ten personas have been defined based on the European statistics randomly assigning age, education, work, family situation, impairments and technology background. As an example next box represents the information about Hannah: Figure 31.Example of persona. Source: (Casas, Blasco et al. 2008) Chapter 6: Reasoning -94While interacting with the e-Servant, it is possible to adjust how and what information is sent to the user. Customizing the physical parameters of the output channels of the system, as volume, pitch, contrast, etc., it is possible to control how the information is send to the user. Also, the e-Servant can increase the help level which is offered to the user as well as the information shown to him/her. The study of the interaction between “personas” and e-Servant has been used to design a user profile that could provide enough information to the system (LU and user interfaces). User profile aims to consider cognitive and sensorial capabilities of the person within the following categories: User level has four different grades: not possible (0) indicates that the user is not able to use the system; of course it could be a temporary situation. Easy (1), standard (2) and expert (3), indicates the understanding that the user has of the system. This understanding can be due to different reasons: e.g. knowing all the system’s features, having memory losses, etc. From the LU’s point of view, this is the most relevant parameter as it is related with the cognitive capabilities and the technological skill of the person. This parameter determines the interaction with the user (number of options, complexity of menus, etc.). Interface makes reference to how the system will show the information to the user: using icons (0) text (1) or both (2). Besides the user preferences, this also has implicit information about the user’s cognitive level. Audio: Inside audio category, three sub-categories are included: Volume, pitch and voice control. First two can help people with aural disabilities to hear the HMI. Voice control indicates if the user would control the system via voice commands. Of course, this would be helpful for people with visual disabilities, but not only. As voice is the most natural way of communicating (compared with remote controls, keyboards, tactile interfaces, etc.), voice control would be helpful for those people with reduced cognitive capacities or low technological skills. Display: it includes common adjustment controls in screens: contrast, brightness and colour settings (physical parameters). These characteristics, besides adapting to the ambient light and user preferences, together with magnification might help people with visual impairments to interact with the display. The user profile has been validated setting a different user configuration for each persona. Next we can see Hannah’s case: User Level Interface Audio Display Icons and textExpert Volume (0-1) Pitch (normal) Voice control (no) Contrast (high) Magnification (high) Brightness (high) Color (2) Figure 32. Hannah user profile. Universidad de Zaragoza -956.4. QUALITY OF LIFE EVALUATION SYSTEM (QOLES) The QoLES is a service that periodically (configurable period) analyses the user`s behaviour looking for changes in his/her habits in the kitchen. These variations could be relevant to detect a loss of physical, cognitive or sensory capabilities. Main objective of this service is to provide objective information about the everyday life of the user. The first step in the design of the service has been to analyse the data available by the e-Servant which could be more relevant in order to detect changes in the user’s habits. Then, the QoLES idea was validated with stakeholders in the co-design session described in the section 3.5.2. In these sessions, participants contributed to the design with new ideas that improved the system. Most relevant contribution was to decide that the system should not automatically change the user level; QoLES service should limit to report the carer with objective data about the person’s quality of life, then the carer will take the final decision about the help level of the system. Finally, according to these information categories, random data were generated to try different algorithms and technics to develop the first version of the QoLE system. This work was carried out by Antonio Bono in his master thesis (Bono Nuez, Roy Yarza 2008). However, to validate this functionality it is needed to evaluate enough number of users during a long period of time (several months). During this time, the user evolution should be contrasted by socio-health professionals who could help to corroborate the QoLES performance; such essay was out of the scope of the research project which framed this PhD and consequently of the PhD. Next sections show my work regarding conceptualization and design of the system as well as implementation of the data-retrieving layers. Implementation of the artificial intelligence layers and its theoretical validation (which was done by Antonio Bono) can be found in the paper: Quality of Life Evaluation of Elderly and Disabled People by Using Self-Organizing Maps (Bono Nuez, Martín del Brío et al. 2009). 6.4.1. QOLE SYSTEM DESIGN It seems evident that changes in HMI’s navigation skill or increasing the times the fridge is opened (without picking anything) might be related to changes in the user’s cognitive capacity or disorientation. The e-Servant is able to gather a huge amount of data which, if not properly selected and handled, would generate unusable outputs and erroneous conclusions. As the main objective was to create a useful tool for the carers, starting with the analysis performed in chapter 3, engineers have worked closely with health and social professionals to determine the relevant data (see section 3.5.2). Also, design of this tool doesn’t restrict to the technology and appliances involved in this AAL (not even to existing technology). This way, the QoLES can be adapted to the technology available and also persist in time. Like in chapter 3, relevant activities in a kitchen are grouped in four areas: Food management and storage: including food storage in cupboards, drawers and fridge, out-of-date food management, cooling interruption of aliments needing refrigeration. Cooking: these activities include from interaction with the hobs, oven and microwaves to actions done in the preparation of the food (following a recipe, cutting, etc.). Washing activities: including washing crockery (dishwasher, sink) and clothes (washing machine). Other no specific: here are included all activities also performed in the kitchen that are not framed by the previous ones. For example, using the water tap, actuating on the lights, interacting with the HMI, throwing the garbage out, etc. Chapter 6: Reasoning -96Next tables outline for each area, the high level information that could be useful for the QoLES as well as the data needed to provide it: Food storage and management High-level information Data needed Changes in shopping habits Food tracking in the kitchen (amount, persistency) Time between shopping Habitual food expiration Expiration date (coded in the RFID labels) Errand behaviour (maybe due to disorientation or bad memory) Number of times the fridge’s door is open without taking/introducing anything Erratic cupboard’s door opening (not implemented) Miss-attend the temperature or door alarms Door and temperature sensor Number of warnings until warning is attended Table 38 Relevant information for the QoLES related with the food storage and management Cooking High-level information Data needed Absence of mind situations (changes may indicate loosing of capacities) Forget the pan on the fire (e-Servant could warn when more than certain time, if the pan is immediately removed, it could be a forgetting) Wrong stove selection Liquid overflowing when cooking Use the pan void Fire or smoke detected by specific sensors Forget switching the appliances off Number of times the oven and/or microwaves’ door is open before ending Table 39 Relevant information for the QoLES related with the cooking activities Washing activities High-level information Data needed Erroneous washing machine operation Program selected Identification of the clothes inside the washing machine Forgetting taking the crockery/clean clothes out Number of warnings until crockery/clothing is removed Changes in washing conducts Time between washings Table 40 Relevant information for the QoLES related with the washing activities Other no specific High-level information Data needed Absence of mind situations (changes may indicate loosing of capacities) Forget about lights on (presence + light sensor) Erratic movement (position sensors) Forget the water-tap opened (sound sensor) Changes in HMI navigation times Time needed by the user in each interaction with the HMI Table 41 Other relevant information parameters for the QoLES Although all these data sources have been identified as relevant in the workshops, several of them were discarded by its complexity or because the sensor setup needed was not reasonable. Every time the context is Universidad de Zaragoza -97evaluated and analysed by the LU, as it is showed in the section 6.1., the information is stored in the context database. Next section shows the information which is stored. 6.4.2. QOLES DATA LU and CM retrieve relevant events and register them in the context database; inside a table called EServant_Events. These registers have the next fields: ID: Auto numeric. This field is used as key. Timestamp: day/month/year Hour:minutes:seconds Event: kind of event (see Table 42) Provider: Name of the bundle which register the event Parameters: Text field where the relevant parameters associated to the event are stored separated by commas (csv format) CM registers all the changes in the context such as the fridge door open event or the hob fire level selected. This kind of events has been defined as low-level events because they don’t require any analysis to be used. LU processes and analyses these events in order to detect more complex situations; defined as high-level events. Next table summarize the high-level events that are used as inputs by the QoLES. Event Description Parameters Related technology F_NewItemDetected New shop: When a new item or group of items is added to the fridge, the LU registers the event Note that it is only possible when the food is wearing a RFID tag. New Item list Fridge (superdevice) Standalone RFID reader F_RemoveTime Time between the user is informed about an expired product and when it’s removed Time (hours) Fridge (superdevice) F_FoodOutOfDate Food out of date Item list Fridge (superdevice) F_NumberExpiredProduct Store the number of expired product in the last month Month, number of expired products Fridge (superdevice) F_ErrandBehaviour Number of times the fridge’s door is open without taking/introducing anything in the last month Month, number of times Fridge (superdevice) F_DoorOpenWarning Number of warnings in the last month and average of reminders Month, number of warnings, average of reminders Fridge (superdevice) F_ AfterhoursActivity Fridge afterhours activity detected - Fridge (superdevice) Chapter 6: Reasoning -98WM_Score Washing machine score (coherence of washing program-clothing colourtemperature) 0 = bad selection 1 = no appropriate selection 2 = suitable selection 3 = most suitable selection Washing Machine (superdevice) WM_Washing Washing Content Washing Machine (superdevice) WM_AfterhoursActivity Washing machine afterhours activity detected - Washing Machine (superdevice) H_LowTimeOn Number of times that the hob is switched on for less than 10 seconds in a day. Monthly average. Month, Time (s) Hob (Appliance) H_ForgottenOn Number of times that the hob is forgotten on in the last month Month, Number of times Hob (Appliance) H_TimeHobOn Average time that the hob is on in the morning, in the afternoon, in the evening and at night in the last month Month, morning Avg time on, afternoon average time, evening average time, night average time Hob (Appliance) H_AfterhoursActivity Hob afterhours activity detected - Hob (Appliance) K_AfterhoursActivity Activity in the kitchen afterhours detected - PIR (Sensor) HMI_times Average navigation times in the interaction with the HMI for each scenario in the last month Month, scenarioID, number of executions, avg time of navigation Communication database analysis Table 42 High-level events relevant to the QoLES 6.5. CONCLUSIONS Following the same philosophy as in previous chapters, LU has been designed modularly in several bundles, isolating the management of the user scenario or the kitchen events from the core layer which manages the intelligence of the system. This intelligence is based on a set of deterministic rules, designed to cover the users’ needs detected in the chapter 3, divided in four areas: food management, cooking, washing and stand-alone RFID reader. Current logic rules have been designed for the assessment of the system with real users. However, thanks to the Core structure which separates the rule management from the rule implementation, it is simple to add/remove rules, even remotely or/and dynamically due OSGi framework’s capacities. Although the first intelligence concept proposed an adaptive system which adjust dynamically the level of help to the capacities of user, this idea was discarded in the co-design sessions (see section 3.5.2). Experts agreed that this feature could disorient the user which detects that, while doing the same things, the behaviour of the system changes. Therefore, deterministic rules assure the same system behaviour to the same user profile, situation and interaction. Thus, carer, relative or social worker is responsible to adjust the user profile and explain the user the changes when required. Universidad de Zaragoza -99Quality of Life Evaluation System has been designed as a support tool in order to provide information to the carer, social worker or relative about the changes in the user’s habits which could be related with a loss of capacities of the user. On that score, this chapter shows the work done in the conceptualization and design of the QoLE System as well as implementation of the data-retrieving layers. Chapter 7: Smart kitchen assessment -100CHAPTER 7: SMART KITCHEN ASSESSMENT This chapter describes the methodology and tools developed for the assessment of the Smart Kitchen. This evaluation was conducted simultaneously in two countries: UK and Spain, by multidisciplinary teams from the Glyndŵr University and the University of Zaragoza (Casas, Blasco et al. 2010). Finally, the results obtained are presented. 7.1. INTRODUCTION When technology is evaluated, it is necessary to consider a range of social, technological, institutional and personal factors (Ballantine, Galliers et al. 1996). Additionally, legal and ethical aspects have mandatory consideration when users are elderly or disabled (Casas, Marco et al. 2006). In this context, there is an interdisciplinary crossing between the technical development of the innovation and its implementation in intervention situations with social assistance which could be considered itself as trans-disciplinary (Nicolescu, Camus et al. 1998). Literature address this situation from different approaches: about user satisfaction (Demers, Monette et al. 2002), about its psycho-social impact (Day, Jutai et al. 2002), about the person, his environment, the technology and the impact of the period training (Goodman, Luft 2002), about the functional independence (Shone, Ryan et al. 2002), or how they affect the quality of life (Scherer, Laura et al. 2001, Schalock, Verdugo Alonso 2003). Regarding to the evaluation of the Assistive Technology (AT), we can find that the use of interaction models in the assessment is quite usual (Hasdoǧan 1996). These models are used mainly for AT outcomes research highlighting the next: Human Activity Assistive Technology (HAAT), Matching Person and Technology (MPT) and the ICF (Lenker, Paquet 2003, Cook, Hussey 2001, Scherer, Craddock 2002). In the end, it is usual to design a specific "tool" in order to obtain evidences about if a particular technological innovation responds or not to the purpose for which it is designed. Although technology assessment has traditionally followed quantitative techniques based on the accepted convention of their rigor, the prevailing reality shows that qualitative assessments can be equally rigorous. In fact, the prevailing trend tends to combine both (Combessie 1986, Groger, Straker 2002, Howe 1992, Jick 1979, Rossi 1994, Morgan 1997, Teddlie, Tashakkori 2003), minimizing the prejudices of each of these methodologies through the contrast processes. This is the philosophy that has been followed in the design of the system assessment: to combine qualitative tools such as observation or interview with quantitative tools such as surveys and data collection. 7.2. ASSESSMENT DESIGN 7.2.1. RATIONALE Although interaction with users and related groups have been considered during the design process, once the Smart Kitchen prototype is available it is necessary to prove its utility. In this sense, the evaluation of the AAL in the kitchen raises three questions: Which people are able to use the system? How easy is to interact with it? Does the Smart Kitchen meet the purpose for which it was designed? Answering these three questions determines the accessibility, usability and functionality of the system. System assessment follows a methodology in line with the research approach proposed in Chapter 3 to study the needs of the target population. That is, it starts with the study of human-device interaction following the model proposed by Abowd and Beale linking it with the parameters studied (accessibility, usability and functionality). This study is the base of the assessment and it is reflected in specific tools in the section 7.2.3. Universidad de Zaragoza -101As it is raised in Chapter 3, user must be able to understand the stimuli perceived from the machine, to process their related information and to produce a response that the system can understand according to the context and the interaction objectives. Therefore, the critical user’s capacities for the interaction are related with: cognition, sensory, physical (movement, pulsation, etc.) and voice. Thus, studying the system interface, it is possible to identify its input and output communication channels; understanding these channels as the different ways a person and device can use to exchange information. In addition, it is needed to consider the cognitive process which governs the stimuli generation depending on the perceived stimuli and the current context. Attending to these processes, the evaluation of the usability, accessibility and functionality has been done as follows: Accessibility is defined by the ISO 9241-171 and 9241-20 (ISO 9241-171 2008, ISO 20282-1 2006) as “usability of a product, service, environment or facility by people with the widest range of capabilities” i.e. it describes how a system is available to a person. Thinking in the human-device interaction, this assessment has defined it in the next way: A system is accessible for a person if he/she can fully interact with it being able to access all of the functionalities, independently of the communication channel used. This interaction implies the perception and comprehension of the information exchanged. Therefore, in the design of the evaluation it is important to know the perception and comprehension of each channel by the user. Usability is defined by the ISO 9241-11 (ISO 9241-11 1998) as “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use." i.e. it assesses the complexity of handling the input and output channels of the system. This parameter is related with the cognitive, physical and sensorial capacities of the person. To assess this parameter, time and capability to resolve several situations as well as the direct user’s opinion will be evaluated. Functionality evaluates if the system accomplishes the initial requirements, this means that the system covers the functions for which it was designed. In this case, main functionalities of the system are:  To facilitate the use of the household appliances adapting to the disabilities or preferences of the user and using adapted interfaces.  To provide useful information and warnings about the use of the household appliances.  To detect emergency situations and automatically take some actions: warn the user, switch off some appliances or request for help. 7.2.2. ASSESSMENT APPROACH Evaluating an AAL system as the one developed poses several challenges. First, the system should be evaluated in a situation as close to its real usage as possible. Ideally, the best option would be to install the Smart Kitchen in the actual houses of the users to avoid introducing further complications which contribute to user disorientation. Furthermore, the Smart Kitchen should be tested for a long time (at least one year) in order to evaluate all the features as the QoLE. Additionally, the number of users should be enough to have a representative sample in order to extract significant assessment evidences. All these factors have been considered inside the Easy Line Plus project as the frame of the development of this thesis. On the one hand, the assessment process was limited in time and number of users. Likewise, existing resources did not allow evaluating the system within the users’ houses so assessment was performed under lab conditions. These labs, described in the section 7.3.1, simulate an environment close to a real situation; that is, Chapter 7: Smart kitchen assessment -102the users’ house. Main drawback of this evaluation design is the associated user disorientation because they are in an unknown kitchen with unknown appliances. One of the possibilities raised in the University of Zaragoza’s lab was to make a “long time” evaluation (one or two weeks) where users had enough time to become familiar with the environment. However, this possibility was discarded due to three reasons: by ethical issues, because the change of location could cause harmful implications in an elderly person (disorientation, loss of capacities, mobility problems in a unknown neighbourhood); by technical reasons, given the time limitation in the evaluation, the number of users involved is drastically limited and therefore, the relevance of their evaluation conclusions; by legal issues caused by any problem or risk that the person could suffer during the assessment. It was finally decided to perform an evaluation where the user first had a brief period of training in the use of the system and then face four situations that may occur in the kitchen. These situations have been designed in order to allow evaluation of the main system’s functionalities studying the interaction between user and system (See Annex III. Functionalities evaluated in each test situation): Situation 1: “Coming home from shopping”. The participant comes home from shopping and he/she is required to store all the items from the shopping bag into the fridge, freezer or cupboards. One of the items is out of date. Situation 2: “Making dinner”. The participant is asked to show a frozen pizza to the standalone reader so the interface identifies the item and asks the user if he/she wants to cook it. If yes, it goes to the set_oven_configuration with the needed parameters. Then the participant goes to the living room (maybe to watch TV) and is warned when the food in the oven is finally done cooking. Situation 3: “Doing my laundry”. The participant is asked to do a laundry (simulated through the spinning program to avoid long waiting times) using different clothing items. Situation 4: “My house is on fire”. The participant is resting and watching TV while the hob is on. The smoke detector is triggered simulating an emergency. There are three people participating in the assessment whose roles have been defined as follows: The user is the person who will evaluate the technology. The test moderator leads the sessions being in charge of interacting with the participants and observing them during the testing. The moderator will introduce the session to the participants and realize several short interviews between the different situations tested. During the test he/she should be neutral. However, he/she can decide to help (for example if user is having a bad time in the interaction) and how much to help. Of course, if the test moderator takes part in the test, this situation will be logged. The test observer is watching the different situations evaluated without contact with the user, taking notes about the participants’ performance and reactions. The observation notes are considered as “objectives” because they are not influenced by the user. Also, the observer will take note about any failure of the system. Once the situations and people involved in the assessment are defined, it is necessary to design tools that will be used; tools are described in section 7.2.3. This process is based in the previous experience of the University of Glyndŵr evaluating the interfaces, adapting their concept to the current assessment and linking it with the human-device interaction model. Stages of the test are as following: Universidad de Zaragoza -109information will be contrasted with the notes taken by the observer who can consider the test is unsuccessful. It is considered that the system is functional for a particular user if the level of usability is greater than or equal to 3. This information has been contrasted with the caregivers’ opinion in order to know both points of view. d) Others parameters User opinion about performance, satisfaction with the product and future outcome is provided by the information collected in the UQ. In the case of performance, the notes of the observer related with system fails are also considered. In the same way that with the rest of parameters, the system has a good performance/satisfaction/future use if its score is greater than or equal to 3. Next table summarizes these evaluations: Parameter evaluated User questionnaire Other related parameters Performance (Q23+Q24)/2 Notes on the observer about system failures Satisfaction (Q25+Q26+Q27)/3 Future use (Q28+Q29+Q30)/3 Table 46 Performance, satisfaction and future use 7.3. EVALUATION RESULTS 7.3.1. TEST PLACES The evaluation of the system was done in two places: Spain and Wales. In Zaragoza, the study took place in a flat owned by the University of Zaragoza and located in a residential area. It was fully furnished and provided with all necessary facilities required to carry out any normal activities in everyday life. The testing environment was exclusively situated at the main entrance of the flat, in the living room (with the TV acting as user interface) and in the kitchen with the full deployment of the AAL. Figure 35.Evaluation Pilot of the University of Zaragoza At Glyndŵr University (Wales, U.K.), the study took place in the CAIR usability laboratory. There is a testing room with a one-way mirror to an observation room. The kitchen part of the room consists of a washing machine, an oven, a cooker hob and a fridge with the system. Chapter 7: Smart kitchen assessment -110Figure 36 Usability lab at Glyndŵr University. 7.3.2. ASSESSMENT PARTAKERS 7.3.2.1. Users Sixty-three users have participated in the evaluation of the system. As it has been commented, disabled people younger than 59 years old have been recruited to increase the ratio of people with disabilities. A relevant limitation that hinders obtaining conclusions about the ability of the system to cope with specific disabilities is that users usually have more than one disability being very difficult to isolate the effects of each one over the system use. As described in section 7.2.3.1, each user has been parameterized in a radial graphic (Figure 33) simplifying the interpretation of these parameters. Next table describe the population participating in the evaluation 10 : Characteristics Recruited participants Impairments/Disability none: visual impairment: hearing impairment: cognitive impairment: motor impairment: 26 13 13 12 23 Age <59: 60–79: 80+: 11 (3 male/8 female) 45 (17 male/28 female) 7 (3 male/4 female) Gender Female: Male: 40 23 Total: 63 participants Table 47 Recruited participants 10 Note that a person could have more than one disability. Universidad de Zaragoza -1117.3.2.2. Caregivers Thirty-one caregivers have participated in the assessment of the system. This sample is composed for professionals and family carers from Wales and Spain, next figures show detailed information about their composition: Figure 37 Caregivers' professional skill Figure 38 Caregivers' work place 7.3.3. SYSTEM DATA RESULTS Data collected during testing were analysed using the process described in Section 7.2.4. Following, the results of the overall evaluation have been summarized; the detail about individual results of each person can be found in the website of the EL+ Project, in the D.7.2. (Blasco, Casas et al. 2010). System usability perceived by users, shows that management of communication channels is not difficult for most people involved in the evaluation. The average score in usability is 3.85 with a confidence interval (95%) between 3 and 5. Observer annotations agree in this point. However, it should be noted that each user has selected the channel or channels of communication best suited to his/her needs. Following chart shows the selected input channels in the evaluations: Chapter 7: Smart kitchen assessment -112Figure 39 Preferred interfaces for the users In this regard, note that the majority of users felt more comfortable using the remote control of television as interface, followed by the touch screen which has proven to be quite intuitive. By contrast, the use of voice commands have only been used by those users who really had problems with other interfaces. Focusing on the system’s accessibility, input and output channels have been evaluated calculating the average score perceived and observed with a confidence interval of 95%, with the following results: Figure 40 Visual output channel 2 62 52 0 20 40 60 80 Speak command control TV remote control Touch screen number of users 5 5 5 5 3 3 2,55 2 4,06 4,68 3,78 3,87 0 1 2 3 4 5 Visual perception (perceived) Visual comprehension (perceived) Visual perception (observed) Visual comprehension (observed) Universidad de Zaragoza -113Figure 41 Aural output channel Figure 42 Speak input channel Figure 43 TV remote control input channel 5 5 5 5 2 1,55 3 3 4,03 3,59 4,21 4,16 0 1 2 3 4 5 aural perception (perceived) aurall comprehension (perceived) aural perception (observed) aural comprehension (observed) 5 5 5 5 3,6 2,2 4,075 5 4,52 4,60 4,75 5,00 0 1 2 3 4 5 Speak command control perception (perceived) Speak command control comprehension (perceived) Speak command control perception (observed) Speak command control comprehension (observed) 5 5 5 5 2,55 3 2 2 4,05 4,87 3,56 3,68 0 1 2 3 4 5 TV remote control perception (perceived) TV remote control comprehension (perceived) TV remote control perception (observed) TV remote control comprehension (observed) Chapter 7: Smart kitchen assessment -114Figure 44 Touch screen control input channel In general, the perceived accessibility is slightly higher than observed. Recalling the criteria set in the preceding sections, it was considered that the system is accessible to a person “if s/he can fully interact with it being able to access to all the functionalities, independently of the communication channel used. This interaction implies the perception and comprehension of the information exchanged”. Analysing the results and considering that a channel is accessible for a user if its score is equal or greater than 3, at least one output channel (visual or aural) is accessible for 98,6% of users. Doing the same with the input channels (tactile, TV remote, spoken), one input channel is at least accessible for 92% of users. Therefore, the system is accessible for the people who can perceive and comprehend at least one input channel and one output channel; in this case the system is accessible for the 90% of the sample. Figure 45 e-Servant functionalities evaluation. User & Carer opinion 5 5 5 5 2 2 2 2 3,81 4,80 3,53 3,85 0 1 2 3 4 5 Touch screen control perception (perceived) Touch screen control comprehension (perceived) Touch screen control perception (observed) Touch screen control comprehension (observed) Universidad de Zaragoza -115In the case of the overall functionality of the system, the average score for this parameter is 3.49 with a confidence interval (95%) between 2.38 and 4.86. Contrasting the assessment of the individual features of the system done by the test participants with the opinion of the carers involved, it is possible to see that they agreed on several points. Figure 45 reflects these opinions from the viewpoint of users (UPV) and of caregivers (CPV). For both, users and carers, “Trigger emergency warnings (fire, smoke, flood) and act in case there is no response” is the functionality rated the highest. For the carers this functionality is followed by “detect routine changes in the kitchen to inform whenever there are changes in conduct patterns that can identify any loss of abilities in the user”. Other functionality with a remarkable evaluation by the user is “Trigger a warning if there is a wrong mix of clothes or unsuitable fabric.” The Figure 46 summarizes the evaluation of the rest of parameters. Performance of the system has been the parameter with worst score: 3,48 over 5. This data, which can be considered as a good result, has been qualified by the observer and by the technical people. The opinion about the future use of the product as well as satisfaction using it has been very positive with a score average close to 4. Figure 46 Performance, satisfaction and future use evaluations 7.3.4. ENHANCEMENT PROPOSALS OF THE SYSTEM AFTER THE EVALUATION Users’ evaluation has been very useful to analyse the strengths and weaknesses of the system. Forms and interviews with users have allowed taking a lot of information about the system. Also, having real users handling the system have helped to detect unknown problems. Also, because the system was working continuously for a period longer than a month and interacting with many people, new problems related with the performance appeared. Next table summarizes the actuations proposed to enhance the system after the user testing. They have been divided in two groups. One group related with the user interface and the intelligence of the system and the other group related with the context awareness: 5,00 5,00 5,00 2,00 3,33 2,70 3,48 3,99 4,00 0 1 2 3 4 5 Performance Satisfaction Future use Chapter 7: Smart kitchen assessment -116User interface and intelligence To solve reliability problems detected thanks to the long-time user interaction To improve the user feedback about the system status (network connection, etc.) To improve the user feedback about the load/unload process of the washing machine To add configurable interface themes and sounds To display the changes in the content of the fridge in an easier way Added several external communication channels to the system as SMS or e-mail Context awareness and intelligence To solve technical problems detected with RFID: - Reliability problems - Heating problems Table 48 Improvement suggested 7.4. EVALUATION CONCLUSIONS The prototype of Smart Kitchen for AAL has been evaluated in real environments by 63 real end users in order to spot any accessibility, usability, performance, etc. issues. Data from the evaluation has been analysed and enhancement proposals have been done to improve the system. From the accessibility point of view, 90% of the users can perceive and understand at least one input channel (tactile screen, remote control or voice) and output channel (visual or aural) and 70% have the opinion that the system is accessible. Usability of the system has been evaluated with a 3.85 over 5. Therefore we can conclude that the system has good usability and physical, sensory and cognitive accessibility. Several suggestions from the users have been taken in account to improve the accessibility and functionality of the system. User’s opinions about the future use (3.98/5) of the product as well as about their satisfaction (3.99/5) have been also very positive. Functionalities of the system has ben also assessed by 31 professionals and carers directly working with end users. It was observed that both for beneficiaries and carers, the highest rated function of the system is security at home: “Trigger emergency warnings (fire, smoke, flood) and act in case there is no response”. For the carers, this functionality is closely followed by “Detect routine changes in the kitchen to inform whenever there are changes in conduct patterns that can identify any loss of abilities in the user” as it can strongly improve the tools available to monitor the evolution of the elderly and disabled people. From the point of view of the beneficiaries security functions are followed by the reminder services “Trigger warnings and reminders when the appliances require attendance”. There are clear indicators that the functionalities of the system have a big potential to support the user in several areas of the Activities of Daily Living (ADLs), reducing the dependence level of the person. This situation, consequently, could be useful to increase his/her time of independent life. The system can support the user in the ADLs’ areas of carrying out domestic tasks (preparing a meal, doing the shopping and doing laundry/ironing) and making decisions (about domestic tasks). Early detection of changes in routines would also help to monitor quality of life of the user in some aspects. For example, it can be detected if the person is washing less often, which might indicate that he/she is wearing dirty clothes. Universidad de Zaragoza -117CHAPTER 8: CONCLUSIONS AND FUTURE WORK Next section summarizes the conclusions of the work described in the thesis; highlights the most important contributions and proposes the future research lines in the different fields addressed. These contributions have been divided in two main areas: methodological with the results of the chapters 3 and 7, and technological, with the advances shown in the chapters 4, 5 and 6. 8.1. INTRODUCTION Deploying an AAL in the kitchen has a relevant impact increasing the autonomy of the person and their quality of life in their own homes. On the one hand, the kitchen is the place where relevant tasks for the autonomous life take place: cooking, clothes washing or food management. On the other hand, most household accidents happen in the kitchen, being elderly people more likely to suffer them (Angermann, Bauer et al. 2007, Van Den Broek, Cavallo et al. 2010). Therefore, by helping the elderly people in the kitchen it is possible to prevent domestic accident and contribute to increase their autonomy. As presented in the review of the state of the art in chapter 2, creating and AmI in the kitchen is not a new idea. Several systems recognize user activities (Kranz, Schmidt et al. 2007, Lei, Ren et al. 2012), guide the user in the cooking process (Hashimoto, Mori et al. 2008, Siio, Hamada et al. 2007), guide the user to have a healthier diet (Chi, Chen et al. 2007, Chen, Chang et al. 2006, Chen, Chi et al. 2010), research in innovative user interfaces (Schwartze, Feuerstack et al. 2009), support the user to make the shopping list (Anastasopoulos, Niebuhr et al. 2005, Chen, Chang et al. 2006, Gárate, Herrasti et al. 2005, Pal Amutha, Sethukkarasi et al. 2012), provide cognitive assistance to help users storing and retrieving items in order to complete a recipe (Ficocelli, Nejat 2012) or research in user interfaces for people with disabilities considering the kitchen appliances as a part of the house’s smart environment (Neßelrath, Haupert et al. 2011). All systems present features that are interesting to build an AAL system in the kitchen; however to my best knowledge, none approaches the needs from an integral point of view. This is precisely the area where this PhD thesis makes its main contribution to the current state of the art: the design and implementation of an innovative AAL architecture, associated context awareness infrastructure and intelligence and the final evaluation of the system with end users. Main objective of the AAL system is to help elderly and/or disabled people to prolong the time of autonomous life, easing and simplifying their daily tasks in the kitchen while providing domestic accidents’ support. From a methodological perspective, this PhD thesis proposes an innovative methodology for identification of needs in the design of technology. This methodology is based in existing Human-Computer Interaction models and International Classification of Functionalities and derived from the conceptualization, systematization and generalization of the process followed in Easy Line Plus research project. Following the same rationale, the assessment methodology of the Smart Kitchen with real users is also designed and put into practice. From a technical standpoint, the objective stated in this thesis has been successfully accomplished: the design and development of an AAL system that assists the user with his daily activities in the kitchen, simplifying them and helping him/her when necessary, detecting changes in his/her abilities. Also the developed system improves the user security in the kitchen, being able to offer a better answer in emergency situations. This work has been carried out considering market issues such as the need of remote updating and maintenance of the system and keeping costs within reasonable margins. Chapter 8: Conclusions and future work -1188.2. METHODOLOGICAL CONTRIBUTIONS 8.2.1. NEEDS IDENTIFICATION FOR THE DESIGN OF TECHNOLOGY FOR SPECIFIC POPULATION GROUPS The identification of user needs is a crucial part of the design process of a project as it affects all subsequent development. Inside the research project, this identification was done through different instruments such as surveys and workshops, but without any defined methodology. During the drafting process of this PhD thesis and with the perspective that time provides in the review of a job done years ago, a systematization and a formal structuration of the methodology applied has been defined. The methodology includes the interaction needs of a specific population in the design process in three steps: a user characterization, interaction characterization and final identification of the users’ needs. It is based on the Interaction framework proposed by Abowd and Beale and supported by the International Classification of Functionalities (ICF) taxonomy, considering which factors influence and describe them in a standardized language. The approach facilitates obtaining standardized and comprehensive needs which simplifies definition of functional specifications easing the results comparison. Use of ICF language in the characterization of the specific population and the subsequent interaction is also a relevant result by itself as can be reused in the design of assessment of the product. It also permits an iterative process of improvement of the interaction with the product or service as new target groups can be included in order to identify further needs. In addition, as a systematic way to find indicators that hinders the interaction between person and technology, the methodology enhances the multidisciplinary work of the actors involved in the design process. On the one hand, it helps designers and technologists to understand the capabilities of the target population and the difficulties they currently have to perform the functions that are to be covered by the new product. On the other hand, personnel close to the user can understand limitations and possibilities of technology. 8.2.2. TECHNOLOGICAL ASSESSEMENT WITH REAL USERS Evaluation of technology with elderly and disabled people is a discipline by itself. Although there are assessment tools with different levels of standardization, the usual practice is to adapt them according to the needs of the product evaluation. This thesis defines a methodology for the assessment of system‘s accessibility, functionality and usability. It combines quantitative and qualitative tools —interviews, questionnaires and objective observations— to obtain complete results in the assessment, spot strengths and weaknesses of the system and provide information for future redesigns. The prototype of Smart Kitchen for AAL has been evaluated in real environments by sixty-three real end users and thirty-one caregivers with good results: the system has been accessible for the 90% of the sample, the usability score has been evaluated with a 3.85 over 5 and its functionalities have been evaluated with a 3.49 over 5. Users and caregivers agree in the system capabilities to support the user in several areas of the Activities of Daily Living (ADLs), reducing the dependence level of the person. Trigger emergency warnings and detect routine changes have been the functionalities better valued. Universidad de Zaragoza -125REFERENCES AAL ASSOCIATION, 22/2/2013, 2013-last update, AAL European Program. Available: http://www.aaleurope.eu/. AARTS, E. and ENCARNAÇÃO, J.L., 2006. Into Ambient Intelligence. True Visions. Springer Berlin Heidelberg, pp. 1-16. AARTS, E. and WICHERT, R., 2009. Ambient Intelligence. In: H. BULLINGER, ed, Technology Guide. Springer Berlin Heidelberg, pp. 244-249. ABOWD, G.D., 1991. Formal Aspects of Human-Computer Interaction. Thesis submitted for the degree of Doctor of Philosophy, University of Oxford. ABOWD, G.D., BEALE, R., DIAPER, D. and HAMMOND, N., 1991. Users, systems and interfaces: A unifying framework for interaction, People and Computers VI: HCI'91: Usability Now: Proceedings of the British Computer Society Special Interest Group on Human-Computer Interaction 1991, Cambridge University Press, pp. 73-87. ANASTASOPOULOS, M., NIEBUHR, D., BARTELT, C., KOCH, J. and RAUSCH, A., 2005. Towards a reference middleware architecture for Ambient Intelligence systems, ACM conference on object-oriented programming, systems, languages, and applications 2005. ANDRUSHEVICH, A., KISTLER, R., BIERI, M. and KLAPPROTH, A., 2009. ZigBee/IEEE 802.15.4 Technologies in Ambient Assisted Living Applications, 3rd European ZigBee Developers' Conference (EuZDC) 2009. ANGERMANN, A., BAUER, R., NOSSEK, G. and ZIMMERMANN, N., 2007. Injuries in the European Union. Statistics Summary 2003-2005. Vienna, Austria: Austrian Road Safety Board. AROZARENA, A., ASENSIO, A., BLASCO, R., CASAS, R., COARASA, V., FALCO, J., GROUT, V., HASSAN, R., IBARZ, A., MARCO, A., PICKING, R. and ROY, A., 2011. Assembly and method for monitoring at least one household appliance, EP2302312 (A1). AROZARENA, A., ASENSIO, A., BLASCO, R., CASAS, R., FALCO, J., GROUT, V., IBARZ, A., MARCO, A., PICKING, R. and ROY, A., 2011. Assembly and method for monitoring a set of household appliances, EP2302605 (A1). AROZARENA, A., ASENSIO, A., BLASCO, R., CASAS, R., FALCO, J., IBARZ, A., MARCO, A., HASSAN, R., NITSCHE, K., KRAHMER, M. and SMIRNOV, A., 2011. Refrigerator and method for operating a refrigerator, EP2306129 (A1). -126AROZARENA, A., ASENSIO, A., BLASCO, R., CASAS, R., MARCO, A., HASSAN, R., NITSCHE, K., KRAHMER, M., SMIRNOV, A., ARTIGAS, J.I. and COARASA, V., 2011. Washing machine and method for operating a washing machine, EP2302122 (A1). AUGUSTO, J.C., NAKASHIMA, H. and AGHAJAN, H., 2010. Ambient intelligence and smart environments: A state of the art. Handbook of Ambient Intelligence and Smart Environments. Springer, pp. 3-31. AZTIRIA, A., IZAGUIRRE, A. and AUGUSTO, J.C., 2010. Learning patterns in ambient intelligence environments: a survey. Artificial Intelligence Review, 34(1), pp. 35-51. BALLANTINE, J.A., GALLIERS, R.D. and STRAY, S.J., 1996. Information systems/technology evaluation practices: evidence from UK organizations. Journal of Information Technology, 11(2), pp. 129-141. BARTRÉS-FAZ, D., CLEMENTE, I. and JUNQUÉ, C., 1999. Alteración cognitiva en el envejecimiento normal: nosología y estado actual. Revista de neurología, 29, pp. 64-67. BAUER, G., STOCKINGER, K. and LUKOWICZ, P., 2009. Recognizing the use-mode of kitchen appliances from their current consumption, Smart sensing and context 2009, Springer, pp. 163-176. BEECHER, V. and PAQUET, V., 2005. Survey instrument for the Universal Design of consumer products. Applied Ergonomics, 36(3), pp. 363-372. BEYER, H. and HOLTZBLATT, K., 1997. Contextual design: defining customer-centered systems. Morgan Kaufmann. BHATTI, S., CARLSON, J., DAI, H., DENG, J., ROSE, J., SHETH, A., SHUCKER, B., GRUENWALD, C., TORGERSON, A. and HAN, R., 2005. MANTIS OS: An embedded multithreaded operating system for wireless micro sensor platforms. Mobile Networks and Applications, 10(4), pp. 563-579. BLASCO, R., CASAS, R., FALCÓ, J.L., GROUT, V. and PICKING, R., 2010. D.7.2. Report with the results of the end users’ test. Easy Line Plus Project public deliverable. Available: http://www.easylineplus.com/. BLASCO, R., CASAS, R., MARCO, Á, COARASA, V., GARRIDO, Y. and FALCÓ, J.L., 2008. Fall Detector Based on Neural Networks, Proceedings of the International Conference on Bio-inspired Systems and Signal Processing BIOSIGNALS 2008, pp. 540-545. BLASCO, R., MARCO, Á, CASAS, R., IBARZ, A., COARASA, V. and ASENSIO, Á, 2009. Indoor localization based on Neural Networks for non-dedicated ZigBee networks in AAL, Bio-Inspired Systems: Computational and Ambient Intelligence 2009, Springer, pp. 1113-1120. Universidad de Zaragoza -127BLUETOOTH, 2012-last update, Bluetooth Technology Web Site. Available: http://www.bluetooth.com/Pages/Bluetooth-Home.aspx, 2012. BONO NUEZ, A., MARTÍN DEL BRÍO, B., BLASCO MARÍN, R., CASAS NEBRA, R. and ROY YARZA, A., 2009. Quality of Life Evaluation of Elderly and Disabled People by Using Self-Organizing Maps, Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living 2009, Springer, pp. 906-913. BONO NUEZ, A. and ROY YARZA, A., 2008. Evaluación de la calidad de vida de ancianos y discapacitados mediante técnicas de reconocimiento de patrones y mapas autoorganizados, Universidad de Zaragoza. BOTTS, M., PERCIVALL, G., REED, C. and DAVIDSON, J., 2008. OGC® Sensor Web Enablement: Overview and High Level Architecture. In: S. NITTEL, A. LABRINIDIS and A. STEFANIDIS, eds, GeoSensor Networks. Springer Berlin Heidelberg, pp. 175-190. BSH, 2012-last update, BSH Electrodomésticos España, S.A. Available: http://www.bsh-group.es. BSH, 2007-last update, Siemens Serve@Home home management system | Appliancist. Available: http://www.appliancist.com/accessories/siemens-home-management-system.html, 2012. CARETECH, 2011, 2012-last update, Digital Carephone CareIP. Available: http://www.caretech.se/en-careip/endigital-carephone-careip.html, 2012. CASAS, R., BLASCO, R., GROUT, V., PICKING, R., ROBINET, A., HASSAN, R., MARCO, A. and ROY, A., 2009. D.1.1.Definition of end users and technical requirements.Easy Line Plus Project public deliverable. Available: http://www.easylineplus.com/. CASAS, R., BLASCO, R., LAÍN, L., MARTÍN-DEL-BRÍO, B., BONO, A. and MARCO, A., 2009. D.3.2. Design, architecture, development and test of e-servant. Easy Line Plus Project confidential deliverable. CASAS, R., BLASCO, R., VAQUERIZO, E., FALCÓ, J.L., GARRIDO, Y., PICKING, R., GROUTH, V., ROBINET, A. and ROY DELGADO, A., 2010. D.7.1. End Users Evaluation Protocol. Easy Line Plus Project confidential deliverable. CASAS, R., BLASCO, R., ROBINET, A., DELGADO, A., YARZA, A., MCGINN, J., PICKING, R. and GROUT, V., 2008. User Modelling in Ambient Intelligence for Elderly and Disabled People. Computers Helping People with Special Needs 2008, pp. 114-122. CASAS, R., MARCO, Á, FALCÓ, J., ARTIGAS, J. and ABASCAL, J., 2006. Ethically Aware Design of a Location System for People with Dementia, K. MIESENBERGER, J. KLAUS, W. ZAGLER and A. KARSHMER, eds. In: Computers Helping People with Special Needs 2006, Springer Berlin Heidelberg, pp. 777-784. -128CHAN, M., CAMPO, E., ESTÈVE, D. and FOURNIOLS, J.Y., 2009. Smart homes—Current features and future perspectives. Maturitas, 64(2), pp. 90-97. CHEN, C., DAS, B. and COOK, D.J., 2010. Energy prediction based on resident's activity, In Proceedings of the International workshop on Knowledge Discovery from Sensor Data 2010. CHEN, J., CHANG, K., CHI, P. and CHU, H., 2006. A smart kitchen to promote healthy cooking, Proceedings of the 8th International Conference on Ubiquitous Computing (ACM UBICOMP 2006) 2006. CHEN, J., KWONG, K., CHANG, D., LUK, J. and BAJCSY, R., 2006. Wearable sensors for reliable fall detection, Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the 2006, IEEE, pp. 3551-3554. CHEN, J.H., CHI, P.P.Y., CHU, H.H., CHEN, C.C.H. and HUANG, P., 2010. A smart kitchen for nutrition-aware cooking. Pervasive Computing, IEEE, 9(4), pp. 58-65. CHENG, C. and BONANNI, L., 2006-last update, counter intelligence - mit media lab. Available: http://www.media.mit.edu/ci/projects/intelligentspoon.html, 2012. CHI, P., CHEN, J., CHU, H. and CHEN, B.Y., 2007. Enabling nutrition-aware cooking in a smart kitchen, CHI'07 extended abstracts on Human factors in computing systems 2007, ACM, pp. 2333-2338. C-LAB, 2012-last update, Cooperative computing and communication laboratory. Available: http://www.clab.de/, 2012. CLARKSON, P., COLEMAN, R., HOSKING, I. and WALLER, S., 2007. Inclusive design toolkit. Engineering Design Centre, University of Cambridge, UK. COMBESSIE, J.C., 1986. A propos de méthodes : Effets d'optique, heuristique et objectivation. Bulletin de Méthodologie Sociologique, 10(1), pp. 4-24. COMYN, G., OLSSON, S., GUENZLER, R., ÖZCIVELEK, R., ZINNBAUER, D. and CABRERA, M., 2006. User needs in ICT research for independent living, with a focus on health aspects. Brussels: European Commission, Directorate-General Joint Research Centre. Institute for Prospective Technological Studies. CONTIKI OS, 2012-last update, Contiki OS. Available: http://www.contiki-os.org/, 2012. COOK, A.M. and HUSSEY, S., 2001. Assistive technologies: Principles and practice. Universidad de Zaragoza -129COOK, D.J., AUGUSTO, J.C. and JAKKULA, V.R., 2009. Ambient Intelligence: Technologies, applications, and opportunities. Pervasive and Mobile Computing, 5(4), pp. 277-298. COPPER, A., 2004. The inmates are running the asylum. AMS publishing. DAY, H., JUTAI, J. and CAMPBELL, K., 2002. Development of a scale to measure the psychosocial impact of assistive devices: lessons learned and the road ahead. Disability & Rehabilitation, 24(1-3), pp. 31-37. DEMERS, L., MONETTE, M., LAPIERRE, Y., ARNOLD, D. and WOLFSON, C., 2002. Reliability, validity, and applicability of the Quebec User Evaluation of Satisfaction with assistive Technology (QUEST 2.0) for adults with multiple sclerosis. Disability and Rehabilitation, 24, 1(3), pp. 21-30. DEMUTH, H. and BEALE, M., 2003. Neural Network Toolbox. For use with MATLAB. Nineth edn. The MathWorks, Inc. DEY, A.K., ABOWD, G.D. and SALBER, D., 2001. A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Human-Computer Interaction, 16(2-4), pp. 97-166. DIX, A., FINLAY, J., ABOED, G.D. and BEALE, R., 2004. Human-Computer Interaction. Prentice hall. DUCATEL, K., BOGDANOWICZ, M., SCAPOLO, F., LEIJTEN, J. and BURGELMA, J., 2001. Scenarios for Ambient Intelligence in 2010, ISTAG, IPTS, Seville 2001. DUMAS, B., LALANNE, D. and OVIATT, S., 2009. Multimodal interfaces: A survey of principles, models and frameworks, Human Machine Interaction 2009, Springer, pp. 3-26. EASY LINE +, 2010, 2010-last update, Easy Line + project website. Available: http://www.easylineplus.com/, 2012. EDEAN, 1999, 1999-last update, Approaches to Design for All. Available: http://www.education.edean.org/index.php?row=3&filters=f16&cardIndex=21. EEML, 2008-last update, Extended Environments Markup Language: EEML. Available: http://www.eeml.org/, 2013. EETIMES EUROPE, 2010-last update, Freescale and Indesit collaborate on smart appliance solutions for ZigBee connected white goods - Electronics Eetimes. Available: http://www.electronics-eetimes.com/en/freescaleand-indesit-collaborate-on-smart-appliance-solutions-for-zigbee-connected-whitegoods.html?cmp_id=7&news_id=222903029, 2012. -130ENOCEAN, 2012-last update, Radio Technology (RF) from EnOcean for Energy Harvesting Wireless Sensor Solutions | EnOc. Available: http://www.enocean.com/en/radio-technology/, 2012. FAYED, N., CIEZA, A. and EDMOND BICKENBACH, J., 2011. Linking health and health-related information to the ICF: a systematic review of the literature from 2001 to 2008. Disability & Rehabilitation, 33(21-22), pp. 19411951. FENG, L., APERS, P. and JONKER, W., 2004. Towards context-aware data management for Ambient Intelligence, Database and Expert Systems Applications. Springer 2004, pp. 422-431. FICOCELLI, M. and NEJAT, G., 2012. The Design of an Interactive Assistive Kitchen System. Assistive Technology, 24(4), pp. 246-258. FREERTOS, 2012-last update, FreeRTOS. Available: http://www.freertos.org/, 2012. FREY, J., NESSELRATH, R. and STAHL, C., 2011. An Open Standardized Platform for Dual Reality Applications, Proceedings of the International Conference on Intelligent User Interfaces (IUI) 2011. G2V, 2012-last update, ISDE ARAGON. Available: http://www.g2v.com/isdearagon/index.htm, 2012. GÁRATE, A., HERRASTI, N. and LÓPEZ, A., 2005. GENIO: An Ambient Intelligence application in home automation and entertainment environment, Proceedings of the 2005 joint conference on Smart objects and ambient intelligence: innovative context-aware services: usages and technologies. ACM 2005, ACM, pp. 241245. GERA-IDENT, 2012-last update, Gis Gera Ident-Systeme GmbH. Available: http://gera-ident.de/, 2012. GIANNAKOURIS, K., 2008. Ageing characterises the demographic perspectives of the European societies. Eurostat, Statistics in focus, 72, pp. 2008. GLYNDŴR UNIVERSITY, 2012-last update. Available: http://www.glyndwr.ac.uk/, 2012. GOODMAN, G. and LUFT, P., 2002. Adoption of assistive technology for computer access among college students with disabilities. Disability and Rehabilitation, 24, 1(3), pp. 80-92. GROGER, L. and STRAKER, J., 2002. Counting and Recounting: Approaches to combining quantitative and qualitative data and methods. Qualitative gerontology: A contemporary perspective. New York: Springer, pp. 179-199. Universidad de Zaragoza -131GROSKY, W.I., KANSAL, A., NATH, S., JIE LIU and FENG ZHAO, 2007. SenseWeb: An Infrastructure for Shared Sensing. MultiMedia, IEEE, 14(4), pp. 8-13. GS1, 2012, 2012-last update, GS1 - The global language of business. Available: http://www.gs1.org/, 2012. GU, G.Z., 2009. Context Aware Computing. Journal of East China Normal University ( Natural Science), 5. GYI, D., SIMS, R., PORTER, J., MARSHALL, R. and CASE, K., 2004. Representing older and disabled people in virtual user trials: data collection methods. Applied Ergonomics, 35(5), pp. 443-451. HASDOǦAN, G., 1996. The role of user models in product design for assessment of user needs. Design Studies, 17(1), pp. 19-33. HASHIMOTO, A., MORI, N., FUNATOMI, T., YAMAKATA, Y., KAKUSHO, K. and MINOH, M., 2008. Smart kitchen: A user centric cooking support system. Proceedings of IPMU, 8, pp. 848-854. HONG, J., SUH, E. and KIM, S.J., 2009. Context-aware systems: A literature review and classification. Expert Systems with Applications, 36(4), pp. 8509-8522. HORNIK, K., STINCHCOMBE, M. and WHITE, H., 1989. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), pp. 359-366. HOWE, K.R., 1992. Getting over the quantitative-qualitative debate, American Journal of Education 1992, pp. 236-256. IBARZ, A., BAUER, G., CASAS, R., MARCO, A. and LUKOWICZ, P., 2008. Design and Evaluation of a Sound Based Water Flow Measurement System, Smart Sensing and Context 2008, Springer Berlin / Heidelberg, pp. 41-54. IEEE 1451, 2011-last update, Smart Transducer Interface Standards. Available: http://www.nist.gov/el/isd/ieee/ieee1451.cfm, 2012. IMSERSO, ed, 2009. INFORME 2008. Las personas mayores en España. Datos estadísticos estatales y por Comunidades Autónomas. ISBN: 978-84-8446-108-1 (Volumen I) edn. Instituto de Mayores y Servicios Sociales (IMSERSO). INE, 2012-last update, Instituto Nacional de Estadistica. Available: www.ine.es. INTERNATIONAL ERGONOMICS ASSOCIATION, 2011-last update, What is Ergonomics. Available: http://www.iea.cc/01_what/What is Ergonomics.html, 2013. -132ISO 20282-1, 2006. Ease of operation of everyday products -- Part 1: Design requirements for context of use and user characteristics. ISO 9241-11, 1998. Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability. ISO 9241-171, 2008. Ergonomics of human-system interaction -- Part 171: Guidance on software accessibility. JARA, A.J., ZAMORA, M.A. and SKARMETA, A.F., 2011. An internet of things---based personal device for diabetes therapy management in Ambient Assisted Living (AAL). Personal and Ubiquitous Computing, 15(4), pp. 431-440. JICK, T.D., 1979. Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24(4), pp. 602-611. JOHNSON, D., CLARKSON, J. and HUPPERT, F., 2010. Capability measurement for inclusive design. Journal of Engineering Design, 21(2-3), pp. 275-288. KEATES, S. and CLARKSON, P.J., 2003. Countering design exclusion: An introduction to inclusive design. Springer Verlag. KILTZ, U., VAN DER HEIJDE, D., CIEZA, A., BOONEN, A., STUCKI, G., ÜSTÜN, B. and BRAUN, J., 2011. Developing and validating an index for measuring health in patients with ankylosing spondylitis. Rheumatology, 50(5), pp. 894-898. KOFLER, M.J., REINISCH, C. and KASTNER, W., 2011. An Intelligent Knowledge Representation of Smart Home Energy Parameters, Proceedings of the World Renewable Energy Congress (WREC 2011) 2011. KOHONEN, T., 1990. The self-organizing map. Proceedings of the IEEE, 78(9), pp. 1464-1480. KRANZ, M., SCHMIDT, A., RUSU, R.B., MALDONADO, A., BEETZ, M., HORNLER, B. and RIGOLL, G., 2007. Sensing technologies and the player-middleware for context-awareness in kitchen environments, Networked Sensing Systems, 2007. INSS'07. Fourth International Conference on 2007, IEEE, pp. 179-186. LEI, J., REN, X. and FOX, D., 2012. Fine-Grained Kitchen Activity Recognition using RGB-D, ACM International Conference on Ubiquitous Computing 2012. LENKER, J.A. and PAQUET, V.L., 2003. A review of conceptual models for assistive technology outcomes research and practice. Assistive Technology, 15(1), pp. 1-15. Universidad de Zaragoza -133LG, 2012-last update, LG Newsroom. Available: http://www.lgnewsroom.com/newsroom/contents/61738, 2012. LI, X. and ZHANG, W., 2004. The design and implementation of home network system using OSGi compliant middleware. Consumer Electronics, IEEE Transactions on, 50(2), pp. 528-534. LIU, W., LI, X. and HUANG, D., 2011. A survey on context awareness, Computer Science and Service System (CSSS), 2011 International Conference on. IEEE 2011, pp. 144-147. LOMBARDI, A., FERRI, M., RESCIO, G., GRASSI, M. and MALCOVATI, P., 2009. Wearable wireless accelerometer with embedded fall-detection logic for multi-sensor ambient assisted living applications, Sensors, 2009 IEEE 2009, pp. 1967-1970. MARCO, A., CASAS, R., BAUER, G., BLASCO, R., ASENSIO, A., JEAN-BART, B. and IBAÑEZ, M., 2009. Common osgi interface for ambient assisted living scenarios. Ambient Intelligence and Smart Environments.Volume 3: Behaviour Monitoring and Interpretation – BMI. Netherlands: IOS Press, pp. 336-357. MARCO, A., CIRUJANO, D., CASAS, R., BLASCO, R. and ASENSIO, A., 2012, 2012-last update, osgi4ami project website. Available: http://sourceforge.net/projects/osgi4ami/, 2012. MARTÍN DEL BRÍO, B. and SANZ, A., 2006. Fundamentos de redes neuronales y lógica borrosa. Apuntes de asignatura. Departamento de Ingeniería Electrónica y Comunicaciones. Universidad de Zaragoza. MARTIN, J.L., NORRIS, B.J., MURPHY, E. and CROWE, J.A., 2008. Medical device development: The challenge for ergonomics. Applied Ergonomics, 39(3), pp. 271-283. MEDLINEPLUS, 2013-last update, Cambios en huesos, músculos y articulaciones por el envejecimiento: MedlinePlus enciclopedia médica [Homepage of National Library of Medicine, USA], [Online]. Available: http://www.nlm.nih.gov/medlineplus/spanish/ency/article/004015.htm, 2013. MERICO, D., MILEO, A. and BISIANI, R., 2009. Intelligent Wireless Sensor Networks for Assisted Living, Emerging Paradigms in Informatics, Systems and Communication 2009, pp. 35. MICROSAR OS, 2012-last update, MICROSAR OS. Available: https://www.vector.com/vi_microsar_os_en.html, 2012. MIELE, 2012-last update, Miele@home - Miele International. Available: http://www.miele.com/international/enint/products/miele_at_home.htm, 2012. -134MIKKONEN, M., VÄYRYNEN, S., IKONEN, V. and HEIKKILA, M., 2002. User and concept studies as tools in developing mobile communication services for the elderly. Personal and Ubiquitous Computing, 6(2), pp. 113124. MORGAN, D.L., 1997. Focus groups as qualitative research. Sage Publications, Inc. MOTIVE TECHNOLOGY, 2012-last update, Motive Technology - Bespoke software, hotel websites, booking engine solutions and ecommerce in Cumbria and North West England - Motive Technology. Available: http://www.motivetechnology.com/Home, 2012. NESSELRATH, R., HAUPERT, J., FREY, J. and BRANDHERM, B., 2011. Supporting Persons with Special Needs in Their Daily Life in a Smart Home, Intelligent Environments (IE), 2011 7th International Conference on 2011, IEEE, pp. 370-373. NICOLESCU, B., CAMUS, M., MAGNIN, T. and VOSS, K., 1998. Levels of representation and levels of reality: towards an ontology of science, The concept of nature in science and theology.Genève: Labor et Fides 1998, pp. 94-103. NIPSSR, 2002. Population Projections for Japan:2001-2050. With Long-range Population Projections: 2051 - 2100. National Institute of Population and Social Security Research. NORMAN, D.A., 1988. The design of everyday things. 2002 edn. United States of America: Basic books. O’GRADY, M.J., MULDOON, C., DRAGONE, M., TYNAN, R. and O’HARE, G.M.P., 2010. Towards evolutionary Ambient Assisted Living systems. Journal of Ambient Intelligence and Humanized Computing, 1(1), pp. 15-29. OSGI ALLIANCE, 2012-last update, OSGi Alliance website. Available: http://www.osgi.org/Main/HomePage, 2012. OSGI ALLIANCE, 2010. Challenges for Todays Consumer Electronics. http://www.osgi.org/wiki/uploads/Markets/Challenges for Todays Consumer Electronics.pdf edn. PAL AMUTHA, K., SETHUKKARASI, C. and PITCHIAH, R., 2012. Smart Kitchen Cabinet for Aware Home, SMART 2012, The First International Conference on Smart Systems, Devices and Technologies 2012, pp. 9-14. PARK, S. and KAUTZ, H., 2008. Hierarchical recognition of activities of daily living using multi-scale, multiperspective vision and RFID, Intelligent Environments, 2008 IET 4th International Conference on 2008, IET, pp. 1-4. PAULO, P. and GAC, H., 2000. Manual de geriatría y gerontología. Universidad Católica de Chile. Universidad de Zaragoza -141Indication of fridge/freezer status: On/Off/Problem/Disconnected, door open or closed, current temperature High Medium High Display of fridge/freezer contents High Medium Medium Support for configuration of fridge/freezer settings: Set target temperature High High Medium Indication of washing machine status: On/Off/Problem/Disconnected, door open or closed, time to finish High Medium High Display of washing machine contents High Medium Medium Support for configuration of washing machine settings: Set washing program, switch on/off High High Medium Indication of hob status: On/Off/Problem/Disconnected High Medium High Support for configuration of hob settings: Switch off High High Medium Indication of oven status: On/Off/Problem/Disconnected, time to finish, temperature High Medium High Support for configuration of oven settings: Set target temperature, switch on/off, set starting cooking time, set duration High High Medium Provide useful information and warnings about the use of the household appliances Advise if the fridge/freezer door is left open High High High -142Advise if food is past its use-by date High High High Advise if food is approaching its use-by date Medium Medium High Warning about fridge/freezer breakdown High High High Advise of wrong mix of clothes (e.g. mix of white and coloured clothes) High High High Advise of unsuitable fabrics (e.g. dry clean only) High High High Advise if machine loaded but not yet on High High High Advise if cycle interrupted High High High Advise if unload incomplete High High High Advise when machine is on final spin High High High Advise when cycle finished High High High Warning about washing machine breakdown High High High Advise if hob is left on with no pan High High High Warning about hob breakdown High High High Advise when food in the oven is ready High High High Warning about oven breakdown High High High Inform how a cloth should be washed, its colour, etc. Medium Medium High Inform how food should be cooked, its expiration date, etc. Medium Medium High Detect emergency situation and automatically take some actions Universidad de Zaragoza -143Advise “Fire detected” emergency High High High Advise “Smoke detected” emergency High High High Advise “Water detected” emergency High High High Table 50Avanced functionalities & Expected benefit -144ANNEX II DATA PROCESSING PAPERS FALL DETECTOR BASED ON NEURAL NETWORKS Universidad de Zaragoza -145- -146- Universidad de Zaragoza -147- -148- Universidad de Zaragoza -149- -150-