This article describes how we go about reviewing digital care products and applications when commissioned to do so or when writing for the Telecare EPG.
The Telecare EPG is our online directory of (mostly) digital care products that can be used to help support the independence, safety and wellbeing of people both when they are at home and when out and about. In addition to developing guidance and good practice to help care and housing professionals decide in which situations a particular product will offer the best outcomes, we also undertake reviews of products to provide additional in-depth knowledge and evidence to help inform their choice.
The Purpose of a Review
Our reviews provide an in-depth and impartial evaluation of how useful, effective and reliable a particular product or system is, enabling the reader to make an informed choice about its suitability for a particular individual in a defined situation. It provides a considered verdict on the overall quality and design of an item and how it compares with some of the alternatives that may be available. A review also provides an insight into how a product performs in the real world as opposed to what is claimed in the sales literature. Reviews should be used alongside product guidance to help choose the best products to meet the needs and circumstances of the person (or people) that will be using the device.
The Anatomy of a Review
Our reviews follow a standard format wherever possible, starting with an introduction which places the product in context by discussing who it is aimed at, what needs it addresses, whether it has any unique features, and the potential benefits to be gained from its use.
This is followed by a more detailed description of the features and characteristics of the product, including (but not limited to): appearance, weight, size, specification, programmable options, user interface, connectivity, price, whether it is stand-alone or to be used as part of a system, business model, and interoperability.
We then provide our opinion on the product – using our 5-point product evaluation framework as the basis for our review (more on this later). The framework considers the following key aspects of the product:
- Ease of Use
- Value for Money
Once we have a good idea of the capabilities of the product, we consider whether the product delivers on what it promises and if it offers anything new or better than what is already available. In order to help the reader get an immediate overall impression of our findings, we include a list of pros and cons that help to summarise a product’s strengths and weaknesses.
We score the product against our 5 evaluation criteria to give a more detailed breakdown of how good (or bad) it is in each category. A score of 5 stars indicates that it is one of the best in the category (for its product type) with one star indicating that it is one of the worst. The scores are intended to give a more detailed indication of where a product’s strengths or weaknesses lie. An overall score is then awarded based on an average of these scores.
Finally, the review is summarised pulling out the key messages and delivering a final verdict on the product especially in relation to benefits, features, cost and how it compares with similar products that may also be available.
The Review Process
We usually obtain equipment for review by being offered a sample from the manufacturer or their distributor; occasionally, we purchase the product ourselves, especially if it is to be used subsequently for research and usability studies over an extended period of time.
Depending on the function and complexity of the product, we spend between half a day and 6 weeks evaluating it; sometimes, we require longer, especially if we need to demonstrate the device to an extended audience. In addition to our own findings and opinion, we generally ask for feedback from our network of health, social care or housing professionals, as well as from actual or potential users who may have qualities or specific disabilities that are relevant to the target audience for the device. Our final analysis includes contributions from all these sources.
Our reviews consider how a product works from installation and setup through to operation and maintenance. We also review the quality and suitability of user documentation and any other support materials or resources that are available. The process of product purchase is also considered; in complex cases there may be various product options and alternative models of payment, including on-going charges for service subscriptions or telephony costs.
When a product is designed to be installed in someone’s home, we will install it in an actual property for ‘real-world’ testing. Similarly, for worn products, we will wear the product for as long as it takes to get a good impression of what it has to offer, and any stigma that might be associated with it if it is visible.
Our review also considers the practical issues related to product maintenance, which includes issues such as cleaning, battery charging and/or replacement. This may not always be straightforward for some individuals who may require additional support to keep the product working effectively.
When undergoing this process, we consider the product within the context of our evaluation framework.
Our five point evaluation framework considers the product holistically and scores the findings against 5 key criteria, centred on the concept of design. The five key criteria are:
- Features – What does the product do? What benefits does it provide?
- Ease of use – How easy is it to install, setup, use and maintain?
- Performance – How well does it work? How reliable is it?
- Value for Money – Do the benefits justify the cost?
- Design – How much thought has gone into the product? Does it meet a genuine need? How well has it been implemented? Is it aesthetically pleasing?
Design involves many disciplines – ranging from getting the product specification right through to implementation with engineering and software development. It determines the shape of a product and the colour or typeface used to label a button. Thus, it relates to both form and function – and we consider both when evaluating a product.
Our main focus when evaluating against these criteria is user experience – how does the product make the user feel when it is in their home or on their wrist or when they are interacting with it? The emotional impact of a product is particularly important in the digital care market as it will have a significant effect on how well it is integrated into a person’s everyday life. Products should give people confidence, making them feel independent and in control of their lives; nobody wants the so-called ‘badge of dependence’ – a label often applied to the ubiquitous wireless pendant supplied with basic telecare packages (a device with notoriously poor engagement with end-users). The more a product makes someone feel positive and fits in with their life, the more likely they are to select it, accept it and use it – leading to better user satisfaction and overall outcomes.
The way a product affects user experience will depend on many factors, some of which are objective and some subjective. We take particular interest in the following areas when evaluating a product:
Appearance & Build Quality
Assisted living and telecare technologies have for too long been neglected in terms of their appearance – how they look and fit within someone’s home or on their person. This is in part a legacy of the fact that the end-user of such products has until very recently tended not to be the purchaser. With a growing private market now for telecare and other digital care and assisted living products, the industry has begun to wake up to the fact that people deserve well-crafted objects that do not look out of place or have an obvious ‘health’ or ‘care’ connotation. The growth in wellness and health related wearable products has re-defined people’s expectations as to what personal healthcare products should look like – moving them into the mainstream (and even being considered ‘cool’). Moving forward, where features and reliability are common, it may be that the appearance and build-quality of an item will be a key product differentiator that dictates whether it is selected over an alternative.
Issues we consider include:
- How well has the product been constructed?
- What materials have been used?
- Does the product look good?
- Is it fit for purpose?
- Will it fit in? Is it non-obtrusive?
- Is it available in a choice of colours (or other features that allow user selection)?
This is concerned with what the product does – what benefits it provides to the user and how many useful features it has. It is important that a device meets a genuine need and that it addresses this need effectively. There are a wide range of products available for providing applications to help support people live independently in their own home or when out and about.
They can use various methods and technologies to detect a particular situation, assist or communicate with a user, or notify a responder. The methods chosen and the way in which users, their family members and care professionals can interact with and/or remotely manage products are increasingly important. For example, the ability to access information remotely using a web browser or smartphone app makes it that much more versatile and convenient to use.
The ability to match the function of a product to the specific needs and circumstances of the user is very important. This might involve altering alert thresholds or times, for instance by setting a low temperature threshold to match the circumstances of the user and the timings to correspond with when they are not in bed (after all, there’s no point raising an alert if the room temperature in the living room is below 16°C if the person is tucked up nice and warm in bed!). Another way of personalising a product may involve adjusting how and when alert notifications are generated. This might include altering who receives such notifications – ranging from a live-in carer, through to family members (by email and/or text message) or to a telecare monitoring centre. Other means of personalising the product may include offering it in different colours to match the colour scheme of their home or offering alternative ways of ‘wearing’ a product e.g. using a lanyard around the neck, wrist-worn, or as a brooch or belt-clip. The modes available to tailor a product to the needs of a particular end user will ultimately depend on the nature of the product itself.
This is concerned with the ability of a product to communicate and interact with other people or products or services either in the home or remotely (e.g. a web/cloud-enabled service). Some products are designed to be stand-alone – to operate in isolation from all others. Some may offer a simple relay output, which might be used to link to an existing telecare platform in the home by using a proprietary radio alarm transmitter.
Other devices need to work as part of a system and will typically use wireless radio communications to communicate with other system components. These components may be in the home or may be situated in a remote location (e.g. cloud-based servers). The wireless technology used will depend on many factors and will range from proprietary radio technologies such as in alarm-based telecare applications to more familiar technologies such as Bluetooth Low Energy, ZigBee and Wi-Fi. The technology used will impact on issues such as range, battery life and various other technical capabilities.
Another issue to consider is how well a particular device is able to work with other devices or systems i.e. how interoperable it is. As a rule of thumb, devices tend only to work with others from the same manufacturer; although this situation is slowly changing as the use of common or open standards increase. While the idea of a smart home full of interconnected appliances and devices has been popular, but elusive, for many years, the growth in ‘Internet of Things’ technologies is likely to bring this closer.
User Interface & Ergonomics
Where there is a need for interaction with the product (and often there is not), then the user interface must be simple and intuitive – special consideration should be made for people with sensory impairments or physical problems such as poor dexterity. People with dementia can also present challenging design issues as they may inadvertently (or deliberately) interfere with device settings if they are easily accessible. Where notifications and alerts are required, these should be deployed in a sensible, proportionate and considerate manner, so as to avoid irritating the user into modifying their behaviour to conceal an issue or just giving up on the product.
We look at how the user can interact with the device or system – does it have LEDs, switches and a display on the device? Are these covered? Are they easily accessible? Can they be read and operated easily? Can the device be setup remotely e.g. by using a web-interface or smartphone app? For wearable devices, is it easy to put on and comfortable to wear?
Setup & Installation
We consider how easy a product is to setup and install in the home or on the person and establish if it can be self-installed using basic DIY skills and tools. This includes a consideration of the user documentation that is included with the product and other forms of support that may be available e.g. videos or a helpline. We also consider any consequences in terms of removing the device at a later date when it is no longer required.
With the emerging private-pay market, there is a growing demand for products that can be self-installed (or that at least do not require professional installation). Devices that can be attached to walls using temporary adhesive or Velcro fixings are increasingly common – with the added benefit of being easy to remove and (in theory) not leaving a mark when they are no longer required.
Once installed, the ease with which a device can be setup to operate as required is also evaluated. Plug and play devices that can automatically register with a base unit and ‘just work’ are increasingly popular. Whether installation is by a member of the public or a professionally accredited installer, the process of installation, setup and removal needs to be as simple as possible.
Product maintenance is generally concerned with various tasks that need to be performed to keep the product running reliably and as intended for a period of years. Issues that are considered include:
- Regular product testing, and the need for special test equipment
- Product cleaning
- Sensor or component replacement
- Battery management – charging or replacing
- Applying any updates – this is usually only applicable to apps
These issues are particularly important when the end user is unable to attend to such tasks either because they will not remember to or because the processes involved are too difficult.
Platform & Standards
As mentioned in the section on connectivity, some products are designed to work within an existing platform or operating system. For example, many sensors and peripherals in the digital care space only work with their corresponding monitoring platform. This is particularly true with telecare or home activity monitoring products, where products tend to work only with systems designed by the same supplier. For apps that run on computers, tablets and smartphones, this also refers to the mobile and desktop operating systems which are supported.
There are also numerous European and British standards that apply to this area and we do take note of which products meet these standards.
Performance & Reliability
During our tests, we use the product in order to establish whether it performs reliably and as advertised.
We take into consideration the up-front cost of a product together with any on-going costs or subscriptions. On-going costs may include a subscription to a monitoring or web service or for on-going mobile telephony charges. Sometimes it is possible to rent equipment rather than purchase it outright. The overall cost of ownership is also considered in relation the need to have to replace consumables such as batteries or limited-use sensors.
Reviewing digital care products is a multifaceted exercise covering a wide range of physical devices and software-based apps aimed at a wide user base with variable needs and capabilities. In order to do this consistently, we have developed a holistic evaluation framework centred on the concept of design.