If you've watched movies like Iron Man, featuring virtual assistants, like JARVIS, which you can just have a conversation with and control your home, you probably think that such virtual assistants belong in the realm of science fiction. Earlier this year, Mark Zuckerberg, who runs Facebook set a personal challenge to create a JARVIS style assistant for his own home. "My personal challenge for 2016 is to build a simple AI to run my home and help me with my work. You can think of it kind of like Jarvis in Iron Man." He may be closer to his goal, as he may be giving a demo something this month. For those that don't have an army of engineers to help them, what can be done today? Well, one interesting piece of technology is Amazon's Echo. So what is it? Amazon describes it as, "Amazon Echo is a hands-free speaker you control with your voice. Echo connects to the Alexa Voice Service (AVS) to play music, provide information, news, sports scores, weather, and more—instantly. All you have to do is ask." Designed for your home, it is plugged into the mains and connected to your wifi. It's been on sale to the general public in the USA since last summer, and was originally available in 2014 for select customers.

This week, it's just been launched in the UK and Germany as well. However, I bought one from America 6 months ago, and I've been using it here in the UK every day since then. My US spec Echo does work here in the UK, although some of the features don't work, since they were designed for the US market. I've also got the other devices that are powered by AVS, the Amazon Tap, Dot and also the Triby, which was the first 3rd party device to use AVS. To clarify, the Echo is the largest, has a full size speaker and is the most expensive from Amazon ($179.99 US/£149.99 UK/179.99 Euros). The Tap is cheaper ($129.99, only in USA) and is battery powered, so you can charge it and take it to the beach with you, but it requires that you push a button to speak to Alexa, it's not always listening like the other products. The Dot is even cheaper (now $49.99 US/£49.99 UK/59.99 Euros) and does everything the original Echo can do, except the built-in speaker is good enough only for hearing it respond to your voice commands. If you want to use it for playing music, Amazon expect you to connect the Dot to external speakers. A useful guide comparing the differences between the Echo, Dot and Tap is here. The Triby ($199.99) is designed to be stuck on your fridge door in the kitchen. It's sold in the UK too, but only the US version comes with AVS. Amazon expect you'd have the Echo in the living room, and you'd place extra Dots in other rooms. Using this range of products has not only given me an insight into what the future looks like, but I can see the potential for devices like the Echo (and the underlying service, AVS) to play a role in our health. In addition, I want to share my research on the experiences of other consumers who have tried this product. There are a couple of new developments announced this week which might improve the utility of the device, which I'll cover towards the end of the post.

You can see in this 3 min video, some of the things I use my Echo for in the morning, such as reading tweets, checking the news, weather/my Google calendar, adding new events to my calendar or turning my lights on. For a list of Alexa commands, this is a really useful guide. If you're curious about how it works, you don't have to buy one, you can test it out in your web browser, using Echosim (you will need an Amazon account though)

What's really fun are experimenting with the new skills [i.e apps] that get added by 3rd parties, one of which is how my Echo is able to control devices in my smart home, such as my LifX lights. I tend to browse the Alexa website for skills and add them to my Echo that way. You can also enable skills just by speaking to your device. At the moment, every skill is free of charge. I suspect that won't always be the case.

Some of the skills are now part of my daily routine, as they offer high quality content and have been well designed. Amazon boast that there are now over 3,000 skills. However, the quality of the skills varies tremendously, just like app stores for other devices we already use. For example, in the Health & Fitness section, sorted by relevance, the 3rd skill listed is one called Bowel Movement Facts.

The Echo is always on, it's 7 microphones can detect your voice even if the device itself is playing music and you're speaking from across the room. It's always listening for someone to say 'Alexa' as the wake up word, but you have a button to mute the Echo so it won't listen. I use Siri, but I was really impressed when I started to use my Echo, it was felt quicker than Siri in answering my questions. Anna Attkisson did a 300 question test, comparing her Echo vs Siri, and found that overall, Amazon's product was better. Not only does the Echo understand my London accent, but it also has no problem understanding me when I used some fake accents to ask it for my activity & sleep data from my Fitbit. I think it's really interesting that I can simply speak to a device in my home and obtain information that has been recorded by the Fitbit activity tracker that I've been wearing on my wrist. It makes me wonder about how we will access our health data in the future. Whilst at the moment, the Echo doesn't speak unless you communicate with it, that may be changing in the future, if push notifications are enabled. I can see it now, having spent all day sitting in meetings, and sat on my smart sofa watching my smart TV, my inactivity as recorded by my Fitbit, triggers my Echo to spontaneously switch off my smart TV, switch my living room lights to maximum brightness, announce at maximum volume that I should venture outside for a 5,000 step walk, and instruct my smart sofa to adjust the recline so I'm forced to stand up. That's an extreme example, but maybe a more realistic one is that you have walked much less today than you normally do, and you end up having a conversation with Echo because your Echo says, "I noticed you haven't walked much today. Is everything ok?"

We still don't know about the impact on society as our homes become smarter and more connected. For example, in the USA, those with GE appliances will be able to control some of them with the Echo. You'll be able to preheat the oven, without even getting off your sofa. That could have immense benefits for those with limited mobility, but what about our children? If they grow up in a world where so much can be done without even having to lift a finger, let alone walk a few steps from the sofa to the kitchen, is this technology a welcome advance? If you have a Hyundai Genesis car, you can now use Alexa to control certain aspects of your car. When I read this part of the Hyundai Genesis article, "Being able to order basic functions by voice remotely will keep owners from having to run outside to do it themselves" it made me think about a future where we just live an even more sedentary lifestyle, with implications for an already over burdened healthcare system. Perhaps having a home that is connected makes more sense in countries like the USA and Australia which on average have quite large houses. Given how small the rooms in my London home are, it's far quicker for me to reach for the light switch than to issue a verbal command to my Echo (and wait for it to process the command)

Naturally, some of us would be concerned about privacy. Right now, anyone could walk into the room and assuming they knew the right commands, could quiz my Echo about my activity and sleep data. One of the things you can do in the US (and now in Europe) is order items from Amazon by speaking to your Echo, and Alex Cranz wrote a post saying, "And today it let my roommate order forty-eight Cadbury Creme Eggs on my account. Despite me not being home. Despite us having very different voices. Alexa is burrowing itself deeper and deeper into owners’ lives, giving them quick and easy access not just to Spotify and the Amazon store, but to bank accounts and to do lists. And that expanded usability also means expanded vulnerability.", he also goes on to say, "In the pursuit of convenience we have to sacrifice privacy." Note that Amazon do offer the ability to modify your voice purchasing settings, so that the device would ask you for a 4 digit confirmation code before placing the order. The code would NOT be stored in your voice history. You can also turn off voice purchasing completely if you wish.

Matt Novak filed a FOI request to ask if the FBI had ever wiretapped an Amazon Echo. The response he got, "we can neither confirm nor deny."

If you don't have an Echo at home, how would you feel about having one? How would you feel about your children using it? One thing I've noticed is that the Echo seems to work better over time, in terms of responding to my voice commands. The way that the Echo works is that it does record your voice commands in the cloud, and by analysing the history of your voice commands, it refines its ability to serve your needs. You can delete your voice recordings, although it may make the Echo less accurate in future. Some Echo users whose children also use the device say their kids love it, and in fact got to grips with the device and it's capabilities faster than the parents. However, according to this Guardian article, if a child under 13 uses an Echo, it is likely to contravene the US Children’s Online Privacy Protection Act (COPPA). This doesn't appear to have put off households installing an Echo in the USA, as research suggests Amazon have managed to sell 3 million devices. Another estimate puts the installed user base significantly lower, at 1.6 million. Either way, in the realm of home based virtual assistants, Amazon are ahead, and probably want to extend that lead, with reports that in 2017 they want to sell 10 million of these speakers.

Can the Echo help your child's health? Well a skill called KidsMD was released in March that allows parents to seek advice provided by Boston Children's hospital. After the launch, their Chief Innovation Officer, John Brownstein said, "We’re trying to extend the know-how of the hospital beyond the walls of the hospital, through digital, and this is one of a few steps we’ve made in that space." So I tested Kids MD back in April, and you can see in this 3 minute video what it's like to use. What I find fascinating is that I'm getting access to validated health information, tailored to my situation, simply by having a conversation with an internet connected speaker in my home. Of course, the conversation is fairly basic for now, but the pace of change means it won't be rudimentary forever.

I was thinking about the news last week here in the UK, where it was announced that the NHS will launch a new website for patients in 2017. My first thought was, what if you're a patient who doesn't want to use a website, or for whatever reason can't use a website. If the Echo (and others like it) launch in the UK, why couldn't this device be one of the digital channels that you use to interface with the NHS? Some of us at a grassroots level are already thinking of what could be done, and I wonder if anyone in the NHS has been formally testing an Echo to see how it might be of use in the future?

@shirleyayres @ManeeshJuneja my first thought is that we could hack a natural language front end to the NHS 111 protocols quite easily
— Keith Grimes (@keithgrimes) September 3, 2016

The average consumer is already innovating themselves using the Echo, they aren't waiting years for the 'system' to innovate. They are conducting their own experiments, buying these new products with their own money. One man in the USA has used the Echo to help him care for his aging mother, who lives in a different location from him.

Brilliant use of #AmazonEcho to help a son care for his #aging mother https://t.co/yEAT2wRVyN #IoT pic.twitter.com/YQicNebdO0
— Maneesh Juneja (@ManeeshJuneja) February 5, 2016

In this post, a volunteer at a hospice asks the Reddit community for input on what the Echo could be useful for with patients.

How about Rick Phelps, diagnosed back in 2010 at the age of 57 with Early Onset Alzheimer's Disease, and now an advocate for Dementia awareness. Back in Feburary, he wrote about his experience of using the Echo for a week. What does he use it for? To find out what day it is, not knowing what day it is because of Dementia.

For many of us, consumer grade technology such as the Echo will be perceived as a gimmick, a toy, of being of limited or no value with respect to our health. I was struck by what Rick wrote in his post, "To many, the Amazon Echo is a cool thing to have. Some what of a just another electronic gadget. But to a dementia patient it is much, much more than that.It has afforded me something that I have lost. Memory. I can ask Alexia anything and I get the answer instantly. And I can ask it what day it is twenty times a day and I will still get the same correct answer." Rick also highlights how he used the Echo to set medication reminders.

I have to admit, the Echo is still quite clunky, but the original iPhone was clunky too, and the 1st generation of every new type of technology is usually clunky. For people like Rick, it's good enough to make a difference to the outcomes that matter to him in his daily life, even if others are more skeptical.

Speaking of medication reminders, there was a 10 day Pymts/Alexa challenge this year, using Alexa to "to reimagine how consumers interact with their payments and financial services solutions providers." What I find fascinating is that the winner was DaVincian Healthcare, and they created something called DaVincianRX, an “interactive prescription, communication, and coordination companion designed to improve medication adherence while keeping family caregivers in the loop." You can read more and watch their video of it in action here. People and organisations constantly ask me, where do we look for innovation and new ideas? I always remind them to look outside of healthcare. From a health perspective, most of the use cases I've seen so far involving the Echo are for older members of society or those that care for them.

I came across a skill called Marvee, which is described as "a voice initiated voice-initiated concierge application integrated with the Alexa Voice service and any Alexa-enabled device, like the Amazon Echo, Dot or Tap." Most of the reviews seem to be positive. It's actually refreshing to see a skill that is purpose built to help those with challenges that are often ignored by the technology sector.

In the shift towards self-care, when you retire or get diagnosed with a long term condition for the first time, will you be getting a prescription for an Amazon Echo (or equivalent)? Who is going to pay for the Echo and related services? Whilst we have real world evidence that shows the Echo is making a positive impact on people's lives, I haven't been able to find any published studies testing the Echo within the context of health. That's a gap in knowledge, and I hope there are researchers out there who are conducting that research. Like any product, there will be risks as well as benefits, and we need to be able to quantify those risks and benefits now, not in 5 years time. Earlier I cited how Rick who lives with Alzheimer's Disease finds the Echo to be of benefit, but for other people like Rick, using the Echo might lead to harm rather than benefit. We don't know yet. However, not every application of the Echo will require a double blinded randomised clinical trial to be undertaken. If I can already use my Echo to order an Uber, or check my bank balance, why can't I use it to book an appointment with my doctor?

In the earlier use case, a son looked through the data from his mother's usage of her Echo to spot the signs when something is wrong. Surely, Amazon could parse through that data for you and automatically alert you (or any interested person) that there could be an issue? Allegedly, Amazon is working on improvements to the service where Alexa could one day recognise our emotions and respond accordingly. I believe our voice data is going to play an increasing role in improving our health. It's going to be a new source of value. At an event in San Francisco recently, I met Beyond Verbal, an emotions analytics company. They are doing some really pioneering work. We already have seen the emergence of the Parkinson's Voice Initiative, looking to test for symptoms using voice recordings.

How might a device like the Echo contribute to drug safety? Imagine it reminds you to take your medication, and in the conversation with your Echo, you reply that you're skipping this dose, and it asks you why? In that conversation, you have the opportunity in your own words to say why you have skipped that dose. Throw in the ability to analyse your emotions during that conversation, and you have a whole new world of insights on the horizon. Some of us might just be under the impression that real world data is limited to data posted on social media or online forums, but our voice recordings are also real world data. When we reach a point when we can weave all this real-world data together to get a deeper understanding of our health, we will be able to do things we never thought possible. Naturally, there are immense practical challenges on that pathway, but progress is being made every day. Having all of this data from all of these sources is great, and even if it's freely available, it needs to be linked together to truly make a difference. Researchers in the UK have demonstrated that it's feasible to use consumer grade technology such as the Apple watch to accurately monitor brain health. How about linking the data from my Apple watch with the voice data from my Amazon Echo to my electronic health record?

An Israeli startup, Cordio Medical has come up with a smartphone app for those patients with Congesitve Heart Failure (CHF) that captures voice data, analyses it in real-time, and "detects early build-up of fluids in the patient’s lung before the appearance of physical symptoms", and deviations found in the voice data would trigger an alert, where "These alerts permit home- or clinic-based medical intervention that could prevent hospitalisation." For those CHF patients without smartphones, could they simply use an Echo at home with a Cordio skill? Or does Amazon offer the voice data directly to organisations like Cordio for remote monitoring (with the patient's consent)? With devices like the Echo, if Amazon (or their rivals) continue to grow their user base over the next 10 years, they could have an extremely valuable source of unique voice based health data that covers the entire population.

At present, Amazon has surprisingly made rather good progress in terms of the Echo as a virtual assistant. However, other tech giants are looking to launch their own products and services. For example, Google Home, that is due to arrive later this year. This short video shows what it will be able to do. Now for me, Google plays a much larger role in my daily life than Amazon, in terms of core services. I use Google for email, for search, for my calendar, and maps for navigation. So, Google's Home might be vastly superior to Echo, simply because of that integration with those core services that I already use. We'll have to wait and see. The battle to be a fundamental part of your home is just beginning, it seems.

The battle to be embedded in every aspect of our lives with extend beyond the home, perhaps in our cars. I tested the Amazon Dot in my car, and I reckon it's only a matter of time before we see new cars on sale with these virtual assistants built into the car's systems, instead of being an add-on. We already have new cars coming with 4G internet connectivity, offering wifi for your devices, from brands like Chevrolet in the USA.

For when we are on the move, and not in our car or home, maybe we'll all have earphones like the new Apple Airpods, where we can discreetly ask our virtual assistants to control the objects and devices around us. Perhaps Sony's product, the Experia Ear, which launches in November, and is powered by something called Sony's Agent, which could be similar to Amazon's AVS, is what we will be wearing in our ears? Or maybe none of these big tech firms will win the battle? Maybe it will be one of us, or one of our kids who comes up with the virtual assistant that will rule the roost? I'm incredibly inspired after watching this video where a 7 year old girl and her father built their own Amazon Echo using a Raspberry Pi. This line in the video's description stood out to me, "She did all the programming following the instructions on the Amazon Github repository." Next time there is a health hackathon, do we simply invite a bunch of 7 year old kids and give them the space to dream up new solutions to problems that we as adults have created? Or maybe it should be a hackathon that invites 7 year olds with their grandparents? Or maybe we have a hackathon where older adults are invited to co-design Alexa skills with younger people for the Echo? We don't just have financial deficits in health & social care, but we have a deficit of imagination. Amazon have a programming tutorial where you can build a trivia skill for Alexa in under an hour. When it comes to our health, do we wait for providers to develop new Alexa skills, or will consumers start to come together and build Alexa skills that their community would benefit from, even if that community happens to be a community of people scattered around the world, who are all living with the same rare disease?

You'll have noticed that in this post, I haven't delved into the convergence of technologies that have enabled something like the Echo to work so well. This was deliberate on this occasion. At present, I'm really interested in how virtual assistants like the Echo make you feel, rather than the technical details of the algorithm being used to recognise my voice. For someone living far away from their aging parents/grandparents, does the Echo make you feel reassured? For someone living alone and feeling social isolated, does the Echo make you feel not alone? For a young child, does it make you feel like you can do magic, controlling other devices just with your voice? For someone considering moving out of their own home into an institution, does the Echo make you feel independent again? If more and more services are becoming digital by default, how many of these services will be available just by having a conversation? I am using my phone & laptop less since I've had my Echo, but I'm not yet convinced that virtual assistants will one day eliminate the need for a smartphone, but some of us are convinced. 50% of urban smartphone owners around the world believe that smartphones will be no longer be needed in 5 years time. That's one of the findings when Ericsson Consumer Lab quizzed smartphone users in 13 cities around the globe last year. The survey is supposed to represent the views of 68 million urban citizens. In addition, they also found, "Furthermore, a third would even rather trust the fidelity of an AI interface than a human for sensitive matters. 29 percent agree they would feel more comfortable discussing their medical condition with an AI system." I personally think the consumer trends identified have deep implications for the nature of our interactions with respect to our health. Far too many organisations are clinging on to the view that the only (and best) way that we interact with health services is face to face, in a healthcare facility, with a human being. Despite these virtual assistants at home not needing a smartphone with a data plan to work, they would need fixed broadband to work. However, looking at OECD data from December 2015, fixed broadband penetration is rather low. The UK is not even at 40%, so products such as the Echo may not be accessible for many across the nation who might find it beneficial with regard to their health. This is an immense challenge, and one that will need joined up thinking, as we need everyone included in this digital revolution.

You might be thinking right now that building a virtual assistant is your next startup idea, it's going to be how you make an impact on society, it's how you can change healthcare. Alas, it's not as easy as we first thought. Cast your mind back to 2014, the same year that the Echo first became available. I was one of early adopters who pledged $499 for the world's first social robot, Jibo [consider it a cuter or creepier version of the Echo with a few extra features] They raised almost $4 million from people like me, curious to explore this new era. Like the Echo, you are meant to be able to talk to Jibo from anywhere in the room, and it will act upon your command. The release got delayed and delayed, and then recently I got an email informing me that the folks behind Jibo have decided that they won't be shipping Jibo to backers outside of the USA, and I was offered a full refund.

One of the reasons that cited was, "we learned operating servers from the US creates performance latency issues; from a voice-recognition perspective, those servers in the US will create more issues with Jibo’s ability to understand accented English than we view as acceptable." How bizarre, my US spec Echo understands my London accent, and even my fake ones! It took the makers of Jibo 2 years to figure this out, and this too from people who are at the prestigious MIT Media Lab. So just how much effort does it take to make something like the Echo? A rather large amount, it seems. According to the Jeff Bezos, the CEO of Amazon, they have over 1,000 people working on this new ecosystem. A very useful read is the real story behind the Echo explaining in detail how it was invented. Apparently, the reason why the Echo was not launched outside of America until now, was to so it could handle all the different accents. So, if you really want to do a hardware startup, then one opportunity is to work on improving the digital microphones found not just in the Echo, but in the our smartphones too. Alternatively, Amazon even have an Alexa Fund, with $100m in funding for those companies looking to "fuel voice technology innovation." Amazon must really believe that this is the computing platform of the future.

Moving on this week's news, the UK Echo will have UK partners such as the Guardian, Telegraph & National Rail. I use the train frequently from my home into central London, the station is a 15 minute walk from my house, so that's one of the UK specific skills I'm most likely to use to check if the train is delayed or cancelled before I head out of the front door. Far easier and quicker than pulling out my phone and opening an app. The UK version will also have a British accent. If you have more than one Echo device at home, and speak a command, chances are that two or more of your devices will hear you and respond accordingly, which is not good, especially if you're placing an order with Amazon. So now they have updated the software with ESP (Echo Spatial Perception) and when you talk to your Echo device, only the closest one to you will respond. It's being rolled out to those who have existing Echo devices, so no need to upgrade. You might want to though, as there is a new version of the Echo Dot (US, UK & Germany), which is cheaper, thinner, lighter and promises better voice recognition than the original model. For those who want an Echo in every room, you can now buy Dots in 6 or 12 packs! In the UK, given that the Echo Dot is just £49.99, I expect this Christmas, many people will be receiving them as presents.

Amazon's Alexa Voice Service is one example of a conversational user interface, and at times it's like magic, and other times, it's infuriatingly clumsy. I'm mindful that my conversations with my Echo are nowhere near as sophisticated as conversations I have with humans. For example, if I say "Alexa, set a reminder to take my medication at 6pm" and it does that, and then I immediately say "Alexa, set a reminder to take my medication at 6.05pm", and so forth, it currently won't say, "Are you sure? You just set a medication reminder close to that time already." Some parents are concerned that the use of an Echo by their kids is training them to be rude, because they can throw requests at Alexa, even in an aggressive tone of voice, with no please, no thank you, and Alexa will always comply. Are these virtual assistants going to become our companions? Busy parents telling their kids to do their homework with Alexa, or lonely elders who find that Alexa becomes their new friend in helping them cope with social isolation? Will we end up with bathroom mirrors we can have conversations with about the state of our skin? Are we ever going to feel comfortable discussing the colour of our urine with the toilet in our bathroom? When you grab your medication box out of the cupboard, do you want to discuss the impact on your mood after a week of taking a new anti-depressant?

Could having conversations with our homes help us to manage our health? It seems like a concept from a science fiction movie, but to me, the potential is definitely there. The average consumer will have greater opportunities to connect their home to the internet in years to come. Brian Cooley, asks in this post if our home becomes the biggest health device of all.

A thought provoking read is a new report by Plextek examining the changes in the medical industry by 2020 from connected homes. I want you to pause for a moment when reading their vision, "The connected home will be a major enabler in helping the NHS to replace certain healthcare services, freeing up beds for just the most serious cases and easing the pressure on GP surgeries and A&E departments. It will empower patients with long-standing health conditions who spend their life in and out of hospitals undertaking tests, monitoring, rehabilitation or therapy, and give them freedom to care for themselves in a safe way."

Personally, I believe the biggest barrier to making this vision a reality is us, i.e people and organisations that don't normally work together will have to collaborate in order to make connected homes seamless, reliable and cost effective. Think of all the people, policies & processes involved in designing, installing, regulating, and maintaining a connected home that will attempt to replace some healthcare services. That's before we even think about who will be picking up the tab for these connected homes.

Do you believe the Echo is a very small step on the path towards replacing healthcare services, one conversation at a time?

[Disclosure: I have no commercial ties with the individuals or organisations mentioned above]

Managing our health: One conversation at a time

Are we building technology for those that need it most?

It's 2014, isn't everyone online?

Surely, all these issues are being discussed at conferences?

So what else can we do?