Should You Take Sleep Advice from ChatGPT?

People have been using Google and the internet to search for sleep advice for a long time. It’s not an unfamiliar picture; a stressed out and tired individual reaching for their mobile phone in the early hours of the morning to find out whether they should be counting sheep or spraying lavender mist on their pillows.

In the last year, Google Searches for ‘insomnia’ are up 5% averaging 47,000 monthly searches in the UK. As more and more people are continuing to look for information and support for poor sleep, it's no wonder the growth in artificial intelligence has users turning to ChatGPT.

What our Sleep Expert says about sleep tips from ChatGPT

Whilst AI and ChatGPT is easily accessible, the lack of regulations on its content and the lack of expert involvement means there are drawbacks to seeking advice on matters as important as sleep from such technology.

ChatGPT Relies on Unreliable Sources

Our resident Sleep Expert, Dr Sophie Bostock, commented the following on sleep advice from ChatGPT, “Studies suggest that ChatGPT provides 'largely accurate' health-related information. This is likely to be better than lifestyle advice from blog posts, which might be an alternative. One study from last year actually found that ChatGPT was judged as responding with greater empathy than human physicians.

ChatGPT responses are drawn from data from many different sources, which include medical websites but also 'wellness publications' which may not be referenced. It's a major limitation of the advice that ChatGPT won't tell us what the source was, medical guidelines or government or NHS sources would be more credible, for example, than an opinion piece.”

ChatGPT information may be out of date

What’s more, ChatGPT itself provides a disclaimer that its latest information is sourced from January 2022, no sooner. Dr Bostock comments, “This means that some advice is either simplified or may not be up to date. For example when prompted, “How long before bed should I exercise?” ChatGPT advises us to avoid exercising a few hours before bedtime. A systematic review in 2019 found that exercise a few hours before bed is OK, but to avoid vigorous exercise within one hour of bedtime. We are warned that screen time before bed may interfere with sleep because 'the blue light emitted by screens can interfere with the production of melatonin'. This is true, but most studies suggest that a bigger effect of screens on sleep is actually that they delay bedtime.

If you ask a follow-up question to ChatGPT you can generate some more specific advice. However, ChatGPT only draws from data up to Jan 2022 so more recent papers have not been included. For example, ChatGPT suggests avoiding drinking coffee at least 6 hours before bed, whereas a recent study recommended avoiding caffeine at 8.8hr before bedtime.”

Is ChatGPT health advice accurate?

According to Dr Bostock, “There are only minor inaccuracies and overall the content of the lifestyle or sleep hygiene advice is sound. There is relatively little mention of the psychological aspects of insomnia (worrying specifically about not sleeping well), but there are recommendations to manage stress, worry and anxiety, and to seek help from a mental health professional for persistent stress. Positively, ChatGPT continually reminds you to consult a healthcare professional if problems persist.”

So, while ChatGPT may not have all the answers, it can be a good starting point to direct you to the experts who can give you more personalised advice.

How to improve the accuracy of ChatGPT advice

Sharing sensitive and personal information with ChatGPT may seem like a way to get better advice, but you should do so with caution. Just like most online safety education, it would be best to refrain from sharing personal information with AI.

Dr Sophie comments on sharing sensitive information with AI, “You may be able to improve the accuracy of ChatGPT advice by telling it more about you and your circumstances. Of course, the advice will be generic unless you can provide personalised information. To avoid sharing sensitive information with the programme, you could ask for advice for a friend. For example, ‘My friend is a 70 year old woman. How much sleep does she need?’ This generates a sensible answer, which includes that there is a lot of natural variation in how much sleep we need based on different circumstances.

Where ChatGPT really gets it wrong with sleep advice

When comparing ChatGPT sleep advice responses with her own expert knowledge, Dr Sophie revealed the softwares lack of knowledge on the UK health system was the most concerning. “Of the responses we have looked at, I think the only inaccuracy I would really be concerned about is around melatonin. Even if you state that you live in the UK and that you are 25 years old, ChatGPT does not mention that in the UK, melatonin is only available on prescription. NICE guidelines recommend that melatonin is prescribed to those aged 55 and above. This is really because the evidence for melatonin for the treatment of insomnia in younger people is very weak.

ChatGPT doesn't mention that melatonin is only available without prescription in some countries, but it does give the following sound advice "Before starting any new sleep aid or supplement regimen, your friend must consult with a healthcare provider, such as a doctor or a pharmacist."

Overall, ChatGPT can be a helpful source of sleep information. It is limited by the absence of explicit sources or references, and the fact that the data may be out of date. It cannot deliver advice which is tailored to your individual experiences, lifestyle, health, medical history and circumstances - for that you need to speak to a qualified healthcare professional.