I was considering trying a sleep tracker again last year and before that happened solved my sleep problems. The amount of things I tried back in my 20s when it all was trivially simple: I just need to get an adequate amount of strength training and cardio exercise during the day and I sleep straight through the night.
I was a hardcore flu.x/nightshift user. I left most lights off once the sun set. Now I'm a lot less convinced that lighting stuff matters for sleeping (but I would still follow it given the the whole cancer-light link thing.)
Of course, all of the common sense stuff still applies: staying up all night, sleeping erratic schedules, drinking a gallon of water before going to bed, drinking a bunch of caffeine late at night, medications, stimulants, cocaine, trying to sleep in a room that's 90F+, having sleep apnea, someone drilling a jackhammer outside your bedroom window etc. If you do a combination of those things there is probably zero you can do to sleep consistently and well.
I really wish I had understood this decades ago. Getting older it hits me how trivially simple solutions to certain problems are despite most running around in circles trying to solve them the entirety of their lives. To be fair, for sleeping poorly, I got an incredible amount of work done sitting in front of a PC 12 hours a day. That's a number you probably can't hit if you are getting adequate exercise, quality food, and consistent deep sleep.
> Getting older it hits me how trivially simple solutions to certain problems are despite most running around in circles trying to solve them the entirety of their lives.
Oh, but you can't (can?) sell a self help book that way. It would only be a few pages long and everyone in the media would shit all over the author if it ever got traction. Hordes of bots on Reddit would be having fake conversations in threads that cry "science!" in an attempt to protect book sales based on TED talks and trending morning show style topics.
Is there a maximum time-of-effect there? i.e, if you work out in the morning, does it have less of an impact on sleep quality than working out in the evening?
For walking, no - it doesn't matter when I do it. Though I would avoid going for more than a light stroll within an hour of bedtime. For resistance training or cycling I'd try to avoid finishing within two hours of bedtime - those two make it harder for me to fall asleep.
I have found that a cool or cold shower before bed really helps. If I am warmed up from exercise (or just sitting in a warm room) I find it much harder to go to sleep. But not too cold for too long - that can be stimulating; not what I want!
> There are limitations to consider in the interpretation of these results. First, devices were worn for one night.
That is odd.
What I noticed is that my Garmin watch may be wrong at times, but overall provides a fairly good and very useful assessment, especially comparatively day vs day and overall for trends.
In addition it takes a bunch of time for the watch to work out enough data to tune the algorithms for a particular person: some UI bits explicitly call out that for accurate results the watch should be worn continuously for ~2 weeks.
I've now been wearing this particular one (Fenix 7) for 6 months, a serious upgrade from a Forerunner 735xt continuously worn for 5 years. 95% of the time the watch is stupidly accurate in ways it has no business being as a simple wrist-worn set of sensors.
My Garmin (255s) is pretty inaccurate when it comes to sleep. It often mistakes REM for awake and vice versa. Honestly, nowadays I only wear it for my runs. Even if it was accurate, I can just tell by feel. All the metrics such as sleep, resting HR and HRV are pretty bad compared to your nervous system.
The metrics aren't really actionable either. I got a bad night of sleep, so what? I know it. I have low HRV because I drank alcohol/over-trained/am ill. I don’t need a watch to tell me that. I know it.
It’s just collecting data for the sake of collecting data.
I've changed my 250 to an Insight 2. It's an improvement and it helped me be more attentive with my sleep, exercise. I don't do runs but walks, hiking, cycling and several water sports during summer. It's also used as a HR sensor for my cycling Garmin device.
> It’s just collecting data for the sake of collecting data.
No, sleep is one of the most important parameters for health!
If in 2024 you are not collecting data on your sleep to take action (and for ex, stop overtraining, reduce drinking etc) to reduce the risks of future brain problems, you are doing it wrong.
> Even if it was accurate, I can just tell by feel
I can't, and I fear it may be the same "overconfidence" issue that cause car accidents when drunk.
Thanks to my garmin, I have identified a pattern where I feel ok, but my garmin reported less REM sleep than usual: in the day, I run more commands than usual but I have way more mistakes (as can be seen by the non-zero return values)
So now, when my watch reports low REM sleep, I try to be very careful to avoid negative-productivity days
Sleep isn't the metric to track here, there are much better things to analyze such as HRV. For reducing drinking you shouldn't be doing that because of your sleep, you should be doing it because of your overall health. If you care about future health, take blood tests, weigh yourself, evaluate your physical fitness.
If the software compares behavior between nights and can then say "that was relatively good", then you're right and the limitation matters
If the software behaves no differently from one night to the next, that is, if you can take any random night's assessment and it would be as likely to be accurate as the first, then it does not matter that they had participants only wear it one night. You'd get more data (from multiple nights) without needing to recruit more participants, but you'd also be biasing towards whether the tracker works well for this set of people (probably a worthwhile trade-off, up to a point)
Your watch says to use it for a while. It could be that that's just because it's not flawless and they want people to see it's often correct even if they got unlucky the first night(s). Or maybe your watch works differently from the ones used in this study
I'm also surprised they only did one night, both because you get a lot more data per amount of effort by having participants keep it for a month, and because I'd not be surprised if the software is more comparative than absolute. But I don't know if we can draw real conclusions from that
Unless they’re spewing out random garbage (which I doubt they are) they’re good for tracking trends. My overnight RHR is about 55. If I’m starting to get sick, even before symptoms set in, it raises to about 60. By looking at trends I can tell when I’m about to start feeling bad.
The YouTube channel, 'The Quantified Scientist', does a great job of comparing sleep tracking wearables to a reference EEG. He collects a lot of data and presents it very well.
Thank you for posting this! I recently started using my Apple Watch for sleep tracking and have been dismayed at my quality and quantity (vs. "bed time") of sleep. Searching the channel for "sleep" I instantly found 3 videos for my "to watch" list.
> The Fitbit Inspire, Fitbit Versa, and Oura Ring estimated light sleep with less bias than deep sleep. Conversely, the Withings Mat and Garmin Vivosmart estimated deep sleep with less bias than light sleep. Across devices, estimates for REM sleep were less biased than estimates of light and deep sleep. Despite low absolute biases, it should be noted that some estimates were still up to 250 min different than their PSG derived measures.
In other words: these devices are all terrible, and shouldn't be trusted. Pretty different from the final conclusion, which is probably "soft" to get additional grants from the commercial entities selling these devices.
You skipped the section where the measurement of total sleep for most devices was found to be more than adequate.
Detecting light versus deep sleep accurately with wristwatches or devices under the mattresses might not be possible. Especially not when accuracy is supposed to be comparable to multiple electrodes places directly on the head to measure brainwaves.
All they have to do is measure time in bed, which for most people most of the time will correlate closely with total sleep. Insomniacs are outliers so you don’t need to worry about measuring them and most are taught to just get up when they awaken anyway.
I mean, given the study they shouldn't be trusted as a single night measure of absolute sleep quality however I don't know of anyone who uses them as such. You use them over long periods of time to either get aggregate statistics to compare against baseline or track personal changes over time.
edit: Also, the last time I did a sleep study at home it had rather little correlation to how I normally sleep except in terms of apnea. Having wires, electrodes, a chest strap and a finger O2 monitor strapped to me did not lead to a pleasant night of sleep.
These plots definitely align with my personal anecdotal experience.
I own both a Fitbit Vivo Active 4S and a Garmin VivoSmart 4.
In the daytime, as a watch, I much prefer the Garmin, but at night, as a sleep tracker, which is the main thing I want from these watches, I only trust the data from the Fitbit.
If I'm concerned about actual total sleep times, I do generally need to edit the Fitbit data in the morning. Over the years, my Fitbit watches have fairly consistently confused idle time sitting on a couch watching videos or lying in bed reading on my phone before or after sleeping as sleep. The data edit tools in the app fortunately make it easy to correct this (I'm not someone who falls asleep with the video playing, so I generally know when I turned off the activity at night and when I woke for the morning).
The data during the night, which is what I care about, I've come to trust on Fitbit watches over the years. I can't know if it's precisely accurate, but it's definitely directionally correct (if there's an overall scale factor error or similar from true sleep quality and time numbers I don't really care as the measurements are consistent enough that I can use them and benefit from them).
My Garmin I view as a random number generator as far as sleep tracking. My perception is it's probably a bit better about not misclassifying idle time before and after sleep, but not accurate enough to be actually correct, and I have not been able to find any simple data editing feature in the app to fix the misclassifications, so the wrong data stays wrong.
The Garmin data during the night I have zero confidence in, as anecdotally it feels far less "believable" than what I see reported by my Fitbit.
Edit added: another commenter just mentioned that Garmin uses a learning algorithm that needs about two weeks of data to hit good data quality. I've done a couple periods of sleep tracking with my Garmin, but I'm guessing I've abandoned it due to poor data quality each time at around two weeks. Sounds like I should give it another try and force myself to wait a month or so before making any assessments.
I've been sleep tracking with my Garmin Fenix 7 for about 18 months, and to me it seems bang on. The HRV stat is particularly good - a decline over about a week predicted the onset of flu symptoms a couple months ago, for example. Body battery is also a really useful feedback measurement. Sleep onset and wake times are right; REM seems right; and deep/shallow measurement at least correlates with how I feel.
I don't have experience with Fitbit, but I have been using Garmin Fenix 7x for the past two years and I can confirm that at least in my case, its measurement of my sleep quality is only loosely connected to reality.
As I am an avid runner and I train daily, I pay close attention to all sleep, heart rate, stress, and running performance numbers.
Unfortunately, it will frequently say I had good night of sleep when I definitely feel groggy or the opposite.
As to misclassifying idle time before and after sleep, I recognise that simple lifestyle adjustment could fix it (just don't browse on your phone before sleep!) But that of course won't fix it if you try to lie still but can't fall asleep.
My anecdata is different: sleep is mostly accurate, if sometimes a bit fuzzy, and occasionally misses the mark.
Regarding grogginess or lack thereof, what I noticed is that sometimes I feel that way even though the watch says I had good rest, but it's only so at the beginning of the day; afterwards I'm okay.
Conversely I may have had a bad night and still feel okay, but what I noticed is that if I don't pay attention to the watch I'm actually plowing through and don't notice I'm wearing myself out, which I then have to repay the following days.
Also the watch gives a) objective measurements (whereas my feeling is subjective) and b) measures only a bunch of things but not everything (e.g I noticed eating pasta or something weighting on my mind and morale can both make me experience brain fog that the watch does not account for)
Drinking one or two cups of green tea helps with my grogginess. Or some light cardio exercise like 3 minutes of rope jumping if you're more inclined to do that rather than stimulents. I also drink a shot of espresso sometimes, but the tea does a much better job without risking a headache. It's Chinese green tea with low caffeine content, Chun Mee or other blends from Yunnan or Anhui. I like their method of processing more than what is used in Taiwan or Japan.
The inability of wrist trackers to identify deep vs light sleep might cause frustration to some, but their automated sleep journaling is actually more than enough to help change the lives of people with insomnia.
For some reason we had to slap complicated sleep scores and sleep classification on these things to take them seriously when all you needed was simple data gathered consistently over time.
I used to use this [1] software about 7 years ago when I first got my CPAP due to a lack of attention from my sleep medicine team. I could dive deep into my downloaded data and set my precise measurements perfectly.
Haven’t used it in years.
Do you have any recommendations for how to find a dependable overnight pulse oximiter on alibaba? I’ve never shopped there.
Apple Watch was not tested, so for anyone interested, here are two videos by "The Quantified Scientist" creator Rob ter Horst, a postdoctoral researcher who specializes in bioinformatics (studying biology using machine learning and statistics).
- WatchOS 9 beta introduces sleep stage tracking to the Apple Watch using a machine learning algorithm trained on polysomnography and Apple Watch heart rate and movement data.
- The author tested the Apple Watch's sleep tracking against an EEG headband over 18 nights and found it agreed very well, especially for deep sleep tracking.
- The Apple Watch detected 89% of deep sleep, 84% of light sleep, and 67% of REM sleep correctly compared to the EEG device.
- Overall sleep stage tracking accuracy was better than the 37 other devices the author has tested.
- The Apple Watch also detected sleep cycles and awake periods reasonably well compared to the EEG.
- Falling asleep and waking up times mostly differed by less than 10 minutes from the EEG.
- Future improvements could include sleep coaching based on metrics.
- Results may vary for other populations not well represented in the algorithm's training data.
- The author used a beta version and polysomnography would provide better validation.
- The Apple Watch's performance makes it a current leader for sleep and heart rate tracking accuracy.
I really, really don't need a tracker and I doubt most people do. Even to satisfy curiosity they seem woefully lacking.
What I do feel I need is an effective way (including cost-wise) to mitigate thermoregulation issues at night, i.e. to avoid waking up hot or cold, dialing in the correct bedding temperature at any moment.
I thought a wool comforter might be the answer given its reputation, but I still fuss around with adding or removing layers. I basically sleep with a stack that changes depending on the weather.
I got a Fitbit in my mid twenties, noticed my heart rate going to 40. Had no idea of what that meant and asking around it turned out “it could be sleep apnea.” Long story short, it was. I no longer fall asleep driving and I assume my heart is healthier than had I not known for many more years when I finally got married and had a sleeping partner (if I hadn’t died on a highway first).
My point is, I think these types of devices are wonderful screening tools. Cheap and imagine all the prevention we could have by screening widely and early.
It's trivial to figure out that apnea is a possible culprit by virtue of the symptoms. If you can buy a fitbit, you can do the bare minimum research to explore the possibility.
Yeah, the former looks like a grift targeting the wealthy, the latter more promising. I'm determined to make things work without that but I am giving ChiliPad serious consideration.
I'm curious to find out why you think Eight sounds like a grift.
We work in the sleeptech/neurotech space, and there is so much crap in this space. We're want to ensure we're taken seriously, so I'd be keen to hear your thoughts on why you think Eight Sleep isn't presented as trustworthy (if that is what you're saying).
I think they actually have the best reputation, and have taken over from Chili.
We're not in the heating/cooling space, so non-competitive.
If it was $2 a month then maybe and I'm a guy that loathes subscription costs.
I'm looking into this sort of thing as i'm hopeful it'll help my wife with thyroid issues get better sleep. but eightsleep is right off the table with those subs prices
After I was using an Oura ring for about a week, I came to the conclusion that knowing the data of how I slept provided very little value.
I personally believe that the consumer grade devices are "good enough" because the micro-data is not that valuable anyway, and the way PSG works, there is no "correct" score. Get 5 different polysomnography's, you'll get 5 different measures of the same sleep.
Once you're doing the sleep hygiene stuff, and experimented with all the magnesium stacks, ashwaganda, CBD, etc, etc, what do you do to actually ensure you are doing the most for your sleep?
This is why I got into the sleeptech/neurotech space, and for the last few years we've been building a headband that monitors brain activity and increases the efficiency of deep sleep. The tech is backed by peer-reviewed and published research from discoveries which began in 2013. https://soundmind.co/research
Yes, people like data and want to "understand" our sleep, but the quality of sleep is more than just the number of minutes you spend in each stage of sleep, just like fitness is more than the number of minutes you spend in each cardiovascular training stage.
Our understanding of sleep today is at the same place our understanding of fitness was in the 70s, and sleep is just beginning to get is due importance. What we'll learn in the next 20 years, I believe, will lead to significant changes in health and longevity.
Semi-related question. I recently took an at-home sleep apnea test using the WatchPAT ONE (https://www.itamar-medical.com/professionals/disposable-hsaw...). It includes three main pieces you can see in the image there: a wristwatch-like sensor, and finger sensor, and a sensor you attach just above your sternum (primarily to track breathing events for sleep apnea). There is an associated phone app.
Of course, thanks to capitalism, the device is marketed as disposable (I love how this is somehow portrayed as a bonus in the marketing materials). You're literally supposed to throw it away after a single use.
So questions:
1. I've searched online but haven't found anything about people successfully hacking the device so it can be used multiple times. Has anyone seen any succesful hacks of this? If not, can anyone point me to some resources where I would even start to try to hack it? It communicates with the app over bluetooth.
2. Does anyone know of other medical-grade sensors that are usable by consumers that are not disposable?
I ask this because in the past I've just used the "Sleep as Android" app with a smartwatch and it was horribly wrong. It just checks for movement, and there are lots of times when I an exhausted, staring up at my ceiling not getting any sleep, but I'm not moving because I'm stuck in this exhaustion-but-not-sleeping state.
Sometimes, I just try to fall asleep, not moving, and the tracker shows me as sleeping. What's worse than no tracking is grossly imprecise tracking as a result of garbage in, garbage out, and making conclusions on a shaky foundation!
This is cool, but it'd be nice if they had evaluated Apple Watch, which is far more common than the devices they measured for. I've been tracking sleep for years with mine. My sense is that it's pretty accurate, but the problem is, I'm asleep during the time in which I could attempt to verify this! ;)
I was a hardcore flu.x/nightshift user. I left most lights off once the sun set. Now I'm a lot less convinced that lighting stuff matters for sleeping (but I would still follow it given the the whole cancer-light link thing.)
Of course, all of the common sense stuff still applies: staying up all night, sleeping erratic schedules, drinking a gallon of water before going to bed, drinking a bunch of caffeine late at night, medications, stimulants, cocaine, trying to sleep in a room that's 90F+, having sleep apnea, someone drilling a jackhammer outside your bedroom window etc. If you do a combination of those things there is probably zero you can do to sleep consistently and well.
I really wish I had understood this decades ago. Getting older it hits me how trivially simple solutions to certain problems are despite most running around in circles trying to solve them the entirety of their lives. To be fair, for sleeping poorly, I got an incredible amount of work done sitting in front of a PC 12 hours a day. That's a number you probably can't hit if you are getting adequate exercise, quality food, and consistent deep sleep.