Cnet’s Smartwatch Step Count Accuracy Test Uses Flawed Methodology

When choosing a fitness tracker, or smartwatch, most people go for style of features other than the actual fitness tracking. Very few reviews seem to think that accuracy is important; I’ve found that many fitness trackers are inaccurate, and I was very pleased to see that Cnet did a test of fitness trackers and smartwatches, comparing them to the Apple Watch.

Unfortunately, the methodology used is flawed. The article says:

“With those caveats in mind, I developed a testing methodology to try and reduce variables as much as possible. I wore each activity tracker or smartwatch on my left wrist at a single time and walked on a treadmill for a mile (as measured by the treadmill’s built-in distance tracker). I then compared the mileage from the treadmill to the mileage recorded on the watch. This test was performed three times with each device I tested to ensure accuracy. The same treadmill was used for the test, and I walked at the same speed (3.5 mph, which came to about 17 minutes each time).”

Most fitness trackers do quite well on treadmills; it’s easy to count steps in a situation like that. It’s much harder to count steps in real-world situations.

When I reviewed the Fitbit Charge, I found that it was very accurate on a treadmill. Compared to the Fitbit One – probably the most accurate fitness tracker, because of the fact that you wear it on your belt – I found the two to be very close:

“Both devices recorded nearly the same number of steps on the treadmill (2,048 steps for the Charge, and 1,997 for the One)…”

But let’s look at the entire paragraph that quote is taken from:

“Here are some examples. One day, I tested the devices by wearing both of them. I went about my usual business, and I walked on my treadmill for 30 minutes. Both devices recorded nearly the same number of steps on the treadmill (2,048 steps for the Charge, and 1,997 for the One), but for the rest of the day, the numbers diverged greatly. Near the end of the day, when the Fitbit Charge was at 5,000 steps, the One was almost exactly at 4,000 steps. These convenient numbers make it very easy to calculate the discrepancy between the two devices. If you take away the 2,000 steps on the treadmill, where the devices nearly matched, the Charge recorded 3,000 steps, and the one 2,000. In other words, the Charge is recording 50% more steps than the One. (I’ve noticed that the Charge records some steps when I’m asleep; a half-dozen here, a dozen there, adding up, some nights, to 50 steps or so. And, no, I don’t sleepwalk.)”

So, comparing the two by only looking at a treadmill test, they look like they’re very close. But comparing the two throughout a full day’s activity, the Fitbit Charge recorded 50% more steps than the Fitbit One.

So it’s good that Cnet decided to test the accuracy of fitness trackers and smartwatches; it’s a shame that they didn’t come up with a valid test.

2 thoughts on “Cnet’s Smartwatch Step Count Accuracy Test Uses Flawed Methodology

  1. Kirk,

    Many thanks for your article about migration and clean install on a Mac.

    I had been having no success with migration assistant (bookmarks but no other settings, photos, iTunes library Calendar etc but most importantly no email files)

    I tried to stay calm while searching the internet for some instruction about this problem but became increasingly concerned as I waded through a sea of mis-information and plenty of simpletons giving advice about things they obviously knew little about.

    Then fortuitously I came across your article and read and then followed your instructions.

    By fine tuning the path for the hidden library that I needed from one of the other drives, I have successfully (so far) managed to retrieve the email files and the others previously mentioned.

    I don’t know what I would have done if I hadn’t come across your instructions.

    Thanks very much, it got me out of a big problem.

    It seems that migration assistant may be only designed to migrate settings from a single drive to a new computer with a single drive.

    My MacPro with multiple drives including an SSD died and the whole sorry saga commenced when I tried to use the drives in an external enclosure connected to an iMac and boot from the ext. SSD. This worked for a couple of days and then disk utility said that the drive could not be repaired and needed to be re-formatted.

    At that point migration assistant couldn’t handle migrating from time machine onto the re-formatted disk and only offered virtually nil content for the last 5 ‘saves’ (The time machine drive contained all the info’ saved correctly so it wasn’t the culprit.)

    Once again thanks, I’ll look forward to following your blog in the future

    regards

    Iain

  2. Kirk,

    Many thanks for your article about migration and clean install on a Mac.

    I had been having no success with migration assistant (bookmarks but no other settings, photos, iTunes library Calendar etc but most importantly no email files)

    I tried to stay calm while searching the internet for some instruction about this problem but became increasingly concerned as I waded through a sea of mis-information and plenty of simpletons giving advice about things they obviously knew little about.

    Then fortuitously I came across your article and read and then followed your instructions.

    By fine tuning the path for the hidden library that I needed from one of the other drives, I have successfully (so far) managed to retrieve the email files and the others previously mentioned.

    I don’t know what I would have done if I hadn’t come across your instructions.

    Thanks very much, it got me out of a big problem.

    It seems that migration assistant may be only designed to migrate settings from a single drive to a new computer with a single drive.

    My MacPro with multiple drives including an SSD died and the whole sorry saga commenced when I tried to use the drives in an external enclosure connected to an iMac and boot from the ext. SSD. This worked for a couple of days and then disk utility said that the drive could not be repaired and needed to be re-formatted.

    At that point migration assistant couldn’t handle migrating from time machine onto the re-formatted disk and only offered virtually nil content for the last 5 ‘saves’ (The time machine drive contained all the info’ saved correctly so it wasn’t the culprit.)

    Once again thanks, I’ll look forward to following your blog in the future

    regards

    Iain

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.