Once again I’ve been reading about the inconsistencies of Strava data – this time from an article claiming that users of the Android app were more likely to record faster times. This was based on the writer’s observation that many of the KOM’s on his local segments were recorded using the Strava android app, and some experimental data that he collected. One of the segments gave a 14 second difference between his Garmin (500) and the app. That’s a big margin of error, when the total segment time was only about 90 seconds.
Evidence is great, so here’s another little experiment.
My commute is about 19 miles each way and there are a number of segments in each direection. I recorded my journey to and from work on two separate days, using my Garmin Edge 800 as usual, and also using the Strava app on my android phone, a Samsung Galaxy S2.
Were there any differences between the two? Here’s the data.
[gview file=”https://www.scarletfire.co.uk/files/garmin android strava.xls”]
Summary of the findings
28 segment recordings were made.
The android app recorded the fastest time in 16 cases.
Garmin recorded the fastest time in 9 cases.
There were 3 segments where the times were identical.
In terms of the size of the discrepancies, 9 of the Android “wins” were by only 1 second, with 3 wins by 2 seconds, and 4 by 3 seconds.
3 seconds was the largest discrepancy observed.
The Garmin wins went as follows: 4 wins by 1 second. 3 by 2 seconds, and 2 by 3 seconds. Again, 3 seconds was the largest discrepancy observed.
There doesn’t appear to be any correlation between the segment length and the size of the discrepancy.
For most popular segments, one or two seconds could mean a big difference in the leaderboard. On one of the segments above, I ranked 2nd with the Android, but 7th with the Garmin.
This evidence could support the hypothesis that the Strava Android app is more likely to result in faster times than Garmin GPS units. Digital doping, if you will.
It might suggest that one piece of hardware is more likely to produce less consistent segment data, but further study would be needed to establish which. From this data, we have no way of knowing if both are inconsistent, or whether the differences are down to the inaccuracies of one device in particular. For example, the Garmin could be recording the true time, with the phone app recording +/- 3 seconds because of different rates of data sampling, for instance. It may also depend on the shape of the segment – long, straight roads may cause lower sampling rates and thus greater errors in segment matching.
The Strava knowledge base on “segment matching” states:
Recording intervals vary between devices – for example, the mobile app records every 3-4 seconds while Garmins usually use smart recording which has a varied recording interval. Segment matching works the same on each GPS dataset, but depending on the unique data and recording interval, can yield different results. Segment matching uses the GPS points in the data closest to the start and endpoints of the segment, and as this can vary with each activity, timing on a segment can vary slightly because of this. At the present time, we don’t interpolate or extrapolate GPS data to normalize for the exact start and end positions of the segment.
Suggestions for further study
It would be interesting to note how consistent the recording devices actually are. For example, if you could set up a controlled journey at a set speed, how accurately would repeated recordings be? Would a phone app be more variable in its output? Would the Garmin be more reliable, producing consistent results with no variation? Or vice versa?
Strava is clearly aware of the issues and working towards smarter algorithms to smooth out the discrepancies.
I guess one lesson from all this is to just chill out and enjoy riding your bike without obsessing over numbers.
You know, just occasionally….
Still interested? Further reading…
Ben at Veloviewer has written a great article explaining exactly why the times can vary so much. He’s also written some code within Veloviewer which creates an “alternative leaderboard” which attempts to compensate for the errors and so may be more accurate than Strava’s own algorithm!
Here’s some tweet responses to this article:
@stravatips that’ll be why I’m always so slow!
— Nick Fox (@foxitwit) March 5, 2013
@stravatips Will give it a try today, have an Edge500 and also the Android App, Will ride both GPS only and get back to you
— Hyde105 (@hyde105) March 5, 2013
@stravatips iPhone’s are a joke!I used an android, much better! Now I use a garmin, I’m starting to beat my android times 3mnths later!
— Marc smyth (@mcws_73) March 5, 2013
@stravatips ive had issues running them side by side and the android app said i was quicker
— OfficeBoy (@0fficeBoy) March 5, 2013