Once again I’ve been reading about the inconsistencies of Strava data – this time from an article claiming that users of the Android app were more likely to record faster times. This was based on the writer’s observation that many of the KOM’s on his local segments were recorded using the Strava android app, and some experimental data that he collected. One of the segments gave a 14 second difference between his Garmin (500) and the app. That’s a big margin of error, when the total segment time was only about 90 seconds.
Evidence is great, so here’s another little experiment.
My commute is about 19 miles each way and there are a number of segments in each direection. I recorded my journey to and from work on two separate days, using my Garmin Edge 800 as usual, and also using the Strava app on my android phone, a Samsung Galaxy S2.
Were there any differences between the two? Here’s the data.
[gview file=”https://www.scarletfire.co.uk/files/garmin android strava.xls”]
Summary of the findings
28 segment recordings were made.
The android app recorded the fastest time in 16 cases.
Garmin recorded the fastest time in 9 cases.
There were 3 segments where the times were identical.
In terms of the size of the discrepancies, 9 of the Android “wins” were by only 1 second, with 3 wins by 2 seconds, and 4 by 3 seconds.
3 seconds was the largest discrepancy observed.
The Garmin wins went as follows: 4 wins by 1 second. 3 by 2 seconds, and 2 by 3 seconds. Again, 3 seconds was the largest discrepancy observed.
There doesn’t appear to be any correlation between the segment length and the size of the discrepancy.
For most popular segments, one or two seconds could mean a big difference in the leaderboard. On one of the segments above, I ranked 2nd with the Android, but 7th with the Garmin.
Conclusion
This evidence could support the hypothesis that the Strava Android app is more likely to result in faster times than Garmin GPS units. Digital doping, if you will.
It might suggest that one piece of hardware is more likely to produce less consistent segment data, but further study would be needed to establish which. From this data, we have no way of knowing if both are inconsistent, or whether the differences are down to the inaccuracies of one device in particular. For example, the Garmin could be recording the true time, with the phone app recording +/- 3 seconds because of different rates of data sampling, for instance. It may also depend on the shape of the segment – long, straight roads may cause lower sampling rates and thus greater errors in segment matching.
The Strava knowledge base on “segment matching” states:
Recording intervals vary between devices – for example, the mobile app records every 3-4 seconds while Garmins usually use smart recording which has a varied recording interval. Segment matching works the same on each GPS dataset, but depending on the unique data and recording interval, can yield different results. Segment matching uses the GPS points in the data closest to the start and endpoints of the segment, and as this can vary with each activity, timing on a segment can vary slightly because of this. At the present time, we don’t interpolate or extrapolate GPS data to normalize for the exact start and end positions of the segment.
Suggestions for further study
It would be interesting to note how consistent the recording devices actually are. For example, if you could set up a controlled journey at a set speed, how accurately would repeated recordings be? Would a phone app be more variable in its output? Would the Garmin be more reliable, producing consistent results with no variation? Or vice versa?
Strava is clearly aware of the issues and working towards smarter algorithms to smooth out the discrepancies.
I guess one lesson from all this is to just chill out and enjoy riding your bike without obsessing over numbers.
You know, just occasionally….
Still interested? Further reading…
Ben at Veloviewer has written a great article explaining exactly why the times can vary so much. He’s also written some code within Veloviewer which creates an “alternative leaderboard” which attempts to compensate for the errors and so may be more accurate than Strava’s own algorithm!
Here’s some tweet responses to this article:
@stravatips that’ll be why I’m always so slow!
— Nick Fox (@foxitwit) March 5, 2013
@stravatips Will give it a try today, have an Edge500 and also the Android App, Will ride both GPS only and get back to you
— Hyde105 (@hyde105) March 5, 2013
@stravatips iPhone’s are a joke!I used an android, much better! Now I use a garmin, I’m starting to beat my android times 3mnths later!
— Marc smyth (@mcws_73) March 5, 2013
@stravatips ive had issues running them side by side and the android app said i was quicker
— OfficeBoy (@0fficeBoy) March 5, 2013
I find that I get quicker times if I throw my Android over the line from about 30 metres back; especially if the hill is steep. I think Strava is bad for the phone though ….. mine keeps breaking.
Why wouldn’t you just run the Garmin and Strava apps on the same ride at the same time? It seems that would be the best test, no?
That’s what I did, twice. If you look at the data you’ll see the garmin results and the android results for the same journey to and from work, on two separate days.
The iPhone app is also noticeably faster for me. I don’t believe it’s an android specific problem, more of a mobile app problem. I really just use Strava to log miles though, so it’s nbd to me.
I got curious about a because I seemed to be faster down a long hill on my mtb than on my tri bike. And, there is a .1m segment that I seem to get wildly varying times for. Finally, I once made a short cut through a forest according to strava. Its obviously estimating times based on intervals. And if the interval doesn’t start and stop with the segment it wont be right. The fix for this is to preload the segments you are interested in, and when the app or the garmin hits that exact stop or start it records the time in addition to its normal interval estimation.
The Aug. 13 poster makes a lot of sense. My girlfriend and I did a ride last night, she with an iphone and I with an Android. She’s fast, but I edged her out on the notable downhill segments. The first by 10-15 seconds, the second and third by about 5 seconds. I stopped and waited for her after each segment. In all cases the iphone recorded faster times.
I was riding with a group yesterday and lead the whole ride. I was using my Edge 500. A guy using an iPhone 4s who was struggling to keep and was dropped twice on a long, fast segment yet ended up with the KOM by one tenth of a second over those of us using Garmins. Shows my speed as .1 MPH slower even though I pulled him. Showed he had a top speed of 44 MPH on the segment, behind us, while we all showed 27-28 MPH on our uploads. Wouldn’t be bothersome but this was a KOM and more importantly proof that rankings separated by less than 2 sec. should all be suspect if a phone was used vs. the more accurate data points of a genuine GPS. http://toddsingleton.net/chronicle/?p=3123