I’ve been reading about stories of GPS inaccuracies on Strava, and this post by Suffolk cyclist got me thinking. How could two riders, ascending a hill together end up recording quite different times?
I decided that some more experimentation was required. Evidence is a wonderful thing, after all, except if you’re a creationist.
Hypothesis:
There may be variation in the way different GPS units record tracks,which results in spurious timings when the data is uploaded to sites such as Strava, which allows the comparison of specific segments of a route. This would essentially undermine the whole principle of Strava leaderboards and make the coveted KOMs rather meaningless.
Aims:
To determine if there is consistency between different GPS tracking devices.
To determine the reliability of segment timings as calculated by Strava.
Apparatus:
One Sunday club run with the fabulous Rhyl Cycling Club.
4 GPS equipped riders, as follows:
Paul, iPhone 4S, strava app. | |
Tim, Garmin 500 | |
Craig, Strava app on HTC Sensation |
|
Me, Garmin edge 800, plus (for this test only) android app MyTracks running on Samsung Galaxy S2 |
A nice hill called Glascoed road, which is exactly 2 miles long, and gains 405ft, average gradient 3.7%. The strava segment is called Craig’s Favourite.
Yes, named after the Craig in our group, as he doesn’t like that particular hill very much.
Method:
For the test hill, the gps riders stayed together in a tight group, so that their segment times should all be within a few seconds of each other.
One rider (myself) ran two GPS logs – one from a Garmin 800, and one from an android phone app.
The data was uploaded to Strava and compared.
Results:
iPhone 4S (running Strava app)
Strangely, although the ride did register many of the segments on the route, this particular segment did not register. On closer inspection, the actual track is all over the place, and I thought the issue was so significant that it deserved its own post.
HTC Sensation (running Strava app)
This phone crashed at some point in the day, and the data was lost.
Galaxy S2 (running Google App MyTracks)
This phone/app combo produced a very accurate track which matched the Garmin Edge 800 exactly.
Garmin
The two Garmin devices in the test both performed impeccably, creating accurate tracks.
Here is the data we did manage to record and review.
GPS device | Time | Rider |
Garmin 500 | 13:06 | Tim |
Garmin 800 | 13:09 | Alan |
MyTracks app on Galaxy S2 | 13:09 | Alan |
Strava app on HTC Sensation | n/a | Craig |
Strava app on iPhone | n/a | Paul |
Paul contacted Strava about the missing segment, and somehow they tweaked it so that it did register, 36 hours later. The time it gave was 13:06, the same as Tim and his Garmin 500. Paul and I did the exact same ride as we live on the same street. Over 58 miles, it was a surprise to discover that the elevation data were very close – only 31 metres difference, in over 1200m of climbing.
I’ve left the iPhone timing data out of the table because it shouldn’t be necessary to have to email Strava to put your data right. The quality of the iPhone track in this experiment was very poor. See the screenshots.
The Garmin 800 and MyTracks andoid app which were on the same rider came out with identical scores.
The Garmin 500 recorded 3 seconds faster, as did the adjusted iPhone data, but there may have been genuine slight variation as it is impossible to know exactly where the segment begins and ends. A more scientific approach would be for one rider to carry all the GPS units. But this was a sunday club ride, not a laboratory controlled experiment, so take it with a pinch of salt.
Conclusion
We didn’t witness any wild variation in timings, as others have reported. It would appear that there is consistency between the Garmin 500 and 800, and it was very reassuring to see the Garmin 800 and the MyTracks app (both carried by the same rider) produce exactly the same times for the segment.
It would be fair to say, from the evidence gathered, that the quality of the iPhone track was inconsistent. It was so poor at times that the recorded track departed significantly from the actual road, which caused the segment to not be recognised by Strava. In contrast, the Garmin tracks from the 500 and the 800 were consistently following the road with no deviation at any time.
I wonder how many iPhone users have stolen KOMs from people simply because their hardware does not track adequately?
Good bit of research. A couple more things that would be worth investigating further:
1. quality of tracking in amongst trees (on the road). That segments looks like it is right out in the open making it fairly easier for the gps units.
2. Also try it in amongst some steep sided dales where satellites might get hidden by the hills.
Those are where I see the most discrepancies.
RaceShape gives a good comparison tool for the timings. See this segment for us Lowe brothers (4th & 5th at time of writing): http://raceshape.com/strava-segments/668080 We were riding next to each other the whole way up, me recording on my 305 and Tones on his Android HTC. Lots of fluctuations although only to a couple of seconds, but even so, that could make all the difference. But if you compare the elevation graphs in strava (Tony’s: http://app.strava.com/rides/10933852#192020139) then you can see the android’s recording is really jittery compared to the Garmin. I wonder if the iPhone’s elevation is worse again?
How about comparing the elevation graphs of your test recordings as well?
One other thing, what do you reckon is the minimum length of a segment for the timings to be reliable (given accurate gps recordings)? The gpx files can have a number of seconds between recorded points, particularly on straight roads, so when Strava adds you to a Segment it seems to use the closest point to the start end, even if that might be a few seconds out.
A good example is this sprint:
KOM: http://app.strava.com/rides/6344524#120283818 – this rider’s effort for the segment (blue line on map) goes from the junction to just over halfway to the A1 bridge. But look at this athlete’s effort http://app.strava.com/rides/11867463#204692309 and he’s gone almost twice the distance! (note: both using Garmin 705’s). More of an art than a science!
Seems like I am being childish given that its free. But I did have a KOM for a sprint. Suddenly someone I know rode 2 mph and 5 seconds faster. Got the same result as above. “His” segment ( which I created ) was shorter than mine by about a third.
Then today I rode with a friend with an app, I have a garmin 500. I toasted him by 100 m over the 3 / 10s course, and he ended up with the same time.
Its not just a GPS thing. I think strava is tweaking to find some similarity and has created some issues.
As an aside, my KOM has disappeared entirely, and a slower day has been ranked. I can’t even locate the other.
And of course there is an issue with the smoothing the watts. I get the explanation, but I do have a watt hub, so I know what my watts are. The new KOM averaged 36.1 mph on a 2.7% grade downhill, (soft) yet pulled only 65 w.
Weird stuff which takes away from the enjoyment of the program. Hopefully its being addressed. I might even pay for it if its wasn’t so bogus.
Malco, after commenting above I put together this page (http://veloviewer.com/NewLeaderboard.php) on my site that, in a fairly lazy way, compares the actual distance travelled and from that the average speed for each athlete for a segment and provides an alternative leaderboard based on that result. Doesn’t work on all segments due to technical reasons but I’d be interested to know the segment you refer to shows a significantly different result to what Strava shows. From what I hear Strava are reworking their segment matching algorithm to make the start/end points for a segment the same for everyone (rather than the closest recorded gps point) although there’s a good chance they won’t apply this to historical data.
You failed to mention what iPhone you were testing. I’ve used an iPhone 4S alongside and Garmin 800 and a Garmin 910XT and found little difference. The iPhone 3gs I used to use was poor, however.
It was a 4S.
I’ll amend the article, thanks.
With iPhone and Galaxy you can have some troubles with battery trying to ride long distances and the they don’t have barometric altimeter, just a reference from maps. Garmin is more acuratelly and fits better for riding, running and swimming.
I would be Intrested to hear if anyone has tested the newer iPhone 5S as this has significatly more processing power for GPS data. I have the Edge 800 & a 5S and although the times are almost identical the climbing data is different so begs the question, which is more accrate?
I found this quite interesting to read and have some insight into the way GPS receivers actually work and why any short route data for strava etc is only a guide and not very good at high accuracy stuff.
The problem is not down to device such as iPhone or galaxy s5 etc…but down to the type of GPS receiver which on a phone is at best 10m accurate at absolute best, probably more like 20 – 30m, which is fine for calculating long routes but when you consider a segment of 500m with the in built receiver of up to 30m error assuming max error at both ends the length of the segment could range from 440m-560m meaning just on device error it has a +/- error of 12% roughly.
The error in verticle is even worse as it is only measured from a long way above it is very hard to get correct even with a sub mm GPS kit so god knows how bad it is on a phone.
There are ways to make your devices work better such as try and find out where the antenna is and point that to the sky and position it where it can see open sky as much as possible. Basically the more sky you see the better, this does mean that the accuracy is very poor in wooded areas.
Hope that helps, and if you want a truly accurate GPS leica do sell them for about £20000 but they are big bits of kit and not to practical.
Good article. I use a Windows Phone on O2 which I track with Endomondo, then export the tracks as GPX to upload to strava. The data matches other rider’s garmin results quite closely. Oddly enough though – since changing phone providers recently (to giffgaff 3g) I find that the tracking is missing several miles, substituting them with straight lines.
Yes Its all looking a bit inaccurate. I was appauled when I compared my average speed on strava vs my average speed on my bike computer after a 4.5 hour ride. 1kmph out. So obvisouly when I was stopped and having some food at the cafe the strava app was none the wiser. Strava great idea, strava developer’s no idea.
The Strava app can be set in the “Settings” to stop recording after 10 seconds of non-movement.
Updated answer…. I use an Samsung S6 and ride with my brother who uses an iPhone S5 and we go on long rides on various occasions and our strava times are always differnt it says the iPhone has done an extra 1.9miles than my android and that the iPhone has climbed alot greater too. Also when it comes to segments me and my brother are usually about the same in speed and get when our strava results have loaded on some segments it says the iPhone has a lot greater time and on others it says the android has better times but most of the time it does favour the iPhone. anybody know why this is like this and any solutions for it? I’m tempted to buy a garmin but not sure for the money it’s gonna make a great deal of difference…..Thanks
iPhone has the superior tech
It’s more accurate !
And to even the playing field. Some new garmens have got the capability of using Russian saterlites.
Most devices use only western saterlites !
More saterlites = great accuracy !
Most new android phones and all iPhones 5 onwards have this capability. It’s called GLONNASS
Manufacturers of GPS devices say that adding GLONASS made more satellites available to them, meaning positions can be fixed more quickly and accurately, especially in built-up areas where the view to some GPS satellites is obscured by buildings. Smartphones generally tend to use the same chipsets and since the versions used since 2015 receive GLONASS signals, smartphones using such chips receive GLONASS positioning information along with GPS. Since 2012, GLONASS was the second most used positioning system in mobile phones after GPS. The system has the advantage that smartphone users receive a more accurate reception of up to 2 meters.
I wondered if all the devices recorded the same distance for the segment and if you still have the distance data recorded by each device. My Garmin 510 and my iPhone 5S running Strava are always different with the Garmin recording a shorter ride every time. The difference between the two is currently running at about 12KM over approx. 1080Kms (the Strava app on the iPhone says its 1081.5 km whereas the Garmin is 1069km, but which is more accurate?
all iphone strava times are gps doping for sure. all iphone/android times should become null and void.
From what I see on many sectors I create, is that garmin 500’s and some 800’s are usually the most inaccurate for recording anything less than 1k. Garmin seems to be a bit more unreliable compared to the android app. iPhone is a close second. It’s strava who is at fault here. In general, even though Strava is the most popular app, it is not the most accurate or reliable app. Ride with GPS is much more accurate and reliable, in terms of elevation, time/distance and segment/sectors. But alas… it’s Strava that gets all the attention! #gofigure
Strava is free so they deserve the attention. My Garmin just died again and I will be suing my Note 3 exclusively because it is free! Garmin wants too much money for their simple technology.