Smartphones Vs Garmin: Which GPS is best for Strava?

 

I’ve been reading about stories of GPS inaccuracies on Strava, and this post by Suffolk cyclist got me thinking. How could two riders, ascending a hill together end up recording quite different times?

I decided that some more experimentation was required. Evidence is a wonderful thing, after all, except if you’re a creationist.

Hypothesis:

There may be variation in the way different GPS units record tracks,which results in spurious timings when the data is uploaded to sites such as Strava, which allows the comparison of specific segments of a route. This would essentially undermine the whole principle of Strava leaderboards and make the coveted KOMs rather meaningless.

Aims:

To determine if there is consistency between different GPS tracking devices.
To determine the reliability of segment timings as calculated by Strava.

Apparatus:

One Sunday club run with the fabulous Rhyl Cycling Club.
4 GPS equipped riders, as follows:

Paul, iPhone 4S, strava app.
 Tim, Garmin 500  
 Craig, Strava app on
HTC Sensation
Me, Garmin edge 800, plus (for this test only) android app MyTracks running on
Samsung Galaxy S2
 

A nice hill called Glascoed road, which is exactly 2 miles long, and gains 405ft, average gradient 3.7%. The strava segment is called Craig’s Favourite.
Yes, named after the Craig in our group, as he doesn’t like that particular hill very much.

Method:

For the test hill, the gps riders stayed together in a tight group, so that their segment times should all be within a few seconds of each other.
One rider (myself) ran two GPS logs – one from a Garmin 800, and one from an android phone app.

The data was uploaded to Strava and compared.

Results:

iPhone 4S (running Strava app)

Strangely, although the ride did register many of the segments on the route, this particular segment did not register. On closer inspection,  the actual track is all over the place, and I thought the issue was so significant that it deserved its own post.

HTC Sensation (running Strava app)
This phone crashed at some point in the day, and the data was lost.

Galaxy S2 (running Google App MyTracks)
This phone/app combo produced a very accurate track which matched the Garmin Edge 800 exactly.

Garmin
The two Garmin devices in the test both performed impeccably, creating accurate tracks.

Here is the data we did manage to record and review.

GPS device Time Rider
Garmin 500 13:06  Tim
Garmin 800 13:09  Alan
MyTracks app on Galaxy S2 13:09  Alan
Strava app on HTC Sensation  n/a Craig
Strava app on iPhone  n/a Paul

Paul contacted Strava about the missing segment, and somehow they tweaked it so that it did register, 36 hours later. The time it gave was 13:06, the same as Tim and his Garmin 500. Paul and I did the exact same ride as we live on the same street. Over 58 miles, it was a surprise to discover that the elevation data were very close – only 31 metres difference, in over 1200m of climbing.
I’ve left the iPhone timing data out of the table because it shouldn’t be necessary to have to email Strava to put your data right. The quality of the iPhone track in this experiment was very poor. See the screenshots.

The Garmin 800 and MyTracks andoid app which were on the same rider came out with identical scores.
The Garmin 500 recorded 3 seconds faster, as did the adjusted iPhone data, but there may have been genuine slight variation  as it is impossible to know exactly where the segment begins and ends. A more scientific approach would be for one rider to carry all the GPS units. But this was a sunday club ride, not a laboratory controlled experiment, so take it with a pinch of salt.

Conclusion

We didn’t witness any wild variation in timings, as others have reported. It would appear that there is consistency between the Garmin 500 and 800, and it was very reassuring to see the Garmin 800 and the MyTracks app (both carried by the same rider) produce exactly the same times for the segment.

It would be fair to say, from the evidence gathered, that the quality of the iPhone track was inconsistent. It was so poor at times that the recorded track departed significantly from the actual road, which caused the segment to not be recognised by Strava. In contrast, the Garmin tracks from the 500 and the 800 were consistently following the road with no deviation at any time.

I wonder how many iPhone users have stolen KOMs from people simply because their hardware does not track adequately?

 

Comments

  1. says

    Good bit of research. A couple more things that would be worth investigating further:

    1. quality of tracking in amongst trees (on the road). That segments looks like it is right out in the open making it fairly easier for the gps units.
    2. Also try it in amongst some steep sided dales where satellites might get hidden by the hills.

    Those are where I see the most discrepancies.

    RaceShape gives a good comparison tool for the timings. See this segment for us Lowe brothers (4th & 5th at time of writing): http://raceshape.com/strava-segments/668080 We were riding next to each other the whole way up, me recording on my 305 and Tones on his Android HTC. Lots of fluctuations although only to a couple of seconds, but even so, that could make all the difference. But if you compare the elevation graphs in strava (Tony’s: http://app.strava.com/rides/10933852#192020139) then you can see the android’s recording is really jittery compared to the Garmin. I wonder if the iPhone’s elevation is worse again?

    How about comparing the elevation graphs of your test recordings as well?

    One other thing, what do you reckon is the minimum length of a segment for the timings to be reliable (given accurate gps recordings)? The gpx files can have a number of seconds between recorded points, particularly on straight roads, so when Strava adds you to a Segment it seems to use the closest point to the start end, even if that might be a few seconds out.

    A good example is this sprint:
    KOM: http://app.strava.com/rides/6344524#120283818 – this rider’s effort for the segment (blue line on map) goes from the junction to just over halfway to the A1 bridge. But look at this athlete’s effort http://app.strava.com/rides/11867463#204692309 and he’s gone almost twice the distance! (note: both using Garmin 705′s). More of an art than a science!

  2. Malco says

    Seems like I am being childish given that its free. But I did have a KOM for a sprint. Suddenly someone I know rode 2 mph and 5 seconds faster. Got the same result as above. “His” segment ( which I created ) was shorter than mine by about a third.
    Then today I rode with a friend with an app, I have a garmin 500. I toasted him by 100 m over the 3 / 10s course, and he ended up with the same time.
    Its not just a GPS thing. I think strava is tweaking to find some similarity and has created some issues.
    As an aside, my KOM has disappeared entirely, and a slower day has been ranked. I can’t even locate the other.
    And of course there is an issue with the smoothing the watts. I get the explanation, but I do have a watt hub, so I know what my watts are. The new KOM averaged 36.1 mph on a 2.7% grade downhill, (soft) yet pulled only 65 w.

    Weird stuff which takes away from the enjoyment of the program. Hopefully its being addressed. I might even pay for it if its wasn’t so bogus.

    • says

      Malco, after commenting above I put together this page (http://veloviewer.com/NewLeaderboard.php) on my site that, in a fairly lazy way, compares the actual distance travelled and from that the average speed for each athlete for a segment and provides an alternative leaderboard based on that result. Doesn’t work on all segments due to technical reasons but I’d be interested to know the segment you refer to shows a significantly different result to what Strava shows. From what I hear Strava are reworking their segment matching algorithm to make the start/end points for a segment the same for everyone (rather than the closest recorded gps point) although there’s a good chance they won’t apply this to historical data.

  3. Steve says

    You failed to mention what iPhone you were testing. I’ve used an iPhone 4S alongside and Garmin 800 and a Garmin 910XT and found little difference. The iPhone 3gs I used to use was poor, however.

    • Ivan Sinigaglia says

      With iPhone and Galaxy you can have some troubles with battery trying to ride long distances and the they don’t have barometric altimeter, just a reference from maps. Garmin is more acuratelly and fits better for riding, running and swimming.

  4. Danny Pelligrin says

    I would be Intrested to hear if anyone has tested the newer iPhone 5S as this has significatly more processing power for GPS data. I have the Edge 800 & a 5S and although the times are almost identical the climbing data is different so begs the question, which is more accrate?

Trackbacks

  1. [...] This is the most common type of problem that effects your VAM and Relative Power stats often giving ludicrously high values. Typically this is down to the GPS device used for the created segment getting confused and not recording the elevations accurately.  On some segments this can be hard to avoid due to severe gradients mixed with tree cover resulting in trouble picking up enough satellites but it usually more of an issue with the device being used for tracking, some are just much better than others. [...]

Leave a Reply

Your email address will not be published. Required fields are marked *