Over the summer there was an interesting discussion in the comments section of this article about the accuracy of dedicated GPS units from companies like Garmin compared to the GPS data collected via smartphones. I’ve personally been biking with a GPS since 2001 (13 years!) and my take was that neither is any more or less accurate than the other. Still, I was curious: was there any difference in accuracy between various types of GPS devices?
To find out, ckdake and I took 10 GPS devices–including 2 wristwatches, 2 bar-mounted units, 3 smartphones, 1 GPS-equipped helmet camera, 1 handheld device, and even a tablet–to a local quarter-mile track to see how each performed. We did our best to mount each unit according to its normal configuration; that is, wristwatches on a wrist, bar-mounted units on the bars, etc. All were rigged to a single rider (me!) and the test started and stopped at the same point on the track after riding 10 laps (2.5 miles). I stuck to the middle of lane 1 (thanks for the velodrome tip Chris!) and each GPS was started (but not moved) one at a time. We didn’t test timing on the devices since each unit is designed to sync time with the GPS satellites themselves, which meant we wouldn’t need to start and stop all devices at the same time.
Here are the results, ordered by distance accuracy.
| Device | Distance | Error |
| iPhone 5 (Strava) | 2.5019 | 0.08% |
| Asus Android Tablet (Strava) | 2.5124 | 0.50% |
| Garmin Forerunner 405CX | 2.5208 | 0.83% |
| Magellan Cyclo505 | 2.5239 | 0.96% |
| iPhone 5 (Garmin Fit) | 2.5328 | 1.31% |
| Garmin Edge 500 | 2.5334 | 1.34% |
| Garmin Fenix2 | 2.5501 | 2.00% |
| Nokia Lumia (GPS Logger) | 2.5542 | 2.17% |
| Garmin VIRB Elite | 2.5795 | 3.18% |
| Garmin 60CSx | 2.2529 | -9.88% |
Looking at the animated graphic at the top of this article, you can get a sense for another measure of accuracy–basically the tightness and “correctness” of the ride data. The actual route hugged the inside lane for each lap, though you can see some of the data veers toward the inside of the field and the edges in some cases (click here for an interactive plot). Subjectively, here’s how I would rank the accuracy based on the route plot:
Top 5: Asus Nexus Tablet (Strava), Nokia Lumia (GPS Logger), Garmin Fenix2, Magellan Cyclo505, and Garmin Forerunner 405.
Bottom 5: Garmin GPSMap60CSx, iPhone 5 (Strava), Garmin Edge 500, iPhone 5 (Garmin Fit), and Garmin VIRB Elite.
Interesting to note that the most accurate in terms of distance are not necessarily the most accurate at plotting route data. Keep this in mind when using GPS to map trails.
[see_also id=”40670″][/see_also]
Accuracy
Overall, the error rate is pretty low–around 3% or less for all but one device. Still, only 4 out of 10 were within 1% of the actual distance. On average (if we throw out the Garmin 60CSx) each device is off by about 1.4%, which is probably a good number to assume whenever analyzing your own data.
Error Direction
Surprisingly, all but one of the devices over-reported the distance, which either means our track is slightly longer than we assumed or something else is going on. In my own experience I’ve noticed most GPS units report distance on the high side (for example, clocking 104 miles in a 100-mile race), and perhaps it’s by design to make consumers feel faster/stronger than they really are. Product engineers know the device isn’t completely accurate so why not just estimate everything on the high side to keep riders happy? 🙂
Another more serious possibility is that distance was added to each track during the time it took us to start and stop all the devices during our test (you can see this in the lower right corner on some of the plots above). Basically if you take pretty much any GPS, hit start, and set it on the ground, after a few minutes you’ll find that your GPS has logged several feet of distance without moving. How is this possible?
GPS units never know exactly where you are located–they’re only accurate to within a few feet. So each time a GPS checks with the satellite, it’s calculating a slightly different position, even when you’re not moving.
Driving Factors
Perhaps one of the biggest misconceptions about GPS accuracy is that it’s all about the chipset (like the SiRFstarIII) in a particular device. While it’s true that some GPS chips are more powerful and/or sensitive than others, there are many, many factors that can affect distance calculations.
But let’s stick with the idea that chipsets affect distance accuracy for a moment. Two of the top four GPS units in our distance accuracy test are regular old smartphones that determine location not just with GPS but also using wi-fi and cell tower signals to triangulate a position. And the Magellan Cyclo505–#4 above–is also wi-fi enabled, though we don’t know if it’s actually using wi-fi to improve location accuracy. Still, having more sensors available to calculate location isn’t a guarantee of accuracy–the Garmin Fit app on an iPhone 5 ended up middle of the pack in our distance test, and the Nokia Lumia smartphone ranked 8 out of 10.
So what the heck happened to the Garmin GPSMap60CSx–it was off by nearly 10%?! There’s another factor at play here, and it’s known as polling frequency. Basically, GPS units check in with the satellite on a regular basis to get the user’s position and save that information, along with the time, to record your ride. Many apps and GPS units allow you to set the frequency yourself. For example, if you record your position once per second, you’ll get better data than if you record only once per minute. The tradeoff is more frequent polling will drain your battery faster and fill up your memory quicker.
In the case of our track test, the GPSMap60CSx was set to record much less frequently than the other units, resulting in “cut off” corners that significantly shortened the distance (remember, a straight line is always the shortest distance between two points). Fortunately this is an easy fix and has nothing to do with the accuracy of the unit itself.
This is what more frequent polling looks like (specifically, 1 point per second):
Some Garmin units (and perhaps units from other GPS manufacturers as well) have a “smart polling” option that doesn’t have a set polling frequency. Instead, the GPS records a point only if it’s not in a straight line with the point before it. A couple of the units we tested used this method which you can see below. In this test smart polling didn’t seem to affect accuracy but for a less-regular course (say, a twisting MTB trail) the GPS may have trouble keeping up with the rider’s constantly-changing vector.
Software Manipulation
One final factor in our GPS distance accuracy test is the use of post-ride software processing. Basically if the GPS data is uploaded to a website (Strava, Garmin Connect, MapMyRide, etc.), chances are that the data will be manipulated. This is particularly true for elevation data (which we’ll discuss in a follow-up post), but I suspect even distances are being tweaked behind the scenes for some of the devices we tested.
In particular, it’s either incredibly lucky that the Strava iPhone app got within 0.1% of our target distance or that Strava is guessing (correctly) that our “workout” occurred on a quarter-mile track and is adjusting distances accordingly. Looking at the trace from the Strava iPhone app, it’s easily the messiest and least “tight” trace which, again, suggests it was either lucky or is making distance adjustments behind the scenes. The fact that the second-most accurate distance tracking also came from the Strava app but running on a low-powered Android tablet lends some credibility to the latter suggestion*.
Conclusions
This is by no means a definitive GPS distance accuracy test. In fact, if we conduct this test again (which we probably will at some point), my hypothesis is the ranking will be completely different. And aside from the track test, a test in an area with zero cell phone or wi-fi signals over a known distance (not a quarter-mile track) could help isolate the additional sensor effect and the software manipulation issues. Bottom line: if you’re choosing a GPS device or app for clocking your rides, don’t focus too much on accuracy… they’re all pretty good, but none are bulletproof.
[see_also id=”51983″][/see_also]
Update: As many commenters have pointed out, the track we used was probably 400m, not a quarter mile. It’s an embarrassing mistake, especially since I ran track all four years of high school. 🙂 However, it still doesn’t change the accuracy rankings. Instead, it just means that all but one unit was off by an additional 0.6% (the Garmin GPSMap 60CSx actually improves by that amount.)
Update 2: After running a second test with a new crop of devices, we’re convinced the track is actually a quarter mile long, and not 400m.
* Being curious, I tested the theory that Strava is recognizing track workouts–and modifying them–by uploading raw data from one of the other devices used during the test. The result: no change in the distance. Still not definitive, but that theory is looking shaky.
52 Comments
Aug 29, 2016
Aug 30, 2016
However, that's not to say that part of the difference isn't due to elevation errors. As we found in our elevation test, GPS units aren't always very good at measuring elevation. So if your GPS thinks a ride was mostly flat, then it will underestimate the actual distance you traveled.
Aug 31, 2016
Dec 16, 2016
Dec 16, 2016
http://spectrum.ieee.org/tech-talk/at-work/test-and-measurement/why_every_gps_overestimates_distance_traveled
We're planning a second round of GPS tests for early next year and are planning to add a cycle computer to the mix this time. I'm stoked to see what we find!
Jan 24, 2017
Jan 24, 2017
I've always assumed that this underreporting is because a trail is constantly twisting and turning, and the Garmin just doesn't register often enough to note all of these changes--so the twistier the trail, the more the GPS alone is underreporting. I've also noticed that the "shape" of our rides on Strava do not look the same (even if we've done the exact same route), with hers featuring both a lot of straightened lines and inconsistencies between multiple trips over the same section of trail, whereas mine track the trail almost perfectly, and much more consistently.
Interestingly, my Edge 500 will be within 1% or so riding on the road with or without the sensor. But the Edge 25 also underreports a bit there, as well--sometimes by about 5%. I'd fix all of this by getting her a sensor for her Edge 25, but as far as I know it won't work with one.
Anyway, the tl;dr version is, if you use a Garmin and want more accurate data when riding on the trail, I think the sensor is the way to go if your model will accept one. And the Edge 25, I'm not impressed with at all.
Nov 6, 2014
Smart phones have the advantage that ephemeris data can be supplied much faster via data from cell towers therefore startup can be in the order of seconds. That so called assisted GPS (A-GPS) is probably what you are thinking of. The term Assisted GPS is however used by many as a catch all phrase for a variety of location assistance, hence sweeping statements like A-GPS is more accurate/better than GPS.
On a smart phone cellular signals can be used to derive a much lower accuracy position independent of GPS therefore a useful backup where GPS coverage is lost or very poor (canyons, tunnels, urban areas, under tree cover). Similarly wifi signals can be used to get a position, say indoors, where no GPS is available. Usually good enough to "check in" on social media etc.
Most devices such as Garmins are standalone and don't have the startup abilities of A-GPS and can't fall back on or utilise wifi or cell signals. The exception is Garmin's 620 and 920xt devices that use satellite ephemeris pre-loading over wifi vastly improving startup times.
I'm sure I'll be corrected in some areas but hopefully the above helps.
Nov 6, 2014
Is this true? If so, then you can't count on a phone when you are riding in the deep woods or mountains.
Nov 6, 2014
Nov 6, 2014
This is also why the offline topos in the Singletracks app are so great. If you download the files in advance, even if you don't have a cell signal you can pull up the map, and hit the locate button to show your position in relation to the trails around you.
Nov 6, 2014
Dec 16, 2016
Nov 6, 2014
Nov 4, 2014
Sep 16, 2017
Nov 4, 2014
-You mentioned GPSs often over-report distance, I don't think this is the case, I think the distance you think you're traveling is actually under-reported. For example, last time I did the Fools Gold 50 it was actually 56 miles.
-I don't know enough about tracks, but as you move further out in the lanes the distance/lap is greater. Which lap is 1/4 mile? Do you actually KNOW you rode 1/4 mile each lap?
-Part 2 - In my mind, for distance accuracy, the gold standard is still the oldschool bike computer with a magnet on the wheel. If it's programed correctly (you actually need to measure the roll out of the wheel) it'll be dead nuts right on. For next part, set up a bike computer properly and go ride a trail and then compare all the distances. The most twists and turns in the trail, the less accurage the GPS devices will be.
Nov 5, 2014
Nov 4, 2014
- Yes. The inside lap on a track is 1/4 mile. As you move outward, the start lines are staggered for each lane to make the lap lengths equal.
- Agreed, it would be interesting to test a bike computer since in theory this should be more accurate. This is how official track and field event courses are measured. Still, it's not perfect--depends on things like tire pressure (which influences the circumference), traction, etc. Hell, even a tape measure isn't 100% accurate depending on the temperature. :)
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Jan 28, 2016
http://spectrum.ieee.org/tech-talk/at-work/test-and-measurement/why_every_gps_overestimates_distance_traveled
To understand it, picture a tight clustering of darts around a bullseye. That the random errors a GPS unit will make when sampling. But, since distance is being calculated by summing up the small lines between each point, any error that is not in the same direction as the next point will be an overestimat!
Jan 28, 2016
A GPS could UNDERESTIMATE distance over a very short distance when looking at a curved track (like a MTB trail.) Again, knowing that the errors are random, it's possible the errors, at least for a few points in a row, could lie inside the arc of the curve instead of along it (or outside of it.) In this case, you're underestimating distance. Particularly curvy GPS tracks potentially suffer from this the most, especially if the polling frequency is low. Still, it's really rare that the stars would be aligned for this to be the case.
Jan 30, 2016
Nov 4, 2014
Jun 13, 2017
Jun 13, 2017
Nov 5, 2014
Both Orux Maps (free) and Backcountry Navigator (practically free) offer offline map support. For that matter, Google Maps now offers offline map support, so having maps available when out-of-service is not an issue.
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Phones can be a good backup, but it's not something functional enough at the prices I'm willing to spend to have it on my bars.
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Nov 5, 2014
Nov 5, 2014
Nov 5, 2014
Nov 5, 2014
Nov 5, 2014
Others have already questioned the assumption 2.5 miles (i.e. is track 400m etc).
You say devices were attached to wrists and bars, given 10 devices they must have been well distributed. I chucked the figures into excel and played about. Assuming half a metre separation between wrists, the additional distance covered by the outer wrist over this distance would have been in the region of 25 metres. That is over 0.5% of the total distance.
This is enough to shift devices around in the rankings and skew things. e.g. if, for example, the forerunner was on the right wrist and the iphone was on the left of the handlebar then applying the adjustment would bring the two devices to within a few metres.
I am not saying this happened here but I came across a similar test a while back. The guy had 5 devices on each wrist and doing several circuits of a lake oblivious to the fact that the extra distance covered by his outer wrist made a mockery of the relative distances not to mention the assumption of the true distance.
Nov 4, 2014
I'd be interested to see the Microsoft Band's accuracy as well.
Nov 5, 2014
Nov 6, 2014
Nov 5, 2014
Some devices offer do offer battery saving modes where the unit does indeed "sleep" for intervals before waking and grabbing a fix. That without a doubt will affect distances.
But the auto modes are mainly to save on storage space. e.g. a 45 minute run the other day with my old 60c (no expansion memory) at 1 second recording chewed up half my device storage! No use for a day's hiking! It makes less of a dent on my new 920xt but over time it would.
The point is auto recording does not necessarily dictate the frequency of "GPS units checking in with the satellite". So there may well be a distinction between "odometer" distance based on actual polling versus the point-to-point distance of a logged track.
So I am used to my hiking handhelds devices reporting a different distance to the downloaded and saved gpx track.
Not sure what happens in the internals of, say, a Garmin fit file in Garmin Connect but I suspect exporting a gpx file to use in another app may lose all but the point-to-point information.
Just another complication to consider! Sigh.
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014
Nov 4, 2014