Sunday, December 14, 2014

How Awesome Is Your Ultrasignup Score?

In October I mentioned that my friend John commented that my UltraSignup score would suffer after choosing to run the Condor 25k with a friend rather than race it. It wasn’t the first time I had heard someone from the Corvallis trail running group make mention of Ultra-Signup scores. My friend Scott once went on about how small races help your USU score whereas the big well known races are likely to hurt your score. I had heard the guys playfully banter about who had a higher ranking. And my friend Gaby asked me at the conclusion of the 2012 Gorge Waterfalls 50k, “Jeez, did you have to win by so much? You made this one of my lowest Ultra-signup results!”


Yes, it is obvious people are paying attention to their Ultra Signup scores, but just how important do people really think they are??


Ultrasignup arrived on the ultrarunning scene in March of 2009 as the brainchild of Mark Gilligan. Today USU handles the registration for approximately 80% of all ultramarathons and many shorter distance events as well. The company tries to upload all of the race results that they are aware of, even if the race does not use USU for registration or timing. RD’s can assist in this massive data collection by submitting results to USU in a spreadsheet format.


Mark admits that he was interested in knowing about his competition going into races, but looking up each person individually took an enormous amount of time, especially for the larger races. He said he was interested in putting together a system to help participants (including himself) easily analyze the competition. The idea for quantifying results came to him one night while having beers with friends after work: The score for each race is calculated by dividing a runner’s time by the winner’s time and the percentage becomes the runner’s score for each event. An individual’s aggregate score is just an average of each individual race score. It was a simple calculation that allowed people to see how many races someone had run and what their quantified average of past results were, which created a general runner rank. The system also benefitted ultra fans, allowing them to quickly identify top competitors or “race favorites”.


The system is decidedly less complicated than something like the BCS ranking system which involves a combination of expert polls and computer calculations. It is this simplicity that it allows it to be applied universally to the ultra-running community, but unlike football there is no accounting for “strength of schedule”. The formula does not take into account the number of participants in a race, the number of people a runner beats, or the strength of the competition in the race. The winner of a ten person fat-ass in the middle of December gets the same reward as the winner of Western States on Ultrasignup. Likewise, a runner could have an exceptionally good race but if the winner destroys the course record on that particular day, the Ultrasignup score may not reflect the accomplishment. But the method of dividing one’s time by the winner’s time was certainly not a novel concept in Ultrarunning: race series, like the Montrail Ultra Cup and the Oregon Trail Series, predated Ultrasignup with their use of the same formula to determine runner rank within the series.


Mark says he has gotten numerous suggestions on how to improve the algorithm, but few are as simple as the current system. And Mark states: “It is not our intent to inject subjectivity, it really is just your average of how you have done over the course of your running career.”
But recently there have been changes to the USU system and the ensuing discussions on the Ultrasignup Facebook page made me realize a lot of runners take this rank really, really seriously. In fact, Mark says he knows of a few situations where egos have been put in check after people have been called out on their low rank. And there are stories where people have sized up potential dates based on their Ultrasignup scores. Now that’s taking it seriously!
One of the changes involved timed race events. Previously these had not been calculated into the score, but these results have now been averaged in (using a modified formula of runner’s distance/winner’s distance). Those who frequently run timed events were strongly in favor of this, basically stating they wanted “credit” for these results. But opponents stated these races are often used for training or running a specific distance with no intention of running the entire time and they didn’t want to be penalized for this. The second change involved listing the DNF’s, though that had no effect on the actual score.
With all the banter and importance placed on the Ultrasignup score, I have decided to share my wisdom and experience (61 Ultrasignup results!) on how to achieve an Awesome Ultrasignup Score:
- If you are having a bad day, you should DNF. Sure the DNF’s are listed now but they don’t actually hurt your overall ranking. On the other hand, sticking it out through a tough day will earn you a score significantly lower than you are used to. My rough day at Western States in 2012 earned the ranking of 58%!
-Never run competitive races. Whenever a lot of fast people show up, the time between you and the first runner is likely to be a lot greater, meaning your score goes down even if you think you had a good day.
- Run a bunch of small, non-competitive races to pad your score. Better yet, go to a fat ass with Ultrasignup registration and race it all out. I like to race and I like to have some runs as training runs. Local races are perfect for this. And high placings at these races that nobody outside of your state has ever heard of (or cares about) can not only boost your ego, but also you USU score!
- Do not do any races for fun. Do not do any races for training. And certainly don’t run with anybody slower than you.  Two years ago, I ran the Pac Rim One Day with my daughter. I thought it was pretty fucking awesome to run an ultra with her when she was only 8 years old. But Ultrasignup didn’t - they reamed me with a 35%. My recent endeavors to help a friend meet her goal at the Condor 25k were also frowned upon in the USU system.
- Do not race outside of your strengths. Last year I thought it would be fun to run a mountain half marathon to mix things up and get some good vert for Western States. But I am not great at steep uphills or steep downhills and I certainly don’t have the speed to compete in a half marathon. Ultrasignup corroborated this with a lower than average score for the day.
- Never race against Ellie Greenwood. A couple years ago, I had a decent run at Chuckanut (it was on the top 20 all time list at the time). But Ellie set the course record that day. Likewise, her two speedy runs at Western States hurt my scores on the day. Fast chicks and dudes can do a number on your score even if you have a good day. It’s best to just not run any races against them at all.

***
;) Ok, hopefully, everyone knows I don’t mean any of that! (Well, except the part about Ellie Greenwood. Nobody has a chance against her :). My point is that ultrarunning is a lot more than a computer generated score. Enjoying new places, hanging with other people, being active, and even toughing it out on a bad day are all things that enrich our lives and make us better people. Being competitive and pushing yourself to do better and even beating others is all part of this sport, too, and I certainly relish that aspect of ultrarunning. But it is good to keep it in perspective.
Amongst all the Facebook feedback and suggestions, a few commenters hit the nail on the head. One said his Ultrasignup score was “not something I think about when I am running an ultra. Stats and rankings don’t mean much.” Another person noted, “I don’t need to be ranked to love this sport.” Even Mark Gilligan, the founder of Utrasignup, says: “the ranking is just for fun so don't take it so seriously. It was added to the site after a few beers so take it for what it is.”
Ok, so fess up! Has the ranking system ever influenced your race decisions, such as what races you run or whether or not to drop? Be honest - How seriously do you take your Ultrasignup score?

23 comments:

Dr. O said...

Actually, I was kind of excited to even see my name on USU. It meant I was out pursuing my goal. Thanks for explaining the system. Looks like I also need to avoid any races in which you are registered! LOL

Olga King said...

You mean my summer run at LaLuz 9M (short!) uphill (I walk!) trail race at altitude (I live in TX!) with 400 local entrants a day after another short altitude race and a year of non-running was a bad idea? No, you don't say so!!! :)

Pam said...

Dr. O - I love that! That is definitely a great outlook on the USU score!

David Jacobson said...

If they added the score to the site "just for fun," it would be even more fun if they made it more relevant. Like an additional ranking that reset every year (like a season series score), a gender or AG category score, etc. The average over the entire "running career" is meaningless.

Daniel Slater said...

Great post. As a barely midpacker, I use the score to get an idea of what time I can generally expect by comparing finishes of those with similar scores to mine. Other than that it is useless to me. I have received low scores (Ice Age 2014 when Max King set course record) and super high scores at small races that I know were inflated.

Pam said...

Olga- I am not saying that kind of race is a bad idea at all; I am just saying it will hurt you USU score!

David- I agree people's motivations, goals, and desires can change a lot over an entire running career. The problem with a reset, is then the scores may be based off of only 1-2 results, which is also pretty meaningless. But I guess that's my point-the scores shouldn't matter that much. They can be a useful tool but they should be kept in perspective.

Olga King said...

I know, I was speaking tongue in cheek:) thus the smiley face. Hard with internet writing these days. In reality, I run because I like the race, the place, it fits my schedule, I haven't been there, am visiting friends, need a training run, have a goal...so, the score doesn't influence my pick, but I do actually like to look at it to see, as others say, what potential time I may be able to shoot for, and also, by looking ahead of registration list, which one of my friends is coming!

mustlovejogs.com said...

funny :)

josh z. said...

I pretty much only run to feed my ego with USU scores. I stick to your set of rules religiously and it pays. the only rule I don't stick to is the DNF-ing one. I have DNS-ed twice and USU maintains they are DNFs.

josh z. said...

more seriously, it'd be fun to go to more competitive races (or races with much deeper fields--there is almost always a handful of badasses at any race around these parts). but that often requires more sacrifice of time (especially when there is travel) and finances (travel, higher entry fees, etc.). it also requires more preparation with qualifiers and/or ridiculously fast sellouts at midnight 6+ months away from the event. I don't have a lot of free time and I like to sign up for stuff on more of a whim. so I stick to your rules as a consequence of this. great ultra geek-out post!

Anonymous said...

Josh if you have a DNS and not DNF you can change it in the settings..

Crankypants said...

So the choices are to have high score or have fun ... let me think, it's a tough one

Laura said...

This was a fun read, Pam! My score is going to plummet (not that it's stellar to begin with) after I do this 10k snowshoe race in February - one of those "seemed like a good idea at the time" sort of things. I guess I need to dust off some old tennis rackets. Or maybe have a spectacular blow up at mile 5 and DNF thus preserving my aforementioned less than stellar USU score ...

SteveQ said...

I saw a guy drop out at mile 95 of a 100 miler, rather than have his ranking drop. It appears sponsors that don't know much about the sport can be impressed by USU scores.

Barry Young said...

The best thing about USU scores is that calculating the impact of various scenarios is a fantastic distractions for the last 10 miles of a 50-miler.

Jennifer O'Connor said...

Another hilarious - because often so wrong - thing USU does is to predict finishing times and places based on those scores. If it's a race with a really fast CR (like Javelina), I know I'll finish hours after they predict. But if it's a small race or one with a slow CR, I get insulted at how slow they say I'll run. I've often raced with the goal of beating my USU prediction.

Heidi Nicole said...

The USU scores and predicted finish times amuse me. I was predicted to finish RRR about 5 hours AFTER the cutoff...which isn't surprising considering USU had almost nothing to base this prediction off of.

In the end, it's just a number...but I'm not going to lie, it felt kind of cool to prove USU wrong at RRR! :)

Lucy said...

Thank you so much, Pam, what a great article. I actually don't care at all about results, I am a mid pack runner, I will run whatever races are convenient and affordable to travel to, be it the ultra competitive TNF 50 or some smaller local race. If DNFed that's fine too; the score bothers me not at all.
I do however wish there was a very basic functionality to UltraSign up - like do you want your information private or public. A very simple thing.
I am nowhere near a top competitor, nobody is going to "size my score up" as a possible competitor.
I am not even worried about my past results and scores, I am talking about future events that also show up if you are signed up for something. Doesn't seem like a big deal. Didn't seem to me either. But this year I was unfortunate enough to get in a situation where a guy I saw a few times started to follow me and show up unexpectedly, and he couldn't show to work or home because we had security in place, so he started showing up to races as those were the easiest things to look up and bothering me there. I had asked ultrasign up numerous times to make my future races private so nobody could see them, explaining the situation, humiliating as it was to have to repeat myself; they'd hide the upcoming races; then 3 days later there they will be again. The attitude that I got from ultrasign up initially was very compassionate and understanding, then when I had to keep contacting them several times because the upcoming races kept showing up, their attitude went to very annoyed and the advice not to sign up for any more races. It's all fine now as my attorney got a legal injunction - but it took time - and a lot of DNSs and wasted money. They also deleted all my results (instead of hiding the upcoming events like I requested). I understand my situation is not very typical and it was Extremely Uncomfortable for me to go through, having done absolutely nothing to cause it; it didn't turn me against the sport, but it did make me feel like why is this so impossible to have privacy settings? Maybe some people want their results hidden, they have a right, I needed to have my upcoming events hidden, I had a compelling reason. Why is such a simple functionality so hard to institute? Seems like it should be people's choice to decide what privacy settings they have on.

Anonymous said...

> I do however wish there was a very basic functionality to UltraSign up - like do you want your information private or public. A very simple thing.

I had the same problem, especially when they started showing future races! That really got me concerned. While I don't have your problem, I am generally concerned with privacy and I strongly prefer it to take a lot of work to figure out my whereabouts from the internet.

I emailed them once and they were able to hide my results. If you go to a specific race, you can still find me ... but if you click my name from there, you only see the results of other people with my name.

I'm not sure if it matters, but I don't have an ultra signup account -- I always register as guest, even though I have ~20-30 results.

But yes, I very much think this should be something built in, rather than special-cased!

Juli said...

The discussion about privacy settings is a good one. Unfortunately, we live in a world that does not feel as safe as it once did. I didn't realize that there was no privacy setting for USU. If you don't want your info public, you shouldn't need to explain why. Regarding the scores, we all know who the elites are and even the very fast runners. The scores should be "just for fun" and they are not always reflective of ability. That being said, no system would be perfect. At TUDC this past Saturday, I made a point of complimenting Trevor for sticking it out after a sub 4 50K and then a struggle and slowdown to finish the 50M in a much slower time than he was capable of. He is only 19 and running his first 50M. He wouldn't have learned much if he had dropped. The kind of person you are, eg. helping a friend finish a race over your own accomplishment can't be measured but it says a lot more about you in a way that really counts. Thanks, Pam! You described it in a way that puts it in perspective.

Guillermo E. Ayala said...

I follow it a bit. I don't take so,serious or get attached to it. I like to see my name with my races and stats, pure motivation. I love to see the record of other runners specially around my age and their stats. Last year I was 4th place in a 50mi race of 80 with only about 44 finishers and that has catapulted my motivation to be on the top 3 or possibly win it one day, Finger Lakes 50s is the event and my favorite. I'm aware it could be a more difficult goal as the race gains popularity. But there's no way this score would be in my head during any race or even worse, to be selective with the events I sign up for. No way, I don't even respect any average score out there since they are ALL subjective to me. I think getting caught up with any type of scores or ranking simply destroys the passion for the sport.

Jason Wheat said...

I agree. Me personally, it pushes me to give it my best. At a race where I know I can't catch the person in front of me and the person behind won't catch me even if I stumble/walk, it pushes me to run hard at my max ability. It's important for me to do my best in all events. But a con to racing for time rather than completion is that I miss out on socializing/eating at aid stations, taking in wonderful views, and missing unique course historicals/landmarks.

Anonymous said...

I will not date a woman with a higher score than me.