Autocross Stats (National & Houston Region)

MartinG · 12-03-2008, 09:39 AM

I recently engaged in a statistics exercise to see how accurately PAX puts different classes on an equal footing. The exercise got a little (okay, a lot) out of hand and I ended up with PAX-like adjustment factors calculated from real run times at national level events. And once I had all the data in a database, I just had to put a website front end on it to make it accessible.

So if you are interested, here it is: hotshoefactor.com. The database contains Houston Region results and National results.

Some of the stats available include:
- Traditional PAX and calculated PAX
- Event results ranked by both adjustment factors
- Driver rankings
- Driver stats
- Stats on which courses favor/disfavor which classes
- Info on which classes have hard/soft PAX

Let me know if you have any questions or comments.

* Registered users of the site do not see these ads.

Mechie3 · 12-03-2008, 10:50 AM

Oh Snap! (sorry had too...still reading though

)

EDIT: So, SNAP is what you believe the "true pax" is based upon the actual cars in a class whereas the real PAX value is what the handicap should theoretically be for a car modified to the complete extent of that classes rules, right? It's some fun data to look at to see how well prepared (or underprepared) certain classes are. I think SM is actually one of the most underprepared classes around (at the local level) because it becomes a catch all instead of a fully prepared class.

KC · 12-03-2008, 10:56 AM

Ok, now... do the same thing, only with ProSolos since they use their own PAX.

National level drivers go to more ProSolos in a year than Tours. (2-3 ProSolos vs. 1 Tour) and I think that would be a better litimus test to see who are the 'best' drivers.

If you can combine them, using the separate PAXs...

Also, 2006 data isn't there yet.

--kC

steverife · 12-03-2008, 12:11 PM

Interesting, but I don't really get how the rankings work. Guys in my class that I beat each time are WAY above me.

MartinG · 12-03-2008, 12:16 PM

Quote:

Originally Posted by Mechie3

Oh Snap! (sorry had too...still reading though

)

EDIT: So, SNAP is what you believe the "true pax" is based upon the actual cars in a class whereas the real PAX value is what the handicap should theoretically be for a car modified to the complete extent of that classes rules, right?

Thanks for checking it out! Yes, your summary is spot on. My approach measures actual performance for the previous season to arrive at the adjustment factors.

That works best for well subscribed classes with enough fully developed cars/drivers. It doesn't work well for classes like SM, AM and perhaps even STU where many cars are not fully prepped. Also, rain at the National Championships can impact the adjustment factor for the affected classes if they are thinly subscribed at National Tours. For well populated classes there is very little difference between PAX and SNAP adjustments.

On the other hand, SNAP is a calculation that doesn't leave any wiggle room and it should be immune to claims that is is hard or soft for certain classes. PAX can be (and has been) debated at length and in some cases the factors were adjusted after the debate, which just shows how much discretion is involved in setting them. Finally SNAP doesn't have artificial restrictions such as AM is automatically the fastest class. It just reflects the previous year's reality.

I think both ways of normalizing performance are useful because they measure slightly different things.

MartinG · 12-03-2008, 12:19 PM

Quote:

Originally Posted by KC

Ok, now... do the same thing, only with ProSolos since they use their own PAX.

National level drivers go to more ProSolos in a year than Tours. (2-3 ProSolos vs. 1 Tour) and I think that would be a better litimus test to see who are the 'best' drivers.

If you can combine them, using the separate PAXs...

Also, 2006 data isn't there yet.

--kC

LOL, are you volunteering to help clean the data? Pro Solos would be very interesting to add for the reasons you state, but it is hard to normalize the data because the event format keeps on changing. Once they settle down to something consistent I'll start collecting data to calculate factors as you suggest.

MartinG · 12-03-2008, 12:29 PM

Quote:

Originally Posted by steverife

Interesting, but I don't really get how the rankings work. Guys in my class that I beat each time are WAY above me.

It is quite possible that there is an error in the data or the calculation. This is very much a beta test. However, keep in mind that the ranking is based on all of your runs, not just the winning run. If your competition has more clean runs than you, they will rank higher unless your dirty times are still faster than their clean ones.

The ranking as it is implemented currently is intended to reward consistency over an entire season. I am open to other ways of doing this if anyone has suggestions.

If you look at your average finishing position in the driver stats, you should see that you finish higher than your competition if there are no data or coding errors.

Please take another look and let me know if this makes more sense now. If not I'll have to take a look at what's going on in the code in which case I'd need your name or DriverID.

steverife · 12-03-2008, 12:46 PM

Quote:

Originally Posted by MartinG

It is quite possible that there is an error in the data or the calculation. This is very much a beta test. However, keep in mind that the ranking is based on all of your runs, not just the winning run. If your competition has more clean runs than you, they will rank higher unless your dirty times are still faster than their clean ones.

Gotcha. I usually put down one +7 run per day, sometimes two, so that's probably why...

KC · 12-03-2008, 01:36 PM

Quote:

Originally Posted by MartinG

It is quite possible that there is an error in the data or the calculation. This is very much a beta test. However, keep in mind that the ranking is based on all of your runs, not just the winning run.

(snip)

The ranking as it is implemented currently is intended to reward consistency over an entire season. I am open to other ways of doing this if anyone has suggestions.

Error in logic. You're missing what national events are all about: National events are about getting the fastest run out of three tries at a course, not about being the fastest in all three runs (or they would be combined).

A super fast driver may hit cones in two of their three runs as they try to find the limits of the course, or a certain corner, adding additional time on to their final time for those runs, but still be faster in their last run, but clean, and winning the class.

Three attempts to have one fast run. Your formula rewards medocrity for mid-to-high pack, but clean, drivers. Whereas the national class champs may have been dirty on onw or two of their three runs gets penalized, but they still win because they had the fastest run. National events are all about that 1 fast time. Period.

If they were consitently winning their class with that one run on different events, great. But adding in dirty runs, unless all three are dirty, mis-represents the real fast people out there.

--kC

KC · 12-03-2008, 01:52 PM

Quote:

Thanks for checking it out! Yes, your summary is spot on. My approach measures actual performance for the previous season to arrive at the adjustment factors.

"Previous season" for the driver or the class itself? If for the class in general, great. If for the driver, again, error in logic.

2 examples:
1) Driver switches class from the previous year. Last year, they were in the class winning car and won many events. Has a change in direction after nats and switches classes. They decide over the winter to build a car. Beginning of the season, it's slow, and is still being worked on. By the end of the season, it's getting up to speed with the rest of the class. Driver can still be an excellent driver, just in an un-prepared car until nationals comes again in Sept, and they trophy. You're penalizing based on the development stage of the car.

2) How do you handle a driver that didn't run last year. Is the driver going to have less of a factor in their points if they didn't run last year? Or will their score be higher this year because all of a sudden, a top driver starts being consistently fast, trophying, but there's no data from last year to corraborate.

Just some ideas.

I personally don't like the idea of using a previous years drivers results to calculate this years performance as a driver because there's so many things that can effect the data/results.

If it's just for class calcs, great.

--kC

moxnix · 12-03-2008, 02:38 PM

There seems to be a problem with the data for 2008 Devens Tour. The top 4 drivers in your listing all have runs that are 0.000 seconds so they are much faster than the other drivers at the event.

Code:

Runs	STS2	177	BAKER, IAN	1	T 	1	1	1	57.804	56.532	56.532	47.284
Runs	STS2	77	SNYDER, MICHAEL	2	T 	2	2	2	58.352	57.068	57.068	47.732
Runs	AS	199	VITAMVAS, JOHN	1	T 	3	3	3	56.766	57.674	57.674	47.797
Runs	AS	99	CAPLAN, CLYDE	2	T 	4	4	4	58.196	59.127	59.127	49.001
Runs	FS	141	STRANO, SAM	1	T 	16	19	5	116.692	116.692	116.692	95.804

Scooby South · 12-03-2008, 02:56 PM

I love that your thinking outside the box....(not a PAX Fan) and hopefully can be used with accurate data...the problem is...it doesn't account for the following

-time of day the run was run
-weather conditions
-the (Ahhhhhhhh what a terrible run) Factor
-Big one, The Excuse factor...

-Nobody...can be 100% perfect all the time...I don't care if you are Mike King...or Tom Hoppe or Strano or Etc.....You get the idea....

I wish I was smart enough to help you find the deciding factor that would Equalize all but I can't....KC has some decent imput...however; Still not the end all beat all...

Bill

bucket7788 · 12-03-2008, 04:00 PM

Nicely done! I think it's great.

MartinG · 12-03-2008, 04:09 PM

Quote:

Originally Posted by KC

If they were consitently winning their class with that one run on different events, great. But adding in dirty runs, unless all three are dirty, mis-represents the real fast people out there.

--kC

This is certainly true. However, you just have to look at the standings in any series to find out who the people are who can put down that single fast run. It would be easy to rank people on their average class finishing position (and I may very well implement that as soon as I have time), but it would tell us pretty much the same thing as looking at a list of trophy winners.

Ranking people on their average performance offers a different perspective by emphasizing consistency over the single hero run. I am well aware that this is not how autocrosses are won or lost, but the intent was to look at the data in a way that doesn't just duplicate the results sheets.

I am in no way married to doing things this way, just explaining the thinking behind my approach.

MartinG · 12-03-2008, 04:15 PM

Quote:

Originally Posted by KC

"Previous season" for the driver or the class itself? If for the class in general, great. If for the driver, again, error in logic.

2 examples:
1) Driver switches class from the previous year. Last year, they were in the class winning car and won many events. Has a change in direction after nats and switches classes. They decide over the winter to build a car. Beginning of the season, it's slow, and is still being worked on. By the end of the season, it's getting up to speed with the rest of the class. Driver can still be an excellent driver, just in an un-prepared car until nationals comes again in Sept, and they trophy. You're penalizing based on the development stage of the car.

2) How do you handle a driver that didn't run last year. Is the driver going to have less of a factor in their points if they didn't run last year? Or will their score be higher this year because all of a sudden, a top driver starts being consistently fast, trophying, but there's no data from last year to corraborate.

Just some ideas.

I personally don't like the idea of using a previous years drivers results to calculate this years performance as a driver because there's so many things that can effect the data/results.

If it's just for class calcs, great.

--kC

The calculation is based roughly on the top 1/3 of run times from clean runs in a given class for each national event. I don't track which driver is responsible for these runs. I am just looking for the fastest runs in class.

A person who starts in a slow car would not be used to calculate the factors until the driver/car combination puts up runs in the top 1/3.

Does this clarify it?

MartinG · 12-03-2008, 04:16 PM

Quote:

Originally Posted by moxnix

There seems to be a problem with the data for 2008 Devens Tour. The top 4 drivers in your listing all have runs that are 0.000 seconds so they are much faster than the other drivers at the event.

That's certainly a problem

Thanks for pointing it out. I'll fix it tonight.

KC · 12-03-2008, 04:39 PM

Quote:

Originally Posted by MartinG

Ranking people on their average performance offers a different perspective by emphasizing consistency over the single hero run. I am well aware that this is not how autocrosses are won or lost, but the intent was to look at the data in a way that doesn't just duplicate the results sheets.

But you're ranking average drivers higher than they should be by factoring in two runs that don't matter to them either.

I respect what you're trying to do, but even with those average drivers, two of their runs don't count. Only one run matters at a national event, for everyone. Hero run or not, factoring in cones to the fast drivers other two runs penalizes them more than someone slower, but clean. Consistency is not a factor in National events. Having the single fastest time for that particular course is all that matters.

Fast drivers hit cones from pushing the limits and boundaries of their driving and car. To them, their two slowest runs don't matter. They're inconsequential in determining their final position. It may not be their last run, it could be that 2nd run... and sometimes 1st run.

Average drivers are too far away from that limit to be 'that close' to the cones to hit them. That doesn't make them better drivers deserving of higher rankings.

Yes, there are some seriously fast drivers, even champs, that also don't hit cones in any of their three runs, but hitting cones are a fact of being a fast driver, being 'too close' to the edge.

--kC

wrx wagone · 12-03-2008, 04:41 PM

I don't know about all of that. I hit cones all the time and I'm not a fast driver. Am I doing something wrong?

KC · 12-03-2008, 04:52 PM

Quote:

Originally Posted by wrx wagone

I don't know about all of that. I hit cones all the time and I'm not a fast driver. Am I doing something wrong?

It depends on the cones and how they are hit, and by what part of the car (or tire).

If you look at national results of the top 10 finishers, you'll see quite a few cones. Are they 'bad' drivers? No. No one can say they are. But the rankings from the OP could conceivably put a 15th place driver in the rankings ahead of a 10th place driver, when that 15th place driver was clean on 3 runs and the 10th place driver hit 4 cones on their 1st 2 runs.

I guess here's where my problem lies... No driver that was clean in three runs should be ranked higher than someone that finished higher, but with one cone on one run. It just doesn't matter. That slower driver couldn't drive the course "faster" than any driver in front of them, and it doesn't make them a better driver. I would arguably call them a consistently SLOW driver. (No offense... just trying to prove a point).

That's akin to saying the 5th place driver is a better driver than the driver that finished in 4th, or 3rd.

If they were a better driver, they would have beaten the 4th or 3rd place. It didn't happen when it mattered by having a faster final time.

That said, most rally-x events are scored in a cumulative time. You could use that to find out who the best rally-x driver is (the one that hits the fewest cones usually). Auto-x, nope. Only one run matters... your fastest. Can't beat the fast time in front of you? You're a slower driver.

--kC

Mind · 12-03-2008, 05:37 PM

Cool stuff. Maybe a useful option would be to include two sets of metrics -- one based on "fastest single run only" in addition to the accumulated average like you're doing now.

Another factor that could distort things - sandbagging.

KC · 12-03-2008, 05:55 PM

Quote:

Originally Posted by Mind

Cool stuff. Maybe a useful option would be to include two sets of metrics -- one based on "fastest single run only" in addition to the accumulated average like you're doing now.

Another factor that could distort things - sandbagging.

That's be good... and willing to bet you'll see those that consistently win ranked higher than those that just stay 'clean'.

--kC

MartinG · 12-03-2008, 06:46 PM

The issues with the Devens data are now fixed. Here are the revised top 10 drivers according to the current ranking method:

1 OTIS, RYAN 5.85 % 5 4
2 STRELNIEKS, ERIK 4.76 % 5 4
3 SHCHIPKOV, ALEX 3.81 % 5 4
4 NOLL, RANDALL 3.58 % 5 4
5 SHIELDS, MIKE 3.04 % 5 4
6 SAUNDERS, TOMMY 2.41 % 5 4
7 BOROWSKI, BARTEK 2.19 % 5 4
8 FENTER, CHRIS 2.18 % 5 4
9 BUETZER, RYAN 1.97 % 5 4
10 WILLIAMS, RON 1.95 % 5 6

I am not sure that all of them would consider themselves mediocre drivers

, but I appreciate KC's & Mind's point and will work up a second ranking that is closer to how autocross events are decided.

Thank you for all the feedback!

BlkWRXWag · 12-03-2008, 07:34 PM

I like that I'm ahead of Mssrs. Thomason and Isley, but pissed that hack Otis is #1

-Max.

Patrick L · 12-03-2008, 11:16 PM

Hmm, 25th driver ranking. Guess that is a good thing.
Still kind on confused on what all this data means.

Evoracer · 12-04-2008, 02:11 AM

Quote:

Originally Posted by BlkWRXWag

I like that I'm ahead of Mssrs. Thomason and Isley, but pissed that hack Otis is #1

-Max.

Hey I'm one ahead of Isley!

Rick

12-03-2008, 10:50 AM	#2
Mechie3 Scooby Guru Member#: 120152 Join Date: Jul 2006 Chapter/Region: MWSOC Location: Indy Vehicle: 2006 06 WRX, 86 FMOD Blue, White/Red	Oh Snap! (sorry had too...still reading though ) EDIT: So, SNAP is what you believe the "true pax" is based upon the actual cars in a class whereas the real PAX value is what the handicap should theoretically be for a car modified to the complete extent of that classes rules, right? It's some fun data to look at to see how well prepared (or underprepared) certain classes are. I think SM is actually one of the most underprepared classes around (at the local level) because it becomes a catch all instead of a fully prepared class. Last edited by Mechie3; 12-03-2008 at 10:57 AM.

12-03-2008, 02:38 PM	#11
moxnix Scooby Specialist Member#: 123798 Join Date: Aug 2006 Chapter/Region: MAIC Location: Woodbridge, VA Vehicle: Any Mazda	There seems to be a problem with the data for 2008 Devens Tour. The top 4 drivers in your listing all have runs that are 0.000 seconds so they are much faster than the other drivers at the event. Code: Runs STS2 177 BAKER, IAN 1 T 1 1 1 57.804 56.532 56.532 47.284 Runs STS2 77 SNYDER, MICHAEL 2 T 2 2 2 58.352 57.068 57.068 47.732 Runs AS 199 VITAMVAS, JOHN 1 T 3 3 3 56.766 57.674 57.674 47.797 Runs AS 99 CAPLAN, CLYDE 2 T 4 4 4 58.196 59.127 59.127 49.001 Runs FS 141 STRANO, SAM 1 T 16 19 5 116.692 116.692 116.692 95.804

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Autocross Stats Revisited	MartinG	Autocross	0	07-17-2009 06:30 PM
SCCA National Tour-Houston Event, who is going?	Robson	Texas Impreza Club Forum -- TXIC	144	04-20-2006 09:04 PM
SCCA Solo II National Tour Houston Event IMPORTANT INFO	Orion	Texas Impreza Club Forum -- TXIC	16	02-22-2002 07:47 PM
LVRSCCA Autocross Rounds 1 & 2	ElDuderino	South West Impreza Club Forum -- SWIC	11	01-05-2002 09:19 PM
SCCA Autocross at Smith & Wesson Sept 30th..	paultg	New England Impreza Club Forum -- NESIC	51	10-02-2001 07:34 AM

12-03-2008, 09:39 AM	#1
MartinG Scooby Specialist Member#: 89610 Join Date: Jun 2005 Chapter/Region: TXIC Location: Houston, TX Vehicle: 2006 STI Silver Honda Civic & CRX Old	Autocross Stats (National & Houston Region) I recently engaged in a statistics exercise to see how accurately PAX puts different classes on an equal footing. The exercise got a little (okay, a lot) out of hand and I ended up with PAX-like adjustment factors calculated from real run times at national level events. And once I had all the data in a database, I just had to put a website front end on it to make it accessible. So if you are interested, here it is: hotshoefactor.com. The database contains Houston Region results and National results. Some of the stats available include: - Traditional PAX and calculated PAX - Event results ranked by both adjustment factors - Driver rankings - Driver stats - Stats on which courses favor/disfavor which classes - Info on which classes have hard/soft PAX Let me know if you have any questions or comments. * Registered users of the site do not see these ads.

12-03-2008, 10:56 AM	#3
KC Scooby Guru Member#: 442 Join Date: Oct 1999 Chapter/Region: NESIC Location: SE Mass/RI Vehicle: 2013 Crosstrek XV 00 Honda S2000	Ok, now... do the same thing, only with ProSolos since they use their own PAX. National level drivers go to more ProSolos in a year than Tours. (2-3 ProSolos vs. 1 Tour) and I think that would be a better litimus test to see who are the 'best' drivers. If you can combine them, using the separate PAXs... Also, 2006 data isn't there yet. --kC

12-03-2008, 12:11 PM	#4
steverife Scooby Newbie Member#: 96749 Join Date: Sep 2005 Location: Knoxville, TN	Interesting, but I don't really get how the rankings work. Guys in my class that I beat each time are WAY above me.

12-03-2008, 02:56 PM	#12
Scooby South Subtly Outspoken Moderator Member#: 238 Join Date: Sep 1999 Chapter/Region: MWSOC Location: 2013 Limited SWP BRZ and a Vehicle: 99 Ver6 Recreation JDM Speedwagon 3 LSDs..:)	I love that your thinking outside the box....(not a PAX Fan) and hopefully can be used with accurate data...the problem is...it doesn't account for the following -time of day the run was run -weather conditions -the (Ahhhhhhhh what a terrible run) Factor -Big one, The Excuse factor... -Nobody...can be 100% perfect all the time...I don't care if you are Mike King...or Tom Hoppe or Strano or Etc.....You get the idea.... I wish I was smart enough to help you find the deciding factor that would Equalize all but I can't....KC has some decent imput...however; Still not the end all beat all... Bill

12-03-2008, 04:00 PM	#13
bucket7788 Scooby Specialist Member#: 124263 Join Date: Aug 2006 Chapter/Region: MAIC Location: Now with DB Super 16g and WI! Vehicle: 2004 WRX Wagon RomRaider + UTEC Tuned	Nicely done! I think it's great.

12-03-2008, 04:41 PM	#18
wrx wagone Scooby Specialist Member#: 97301 Join Date: Sep 2005 Chapter/Region: Tri-State Location: Phillytown, PA Vehicle: 2005 WRX Wag-Tow Rig 95 FM - QRE Invader	I don't know about all of that. I hit cones all the time and I'm not a fast driver. Am I doing something wrong?

12-03-2008, 05:37 PM	#20
Mind Scooby Specialist Member#: 109216 Join Date: Mar 2006 Chapter/Region: MAIC Location: PA Vehicle: 08 STI DGM 76-77 Lancia Scorpion(s)	Cool stuff. Maybe a useful option would be to include two sets of metrics -- one based on "fastest single run only" in addition to the accumulated average like you're doing now. Another factor that could distort things - sandbagging.

12-03-2008, 06:46 PM	#22
MartinG Scooby Specialist Member#: 89610 Join Date: Jun 2005 Chapter/Region: TXIC Location: Houston, TX Vehicle: 2006 STI Silver Honda Civic & CRX Old	The issues with the Devens data are now fixed. Here are the revised top 10 drivers according to the current ranking method: 1 OTIS, RYAN 5.85 % 5 4 2 STRELNIEKS, ERIK 4.76 % 5 4 3 SHCHIPKOV, ALEX 3.81 % 5 4 4 NOLL, RANDALL 3.58 % 5 4 5 SHIELDS, MIKE 3.04 % 5 4 6 SAUNDERS, TOMMY 2.41 % 5 4 7 BOROWSKI, BARTEK 2.19 % 5 4 8 FENTER, CHRIS 2.18 % 5 4 9 BUETZER, RYAN 1.97 % 5 4 10 WILLIAMS, RON 1.95 % 5 6 I am not sure that all of them would consider themselves mediocre drivers , but I appreciate KC's & Mind's point and will work up a second ranking that is closer to how autocross events are decided. Thank you for all the feedback!

12-03-2008, 07:34 PM	#23
BlkWRXWag Scooby Specialist Member#: 31821 Join Date: Jan 2003 Chapter/Region: SCIC Location: Vorshlag Motorsports/Chase Cam Vehicle: 2006 STR MX5 WRB BRZ Limited	I like that I'm ahead of Mssrs. Thomason and Isley, but pissed that hack Otis is #1 -Max.

12-03-2008, 11:16 PM	#24
Patrick L Scooby Specialist Member#: 634 Join Date: Dec 1999 Chapter/Region: TXIC Location: LMSports/Kartboy/Izoomgraphics Vehicle: 2000 Impreza 2.5RS DSP#195/ZzyzxMotorsports	Hmm, 25th driver ranking. Guess that is a good thing. Still kind on confused on what all this data means.