Welcome to the North American Subaru Impreza Owners Club Wednesday September 28, 2016
Home Forums WikiNASIOC Products Store Modifications Upgrade Garage
NASIOC
Go Back   NASIOC > NASIOC General > Motorsports

Welcome to NASIOC - The world's largest online community for Subaru enthusiasts!
Welcome to the NASIOC.com Subaru forum.

You are currently viewing our forum as a guest, which gives you limited access to view most discussions and access our other features. By joining our community, free of charge, you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is free, fast and simple, so please join our community today!

If you have any problems with the registration process or your account login, please contact us.
* Registered users of the site do not see these ads. 
Reply
 
Thread Tools Display Modes
Old 12-03-2008, 09:39 AM   #1
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default Autocross Stats (National & Houston Region)

I recently engaged in a statistics exercise to see how accurately PAX puts different classes on an equal footing. The exercise got a little (okay, a lot) out of hand and I ended up with PAX-like adjustment factors calculated from real run times at national level events. And once I had all the data in a database, I just had to put a website front end on it to make it accessible.

So if you are interested, here it is: hotshoefactor.com. The database contains Houston Region results and National results.

Some of the stats available include:
- Traditional PAX and calculated PAX
- Event results ranked by both adjustment factors
- Driver rankings
- Driver stats
- Stats on which courses favor/disfavor which classes
- Info on which classes have hard/soft PAX

Let me know if you have any questions or comments.
* Registered users of the site do not see these ads.
MartinG is offline   Reply With Quote
Sponsored Links
* Registered users of the site do not see these ads.
Old 12-03-2008, 10:50 AM   #2
Mechie3
Scooby Guru
 
Member#: 120152
Join Date: Jul 2006
Chapter/Region: MWSOC
Location: Indy
Default

Oh Snap! (sorry had too...still reading though )

EDIT: So, SNAP is what you believe the "true pax" is based upon the actual cars in a class whereas the real PAX value is what the handicap should theoretically be for a car modified to the complete extent of that classes rules, right? It's some fun data to look at to see how well prepared (or underprepared) certain classes are. I think SM is actually one of the most underprepared classes around (at the local level) because it becomes a catch all instead of a fully prepared class.

Last edited by Mechie3; 12-03-2008 at 10:57 AM.
Mechie3 is offline   Reply With Quote
Old 12-03-2008, 10:56 AM   #3
KC
Scooby Guru
 
Member#: 442
Join Date: Oct 1999
Chapter/Region: NESIC
Location: RI/SE Mass
Vehicle:
16 SS
00 S2000

Default

Ok, now... do the same thing, only with ProSolos since they use their own PAX.

National level drivers go to more ProSolos in a year than Tours. (2-3 ProSolos vs. 1 Tour) and I think that would be a better litimus test to see who are the 'best' drivers.

If you can combine them, using the separate PAXs...

Also, 2006 data isn't there yet.

--kC
KC is offline   Reply With Quote
Old 12-03-2008, 12:11 PM   #4
steverife
Scooby Newbie
 
Member#: 96749
Join Date: Sep 2005
Location: Knoxville, TN
Default

Interesting, but I don't really get how the rankings work. Guys in my class that I beat each time are WAY above me.
steverife is offline   Reply With Quote
Old 12-03-2008, 12:16 PM   #5
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default

Quote:
Originally Posted by Mechie3 View Post
Oh Snap! (sorry had too...still reading though )

EDIT: So, SNAP is what you believe the "true pax" is based upon the actual cars in a class whereas the real PAX value is what the handicap should theoretically be for a car modified to the complete extent of that classes rules, right?
Thanks for checking it out! Yes, your summary is spot on. My approach measures actual performance for the previous season to arrive at the adjustment factors.

That works best for well subscribed classes with enough fully developed cars/drivers. It doesn't work well for classes like SM, AM and perhaps even STU where many cars are not fully prepped. Also, rain at the National Championships can impact the adjustment factor for the affected classes if they are thinly subscribed at National Tours. For well populated classes there is very little difference between PAX and SNAP adjustments.

On the other hand, SNAP is a calculation that doesn't leave any wiggle room and it should be immune to claims that is is hard or soft for certain classes. PAX can be (and has been) debated at length and in some cases the factors were adjusted after the debate, which just shows how much discretion is involved in setting them. Finally SNAP doesn't have artificial restrictions such as AM is automatically the fastest class. It just reflects the previous year's reality.

I think both ways of normalizing performance are useful because they measure slightly different things.
MartinG is offline   Reply With Quote
Old 12-03-2008, 12:19 PM   #6
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default

Quote:
Originally Posted by KC View Post
Ok, now... do the same thing, only with ProSolos since they use their own PAX.

National level drivers go to more ProSolos in a year than Tours. (2-3 ProSolos vs. 1 Tour) and I think that would be a better litimus test to see who are the 'best' drivers.

If you can combine them, using the separate PAXs...

Also, 2006 data isn't there yet.

--kC
LOL, are you volunteering to help clean the data? Pro Solos would be very interesting to add for the reasons you state, but it is hard to normalize the data because the event format keeps on changing. Once they settle down to something consistent I'll start collecting data to calculate factors as you suggest.
MartinG is offline   Reply With Quote
Old 12-03-2008, 12:29 PM   #7
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default

Quote:
Originally Posted by steverife View Post
Interesting, but I don't really get how the rankings work. Guys in my class that I beat each time are WAY above me.
It is quite possible that there is an error in the data or the calculation. This is very much a beta test. However, keep in mind that the ranking is based on all of your runs, not just the winning run. If your competition has more clean runs than you, they will rank higher unless your dirty times are still faster than their clean ones.

The ranking as it is implemented currently is intended to reward consistency over an entire season. I am open to other ways of doing this if anyone has suggestions.

If you look at your average finishing position in the driver stats, you should see that you finish higher than your competition if there are no data or coding errors.

Please take another look and let me know if this makes more sense now. If not I'll have to take a look at what's going on in the code in which case I'd need your name or DriverID.
MartinG is offline   Reply With Quote
Old 12-03-2008, 12:46 PM   #8
steverife
Scooby Newbie
 
Member#: 96749
Join Date: Sep 2005
Location: Knoxville, TN
Default

Quote:
Originally Posted by MartinG View Post
It is quite possible that there is an error in the data or the calculation. This is very much a beta test. However, keep in mind that the ranking is based on all of your runs, not just the winning run. If your competition has more clean runs than you, they will rank higher unless your dirty times are still faster than their clean ones.
Gotcha. I usually put down one +7 run per day, sometimes two, so that's probably why...
steverife is offline   Reply With Quote
Old 12-03-2008, 01:36 PM   #9
KC
Scooby Guru
 
Member#: 442
Join Date: Oct 1999
Chapter/Region: NESIC
Location: RI/SE Mass
Vehicle:
16 SS
00 S2000

Default

Quote:
Originally Posted by MartinG View Post
It is quite possible that there is an error in the data or the calculation. This is very much a beta test. However, keep in mind that the ranking is based on all of your runs, not just the winning run.

(snip)

The ranking as it is implemented currently is intended to reward consistency over an entire season. I am open to other ways of doing this if anyone has suggestions.
Error in logic. You're missing what national events are all about: National events are about getting the fastest run out of three tries at a course, not about being the fastest in all three runs (or they would be combined).

A super fast driver may hit cones in two of their three runs as they try to find the limits of the course, or a certain corner, adding additional time on to their final time for those runs, but still be faster in their last run, but clean, and winning the class.

Three attempts to have one fast run. Your formula rewards medocrity for mid-to-high pack, but clean, drivers. Whereas the national class champs may have been dirty on onw or two of their three runs gets penalized, but they still win because they had the fastest run. National events are all about that 1 fast time. Period.

If they were consitently winning their class with that one run on different events, great. But adding in dirty runs, unless all three are dirty, mis-represents the real fast people out there.

--kC

Last edited by KC; 12-03-2008 at 01:43 PM.
KC is offline   Reply With Quote
Old 12-03-2008, 01:52 PM   #10
KC
Scooby Guru
 
Member#: 442
Join Date: Oct 1999
Chapter/Region: NESIC
Location: RI/SE Mass
Vehicle:
16 SS
00 S2000

Default

Quote:
Thanks for checking it out! Yes, your summary is spot on. My approach measures actual performance for the previous season to arrive at the adjustment factors.
"Previous season" for the driver or the class itself? If for the class in general, great. If for the driver, again, error in logic.

2 examples:
1) Driver switches class from the previous year. Last year, they were in the class winning car and won many events. Has a change in direction after nats and switches classes. They decide over the winter to build a car. Beginning of the season, it's slow, and is still being worked on. By the end of the season, it's getting up to speed with the rest of the class. Driver can still be an excellent driver, just in an un-prepared car until nationals comes again in Sept, and they trophy. You're penalizing based on the development stage of the car.

2) How do you handle a driver that didn't run last year. Is the driver going to have less of a factor in their points if they didn't run last year? Or will their score be higher this year because all of a sudden, a top driver starts being consistently fast, trophying, but there's no data from last year to corraborate.

Just some ideas. I personally don't like the idea of using a previous years drivers results to calculate this years performance as a driver because there's so many things that can effect the data/results.

If it's just for class calcs, great.

--kC
KC is offline   Reply With Quote
Old 12-03-2008, 02:38 PM   #11
moxnix
Scooby Specialist
 
Member#: 123798
Join Date: Aug 2006
Chapter/Region: MAIC
Location: Woodbridge, VA
Vehicle:
Any Mazda

Default

There seems to be a problem with the data for 2008 Devens Tour. The top 4 drivers in your listing all have runs that are 0.000 seconds so they are much faster than the other drivers at the event.

Code:
Runs	STS2	177	BAKER, IAN	1	T 	1	1	1	57.804	56.532	56.532	47.284
Runs	STS2	77	SNYDER, MICHAEL	2	T 	2	2	2	58.352	57.068	57.068	47.732
Runs	AS	199	VITAMVAS, JOHN	1	T 	3	3	3	56.766	57.674	57.674	47.797
Runs	AS	99	CAPLAN, CLYDE	2	T 	4	4	4	58.196	59.127	59.127	49.001
Runs	FS	141	STRANO, SAM	1	T 	16	19	5	116.692	116.692	116.692	95.804
moxnix is offline   Reply With Quote
Old 12-03-2008, 02:56 PM   #12
Scooby South
Subtly Outspoken
Moderator
 
Member#: 238
Join Date: Sep 1999
Chapter/Region: MWSOC
Location: 2013 STX National BRZ and a
Vehicle:
99 Ver6 Recreation
JDM Speedwagon 3 LSDs..:)

Default

I love that your thinking outside the box....(not a PAX Fan) and hopefully can be used with accurate data...the problem is...it doesn't account for the following

-time of day the run was run
-weather conditions
-the (Ahhhhhhhh what a terrible run) Factor
-Big one, The Excuse factor...
-Nobody...can be 100% perfect all the time...I don't care if you are Mike King...or Tom Hoppe or Strano or Etc.....You get the idea....

I wish I was smart enough to help you find the deciding factor that would Equalize all but I can't....KC has some decent imput...however; Still not the end all beat all...

Bill
Scooby South is offline   Reply With Quote
Old 12-03-2008, 04:00 PM   #13
bucket7788
Scooby Specialist
 
Member#: 124263
Join Date: Aug 2006
Chapter/Region: MAIC
Location: Now with DB Super 16g and WI!
Vehicle:
2004 WRX Wagon
RomRaider + UTEC Tuned

Default

Nicely done! I think it's great.
bucket7788 is offline   Reply With Quote
Old 12-03-2008, 04:09 PM   #14
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default

Quote:
Originally Posted by KC View Post
If they were consitently winning their class with that one run on different events, great. But adding in dirty runs, unless all three are dirty, mis-represents the real fast people out there.

--kC
This is certainly true. However, you just have to look at the standings in any series to find out who the people are who can put down that single fast run. It would be easy to rank people on their average class finishing position (and I may very well implement that as soon as I have time), but it would tell us pretty much the same thing as looking at a list of trophy winners.

Ranking people on their average performance offers a different perspective by emphasizing consistency over the single hero run. I am well aware that this is not how autocrosses are won or lost, but the intent was to look at the data in a way that doesn't just duplicate the results sheets.

I am in no way married to doing things this way, just explaining the thinking behind my approach.
MartinG is offline   Reply With Quote
Old 12-03-2008, 04:15 PM   #15
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default

Quote:
Originally Posted by KC View Post
"Previous season" for the driver or the class itself? If for the class in general, great. If for the driver, again, error in logic.

2 examples:
1) Driver switches class from the previous year. Last year, they were in the class winning car and won many events. Has a change in direction after nats and switches classes. They decide over the winter to build a car. Beginning of the season, it's slow, and is still being worked on. By the end of the season, it's getting up to speed with the rest of the class. Driver can still be an excellent driver, just in an un-prepared car until nationals comes again in Sept, and they trophy. You're penalizing based on the development stage of the car.

2) How do you handle a driver that didn't run last year. Is the driver going to have less of a factor in their points if they didn't run last year? Or will their score be higher this year because all of a sudden, a top driver starts being consistently fast, trophying, but there's no data from last year to corraborate.

Just some ideas. I personally don't like the idea of using a previous years drivers results to calculate this years performance as a driver because there's so many things that can effect the data/results.

If it's just for class calcs, great.

--kC
The calculation is based roughly on the top 1/3 of run times from clean runs in a given class for each national event. I don't track which driver is responsible for these runs. I am just looking for the fastest runs in class.

A person who starts in a slow car would not be used to calculate the factors until the driver/car combination puts up runs in the top 1/3.

Does this clarify it?
MartinG is offline   Reply With Quote
Old 12-03-2008, 04:16 PM   #16
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default

Quote:
Originally Posted by moxnix View Post
There seems to be a problem with the data for 2008 Devens Tour. The top 4 drivers in your listing all have runs that are 0.000 seconds so they are much faster than the other drivers at the event.
That's certainly a problem Thanks for pointing it out. I'll fix it tonight.
MartinG is offline   Reply With Quote
Old 12-03-2008, 04:39 PM   #17
KC
Scooby Guru
 
Member#: 442
Join Date: Oct 1999
Chapter/Region: NESIC
Location: RI/SE Mass
Vehicle:
16 SS
00 S2000

Default

Quote:
Originally Posted by MartinG View Post
Ranking people on their average performance offers a different perspective by emphasizing consistency over the single hero run. I am well aware that this is not how autocrosses are won or lost, but the intent was to look at the data in a way that doesn't just duplicate the results sheets.
But you're ranking average drivers higher than they should be by factoring in two runs that don't matter to them either.

I respect what you're trying to do, but even with those average drivers, two of their runs don't count. Only one run matters at a national event, for everyone. Hero run or not, factoring in cones to the fast drivers other two runs penalizes them more than someone slower, but clean. Consistency is not a factor in National events. Having the single fastest time for that particular course is all that matters.

Fast drivers hit cones from pushing the limits and boundaries of their driving and car. To them, their two slowest runs don't matter. They're inconsequential in determining their final position. It may not be their last run, it could be that 2nd run... and sometimes 1st run.

Average drivers are too far away from that limit to be 'that close' to the cones to hit them. That doesn't make them better drivers deserving of higher rankings.

Yes, there are some seriously fast drivers, even champs, that also don't hit cones in any of their three runs, but hitting cones are a fact of being a fast driver, being 'too close' to the edge.

--kC
KC is offline   Reply With Quote
Old 12-03-2008, 04:41 PM   #18
wrx wagone
Scooby Specialist
 
Member#: 97301
Join Date: Sep 2005
Chapter/Region: Tri-State
Location: Phillytown, PA
Vehicle:
2005 WRX Wag-Tow Rig
125 FM - QRE Invader

Default

I don't know about all of that. I hit cones all the time and I'm not a fast driver. Am I doing something wrong?
wrx wagone is offline   Reply With Quote
Old 12-03-2008, 04:52 PM   #19
KC
Scooby Guru
 
Member#: 442
Join Date: Oct 1999
Chapter/Region: NESIC
Location: RI/SE Mass
Vehicle:
16 SS
00 S2000

Default

Quote:
Originally Posted by wrx wagone View Post
I don't know about all of that. I hit cones all the time and I'm not a fast driver. Am I doing something wrong?
It depends on the cones and how they are hit, and by what part of the car (or tire).

If you look at national results of the top 10 finishers, you'll see quite a few cones. Are they 'bad' drivers? No. No one can say they are. But the rankings from the OP could conceivably put a 15th place driver in the rankings ahead of a 10th place driver, when that 15th place driver was clean on 3 runs and the 10th place driver hit 4 cones on their 1st 2 runs.

I guess here's where my problem lies... No driver that was clean in three runs should be ranked higher than someone that finished higher, but with one cone on one run. It just doesn't matter. That slower driver couldn't drive the course "faster" than any driver in front of them, and it doesn't make them a better driver. I would arguably call them a consistently SLOW driver. (No offense... just trying to prove a point).

That's akin to saying the 5th place driver is a better driver than the driver that finished in 4th, or 3rd.

If they were a better driver, they would have beaten the 4th or 3rd place. It didn't happen when it mattered by having a faster final time.

That said, most rally-x events are scored in a cumulative time. You could use that to find out who the best rally-x driver is (the one that hits the fewest cones usually). Auto-x, nope. Only one run matters... your fastest. Can't beat the fast time in front of you? You're a slower driver.

--kC
KC is offline   Reply With Quote
Old 12-03-2008, 05:37 PM   #20
Mind
Scooby Specialist
 
Member#: 109216
Join Date: Mar 2006
Chapter/Region: MAIC
Location: PA
Vehicle:
08 STI DGM
76-77 Lancia Scorpion(s)

Default

Cool stuff. Maybe a useful option would be to include two sets of metrics -- one based on "fastest single run only" in addition to the accumulated average like you're doing now.

Another factor that could distort things - sandbagging.
Mind is offline   Reply With Quote
Old 12-03-2008, 05:55 PM   #21
KC
Scooby Guru
 
Member#: 442
Join Date: Oct 1999
Chapter/Region: NESIC
Location: RI/SE Mass
Vehicle:
16 SS
00 S2000

Default

Quote:
Originally Posted by Mind View Post
Cool stuff. Maybe a useful option would be to include two sets of metrics -- one based on "fastest single run only" in addition to the accumulated average like you're doing now.

Another factor that could distort things - sandbagging.
That's be good... and willing to bet you'll see those that consistently win ranked higher than those that just stay 'clean'.

--kC
KC is offline   Reply With Quote
Old 12-03-2008, 06:46 PM   #22
MartinG
Scooby Specialist
 
Member#: 89610
Join Date: Jun 2005
Chapter/Region: TXIC
Location: Houston, TX
Vehicle:
2006 STI Silver
Honda Civic & CRX Old

Default

The issues with the Devens data are now fixed. Here are the revised top 10 drivers according to the current ranking method:

1 OTIS, RYAN 5.85 % 5 4
2 STRELNIEKS, ERIK 4.76 % 5 4
3 SHCHIPKOV, ALEX 3.81 % 5 4
4 NOLL, RANDALL 3.58 % 5 4
5 SHIELDS, MIKE 3.04 % 5 4
6 SAUNDERS, TOMMY 2.41 % 5 4
7 BOROWSKI, BARTEK 2.19 % 5 4
8 FENTER, CHRIS 2.18 % 5 4
9 BUETZER, RYAN 1.97 % 5 4
10 WILLIAMS, RON 1.95 % 5 6

I am not sure that all of them would consider themselves mediocre drivers , but I appreciate KC's & Mind's point and will work up a second ranking that is closer to how autocross events are decided.

Thank you for all the feedback!
MartinG is offline   Reply With Quote
Old 12-03-2008, 07:34 PM   #23
BlkWRXWag
Scooby Specialist
 
Member#: 31821
Join Date: Jan 2003
Chapter/Region: SCIC
Location: FRSport.com
Vehicle:
2015 DS WRX

Default

I like that I'm ahead of Mssrs. Thomason and Isley, but pissed that hack Otis is #1

-Max.
BlkWRXWag is offline   Reply With Quote
Old 12-03-2008, 11:16 PM   #24
Patrick L
Scooby Specialist
 
Member#: 634
Join Date: Dec 1999
Chapter/Region: TXIC
Location: Tulsa, OK
Vehicle:
2000 Impreza 2.5RS
Becasue ITS racecar

Default

Hmm, 25th driver ranking. Guess that is a good thing.
Still kind on confused on what all this data means.
Patrick L is offline   Reply With Quote
Old 12-04-2008, 02:11 AM   #25
Evoracer
Scooby Newbie
 
Member#: 91529
Join Date: Jul 2005
Location: OC
Vehicle:
'03 Evo #179 STU
Bilstein/Dunlop/ChaseCam

Default

Quote:
Originally Posted by BlkWRXWag View Post
I like that I'm ahead of Mssrs. Thomason and Isley, but pissed that hack Otis is #1

-Max.
Hey I'm one ahead of Isley!

Rick
Evoracer is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Autocross Stats Revisited MartinG Autocross 0 07-17-2009 06:30 PM
SCCA National Tour-Houston Event, who is going? Robson Texas Impreza Club Forum -- TXIC 144 04-20-2006 09:04 PM
SCCA Solo II National Tour Houston Event IMPORTANT INFO Orion Texas Impreza Club Forum -- TXIC 16 02-22-2002 07:47 PM
LVRSCCA Autocross Rounds 1 & 2 ElDuderino South West Impreza Club Forum -- SWIC 11 01-05-2002 09:19 PM
SCCA Autocross at Smith & Wesson Sept 30th.. paultg New England Impreza Club Forum -- NESIC 51 10-02-2001 07:34 AM

All times are GMT -4. The time now is 12:07 PM.


Powered by vBulletin® Version 3.7.0
Copyright ©2000 - 2016, Jelsoft Enterprises Ltd.
Powered by Searchlight © 2016 Axivo Inc.
Copyright ©1999 - 2016, North American Subaru Impreza Owners Club, Inc.