Login
01222019, 10:31 AM  #111  
Private
40
Rep 79
Posts 
Quote:
The general idea of measuring whether or not one car is truly faster than another, whether one marketing campaign is better than another, etc, falls under the framework of null hypothesis testing. The concept is to determine, without chance, whether some variant outperforms another. To do so we assume a distribution of the data, state a hypothesis, we calculate the probability of seeing the observed result given that the stated hypothesis is true, and report our findings. In more friendly terms, the times given for each car on a specific track represent the sampling distribution of that track. We can then calculate a sample average and use a hypothesis test to test the assumption that a particular time is faster than the track average or different than a specific time. When you do this for the OP you’ll find that the Pista and the TypeR represent opposing ends of the spectrum. I.e. there is enough evidence to support that the Pista is faster than the TypeR and we might expect this conclusion to be reasonably accurate across varying tracks. Ofcourse, there are downfalls with these assumptions and some techniques hold more validity than others. With a small sample size and unclear test conditions, it easy to logically dismiss a 2 millisecond time difference as chance. Further, if we take the hypothesis that the m2c and m3 zcp are equally as fast and calculate the probability, you’ll find that value to be > .9. In other words, it is highly likely that the two cars are equally fast. To determine whether one of them is faster than the other, we would need to run more tests and interpret those results as well. So to answer your question more directly, an accepted way of determining whether a result is significant or not falls under the framework of hypothesis testing that has been established by mathematicians and statisticians, which has been adopted in business, engineering, STEM fields etc. Last edited by jlhymb; 01222019 at 11:08 AM.. Reason: spelling 

Appreciate
0

01222019, 11:26 AM  #112 
S0THPAW
5070
Rep 7,487
Posts
Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

But with the facts we have just now, which car had the faster laptime at Magny Cours in this specific case? In this specific video with this driver?
Why is one car positioned above the other one if they're equally fast? Cheers Robin 
Appreciate
0

01222019, 12:08 PM  #113  
Private
40
Rep 79
Posts 
Quote:
1) The editorial in the OP shows that given their testing methodology, the m2c was 2ms faster than the m3 zcp around that circuit. 2) It's unclear that the m2c is actually faster than the m3 zcp or whether the result is just by chance. Both 1 and 2 can be true simultaneously. Aka that is the case here. For proof of 1, please see OP. For proof of 2, perform hypothesis test. We can conclude both 1 and 2 from the data we have. So for instance, lets say you and I play a game of basketball one on one, and I beat you. Does this mean that I am better at basketball than you? No, it doesn't. It means I beat you at one game of one on one. So then what if we play 500 games, and you win the vast majority of games, does that make you a better one on one basketball player than me? Most likely yes, we could say that there is evidence to prove that you are better than me at basketball and that your winning results are not luck (aka not by chance). Back to the car scenario, a 2ms difference is not enough to determine that the result of the OP is not due to chance. Hence 1 and 2 are simultaneously true. Last edited by jlhymb; 01222019 at 12:13 PM.. 

Appreciate
1
Project_F87394.50 
01222019, 12:29 PM  #114  
S0THPAW
5070
Rep 7,487
Posts
Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

OK understand your basketball analogy and I agree.
I'm more into football(that's playing a round ball kicking it with your feet ) and in tournaments sometimes the number one just got lucky or whatever and the runner up deserved it because they played better. But that's FIFA's 'choice' so to speak. Anyway... Quote:
In this case(never) they won't go repeating 'the same old song' over and over again, just because they only just have one car to do their laptimes with and many times the other car just isn't there to try it out. There are supertests with '10 new performance cars' but I haven't seen any of them doing a laptime over again because another car was just faster or whatever the reason. It is what it is. I think:Sorry for the car which should be faster but performed less than expected, kudos to the car which performed better. But that's the way it is. One chance, don't screw it. When there is a new(er) model, they will use that one to see what's the difference. M2 vs M2C in the MagnyCourse case. 2) I won't say 'by chance' because M2C is competitive being just as fast as M4 on the Nordschleife. But in this particular case (just this case)at this time the M2C had the faster laptime. What can we do about it. Nothing. If it was the other way around we wouldn't have this discussion I guess Cheers Robin 

Appreciate
0

01222019, 01:04 PM  #115 
Private
40
Rep 79
Posts 
Robin_NL
Lol, we are getting close. The term 'by chance' is used to indicate that diference in performance could occur from something that is not attributable to the cars actual capabilities. I.e. Lets say your car does 060 in 4.3 seconds consistently, but on your last launch your foot slipped and you recorded a time of 5.0 seconds 060... Does this mean your car is no longer a 4.3 second car? No, your car is still a 4.3 second car. The 5 second time occurred by chance circumstance aka your foot slipping. The idea is that the difference of 2ms can be attributed to these kind of chance circumstances. Noooooo no no no haha, this is precisely the point  even if the m3 zcp finished 2ms ahead of the m2c, the resulting interpretation would still be the same: not enough evidence to conclude that one is faster than the other (!) 
Appreciate
0

01222019, 01:51 PM  #116 
Brigadier General
4460
Rep 3,572
Posts 
If they ran each car once and only once on the track and came up with these results, I'd agree that it doesn't tell you much. However, car magazines or channels usually run several laps around the same track and publish the best time. I don't know about this channel but I do know SportAuto, MotorTrend or factory tests use the best of several laps. Unfortunately I don't know how many laps these guys did with each car but I bet it's not a single lap. Randy Pobst for instance does at least 10laps with each car on Head 2 Head. In any case, statistically significant or not, M2C was barely quicker than the M3 Comp on this track and nobody is claiming it'll be consistenly quicker than M3/M4 Comp on every track.

Appreciate
1
bradleyland1416.50 
01222019, 04:10 PM  #118  
Brigadier General
1667
Rep 3,307
Posts 
Quote:
Alas, you also have the track condition (temps are going to affect that as well) As jlhymb points out, these are chance circumstances and they get more prevalent when the cars are not run on the same day/time. And I think a lot of people believe we would not be having these arguments if the M3 was .02 faster  but once again jlhymb points out, that's not the case at all. In fact, what a lot of people fail to realize is that if the M2 and M3 were run on the same day, side by side who knows, maybe the M2s time would be much better?
__________________
2018 M3, Mineral White, 6 speed, ZCP, loaded sans CCB
2011 1M, Black loaded sans sat radio 

Appreciate
1
jlhymb40.00 
01222019, 04:18 PM  #119  
TIM YOYO
1417
Rep 3,264
Posts
Drives: 2013 M3
Join Date: Jun 2010
Location: Vero Beach, FL

Quote:
In statistical hypothesis testing, a result has statistical significance when it is very unlikely to have occurred given the null hypothesis. Another (from Harvard Business Review): “Statistical significance helps quantify whether a result is likely due to chance or to some factor of interest,” says Redman. When a finding is significant, it simply means you can feel confident that’s it real, not that you just got lucky (or unlucky) in choosing the sample. You keep saying that this outcome is not statistically significant, but you are grossly misusing that term while at the same time insisting upon my ignorance. So I'm going to try to clarify your argument for you, while showing just how ignorant you really are. What you intend to argue is that the result is within the margin of error. Since you turned this into some kind of statistics knowledge pissing match, I'll point out that we have no idea what the margin of error is (because we only know what we saw in the video). I'm nothing if not practical, so I'll grant that no car/driver is consistent to within 0.02s around any circuit. So yes, the results are well within the margin of error, and that means if you reran the race, it is just as likely that the M3C would win in any given comparison. That still is not the same as statistical significance, and continuing to insult me while demonstrating zero knowledge doesn't change that fact.
__________________
His: 2013 ///M3  Interlagos Blue Black MDCT
Hers: 2013 X3 28i  N20 Mineral Silver / Sand Beige / Premium, Tech Past: 2010 135i  TiAg Coral Red 6MT ///MSport 

Appreciate
0

01222019, 05:01 PM  #120  
Brigadier General
4460
Rep 3,572
Posts 
Quote:
Quote:


Appreciate
0

01222019, 05:39 PM  #121  
Private
40
Rep 79
Posts 
Quote:
The pvalue for a t test run under the null hypothesis that the m2c time = the m3 zcp time is > .9. In null hypothesis testing, it is generally accepted that < .05 is the rejection region. Since .9 > .05 we fail to reject the null hypothesis and conclude that there is strong evidence to suggest the times are equal. If you read the next two sentences in the wikipedia page you quoted, you will see the same. It states that a result is statistically significant when p<a. In our test, p>a, so the 2ms result is not considered statistically significant. Also, if you read my comments to Robin, you'll see the same sentiment shared with your HBR quote. So thanks(?) for proving my point lol. Last edited by jlhymb; 01222019 at 05:47 PM.. Reason: spelling 

Appreciate
1
Project_F87394.50 
01232019, 12:13 AM  #122 
S0THPAW
5070
Rep 7,487
Posts
Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

Well I stand corrected by the fact those journos do more than one FAST lap per car (I'm not sure about Sportauto Nordschleife, that's one fast lap) and off course that is just fair. What was I thinking?
There is only so much in a particular car when driven one fast(final) lap. The next one(or the previous ones) must be with no overheated tyres and brakes off course. Those are streetcars...you know it. Anyway, conspiracy theory. Those journos on Magny Cours tried a few laps and laptimes were still slower than M3 Comp. But the ultimate one got them being 0.02s faster in M2C than M3C so they put that one on the list. I just want to know how many FAST laptimes particular journos do in particular cars on particular tracks. Cheers Robin 
Appreciate
0

01232019, 12:18 AM  #123 
S0THPAW
5070
Rep 7,487
Posts
Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

And only that carjourno of that particular carmag/channel comes up with those laptimes and decides which one is fastest for them to show to the 'world'(us)
We as car enthusiasts/bimmerpost member/fanboys cannot do a thing about it. We can discuss/argue/let scientific words and graphs do their thing etcetera, but at the end of the day one car will still be higher on that leaderboard than the other one. Hate to say it. I remember driving my E90 M3 6MT @ Assen Moto GP Netherlands,>10 years ago, after 4 laps my front brakes were on fire and understeer like hell. ROFLOL. Cheers Robin 
Appreciate
0

01232019, 07:30 AM  #124 
Brigadier General
1667
Rep 3,307
Posts 
Let's just say they get a lot more discussion this way  if they had all four of those cars tied it would be a BORING article
__________________
2018 M3, Mineral White, 6 speed, ZCP, loaded sans CCB
2011 1M, Black loaded sans sat radio 
Appreciate
0

Post Reply 
Bookmarks 
Thread Tools  Search this Thread 

