BMW Garage BMW Meets Register Search Today's Posts Mark Forums Read
BMW M2 Forum > BMW M2 Competition Model > BMW M2 Competition faster than Cayman S, M3 ZCP, Type R, C63S on Magny-Cours GP Track

Post Reply
 
Thread Tools Search this Thread
      01-22-2019, 11:31 AM   #111
jlhymb
Private
23
Rep
54
Posts

Drives: ymb
Join Date: Sep 2017
Location: location

iTrader: (0)

Quote:
Originally Posted by Robin_NL View Post
Serious post:
...
Or can we just dismiss this type of laptime leaderboards and which one is correct if any?
Why do carmags/journos etc do lap their cars on different tracks, just for nothing? Is Hockenheim Short/GP more important than Magny-Cours and why?
The Mother of all Tracks(Nordschleife) is pretty important(doh), but every car is also driven in different circumstances/temperatures and ambient temperatures and not on the same day...
When does a laptime count? When one or two rivals are not within, let's say 1 second of the car currently driven? That would be strange....
And who decides if/when a laptime counts/is valid vs the rest?
-Some Bimmerpost forummembers?
-The carjournos/ the guys who drove the car and made it possible?
-???
Concluding that two cars have the same performance is not dismissing results. It maybe a boring and unfavorable result to forum readers, but it is still a result.

The general idea of measuring whether or not one car is truly faster than another, whether one marketing campaign is better than another, etc, falls under the framework of null hypothesis testing.

The concept is to determine, without chance, whether some variant outperforms another. To do so we assume a distribution of the data, state a hypothesis, we calculate the probability of seeing the observed result given that the stated hypothesis is true, and report our findings.

In more friendly terms, the times given for each car on a specific track represent the sampling distribution of that track. We can then calculate a sample average and use a hypothesis test to test the assumption that a particular time is faster than the track average or different than a specific time. When you do this for the OP you’ll find that the Pista and the Type-R represent opposing ends of the spectrum. I.e. there is enough evidence to support that the Pista is faster than the Type-R and we might expect this conclusion to be reasonably accurate across varying tracks.

Ofcourse, there are downfalls with these assumptions and some techniques hold more validity than others. With a small sample size and unclear test conditions, it easy to logically dismiss a 2 millisecond time difference as chance. Further, if we take the hypothesis that the m2c and m3 zcp are equally as fast and calculate the probability, you’ll find that value to be > .9. In other words, it is highly likely that the two cars are equally fast. To determine whether one of them is faster than the other, we would need to run more tests and interpret those results as well.

So to answer your question more directly, an accepted way of determining whether a result is significant or not falls under the framework of hypothesis testing that has been established by mathematicians and statisticians, which has been adopted in business, engineering, STEM fields etc.

Last edited by jlhymb; 01-22-2019 at 12:08 PM. Reason: spelling
Appreciate 0
      01-22-2019, 12:26 PM   #112
Robin_NL
S0THPAW
Robin_NL's Avatar
Netherlands
4280
Rep
7,223
Posts

Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

iTrader: (0)

But with the facts we have just now, which car had the faster laptime at Magny Cours in this specific case? In this specific video with this driver?
Why is one car positioned above the other one if they're equally fast?

Cheers
Robin
Appreciate 0
      01-22-2019, 01:08 PM   #113
jlhymb
Private
23
Rep
54
Posts

Drives: ymb
Join Date: Sep 2017
Location: location

iTrader: (0)

Quote:
Originally Posted by Robin_NL View Post
But with the facts we have just now, which car had the faster laptime at Magny Cours in this specific case? In this specific video with this driver?
Why is one car positioned above the other one if they're equally fast?

Cheers
Robin
I think the part we are hung up on is generalization. There are two conclusions that are widely acceptable from the OP, let's be specific:

1) The editorial in the OP shows that given their testing methodology, the m2c was 2ms faster than the m3 zcp around that circuit.
2) It's unclear that the m2c is actually faster than the m3 zcp or whether the result is just by chance.

Both 1 and 2 can be true simultaneously. Aka that is the case here. For proof of 1, please see OP. For proof of 2, perform hypothesis test. We can conclude both 1 and 2 from the data we have.


So for instance, lets say you and I play a game of basketball one on one, and I beat you. Does this mean that I am better at basketball than you?

No, it doesn't. It means I beat you at one game of one on one.

So then what if we play 500 games, and you win the vast majority of games, does that make you a better one on one basketball player than me?

Most likely yes, we could say that there is evidence to prove that you are better than me at basketball and that your winning results are not luck (aka not by chance).
Back to the car scenario, a 2ms difference is not enough to determine that the result of the OP is not due to chance. Hence 1 and 2 are simultaneously true.

Last edited by jlhymb; 01-22-2019 at 01:13 PM.
Appreciate 1
      01-22-2019, 01:29 PM   #114
Robin_NL
S0THPAW
Robin_NL's Avatar
Netherlands
4280
Rep
7,223
Posts

Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

iTrader: (0)

OK understand your basketball analogy and I agree.
I'm more into football(that's playing a round ball kicking it with your feet )
and in tournaments sometimes the number one just got lucky or whatever and the runner up deserved it because they played better. But that's FIFA's 'choice' so to speak. Anyway...



Quote:
Originally Posted by jlhymb View Post
1) The editorial in the OP shows that given their testing methodology, the m2c was 2ms faster than the m3 zcp around that circuit.
2) It's unclear that the m2c is actually faster than the m3 zcp or whether the result is just by chance.
1) That's THEIR testing methodology and many(let's say 9-10)car testers, and I think it's 10-10) test/lap their cars like that afaik.

In this case(never) they won't go repeating 'the same old song' over and over again, just because they only just have one car to do their laptimes with and many times the other car just isn't there to try it out. There are supertests with '10 new performance cars' but I haven't seen any of them doing a laptime over again because another car was just faster or whatever the reason. It is what it is.

I think:Sorry for the car which should be faster but performed less than expected, kudos to the car which performed better. But that's the way it is. One chance, don't screw it.

When there is a new(er) model, they will use that one to see what's the difference. M2 vs M2C in the Magny-Course case.



2) I won't say 'by chance' because M2C is competitive being just as fast as M4 on the Nordschleife.
But in this particular case (just this case)at this time the M2C had the faster laptime. What can we do about it. Nothing.
If it was the other way around we wouldn't have this discussion I guess

Cheers
Robin
Appreciate 0
      01-22-2019, 02:04 PM   #115
jlhymb
Private
23
Rep
54
Posts

Drives: ymb
Join Date: Sep 2017
Location: location

iTrader: (0)

Robin_NL
Lol, we are getting close. The term 'by chance' is used to indicate that diference in performance could occur from something that is not attributable to the cars actual capabilities.
I.e. Lets say your car does 0-60 in 4.3 seconds consistently, but on your last launch your foot slipped and you recorded a time of 5.0 seconds 0-60...
Does this mean your car is no longer a 4.3 second car? No, your car is still a 4.3 second car. The 5 second time occurred by chance circumstance aka your foot slipping.

The idea is that the difference of 2ms can be attributed to these kind of chance circumstances.

Quote:
Originally Posted by Robin_NL View Post
...
If it was the other way around we wouldn't have this discussion I guess
...
Noooooo no no no haha, this is precisely the point - even if the m3 zcp finished 2ms ahead of the m2c, the resulting interpretation would still be the same: not enough evidence to conclude that one is faster than the other (!)
Appreciate 0
      01-22-2019, 02:51 PM   #116
cntzl
Brigadier General
cntzl's Avatar
United_States
3699
Rep
3,126
Posts

Drives: 2019 M2C 6MT
Join Date: Mar 2015
Location: San Mateo, CA

iTrader: (0)

If they ran each car once and only once on the track and came up with these results, I'd agree that it doesn't tell you much. However, car magazines or channels usually run several laps around the same track and publish the best time. I don't know about this channel but I do know SportAuto, MotorTrend or factory tests use the best of several laps. Unfortunately I don't know how many laps these guys did with each car but I bet it's not a single lap. Randy Pobst for instance does at least 10laps with each car on Head 2 Head. In any case, statistically significant or not, M2C was barely quicker than the M3 Comp on this track and nobody is claiming it'll be consistenly quicker than M3/M4 Comp on every track.
Appreciate 1
      01-22-2019, 02:59 PM   #117
akkando
Brigadier General
2214
Rep
3,081
Posts

Drives: 2017 M2 LBB DCT
Join Date: Jan 2017
Location: Bay Area

iTrader: (0)

This is quite the hill to die on!
Appreciate 0
      01-22-2019, 05:10 PM   #118
doug_999
Colonel
doug_999's Avatar
1332
Rep
2,908
Posts

Drives: 2018 M3/2011 1M
Join Date: Mar 2008
Location: Chicago, IL

iTrader: (0)

Quote:
Originally Posted by cntzl View Post
If they ran each car once and only once on the track and came up with these results, I'd agree that it doesn't tell you much. However, car magazines or channels usually run several laps around the same track and publish the best time. I don't know about this channel but I do know SportAuto, MotorTrend or factory tests use the best of several laps. Unfortunately I don't know how many laps these guys did with each car but I bet it's not a single lap. Randy Pobst for instance does at least 10laps with each car on Head 2 Head. In any case, statistically significant or not, M2C was barely quicker than the M3 Comp on this track and nobody is claiming it'll be consistenly quicker than M3/M4 Comp on every track.
The other thing that has to be taken into account is atmospheric conditions. I believe C&D adjusts for these, but R&T was not last time I looked - and these guys I have no idea. So if the M2 ran on a 85 degree day and the M3 ran on a 70 degree day, one could argue that the M2's times were high and it would have been even quicker.

Alas, you also have the track condition (temps are going to affect that as well)

As jlhymb points out, these are chance circumstances and they get more prevalent when the cars are not run on the same day/time.

And I think a lot of people believe we would not be having these arguments if the M3 was .02 faster - but once again jlhymb points out, that's not the case at all. In fact, what a lot of people fail to realize is that if the M2 and M3 were run on the same day, side by side who knows, maybe the M2s time would be much better?
__________________
2018 M3, Mineral White, 6 speed, ZCP, loaded sans CCB
2011 1M, Black loaded sans sat radio
Appreciate 1
jlhymb23.00

      01-22-2019, 05:18 PM   #119
bradleyland
TIM YOYO
United_States
1404
Rep
3,266
Posts

Drives: 2013 M3
Join Date: Jun 2010
Location: Vero Beach, FL

iTrader: (0)

Garage List
Quote:
Originally Posted by jlhymb View Post
You're lost in your own sauce buddy ... no one can help you

Here's the executive summary of our conversation:
Me: google hypothesis testing
You:
Maybe this will help you (from Wikipedia):

In statistical hypothesis testing, a result has statistical significance when it is very unlikely to have occurred given the null hypothesis.

Another (from Harvard Business Review):

“Statistical significance helps quantify whether a result is likely due to chance or to some factor of interest,” says Redman. When a finding is significant, it simply means you can feel confident that’s it real, not that you just got lucky (or unlucky) in choosing the sample.

You keep saying that this outcome is not statistically significant, but you are grossly misusing that term while at the same time insisting upon my ignorance. So I'm going to try to clarify your argument for you, while showing just how ignorant you really are.

What you intend to argue is that the result is within the margin of error. Since you turned this into some kind of statistics knowledge pissing match, I'll point out that we have no idea what the margin of error is (because we only know what we saw in the video). I'm nothing if not practical, so I'll grant that no car/driver is consistent to within 0.02s around any circuit.

So yes, the results are well within the margin of error, and that means if you re-ran the race, it is just as likely that the M3C would win in any given comparison.

That still is not the same as statistical significance, and continuing to insult me while demonstrating zero knowledge doesn't change that fact.
__________________
His: 2013 ///M3 - Interlagos Blue Black M-DCT
Hers: 2013 X3 28i - N20 Mineral Silver / Sand Beige / Premium, Tech
Past: 2010 135i - TiAg Coral Red 6MT ///M-Sport
Appreciate 0
      01-22-2019, 06:01 PM   #120
cntzl
Brigadier General
cntzl's Avatar
United_States
3699
Rep
3,126
Posts

Drives: 2019 M2C 6MT
Join Date: Mar 2015
Location: San Mateo, CA

iTrader: (0)

Quote:
Originally Posted by doug_999 View Post
The other thing that has to be taken into account is atmospheric conditions. I believe C&D adjusts for these, but R&T was not last time I looked - and these guys I have no idea. So if the M2 ran on a 85 degree day and the M3 ran on a 70 degree day, one could argue that the M2's times were high and it would have been even quicker.

Alas, you also have the track condition (temps are going to affect that as well)

As jlhymb points out, these are chance circumstances and they get more prevalent when the cars are not run on the same day/time.

And I think a lot of people believe we would not be having these arguments if the M3 was .02 faster - but once again jlhymb points out, that's not the case at all. In fact, what a lot of people fail to realize is that if the M2 and M3 were run on the same day, side by side who knows, maybe the M2s time would be much better?
Of course! Ideally for the most accurate comparison these tests should be done on the same day and by the same driver. Even then it may not be perfect since the temp and track conditions can change depending on the time of the day. I already stated the weather factor in my earlier post on the second page.

Quote:
Originally Posted by cntzl View Post
Even though I agree that these tests weren't done in the same day under the same weather conditions, the drivers who did tests aren't your average HPDE driver. Btw, most of the tests were done by the same driver (Romain Monti). He was the driver for M2 Competition, M3 Competition, M2, 718 Cayman S, C63S, Alpine A110, and Camaro.

Some of their video descriptions also include what tires they had on the cars. M2C for instance had stock Pilot Super Sport while Alpine had Pilot Sport 4S. It doesn't mention but I believe M2 and M3C had the stock PSS tires as well. However, M3C stock tire sizes are 265/285 (vs 245/265) so it has some advantage there in addition to the power difference.

Here are videos of some other cars at the same track and by the same driver:
Appreciate 0
      01-22-2019, 06:39 PM   #121
jlhymb
Private
23
Rep
54
Posts

Drives: ymb
Join Date: Sep 2017
Location: location

iTrader: (0)

Quote:
Originally Posted by bradleyland View Post
Maybe this will help you (from Wikipedia):

In statistical hypothesis testing, a result has statistical significance when it is very unlikely to have occurred given the null hypothesis.
...
“Statistical significance helps quantify whether a result is likely due to chance or to some factor of interest,” says Redman. When a finding is significant, it simply means you can feel confident that’s it real, not that you just got lucky (or unlucky) in choosing the sample.
...
Lol, the wheels never stop spinning with you... but at least your using google now and you're getting closer. Try googling the calculation for t test statistic and you will see that the denominator of the equation is part of the calculation used for the margin of error (critical value x standard deviation). You will notice that the margin of error and the p-value are related, and either or can be used to inform the interpretation of a hypothesis test.

The p-value for a t test run under the null hypothesis that the m2c time = the m3 zcp time is > .9. In null hypothesis testing, it is generally accepted that < .05 is the rejection region. Since .9 > .05 we fail to reject the null hypothesis and conclude that there is strong evidence to suggest the times are equal.

If you read the next two sentences in the wikipedia page you quoted, you will see the same. It states that a result is statistically significant when p<a. In our test, p>a, so the 2ms result is not considered statistically significant. Also, if you read my comments to Robin, you'll see the same sentiment shared with your HBR quote.

So thanks(?) for proving my point lol.

Last edited by jlhymb; 01-22-2019 at 06:47 PM. Reason: spelling
Appreciate 1
      01-23-2019, 01:13 AM   #122
Robin_NL
S0THPAW
Robin_NL's Avatar
Netherlands
4280
Rep
7,223
Posts

Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

iTrader: (0)

Well I stand corrected by the fact those journos do more than one FAST lap per car (I'm not sure about Sportauto Nordschleife, that's one fast lap) and off course that is just fair. What was I thinking?

There is only so much in a particular car when driven one fast(final) lap.
The next one(or the previous ones) must be with no overheated tyres and brakes off course. Those are streetcars...you know it.

Anyway, conspiracy theory. Those journos on Magny Cours tried a few laps and laptimes were still slower than M3 Comp. But the ultimate one got them being 0.02s faster in M2C than M3C so they put that one on the list.

I just want to know how many FAST laptimes particular journos do in particular cars on particular tracks.

Cheers
Robin
Appreciate 0
      01-23-2019, 01:18 AM   #123
Robin_NL
S0THPAW
Robin_NL's Avatar
Netherlands
4280
Rep
7,223
Posts

Drives: HS M2 Competition
Join Date: Apr 2005
Location: The Netherlands/ Lilyhammer, Norway

iTrader: (0)

And only that carjourno of that particular carmag/channel comes up with those laptimes and decides which one is fastest for them to show to the 'world'(us)

We as car enthusiasts/bimmerpost member/fanboys cannot do a thing about it.

We can discuss/argue/let scientific words and graphs do their thing etcetera, but at the end of the day one car will still be higher on that leaderboard than the other one. Hate to say it.

I remember driving my E90 M3 6MT @ Assen Moto GP Netherlands,>10 years ago, after 4 laps my front brakes were on fire and understeer like hell. ROFLOL.

Cheers
Robin
Appreciate 0
      01-23-2019, 08:30 AM   #124
doug_999
Colonel
doug_999's Avatar
1332
Rep
2,908
Posts

Drives: 2018 M3/2011 1M
Join Date: Mar 2008
Location: Chicago, IL

iTrader: (0)

Quote:
Originally Posted by Robin_NL View Post
And only that carjourno of that particular carmag/channel comes up with those laptimes and decides which one is fastest for them to show to the 'world'(us)
Let's just say they get a lot more discussion this way - if they had all four of those cars tied it would be a BORING article
__________________
2018 M3, Mineral White, 6 speed, ZCP, loaded sans CCB
2011 1M, Black loaded sans sat radio
Appreciate 0
Post Reply

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -5. The time now is 06:03 AM.




m2
Powered by vBulletin® Version 3.7.0
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
1Addicts.com, BIMMERPOST.com, E90Post.com, F30Post.com, M3Post.com, ZPost.com, 5Post.com, 6Post.com, 7Post.com, XBimmers.com logo and trademark are properties of BIMMERPOST