Don't Count the Dots (Consumer Reports)

[CaMIRO]

Don't Count the Dots

Brief quotes below...

GM Executive Lori Queen is reported to have fired back this week at Consumer Reports, through Automotive News, for the perceived injustice of its top-ten auto list this year. While we'll admit both that several vehicles on that list have better competitors, and that by corollary an all-Japanese list seems peripherally disingenuous, the real question is whether obvious inadequacies in Consumer Reports' methodology have any chance of being rectified.

  

It is these we propose to discuss, briefly, this morning.

How can a self-selective survey of subscribers possibly have external validity? Only Consumer Reports subscribers may fill out the survey, and they are selected not at random but, rather, participate on a volunteer basis. It is unclear what verification of their responses is performed but, this and other issues notwithstanding, the self-selective nature of this survey is a massive barrier to its trustworthiness and, ultimately, its relevance. By Champion's own comments to McElroy - "our job... is to serve our subscribers... the people that we are helping (are) the same people that are filling-in the survey" - Consumer Reports' findings require a serious disclaimer! (Autoline Detroit, October 30th, 2005)

How can one possibly compare the reliability factors of two vehicles to which (potentially) wildly different numbers of responses have been submitted? Consumer Reports has never disclosed these numbers, and will not upon request. To attempt to compare Consumer Reports' 'dots' between a Chevrolet Malibu and a Toyota Camry, without clear indication of the number of responses, flies in the face of margin-of-error, which holds that the accuracy of an estimate varies with the numbers of responses used in determining it.

However, given the clear problems noted here, Consumer Reports must be careful of perpetuating the perception gap. Even when done unintentionally, through inaccuracy and through the failure to provide critical information to ensure that a survey's results are properly interpreted, disingenuousness remains this end result. Moreover, perpetuating the perception gap does not serve the consumer - it does not "maintain and enhance the quality of life for consumers" (as Consumers Union claims is its mission) but, rather, it maintains the status quo, perhaps merely because not enough readers have caught on to these basic statistical faux-pas.

[Dsuupr]

I like how she went at it with specific comment rather than blank I hate them statements.

One has to wonder when or if people will catch on to their flawed thinking.

[CARBIZ]

I've already had one customer speculate about the accuracy of CR because of the Japanese sweep. Perhaps this is a good thing. Of course the sheeple will just nod and continue buying Japanese, but the larger majority and the cynics may start to scratch their heads and begin to wonder about biases and inconsistencies BECAUSE everybody else was shut out of these latest results.

For my part, I am not the least surprised or bothered by this. It merely confirms what I've believed all along - many automotive "journalists" can't see beyond their own prejudices.

[Newbiewar]

really great read...

have i ever got surveyed? nope...

but has my dad who has owned like 5 toyotas and 2 fords (which the only reason he owns the fords are because the rig is 20k + pounds and toyota can handle that... so if you can imagine if the F350's never see a day unloaded... they are likely to have a problem here or there for such a burdon of weight strain) so because of his problematic ford (1989 F350), on his 2003 F350 he has now purchased a 10 year 100k mile mechanical insurance for the vehicle to insure it never has problems that come out of his pocket...

[FUTURE_OF_GM]

People are starting to turn on CR...

Everyone I've talked to is suspicious.

[Guest Josh]

J.D. Power sent me a survey about my Solstice. I answered it honestly.

[regfootball]

People are starting to turn on CR...

Everyone I've talked to is suspicious.

if the public ever does turn on CR, I hope they go DOWN. Still, the clueless yuppie liberal mondo urban bunch who always buy Toyotas and Hondas are so thick around here these days and they have all the big paying jobs and they are the ones who are dropping big coin like nothing on all the toyohondas etc. Between a CR backlash and a rise in Hyundai though could be what cracks away at toyota's position in the US.

Seriously, folks spend 30k on used Camry's and 25k on 4 year old A4's........but won't buy a new Impala for 20g.

Edited March 17, 2006 by regfootball

[Deuce]

I haven't posted a while (EDIT: First Post) but here I go. I was also a little PO'd when I heard of the Japanese sweep and read through my free copy of the CR automotive issue. Sadly, even though there may be an outside chance this will affect CR a little bit the bigger issue is more excuses and finger-pointing by a GM executive.

I work for GM like many on this board and am not one to bash the executives, but it seems like they are the worst at dealing with the press. I know it's easier for Toyota's Jim Press to stay positive since they are on top of the world right now, but I don't remember Dieter Zetsche or Carlos Ghosn always talking about things outside their influence holding them back. That's why Chrysler and Nissan are hot, and GM is on a downhill slide. Just my 2 cents.

Edited March 17, 2006 by Deuce

[boblutzfan]

Great read. Automobear has some great points. I would love for the mainstream media to do a story on CR - but you know they wont. The mainstream media has been party to the perception gap that exists. Unfortunately I think the only solution is for GM, Ford and Chrysler to continue to produce high quality products that compete well with the Japanese and wait for the perception gap to close in time.

[FUTURE_OF_GM]

It's not that CR picked all jap cars as their best that alarms people...

I mean, hell, they know and we all know that jap cars are VASTLY superior to everything american, as does the public (I'm saying this in the P.O.V. of the average car buying idiot)

What DOES alarm people is the fact that 8 out of 10 of the vehicles came from 2 companies... It just doesn't hold water and sets off a question mark. Of course, I'm sure A LOT of sheep will still take it as gospel and Detroit will be eroded further.