Image via Wikipedia
My friend Bill Brenner has a post up on his Salted Hash blog today about a recent browser security study done by Accuvant LABS. The study shows that Google Chrome was the safest browser tested. Like Bill, I use Chrome too and agree with the Accuvant study.
The problem for Bill is that the study was sponsored by Google, the makers of Chrome. The fact that they paid for this study in Bill’s mind and in many other people’s minds calls the legitimacy of the entire study into question. Even if it is correct, it just doesn’t sit well with Bill.
Frankly I have the same problems with most product reviews, bake offs, analysis reports, etc. I have written about this before as well. In my mind it is a big reason why no one seems to pay attention to product reviews anymore.
It doesn’t make a difference if it is an “independent lab” doing the testing, a magazine’s testing department, industry awards or an analyst firm analyzing the market, the first thing I look at is who is paying for it. Sometimes finding out who is paying for it is not so easy or transparent either.
To be fair, some firms like Securosis for instance will say upfront that some research they are doing is being financed by a paying customer. Such was the case when Mike, Rich and Adrian did a dive on “fact based security security metrics”. The boys said upfront and at the bottom of the page that they thanked Red Seal for sponsoring the research.
Now does that mean that everything they wrote was for the benefit of Red Seal? I know Rich, Mike and Adrian too well to believe that. But it does give me pause when I read the report to remember that fact.
But I will say that over time I have come to soften my attitude on this issue (I must be getting old). For me it is a case of forewarned is forearmed and I take that disclosure in terms of evaluating how much weight to give the research. The same way a juror has to weigh the testimony of a witness depending on their believability.
In the case of Bill’s browser study, same thing. The chances that Google had a heavy hand in the study by Accuvant is pretty low, but it is something to consider. That is just the way it is.
But for Bill and the other doubting Thomas’s out there, what is the alternative?
One alternative is what Rick Moy and the guys at NSS Labs are doing. They have turned this equation on its head. They make their money from the end user, so the vendors being tested have little to no influence.
As Bill says just because Google paid for it doesn’t mean the study is wrong, but it does give you something else to consider. But since no one wants to do these tests or studies for free, someone has to pay and that is the truth of it.


![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=c21a029e-dd27-48db-8667-3a04d72b2baf)


