Last week I wrote about how Mozilla is working to change the way it measures security. That effort will probably help eliminate some of the misinformation that gets spread around since measuring the number of bugs doesn't exactly provide an accurate perspective.
We can look at a recent survey conducted by Pingdom as an example how the same principles apply in other areas. The company, which monitors up-time for various systems for a fee, recently monitored the servers that provide software updates for Microsoft, Apple, and Ubuntu. When the survey was over the company said that Microsoft's servers had the best up-time while Ubuntu's had the worst. As a result of that claim many news stories appeared with a misleading headline. If readers didn't bother to read the story details or investigate Pingdom's claims on their own then they probably went away with a totally false perception.
The primary problem is that, in my perspective, Pingdom's study was done is an irresponsible and lazy manner. They knew full well that Microsoft uses load balancing and DNS techniques to maintain availability for their update servers. And, they know that Ubuntu has a long list of mirror sites around the world that provide software updates, and that Ubuntu does not use DNS techniques to spread the load. Instead Ubuntu leaves it up to users to choose which mirrors they want to use.
Nevertheless, Pingdom totally ignored the update provisioning methods used by Canonical (makers of Ubuntu) and chose to only monitor the main update site for up-time while ignoring the availability of all the mirrors. They also failed to do any research regarding the 'here and now' regarding Ubuntu - if they had then they'd have realized that the company was about to release a new OS version, which of course means that countless people would be downloading untold numbers of megabytes of data each. In other words, when you've got 10's of thousands of people all trying to download huge amounts of data simultaneously a small amount of downtime might occur intermittently - maybe. That depends on infrastructure and administrative needs.
So, after Pingdom finished monitoring per their chosen methods (e.g. ignoring relevant factors) over the course of April, May, and June of this year they then issued a proclamation that "Microsoft’s software update beats Apple and Ubuntu" and various news media picked up on that and ran with it. Fortunately some of those outlets provided objective reporting - and unfortunately some of those news outlets used deceptive headlines as I already pointed out. I even saw one headline that implied that Windows is more reliable than OS X and Ubuntu when in reality the data has nearly nothing to do with OS reliability.
The bottom line is that Pingdom report didn't act responsibly and as a result they became the source of misinformation - much to the detriment of innocent computer users. Various news outlets were also willing accomplices.
