I saw this article summarized on Slashdot this morning. Headline: Microsoft Claims ‘Vista Has Fewer Flaws Than Other First-Year OSes.’ According to the article, Vista released 17 security bulletins and fixed 36 vulnerabilities in the first year, which is a big improvement over XP’s hit counts of 30 and 65 respectively. The article continues by comparing these figures to the first-year vulnerability numbers for Red Hat (360), Ubuntu (224) and Mac OS X (116). One security specialist is quoted, stating that these numbers “prove that [Vista] is quantitatively more secure.” He then chastises other OS vendors for their negligence in QA and security testing.
When you read this article, does it make you doubt the security of Linux and OS X? That’s the intended message, no?
I find this article misleading for a few reasons:
- Vista numbers are based on actions Microsoft decided to take: release bulletins and fix vulnerabilities. There is an element of subjectivity here.
- The article discloses that 30 Vista vulnerabilities remain, which brings the number of known vulnerabilities to 66. Sometimes, ‘known’ translates into ‘disclosed’.
- An assumption is made that the writing of Vista and the compilation of a Linux distro are comparable activities. They are not, and the vulnerabilities associated with these activities likely have different statistical distributions.
- The scope and risk of the vulnerabilities fixed are not discussed. Microsoft may have fixed 17 big problems, where as Ubuntu fixed 224 small ones. For all we know, the cumulative effect could be equal.
The security specialist states that Vista underwent more testing than the other OSes. He makes no reference to the relative quality of that testing. “More” doesn’t mean “better.” And what does “more” mean anyway? Was the testing measured in dollars spent? Number of testers? Number of test cases?
An analysis of the actual report would probably provide more clarity. Moreover, independent verification of the numbers would boost the integrity of the conclusions drawn. The validity of the actual results, however, does not change the intent of those who report on them. FUD, I say!