The results are in. Coverity performed bug testing on 30 of the most popular Open Source programs, and have compiled their results. This study was done for the Department of Homeland Security.
The results? The most bugs per 1,000 lines of code was in AMANDA (which is a method of backing up your network to a tape drive). It had 108 bugs in 88,950 lines of code (which translates to about 1.214 bugs per 1,000 lines of code). The cleanest program is XMMS, which had 6 bugs in 116,899 lines of code (which translates to about .51 bugs per 1,000 lines).
Even better news is, AMANDA fixed all 86 bugs within the last couple of weeks. So, now they are the cleanest program with 0 bugs for 88,950 lines of code.
Two points to quote from the article.
“Generally speaking, it is difficult to determine how well these open-source programs compare with their proprietary counterparts, Chelf said. Coverity has tested only a few commercial products, so direct comparisons cannot be made. ” This is directed to the people who will use this to say that Open Source is most definitely more error-free then commercial programs.
The second quote is ” Quality is more accurately reflected by the ratio of developers to the size of the code base and by the number of users who use the software (and provide feedback). ” This is actually a point that commercial software creators should consider. The more eyes that are looking at the code, the better your chances of finding and fixing the bugs.