Common Weakness Enumeration

The total number of publicly reported web application vulnerabilities has risen sharply, to the point where they have overtaken buffer overflows. This is probably due to ease of detection and exploitation of web vulnerabilities, combined with the proliferation of low-grade software applications written by inexperienced developers. In 2005 and 2006, cross-site scripting (XSS) was number 1, and SQL injection was number 2.

PHP remote file inclusion (RFI) skyrocketed to number 3 in 2006, almost a 1000% increase over the previous year. Because RFI allows arbitrary code execution on a vulnerable server, this is a worrisome trend, although proper configuration is frequently sufficient to eliminate it. This trend is likely a reflection of RFI's role in creating botnets using web servers [Evron].

Buffer overflows are still the number 1 issue as reported in operating system (OS) vendor advisories. XSS is still high in this category, at number 3 in both 2005 and 2006, although other web application vulnerabilities appear much less frequently.

Integer overflows, barely in the top 10 overall in the past few years, are number 2 for OS vendor advisories, behind buffer overflows. This might indicate expert researcher interest in high-profile software.

There are noticeable differences in the types of vulnerabilities being reported in open and closed source OS vendor advisories. These merit further investigation because they might reflect important differences in development, research, and disclosure practices.

The data is inconclusive regarding whether there is a concrete improvement in overall software security. While there is a rise in new vulnerability classes, and increasing diversity of vulnerability types, the raw numbers for older classes have not changed significantly. Further investigation is also required in this area.

The 2006 statistics cover the entire year.

An important statistical gap with CSRF is reported; see Table 1 analysis.

PHP remote file inclusion (RFI) is #4 overall, not #5, and is much closer to SQL injection in 2006 than originally reported. RFI's linkage to web server botnets is mentioned in the 'Summary of Results' and 'Overall Trends' sections.

The complete report has minor statistical discrepancies with the October report regarding total numbers of CVEs, due to (a) incidental additions of IDs for older issues occurring in late 2006, or (b) removal of some IDs because they were duplicates or later proven to be false reports. Both occurrences are relatively common for any vulnerability information repository that seeks to maintain historical accuracy.

Unsafe storage under the web document root (webroot) is number 10 for all of 2006, not 13.

The most basic data manipulations for these vulnerabilities are very simple to perform, e.g., ''' for SQL injection and '<script>alert('hi')</script>' for XSS. This makes it easy for beginning researchers to quickly test large amounts of software.

There is a plethora of freely available web applications. Much of the code is alpha or beta, written by inexperienced programmers with easy-to-learn languages such as PHP, and distributed on high-traffic sites. The applications might have a small or non-existent user base. Such software is often rife with easy-to-find vulnerabilities, and it is often a target for beginning researchers. The large number of these 'fish-in-a-barrel' applications is probably a major contributor to the overall trends.

With XSS, every input has the potential to be an attack vector, which does not occur with other vulnerability types. This leaves more opportunity for a single mistake to occur in a program that otherwise protects against XSS. SQL injection also has many potential attack vectors.

Despite popular opinion that XSS is easily prevented, it has many subtleties and variants. Even solid applications can have flaws in them; consider non-standard browser behaviors that try to 'fix' malformed HTML, which might slip by a filter that uses regular expressions. Finally, until early 2006, the PHP interpreter had a vulnerability in which it did not quote error messages, but many researchers only reported the surface-level 'resultant' XSS instead of figuring out whether there was a different 'primary' vulnerability that led to the error.

There is some evidence that over the past couple of years, web defacers have taken an interest in performing and publishing their own research. This is probably due to the ease of finding vulnerabilities, combined with the presence of high-risk problems such as PHP file inclusion, which can be used to remotely install powerful, easily-available backdoor code. Based on customer posts to numerous vendor forums, there is solid evidence that remote file inclusion is regularly used to compromise web servers, which also helps to explain its popularity.

PHP remote file inclusion skyrocketed in 2006, nearly 1000% over the previous year. This is most likely a reflection of RFI's role in creating botnets using web servers [Evron].

For 2006, the top 5 vulnerability types are responsible for 57% of all CVEs. With over 35 vulnerability types used in this report, and dozens more as currently identified in CWE, this shows how most public reports concentrate only on a handful of vulnerability types.

Cross-Site Request Forgery (CSRF) remains a 'sleeping giant' [Grossman]. CSRF appears very rarely in CVE, less than 0.1% in 2006, but its true prevalence is probably far greater than this. This is in stark contrast to the results found by web application security experts including Jeremiah Grossman, RSnake, Andrew van der Stock, and Jeff Williams. These researchers regularly find CSRF during contract work, noting that it is currently not easy to detect automatically. The dearth of CSRF in CVE suggests that non-contract researchers are simply not investigating this issue. If (and when) researchers begin to focus on this issue, there will likely be a significant increase in CSRF reports.

Over the years, there has been a noticeable decline in shell metacharacters, symbolic link following, and directory traversal. It is unclear whether software is actually improving with respect to these problems, or if they are not investigated as frequently.

Information leaks appear regularly. There are 2 main reasons for the prominence: 'information leak' is a more general class than others (see CWE for more precise sub-categories), and when an error message includes a full path, that is usually categorized as an information leak, although it might be resultant from a separate primary vulnerability.

The inability to handle malformed inputs (dos-malform), which usually leads to a crash or hang, is also a general class. Malformed-input vulnerabilities have not been studied as closely as injection vulnerabilities, at least with respect to identifying the root cause of the problem. Also, many reports do not specify how an input is malformed. There are likely many cases in which a researcher accidentally triggers a more serious vulnerability but does not perform sufficient diagnosis to determine the primary issue. Finally, vendor reports might only identify an issue as being related to 'malformed input,' which obscures the primary cause.

As the percentage of buffer overflows has declined, there has been an increase in related vulnerability types, including integer overflows (int-overflow), signedness errors, and double frees (double-free). These are still very low-percentage, probably due to their relative newness and difficulty of detection compared to classic overflows. In addition, these newly emerging vulnerability types might be labeled as buffer overflows, since they often lead to buffer overflows, and the 'buffer overflow' term is used interchangeably for attack, cause, and effect.

Other interesting web application vulnerabilities are webroot (storage of sensitive files under the web document root), form-field (web parameter tampering), upload of files with executable extensions (e.g., file.php.gif), eval injection, and Cross-Site Request Forgery (CSRF).

Integer overflows are heavily represented in OS vendor advisories, rising to number 2 in 2006, even though they represent less than 5% of vulnerabilities overall. This probably reflects growing interest by expert researchers in finding integer overflows, along with the tendency of expert researchers to evaluate widely deployed software. The affected software ranges widely, including the kernel, cryptographic modules, and multimedia file processors such as image viewers and music players. After 2004, many of the reported issues occur in libraries or common DLLs.

Buffer overflows are still #1. This is probably due to under-representation of web applications in OS advisories, relative to other CVEs. In addition, as related issues like integer overflows increase, they might be detected or reported as buffer overflows, since buffer overflows are frequently resultant from integer overflows.

XSS is still very common, even in OS advisories, and it appears with nearly the same frequency as integer overflows in 2006. An informal analysis shows that the affected software includes web servers, web browsers, email clients, administrative interfaces, and Wiki/CMS.

With the exception of XSS, there is a wide gulf between web-related vulnerabilities in OS advisories and other issues. SQL injection is at number 7 for OS advisories, and PHP remote file inclusion is practically nonexistent. Many other web-related vulnerabilities occupy the bottom of the chart. For SQL injection, it is possible that most OS-supported applications do not use databases, or aren't web accessible. SQL injection vulnerabilities are not web-specific, but it seems that they are rarely reported for non-web applications, so it is possible that this reflects some researcher bias.

Directory traversal and format string vulnerabilities are frequently reported at a higher rate in OS vendor advisories than elsewhere. The reason is unclear, because these vulnerabilities are not restricted to local attack vectors, so one might expect that they would also appear regularly in web applications. However, it is likely that researchers do not focus on format strings because they are rarely exploitable for code execution in languages other than C. In the case of PHP, many PHP functions are subject to both remote file inclusion and directory traversal, and it might be that only the file inclusion is publicly reported. (In fact, the overlap is so close that this sometimes causes difficulties with classification).

In 2006 so far, more than a quarter of the OS vendor advisories did not have sufficient details to actually classify the vulnerability (type 'unk'), at 26.8%. This is in sharp contrast to the non-OS issues, which comprise less than 8%. However, because of the data sets in question, the non-OS CVEs will include many non-coordinated disclosures that would, by their nature, provide more details. Table 4 demonstrates that it is not just closed source vendor advisories that omit sufficient details for vulnerability classification.

The 'top 5' and 'top 10' vulnerabilities in each year are a much smaller percentage of total vulnerabilities in OS vendor advisories than non-OS issues. For example, the 5 most common vulnerabilities in 2006 accounted for 30.2% of OS vendor issues, but 65.3% for non-OS. For OS issues, this suggests an increasing diversity in the kinds of vulnerabilities being reported, whereas for other issues, that diversity appears to be decreasing. This is also reflected in the 'other' category, in which OS vendors have a much larger percentage of 'other' issues in 2006 than non-OS. However, this could be another reflection of the domination of web application vulnerabilities.

The percentage of 'unknown' vulnerabilities - those that could not be classified due to lack of details - is significantly higher in closed source than open source advisories, at 43% in 2006, compared to only 8% for open source. With such a wide discrepancy, it is difficult to know whether any of the remaining results in this section are significant.

Buffer overflows are number 1 for both open and closed, with roughly the same percentage in each year, with the exception of 2004.

Symbolic link vulnerabilities appear at a higher rate in open source than closed source, although this might be due to the non-Unix OSes in the data set. While Windows has 'shortcuts' (.LNK) that are similar to Unix links, they appear very rarely in Microsoft advisories, or for Windows-based applications. It is not clear whether this is due to under-research or API/development differences. The authors recall that at least one researcher for a Linux distribution regularly investigated symbolic link issues in 2004 and 2005, so researcher bias might also be a factor.

Format string vulnerabilities appear more frequently in open source. There are probably several factors. First, susceptible API library calls such as printf() are easily found in source code using crude methods, whereas binary reverse engineering techniques are not conducted by many researchers (this might also be an explanation for symbolic link issues). Second, many format string problems seem to occur in rarely-triggered error conditions, which makes them more difficult to test with black box methods.

Perhaps most surprising: in 2006, the non-Unix closed source advisories barely covered any format strings at all. It is not clear why there would be such a radical difference.

Malformed-input vulnerabilities usually appeared more frequently in closed source advisories than open source, except for 2006. This historical tendency might be due to a lack of details in closed source advisories. If an advisory mentions a problem due to 'malformed data,' it might be assigned the dos-malform type. Another factor might be due to black box techniques. It seems likely that fuzzers and other tools would be used more frequently against closed source products than open source, but this is not known. A third factor might be modifications in CVE's data entry procedures, which eventually began to enter 'unknown' flaw types for vague terms such as 'memory corruption.'

XSS vulnerabilities appear more frequently in open source advisories than closed, but this might be a reflection of vendor release policies for advisories. It seems that open source vendors are more likely to release advisories for smaller packages.

Integer overflows have been roughly the same rank for open and closed source. This is a curious similarity, since one might not expect open and closed source analysis techniques to be equally capable in finding these problems.

Another interesting example is in the use of default or hard-coded passwords. Over the years, very few open source vendor advisories have mentioned default passwords, whereas they appear with some regularity in closed source advisories, even in the top 10 as recently as 2005. It is not clear whether this is a difference in shipping/configuration practices or vendor disclosure policies.

During the October 2006 analysis, it was discovered that shell metacharacter issues appear less frequently in non-Unix closed source than other closed source advisories. This result was verified using the latest data; it is not evident in Table 4. This could be due to usage patterns of API functions such as CreateProcess() for Windows, and system() for Unix. This result is being reported because it is the most concrete example of how API functions might play a role in implementation-level vulnerabilities.

The vulnerability types could be tied to other CVE-normalized data, such as IDS, incident databases, or vulnerability scanning results. This could determine the types of vulnerabilities that are being actively exploited or detected in real-world enterprises.

More precise classification could be informative. Approximately 15% of CVEs have vulnerability types that cannot be described using the current classification scheme. Another 10% are 'unknown' vulnerabilities whose disclosures do not have sufficient details to determine any vulnerability type, but this problem is unavoidable, since some vendors do not release these details.

A crude measure of researcher diversity might be possible by linking data to other vulnerability databases that record this information. This could be used to determine if the raw number of researchers is increasing (probably), how that rate is increasing relative to the number of vulnerabilities (unknown), and how many different bug types are found by the average researcher (probably fairly small). If such data is available, then a further breakdown could be performed based on professional researchers versus others.

More precise data sets could be identified, such as a cross-section of market leaders in various product categories, not just OS vendor advisories. CVE does not record this type of information.

Why aren't you giving out raw numbers for open vs. closed source?

Answer: we already said why. See paragraph 2 of the Table 4 analysis for a reminder, the one marked 'IMPORTANT.'

Why did you release the draft report in October, without waiting for complete 2006 data?

Answer: when MITRE mentioned the preliminary results at the Cyber Security Executive Summit on September 13, there was a lot more interest than we had originally anticipated. We hoped that follow-up discussion of the results might help us to provide a better report when 2006 was complete.

How does this compare with the other summaries you've posted in the past? Why have the numbers and percentages changed for older years?

Answer: (1) we occasionally add CVEs for older issues, (2) some of the previously released summaries were cumulative instead of offering a year-by-year breakdown, and (3) eventually, as a new type of vulnerability is reported more frequently, the CVE project notices it enough to give it a name, or at least a type. Once we do that, we can go back and update the older CVEs that also had the issue. However, we often rely on keyword searches in CVE descriptions for doing these kinds of updates. The earliest reports of new vulnerability types probably don't get captured fully, because CVE descriptions frequently vary in the early days or months of a new vulnerability type. Most updates to these vulnerability trends trigger an informal review of the 'other' vulnerabilities for the data set in order to update the type fields.

There are a lot more vulnerability types than what you've covered.

Answer: That's an observation, not a question. If a certain vulnerability type is not on the list, then it probably didn't appear frequently enough for the CVE project to track closely. There are several reasons: (1) the vulnerability type is selected from a large dropdown menu during CVE refinement, but also (2) our work in the Common Weakness Enumeration (CWE) is producing hundreds of vuln types, and we want that to become a little more stable before doing the next round of modifications to CVE data. Finally, (3) with approximately 4,000 vulnerabilities marked 'other' or 'not specified', it is cost-prohibitive to review each CVE when the set of categories is updated.

Why isn't my favorite web vulnerability here?

Answer: Many web vulnerabilities are difficult to classify because they are 'multi-factor,' i.e., they are composed of multiple bugs, weaknesses, and/or design limitations. Other web issues are really just specialized attacks that use other primary vulnerabilities. For example, most HTTP response splitting problems rely on CRLF injection, so they are classified under CRLF injection.

[Christey] 'Open Letter on the Interpretation of 'Vulnerability Statistics'', Steve Christey, Bugtraq, Full-Disclosure January 5, 2006, http://lists.grok.org.uk/pipermail/full-disclosure/2006-January/041028.html

[Evron] 'Web server botnets and hosting farms as attack platforms', Gadi Evron, Kfir Damari & Noam Rathaus, Virus Bulletin, February 2007

[Grossman] 'CSRF, the sleeping giant', Jeremiah Grossman, http://jeremiahgrossman.blogspot.com/2006/09/csrf-sleeping-giant.html