Common Weakness Enumeration

CWE Glossary Definition

CWE Mapping Analysis
CWE Mapping Analysis

Document version: 1.0    Date: September 9, 2008

This is a draft document. It is intended to support maintenance of CWE, and to educate and solicit feedback from a specific technical audience. This document does not reflect any official position of the MITRE Corporation or its sponsors. Copyright © 2008, The MITRE Corporation. All rights reserved. Permission is granted to redistribute this document if this paragraph is not removed. This document is subject to change without notice.

Author: Mark Loveless
URL: http://cwe.mitre.org/documents/mapping_analysis/index.html

As CWE evolves and grows, the CWE Team felt it was very important to understand how tools and different types of knowledge repositories can use CWE's content, and how the usage would be affected by the amount and quality of coverage within CWE. To attempt to measure and understand coverage, an analysis was performed for CWE mappings from the weakness repositories of three sources - two commercial analysis tools and a secure programming reference. This analysis of CWE's content structure should help determine whether the changes incorporated into Version 1.0 of CWE are sufficient to support accurate and clean mappings of the content in external repositories of security weakness information.

If CWE entries are too vague or general – or missing entirely – this suggests areas for improvement. If the same overall issues or entries tend to correlate, this suggests either a need for further clarity and depth in CWE, or possibly too much depth. The goal of this exercise was to improve the overall quality of CWE, as well as provide a point of reference for individuals and organizations wishing to map their content to CWE. This point of reference will hopefully provide some level of guidance and advice in the mapping process. It will also provide a beginning vocabulary for issues the CWE Team has dealt with during the CWE creation and mapping process. This same vocabulary could be used by others to give detailed feedback.

A total sample of 573 external items was examined from the 3 repositories for a cross-check of CWE 1.0's coverage.

While examining each individual item and its mapping, we asked the following questions:

• Did the choice of CWE entry mapping make sense? Was it accurate?
• If an issue was not mapped to a CWE entry, was an accurate mapping possible?
• Were there any entries missing from CWE, or CWE entries that needed to be corrected?
• If an issue was mapped to a CWE entry that was deemed incorrect, what can we do to correct it? Is the CWE entry too vague, too general, or just not quite a good fit?

While we used the vendor-supplied CWE mappings as guidance, we performed our evaluations in light of what the most appropriate mapping was. In some cases, the vendor would map to the wrong CWE entry due to a typo (although this was rare), a misunderstanding of the entry, or due to outdated mappings (i.e. a new entry was created after the original mapping was performed).

In the following table, the mappings of the 573 individual items were separated into 10 different categories. The names of the categories are self-explanatory for the most part, but they will be discussed individually below.

It should be noted that while the three external resources had different goals and coverage, the appropriateness or "fit" of each individual mapping could be placed into one of these categories.

While the individual categories are listed in the above table in alphabetical order, they will be discussed with the most relevant categories listed first.

EXACT – Means there was an exact match with a CWE entry. By far, this was the most common type of fit we found during this exercise. There were some cases where the repository had an incorrect or missing mapping, but since there was a CWE entry that was conceptually the same as the issue in the repository, the fit was listed as exact. When the repository specified an incorrect CWE, we tried to diagnose the cause of the incorrect mapping, but it generally fell into one of the other fitness categories.

BAD ENTRY – Means there is an appropriate CWE entry, but a quality problem with the entry itself prevented the original mapper from using it. During the mapping process, the CWE entry seemed to fit; however, vagueness, inconsistencies, or over-generalization within the CWE entry could cause confusion or lead to incorrect mappings in the future. During this exercise, entries like this were flagged for post CWE 1.0 editing and revision. A good example of this is CWE-391, which covers several distinct issues relating to improper error handling. In other cases, a vendor mapping was made to two CWE entries in an attempt to cover what they perceived as one issue. This was often treated as an indicator that there needed to be a more specific entry that covered the issue, as opposed to trying to map to multiple entries.

MISSING – Means the issue did not have a matching CWE, but it is the kind of issue that would be covered by CWE. This identifies gaps within CWE itself. Either there was no existing entry, or the entry was "immature" and required expansion to fully cover the issue. For example, there were multiple issues related to insufficient logging, which is not well-covered in CWE 1.0 except under the Protection Mechanism Failure pillar (CWE-693), but logging is clearly security-relevant.

ABSTRACTION – Means that the issue being mapped is either more specific or more abstract than the CWE entry to which it corresponds. For example, one repository covered three SSL-specific exceptions in Java. While a mapping to CWE-391 Unchecked Error Condition makes sense, that entry is written in a more general fashion; however, the vendor is checking for a specific variant related to specific errors implemented under Java. We did not distinguish between cases when the repository item was more abstract or more specific than the closest CWE, but in general, it was more specific.

INCLUSION – Means the repository issue does not have a CWE mapping, and it could be argued that CWE should not cover that type of issue. These are related to issues with non-security impacts (code quality, environment and build issues, configuration issues) and ones with security impacts where the issue referenced is more of a consequence of a weakness instead of a root cause, such as a generic item labeled "denial of service." Sometimes, a repository item is intended to communicate tool information back to the consumer, e.g. if a library file could not be found, or if a scanning module was disabled; this was treated as an inclusion issue. Other subtler issues would arise, however. While CWE has covered some code quality issues with direct links to more serious weaknesses, some of the repository items seemed like a less obvious choice for inclusion. Consider the assignment of a variable in which the value is never read. One cause of this might be when a value is assigned to a variable, but the variable is never used. Another cause might be when a variable is initialized to a value, followed by an explicit assignment using the same value. There is no actual security impact, however many resources – particularly software analysis tool vendors – will cover general code quality issues in addition to explicitly security-related issues, since there is often a close association between the two. It is an open question as to how CWE should cover such issues, if at all.

PERSPECTIVE – Means the repository item has multiple possible CWE matches that could vary depending on the perspective of the mapper, i.e. which mapping made the most sense to them. Perspective issues depend on what perspective one has as to which CWE entry mapping makes the most sense to them. Different mapping choices could have been made, and still seem correct depending on the view of the person doing the mapping. This area generated more discussion and changing (and rechanging) for the appropriate CWE mapping than any other group.

Some issues fall into seemingly different sections of CWE. For example, a Double Unlock on a software component could fall under CWE-404 Improper Resource Shutdown or Release ("The program fails to release – or incorrectly releases – a system resource before it is made available for re-use."), CWE-413 Insufficient Resource Locking ("A product does not sufficiently lock resources"), or CWE-675 Duplicate Operations on Resource ("The product performs the same operation on a resource two or more times, when the operation should only be applied once."). Each mapping is defensible, depending on the perspective of the individual reviewing the information at hand. We view this type of perspective issue as an active research topic for CWE.

A final example would be a case where a resource's mapping is to one element of a chain when another element of the same chain could be appropriate. For example, the vendor tool might be covering cases in which the code fails to check for a NULL return value, which later leads to a NULL pointer dereference. The tool might map to CWE-252 Unchecked Return Value or to its consequence, CWE-476 NULL Pointer Dereference.

IMPRECISE – Means the resource had a clearly defined issue, but the best matching CWE entry is not an exact correspondence, often only a partial match. Whether the resource mapped to a CWE or not, it was determined that there was not a precise fit. The issue may cover multiple distinct concepts, or the issue may map to one element in a chain. An example of an imprecise mapping occurred when mapping an issue related to signed comparison. The issue involved using a signed comparison to check a value that is later treated as unsigned, causing the program to read data from outside the bounds of allocated memory. This issue was mapped to CWE-126 Buffer Overread ("The software reads data past the end of the intended buffer") which is close, but the issue specifically involves the signed comparison – the "root cause" link in a chain that leads to CWE-126. CWE 1.0 has entries related to signed/unsigned conversion, but not signed comparison.

NEEDS DETAILS – Means the item did not provide enough details to assess the CWE mapping (if any). Some items were either extremely complex in nature, or contained vague references to important aspects of security. To fully understand the issues would require further consultation with the repository owner to determine the original intent of the item.

CONFUSING – Means the external repository item has details, but it is not written clearly, or there is a CWE mapping that suggests a different interpretation than the description implies. Like the NEEDS DETAILS category, these require input from the repository owner to determine whether the mappings are correct, or if new entries are needed or existing entries will suffice.

OTHER – This was a "catch all" category for those issues whose mapping fit could not be categorized. There were two main reasons items ended up in the OTHER category. The first one involved issues that had multiple mapping fits, which will ultimately require further research as to how to approach them. The second group was rather specific, in that it involved issues that had multiple possible CWE mappings, depending on whether the issue was introduced during implementation/design or during configuration.

When mapping issues to CWE entries, things are not always as clear-cut as they might appear. Extra analysis into the mappings by content creators can lead to improved and more accurate CWE mappings. If those doing the mapping analysis provide feedback to the CWE Team or ask questions, there can be major improvements in CWE itself, in the form of clarified entries or new entries entirely. The model we have outlined for this mapping analysis was originally designed as a quality control exercise for CWE; however, we felt that others could benefit from the questions we asked and how we categorized the mapping fit.

The main concern is resolution – how do we reconcile issues that are not an exact match? Resolution will come in the form of CWE entry alteration, external repository content alteration, or a combination of the two. Other alternatives might include allowing multiple node mappings for a single issue, although this is not always desired. But this analysis does provide a common ground for discussion of potential resolution points and should help all parties involved in understanding the complexity of mapping issues to CWE.