Common Weakness Enumeration

CWE Glossary Definition

2010 CWE/SANS Top 25 - Final Draft Changes and Discussion

Date: December 13, 2010

• Published 1.07 of Top 25 - Updated details to be consistent with the release of CWE 1.11, primarily with mitigations and some name changes.

Date: September 27, 2010

• Published 1.06 of Top 25 - Updated details to be consistent with the release of CWE 1.10, primarily with mitigations.

Date: June 29, 2010

• Published 1.05 of Top 25 - Modified official name of the list, replacing "Programming" with "Software"

Date: June 21, 2010

• Published 1.04 of Top 25 - Updated details to be consistent with the release of CWE 1.9, primarily with mitigations.

Date: April 5, 2010

• Published 1.03 of Top 25 - Updated details to be consistent with the release of CWE 1.8.1, primarily with improved mitigations and additional CAPEC mappings.
• Removed duplicate entry in Focus Profile summary

Date: February 25, 2010

• Top 25 document, version 1.02
• Fixed "Bypass security" impact in CWE-306; added pointers to SANS Application Security blog.

Date: February 17, 2010

• Top 25 document, version 1.01
• Added links to Microsoft SDL
• Added page numbers to PDF

Date: February 17, 2010

• Initial public version of 2010 Top 25 document (1.0)

Date: February 13, 2010

CWE Top 25 - Voting Results

All,

As mentioned before, a number of Top 25 voters did not like the restriction of 4 Critical votes for importance, and 4 Widespread votes for prevalence.

Even if this restriction did not exist, it appears that the impact on the Top 25 would have been minimal.

A number of voters initially supplied me with ballots that exceeded these restrictions. I took these ballots, combined them with other voters' ballots that satisfied the restrictions, and generated the voting results to see what happened.

The comparison is attached. Only one item would have fallen off the Top 25. The top 5 items did not even change positions, and neither did the bottom 5 of the original 41. Only about 25% of the entries rose or fell more than 2 places.

Since about half of the voters supplied ballots after reminders about the restrictions, the data set for unrestricted voting is incomplete. So, I don't think it's appropriate to make a separate profile using this data set just to provide alternate rankings. So I've decided to drop it. However, I don't think this is a big loss, since the comparison suggests there was a good chance that the difference wouldn't be that significant anyway. I will chalk this up to "lessons learned" to consider for next year, and document it accordingly in the methodology portion of the document.

- Steve

Date: February 3, 2010

CWE Top 25 - Summary of Activities

All,

Here is a summary of recent activities and what's coming ahead.

1. As a reminder, the public release of the Top 25 has been postponed until Tuesday, February 16.
2. Voting for the Nominee List is closed. We received ballots from 28 different organizations! We're thrilled with the response.
   Within a couple of days, I will contact each voter and send their ballots back to them for review, with their own score and ranking, along with the final scores/rankings.
3. I have decided on which metric will be used for the general list. Basically, the rationale is to keep it simple. For each vote for a CWE, a sub-score will be assigned based on:
   (Importance X Prevalence)
   where each factor gets a rating from 1 to 4, i.e., no "extra credit" for the forced limits on Widespread and Critical ratings. The final score is a summation of individual ratings per CWE.
   Betsy Nichols is evaluating the quality of the data (e.g. distributions and factor independence) but at this point, we have what we have.
4. Many voters wanted to use more than 4 Widespread or Critical ratings, so we will account for this in a separate "focus profile" (formerly referred to as a "technology profile"). I will reuse the original ballots that I received from people; if your first ballot followed the restriction, feel free to re-submit an updated ballot that assigns as many "Widespread" and "Critical" ratings as you feel is appropriate.
5. The following Focus Profiles are likely to make it into the final version. These will be posted to the discussion list, and we'd love to have active participation and review for any of these.
   • Unrestricted Widespread/Critical ratings: previously discussed.
   • Educational Emphasis: which weaknesses should be emphasized in an educational setting; uses alternate metrics and 3 factors.
   • Automated vs. Manual Analysis: which weaknesses are detected by automated techniques vs. manual.
   • Language-independent and Language-specific issues: may contain a small breakdown of each item by its prevalence in major languages.
   • Priorization by Importance: an alternate view of the voting data in which the metric emphasizes Importance, intended for software consumers or those just getting started in security.
   • Design vs. Implementation: checklist view of which weaknesses are fixed in design, implementation, or both.

   Other considerations include:

   • Established developer ratings: using ONLY submitted votes from software vendors who participated, create a separate ranking - showing what vendors with established secure development practices care about.
   • Web-based and Unmanaged code may be subsumed by the language-focused/language-independent profiles.
6. I plan to post the first full draft of the Top 25 document on Thursday evening, February 4. Individual focus profiles will also be posted for separate review.

For those who can participate, we will be looking for people to review the draft document (any section you wish), contribute to individual focus profiles, or evaluate the detailed writeups of the individual entries.

Thanks again to everybody who's been helping us out!

- Steve and Bob

Date: January 26, 2010

CWE Top 25 - Progress Update and Upcoming Activities

All,

Here is an update on the CWE Top 25.

1. We have received at least 15 voting ballots for the Nominee List, and we may get 20 or more. This was more than we could hope for, so thank you for all your efforts!
   The voting deadline has been extended two more days, to Thursday, January 26 - a week before the publication of the official Top 25.
2. Many voters did not like the restriction in which Critical importance and Widespread prevalence were only allowed a maximum of four votes each. This will be accounted for in the scoring metric by giving "extra credit" to these ratings, and it will be a lesson learned for next year.
3. Now that the data collection for voting is almost complete, it is time to discuss some scoring methods to use. Shortly after this email, I will propose various metrics and discuss the pros and cons. I will be working with Betsy Nichols in the next couple days to analyze the utility of the proposed metrics and whether they pass the appropriate statistical smell tests.
   A final decision on the metric to use will be made on Friday. This will then be used to rank, and most likely select, the official entries for the General List.
4. The various comments will be more closely reviewed and synthesized into a summary of the final general list. One of the most common themes is that many of the assessments of importance varied heavily on context, such as the type of application or environment. My hope is that some of this context will be elucidated in tech profiles. At the very least, they become important inputs for MITRE's work on a Common Weakness Scoring System (CWSS).
5. I have been working on the technology profiles concept and have a general sense of which profiles would be nice to include in the first public release. Some of these ideas will be proposed in the next day. I think the approach will be to recruit contributors to individual tech profiles, who may review the results and offer any commentary. Pascal Meunier has spent a good deal of effort in formulating an educational profile, complete with an alternate quantitative ranking with three different factors.
6. The original draft for the most effective mitigations has been improved, though it needs a few more edits. I will have a proposal within a couple days.
7. We are beginning to solicit supporting quotes, as well as the experiences from anybody who used the 2009 Top 25 list and saw benefit from it. If you're interested, contact Bob Martin (ramartin@mitre.org) and Steve Christey (coley@mitre.org).
8. Review of mitigations for individual CWE entries, as well as the associated commentary, will begin no later than Monday, February 1.
9. The publication of the Top 25 will be Thursday, February 4.

Thanks again to everyone for their continued support of this effort!

Steve and Bob

Date: January 11, 2010

Scoring and Ranking

All,

I spoke with Elizabeth Nichols about different ways to quantitatively rank or score weaknesses on the Top 25. While there were a number of ideas covered, most of them would be too time-consuming, subject to error because of small sample size, etc. Next year, I hope to take some more time to explore some of the ideas.

However, for this year, I believe we have hit upon a basic approach that will have some flexibility and provide additional lessons learned to help us improve techniques in the future.

1. The Keep/Remove/Add votes that I've received so far have been used to create a draft list of weaknesses. Additional opinions will be solicited for the next few days.
2. A "Nominee List" will be gathered from the recommendations, adjusting for abstraction and general feedback.
3. A polling period will take place for a week. Each contributor will be polled to assess each weakness based on the criteria we select - such as prevalence and severity - using a selection of fixed, discrete values (e.g., High, Medium, Low). The use of discrete values will simplify a number of problems I'm facing because of the wide variety of prevalence data I'm using.
4. We can then devise scoring functions with appropriate weights, then use the resulting scores to rank the entries. I intend to propose some type of scoring function, and Elizabeth will do some data-munging magic to see if she can infer appropriate weights from the submitted data itself. This would be an interesting exercise in and of itself, but if we both come up with roughly-equivalent answers, then it will be some kind of validation.

These items are included in the updated schedule that I just sent.

- Steve

Proposal: Use "Importance" instead of "Severity"

For scoring and ranking items on the master list, I suggest that we use these two factors:

• Prevalence
• Importance

Given the multitude of threat profiles and perspectives, and some of the discussion on this list, using a factor of "Severity" poses some difficulties. First, as evidenced by the CWE-209 discussion, the perception of severity can vary widely. Several people have said that even though it doesn't cause that much damage by itself, it is so important for defense-in-depth that it should still be included on the list. Second, alternate perspectives such as educational priority may not consider severity to be the most important factor. There is also the desire to quantify results and come up with appropriate rankings.

I propose that we use a new factor, instead of severity: "importance."

This will allow individuals to vote on how important a weakness is, using their own criteria that reflects their daily work. Top 25 contributors can effectively encode what is most important to them - whether it's severity, frequency of exploit, need for education, etc. - in a way that is specific to their own needs (and their customers' needs).

Using some kind of mechanism that combines Prevalence and Importance (to be determined), we can then assign individual scores to each CWE entry, then rank the Top 25 entries according to those scores.

The result will be a generalized recommendation that merges diverse perspectives to come up with something approaching a consensus. Each individual technology profile could still define its own criteria, scores, and ranking.

Draft 1 - Status Summary (January 11, 2010)

Draft 1 - Status Notes

There are 17 entries with this status.

There are 3 entries with this status.

There are 1 entries with this status.

There are 4 entries with this status.

There are 9 entries with this status.

There are 5 entries with this status.

There are 4 entries with this status.

There are 13 entries with this status.

Date: January 5, 2010

Here is a high-level summary of the discussions and feedback that we have received so far. I apologize for over-simplifying the extensive feedback we have already received.

• Technology Profiles
• Structure of Top 25 Document
• Threat Models
• Overlap / Abstraction Issues Between Entries
• Prioritization and Ranking
• Section on the Most Effective Mitigations
• Suggested Additions / Removals

Technology Profiles

The opinions on the use of technology profiles were largely positive.

Several new profiles were recommended - on the list and privately - including:

• education
• client-side software (and server-side software)
• managed code
• mobile devices

There were some concerns about the originally-proposed profiles: web-based, unmanaged, and embedded. There is overlap between these profiles; for example, embedded code might provide web-based functionality that is written in unmanaged code.

It is not clear whether this type of overlap is problematic.

Possible process:

• identify a small number of important technology profiles to be ready for release in the final document (early February).
• each profile defines and uses its own criteria for prioritizing items.
• other profiles can be added to the Top 25, or published separately, well after initial release.

Next steps:

• MITRE updates the example document to reflect the use of these profiles.
• MITRE and Top 25 contributors determine whether profile overlap is acceptable, and if so, what actions (if any) need to be taken to minimize or clarify the overlap
• Top 25 contributors may request (or be requested by MITRE) to further concentrate on refining one or more of these profiles, including the prioritization criteria. (Later sections in this summary will cover some specifics.)

Structure of Top 25 Document

Assuming the use of technology profiles, there was no support for assigning 25 items to each profile, then combining them to create a master list that would most likely consist of more than 25 items.

There has been solid support for building a single master list - 25 items - then providing separate appendices for each technology profile. Each profile could use a profile-specific ranking of items from the master list, and a supplementary group of on-the-cusp (watch list) entries could be used to identify issues that are critical for that technology profile, even if they are not important enough to make the master list.

Next steps:

• MITRE will post an updated example document that reflects this structure.

Threat Models

The 2009 Top 25 assumed a simple threat model: a skilled and determined attacker for whom integrity and confidentiality was more important than denial of service.

Since the 2010 Top 25 will use multiple profiles, it may make more sense to allow each profile to identify its own threat model - assuming that the profile's prioritization criteria want to consider a threat model. (For example, an educational profile might choose to prioritize weaknesses based on ease-of-understanding by students instead of considering the real-world threats posed by those weaknesses.)

Given the use of multiple profiles this year, and the fact that last year's threat model was not always applicable, the pursuit of more detailed threat models will probably be abandoned for this year, except possibly for the master list.

Next steps:

• TBD; partially covered by profile-specific actions in the previous section.

Overlap / Abstraction Issues Between Entries

2009 entries such as CWE-20 (improper input validation) had a significant amount of overlap with other items on the list, such as XSS and SQL injection. Some entries were at a very high level, such as improper authorization, while others were at a lower level. In some cases, the Top 25 would include both a low-level and its parent (such as the entries related to external control of filenames and critical data).

Entries such as these could be addressed by one or more of the following:

1. Using lower-level CWE entries (possibly creating them)
2. Resolving / normalizing cases when both an entry and its parent/grandparent were included in the 2009 list.
3. De-prioritizing the entries related to "root cause," possibly changing them to a cusp/watch-list role.
4. Moving some entries, such as CWE-20, to a separate section that identifies the most important mitigations.

Next steps:

• MITRE will suggest abstraction changes and build a straw-man

Prioritization and Ranking

While the topic was not raised explicitly, there seems to be support for prioritizing the items on this year's list.

There were not any objections to continuing last year's emphasis on prevalence of occurrence and severity to determing inclusion on the Top 25. Some people advocated using a third factor that captures the frequency of real-world attacks that exploit the issues.

There has also been solid support for using quantitative methods to rank items, which also would ultimately determine which items make it onto the Top 25 list.

MITRE has received several submissions, privately and to the discussion list, which contain real-world weakness or vulnerability data that might be difficult to combine. For example, some submissions use low-level CWEs, and others use high-level ones. CVE vulnerability data is also available; a summary of 2008/2009 results was posted. However, it has been noted that CVE is subject to unpredictable fluctuations, such as the spike in symbolic link issues in 2008 due to a single researcher's efforts.

As mentioned previously, there is a possibility that each technology profile might use its own prioritization scheme.

Next steps:

• During the first week of January, MITRE will work with Elizabeth Nichols of PlexLogic to determine if there are suitable approaches for data collection or polling that will support quantitative analysis.
• MITRE will investigate the use of attack frequency as another factor in ranking.
• Supporters of individual profiles can begin discussion of criteria for profile-specific ranking.

Section on the Most Effective Mitigations

There was solid support for creating a separate section that covers important mitigations - or "safe practices" - that could eliminate or reduce the severity of many items on the Top 25. Such mitigations might include running with low privileges, input validation, using well-vetted security features instead of inventing new ones, etc.

One benefit of this approach would be to reduce the apparent overlap of many issues on the 2009 Top 25.

This is being informally referred to as "Top 10 Mitigations," but there is not a clear need to limit (or expand) the list to 10 items.

Next steps:

• MITRE will identify entries from the 2009 Top 25 that might make more sense to relocate to the mitigation section
• MITRE will create a sample section containing sample mitigations, and solicit input on those suggestions.
• Top 25 contributors can nominate the most effective mitigations

Suggested Additions / Removals

Contributors were given last year's 2009 list, and asked whether the items should be kept or removed. Contributors were also given a list of "on the cusp" from last year, as well as other recommendations, then asked whether those entries should be added.

The most controversial item has been CWE-209 (error message information leak), which was on the 2009 Top 25. Supporters state that even if it is not severe on its own, it is often an important part of real-world attacks, so for that reason it should be considered. Dissenters acknowledge the role in attacks but believe that the Top 25 should have a greater emphasis on the critical weaknesses for which CWE-209 is an intermediate step.

There were some suggestions for error handling and/or error checking, for which several entries were nominated.

As the discussion of addition/removal evolved, it became apparent that the 2009 Top 25 does not appropriately clarify what its primary purpose is, which would then affect which items should be added or removed.

Next steps:

• MITRE will process and summarize the specific feedback that has already been received
• MITRE and Top 25 contributors will clarify the purpose of the Top 25 (e.g., a general educational/awareness tool, or a utility to help focus the development of software assurance requirements).
• Top 25 contributors will continue to offer their feedback/suggestions for more additions and removals