Name of Your Organization:
U.S. National Institute of Standards and Technology (NIST)
Web Site:
www.nist.gov
Compatible Capability:
Software Assurance Reference Dataset (SARD)
Capability home page:
http://samate.nist.gov/SARD/
General Capability Questions
Product Accessibility <CR_2.4>
Provide a short description of how and where
your capability is made available to your customers and the public (required):
NIST's Software Assurance Reference Dataset (SARD) provides a public repository of more than 61,000 test cases for software assurance tool assessment via Internet.
Mapping Questions
Map Currency Indication <CR_6.1>
Describe how and where your capability indicates the most recent CWE content used to create or update its mappings (required):
The section in the SARD designer's notes that discusses the weakness naming convention and CWE Compatibility states the version of CWE that the software's set of mappings is based on.
Map Currency Update Approach <CR_6.2>
Indicate how often you plan on updating the mappings to reflect the current CWE content and describe your approach to keeping reasonably current with the CWE content when mapping them to your repository (recommended):
When a test case is submitted to the SARD repository, its weakness(es) will be stored, according to the CWE mapping, as part of test case record. If the weakness(es) does not exist in the SARD CWE weaknesses table, the table will be updated accordingly.
When there is a new release of CWE, the CWE mapping will be reviewed and a relevant change will be made if it applies.
Documentation Questions
CWE AND COMPATIBILITY DOCUMENTATION <CR_5.1>
Provide a copy, or directions to its location, of where your documentation describes CWE and CWE compatibility for your customers (required):
CWE and CWE compatibility are described in a section of the SARD Designer's
Notes, titled "Naming Convention of Weakness – CWE Compatibility". It is
accessible via
http://samate.nist.gov/SARD/.
DOCUMENTATION OF FINDING ELEMENTS USING CWE IDENTIFIERS <CR_5.2>
Provide a copy, or directions to its location, of where your documentation describes the specific details of how your customers can use CWE identifiers to find the individual security elements within your capability's repository (required):
On the screen of searching test cases, click on the tab of Extended Search. Then, there are two ways presented to search test cases using CWE Identifiers:
On the right portion of the screen, expand the CWE weaknesses under "Weakness". A hierarchical tree of software weaknesses in CWE Identifiers that
SARD supports will display. Selecting one of the CWE Identifier will add that weakness as one of searching criteria. It will show at searching field
"Weakness" on the left portion of the screen. Fill in other searching fields on this screen, if applicable. Clicking on
"Search Test Cases" will return a list of test cases that meet the searching criteria.
Figure 1: Search Screen using JavaScript tree
If your browser does not support JavaScript tree, click on "Weakness" will not expand the hierarchical tree of CWE Identifiers. Move the mouse onto
"Click here if experiencing some trouble with the JavaScript trees" at the right-upper corner of the screen and click. Then, use the drop down menu of
"Weakness" field on the left portion of the screen. The menu has the complete list of CWE Identifiers that
SARD supports. Select the desired software weakness from that menu.
Figure 2: Search Screen using Pull down Menu
DOCUMENTATION OF FINDING CWE IDENTIFIERS USING ELEMENTS
<CR_5.3>
Provide a copy, or directions to its location, of where your documentation describes the process a user would follow to find the CWE identifiers associated with individual security elements within your capability's repository (required):
In SARD, the CWE-ID appears in every place where weakness needs to be posted, e.g., the screen display of a test case, in the list of test cases, the output zipped file of test cases, etc. The complete list of CWE weakness used in
SARD is provided on line.
Figure 3: List of Test Cases
Figure 4: Display of a Test Case
Type-Specific Capability Questions
Tool Questions
FINDING TASKS USING CWE IDENTIFIERS <CR_A.2.1>
Give detailed examples and explanations of how a user can locate tasks in the tool by looking for their associated CWE identifier (required):
On the screen of searching test cases, SARD provides two ways to locate test cases by looking for their associated CWE Identifier. If your browser supports JavaScript tree, expand the tree on the right side of the screen and select one of weaknesses. Fill in other search criteria and click on
"Search Test Cases". A list of test case with the specified CWE Identifier will display on the next screen.
Figure 5: Search Screen with Search Criteria Filled In
Figure 6: Search Result
If your browser does not support JavaScript tree, click on "Click here if experiencing some trouble with the JavaScript trees" at the right-upper corner of the screen. Then, use the drop down menu of
"Weakness" field on the left portion of the screen to select a CWE Identifier. Fill in other search criteria, if applicable, and click on
"Search Test Cases". A list of test case with the specified weakness will display on the next screen.
Figure 7: Search Screen with Search Criteria Filled In
Figure 8: Search Result
FINDING CWE IDENTIFIERS USING ELEMENTS IN REPORTS <CR_A.2.2>
Give detailed examples and explanations of how, for reports that identify individual security elements, the tool allows the user to determine the associated CWE identifier for the individual security elements in the report (required):
When a user requests downloading a test case, SARD will generate a zip file that contains the test file(s) along with a manifest file. The manifest file contains the location of the test file, the weakness of the test case, and the location of the weakness. The weakness is in the form of CWE Identifier.
Here is an example:
Figure 9: Example of Report
GETTING A LIST OF CLAIMED CWE IDENTIFIER COVERAGE <CR_A.2.3>
Give detailed examples and explanations of how a user can obtain a listing of all of the CWE identifiers that the owner claims the tool is effective at locating in software (required):
On the screen of "More downloads from the Samate Reference Dataset", there is a bullet item
"Weakness". Clicking on it, a listing of all of the CWE identifies that SARD supports will show up on next screen:
Figure 10: Listing of all CWE-IDs
USING CCR TO PROVIDE CLAIMED CWE IDENTIFIER COVERAGE <CR_A.2.4>
Give a detailed explanation of how a user can find the Coverage Claim Representation (CCR) XML document with all of the CWE Identifiers that the owner claims the tool is effective at locating in software (recommended):
Please see answer to <CR_A.2.3>.
Service Questions
FINDING TASKS USING CWE IDENTIFIERS <CR_A.3.1>
Give detailed examples and explanations of how a user can locate tasks in the tool by looking for their associated CWE identifier (required):
SARD provides various lists of CWE Identifier – in the form of JavaScript tree or Pull down menu. Users are able to search for a set of test cases by specifying a particular CWE Identifier in the list.
For detailed example, please see answer to <CR_A.2.1>.
FINDING CWE IDENTIFIERS USING ELEMENTS IN REPORTS <CR_A.3.2>
Give detailed examples and explanations of how, for reports that identify individual security elements, the tool allows the user to determine the associated CWE identifier for the individual security elements in the report (required):
SARD provides a facility of downloading test cases for generating service report. The download files consist of test file (one or multiple files of source code) and a manifest file (attributes of the test case). CWE Identifier is one of the test case attributes in the manifest file.
For detailed example, please see answer to <CR_A.2.2>.
GETTING A LIST OF CLAIMED CWE IDENTIFIER COVERAGE <CR_A.3.3>
Give detailed examples and explanations of how a user can obtain a listing of all of the CWE identifiers that the owner claims the tool is effective at locating in software (required):
Give detailed examples and explanations of how a user can obtain a
listing of all of the CWE Identifiers that the owner claims the service
is effective at locating in software (required):
Please see answer to <CR_A.2.3>.
GETTING A LIST OF CWE IDENTIFIERS ASSOCIATED WITH TASKS <CR_A.3.4>
Give a detailed explanation of how a user can find the Coverage Claim Representation (CCR) XML document with all of the CWE identifiers that the owner claims the service is effective at locating in software (recommended):
Please see answer to <CR_A.2.3>.
Online Capability Questions
FINDING ONLINE CAPABILITY TASKS USING CWE IDENTIFIERS <CR_A.4.1>
Give detailed examples and explanations of how a "find" or
"search" function is available to the user to locate tasks in the online capability by looking for their associated CWE identifier or through an online mapping that links each element of the capability with its associated CWE identifier(s) (required):
SARD provides various lists of CWE Identifier – in the form of JavaScript tree or Pull down menu. Users are able to search for a set of test cases by specifying a particular CWE Identifier from the list.
For detailed example, please see answer to <CR_A.2.1>.
FINDING CWE IDENTIFIERS USING ONLINE CAPABILITY ELEMENTS <CR_A.4.2>
Give detailed examples and explanations of how, for reports that identify individual security elements, the online capability allows the user to determine the associated CWE Identifiers for the individual security elements in the report (required):
A user can request downloading individual test cases in SARD. The download files consist of test file (one or more files of source code) and a manifest file (attributes of the test case). CWE Identifier is one of the test case attributes in the manifest file.
For detailed example, please see answer to <CR_A.2.2>.
GETTING A LIST OF CLAIMED CWE IDENTIFIER COVERAGE <CR_A.4.3>
Give detailed examples and explanations of how a user can obtain a listing of all of the CWE Identifiers that the owner claims the online capability's repository covers (required):
For detailed example, please see answer to <CR_A.2.3>.
USING CCR TO PROVIDE CLAIMED CWE IDENTIFIER COVERAGE <CR_A.4.4>
Give a detailed explanation of how a user can find the Coverage Claim Representation (CCR) XML document with all of the CWE Identifiers that the owner claims the online capability's repository covers (recommended):
For detailed example, please see answer to <CR_A.2.3>.
ONLINE CAPABILITY ELEMENT TO CWE IDENTIFIER MAPPING <CR_A.4.5>
If details for individual security elements are not provided, give examples and explanations of how a user can obtain a mapping that links each element with its associated CWE Identifier(s), otherwise enter N/A (required):
On the top menu of SARD home page, users can select menu item "view/download" to list all the
SARD test cases, a mapping that links each SARD element (i.e., test case) with its associated CWE Identifier will display:
Figure 11: List of Test Cases
Media Questions
ELECTRONIC DOCUMENT FORMAT INFO <B.3.1>
Provide details about the different electronic document formats that you provide and describe how they can be searched for specific CWE-related text (required):
SARD provides electronic document for test cases and test suites. The user can specify test cases or test suites to download;
SARD will generate a zip file that consists of the test files and a manifest file. In the manifest file, the metadata of each test case, including the file path of the test file, the weakness of the test case, the location of the weakness and other attributes of the test case are listed. The weakness is in the form of CWE Identifier. Here is an example:
Figure 12: List of Test Suite 45
Figure 13: Manifest file of Test Suite 45
ELECTRONIC DOCUMENT LISTING OF CWE IDENTIFIERS <CR_B.3.2>
If one of the capability's standard electronic documents only lists security elements by their short names or titles provide example documents that demonstrate how the associated CWE identifiers are listed for each individual security element
(required):
Please see answer to <CR_A.2.3>.
Graphical User Interface (GUI) Questions
FINDING ELEMENTS USING CWE IDENTIFIERS THROUGH THE GUI <CR_B.4.1>
Give detailed examples and explanations of how the GUI provides a
"find" or "search" function for the user to identify your capability's elements by looking for their associated CWE identifier(s) (required):
Please see answer to <CR_A.2.1>.
GUI ELEMENT TO CWE IDENTIFIER MAPPING <CR_B.4.2>
Briefly describe how the associated CWE identifiers are listed for the individual security elements or discuss how the user can use the mapping between CWE identifiers and the capability's elements, also describe the format of the mapping (required):
Please see answer to <CR_A.4.5>.
GUI EXPORT ELECTRONIC DOCUMENT FORMAT INFO <CR_B.4.3>
Provide details about the different electronic document formats that you provide for exporting or accessing CWE-related data and describe how they can be searched for specific CWE-related text (recommended):
Please see answer to <CR_B.3.1>.
Questions for Signature
STATEMENT OF COMPATIBILITY <CR_2.11>
Have an authorized individual sign and date the following Compatibility Statement (required):
"As an authorized representative of my organization I agree that we will abide by all of the mandatory CWE Compatibility Requirements as well as all of the additional mandatory CWE Compatibility Requirements that are appropriate for our specific type of capability."
Name: Barbara Guttman
Title: Manager, Software Quality Group, Software and System Division, Information Technology Laboratory
STATEMENT OF ACCURACY <CR_3.4>
Have an authorized individual sign and date the following accuracy Statement (recommended):
"As an authorized representative of my organization I agree that we will abide by all of the mandatory CWE Compatibility Requirements as well as all of the additional mandatory CWE Compatibility Requirements that are appropriate for our specific type of capability."
Name: Barbara Guttman
Title: Manager, Software Quality Group, Software and System Division, Information Technology Laboratory
STATEMENT ON FALSE-POSITIVES AND FALSE-NEGATIVES <CR_B.2.10> and/or <CR_B.3.7>
FOR TOOLS AND SERVICES ONLY – Have an authorized individual sign and date the following statement about your tools efficiency in identification of security elements (required):
"As an authorized representative of my organization I agree that we will abide by all of the mandatory CWE Compatibility Requirements as well as all of the additional mandatory CWE Compatibility Requirements that are appropriate for our specific type of capability."
Name: Barbara Guttman
Title: Manager, Software Quality Group, Software and System Division, Information Technology Laboratory
More information is available — Please edit the custom filter or select a different filter.
|