| |
NASA Software Documentation Standard Appendix E
|
ASSURANCE AND TEST PROCEDURES DATA ITEM DESCRIPTIONS
This appendix contains the specifications for the format, outline, and content of the Assurance and Test Procedures and rolled-out sections. Major sections of the Assurance and Test Procedures have been rolled-out into separate Data Item Descriptions (DIDs) using the template DID (NASA-DID-999) for purposes of clarity and manageability.
The Assurance and Test Procedures DIDs provide outlines for the complete Assurance and Test Procedures. Major sections of the Assurance and Test Procedures point to lower level DIDs that contain more detailed descriptions of these major sections.
The number of Assurance and Test Procedures documents generated does not have to match the number of DIDs presented in this section. Lower-level detailed DIDs provide additional substructure and contain content discussion which should be reviewed even when the content is recorded in-line (i.e., not rolled-out).
The detailed DIDs in this appendix may be used as they stand to produce separate documents of the Assurance and Test Procedures.
Note that the DIDs for the Assurance Procedures (NASA-DID-A100) and the Test Procedures (NASA-DID-A200) are to be used multiple times for various sections of Assurance and Test Procedures.
Table E-1. DID Index (Numeric Order)
DID Number Title
NASA-DID-A000 Assurance and Test Procedures NASA-DID-A100 Assurance Procedures DID NASA-DID-A200 Test Procedures DID
Table E-2. Complete DID Set for Assurance and Test Procedures
NASA-DID-A000 Assurance and Test Procedures NASA-DID-A100 Quality Assurance Procedures Verification and Validation Procedures NASA-DID-A100 Inspection, Review, and Analysis Procedures Testing Procedures NASA-DID-A200 Unit Test Procedures NASA-DID-A200 Integration Test Procedures NASA-DID-A200 Acceptance Test Procedures NASA-DID-A100 Quality Engineering Assurance Procedures NASA-DID-A100 Safety Assurance Procedures NASA-DID-A100 Security and Privacy Assurance Procedures NASA-DID-A200 Certification Procedures
TABLE OF CONTENTS
1.0 INTRODUCTION 2.0 RELATED DOCUMENTS 3.0 QUALITY ASSURANCE 4.0 VERIFICATION AND VALIDATION 4.1 Inspections, Reviews, and Analyses 4.2 Testing 4.2.1 Unit Testing 4.2.2 Integration Testing 4.2.3 Acceptance Testing 5.0 QUALITY ENGINEERING ASSURANCE 6.0 SAFETY ASSURANCE 7.0 SECURITY AND PRIVACY ASSURANCE 8.0 CERTIFICATION 9.0 ABBREVIATIONS AND ACRONYMS 10.0 GLOSSARY 11.0 NOTES 12.0 APPENDICES
The purpose of the Assurance and Test Procedures is to document all of the technical procedures (such as test or assurance procedures) used to assure software. The types of assurance and the organizations responsible for performing that assurance activity are specified in the Management Plan. In particular, Independent Verification and Validation (IV&V) organizations will be tasked to produce separate Assurance and Test Procedures for their activities using this DID.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
The purpose of this section is to document the objective of, and procedures for, quality assurance (QA) activities specified in the Assurance Plan section of the Management Plan, including reviews and audits for the purpose of evaluating quality. In general, QA activities focus on conformance to standards, procedures, and plans.
Use the Assurance Procedures DID (NASA-DID-A100) for each QA activity. Document all reports from each activity in the Management, Engineering, and Assurance Reports using the Audit Report DID (NASA-DID-R002) or Assurance Activity Report DID (NASA-DID-R008) or both.
The purpose of this section is to document the objective of, and procedures for, verification and validation (V&V) activities specified in the Assurance Plan section of the Management Plan.
The purpose of this section is to document the objective of, and procedures for, V&V activities specified in the Assurance Plan section of the Management Plan, including inspections or walkthroughs, phase transition reviews, and static analyses, such as code evaluation or algorithm analysis.
Use the Assurance Procedures DID (NASA-DID-A100) for each V&V activity. Document all reports from each activity in the Management, Engineering, and Assurance Reports using the Inspection Report DID (NASA-DID-R003), the Review Report DID (NASA-DID-R011), and/or the Assurance Activity Report DID (NASA-DID-R008), as applicable.
The purpose of this section is to document the test objectives, procedures, criteria, expected results, and actual results of tests to demonstrate that the software meets requirements and is acceptable. Separate subsections will be generated for each level of testing (such as unit, integration, and acceptance) specified in the Assurance Plan section of the Management Plan and for each major group of tests within a testing level.
The primary topics of each testing subsection include:
The purpose of this section is to document the test objectives, procedures, criteria, expected results, and actual results of unit tests. The procedures for unit tests are based on, and are intended to verify, the detailed design.
For each major group of unit tests, a subsection detailed according to the Test Procedures DID (NASA-DID-A200) should be generated. Summaries of test results should be recorded in the Test Report DID (NASA-DID-R009).
The purpose of this section is to document the test objectives, procedures, criteria, expected results, and actual results of integration tests. The procedures for integration tests are based on the detailed design wherever two or more software entities (CSUs, CSCs or CSCIs) are combined for testing (i.e., the level of the architectural design).
For each major group of integration tests, a subsection detailed according to the Testing Procedures DID (NASA-DID-A200) should be generated. Summaries of test results should be recorded in the Test Report DID (NASA-DID-R009).
The purpose of this section is to document the test objectives, procedures, criteria, expected results, and actual results of acceptance tests. The procedures for acceptance tests are based on the functional and acceptance requirements for that entity of software (CSU, CSC, or CSCI).
For each major group of acceptance tests, a subsection detailed according to the Testing Procedures DID (NASA-DID-A200) should be generated. Summaries of test results should be recorded in the Test Report DID (NASA-DID-R009).
The purpose of this section is to document the objective of, and procedures for, quality engineering (QE) activities specified in the Assurance Plan section of the Management Plan, including the assurance of reliability, maintainability, and other quality factors.
Use the Assurance Procedures DID (NASA-DID-A100) for each QE activity. Document all reports from each activity in the Management, Engineering, and Assurance Reports using the Assurance Activity Report DID (NASA-DID-R008).
The purpose of this section is to document the objective of, and procedures for, safety assurance activities specified in the Assurance Plan section of the Management Plan.
Use the Assurance Procedures DID (NASA-DID-A100) for each safety assurance activity. Document all reports from each activity in the Management, Engineering, and Assurance Reports using the Assurance Activity Report DID (NASA-DID-R008).
The purpose of this section is to document the objective of, and procedures for, security and privacy assurance activities specified in the Assurance Plan section of the Management Plan.
Use the Assurance Procedures DID (NASA-DID-A100) for each QA activity. Document all reports from each activity in the Management, Engineering, and Assurance Reports using the Assurance Activity Report DID (NASA-DID-R008).
The purpose of this section is to document the test objectives, procedures, criteria, expected results, and actual results of tests for all certification tests, as specified in the Assurance Plan section of the Management Plan, to demonstrate that the software meets requirements and is acceptable. (Note that certification is typically done only at the CSCI or system level.)
Use the Testing Procedures DID (NASA-DID-A200) for each certification test. Summaries of certification test results should be recorded in the Certification Report DID (NASA-DID-R001).
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Return to Beginning of Appendix
TABLE OF CONTENTS
1.0 INTRODUCTION 2.0 RELATED DOCUMENTS 3.0 OBJECTIVES 4.0 PROCEDURES 5.0 GLOSSARY 6.0 NOTES 7.0 APPENDICES
The purpose of the Assurance Procedures section is to record the objectives, procedures, and other technical information related to assurance activities for either a product or a process.
NOTE: This DID is used multiple times to document Quality Assurance, Verification and Validation, Quality Engineering Assurance, Safety Assurance, and Security and Privacy Assurance activities.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Specify the objectives of the assurance activity and the specific quality attributes for which it is being evaluated. Trace the assurance activities to the appropriate section of the Assurance Plan section of the Management Plan where the plans have been described and methods to be employed have been stated. When appropriate, trace assurance activities to either the appropriate Requirements or Design section(s) of the Product Specification or process description(s) in the Management Plan.
Describe the details required to conduct the specific assurance activity. Describe the overall criteria used for evaluation of this activity. When appropriate, provide a range of acceptability or numeric measures. Describe the specific measurement criteria against which the product or process is to be evaluated, such as a checklist.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the template DID (NASA-DID-999) for a detailed structure and content description of this section.
Return to Beginning of Appendix
TABLE OF CONTENTS
1.0 INTRODUCTION 2.0 RELATED DOCUMENTATION 3.0 TEST IDENTIFICATION AND OBJECTIVE 4.0 PROCEDURES 5.0 EVALUATION CRITERIA 6.0 EXPECTED RESULTS 7.0 ACTUAL RESULTS 8.0 ABBREVIATIONS AND ACRONYMS 9.0 GLOSSARY 10.0 NOTES 11.0 APPENDICES
The purpose of a Test Procedures section is to record the objectives, procedures, results, and other technical information related to a test or a group of tests. The title of the section or document should indicate the level and, if appropriate, type of test and the product being tested.
This Test Procedures DID is applicable at all levels of testing (unit, integration, acceptance, and certification) and for software testing. The DID is used either as a separate document or in-line for each test or group of tests associated with a specific level of testing.
Different categories of testing may be performed by either an engineering or assurance organization. While testing does assure a product, it may be a large undertaking, which requires the development of its own set of products, such as test procedures and test cases. Whether test products are developed by an engineering or assurance organization, provisions should also be made to have assurance performed on the test products.
NOTE: This DID is use multiple times to document Unit, Integration, Acceptance, and Certification tests.
Refer to the Template DID (NASA-DID-999) for a detailed structure and content description of this section.
Refer to the Template DID (NASA-DID-999) for a detailed structure and content description of this section.
Identify the test or set of tests. Provide a link between this test section and the tests specified in the Assurance Plan section of the relevant Management Plan for this software.
Describe the test objectives. For example, for software unit testing, one test objective is to demonstrate that the detailed design has been correctly represented in the code. For acceptance testing, the objective might be to determine that the system meets a selected set of requirements from the Product Specification.
Describe the specific sections of the Product Specifications (Requirements, Design, etc.) that are to be demonstrated by this test. This specification should provide the appropriate traceability in the Product Specification. (i.e., to Detailed Design for Unit Test, to Architectural Design for Integration Test, to Requirements for Acceptance Test).
Describe the procedures necessary to support the test(s) in terms such as:
Describe the test case(s) to be used in this test or set of tests in terms such as:
Describe the criteria used to determine the success or failure of the test(s) in terms such as:
Describe the expected results from the test(s) in terms such as:
If this information is available in electronic form, it should be maintained in that form for possible future regression testing.
Identify the particular version of the product tested and the specifics of the environment (support software, hardware, etc.) in which it was tested and the actual test date. Describe the actual results from the test(s). The content and format of this section should mirror that of expected test results for ease of comparison.
A statement of the success or failure of this test or set of tests based on the criteria defined in Section 5.0 is given in a test report to management. The relevant test report (or set of reports) should be referenced in this section.
Refer to the Template DID (NASA-DID-999) for a detailed description of content for this section.
Refer to the Template DID (NASA-DID-999) for a detailed description of content for this section.
Refer to the Template DID (NASA-DID-999) for a detailed description of content for this section.
Refer to the Template DID (NASA-DID-999) for a detailed description of content for this section.
Return to Beginning of Appendix
Return to NASA Software Documentation Standard
If you have any questions or comments about the SATC, contact:
Dr. Linda Rosenberg
NASA/GSFC
Code 302 - Bldg 6
Greenbelt, MD 20771
Linda.Rosenberg@gsfc.nasa.gov
This page was last updated on:
06/29/99