II. ESTABLISHING A PROJECT SOFTWARE ASSURANCE ACTIVITY A. Concepts and Definitions Every software development, enhancement, or maintenance project includes some assurance activities. The types, amount, and formality of such activities are decisions of the project manager, based on an assessment of the project, its risks, and its development and operational environments. Even a simple, one person development job has assurance activities embedded in it, even if the programmer denies that "quality assurance" plays any part in what is to be done. Each programmer has some idea of how code should be written, and this idea functions as a coding standard for that programmer. Likewise, each of us has some idea of how documentation should be written, and this is a personal documentation standard. Each programmer reviews his/her products to make sure they meet their internal standards, and this is an assurance review or audit. Each programmer tests and inspects his/her own work, and these are verification and validation processes. The list could go on, but the idea should be clear. A project software assurance program involves the processes that each programmer goes through, but requires the planning and formal establishment of project, rather than personal, standards and processes. B. Tailoring Software Assurance to the Project Specific project characteristics and risks influence assurance needs, and assurance planning should be tailored to reflect this fact. Characteristics that should be considered include safety and mission criticality of the software, schedule and budget, size and complexity of the product to be produced, and size and organizational complexity of the development staff. The relationship of criticality to assurance is as one would expect: the more critical the software, the more important and formal the software assurance effort must be. The relationship of schedule and budget is not intuitive, however; the tighter the budget and schedule, the more critical it is to have a well planned and effective assurance effort. This does not mean that projects with more resources can afford to be lax, it just means that tight resources increase risks that should be offset by a strong assurance program. The projected size of the software to be produced influences the level of assurance required. A large project requires explicit and detailed standards for all of the products in order to get at least a minimum standard of quality from the varied ideas and experience of many different programmers. In addition, a large project requires significant efforts in testing and other verification activities, which have to be planned and the plans followed. In short, just due to the size of the activity, a significant and formal assurance program must be established or risks of poor quality products must be accepted. On the other hand, a small project may require little formal assurance, and on a very small one, the assurance efforts may be left to the programmer involved if adequate, informal planning is done. Another factor that influences assurance planning is the project's organizational structure. A small, centralized development staff can easily participate in reviews and inspections, keep each other informed on the status of nonconformances, and help each other in meeting coding and documentation standards. A large or dispersed staff will have many different ideas of the best ways of doing things and many more difficulties in communicating them. In the latter case, a more formal assurance program and a larger assurance effort will be needed. A last but very important characteristic is the difference between the requirements of a software providing organization and a software acquiring organization. A software provider actually develops the products by developing designs and writing code, etc., and therefore needs a full assurance program. An acquirer does not develop software and thus can limit its assurance activities to those that ensure that the provider is adhering to agreed- to methods and standards and producing the agreed-to products. C. Creating the Software Assurance Plan An effective assurance program requires planning and follow through; it cannot simply evolve along with the project. Adequate assurance planning ensures that the assurance activities are focused on the quality requirements and risks associated with the specific project. The purpose of creating a software assurance plan is to document/specify the conduct of the activities that will comprise software assurance for a specific project. Armed with information about the project and the available software assurance resources, the project manager is ready to develop the plan. A useful guide for documenting assurance plans is provided in the assurance sections of the SMAP Management Plan Documentation Standard. In addition, the following should be considered: Plan software assurance in conjunction with management and engineering planning, i.e., during the project concept and initiation phase. Phase assurance activities properly. For example, design standards must be produced well before design is to be done. Complete tool development or procurement before the tools are needed. Especially important is the development of test tools and test data sources. A summary of software assurance activities grouped by software development phases is provided in Appendix A. D. Project Structure Considerations In planning and establishing a software assurance program, one consideration is the software project organization and the location in that organization of the assurance activities. Experience has indicated, both in hardware and software, that some assurance functions are best done by organizational entities that are separate from the ones doing engineering activities. Software Quality Assurance (SQA) is one activity that should be organizationally separated from the producing organizations. Administratively, the SQA organization should report no lower than the project manager; indeed, many large successful software producing organizations have the SQA organization report administratively to top corporate management and interface with the project manager. The reason for this separation of function is that the SQA organization is management's arm that assures that standards are met and that procedures are followed. If SQA is not independent of the development activity, clear and impartial assessment will be difficult. In addition, many organizations have had success using an independent test team, or at least an independent test development team. The team is responsible for developing test plans, procedures, and test cases for formal acceptance tests. Independence is required because these tests should be requirements driven and not influenced by the design structure and coding details. E. Completion Criteria Because of the nature of software, it is difficult to ascertain the status of a development or maintenance activity. It is important, therefore, to define criteria for the completion of specific development stages. For example, during the implementation phase, one has to do the lowest level detailed design of small program elements, code the elements, and unit test them. When a significant number of program elements are involved, it is difficult for anyone to ascertain the status of the units without specific completion criteria. For example, if there is a criterion that detailed design is complete only after the rework that finishes a design inspection, then the design can be said to be either complete or incomplete depending on the status of the rework. The setting of completion criteria is a management activity, but the audit of records is an SQA activity. The accuracy of the reported status can then be determined. This is important to both providers and acquirers of software, and this "status auditing" is an important SQA function. F. Implementation of the Software Assurance Plan Once the project needs have been determined and the software assurance planning is complete, the plan must be implemented. Qualified, trained staff must be obtained, and special training must be made available where needed. If standards and procedures are not available for reuse on this project, they must be written. Staff must be trained in the standards and procedures, since merely writing them down does not guarantee compliance. All of the above are management activities, but the assurance staff is a resource to help complete them. Staff devoted purely to assurance activities is usually small compared to the project staff. On the other hand, it is important to have people with specific assurance responsibilities, even if they must be shared organizationally with other duties. Too often the truism that "quality is everybody's business" becomes "quality is nobody's business" if specific responsibilities are not assigned. G. Sources of Help In addition to this guidebook, there are other sources of help in planning and implementing a software assurance program. First, there is a NASA software planning requirement, stated in NMI 2410.10. In addition, there are Center requirements and guidance documents. Many of these are listed in Appendix B, which also contains other useful reference material used in the development of this guidebook. All NASA Centers have assurance organizations that provide varying degrees of support, assistance, and actual performance of software assurance activities. H. Summary Software assurance is an essential part of the development and maintenance of software. Software assurance forms part of the triad of activities, along with software management and software engineering that, taken together, can provide a successful software development, enhancement, or maintenance activity. This guidebook is intended to increase the general understanding in NASA of what comprises software assurance and how it is to be planned and implemented.