Spread Knowledge

CS615 - Software Project Management - Lecture Handout 42

User Rating:  / 0

Related Content: CS615 - VU Lectures, Handouts, PPT Slides, Assignments, Quizzes, Papers & Books of Software Project Management


Quality Concept

What is it? It's not enough to talk the talk by saying that soft ware quality is important, you have to (1) explicitly define what is meant when you say 'software quality, (2) create a set of activities that will help ensure that every software engineering Work product exhibits high quality, (3) perform quality assurance activities on every software project, (4) use metrics to develop strategies to improving your software process and as a consequence the quality of. the end product.

Who does it? Everyone involved in the software engineering process is responsible for quality.

Why is it important? You can do it right, or you can do it over again. If a software team stresses quality in all software engineering activities, it reduces the amount of rework that it must do that results in lower costs, and more importantly,
improved time-to-market.

What are the steps? Before software quality assurance activities can be initiated, it is important to define 'software quality' at a number of different levels of abstraction, Once you understand what quality is, a software team must identify a set of SQA activities that will filter errors out of work products before they are passed on.

What is the work product? A Software Quality Assurance Plan is created to define a software team's SQA strategy. During analysis, design, and code generation, the primary SQA work product is the formal technical review summary report. During testing, test plans and procedures are produced. Other work products associated with process improvement may also be generated.

How do I ensure that I've done it right? Find errors before they become defects! That is, work to improve your defect removal efficiency, thereby reducing the amount of rework that your software team has to perform.

SQA encompasses:

  1. A quality management approach
  2. Effective software engineering technology (methods and tools)
  3. Formal technical reviews that are applied throughout the software process
  4. A multi-tiered testing strategy
  5. Control of software documentation and the changes made to it
  6. A procedure to ensure compliance with software development standards (when applicable)
  7. Measurement and reporting mechanisms.

Software quality is defined as conformance to explicitly stated functional and performance requirements, documents, and standards. The factors that affect software quality are a complex combination of conditions that can be measured based on data, such as audit-ability, completeness, consistency, error tolerance, and expandability.1n addition, this data includes hardware independence, software system independence, modularity, security, and simplicity.

Software quality assurance is a planned and systematic approach necessary to ensure the quality of a software product. Software reviews filter the product of all errors. Software reviews can be conducted at all stages in the DLC of a software product, such as analysis, design, and coding.

Testing is an important element of SQA activity. There are various testing tools to automate testing process. The SQA plan is used as the template for all SQA activities planned for a software project and includes details of the SQA activities to be performed during project execution.

SCM is used to establish and maintain integrity of software items and ensure that they can be traced easily. SCM helps define a library structure for storage and retrieval of software items. SCM helps assess the impact of a recommended
change and make decisions depending on the costs and benefits. SCM needs to be performed at all phases in the DLC of a software project

The various SCM activities are identifying changes, controlling changes, controlling versions, implementing changes, and communicating changes. These activities are independent of the supervision of the project or the product manager. This is to ensure objectivity in SCM. The scope of SCM is not limited by code and includes requirements, design, database structures, test plans, and documentation. SCM procedures vary with the project.

Producing quality software

As we have seen, one of the main problems in producing quality software is the difficulty in determining the degree of quality within the software. As there is no single widely accepted definition for quality, and because different people perceive quality in different ways, both the developer and the customer must reach agreement on metrics for quality' (this is discussed in more detail later). The method of measuring quality may differ for different projects.

This problem is discussed in a paper by Wesselius and Ververs (1990), in which they conclude that complete objectivity in quality assessment cannot be achieved.
They identify three distinct components of quality:

  • An objectively assessable component
  • A subjectively assessable component
  • A non-assessable component

The quality of a product is objectively assessable when the characteristics of the product, as stated in the requirements specification, can be identified.

The quality of a product is subjectively assessable when the characteristics of the product comply with the customer's expectations.

The quality of a product is non-assessable when it behaves according to our expectations in situations that have not been foreseen.

Wesselius and Ververs suggest that, for the quality of a software product to be assessable, as many characteristics as possible should be moved from the subjective and non-assessable components to the assessable component.
Essentially, this means that the requirements specification must describe as many measurable characteristics of the product as possible.

Experience supports Wesselius and Ververs' conclusions. Badly defined requirements are always a source of dispute between developer and customer.
Well-defined, detailed and measurable requirements minimize disputes and disagreements when the development of the product is complete.

However, many development methods have a prolonged interval between the specification of requirements and the delivery of the product (refer to Chapter 4 for a discussion of the software development cycle). The determination of quality should not be postponed until development is complete. Effective software quality control requires frequent assessments throughout the development cycle. Thus, effective quality control coupled with a good requirements specification will clearly increase the quality of the final product.

Quality Control

Variation control may be equated to quality control. But how do we achieve quality control? Quality control involves the series of inspections, reviews, and tests used throughout the software process to ensure each work product meets the
requirements placed upon it.

Quality control includes a feedback loop to the process that created the work product. The combination of measurement and feedback allows us to tune the process when the work products created fail to meet their specifications. This approach views quality control as part of the manufacturing process.

Quality control activities may be fully automated, entirely manual, or a combination of automated tools and human interaction. A key concept of quality control is that all work products have defined; measurable specifications to which we may compare the output of each process. The feedback loop is essential to minimize the defects produced.

Quality Control Myths

The establishment of effective quality control frequently encounters various misconceptions and myths, the most common of which is related to the cost effectiveness of quality control. Cobb and Mills (1990) list several of these myths, and suggest methods of combating them. Two of the more prevalent myths identified by Cobb and Mills are described below.

Myth: Quality costs money. This is one of the most common myths (not only in software development). In fact, quality in software usually saves money. Poor quality breeds failure. There is a positive correlation between failures and cost in that it is more expensive to remove execution failures designed into software than to design software to exclude execution failures.

Myth: Software failures are unavoidable. This is one of the worst myths because the statement is partly true, and is therefore often used as an excuse to justify poor quality software. The claim that ‘there is always another bug' should never be a parameter in the design or implementation of software

As these myths lose ground in modern approaches to software development, the demand for suitable quality control standards and procedures increases. The IEEE issued their first standard for software quality assurance plans in 1984 (IEEE .1984), followed by a detailed guide to support the standard, issued in 1986. The US Department of Defense issued a separate standard 2168 for defense systems software quality programs (DOD 1988b), which forms a companion to the famous US DOD standard 2167A (DOD 1988a) for defense systems software development. The European ISO standard 9000-3, of 1990 (ISO 1990) gives a broader meaning to the term quality assurance and covers configuration control too.

Resources for quality control

When the SQA mandate includes configuration control activities, the required resources will also include those required for configuration control. Merging SQA and configuration control is not uncommon, and can eliminate some duplication of assignments and activities. Two alternative organizational charts are shown in Fig. 8.5. Note that for small projects, merging the two groups may mean simply assigning both responsibilities to the same person.

Though many tools are common to quality control and configuration control, few tools are specifically designed for quality control. The following are some of the general support tools that can be useful in supporting SQA activities:

  • Documentation utilities
  • Software design tools
  • Debugging aids
  • Structured preprocessors
  • File comparators
  • Structure analyzers
  • Standards auditors
  • Simulators
  • Execution analyzers
  • Performance monitors
  • Statistical analysis packages
  • Integrated CASE tools
  • Test drivers
  • Test case generators

These tools support quality control in all phases of software development.
Documentation aids can provide partially automatic document writers, spelling checkers and thesauruses etc. Structured preprocessors (such as the UNIX utility lint) are useful both to standardize code listings. And to provide additional compile-time warnings that compilers often overlook. Early warnings regarding possible execution time problems can be provided by simulators, execution time analyzers and performance monitors. Substantial software system testing can often be performed automatically by test suite generators and automatic test executors.

All SQA tools to be used during software development should be identified and described in the SQA plan. This plan includes a description of all required quality assurance resources and details of how they will be applied. Thus, at the start of the project SQA resources can be budgeted and procured as part of the required project development resources.

Project manager

Quality Assurance

Quality assurance consists of the auditing and reporting functions of management. The goal of quality assurance is to provide management with the data necessary to I be informed about product quality, thereby gaining insight and confidence that product quality is meeting its goals. Of course, if the data provided through quality assurance identify problems, it is management's responsibility to address the problems, and apply the necessary resources to resolve quality issues.

Software Quality Assurance

Even the most jaded software developers will agree that high-quality software is an important goal. But how do we define quality? A wag once said, 'Every program does something right, it just may not be the thing that we want it to do.'

Many definitions of software quality have been proposed in the literature. For our purposes, software quality is defined as:

Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software.

There is little question that this definition could be modified or extended. In fact, a definitive definition of software quality could be debated endlessly. For the purposes of this book, the definition serves to emphasize three important points:

  1. Software requirements are the foundation from which quality is measured.
    Lack of conformance to requirements is lack of quality.
  2. Specified standards define a set of development criteria that guide the manner in which software is engineered. If the criteria are not followed, lack of quality will almost surely result.
  3. A set of implicit requirements often goes unmentioned (e.g., the desire for ease of use and good maintainability). If software conforms to its explicit requirements but fails to meet implicit requirements, software quality is suspect.

Background Issues

Quality assurance is an essential activity for any business that produces products to be used by others. Prior to the twentieth century, quality assurance was the sole responsibility of the craftsperson who built a product. The first formal quality assurance and control function was introduced at Bell Labs in 1916 and spread rapidly throughout the manufacturing world. During the 1940s, more formal approaches to quality control were suggested. These relied on measurement and continuous process improvement as key elements of quality management.

Today, every company has mechanisms to ensure quality in its products. In fact, explicit statements of a company's concern for quality have become a marketing ploy during the past few decades.

The history of quality assurance in software development parallels the history of quality in hardware manufacturing. During the early days of computing (1950s and 1960s), quality was the sole responsibility of the programmer. Standards for quality assurance for software were introduced in military contract software development during the 1970s and have spread rapidly into software development in the commercial world [IEE94].

Extending the definition presented earlier, software quality assurance is a planned and systematic pattern of actions [SCH98] that are required to ensure high quality in software.

The scope of quality assurance responsibility might best be characterized by paraphrasing a once-popular automobile commercial:

Quality is Job # 1'. The implication for software is that many different constituencies have software quality assurance responsibility-software engineers, project managers, customers, salespeople, and the individuals who serve within an
SQA group.

The SQA group serves as the customer's in-house representative. That is, the people who perform SQA must look at the software from the customer's point of view. Has software development been conducted according to pre-established
standards? Have the technical disciplines properly performed their roles as part of the SQA activity? The SQA group attempts to answer these and other questions to ensure that software quality is maintained.

SQA Activities

Software quality assurance is composed of a variety of tasks associated with two different constituencies-the software engineers who do technical work and an SQA group that has responsibility for quality assurance planning, oversight, record keeping, analysis, and reporting. 'If Software engineers address quality and perform quality assurance and quality control activities by applying solid technical methods and measures, conducting formal technical reviews, and performing well-planned software testing.

The charter of the SQA group is to assist the software team in achieving a highquality end product. The Software Engineering Institute [PAU93] recommends a set of SQA activities that address quality assurance planning, oversight, record keeping, analysis, and reporting.

These activities are performed (or facilitated) by an independent SQA group that:

Prepares SQA plan for a project

The plan is developed during project planning and is reviewed by all interested parties. Quality assurance activities performed by the software engineering team and the SQA group are governed by the plan. The plan identifies

  • evaluations to be performed
  • audits and reviews to be performed
  • standards that are applicable to the project
  • procedures for error reporting and tracking
  • documents to be produced by the SQA group
  • amount of feedback provided to the software project team

Participates in the development of the project's software process description

The software team selects a process for the work to be performed. The SQA group reviews the process description for compliance with organizational policy, internal software standards, externally imposed standards (e.g., ISO- 900 I), and other parts of the software project plan.

Reviews software engineering activities to verify compliance with the defined software process. The SQA group identifies, documents, and tracks deviations from the process and verifies that corrections have been made.

Audits designated software work products to verify compliance with those defined as part of the software process. The SQA group reviews selected work products; identifies, documents, and tracks deviations; verifies that corrections have been made; and periodically reports the results of its work to the project manager.

Ensures that deviations in software work and work products are documented and handled according to a documented procedure. Deviations may be encountered in the project plan, process description, - applicable standards, or technical work products.

Records any noncompliance and reports to senior management.
Noncompliance items are tracked until they are resolved.

In addition to these activities, the SQA group coordinates the control and management of change and helps to collect and analyze software metrics.

SQA Plan

The SQA plan serves as the template for SQA activities planned for each software project. The SQA group and the project team develop the SQA plan. The initial two sections of the plan describe the purpose and references of the SQA plan. The next section records details of the roles and responsibilities for maintaining software product quality.

The Documentation section of the SQA plan describes each of the work products produced during the software process. This section defines the minimum set of work products that are acceptable to achieve high quality. The Documentation section consists of:

  • Project document such as project plan, requirements document, test cases, test reports, user manual, and administrative manuals
  • Models such as ERDs, class hierarchies
  • Technical document such as specifications, test plans
  • User document such as help files

All applicable standards to be used in the project are listed in the Standards and Guidelines section of the SQA plan. The standards and practices applied are the document standards, coding standards, and review guidelines.

Example of the contents of a software quality assurance plan

  1. Software quality assurance organization and resources
    Organization structure.
    Personnel skill level and qualifications
  2. SQA standards, procedures, policies and guidelines
  3. SQA documentation requirements
    List of all documentation subject to quality control
    Description of method of evaluation and approval
  4. SQA software requirements
    Evaluation and approval of software
    Description of method of evaluation
    Evaluation of the software development process
    Evaluation of reused software
    Evaluation of non-deliverable software
  5. Evaluation of storage, handling and delivery
    Project documents
    Data files
  6. Reviews and audits
  7. Software configuration management (when not addressed in
    a separate document)
  8. Problem reporting and corrective action
  9. Evaluation of test procedures
  10. Tools techniques and methodologies
  11. Quality control of subcontractors, vendors and suppliers
  12. Additional control
    Miscellaneous control procedures
    Project specific control
  13. SQA reporting, records and documentation.
    Status reporting procedures
    Storage and security
    Retention period

Software quality metrics

Much attention has been devoted to questions associated with the measurement of quality. How do we determine the extent to which a software product contains this vague attribute called quality? When is the quality of a software product high and when is it low?

One of the more recent developments in quality assurance (not only for software) is the realization that quality is not a binary attribute that either exists or does not exist. Kaposi and Myers (1990), in a paper on measurement-based quality
assurance, have stated their belief that 'the quality assurance of products and processes of software engineering must be based on measurement 7. The earlier the measurement of quality begins, the earlier problems will be located. Cohen et al. (1986), in addressing the cost of removing errors during the early phases of software development, proclaim the existence of the famous exponential law8.

The quality of two products can be compared, and it is perfectly acceptable to claim that the quality of one product is greater than the quality of another. It is also acceptable to measure quality and deduce the extent of expected faults based on the measured result.

The set of measurable values associated with the quality of a product is referred to as the product's quality metrics. Software quality metrics can be used to determine the extent to which a software product meets its requirements. The use of quality metrics increases the objectivity of the evaluation of product quality. Human evaluation of quality is subjective, and is therefore a possible source of disagreement, particularly between customer and developer.

A number of methods for establishing software quality metrics are currently being developed, though no generally accepted standard has yet emerged. For example, an initial draft of IEEE Std-1061 (1990) includes a detailed discussion of software quality metrics in general, including a suggested methodology for applying metrics, and many examples and guidelines. Quality metrics, once defined, do indeed increase objectivity, but the definition itself is not necessarily objective and greatly depends upon the needs of the organization that produces the definition.

The basic approach for applying software quality metrics requires:

  • The identification of all required software quality attributes. This is usually derived from the software requirements specification.
  • Determinations of measurable values to be associated with each quality attribute. A description of the method by which each measurable value will be measured.
  • A procedure for documenting the results of measuring the quality of the software product.

A set of many values can be used to determine the overall quality of a software product. However, a single measure can be created to represent the overall quality of the software product. This requires:

  • A weighted method for combining the measured quality attributes into a single measure of quality for the product.

Some examples of software metrics are:

Reliability: The percentage of time that the system is successfully operational (e.g. 23 out of 24 hours produces: 100 x (23/24) percent)

Recoverability: The amount of time it takes for the system to recover after failure (e.g. 1hour to reload from backups and 30 minutes to reinitialize the system)

User-friendliness: The amount of training time needed for a new user

The measurement of software quality should not be performed only at the end of a project. The degree of quality should be measured at regular intervals during development. Thus, any major reduction in the overall measure of quality should
act as a warning for the project manager that collective action is required. High quality at the end of the project is achieved by assuring high quality throughout the development of the project.

Some general guidelines

The basic software quality assurance activities cover the review and approval of the development methodology, the software and documentation, and the supervision and approval of testing. Other SQA activities, such as the supervision
of reviews, the selection and approval of development tools, or the administration of configuration control, depend on the way SQA is adapted to a specific project.
The size of the project is usually the determining factor. The following guidelines discuss some of the parameters to be considered for different types of project when planning SQA.

  • In small projects, many SQA activities can be performed by the project manager. This includes the organization and supervision of reviews and audits, the evaluation and selection of development tools, and the selection and application of standards.
  • Test procedures and testing are always best when conducted by a separate independent team (discussed later). The decision on whether supervision of testing activities can be assigned to SQA depends on many factors, including the independence of the SQA team, the size of the project and the 'complexity of the project.
  • When testing is performed by an independent test team, SQAS involvement will be minimal. In most other cases it will be the responsibility of the SQA team to plan and supervise the testing of the system.
  • As a general guideline, it is usually undesirable for SQA to be performed by a member of the development team. However, small projects often cannot justify the cost of a dedicated SQA engineer. This problem can be solved by having a single SQA engineer responsible for two or three small projects (with each project funding its share of the SQA services).

One additional guideline is based on the conclusions of Wesselius and Ververs (1990) for the application of effective quality control:

  • The ability to control software quality is directly linked to the quality of the software requirements specification. Quality control requires the unambiguous specification of as many of the required characteristics of the software product as possible.