Vol. 13, No. 1 …

1 2006 Vol. 13, No. 1 IMPLEMENTING PRODUCTAND process QUALITY ASSURANCEWith permission from a client of The process GroupOne of the process Areas (PA's) in the CMMI* is process andProduct Quality Assurance (PPQA). This process area supportsthe delivery of high-quality products and services by providingthe project staff and managers with appropriate visibility into, andfeedback on, processes and associated work products throughoutthe life of the this article we share an example process definition for PPQA from one of our clients. The company uses seven part-time Software Quality Assurance(SQA) staff, each person working eight hours per month onaverage performing audits. Each auditor is either a developer ortester as his or her primary job. To ensure objectivity, each personis assigned to a project other than the one they work on. All auditactivities are managed by the leader of the Engineering ProcessGroup (EPG), a part-time position of seven hours per month for apopulation of 80 software benefits of implementing a lightweight audit process are to: Maintain the gains made in process improvement over theprevious five years Detect process refinements needed to simplify process imple-mentation Share engineering and management practices across the devel-opment groupsProcess and product Quality Assurance (PPQA) Audit PURPOSEThe purpose of this Audit process is to provide SQA staffwith a standard method for performing process applies to software projects within the RESPONSIBILITIESThe SQAstaff is responsible for conducting audits using thisprocess and the audit checklist(s) ( ).

2 * Capability Maturity Model Integration / CONDUCTING Audit PreparationEach SQA staff member (auditor) will keep appraised ofthe project he or she has been assigned to by reviewingthe project's Software Project Management Plan (SPMP)and schedule. The auditor will meet with the ProjectLeader approximately one week prior to a scheduledaudit. At this time the following shall be discussedand/or identified: The actual date of the audit The location of project plans and artifacts to beaudited The names of project team members whom theauditor can contact should questions arisePrior to the actual date of the audit, the auditor shall: Identify and prepare copies of the required auditchecklists Review associated project plans and AuditsThe auditor shall perform audits of the project teamworkactivities by reviewing resulting work artifacts andmaking comparisons to the associated process within theteam's documentation.

3 As a minimum, the audit shallfollow the items contained in the audit checklist(s). Theresults will be recorded on the Project Leader will be informed of the audit resultsvia the reporting activity and must resolve any noncom-pliant issues. Resolving noncompliant issues should behandled in a timely fashion (less than two weeks isexpected) unless some unavoidable circumstancesprohibit resolution. Noncompliant issues that are not resolved promptly andare not included in a plan for resolution shall beescalated by notifying the EPG Lead of the situation. Ifthe EPG Lead cannot bring about resolution of noncom-pliant issues, then he or she will escalate the issues tothe Technical ON PAGE 2 IMPLEMENTING product AND process QUALITY ASSURANCE(Continued from page 1) AUDIT REPORTINGThe completed checklist acts as the audit Naming ConventionsA standard naming convention for audit report files isused.

4 File names will be a concatenation (with under-scores) of: Group name_Project group, project, and audit names can be abbreviatedto avoid long file names. The audit name shallcorrespond to one of the audit types in Table 3. The dateis in the form, mmddyy, and is the actual date of theaudit. An example would be Location of ReportsThe audit reports shall be stored into the groupConfiguration Management (CM) system (//qa-reports/project-X/<file>). Auditors shall email a copy of each audit report to theassociated Project Leader and the EPG at the report is completed. The EPG shall be respon-sible for storing the file in the appropriate shall be responsible for providing the EPG withmetric information on an individualproject basis. Auditors shall use acopy of a blank metrics spreadsheetobtained from the process AssetLibrary (PAL)( ). An exampleis shown in Table 1. Once filled in,the auditor shall email the spread-sheet to the EPG at ProjectIn the spreadsheet, "ProjectName" shall be replaced by theactual name of the project, usingan abbreviation that makes senseif necessary.

5 Below the projectname, the project leader's nameshall be entered in the Project and Project Leader, a separate row shall beused to record metric data for each audit. If multiple audittypes are audited together ( Project Planning, ProjectMonitoring and Control, and Requirements), a unique rowshall be used to record the data for each audit addition, a separate row shall be used for each audit torecord issues found in each process area (PA). In the sample(Table 1) a project-planning audit was conducted on11/03/xx. During that audit, four major and two minor issueswere found in Project Planning (PP), two major and fiveminor issues in CM, and three major and one minor issue(s)in Measurement & Analysis (MA). Both the Audit Name and process Area columns are populated using pre-defined pull-down menus. The Audit Name correlates to the checklistname while the process Area corresponds to the CMMI PAname.

6 Ablank entry is provided in the Audit Name drop-down menu for audits with multiple rows. The checklist has acolumn to identify the PA for each item checked. Below is anexample of a checklist to AuditsThis metric allows the auditor to specify if the audit wasplanned (scheduled in the SPMP) or unplanned. Unplannedaudits are conducted at the discretion of the auditor, basedon previous evaluation of process ON PAGE 3 ITEMPROCESS VERIFICATION the requirements documented using the template in the PAL or using a customer-defined format? there a traceability matrix for the requirements? Note: a traceability matrix implies that a requirement is traced to a design element, test case and area of requirements tracked and updated according to the Software Project Management Plan (SPMP), or the Software Configuration Management Plan (SCMP) if it is a separate document?

7 Need to verify that changes to requirements are requirements analyzed using a technique other than a peer review ( use cases, Object Oriented Design methods, writing test cases for each requirement, etc.)? requirements under Configuration Management?Table 2 Excerpt from an Audit ChecklistTable 1 Record of PPQAA udit MetricsProjectNameProjectLeaderProject 1(Smith)Project 3(Smith)AuditDate11/03/xx11/25/xxPlanned /UnplannedAuditsPlannedPlannedAuditNameP PPMCP rocessAreaPPCMMAPMCWorkProductsReviewed2 120 Number ofMajorNon-CompliantIssues4230 Number ofMinorNon-CompliantIssues2510 Number ofMajorNon-CompliantIssuesClosed0000 Number ofMajorNon-CompliantIssuesElevated toEPG Lead0000 Number ofMajor Non-CompliantIssuesElevated toSeniorManagement0000 Number ofHi-PriorityRisk Itemsin ProjectRisk List4000 Hours toPrepareandConductthe Audit34 AuditorLiuLiuIMPLEMENTING product AND process QUALITY ASSURANCE(Continued from page 2)This metric serves to compare planned work to actual work,which has a direct impact on hours and funding.

8 This metricmust be tracked as each SQA audit is completed. A signif-icant number of unplanned audits shall trigger analysis ofactual versus planned funding for SQA activities. Audit NameThis column is used to record the type of audit beingconducted. A separate row shall be used to record metric datafor each audit type. Table 3 illustrates the phases of a projectand the abbreviation to be entered into this column of themetric spreadsheet. This table is provided on the Info tab ofthe spreadsheet. Use a blank in the Audit Name column whenan audit requires more than a single row as described insection Options QA and EPG are provided for use inrecording the results when QA and EPG activities process AreaThis column contains the CMMI process Area of the issuebeing reported. Work Products ReviewedDuring the life cycle of a project, the team will generateproject work products ( Software RequirementsSpecification, Design Review, etc.)

9 The SQA auditor willenter the number of work products that were assessed duringthe audit into the Number of Work Products Reviewedcolumn. Note that this figure is the total number of workproducts reviewed. For example, if 20 test logs werereviewed in addition to the test plan the total number of workproducts reviewed would be 21. Number of Major Noncompliant IssuesThis metric tracks the number of major , Conducting Audits, describes escalating noncom-pliant issues. A major noncompliant issue is identified as asignificant deviation from the documented process , forexample, when the project lacks something that is necessaryand specifically named in the process documentation. Thefollowing are examples of major noncompliant issues: Lack of an SPMP or other plan Projects not following what is documented in the SPMP SPMP missing major section(s) ( MA) Documents not being kept up to date Lack of a schedule and/or Work Breakdown Number of Minor Noncompliant IssuesMinor noncompliant issues shall be annotated as such in thecomments section of the associated checklist.

10 Examplenoncompliant issues include: Typographical errors Documents with titles or locations that are not exactly asreferenced in a plan Documents with noncurrent release attributes Late posting of meeting minutes, peer reviews, noncompliant issues are problems that should be trackedto closure. However, they are not collected for metrics MajorNoncompliant Issues ClosedThis metric tracks the number of noncompliances closed orresolved. There should be no noncompliant issues leftunresolved. Section , Conducting Audits, describesescalating noncompliant MajorNoncompliant Issues Elevated to EPG LeadThe EPG Lead shall review the audit reports and will beapprised of any noncompliant issues. Section , ConductingAudits, describes escalating noncompliant MajorNoncompliant Issues Elevated toSeniorManagementThis metric tracks the number of noncompliances elevatedto senior management.