Best Practices for the Formal Software Testing Process

1 BEST Practices FOR THEFORMAL SOFTWARETESTING PROCESSAlso Available from Dorset House PublishingThe Deadline: A Novel About Project Managementby Tom DeMarcoISBN: 0-932633-39-0 Copyright 1997 320 pages, softcoverDr. Peeling's Principles of Management:Practical Advice for the Front-Line Managerby Nie PeelingISBN: 0-932633-54-4 Copyright 2003 288 pages, softcoverFive Core Metrics: The Intelligence Behind Successful Software Managementby Lawrence H. Putnam and Ware MyersISBN: 0-932633-55-2 Copyright 2003 328 pages, softcoverMeasuring and Managing Performance in Organizationsby Robert D. Austin foreword by Tom DeMarco and Timothy ListerISBN: 0-932633-36-6 Copyright 1996 240 pages, softcoverPeopleware: Productive Projects and Teams, 2nd Tom DeMarco and Timothy ListerISBN: 0-932633-43-9 Copyright 1999 264 pages, softcoverProject Retrospectives: A Handbook for Team Reviewsby Norman L.

2 Kerth foreword by Gerald M. WeinbergISBN: 0-932633-44-7 Copyright 2001 288 pages, softcoverSurviving the Top Ten Challenges of Software Testing :A People-Oriented Approachby William E. Perry and Randall W. RiceISBN: 0-932633-38-2 Copyright 1997 216 pages, softcoverWaltzing with Bears: Managing Risk on Software Projectsby Tom DeMarco and Timothy ListerISBN: 0-932633-60-9 Copyright 2003 208 pages, softcoverFor More Information Contact us for prices, shipping options, availability, and more. Sign up for DHQ: The Dorset House Quarterly in print or PDF. Send e-mail to subscribe to e-DHQ, our e-mail newsletter. Visit for excerpts, reviews, downloads, and HOUSE PUBLISHINGAn Independent Publisher of Books onSystems and Software Development and Management.

3 Since West 12th Street New York, NY 10014 USA1-800-DH-BOOKS 1-800-342-6657212-620-4053 fax: Practices FOR THEFORMAL SOFTWARETESTING PROCESSA MENUOFTESTINGTASKSR odger D. Drabicks\DHDorset House Publishing353 West 12th StreetNew York, NY 10014 Library of Congress Cataloging-in-Publication DataDrabick, Practices for the Formal Software Testing Process : a menu of testingtasks / Rodger bibliographical references and 0-932633-58-7 (softcover)1. Computer Software - Testing . I. '4-dc222003062472 All product and service names appearing herein are trademarks or registeredtrademarks or service marks or registered service marks of their respective own-ers and should be treated as Design: Nuno AndradeCover Graphic: Detail from Figure 1-5 Cover Author Photograph: Courtesy of Lockheed MartinCopyright 2004 by Rodger D.

4 Drabick. Published by Dorset House PublishingCo., Inc., 353 West 12th Street, New York, NY rights reserved. No part of this publication may be reproduced, stored in aretrieval system, or transmitted, in any form or by any means, electronic, mechani-cal, photocopying, recording, or otherwise, without prior written permission of in the English language in Singapore, the Philippines, and South-east Asia by Alkem Company (S) Pte. Ltd., Singapore; in the English languagein India, Bangladesh, Sri Lanka, Nepal, and Mauritius by Prism Books Pvt.,Ltd., Bangalore, India; and in the English language in Japan by Toppan Co.,Ltd., Tokyo, in the United States of AmericaLibrary of Congress Catalog Number: 2003062472 ISBN: 0-932633-58-7 12 11 10 987654321 Digital release by Pearson Education, Inc.

5 , June, 2013 DedicationTo my wife, Karen, without whose love and support this book and theother things I do would not be possible; and to my daughters, Alysonand Liz I love you page intentionally left blank AcknowledgmentsMany people including coworkers, managers, SQA and Testing associ-ates, family, and friends have earned a special Thank You for helpingme bring my ideas from a concept to a published 'd like first to express my thanks in memoriam to Peter Natale,formerly of Eastman Kodak Company, who provided me with my firstopportunity in Software Testing . Without Pete's support and mentor-ship, this book would not have been possible. Thanks also to FrankMondziel, who provided guidance in our first efforts in Software QA andtesting at Kodak, and to Ed Cattron, a long-time friend, coworker, andmanager at Kodak, who supported my initial efforts to develop and for-malize this Process model on a large and complex Internal RevenueService program in the early 1990' also owe a great debt of gratitude to Bill Perry of the QualityAssurance Institute and Dr.

6 Edward Miller of Software Research, Inc.,both of whom gave me the opportunity to present portions of thismodel at their Testing conferences. I made my first presentation on thetopic as a keynote address at a Software Quality Week, hosted by Edand his company. Independent consultant Denis Meredith gave methe opportunity to present the model end-to-end at one of the TestingComputer Software Conferences he coordinated. Denis remains my"go to guy" for questions about requirements and risk ACKNOWLEDGMENTST hank you very much, gentlemen! It's been a pleasure knowing you,and learning from so many at Software Quality Engineering who have been helpfulto me along the way, I give special thanks: Bill Hetzel, former presidentof SQE, and Dave Gelperin and Rick Craig, associates at SQE all ofyou have provided invaluable technical and moral support throughoutmy Testing Crispin, coauthor of Testing Extreme Programming, introducedme to special techniques for approaching Testing of that agile that knowledge and for reviewing a number of draft chapters, Lisa,I am deeply 's difficult to adequately express the gratitude I owe to several edi-tors.

7 First, my friend of many years, Debbie Lafferty, formerly Acquisi-tions Editor at Addison-Wesley, started me working on this book. Deb-bie introduced me on-line to Wendy Eakin at Dorset House Publishing,and that introduction has resulted in the final product you hold inyour hands. To Wendy and the rest of the Dorset House staff, includ-ing Nuno Andrade, Vincent Au, and David McClintock, many thanksfor helping me turn a concept into a real book that I hope will providevaluable information to the Testing community! Thanks also to DorsetHouse author and Software systems consultant Tim Lister, for his rec-ommendation that this book be finally, a lifetime of gratitude and thanks to Karen, my wifeand friend of many years. Without your help and encouragement, thisbook would never have been started, much less xiiiPreface xvChapter 1 Introduction : OVERVIEW : THE Software DEVELOPMENT LIFE CYCLE : THE Formal Testing LIFE CYCLE : ORGANIZATIONAL CONSIDERATIONS : THE NEED FOR THE Process MODEL : How то USE THE Formal Testing Process MODEL : INPUT- Process -OUTPUT MODELS : LEVEL 0 IPO MODEL FOR Formal Testing : LEVEL 1 IPO MODEL FOR Formal Testing : LIMITATIONS INHERENT IN THIS BOOK'S SCOPE : WHAT'S NEXT?

8 27 Chapter 2 The Formal Testing Process Model: Level 1 IPOD iagrams : OVERVIEW : EXTRACT TEST INFORMATION FROM PROGRAM PLANS LEVEL 1 IPODIAGRAM 28ixx : CREATE TEST PLAN : CREATE TEST DESIGN, TEST CASES, TEST Software , AND TESTPROCEDURES : PERFORM Formal TEST : UPDATE TEST DOCUMENTATION : WHAT'S NEXT? 50 Chapter 3 Extract Test Information from Program Plans: Levels 2and 3 IPO Diagrams : OVERVIEW : EXTRACT TEST INFORMATION FROM PROGRAM PLANS LEVEL 2 IPODIAGRAM : REVIEW PROGRAM MANAGEMENT PLAN LEVEL 3 IPO DIAGRAM : REVIEW QUALITY ASSURANCE PLAN LEVEL 3 IPO DIAGRAM : REVIEW Software DEVELOPMENT PLAN LEVEL 3 IPO : REVIEW CONFIGURATION MANAGEMENT PLAN LEVEL 3 IPODIAGRAM : WHAT'S NEXT?

9 72 Chapter 4 Create Test Plan: Levels 2 and 3 IPO Diagrams : OVERVIEW : CREATE TEST PLAN LEVEL 2 IPO DIAGRAM : ANALYZE REQUIREMENTS LEVEL 3 IPO DIAGRAM : WRITE TEST PLAN LEVEL 3 IPO DIAGRAM : REVIEW TEST PLAN LEVEL 3 IPO DIAGRAM : WHAT'S NEXT? 93 Chapter 5 Create Test Design, Test Cases, Test Software , and TestProcedures: Levels 2 and 3 IPO Diagrams : OVERVIEW : CREATE TEST DESIGN, TEST CASES, TEST Software , AND TEST PRO-CEDURES LEVEL 2 IPO DIAGRAM : REVIEW TEST PLAN LEVEL 3 IPO DIAGRAM : CREATE TEST DESIGN LEVEL 3 IPO DIAGRAM : REVIEW TEST DESIGN LEVEL 3 IPO DIAGRAM : CREATE TEST CASES LEVEL 3 IPO DIAGRAM : ACQUIRE TEST Software -BUILD LEVEL 3 IPO DIAGRAM 119 CONTENTS : ACQUIRE TEST Software -BUY LEVEL 3 IPO DIAGRAM : CREATE TEST PROCEDURES LEVEL 3 IPO DIAGRAM : WHAT'S NEXT?

10 131 Chapter 6 Perform Formal Test: Levels 2 and 3 IPO Diagrams : OVERVIEW : PERFORM Formal TEST LEVEL 2 IPO DIAGRAM : HOLD PRETEST MEETING LEVEL 3 IPO DIAGRAM : EXECUTE TEST LEVEL 3 IPO DIAGRAM : DETERMINE DISPOSITION OF INCIDENTS LEVEL 3 IPODIAGRAM : HOLD POSTTEST MEETING LEVEL 3 IPO DIAGRAM : WRITE TEST REPORT LEVEL 3 IPO DIAGRAM : WHAT'S NEXT? 159 Chapter 7 Update Test Documentation: Levels 2 and 3 IPO Dia-grams : OVERVIEW : UPDATE TEST DOCUMENTATION LEVEL 2 IPO DIAGRAM : ANALYZE TEST DOCUMENTATION PROBLEMS LEVEL 3 IPODIAGRAM : UPDATE TEST DOCUMENTS LEVEL 3 IPO DIAGRAM : REVIEW AND APPROVE Software TEST DOCUMENTS LEVEL 3 IPODIAGRAM : WHAT'S NEXT?