Quality 101 Session 2 Supplier Quality - The Global Voice ...

1 Quality 101 Session 2. Supplier Quality ASQ Aviation, Space & Defense Division Collaboration on Quality in the Space and Defense Industries (CQSDI) Forum March 13, 2017. Anne Hennessy Rick Coberly Dave Newton James Banks Approved for Public Release Quality 101 Session Two (2:30 4:45 pm). Agenda Supplier Quality Process 2:30 2:55 Risk Identification and Mitigation 2:55 3:05 Supplier Evaluation 3:05 3:10 Supplier Approval and Selection 3:10 3:25 Supplier Quality /Mission Assurance Requirements Activity 3:25 3:30. Supplier Quality Execution 3:30 3:45 Supplier Oversight 3:45 3:50 Supplier Ratings - Scorecard 3:50 4:05 Actions for Non-conformance - Root Cause Corrective Action Activity 4:05 4:40. Summary and Resources 4:40 4:45. 2 Approved for Public Release Supplier Quality Process Approved for Public Release Supplier Quality Process Managing Supplier Risk Closed Loop System of Checks and Balances Manages Risk 4 Approved for Public Release Managing the Risk of External Providers Pre-Award Evaluation Source Selection, approval status, controlled use, Scope of approval product type Purchase Order Coding for Quality Requirements Periodic reviews of Supplier performance including process, products, services conformity, on-time delivery performance Reviews types include Metrics for Supplier , Readiness Reviews and Discrepancy Reviews during Qual/ATP.

2 Product acceptance inspections Supplier Ratings Register of suppliers approved, conditional, disapproved Actions for non-conformances Root Cause/Corrective Action Reference: AS 9100D Control of Externally Provided Processes, Product and Services 5 Approved for Public Release Risk Identification and Mitigation Approved for Public Release Managing Supplier Risk Why perform a risk assessment of suppliers What is the risk management process What are sources of Supplier risk How to assess risk How to manage risk Mapping risk potential to mitigation strategies 7 Approved for Public Release Why Perform a Risk Assessment of Suppliers? It's a requirement of AS9100 D. The organization shall identify and manage the risks associated with the external provision of processes, products, and services, as well as the selection and use of external providers . Provides a basis for source selections Understanding of efforts to manage the risk Technical, cost, schedule, sub-tier management Total cost and complexity of an acquisition Disciplines, effort, level of skills to execute They Contribute to Our Success 8.

3 Approved for Public Release What is the Risk Management Process? 1. Identify risk Use Models to segment the supply base by Identify Supplier or product Risk 2. Assess risk Probability and Impact Review Assess Controls Risk 3. Choose mitigation strategy Map risks to mitigation strategies unique to each situation 4. Implement controls Choose Implement Mitigation Controls Strategy 5. Review controls 9. Approved for Public Release Potential Sources of Supplier Risk Loss of critical Schedule Cost vs. skills performance Competency Sub-tier Manufacturing Supplier site change control Process Quality control performance Conversion of customer Requirements Requirements Supplier Counterfeit Risk parts 10 Approved for Public Release A Quick View of Risk Assessment Within the aviation, space, and defense industry, risk is generally expressed in terms of the likelihood of occurrence and the severity of the consequences A formal risk mitigation process addresses risks that exceed a defined criteria (such as endangers life, mission failure, loss of functionality).

4 Focuses resources on highest risk References: DCMA-INST 219 Supplier Risk Management Through Standard Contract Surveillance . Risk Management Guide for DoD Acquisition, (Sixth Edition, Version ). 11 Approved for Public Release Methods to Manage Risk Early understanding of risk potential Ensure requirements are complete, accurate, unambiguous, verifiable Kick-off review performed by one or more disciplines depending on complexity Use engineering oversight Understand changes from baseline products Manufacturing readiness reviews Regular product reviews Challenge qualification by similarity Periodic QMS evaluation Customer source inspection / testing Incoming inspection and screening 12. Robust testing Approved for Public Release Segment by Product Classification Who is responsible for engineering? Where the product is in the life cycle? Map product types within the matrix Quadrant 1 Build-to-print and Off-the-Shelf Quadrant 2 Build-to-print and Development Quadrant 3 Build-to-Spec and Off-the-Shelf Quadrant 4 Build-to-Spec and Development Apply unique risk mitigation strategies Reference: Aerospace Report No TOR-2011 (8591-18), Supplier Risk Evaluation and Control, June 1, 2011, TOR-2011.

5 13 Approved for Public Release Product Segmentation: Risk and Mitigation Strategies Product Prime's Supplier 's Source of Technical Risk Mitigation Category Responsibility Responsibility Strategies Low technical/moderate Quality risk Decide appropriateness Pre-award Changes in designs or processes, Build-to-print of product Manufacturing processes evaluation of materials, site and Off-the- Create a drawing from Standard product lists similarities of Uses Qual by Similarity Shelf Supplier 's to manage Re-use of existing code design and Control of processes unique requirements processes Control of Quality Technology High technical/ high Quality risk Engineering Manufacturing capability Build-to-print Assumptions of the Supplier 's Software design (data and technology Careful reviews and capabilities or process controls flows, hierarchy Generates code Strong process Development Changes during development diagrams, data Performs software controls Stability of Supplier 's technical staff structures, etc.)

6 Qualification testing Low technical/ low Quality risk Standard product line Supplier makes minor changes and Development is complete Engineering Build-to-spec Specifies requirements qualifies by similarity Configuration oversight by and Off-the- and buys suppliers Supplier changes material management use of subject Shelf standard product Differences in application from As-Is re-use of existing matter experts intended use code High technical/moderate Quality risk Prime must ensure Supplier is applying Engineering reqmts to mature technology (post PDR focus). Build-to-spec Thorough pre- proof of design Supplier 's capabilities need to and Specifies requirements award Manufacturing capability accommodate the complexity Development evaluation Code to requirements Supplier 's ability to transition to production 14 Reference: Aerospace Report No TOR-2011 (8591-18), Supplier Risk Evaluation and Control, June 1, 2011, TOR-2011. Approved for Public Release Supplier Evaluation Approved for Public Release Supplier Evaluation Pre-Award Evaluation Supply Ratings Source Selection Supplier Performance Oversight Is the Supplier qualified to perform their anticipated work scope prior to commencing source solicitation and selection or contractual arrangement, including teaming agreements?

7 Qualified suppliers will exhibit four key attributes: 1. Compliance to an appropriate, functioning, Quality Management System Example: Industry standards, such as CMMI, ISO9001 and AS9100. 2. Possess adequate financial health and historically demonstrates reasonable financial stability 3. Demonstrated capability, qualifications and technical maturity for the type of work under consideration Example: Special process validation audits by National Aerospace and Defense Contractors Accreditation Program NADCAP. 4. Adequate current and future capacity (both within the Supplier and the Supplier 's supply chain) to perform the work scope 16 Approved for Public Release Supplier Evaluation Quality Management System (QMS) Supply Ratings Source Selection Supplier Performance Oversight Industry standards CMMI, ISO9001 and AS9100 describe QMS elements Significant improvements in Quality , cost reductions, process repeatability Confirm that QMS certification is not merely virtual certifications Review the Supplier 's documented procedures and implementation Including specific methods for the Supplier 's control of goods and services from his sub-tier suppliers Must describe the Supplier 's organizational structure and reporting lines, including clear delineation of responsibilities Scope should include all functions involved Engineering, development, systems management and integration, production, installation, testing, support, modification and servicing of both the hardware and software of the products and services being procured Special process validation audits by third party entities such as National Aerospace and Defense Contractors Accreditation Program (NADCAP).

8 17. Supplier Evaluation Financial Health Assessment Supply Ratings Source Selection Supplier Performance Oversight Financial stability is essential to a Supplier 's stable, predictable and dependable performance Means to identify potential risk, key in times of economic stress Various financial metrics to evaluate a Supplier 's financial health Current Ratio Quick Ratio Debt to Equity Return on Sales Return on Equity Credit Risk Monitor (CRM) service integrates several of them into an aggregated score Dun and Bradstreet (D&B) and Equifax offer similar estimate of financial distress for private firms without publically-available financial data Incorporate into new Supplier qualification & periodic reviews 18 Approved for Public Release Supplier Evaluation Process Capability Assessment Supply Ratings Source Selection Supplier Performance Oversight Ensures Supplier is a good fit for the anticipated work scope Ensures Supplier has the tools, processes and infrastructure to succeed Capabilities, equipment, personnel, process controls and management systems Ensures robust sub-contractor selection and control Perform by subject matter experts Ensuring the Supplier 's product level of technical maturity increases in line with the contract delivery schedule 19 Approved for Public Release Supplier Evaluation Technical Maturity Level Supply Ratings Source Selection Supplier Performance Oversight Definitions of product technical maturity from the Department of Defense (DoD)

9 , National Aeronautics and Space Administration (NASA), European Space Agency (ESA), European Commission (EC) and Oil & Gas Industry, US. Department of Energy (DOE). Reference : Level 1: Undefined and not capable: No Process, Methods, Tools and/or inappropriate behaviours. Level 2: Defined and applied but not 100% effective or not applied everywhere in the company (capable for low risk products and services). Level 3: Defined, applied and effective: Repeated satisfactory performance capable. Level 4: Performance of proactive improvements towards planned targets, but not systematically on all processes / areas / products. Level 5: Best in class, continual improvement fully deployed, involving all stake holders as part of company culture. Reference : Supply Chain Management Handbook (SCHM), Section , Supplier Selection & Capability Assessment Model Guidance Notes, 20 Approved for Public Release Supplier Evaluation Capacity Assessment Supply Ratings Source Selection Supplier Performance Oversight Evaluate the level at which a Supplier is managing their workload Entire plant Cell level Machine level Management of capacity at the lowest level is most desirable Is the Supplier using capacity data for long range planning?

10 The Defense Contracts Management Agency (DCMA) guidebook provides excellent guidelines for pre-award surveys: 21 Approved for Public Release Supplier Evaluation Warning Signs Supply Ratings Source Selection Supplier Performance Oversight Pre-award evaluation elements may uncover indications of potential risk Warning Signs Quality Management System Verification Quality Capacity QMS in-place and working Reliance on QMS certification only CARS. Defense business segment small Financial Assessment Sales skewed to commercial Credit Risk Monitoring (Frisk Score) Commercial decline/strike Equifax Marginal Score Self-financing Financial Ratios Multiple sub-tier suppliers Process Capability Assessment Work scope outside normal bounds Capable for anticipated scope Diminishing sources Tools, processes and infrastructure Key personnel change Employee turnover rate Technical Assessment High absenteeism Technical Maturity (TRL) Lack of cross training Capacity shortages Capacity % Overtime Long range planning Managing capacity at site vs.