Meaningful Competency Assessment - COLA

1 1 Meaningful Competency Assessment Kathy Nucifora, MPH, MT(ASCP) COLA Objectives Differentiate between Competency Assessment and the traditional performance evaluation Outline the six CLIA-required components of Competency Assessment Apply methods described to conduct and document Meaningful Competency assessments Comply with COLA criteria that address Competency Assessments 2 The problem with Competency It s hard to see a problem clearly if you ve already decided what to do about it. Levitt and Dubner 3 4 We can easily define Competency , but evaluating Competency in a Meaningful way can be challenging. 5 Why the Emphasis on Competency ? Studies indicate that more education & training produce higher quality results. Judy Yost, CLIA Compliance, PT, and Quality Control: What s New for 2012? G2 Webinar 2012 Competency confirms the effectiveness of training CLIA-defined Lab Director qualifications are stringent due to the overall responsibilities, but testing personnel qualifications are minimal, based upon complexity.

2 Therefore, competence is critical. Errors may have impact on patients. 6 Assessing Competency vs. Performance Evaluations 7 Team -work Attendance and Punctuality Productivity Customer Service skills Technical Competency Personnel Evaluation Some items that might be included on a personnel evaluation 8 Direct Observation routine patient testing Direct Observation instrument maintenance Review of Records - intermediate test records Monitor Result Reporting Blind Sample Testing - PT can satisfy this component Evaluation of Problem Solving Skills Required for Competency Assessment for testing personnel Assessing Competency WHO is required to undergo Competency Assessment ? Technical Consultant, Technical Supervisor Clinical Consultant General Supervisor Testing Personnel 9 Assessing Competency The Laboratory Director is not required to have Competency Assessment BUT is responsible for all CLIA defined responsibilities.

3 10 Competency Evaluations WHO should evaluate Competency ? First it is important to have competent individuals assess the Competency of the personnel under review. The reviewer should not be the Office Manager, the Nursing Supervisor, or others who do not perform lab testing. CLIA gives this responsibility to the TC (moderate complexity) or TS (high complexity). Competency must be evaluated by qualified individuals (TC/TS/GS). 11 CLIA Requirements for Personnel Competency Assessment Six required elements of Competency Assessment 12 Competency Evaluations Two of the six required Assessment activities involve Direct Observation: 1. Direct observation of routine patient test performance; 2. Direct observation of performance of instrument maintenance and function checks; This can be documented with a detailed process oriented check list, per test or instrument.

4 13 Competency Evaluations Two of the six required Assessment activities involve Review of Records: 3. Monitoring the recording and reporting of test results; 4. Review of intermediate test results, QC records, proficiency testing results, and preventive maintenance records; Include with the Competency documentation records that were reviewed. 14 Competency Evaluations 5. Assessment of test performance through testing previously analyzed specimens, internal blind testing samples, or external proficiency testing samples; Rotate the performance of PT samples among testing personnel! 6. Assessment of problem-solving skills . Use problem logs, QC corrective action, complaint investigations, specimen rejection 15 Competency Assessment Don t waste paper Don t waste time Make it Meaningful ! 16 What does documentation of Competency Assessment look like?

5 17 18 19 20 21 Patient testing Maintenance Direct observation Intermediate records Reporting Record review Blind sample testing Problem solving Assess Competency Competency Assessment Serum Osmolarity Direct observation with DETAILS listing each critical step Instrument maintenance Test performance Review of Records Intermediate records attach copy of Osmo worksheet with controls recorded Result reporting attach copy of final result with any result comments, critical values, etc Blind Sample analysis Attach copy of report and expected results Problem Solving What if scenarios Attach problem logs with resolution 22 Competency Assessment Serum Osmolarity Your documentation should include: Notation of direct observations Includes each critical step in the test performance Copies of records reviewed Blind sample testing with results and acceptability Problem logs, quizzes, or what would you do if scenarios Conclusion is the testing personnel competent to perform this test?

6 Need for further training, if necessary Signature of testing personnel and person evaluating Competency 23 FAQ Do personnel performing PPM require Competency Assessment ? Who can evaluate Competency ? Do all elements of Competency Assessment need to be done at once? Does PT satisfy Competency Assessment requirements? Profiles can you do one Competency Assessment for a panel of tests run on a single platform? 24 Waived Testing COLA WAV criteria section includes Competency requirement for waived testing. Why? Many tests categorized as waived can have significant patient impact if not performed correctly. 25 Competency Assessment for specimen collection and processing personnel Not required by CLIA, but certainly recommended Required by COLA 26 27 IQCP Evaluating potential errors related to your testing personnel is required as part of the risk Assessment .

7 Complete and Meaningful Competency Assessment of staff involved in ALL phases of testing should be part of ANY Individualized Quality Control Plan. 28 29 How to contact COLA COLA central: Email: Address: 9881 Broken Land Parkway, Suite 200 Columbia, MD 21046 Website: Phone: 800 981 9883 Lab University Competency Assessment course Email Questions and Answers from Competency Assessment Webinar Who should perform the Competency Assessment on the Technical Consultant? A: Ideally, the Lab Director, but this can also be done for example, by another Technical Consultant within the same group. Does Competency have to be exactly every 12 months? A: Competency can be done throughout the year; it does not have to be done all at once. I suggest that for your annual competencies, you do this each calendar year for your staff.

8 COLA will not get real particular about it being exactly 12 months. That said; avoid doing Competency Assessment for example at the beginning of one year and at the end of the next year. You might miss an opportunity for improvement which we should be celebrating! How can we document the direct observation requirements for Competency Assessment ? A: For each test performed by the individual, document your observation of each step of the procedure. Use a list that denotes of each major step of the procedure, and check off each step as you observe. For example for CBC did they check for adequate specimen, and properly labeled specimen? Did they make sure that the sample was well mixed? If you make manual slides for differential, observe them making the slide to ensure that they achieve a feather edge. Do they recognize instrument flags and follow the procedure for resolving flags?

9 Indicate in your documentation the date of the observation, and the assession number of the patient testing observed. If you provide any feedback or if you identify a need for retraining, indicate this as well. How can we perform and document Competency Assessment for phlebotomists? A: My opinion on this is that there is no substitute for direct observation when performing Competency Assessment on phlebotomy staff. To supplement this, you can also do a quiz to cover things that may happen infrequently and are therefore not easy to observe. Such as what to do where should you obtain the specimen when a patient has had a mastectomy on one side. Competency for phlebotomy staff does not have to include the six required elements as for testing personnel. How is Competency Assessment related to IQCP? A: As part of the risk Assessment for IQCP, labs must evaluate potential errors that are related to personnel issues, such as lack of training or lack of Competency .

10 IQCP gives us a great opportunity to make sure that our Competency Assessment programs are Meaningful and effective. The risk Assessment should include a review of any problems or failures a lab has had with the test(s) and should incorporate any corrective actions in to the Competency Assessment for that test. It s a great opportunity to connect the dots. Who should do the Competency Assessment on the Clinical Consultant? A: The Lab Director. If the LD/ CC are the same person, then Competency is not required BUT here s an idea how about getting feedback from other physicians that use the lab s services? Ask if the reports include all information required and if they are easy to interpret? Ask the physicians if they know who to contact if they have questions about what test(s) should be ordered in a particular situation? I work for a large group of labs in a health system and the labs are geographically far apart.