Identifying the Most Important 21st Century Workforce ...

1 Research Report ETS RR 13-21. Identifying the most Important 21st Century Workforce competencies : An Analysis of the Occupational Information Network (O*NET). Jeremy Burrus Teresa Jackson Nuo Xi Jonathan Steinberg November 2013. ETS Research Report Series EIGNOR EXECUTIVE EDITOR. James Carlson Principal Psychometrician ASSOCIATE EDITORS. Beata Beigman Klebanov Gary Ockey Research Scientist Research Scientist Heather Buzick Donald Powers Research Scientist Managing Principal Research Scientist Brent Bridgeman Gautam Puhan Distinguished Presidential Appointee Senior Psychometrician Keelan Evanini John Sabatini Managing Research Scientist Managing Principal Research Scientist Marna Golub-Smith Matthias von Davier Principal Psychometrician Director, Research Shelby Haberman Rebecca Zwick Distinguished Presidential Appointee Distinguished Presidential Appointee PRODUCTION EDITORS.

2 Kim Fryer Ruth Greenwood Manager, Editing Services Editor Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS Research Report series, undergo a formal peer-review process by ETS. staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes.

3 Peer review notwithstanding, the positions expressed in the ETS Research Report series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of Educational Testing Service. The Daniel Eignor Editorship is named in honor of Dr. Daniel R. Eignor, who from 2001 until 2011 served the Research and Development division as Editor for the ETS Research Report series. The Eignor Editorship has been created to recognize the pivotal leadership role that Dr. Eignor played in the research publication process at ETS. Identifying the most Important 21st Century Workforce competencies : An Analysis of the Occupational Information Network (O*NET).

4 Jeremy Burrus, Teresa Jackson, Nuo Xi, and Jonathan Steinberg Educational Testing Service, Princeton, New Jersey November 2013. Find other ETS-published reports by searching the ETS ReSEARCHER. database at To obtain a copy of an ETS research report, please visit Action Editor: John Sabatini Reviewers: Ross Markle and Richard Coley Copyright 2013 by Educational Testing Service. All rights reserved. ETS, the ETS logo, and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). Abstract To identify the most Important competencies for college graduates to succeed in the 21st Century Workforce , we conducted an analysis of the Occupational Information Network (O*NET).

5 Database. O*NET is a large job analysis operated and maintained by the Department of Labor. We specifically analyzed ratings of the importance of abilities (52 ratings), work styles (16 ratings), skills (35 ratings), and knowledge (33 ratings) to succeed in one's occupation. First, we conducted descriptive analyses. Next, data were split into 2 sets, according to the theoretical structure proposed by the O*NET content model, and principal component analyses (PCAs). were run on each dataset. The PCAs identified 15 components: problem solving, mechanical skills, service orientation, cultural literacy, business literacy, science literacy, civic literacy, information processing, athleticism, visual acuity, fluid intelligence, communication skills, teamwork, achievement/innovation, and attention to detail/near vision.

6 Components were then ranked in importance using the mean component scores over all occupations. A comparison of this ranking with previous 21st Century competencies frameworks suggested that 5 competencies stand out as Important for most occupations: problem solving ( , complex problem solving), fluid intelligence ( , category flexibility), teamwork ( , cooperation), achievement/innovation ( , persistence), and communication skills ( , oral expression). Consistent with this conclusion, a correlation of component scores with wages found that 4 of these 5 competencies were strongly related to wages, with the exception being teamwork. Key words: O*NET, Workforce readiness, student learning outcomes, noncognitive skills, job competencies , KSA, job skills i Acknowledgments This paper was supported by the Research and Development division of ETS.

7 We would like to thank the following individuals for comments and suggestions on an earlier version of this report: Patrick Kyllonen, Richard Coley, Ross Markle, and John Sabatini. ii Table of Contents Page Identifying 21st Century Skills .. 2. O*NET .. 5. 5. History of O*NET .. 5. O*NET Framework .. 6. Development of the O*NET Database .. 9. Current Study .. 12. Method .. 12. Database .. 12. Measures .. 12. Results .. 13. Descriptive Analyses .. 13. Across All competencies .. 13. Knowledge .. 13. Skills .. 15. 15. Work Styles .. 15. Discussion of Descriptive Analyses .. 16. Principal Component Analyses (PCAs).. 17. Discussion of the Principal Component Analyses (PCAs).

8 18. Ranking of Principal Components .. 22. Correlation With Wages .. 23. Final 21st Century Workforce competencies Framework .. 27. General Discussion .. 30. Summary of Findings .. 30. Does the Workforce Possess These Skills? .. 30. Future Research .. 33. Additional Study Limitations .. 34. Conclusion .. 34. iii References .. 36. Appendix .. 39. iv List of Tables Page Table 1. Skills Identified by Three 21st Century Skills Frameworks .. 3. Table 2. O*NET Content Model, Number of Ratings per Descriptor, and Rating Source for Each Rating .. 8. Table 3. Descriptive Statistics for Top 10 competencies Across All Domains and Zones .. 14. Table 4. Descriptive Statistics for Top 10 competencies in the O*NET Knowledge Domain.

9 14. Table 5. Descriptive Statistics for Top 10 competencies in the O*NET Skill Domain .. 15. Table 6. Descriptive Statistics for the Top 10 competencies in the O*NET Abilities Domain 16. Table 7. Component Loading Matrix of the O*NET Worker Requirements Variables .. 19. Table 8. Component Loading Matrix of the O*NET Worker Characteristics Variables .. 21. Table 9. Composite Scores and Rank Orders of O*NET Worker Requirements .. 24. Table 10. Composite Scores and Rank Orders of O*NET Worker Characteristics .. 24. Table 11. Correlation of Components With Log-Transformed Wages .. 27. Table 12. Skill Deficiencies Identified in Casner-Lotto and Barrington (2006) and Crosswalk to Current Results.

10 31. v List of Figures Page Figure 1. Histograms of O*NET worker requirements components.. 25. Figure 2. Histograms of O*NET worker characteristics components.. 26. Figure 3. Twenty-first Century Workforce competencies framework.. 29. vi What are the most essential skills for the 21st Century Workforce ? This question would not be necessary in an ideal world, as educational systems would continually adapt to meet the demands of the economy. There is growing evidence, however, that a mismatch exists between the skills possessed by much of the Workforce and the skills required by employers. In 2011, a survey of 2,000 companies revealed that two thirds of these companies reported difficulties finding people qualified to fill some of their open positions (Manyika et al.)