Quality Assurance Framework - cll.mcmaster.ca

1 1 Quality Assurance in Higher Education: A Review of the Literature Karen nicholson Teaching and Learning Development Officer Council of Ontario Universities Degree Level Expectations project McMaster University January 3, 2011 Introduction Over the past two decades, Quality Assurance processes in higher education have become increasingly common in Europe, the United Kingdom and Australia, and are steadily gaining in importance in Canada and the United States. A number of factors have contributed to this development: government and industry have advocated that a well-educated workforce is essential to increased productivity and to maintaining a competitive edge in the global knowledge economy, resulting in an increase in public funding for higher education and a drive to make postsecondary education more accessible, particularly for under-represented populations. This, in turn, has brought about calls for greater accountability on the part of educational providers and the measuring of outputs through Quality Assurance processes (Australian Universities Quality Agency, 2009; Harvey, 1998 Harvey & Knight, 1996; Koslowski, 2006; Quality Assurance Task Force, 2010; Woodhouse, 1999).

2 In Ontario, external audit and appraisal of graduate programs has been in place since 1968; for undergraduate programs, it has existed since 1996. 2010 marked the establishment of the Ontario Universities Council on Quality Assurance (the Quality Council) and a new Quality Assurance protocol, the Quality Assurance Framework . The Quality Council is responsible for assuring (a) the Quality of all programs leading to degrees and graduate diplomas granted by Ontario s publicly assisted universities, and (b) the integrity of university Quality Assurance processes. Through these practices, the Quality Council seeks to assist with the improvement and enhancement of university programs. The Quality Council has the following mandate: to guide Ontario s publicly assisted universities in the ongoing Quality Assurance of their academic programs; to review and approve proposals for new graduate and undergraduate programs; to ensure through regular audits that Ontario s publicly assisted universities comply with Quality Assurance guidelines, policies and regulations for graduate and undergraduate programs; to communicate final decisions to the Ministry of Training, Colleges and Universities; 2 to review and revise, from time to time for future application, the Council of Ontario University s Quality Assurance protocols in light of its own experiences and developments in the field of Quality Assurance ; to liaise with other Quality Assurance agencies, both provincially and elsewhere; and to undergo regular independent review and audit at intervals of no longer than eight years.

3 ( Quality Assurance Task Force, 2010, p. 35) With the implementation of the Quality Assurance Framework , publicly assisted universities in Ontario have placed themselves in the mainstream of Quality Assurance . The following literature review provides an overview of some of the main themes related to Quality Assurance in higher education in order to provide background information and context for the Framework . First, we will examine the debate surrounding the meaning and measures of Quality in higher education. Next, we will discuss the dual purpose of Quality Assurance to ensure accountability and promote enhancement and the processes of accreditation, assessment and audit of Quality Assurance by which it is achieved. Finally, we will review common approaches to Quality Assurance in higher education and examine the main features of outcome-based education. Defining Quality One of the overarching themes in the higher education literature on Quality Assurance is how to define and measure Quality .

4 Harvey and Knight (1996) identify the following meanings attributed to Quality : Quality as exceptional, , exceptionally high standards of academic achievement; Quality as perfection (or consistency), which focuses on processes and their specifications and is related to zero defects and Quality culture; Quality as fitness for purpose, which judges the Quality of a product or service in terms of the extent to which its stated purpose defined either as meeting customer specifications or conformity with the institutional mission is met; Quality as value for money, which assesses Quality in terms of return on investment or expenditure and is related to accountability; and Quality as transformation, which defines Quality as a process of qualitative change with emphasis on adding value to students and empowering them. Quality as transformation is further described as critical transformation and as a meta- Quality concept : The transformative notion of Quality presupposes a fundamental purpose of higher education.

5 It assumes that higher education must concern itself with transforming the life experiences of students, by enhancing or empowering them. The transformative conception is, in effect, a meta- Quality concept. Other concepts, such as perfection, high standards, fitness for purpose and value for 3 money, are possible operationalizations of the transformative process rather than ends in themselves. (Harvey & Knight, 1996, pp. 15-16) Bogue (1998) highlights the following three perspectives on Quality common to institutions of higher education: limited supply, often used in institutional rankings such as Macleans; Quality within mission, defined as fitness for purpose ; and value-added, or Quality in results, defined by Astin (1985, cited in Bogue, 1998, p. 9) as the impact on the student s knowledge and personal development and on the faculty member s scholarly and pedagogical ability and productivity . The definitions put forth by Harvey and Knight (1996) and Bogue (1998) can be aligned as presented in Table 1.

6 Definitions of Quality Compared. Table 1. Definitions of Quality Compared Harvey and Knight (1996) Bogue (1998) Quality as exceptional Quality as limited supply Quality as perfection or consistency Quality as fitness for purpose (mission) Quality within mission Quality as value for money Quality as transformation Quality as value-added According to Koslowski (2006), in the 1980 s and 1990 s, Quality was defined as excellence or limited supply and assessed according to the internal resources of an institution, such as the number of faculty with terminal degrees, the number of volumes in the library, reputation, endowment, etc. Emphasis was placed on high Quality inputs and producing excellent outcomes. Total Quality Management and Continuous Quality Improvement, approaches borrowed from corporate management paradigms, followed in the late 1990 s. These were replaced in turn by performance-oriented, outcomes-based approaches to Quality that stress identifying and measuring competencies students should gain through a university education.

7 Since then, the most widely accepted definition of Quality is fitness for purpose (Harvey, 1998; 4 Woodhouse, 1999). One can surmise that its appeal stems from its inherent flexibility, which allows institutions to measure Quality in terms of their ability to meet their respective missions and objectives. For many scholars, the lack of agreement surrounding the meaning of Quality in higher education suggests that the concept borrowed from business and industry is ill suited to the educational context. While customer-based definitions of Quality have gained pre-eminence in business, in higher education, they are viewed as problematic for a number of reasons. First, a singular view of Quality is not representative of the varied and sometimes conflicting views of stakeholder groups: The key issue is the ability of the Quality concept to facilitate the perspective of a range of stakeholders who have different conceptions of higher education (Cullen, Joyce, Hassall & Broadbent, 2003, p.)

8 6). For example, in determining whether or not their educational experience has met their expectations, students are most likely to judge Quality as fitness for purpose, while faculty members are apt to measure Quality in terms of inputs and outputs, such as research dollars and productivity, number of publications, number of courses taught, etc., or outcomes such as improved student learning. In contrast, external stakeholders such as government and the public would almost certainly agree that Quality equals value for money and doing more with less. Moreover, because the two university mainstays of research and teaching differ in terms of purpose, process and outcomes, they require different approaches to Quality Assurance (Marshall, 1998). In light of this, it is not surprising that Harvey and Green (1993) suggest that the only practical solution to this complex philosophical question is to recognize and validate all of these diverse perspectives and reject the possibility of accepting a singular definition of Quality .

9 Building on work by Koslowksi (2006), Table 2. Definitions of Quality in Business and Higher Education brings together the definitions of Quality identified above and aligns them with the stakeholder group most likely to support each. Table 2. Definitions of Quality in Business and Higher Education BUSINESS EDUCATION Definition Definition Stakeholders Transcendent Quality results from producer s expertise Exceptional Quality results from expertise of professoriate Faculty Manufacturing-based Product conforms to specifications; Fitness for purpose (mission) Institution is capable of External stakeholders, 5 Table 2. Definitions of Quality in Business and Higher Education BUSINESS EDUCATION Definition Definition Stakeholders fitness for purpose meeting educational aims and objectives accreditation agencies Product-based Quality is determined by the presence or absence of an ingredient Transformative Value-added Linked to assessment; evidence of Quality is increased student learning Accreditation agencies, employers Value-based Acceptable performance at an acceptable price Value for money Limited supply External rankings, such as Macleans Resource orientation Administrators, parents, students User-based Quality defined by consumers needs and preferences Fitness for purpose (customer specification) Outcomes meet specified requirements Students, government (depending on who is identified as the customer ) Within Ontario s Quality Assurance Framework ( Quality Assurance Task Force, 2010), Quality is operationalized in several ways including: (a) fitness for purpose (mission), (b) exceptional, and (c) value-added.

10 For example, the criteria for the evaluation of new undergraduate and graduate programs, which include consistency of the program with the institution s mission and academic plans and clarity and appropriateness of the program s requirements and associated learning outcomes in addressing the institution s own undergraduate and graduate Degree Level Expectations (p. 8), assess fitness for purpose. The view of Quality as exceptional is evident in the measures described in section , Quality and other indicators . These measures, intended to provide evidence of Quality of the faculty and evidence of a program structure and faculty research that will ensure the intellectual Quality of the student experience , include 6 qualifications, research, innovation and scholarly record, and appropriateness of collective faculty expertise to contribute substantively to the proposed program (p. 11). Finally, the Degree Level Expectations included in the Framework , which represent the threshold level skills and knowledge Ontario students must demonstrate in order to successfully complete their programs, measure fitness for purpose at the program level: The Degree Level Expectations, combined with the expert judgment of external disciplinary scholars, provide the benchmarks for assessing a program s standards and Quality ( Quality Assurance Task Force, 2010, p.)