Issues in testing business English

1 Issues in testing businessEnglish The revision of the Cambridge BusinessEnglish CertificatesAlso in this series:An investigation into the comparability of two tests of English as a Foreign Language: TheCambridge-TOEFL comparability studyLyle F. Bachman, F. Davidson, K. Ryan, ChoiTest taker characteristics and performance: A structural modeling approachAntony John KunnanPerformance testing , cognition and assessment: Selected papers from the 15th Language TestingResearch Colloquium, Cambridge and ArnhemMichael Milanovic, Nick SavilleThe development of ielts : A study of the effect of background knowledge on readingcomprehensionCaroline Margaret ClaphamVerbal protocol analysis in language testing research: A handbookAlison GreenA multilingual glossary of language testing termsPrepared by ALTE membersDictionary of language testingAlan Davies, Annie Brown, Cathie Elder, Kathryn Hill, Tom Lumley, Tim McNamaraLearner strategy use and performance on language tests: A structural equation modellingapproachJames Enos PurpuraFairness and validation in language assessment.

2 Selected papers from the 19th Language TestingResearch Colloquium, Orlando, FloridaAntony John KunnanIssues in computer-adaptive testing of reading proficiencyMicheline Chalhoub-DevilleExperimenting with uncertainty: Essays in honour of Alan DaviesA. Brown, C. Elder, N. Iwashita, E. Grove, K. Hill, T. Lumley, K. O Loughlin, T. McNamaraAn empirical investigation of the componentiality of L2 reading in English for academic purposesCyril WeirThe equivalence of direct and semi-direct speaking testsKieran O LoughlinA qualitative approach to the validation of oral language testsAnne LazaratonContinuity and innovation: Revising the Cambridge Proficiency in English Examination1913 2002 Edited by Cyril Weir and Michael MilanovicEuropean language testing in a global contextEdited by Cyril Weir and Michael MilanovicA modular approach to testing English language skills: The development of the Certificates in English Language Skills (CELS) examinationsRoger HawkeyChanging language teaching through language testing : A washback studyLiying ChengUnpublishedThe impact of high-stakes examinations on classroom teaching: A case study using insights from testing and innovation theoryDianne WallImpact theory and practice.

3 Studies of the ielts test and Progetto Lingue 2000 Roger HawkeyIssues in testing businessEnglish The revision of the Cambridge BusinessEnglish CertificatesBarry O SullivanivCAMBRIDGE UNIVERSITY PRESSC ambridge, New York, Melbourne, Madrid, Cape Town, Singapore, S o PauloCambridge University PressThe Edinburgh Building, Cambridge CB2 2RU, on this title: UCLES 2006 This publication is in copyright. Subject to statutory exceptionand to the provisions of relevant collective licensing agreements,no reproduction of any part may take place without the written permission of Cambridge University published 2006 Printed in the United Kingdom at the University Press, CambridgeA catalogue record for this publication is available from the British LibraryISBN-13 978-0-521-01330-7 paperbackISBN-10 0-521-01330-5 paperbackThe author is grateful to the copyright holders for permission to use the copyrightmaterial reproduced in this book.

4 Every effort has been made to trace the copyrightholders. Cambridge University Press apologises for any unintentional omissions andwould be pleased, in such cases, to add an acknowledgement in further 2006To MauravviiContentsAcknowledgementsixSerie s Editors notexAbbreviationsxiiChapter 1 Introduction to the testing of language for business purposes1 Chapter 2 The revision of BEC82 Chapter 3 Major changes to the suite119 Chapter 4 Changes in the BEC papers130 Chapter 5 Conclusions and the way forward174 AppendicesAppendix : JOCT Evaluation Criteria196 Appendix : CEFLS Pilot Test197 Appendix : CEIBT Test of Reading and Writing June and November 1992205 Appendix : BULATS Standard Test English225 Appendix : BULATS Speaking Test241 Appendix : BULATS Standard Test German244 Appendix : ALTE Work Typical Abilities260 Appendix : BEC 1 Sample Paper261 Appendix : BEC 2 Sample Paper277 Appendix : BEC 3 Sample Paper291 Appendix.

5 BEC Preliminary Sample Paper302 Appendix : BEC Vantage Sample Paper328 Appendix : BEC Higher Sample Paper353viiiContentsAdditional information on tests of language for business purposes378 References381 Index393 Acknowledgements I would like to thank the many people at Cambridge ESOL who contributed tothis volume. I was lucky enough to be able to interview many of the individualswho were involved in the development and administration of their tests ofEnglish for business purposes, particularly the BULATS and BEC groups. Inparticular, I am indebted to Hugh Bateman, without whom the book could nothave been completed. Others I would like to single out include Mike Milanovicand Nick Saville, who provided historical information and access to internalreports, and Neil Jones, who provided documentation on the grading proceduresfor BEC. I would also like to thank David Thighe who clarified a number ofgrading-related Issues , and who made valuable comments on the later drafts ofthis book.

6 Finally, I would like to thank Rowena Akinyemi for all her work inensuring that this book made it to would also like to thank the following individuals and institutions forproviding information on their tests and for giving their permission for itemsfrom the tests to be included in the book: Certificazione della conoscenza dell italiano commerciale (Certificate in Italianfor Commerce CIC): Professor Giuliana Grego Bolli and Francesca Parizzi ofthe Universit per Stranieri di tests: Professor Kiyokata Katoh of Tokyo Gakugei University andReiko Kimura of the Japan External Trade tests: Glyn Jones of City and Guilds, ESOL tests: Dr Mike Milanovic of Cambridge on page 101: reproduced by permission of Oxford University Press fromOxford Applied Linguistics: Fundamental Considerations in Language Testingby Lyle F Bachman Lyle F Bachman , I would like to thank Professor Cyril Weir who read various parts of thebook and offered invaluable critical comments and Editors noteThe language testing world has flirted with the testing of English for specificpurposes for many years.

7 In small scale testing contexts there have been andcontinue to be numerous specific assessments tailored to particular needs but inthe context of large scale international language testing , specific purpose assess-ments have been far less common. Cambridge ESOL started testing English in 1913. In some ways you mightconsider the original Certificate of Proficiency in English (CPE) a specificpurpose examination designed to meet the needs of those teaching Englishalthough over the years it has become far more general in its ESOL also collaborated with the British Council on the devel-opment of the English Language testing Service (ELTS) in the 1970s which hadsix subject specific modules. ielts , which evolved from ELTS in 1990, saw areduction in the academic modules to 3 and the 1995 revision of ielts led to thesingle academic module alongside a general training are many reasons for this retreat by ielts , both practical andtheoretical.

8 Developing numerous multiple modules in the quantities requiredwas a far from easy task. Ensuring that candidates took the right module wasdifficult. Equating modules proved technically very demanding. Ensuringcontent appropriateness required access to experts in a number of fields and soon. However, the need to extend the remit of General English assessmentremains. The context of Cambridge ESOL English language assessmentcurrently falls into four broad categories. Academic English ( ielts ), BusinessEnglish (BEC and BULATS), Young Learners English (YLE) and GeneralEnglish (KET, PET, FCE, CAE, CPE and CELS) and in 2006 Cambridge ESOL will launch the International Legal English Certificate. This test seeks to addressmore specifically English in the legal domain of use. In Issues in testing business English , Barry O Sullivan provides a frameworkfor classifying and understanding specific purpose language assessment.

9 Thefirst part of the volume provides the reader with a comprehensive review ofnumerous business English tests as well as business language tests in otherlanguages. Some of the tests described no longer exist so the volume also servesas a useful historical record. This is followed by a detailed look at the revision ofthe Cambridge business English Certificates (BEC).Chapter 1 considers the relationship between general English and English forspecific purposes and the definition of a business English construct. O Sullivanpresents a continuum ranging from an unspecified purpose to one that is highlyxispecified. To this he adds construct, test method, skills coverage measurementqualities, degree of specificity/authenticity, non language factors and thereporting of test performance. This provides him with a framework forcomparison and he proceeds to evaluate a series of business language tests onthis basis.

10 Particular attention is paid to some very widely used tests such asTOEIC, BULATS and BEC although the coverage of less widely known assess-ments is comprehensive. The text is illustrated with numerous examples of testitem types which make interesting reading. Having provided a detailed context against which to understand BEC, subse-quent chapters consider BEC s revision and look in detail at each of the threeBEC levels. The discussion of development methodology is interesting as is thefocus on test reliability. It is gratifying to note that an examination like BEC,operating on a truncated sample of the test taking population at each of its threelevels, demonstrating very good construct and content validity features andusing a good variety of realistic material with an authentic orientation, cannonetheless achieve respectably high reliability estimates. Throughout thisvolume readers are referred to Volume 15 (Weir, Cyril and Milanovic, Michael(Eds) (2003) Continuity and innovation: Revising the Cambridge Proficiency inEnglish Examination 1913 2002 ) in the same series which gives an even moredetailed account of the principles that underline the Cambridge approach to testdevelopment and validation.