A Competency and Skills Framework for the …

1 A Competency and SkillsFramework for the Assessmentof Software Engineering in theRoyal Air ForceDavid GillJanuary 2005A dissertation submitted in partial fulfilment of therequirements for the degree of Master of Sciencein Software EngineeringKellogg CollegeUniversity of Oxford British Crown Copyright 2005iAbstractThe idea for this dissertation was borne out of a study into military aviation designsupport services that made numerous recommendations, one of which was that a fullreview of software maintenance core competencies should be undertaken. However,what I initially thought would be a simple enough task soon revealed itself to bequite a complex and labour-intensive order to conduct a review of corecompetencies, I first had to revisit the acceptedacademic definitions of Software Engineering (SE), dissecting and examining themin the context of military usage. From this work, I was able to arrive at a newdefinition that I subsequently used to create a SE Activity Model that encompassedboth the development and support then developed a strategy for the creation of a SE Competency and Skills (C&S)Assessment Framework , and subsequently created the Framework itself.

2 This wasby far the longest phase of the dissertation I had completed the C&S Framework , I then trialled it on a military softwaremaintenance team. The results of this trial were encouraging and enabled me tomake recommendations for improving the Framework , potential extensions to its useand suggestions for future work in other dissertation would not have been possible were it not for the patience andsupport of my friends and colleagues. In particularI would like to thank thefollowing for their specific contributions:The staff of the Software Engineering Programme, especially my projectsupervisor, Dr Jim Davies, for his wisdom and Weekes, Jamie Brooks and Ian Grose for their encouragement,constructive criticism and challenging insight. Special thanks should begiven to Jamie for his readiness to thoroughly absorb and discuss my ideasand Ian for taking the time to edit my work and save me from potentialembarrassment!

3 Mike Placeand the personnel of HSMU who took the time to take part in theframework evaluation, giving me valuable feedback in the , the greatest appreciation by far must go to my family and Mary for givingme the strength and impetus to finish this author confirms that this dissertation does not contain material previouslysubmitted for another degree or academic award and the work presented here is theauthor s own, except where otherwise opinions expressed in this dissertation are solely those of the author and can notbe attributed to the Royal Air Force or Ministry of MatterAbstractiAcknowledgementsiiContent siiiMain Software Definitive Process a Model of Software Engineering Software Supporting Activities Supporting Infrastructure Engineering Activity Engineering Assessment Engineering Competencies and and Skill Assessment Framework C&S Assessment of Evaluation the Conclusions This Work Should be Customers77ivBack MatterReferences78 Definition of Terms81 Abbreviations83 Appendix A SE Topics85 Appendix B C&S Mapping to SE87 Appendix C SFIA Skills and Levels95 Appendix D SFIA Level Definitions96 Appendix E C&S Assessment Engineering Modification Activities Infrastructure Management Function144 Appendix F Assessment

4 Pro-forma155 Appendix G Completed HSMU Assessment D166 Appendix H Proposed Statements for Unpopulated Competencies170 TablesTable SE Knowledge Areas, after[SWEBOK, 2001]14 Table Related Disciplines[SWEBOK, 2001]23 Table Life-Cycle Evaluation Stage Comparison of System and SoftwareDevelopment34 Table SW-CMM KPAs and Common Features60 Table SE C&S Assessment Table62 Table HSMU Coupling to SE Activity Model67 Table Software Engineering Topics[Open, 1997]86 Table C&S Mapping to SE Functions and Activities94vFiguresFigure Academic SE Topics6 Figure Software Support Model, based on an idea by [Brooks, 2004]8 Figure RAF Software Support Team Functional Areas12 Figure Basic RAFSST Manning Model12 Figure Model of a Process-Based QMS [ISO 9001, 2000]18 Figure Proposed Data Life-Cycle21 Figure System Life-Cycle23 Figure System Engineering Process[Blanchard, 1998]25 Figure System Evaluation Stages During the Life-Cycle[Blanchard, 1998]32 Figure V Model Software Development Life-Cycle[McDermid, 1994]33 Figure System Activities Related to SM, after[Blanchard, 1998]35 Figure Illustration of Software Users From a Military Perspective41 Figure Illustrative User Class Diagram42 Figure Related Disciplines, after[SWEBOK, 2001]48 Figure Software Engineering Activity Model49 Figure Assessment Guidance[IEE, 1999]52 Figure SE Competency Assessment, after[IEE, 1999]53 Figure SFIA Framework [SFIA, 2004]54 Figure Assessment Guidance[SFIA, 2004]54 Figure Outline SE C&S Assessment Framework57 Figure The 5 Levels of Software Process Maturity[SW-CMM, 1997]

5 58 Figure Assessment Guidance[SW-CMM, 1997]59 Figure SE C&S Assessment Framework Creation Strategy60 Figure SE C&S Assessment Guidance Creation61 Figure HSMU Software Life-Cycle66 Figure HSMU Manning Structure Royal Air Force (RAF) is no stranger to software; indeed, when compared tothe British Army and Royal Navy, RAF defence capability has relied in part uponsoftware-intensive systems since the early 1970s with the introduction of theNimrod R1. It should therefore come as no surprise to the reader that there currentlyexists a number of RAF software maintenance teams, collectively referred to asSoftware Support Teams (SSTs). By and large, these SSTs each maintain a real-time application, effectively complimenting a platform s1 Design Authority2 (DA),and are staffed almost exclusively by military personnel giving them the prefix ofin-Service exact rationale behind the creation of each SST, however, is not altogether clearbut can be surmised from their common tasking activities.

6 Essentially, the existenceof a platform SST affords the RAF a measured degree of control when consideringcertain software maintenance activities that may directly impinge upon theplatform s operational capability and/or availability. In reality, this means that theRAF is relatively free to dictate the level and extent of some platform softwaremaintenance activities undertaken inorder to meet its wider operational commitmentas long as it is willing to accept the risk. As such, most SST modifications aretherefore released as Service Engineered Modifications (SEMs).This tasking freedom is a feature that the RAF frequently exploits as it allows for theproduction of mission-critical software independent of the respective platform s the need for this design contractor approval, the RAF can introduce newfunctionality, correct faults and enhance a system s capability all with typicalmilitary expediency.

7 In essence the RAF has, to a certain extent, takenresponsibility for mission-critical software maintenance in theoperation/maintenance phase of a system s , the individual nature of each SST has essentially meant that there hasbeen little by way of standardisation between them. The reader might be forgivenfor thinking that this in itself should not present too much of a problem. After all, aslong as there is a dedicated team established to support each platform then whatshould it matter if they all do things slightly differently? Indeed, this is analogous tomost similar-product civilian manufacturing organisations (for example the motormanufacturing industry). This argument may well have held true when the SSTswere first formed, when funds and personnel were aplenty and when platforms wereintroduced by multifarious project Generally speaking, supported equipment or Normally the original design contractor for the , problems arise when one considers the in-house nature of SSTs.

8 As theyare effectively owned and staffed almost exclusively by the RAF, consideration hasto be given to the resource provisioning implications of bespoke formation highinitial set up costs, low opportunities for Skills cross-pollination, difficulties incareer progression. Today, withan increasingly diminishing defence budget, theRAF can no longer afford the luxury of uniqueness when considering the support ofsoftware intensive systems. Likewise, as software becomes the all pervasivedeliverer of functionality, it can not afford toignore it either. The key to supportflexibility and mission availability now rests on careful resource management, andin particular one of the most valuable resources an organisation can ever have primary aim of this dissertation is the enabling of a Software Engineering (SE) Competency evaluation of RAF SSTs. Of secondary importance is the enabling ofthe identification and definition of RAF SST manning requirements.

9 It is intendedto achieve both of these aims through the creation and application of a SECompetency and Skills (C&S) Assessment OutlineThis dissertation will comprise a thesis with supporting project work. An outline ofthe project itself is as follows:Defining Software EngineeringSoftware maintenance and SE can have significantly diverse meanings dependent onthe task that they are required to describe. Indeed, even within the academiccommunity itself, there exists a disparity of understanding over the termsthemselves, and to make the assertion that all RAF SSTs undertake one over theother is not as straightforward as you may think. In order to address this problem,however, it is necessary to begin somewhere so I have chosen to analyse SE with theintention of arriving at a common definition and adapting it to the functionalrequirements of the RAF. Once this is achieved, however, it is then a relativelystraightforward case of identifying and defining SE Functions and of a Competency & Skills AssessmentFrameworkOnce the SE Functions and Activities have been defined, the next stage is to producecompetency statements for each.

10 This process requires the formation of a C&Screation strategy based on established sources of information, and then theapplication of this strategy in order to populate a C&S Framework . It is not possible,however, to create statements for all of the competencies listed as this requires adegree of application domain knowledge that I do not possess. In order to overcome3this,I proceed on the basis that I can use the application phase of the frameworkitself to address these elements, drawing on the knowledge of interviewees AssessmentIt would be preferable to trial the Framework on as many SSTs aspossible beforepublishing the final product. However, as dissertation size and available work timeconstraints do not allow for this to happen, I trial it on one SST only. I do notbelieve that this poses too great a handicap, as I am still able to effectively appraisethe shortfalls of the developed Framework assessment process ReviewOnce the practical application of the Framework is complete, I report the results andconduct a critical appraisal of the dissertation.