The Real Costs of e-Learning What are they

The Real Costs of e-Learning What are they? How can we find out? Professor Paul Bacsich UK e. Universities Worldwide Limited Presentation to Venezualan Delegation, 16 March 2004 1

Contents ¨ Key issues in a borderless world ¨ UK Transparency Review ¨ Activity Based Costing [CNL 2] ¨ Activities for Networked Learning [CNL 1] ¨ Application to e-learning 2

Key issues and a challenge ¨ ¨ There is now increasing agreement on the methodologies of costing e-learning So why is there a dilemma where education see “No Significant Difference” whereas training sees “Return on Investment”? The challenge is to find a uniform evaluation methodology, including costs, which copes with a world without borders… Borders are not only geographic 3

UK Transparency Review ¨ HEFCE Initiative - and other Funding Councils ¨ Focus on costs of research and teaching ¨ Compulsory ¨ Reports on 5 main activities: T 2 R 2 O ¨ Uses “Activity Based Costing” terminology ¨ Uses Activity Time Sheets ¨ Phased implementation, mainly led by research-intensive universities 4

Transparency - conclusions It has been a useful if painful exercise for universities and funding agencies ¨ Pressure is increasing on research funding agencies to raise the overheads they allow as real costs become clear BUT, in reference to the “eternal dilemma” ¨ It does not seem to have led to operational conclusions ¨ Because it is not detailed enough? ¨ 5

Scope of CNL 2 project ¨ ¨ ¨ Sarah Heginbotham (SHU then UKe. U then Canada) Funded by JISC (Joint Info Systems Cttee) From CNL 1: Recommendation that activitybased costing be trialled in a university Chose the School of Computing and Management Sciences at Sheffield Hallam University, UK 6 months project , started summer 2000 Transparency Review was going on also 6

Cooper and Kaplan - ABC ¨ ¨ Developed in 1988 as an alternative costing methodology at Harvard Business School They argued that traditional costing distorted product costs ¨ ¨ There is a cost to all activities carried out within an organisation ¨ ¨ “peanut butter spread” approach e. g. transport of peanuts Activity costs should be distributed to products in relation to their use of resources 7

Advantages of ABC ¨ Gives more than just financial information ¨ “Fairer” system of overhead allocation ¨ Accountability of central services e. g. IT ¨ Highlights cross-subsidisation ¨ ¨ Recognises the changing cost behaviour of different activities as they grow and mature Can provide data for other initiatives ¨ e. g. Transparency Review and Quality 8

Disadvantages of ABC ¨ ¨ Data collection can be costly and time consuming Initially it can be challenging to collect the data you want Need to determine appropriate and acceptable cost drivers ABC is a more complex system than traditional management accounting ¨ and relies on that being done well ! 9

CNL 2 Conclusions ¨ ¨ ¨ ABC does work in HE and will be increasingly used in institutions, especially “ABC-lite/ABM” It works on different levels within the institution Suitable software and professional support is essential, in the early days at least ¨ Academic scepticism can be overcome ¨ Costings data needs to be placed in context 10

Networked Learning - CNL 1 ¨ Paul Bacsich ¨ ¨ ¨ and Charlotte Ash (now in public sector “Best Value” programme) Sheffield Hallam University, UK 6 month study (in 1999 -2000) Funded by JISC Aim - to produce a planning document and financial schema that together accurately record the Hidden Costs of Networked Learning 11

Examples of Hidden Costs ¨ ¨ ¨ ¨ Increased telephone call and printing bills for students due to Internet usage Entertainment expenses “necessarily” incurred by academic staff at conferences but not reimbursed by the Institution Administrator time answering student queries Support costs of a new Learning Environment Costs of content - “created in one’s spare time” Costs of institutional collaboration Costs of conformance to standards 12

Mega-activities for teaching: Lifecycle model ¨ ¨ ¨ Cost items recorded from literature All current educational and training costing models analysed (Rumble, Bates, etc) Traditional financial accounting analysed Consultation with academic staff Various cyclic models proposed, leading to. . . 13

Course Lifecycle Model Three-phase model of course development Planning and Development Maintenance and Evaluation 14 Production and Delivery

Breakdown of threephase model (activities) 15

Financial Schema “Stakeholders” 16

Conclusions from CNL 1 In order to accurately record the Costs of Networked Learning you must have a sectorally-accepted method of what costs to record and how, in place from the start, which takes into account all stakeholders and uses Activity Based Costing within a framework of phases of the course development life cycle 17

What are the True Costs? ¨ ¨ Q: What is the rate per study hour? Different kinds of study hours: ¨ ¨ ¨ Active engagement with specifically created “teaching” material (expensive) Thinking, conversation Doing assessment Studying “resource” e-content (cheaper) Surfing the open web (cheapest, but worst? 18

Rough guide - rule of thirds ¨ ¨ 1: Study of “engaging” multimedia 2: Study of existing or slightly modified learning resources 3: Working on assignments (maybe in collaboration) Costs (US, UK and Germany) ¨ ¨ ¨ average $12000 per m/m hour, but v. large 10% of this for simple material could one aim for $1000 per hour? 19

Content development 1 ¨ Determine the level of demand ¨ Produce an outline plan of the courseware ¨ Conceptualise the subject knowledge ¨ ¨ Structure and design the learning experience to be provided by the courseware materials Decide interactivity with the learner, the level and ways of building tutorial support into the material itself and the links with other materials. 20

Content development 2 ¨ ¨ Decide the formative assessment exercises and the ways of providing feedback to the learner (and to the tutor if appropriate) Decide the summative assessment associated with each module Establish a range for the possible levels of on-line tutor support for each module. Consider the interface between the learning programme and its sub-parts and the administrative support software tools 21

Consortia - Critical Success Factors ¨ High “binding energy” ¨ ¨ ¨ ¨ may come from top-down or funding-driven but more likely from friendship or shared vision does not depend much on legal aspects Organisational homogeneity or managed diversity Linguistic and cultural similarity Stratification “Safely far away” [Source: TL-NCE, UNESCO report] 22

Other topical questions ¨ Will research advances reduce costs? ¨ Will Open Source reduce costs? 23

Research ¨ ¨ This may be too much of a personal view I have been conference organiser, evaluator, reviewer, . . . Look at impact from EU research work Look at impact of work elsewhere ¨ ¨ UK TL-NCE (Canada) US Australia, New Zealand, Singapore Hong Kong…. 24

Research conclusions ¨ ¨ ¨ European research: FP 3 set the scene; FP 4 added little, FP 5 more promising Canadian work credible and integrated, but lacks evidence of scalable approaches Too much gap between computing theorists and industrial-strength pedagogic practice ¨ ¨ few theorists in universities are seriously active in e-learning services US is very synchronous and transmissive, paradigms that do not fit a time-poor world 25

What research might reduce costs? ¨ Focus on co-operative learning ¨ ¨ ¨ Develop scalable approaches ¨ ¨ ¨ Start with basic asynchronous “BBS” model Allow new models to be supported, especially those with business potential more focus on automated assessment? Support multiple media and devices using same content Develop more effective learning esp time-effective 26

Can e-learning save time? ¨ ¨ travel time reduce “time on task” by “better” training (25 -40%) by individualised training. . . by using “time of the 3 rd kind” ¨ ¨ ¨ Time 1 : on duty - used to be called “at work” Time 2 : off duty (not at work) by consolidating time fragments 27

Open source issues ¨ Exemplars: ¨ ¨ Purpose: ¨ ¨ ¨ Linux, MIT, Canadian, Finnish, IMS, UK interest Challenge commercial vendors Facilitate research by providing flexible system Cheaper? ¨ ¨ ¨ Yes, to buy No, to support (scalability, robustness? ) Few successful large e-learning players use “pure” open source, but several use modified 28

Features of a uniform cost -aware evaluation methodology ¨ ¨ ¨ Costing basis must be clear To gain political buy-in and breadth (multisector, etc), depth will be small Students treated as people not only learners Focus on level of knowledge and skill gained Focus on time issues, not only time on task ¨ ¨ time 1, time 2, time 3 - and differential value? More feasible when there is a more uniform curriculum or exams (as in schools) 29

Details about CNL costings projects At http: //www. shu. ac. uk/cnl/ thanks to support from SHU and JISC Professor Paul Bacsich pbacsich@ukeu. com 30