Costs of e-learning in HE — and the

Costs of e-learning in HE - and the benefits Professor Paul Bacsich Sheffield Hallam University, UK

Methodological foundation “Suitably interpreted and broadened, and leavened with pedagogy, the rather dry methods of business costing and planning can provide a sound basis for analysing e-learning developments and their impact. ” 2

Summary ¨ CNL Phase 1 [JISC JCALT] ¨ CNL Phase 2 [JISC JCALT] ¨ Effectiveness: impact of other work including analysis of the training sector [EU Telelearn etc] and the NLN Evaluation work [FEDA] ¨ Some impact from related work including RCIT (CNL methods applied to admin IT systems) [JCALT] and soon perhaps from “Managing Change” (formerly TLTR) on change management in HEIs [JCIEL] 3

Main issues What learners want - must be first! Costs Effectiveness 4

What learners want. . . ¨ From our own work on learners (CNL): ¨ ¨ students believe that e-learning increases costs to them but they behave as though it saves costs ¨ and saves time! 5

What learners want ¨ From our own work on learners (CNL): ¨ ¨ students believe that e-learning increases costs to them but they behave as though it saves costs ¨ and saves time! ¨ From other work (including USA): ¨ ¨ they want flexibility (but not too much) some want overclocking (ie faster throughput) 6

Hidden Costs - examples ¨ Increased home telephone call bills for students due to Internet usage ¨ Entertainment expenses incurred by lecturers but not reimbursed by the HEI ¨ The usage of teaching or research budgets to bolster IT budgets - or vice versa 7

CNL 1

Conclusions - CNL 1 ¨ In order to accurately record the Costs of Networked Learning you must have a universally accepted method of what costs to record and how. ¨ Much of the literature formed interesting background reading but failed to travel far enough towards operational conclusions to be taken (individually) as a basis. ¨ The method must include a lifecycle model, stakeholders, activity based costing and a “costaware” approach to planning. 9

Course Lifecycle Model Planning and Development Maintenance and Evaluation Three-phase model of course development Production and Delivery 10

Breakdown of threephase model 11

The main Stakeholders ¨ The HEI ¨ and the staff of that HEI ¨ The learners 12

Stakeholders two more special cases ¨ The HEI ¨ and the staff of that HEI ¨ The learners ¨ and their organisation and in some cases their parents/partners 13

The Financial Schema - cf Transparency Review ¨ The traditional financial model underpinning work in most HEIs is defective in four ways: No Stakeholders No agreed set of activities Crude Overhead Allocation No time recording 14

Planning Document and Financial Schema Multi-level Activity-Based Costing Flexible Overheads Three-phase Course Lifecycle Model Based on previous works Multi-stakeholder Time recording Time Division 15

Activity-Based Costing

Activity-based costing (ABC) ABC was developed as an alternative costing methodology by Robin Cooper and Robert Kaplan of the Harvard Business School (1988) in their research into product costing in the manufacturing industry.

Cooper and Kaplan (1988) ¨ They argued that traditional costing distorted product costs (“peanut butter spread approach”) ¨ There is a cost to all activities carried out within an organisation ¨ Activity costs should be distributed to products in relation to their use 18

ACTIVITY DRIVERS method Staff time Activities Hold Tutorials Resources Cost for each Customer/product/channel group FT Business Intelligence FT I. T. & Management DL I. T. & Management Cost driver No. Students

Definitions A series of activities required to achieve an outcome Invoicing Process Activity A series of related tasks carried out repeatedly Chase a student for late payment Task One action by one person at one point in time Telephone student Process

Advantages of ABC ¨ Gives more than just financial information ¨ Fairer system of overhead allocation ¨ Accountability of central services ¨ Highlights cross-subsidisation ¨ Recognises the changing cost behaviour of different activities as they grow and mature ¨ Can provide data for other initiatives 21

Disadvantages of ABC ¨ Data collection is costly and time consuming ¨ Initially it can be difficult to collect the data you want ¨ Determining appropriate cost drivers ¨ More complex system 22

Implementing ABC takes time “Its important in a service organisation to get started with ABM and not worry about total accuracy from the start. Accuracy will improve with time” Antos (1992) 23

CNL 2 ¨ This was a trial of ABC in one faculty of Sheffield Hallam University, using industrial-strength ABC software and consultancy (Armstrong-Laing) ¨ The trial was funded by JISC JCALT ¨ The report of the trial (plus workbook) will be available in August 2001 ¨ The full set of costs are not reported! 24

Tiptoeing into effectiveness issues

History ¨ Relatively early in the history of CNL 1, effectiveness issues were looked at (Ash, November 1999) ¨ Main conclusion was that a uniformly accepted methodology was needed there also; and some suggestions were given, based to a certain extent on the US Flashlight work. ¨ This work led into the NLN evaluation studies (English FE sector) 26

Effectiveness: a big “border” between E & T ¨ The NSD effect in academia (No Significant Difference) ¨ versus learning gains and large Return on Investment in training world: ¨ ¨ ¨ 700% ROI at Royal Bank of Scotland (Shepherd) routine 10 -25 -40% learning gains and situations with 60% learning gains 27

NSD in academia. . . ¨ It is commonly believed, and apparently validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students ¨ The Web site teleeducation. nb. ca/nosignificantdifference/ rounds out and updates this earlier work. 28

NSD in academia ¨ It is commonly believed, and apparently validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students. ¨ The Web site teleeducation. nb. ca/nosignificantdifference/ rounds out and updates this earlier work. ¨ But we contend that the NSD work has several major flaws - in particular many key variables were not controlled. 29

Self-incrimination? ¨ “There were no significant pre- and postcourse differences on the items related to students’ expectations and perceived benefits. . . ¨ There were no significant differences between students in the [online] compared to students in the [face to face] courses for course retention or course grade. . . ¨ but there were increases in the amount of time and the purposes for which students used computers. " (Green et al 2001) 30

So is a key difference to do with time? ¨ And what is time? 31

What is time? ¨ “Time is the new distance” (Mason, inaugural lecture 2001) 32

Can e-learning save time? ¨ travel time 33

Can e-learning save time? ¨ travel time ¨ reduce “time on task” ¨ by “better” training (25 -40%) ¨ by individualised training. . . 34

Can e-learning save time? ¨ travel time ¨ reduce “time on task” ¨ by “better” training (25 -40%) ¨ by individualised training. . . ¨ by using “time of the 3 rd kind” ¨ by consolidating time fragments 35

Time of the 3 rd kind ¨ Time 1 : on duty - used to be called “at work” ¨ Time 2 : off duty (not at work) 36

Time of the 3 rd kind ¨ Time 1 : on duty - used to be called “at work” ¨ Time 2 : off duty (not at work) ¨ Time 3 : in-between ¨ on duty but less productive ¨ ¨ ¨ eg travelling eg slots in between other tasks off-duty but somewhat productive ¨ ¨ eg in bar with colleagues at dinner with customers 37

One key conclusion ¨ To understand effectiveness issues we need a much better understanding of time ¨ and its monetary and other value to students ¨ Only when one understands time should one move on to other measures of effectiveness 38

Details about our work at: http: //www. shu. ac. uk/cnl/ Paul Bacsich Professor of Telematics School of Computing and Management Sciences Sheffield Hallam University