A Process to Justify e Learning How to

A Process to Justify e. Learning How to Use a Systems Approach for e. Learning e. Valuation Based Upon Enabling Objectives International Society of Performance Improvement Boston - 2003 e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

A Cooperative Team Approach BAYLO R UNIVERSIT e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Meet the e. Valuation Team • Mr. Ara Ohanian, CEO, Vuepoint Learning System aohanian@vuepoint. com • Dr. Jim (Mo) Moshinskie, CPT, Baylor University james_moshinskie@baylor. edu • Ms. Ann Kennedy, e. Learning Director, David Weekly Homes, Akennedy@dwhomes. com e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Meet David Weekly Homes • 8 Southern states • 1, 200 team members • 18 courses developed by Dr. Moshinskie and powered by Vuepoint in past 3 years. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Why Vuepoint: An Integrated System • Content Creator • Learner • Administrator e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Vuepoint Learning System – ISPI – Boston - 2003 e. Learning e. Valuation Model Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

• Short modules – 15 -20 screens (1 minute per screen) • Teach the “need to. Model – ISPI–– link to 2003 to know” e. Learning e. Valuation know” Boston - “nice Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

• Make it as interactive as possible – be creative - fun • Usee. Learning e. Valuation Model – ISPI – Bostonvary pace and appe variety of VLS Templates to - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Screen shot - template • 18 types of interactive templates • Wizard to guide course development fast. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Award Winning Winner of the 2001 Award of Excellence from the International Society of Performance Improvement (ISPI) e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Award Winning 2003 Professional Builder Work. Force Development Achievement Award e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

But Is It Working? • • Do learners like it? Is their knowledge increased? Is it a significant change, or random? Can mangers see a difference? Are house sales up? Are customers happier? What are our lessons learned? e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

What Is Our e. Valuation Process? • Three phases broken down into stages. • Each stage breaks down into procedures (steps) and principles (tips and guidelines) Each Stage Includes: Procedures Principles Exact steps to follow Tips and Guidelines e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Handout: Our Working Document • Our process is divided into: Stages Procedures Principles e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

PHASE 1: BEFORE 1. Select two SME’s. 2. Meet with Dr. Moshinskie to develop the Content. 3. Form Content into a Process. 4. Determine key Procedures and Principles. e. Learning e. Valuation 5. Baylor University – David Weekly Homes. Model – ISPI – Boston - 2003 Black & Decker Write the Enabling – Vuepoint Learning System –

What Are Enabling Objectives? • A short statement of the procedure or principle learner expected to perform. • Start with a simple measurable verb. • Form into an organized list. • Write the questions to them. • Tie blended workshop to them (checklist). • Develop coaching and evaluation checklists. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Type of content Critical? Link to question Numbered • Short • Start with verb • Measurable e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

PHASE 2: DURING • Classify Enabling Objectives (critical or not? ) • Write Pre-Test Questions - Tie to specific critical Enabling Objectives - VLS links Enabling Objectives Screens - Results in a customized Roadmap • Design Each Module using ROPES • Prepare Post Assessment - VLS e. Learning e. Valuation Model –randomization allows pooling, ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Using the ROPES Model R O P E S Review previous topics and give needed prerequisite information Overview Explain enabling objectives and WIIFMs Present Teach enabling objectives interactively Exercise Provide realistic scenarios based on enabling objectives Summari Restate enabling objectives ze e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

SMEs Used the VLS Comment feature for quick editing. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Screen shot - roadmap • Base questions on exact enabling objectives • Customized Roadmap based upon Pre-Test e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Motivate Learners – Give Awards Service Portfolio Sales B TWW Change Email Warranty Communications Builder Selling Safety Houses Diversity OSHA Safety Closing DWH 101 Posting 101 e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

PHASE 3: FOLLOWING 1. Analyze Level 1 data (VLS) 2. Analyze Level 2 (VLS) 3. Conduct Blended Workshops 4. Allow selfassessment 5. Query Managers 6. Correlate Results 7. Analyze Job Performance e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Did learners like the course? e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Role of Blended Workshops • Direct the learners in applying the enabling objectives by teams. • Allow coaches to evaluate learners using checklist (1 -5 scale). • Calculate scores. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Randomly Select Learners • Select 18 -20 learners • Query their line managers. • Survey their customers. (J. D. Powers) • Analyze their job performance. - Quality walk at home sites - Mystery shop sales calls - Videotape performance • Tie results to the enabling objectives. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Self Assessment • Ask e. Learners to evaluate their confidence levels on performing critical enabling objectives • Rank 1 -5 (Strongly disagree to strongly agree) • Include open ended: “Which performance areas do you need specific help on now? ” e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Take the Demo • www. vuepoint. com/vlsdemo/default. aas p e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Statistical Toolbox - e. Xcel 1. Means – What are the averages? 2. Standard Deviations – How consistent? 3. Paired T-Test – Is the difference significant? 4. Pearson Coefficient – Is there a See pages 7 and 8 relationship? of handout. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Statistical Category e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Calculating Early Results Level 1 Average: 4. 6 (out of 5) Level 2 – Average Pre. Test = 73 Average Post Test = 92 Level 3 – Line Managers My employees know more – 4. 5 (out of 5) Qualitative Comments - Too many interruptions - Regional content varies Confidence Level - 89% e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Calculating Early Results Blended Workshop Evaluations Course 1 – 2. 73 out of 5 Course 2 – 2. 71 out of 5 e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Pearson Correlation possibilities? Is there a relationship between: • post-test scores + sales performance? • post-test scores + customer surveys? • blended workshop scores + sales performance? • walk throughs + customer satisfaction? • J. D. Powers surveys + blended workshop scores? e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Return on Investment angles? • • • Fewer warranty calls Fewer defects Fewer re-do’s Fewer repairs Increased customer satisfaction Increased house sales – relate to recession trends • Performance improvement analysis e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Barriers to Success? • Group Exercise e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Avoiding Interruptions CAUTION: Brain Under Construction e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

ROI – Maybe? e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Next: Black & Decker • 15 courses finished using Vuepoint Learning System; 8 more under development. • Modules based upon ROPES. • Combine with hands-on blended training in corporate university workshops in Baltimore. • Collect data based upon our e. Learning e. Valuation model. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Summary • Our e. Learning e. Valuation uses a systems approach for corporations to evaluate e. Learning. • It was developed by a team approach: Client + Consultant + Vendor • It offers a process consisting of 3 phases: BEFORE, DURING, and AFTER • Each phase contains a series of stages. • The key is developing solid enabling objectives. • Using statistical tools helps collect- 2003 e. Learning e. Valuation Model – ISPI – Boston defendable Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

Want to Learn More? Dr. Jim Moshinskie’s Website on e. Learning http: //business. baylor. edu/james_moshi nskie On-site Workshops available - Converting Content into e. Learning: ROPES. - How to e. Valuate e. Learning with Confidence. - FLASH-MX Basics Boot Camp. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker

e. Learning Contract Work • Dr. Jim Moshinskie • Office: 254 -710 -4097 • Cell: 254 -744 -7934 We accept corporate e. Learning development projects. e. Learning e. Valuation Model – ISPI – Boston - 2003 Baylor University – David Weekly Homes – Vuepoint Learning System – Black & Decker