Mobile Computing and Technology Assessment: A Case Study with Results Stephen G. Landry, Ph. D. Chief Information Officer Heather Stewart, Ph. D. Assistant Vice President, Finance and Technology Copyright Stephen G. Landry and Heather Stewart, 2003
Seton Hall University Who Are We? • We Are A Mid Sized, Private, Catholic Affiliated University in Central New Jersey • Carnegie Classification: Research / Doctoral II • Main Campus in South Orange, NJ – 15 miles from New York City • 10, 000 Students – 4, 400 Full Time Undergraduates (50% Residential) • 350 FT Faculty (450 FTE Faculty) • FY’ 01 Annual Operating Budget Approx. $150 million – FY’ 00 G&E for South Orange Campus Approx. $115 million
Pace Of Change at Seton Hall University • Before 1995: – Inadequate Computer Labs – Inadequate Local Networks/E-mail – Lack of Integration of Technology in Teaching – Disorganized Support/Allocation of Resources
Pace Of Change (Cont. ) Now: – Seton Hall University is doing IT “right” • 2000 EDUCAUSE Award (Honorable Mention) for Systemic Progress in T&L with Technology • 1999 EDUCAUSE Award for Campus Networking Excellence • In Top 50 of Yahoo! Internet Life Survey of “Most Wired” Universities • Growing National Reputation of our Mobile Computing Initiative and other Teaching, Learning, and Technology Initiatives • Alliance with IBM Corporation • Site Visits by National/International Universities • Presentations at National Conferences/Workshop
A Vehicle for Change: University Strategic Planning • Seton Hall University has accomplished heightened recognition among top-tier Catholic Universities nationally • The strategic goals include a critical focus on the intellectual, personal and spiritual development of all students • Students experience a rigorous, value-centered and technologically enhanced environment
Advancing Strategic Goals with Technology • To advance the strategic goals, a strong focus was placed on communication, teaching and learning as well as support services for students using technology • It was recommended that a strategic agenda include an Information Technology Strategic Planning process to build an IT Long-Range Plan
Seton Hall University How Did This Happen? • Began With Development of IT Plan in 1995 – Part of the University’s Strategic Planning Process – Cross Functional Team • Included Faculty and Administrators • Given Half-Time Release to Work on IT Plan • Intensive Immersion in IT Issues, Trends, Directions – Executive Sponsorship/Commitment to Implement • Chief Academic Officer/Chief Financial Officer – Involvement of User Community • Focus Groups/Surveys • Involvement of Faculty and Administrator Constituency Groups (e. g. , Academic Computing Advisory Committee, Administrative Computing User Groups, etc. ) – Technology Planning Consultant from IBM Education
Seton Hall University’s Strategic Technology Plan • Our Technology Strategy: – Seton Hall University will develop and implement a • learner-centered, • network-centered, • distributed (mobile) learning environment – The core of our distributed learning environment will be a robust set of digital information resources (e. g. , a “digital library”) in support of teaching, learning, scholarship, and institutional transformation
SHU’s Strategic IT Plan • Implications of the IT Plan: Focus on student experience – Students Want / Need: • • • Ubiquitous Access to Technology University-wide Hardware and Software Standards University-wide Network Central IT Support Services Greater Curricular Integration of Technology More Efficient Business Processes / Greater Access to Institutional Information and Services / Bringing Together Academic and Administrative Computing
Implementing the Strategy • In order to implement the strategy, the University undertook a number of teaching, learning, and technology initiatives: – Mobile (Ubiquitous) Computing Program – Seton. World. Wide (Online Professional Programs) – Web-enabling Enrollment Services – Reorganization of Computing Services • Created New User Services Organization, Including a Teaching, Learning, and Technology Center – Major Technology Infrastructure Upgrade • Gigabit Ethernet Campus Backbone; Server Consolidation; Campus-wide Email System; 802. 11 b Wireless; Internet Upgrade; etc.
Mobile Computing at Seton Hall University • Seton Hall University’s Mobile Computing Program is an innovative academic program involving three components: – Access: The University licenses the use of a laptop computer to all undergraduates as part of their tuition and fees – Curricular Integration: The University provides support and incentives to faculty to use technology in innovative ways to enhance teaching and learning – Network and Support Services: The University provides the infrastructure and support services that enable the effective use of technology in teaching and learning
Mobile Computing (cont. ) • Current Model: IBM Think. Pad R-series computer – 1. 2 MHz PIII; 14” TFT Screen; 256 Mb RAM; 30 Gb HDD; CDRW; Built-in 802. 11 b wireless networking; 10/100 Ethernet; 56 Kb Modem; Li. ON Battery – Computer is replaced every two years • Current Technology Fee: $700 per semester for students in the program (“non mobile” students, e. g. , many graduate students, pay a technology fee of $200 per semester) • Bundled software includes: MS Windows ME, MS Office 2000, SPSS, Maple V, various utilities • Bundled services include: – Technology Help Desk, PC Repairs, Loaner Computers – Network Services, including wireless network access from most academic and public spaces
Mobile Computing (cont. ) • Program Phase-In – 1995 – 1997: Pilot Programs • Focus on Teaching and Learning • Based on cohorts with common academic experiences (e. g. , Business Honors Program in 1995, expanded to all 1 st Year Business, Biology, and Honors students in Fall 1997) • Began Teaching, Learning, and Technology Center and internal grant program to provide incentives to departments willing to integrate technology into large enrollment core courses
Mobile Computing (cont. ) • Program Phase-In (cont. ) – 1998 – 1999: Full Roll Out of Program • Program is Mandatory for ALL Incoming FT Undergraduates • Focus on Infrastructure • Major Network / Server Upgrades – Lotus Notes Email / Lotus Learning. Space Course Management System – MS Front. Page Enabled Web Servers for Faculty and Students – ATM Backbone / Campus Internet Upgrade / Server Consolidation / Y 2 K Preparation
Mobile Computing (cont. ) • Program Phase-In (cont. ) – 2000 – Present: Enterprise Perspective • Focus on Enterprise Systems – Implementation of Web services for students (e. g. , online registration, course audit, grades, etc. ) – Automation of Mobile Computing processes (e. g. , asset management, account generation, etc. ) – Implementation of Black. Board course management / portal system; 24/7 support services through Eduprise – Campus-wide wireless networking / Campus network upgraded to gigabit Ethernet backbone
Mobile Computing (cont. ) • Major investment on the part of the University – Central IT Budget (incl. laptop leases /student fees) for FY’ 02 approx. $15 million or 13% SO Campus G&E (10% of the total University budget) • Almost equally divided between personnel, operating, and Mobile Computing Program (technology fee) budgets • Most costs are operationalized / very little capital funding • Seton Hall University is a different place because of the University’s strategic initiatives (enrollments up, SAT’s up, new programs launched, etc. ) • Mobile Computing Assessment Project indicates the program has had positive impact on learning / student recruitment
• Value of Seton Hall Results - DATA • Underlying, Theory and Process • Technology Assessment in Larger Institutional Context
Charge of the Mobile Assessment Team • Assess the impact of mobile computing on fall 1998 entering freshman • This includes student satisfaction with the learning environment and student learning outcomes
Full Report and Resources Available On-Line at: http: //tltc. shu. edu/initiatives/assessment/index. html Key Researchers and Co-Authors Eric Fountain, Ed. D. John Collins, Ed. D. Janet Easterling
Circles of Assessment Students Faculty Institution External
When Assessment Occurs Before Determine Needs During Make Changes in Process After Evaluate Experience
Examples of Assessment Topics Faculty Students Institution • Technical Skills • Instructional Goals • Technical Skills • Enrollment Decisions • Community Engagement • Resources • Support • Time Distribution • Access • General Satisfaction • Bottlenecks • Network Stability • Info. Access • Communication • Technical Skills • Delivery Change • Evaluate Tech. Options B • Training Needs • Tool Fits Goal D A
Types of Assessment • • • Focus Groups Interviews Journals Observations Portfolios Performances • • • Standardized Tests Surveys Tracking Systems Longitudinal Records Document Reviews Skills Application Demonstrations
Narrowing the Question Fantasy: a+b=c a = teaching, b = technology, c = learning Reality: a + b + c + d + e + f… discipline, teaching style, technology proficiency, fit of application to task, desired learning objectives. . .
Faculty Journals (1997) • “Cognitive, behavioral and affective sides of transition” • “Technology has become a natural part of my teaching” • “Be prepared to punt” • “A year ago I couldn’t imagine using all this stuff: now I can. ” • “Technology is only, alas, a dumb machine that is no more creative, inventive, or productive than the human using it”
Technology Functions • • Edit Collaborate Create Compose Automate Visualize Simulate • • Communicate Store Structure Practice Retrieve
Seven Principles (Chickering and Gamson, 1989) • Contact between student and professor • Cooperation among students • • • Active learning Prompt feedback Time on Task High Expectations Respect diversity in talents and learning
Nature of Learning (AAHE, NASPA, 1998) • Making Connections • Compelling Situation • Active search for meaning by learner • Developmental, cumulative process • Individual learners are social beings • Strongly affected by educational climate • Rapid, rich, frequent feedback • Much is informal and incidental • Requires transfer of specific knowledge • Ability of learner to monitor own learning
Evaluation of General Satisfaction, Technology Use, and Student Perceptions of the Impact of Mobile Computing on the Learning Environment at Seton Hall University Spring 1998 – Fall 2000
Independent Variables Gender R 1 a. Does a student’s gender have an effect on their perceptions regarding the satisfaction of the SHU Mobile Computing Program? R 2 a. Does a student’s gender have an effect on their perceptions regarding the use of computers in the SHU Mobile Computing Program? R 3 a. Does a student’s gender have an effect on their perceptions regarding the desired outcomes of the SHU Mobile Computing Program?
Independent Variables Housing Status R 1 b. Does a student’s housing status have an effect on their perceptions regarding satisfaction of the SHU Mobile Computing Program? R 2 b. Does a student’s housing status have an effect on their perceptions regarding the use of computers in the SHU Mobile Computing Program? R 3 b. Does a student’s housing status have an effect on their perceptions regarding the desired outcomes of the SHU Mobile Computing Program?
Independent Variables Ethnicity R 1 c. Does a student’s ethnic affiliation have an effect on their perceptions regarding satisfaction of the SHU Mobile Computing Program? R 2 c. Does a student’s ethnic affiliation have an effect on their perceptions regarding the use of computers in the SHU Mobile Computing Program? R 3 c. Does a student’s ethnic affiliation have an effect on their perceptions regarding the desired outcomes of the SHU Mobile Computing Program?
Limitations • Structural • Use of Mobile vs. Non-Mobile Construct • Neutral Category • Administrative • Sampling • Change from paper to online.
RESULTS
Response Rates Term Target Popn: #FR Target Popn: #Non-FR/U (returning students) 365 (20+20+325) Survey Responses #Non-FR/U (Non-FR/U Resp. Rate) 90 (25%) Target Popn: Comb. Total Survey Responses (Comb. Resp. Rate) 1591 414 (26%) Fall 1998 1226 Survey Responses #FR (FR Resp. Rate) 324 (26%) Spring 1999 1226 216 (18%) 365 (20+20+325) 124 (34%) 1591 340 (27%) Fall 1999 1070 311 (29%) 1591 (365+1226) 453 2661 764 (29%) Spring 2000 1070 179 (17%) 1591 (365+1226) 2661 197 ( 7%) Fall 2000 1125 235 (21%) 2661 (1591+1070) 476 3786 711 (19%) Totals – Fall Terms Only 3421 870 (25%) 4617 1019 (22%) 12, 290 2, 426 (20%) All 5 Terms (28%) 18 (01%) (18%)
Response Demographics Gender Ethnicity Housing Status Fall Terms Only 46% FR Only Male N=870 35% Non-White 39% or Commuters other/(missing) SHU Freshmen 48% (Fall ’ 98, ’ 99, ’ 00) Male 52% Non-White 37% or Commuters other/(missing)
Top Five Responses – Satisfaction 68. 4 63. 9 56. 1 55. 5 53. 2
Top Five Responses – Technology Use
Top Five Responses – Outcomes
Examining the Research Questions
Gender R 1 a. Does a student’s gender have an effect on their perceptions regarding the satisfaction of the SHU Mobile Computing Program? R 2 a. Does a student’s gender have an effect on their perceptions regarding the use of computers in the SHU Mobile Computing Program? R 3 a. Does a student’s gender have an effect on their perceptions regarding the desired outcomes of the SHU Mobile Computing Program?
Gender
Housing Status R 1 b. Does a student’s housing status have an effect on their perceptions regarding satisfaction of the SHU Mobile Computing Program? R 2 b. Does a student’s housing status have an effect on their perceptions regarding the use of computers in the SHU Mobile Computing Program? R 3 b. Does a student’s housing status have an effect on their perceptions regarding the desired outcomes of the SHU Mobile Computing Program?
Housing Status
Ethnicity R 1 c. Does a student’s ethnic affiliation have an effect on their perceptions regarding satisfaction of the SHU Mobile Computing Program? R 2 c. Does a student’s ethnic affiliation have an effect on their perceptions regarding the use of computers in the SHU Mobile Computing Program? R 3 c. Does a student’s ethnic affiliation have an effect on their perceptions regarding the desired outcomes of the SHU Mobile Computing Program?
Ethnicity
Changes in MC 2002 Results • 81% of the respondents said they had used email to communicate on classrelated information on a daily or weekly basis. • 68% of the respondents used the internet for research on a daily or weekly basis. • 56% submitted work electronically.
Changes in MC 2002, cont. • 70% had contact with their professors often or very often. An additional 20% reported occasional contact. • 68% reported that it helped them take control of their own learning. • 70% used the internet to access class materials online. • While 66% of students felt their laptop use did not detract from their ability to pay attention in class, 45% felt that laptop use was distracting other students.
Conclusions The typical Seton Hall University student, whether representing all respondents, or particular groups of students (groups identified by gender, race/ethnicity, and residence status), is generally: • Attracted to Seton Hall University by the availability of technology at the University and the infusion of technology in the curriculum, as suggested by student reported positive influences on the decision to attend the University, • Satisfied with the Mobile Computing Program as well as with the laptop and support services once enrolled at the University, • Making good use of the technology available to all students, at least in terms of certain types of technology use, and. • Perceiving a substantive positive impact of Seton Hall University’s Mobile Computing Program on the learning environment.
Technology Assessment Projects Recurring Program Assessment University Assessment, Planning and Budget
Technology Assessment Initiatives Programs for Coordinating Assessment Major University Assessment, Budgeting and Planning Mobile Computing Outcomes Sesquicentennial Assessment Team Strategic Plan Tablet PC Pilot TLTR Subcommittee IT 2 Large Course Redesign Middle States Association
Questions?
Скачать презентацию Mobile Computing and Technology Assessment A Case Study
68bb663722981bc6f85ad7954f16a27b.ppt