Ivan Pribela Mirjana Ivanović EVOLUTION OF THE SVETOVID

Ivan Pribela, Mirjana Ivanović EVOLUTION OF THE SVETOVID SYSTEM

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 2

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 3

The Svetovid system Special environment for development and submission of student programming assignments Implemented purely in Java 4

Background Before reforms, students were assessed based on their solutions written on paper After reforms, we concentrate on practical exercises as a form of continual assessment 5

Motivation Now instructors have to much work Compile each program Run it with some test cases Examine all of the output Not all students are producing their own solutions Solutions are easily shared Awareness and honesty is on a poor level 6

The goals of Svetovid Allow students to comfortably develop their programs Allow access to support literature Disallow students to share programs and solutions, intentionally or unintentionally Keep a detailed log of student efforts Help instructors to mark student solutions Allow students to test their programs before submission Be flexible enough and usable for different courses Wide range of programming languages Different project stages: coding, documenting… 7

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 8

The first version Command line based Set of command line tools for assignment retrieval and solution submission Used virtual student directories for assignment and solution storage 9

Virtual directories Root directory One virtual directory John Smith Marc Tyler Assignment 1 Lazar Hans Lucy May Peter White Marc Tyler Peter White Fred Carlen Joanna Blue All students John Smith Assignment 2 Assignment 3 Assignment 1 Assignment 2 Assignment 3 All students 10

Usage Students used favorite development tools and submitted solutions at the end Command line tools are cumbersome Had to type name and password every time Deemed uncool by the students Easy to cheat 11

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 12

The second version GUI based Introduced integrated development environment as student client Introduced instructor control client 13

The student client IDE allowing the student comfortable work Prompts a login dialog The files owned by a student are contained in virtual directory Content of virtual directories can be saved to local file system or sent via e-mail only after assessment 14

The student client 15

The instructor client Set of configuration dialogs Setting up courses Managing students Prompts a login dialog Full control over the conduction of practical exercises Groups Order Timing 16

The instructor client 17

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 18

Encountered problems The text editor component should accommodate all programming needs A third party component was built in Syntax highlight, parenthesis matching… Main problems No keyboard shortcuts while editing text No automatic indention 19

Goals of the extension Create a new text editor component Use of java. swing. text package Keep all of the previous features and add new ones 20

Code. Editor. Pane New textual component Extends JEditor. Pane Main features Syntax highlighting Marking parenthesis pairs Line numbering Indent preservation Measures against cheating Convenience methods for applications 21

Syntax highlighting Done in a separate object representing the language Supported languages Java Scheme Modula 2 Te. X 22

Marking parenthesis pairs Pairs of parenthesis usually have meaning in program code 23

Line numbering Provides useful information during debugging 24

Indent preservation Indentation helps to make code more readable On line breaks, the indent from the previous line is copied 25

Measures against cheating Usage of the system clipboard is forbidden Overriding cut, copy and paste methods Using a local static clipboard 26

Convenience methods Status bar Caret row and column Active language Toolbar Language selection Manual syntax highlight refresh Line numbering component Fully set up JPanel with the component 27

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 28

The Testovid system Interactive and batch testing system Framework for testing systems Implemented using Apache Ant 29

Niklaus Wirth It is easier to build many domain specific assessment systems than to create one universal 30

The goals of Testovid Allow students to check their solutions directly form their environment Allow assessment of all collected solutions after the assignment deadline Be flexible enough and usable for different courses Wide range of programming languages Diverse set of testable aspects 31

Testing modes Students can test their solutions at any time Instructor can test all submitted assignments at once In both cases detailed reports are generated 32

Interactive test mode report Predmet: Operativni sistemi 1 Zadatak: Obilazak direktorijuma Student: Pera Perić Provera: Direktorijum bez poddirektorijuma Uspeh: 1 poen. Provera: Direktorijum sa poddirektorijumima Neuspeh: Rezultat izvrsavanja se ne slaze sa ocekivanim. Provera: Opsti slucaj direktorijuma Neuspeh: Rezultat izvrsavanja se ne slaze sa ocekivanim. -+-+-+-+-+-+Ukupno: 1 poen. 33

Batch test mode report Predmet: Operativni sistemi 1 Zadatak: Obilazak direktorijuma Bajcetic Vladimir 3 poena Baranovski Nenad 0 poena Barjaktarovic Marko Blagojevic Katarina Damjanovic Jelena 0 poena Djurkovic Velimir 3 poena Jovanovic Jelena 1 poen Kovacevic Miroslav Krickovic Valentina. . 1 poen 2 poena 1 poen 34

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 35

Integration One of Svetovid goals was automatic assessment Not implemented directly Delegated to another system Testovid was designed with integration in mind Internal logic is the same Directory structure is compatible Integration was straightforward 36

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 37

The Mag system Tutoring system for distance learning Provides three types of activities Tutoring Quiz and feedback Online programming 38

The goals of Mag Be platform independent Create useful reports for instructors Have rapid development tools for creating basic elements of tutoring system 39

The user interface 40

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 41

The MILE Multifunctional Integrated Learning Environment Modular design Learning environment (Mag) Submission system (Svetovid) Testing system (Testovid) 42

The goals of MILE Provide adequate tutorials to students Allow students to test their understanding of new material Provide online programming support Provide students with automated feedback Allow final exams via internet or in computer laboratory 43

The environment provides Communication Student-student Student-instructor Tools to simplify maintenance Student web pages 44

Main problems Mag was designed as a Java programming tutor Svetovid and Testovid are designed for computer laboratories Great difference between structures of student modeling 45

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 46

Svetovid goals are fulfilled Less time spent on submission Less effort spent on assessment Has been successfully used in many courses Operating systems Programming languages Computer graphics Compiler construction 47

Text editor component Goals are fulfilled Support for all existing features Added new features Flexible syntax highlight system Simple to build in into existing applications Should be used from this year All courses using Svetovid 48

Testovid goals are fulfilled Less effort spent on assessment Assessment available at any time Has been successfully used Operating systems 49

MILE goals are fulfilled Showed itself as a modern learning tool Student satisfaction was the same as with the manual method Less effort spent on assessment Has been used by first year students As a complement to classroom teaching 50

Published work Svetovid – Special submission environment for student’s assessment Ivan Pribela, Nataša Ibrajter, Mirjana Ivanović Proceedings of 2 nd Balkan Conference in Informatics, BCI’ 2006 Ohrid, Macedonia, November 17 -19, 2005, pages 228 -237 Testing almost any aspect of student’s assignments Ivan Pribela, Mirjana Ivanović, Zoran Budimac Proceedings of 3 rd Balkan Conference in Informatics, BCI’ 2007 Sofia, Bulgaria, September 27 -29, 2007, pages 173 -182 Svetovid—Interactive development and submission system with prevention of academic collusion in computer programming Ivan Pribela, Mirjana Ivanović, Zoran Budimac British Journal of Educational Technology (in print) MILE – Multifunctional Integrated Learning Environment Boban Vesin, Mirjana Ivanović, Zoran Budimac, Ivan Pribela Proceedings of e-learning 2008, IADIS Multi Conference on Computer Science and Information Systems Amsterdam, Netherlands, July 22 -27, 2008, pages 104 -108 51

Table of contents Overview of Svetovid The first Svetovid version The second Svetovid version Extension of the second version Overview of Testovid Integration of Svetovid and Testovid Overview of Mag The MILE environment Conclusions Further work 52

Continual improvement Continue to improve usability and user experience of GUI clients Adding new features and tools Adding support documentation and manuals Leveraging effort of test creation Creating a solid base of reusable test modules for multitude of aspects Creation of a test module generator Adding plagiarism detection Just in case 53

Thank you for your attention 54

Questions 55