Betzavta-Together Research project on Evaluation 2001 -2002 Michael

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together Research project on Evaluation 2001 -2002 Evaluation team: Michael Bommes and Ulrike Wolff. Jontofsohn (University of Education Freiburg)

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Evaluation • The Research Process: ◦ ◦ The The Programme Funders: Demands and Expectations Research team: basic assumptions about evaluation Evaluation Design • Main Findings and Conclusions ◦ ◦ * Programme Implementation –the Organisational Framework Training Quality: Staff and Multipliers Success and/ or Failure? ? Slide Nr. 2

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together • Betzavta-a Democracy and Human Rights education Programme -Developed by the Adam. Institute in Jerusalem/Israel(1986) • Closely connected to the Israeli Situation • Educational Goals: educate democratic citizens, educate all members of a community to accept equality and freedom as basic democratic principles • Approach: Social and moral education by creating dilemmas and conflictual situations(Piaget, Kohlberg) • Facilitation Techniques: focus on group dynamics and conflict elaboration * Slide Nr. 3

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Methods: Two Types of Activities • Type 1: Creating a Dilemma Structure: ◦ group-dynamic activity ◦ Perception and reflection ◦ Conflict elaboration within normative aspects • Type 2: Learning the Democratic Discourse Structure: ◦ individual task of opinion forming ◦ Search for consent in small groups ◦ Market Place - Exchange of Opinions * Slide Nr. 4

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn German Version of Betzavta-Together • 1996: Adaptation initiated by the Bertelsmann. Foundation and the University of Munich (CAP) • Modular combination of three different programs: ◦ Betzavta (1886), ◦ Peace Education Program (1995) ◦ Building Blocks of Democracy (1996). • Since 1997: Training of multipliers • Implementation Process * Slide Nr. 5

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn The Funders: Demands and Expectations • Target Group: “ The Multipliers from 1997 to 2001” • Innovative Research • Combination of qualitative and quantitative methods • A democratic and participatory process of evaluation corresponding to the spirit of the programme • Valid assessment of progress or failure • Inherent Definition of Success: cognitive learning, socio-moral education, attitude change, action planning, systemic changes * Slide Nr. 6

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Basic assumptions about evaluation of the research-team (Part I) • Complexity-Learning processes are complex phenomena • Validity: Control groups are limited in their testimony • Reliability: Effects and effect-attribution are not identical * Slide Nr. 7

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Our Basic assumptions -Part II Systemic View: Evaluation is an attempt to measure the outcomes and effects of an individual learning experience within a specific system Perception of effects and effect-attributions are context-related • Interpretation of Goals is dependent on the occupational environment • Interpretation of Effects/Outcomes dependent on the definition of progress and failure in a certain milieu * Slide Nr. 8

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Evaluation design Database: 250 addresses • Partly standardized questionnaires • 11 individual interviews • Group discussion • Cooperation with a practitioner’s group • Interviews with experts * Slide Nr. 9

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Main results (Part I) • Programme : • enthusiastic acceptance of its methodology and interactive learning strategies • Effect-attributions are very positive • But: positive judgment did not depend on duration of the personnel and/or professional experience • Modest degree of distribution and implementation • Circularity in specific milieus and institutions * Slide Nr. 10

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Main results (Part II) • Trainers and multipliers: majority from pedagogical and psychological occupations • One Third complain about training quality • High commitment to the issues-certain lack of professionalism • Rare use of the whole Programme • Preference for group-dynamic activities • Modest integration into school curricula • Educational practice shows tendancy towards a specific interpretation of the programme • Rather depoliticized usage as programme for Social Learning; * Slide Nr. 11

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Recommendations and conclusions • • • Improvement of staff and organisation Theory-input Clarification of goals and objectives Monitoring and consulting Context-orientation -different trainings for different target groups • Improvement of implementationstrategies * Slide Nr. 12

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Discussion: What might be useful for us? ? • Our understanding of political education? • Our goals and expectations concerning our work with a specific programme? • Do we pay attention to needs and requirements of different target group? • Long-term or short- term intervention? Follow-up? • Our indicators for success or failure? * Slide Nr. 13

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Obstacles and conflicts during the evaluation process • The practitioner‘s group ◦ Fluctuation of members ◦ Collisions of interests ◦ Problems with solidarity • Negotiation processes ◦ Different perspectives of Stakeholders ◦ Conflict of interests * Slide Nr. 14

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Preconditions for Success • • • Quality of the programme Quality of the facilitators Structure of the learning process Reference to the Participants´context Integration into Curricula • • Learning arrangements Long-Term Intervention Continuity of the process Follow-up * Slide Nr. 15

Betzavta-Together Research project on Evaluation 2001 -2002 Michael Bommes and Ulrike Wolff-Jontofsohn Workshop in Feldafing-Overview • Introduction • Evaluating a Peace Education Programme: ” Betzavta-Together” • Evaluation Process and Main Findings • Discussion • Closing round * Slide Nr. 16