Number 4, Summer 1996

CCCRTE

by Kathleen Shepherd, Global Security Fellows Initiative

After Clem's talk, four breakout groups used the rest of the morning to generate, from their own experiences and perspectives, the key issues for evaluation. Our task was to examine the dilemmas of evaluation facing conflict resolution trainers and educators in the wide range of conflicts in which members are working. People working in quite different settings asked similar questions: What are the purposes of evaluation? Who does it serve? How can it be adapted for different purposes? How do we know what we've accomplished? Can we measure the effects of our work beyond the immediate participants? Underlying the discussion was the tension, on the one hand, between the desire to devote every bit of energy to help those in conflict and, on the other, to know the value of our efforts.

The rich outpouring of questions and ideas generated within each of the small groups was reported out and summarized on flipcharts by the lead facilitators -- Guus Meijer, Diana Francis and Clem McCartney. The four topic areas which follow attempt to represent the breadth, and also the overlapping concerns, expressed in the discussions of evaluation.

Stakeholders

One broad topic was the purposes and audiences of evaluation. Who is evaluation FOR? How can evaluation be made accountable to those who are served and those who support CR training? Can donors' needs for objectivity and certainty be balanced with trainers' needs for feedback to evaluate their own work? Can we balance the needs of organizational cultures which expect summative evaluations with the needs of trainers for formative methods? Are there ways to collaborate with donors to create responsive approaches which are better adapted to the special demands of evaluating peacemaking and peacebuilding? What can we do to prepare donors to understand relatively invisible changes?

A related question was raised about who should be included in the evaluation process, which could include evaluation design. What kind of evaluation balances a focus on the trainers' purpose with responsiveness to the participants' needs? Whose views count? How selectively should participants in evaluation be chosen? Are there better ways to use the growing know-how of the stakeholders to formulate criteria for intervention and evaluation?

Measuring outcomes

A second topic area grappled with questions of what, exactly, are we trying to measure, and how it can be done. Attention needs to be given to outcomes both intended and unintended, because benefits or problems may occur unpredictably. How can we measure long-term effects on the conflict as well as long-term gains for individuals? How do levels of intervention influence each other? Can we develop indicators for measuring impacts of conflict resolution training at different levels -- the person, the group, the conflict context itself? Can we track what people DO afterwards, possibly as a result of training? Could such indicators help determine where to make future interventions?

A widespread problem of measurement is how to fit it into training without disrupting the training experience. How can we use evaluation techniques without disturbing the flow, atmosphere, and confidentiality of training processes? How can we get honest evaluation from participants, who may find criticism difficult? How can we evaluate "inner change" of attitude, insight, or group relations?

A thorny problem is presented by the preventive nature of our work. Because the goal is to interrupt violent conflict, how is it possible to measure a disaster which did NOT occur? Could early-warning data provide a base-line of what might have occurred had no intervention been made?

Assumptions and worldviews

A third major theme was the way in which underlying assumptions and philosophies held by the trainers affect training. How do we know if we are asking the right questions? Isn't this based on the world views of trainers and participants? Can we define conflict resolution training more clearly, in order to distinguish between different types of training and their evaluation? For example, if the purpose of a particular conflict resolution training is to develop awareness, or transfer skills, or build solidarity, how does purpose shape evaluation? How can we base a better explanation of what we are trying to measure on a better analysis of the contexts in which we practice? How can we locate training as part of a larger process, in order to overcome criticism of the validity of training as an isolated event? Will attention to underlying assumptions help us overcome the tendency to evaluate events rather than processes?

Process

A fourth broad topic addressed practical and strategic concerns in the process of conflict resolution training and its evaluation. At what points in the training process should evaluation take place? What are useful divisions of labor in the tasks of evaluation? How can we clarify the objectives of training? Can we distinguish outputs from outcomes (in Clem's framework) and does that help clarify our task, our communication about it, and the evaluation processes? How can evaluation be designed to capture both intended and unintended outcomes? How can evaluation serve as a specific guide to future intervention? Perhaps the question is not so much what is "good" or "bad" training technology but WHEN is it appropriately used?

It was clear that trainers face enormous challenges from the contexts in which they work. There are no easy answers for non-violent conflict resolution trainers to such questions as Peter Jarman's -- how can armed peacekeepers and unarmed peacebuilders be trained to work together?

The determination of trainers to think rigorously about every conflict situation and to improve their interventions through effective evaluation was equally apparent. The beneficiaries of their care and reflection will be those struggling in the midst of violent conflicts.

newsletter  |  ccts