Small groups discussed several questions raised by Puntos experience of evaluation.
The process as important as the outcomes
It is important to focus on relationships in understanding processes of change, and to better understand the complex way people learn and act. Puntos used their evaluation data to further explore some of the indirect effects of the project and the important indirect role of promoting interpersonal discussion. It is important to recognise the complex social process where-by people become aware of social norms and issues, reflect on them, and work through them together, with contradictary action and opinions in different settings.
Contribution rather than attribution
In a complex social setting, it was hard to say that particular projects had a direct causal effect on social outcomes, since there were so many influences at work together from different aspects of social life. In this way it was important to try and more humbly assess ‘contribution’ to outcomes and understand the way the process unfolds overall. This is more useful than only narrowly focusing on direct impacts which are often elusive in real social life. Despite the existence of useful tools to measure contribution, such as IDRC’s Outcome Mapping, and some new thinking from the complex systems field, new tools for evaluating social change are urgently needed.
Developing ‘collective messages’
Rather than develop messages, it was useful to look at key issues and work with a range of groups to promote discussion of these issues, that was open and diverse. The main emphasis should be on promoting debate and discussion, so that people can make informed choices and take responsibility rather than be told what is ‘correct’ behaviour. Given the fact that much of the criticism of Puntos approach came from the public health and communication community, the rationale for this approach may still need to be better articulated.
At the same time, it is important to be transparent about the changes you are trying to achieve, and to understand the networks of organisations who are working collectively to open up debate on issues.
