Summary evaluation of 2003 experiences
In introducing Stories of Change in 2003 we failed to distinguish their difference from human-interest stories. As a result, staff viewed Stories as just a more systematic way of reporting qualitatively on the (positive) aspects of the project. They often neglected to involve stakeholders and to explain why the story was significant, or what would be done differently in future to achieve the same successes. This learning is crucial to organizational learning.
While project staff identified with the need for a method like Stories of Change, it is not yet prioritised within the project. It is an initiative being pushed from QA with little reinforcement from the project managers or Aim Leaders. Having the QA and project managers reintroduce the method together might help staff to invest more effort into working with beneficiaries or partners to identify the most significant Stories of Change. Stories need a champion at the highest levels of a project and programme.
Practical training at the field level should make a real impact on the uptake of Stories as a core approach to evaluating impacts. Introducing the method through the Annual Review and expecting it to disseminate down meaningfully to project staff was unrealistic and counter to the bottom-up nature of the approach. No amount of written guidelines can substitute for going through the process with field staff and with partners. Practical training should help staff see the value of sharing stories - good and bad - as opposed to just passing them up the line, as the Annual Project Review has done in the past.
In the 2003 Review, it was found that a majority of projects responded to this section with ‘stories of success’ often affecting ‘communities of people‘ with no clear identification with the partner or pointer to the lessons learned. It is felt that ‘Domains of Change’ could help focus Stories of Change on a particular set of ‘relevant’ issues that were largely missed by he 2003 Annual Project Reviews. 2004 includes a domain for stories of disappointment. These are encouraged since they help project staff to make critical Action Recommendations for the year ahead.This could compensate for a tendency of staff to see M&E, the Annual Project Review and Stories of Change as important only for meeting accountability requirements. What we need to achieve is a constant feedback into lesson learning and better implementation approaches.
1 comment:
Reflections on SOC at our team meeting last year:
Stories of Change - methodology
• Jayantha - Important to be clear why we are collecting a particular SOC – the purpose and audience of the story. This is determined by the way we define the domain of change.
• When we write SOC, style should not refer to the intervention activities but rather to the impact on beneficiaries
• Example discusses qualitative changes in the attitudes of farmers – this is effective.
• Rob – many of the stories lack focus, and miss highly relevant information. Weak interpretation sometimes kills the story: e.g. Sudan story reveals immense inner strength of story teller but doesn’t seem to value this.
• Alison – Aim 3 SOC experiences indicate that project managers appear to be writing stories at the time of annual review – with a strong bias towards their own perspectives / telling their own story!
• Mike – the SOC methodology should involve much more than just the stories. It also must include the information about how stories are selected as significant – the criteria used.
• Alex – but in practice field staff are just not collecting enough material.
• Rob – and quality of story collecting is poor – requires significant training for field staff
• Shibeika – as a team we are not aware of importance of collecting SOC. Staff focus just on implementing activities. Need to build capacity to take care of this role. It should be a normal part of our work, rather than a one-off special task for the annual review.
• Alison – maybe project staff are not the best people for collecting stories. Need to explore options such as using communications staff.
• Mike – this is true, but can’t avoid the need to ‘normalise’ story collecting in the work of project team – so that we have enough stories to make the selection process / criteria revealing.
• Alison – potential v interesting to look at how to use SOC to understand perspectives of MOG. What stories do they want to tell. E.g imagine Guar women talking about the process
• Jayantha – why do we need SOC – for M&E, for project management
• Alison – PROMISS is set up around learning cycles. As an organisation we want to be “improving” our work. How can we use information about impact, influence, results in our fieldwork to influence Group Strategy
• Shibeika – what incentives can we put in place for staff
Post a Comment