December 10, 2014 at 3:48 pm #143
Some DDD Manifesto Community members have asked about the definition of ’success’ in DDD processes. How do we know if a DDD process is yielding success? How do we ensure that these kinds of processes are accountable and productive? Do you have any experience thinking about—and measuring—progress and results in a DDD process? When sharing, please try and break down the experience to offer lessons to those reading your comments—or suggest questions.December 11, 2014 at 11:00 am #262
This is an important question. In addition to “How do we define success?” we should also ask ourselves “Who defines success?” Because control over the definition of success means power and as long as we (the external development actors) insist on being the ones to define success we are not willing to hand over power to those who own the problem.December 11, 2014 at 11:24 am #264
Great question, Eva. I wonder if anyone has any experience in defining success metrics. Who was involved in the process? what did they focus on?December 11, 2014 at 11:27 am #265
In our PDIA work we work with local counterparts, usually in government, to identify problems as entry points. The results we are focused on are then simple, ‘problem solved’, which is usually a set of measurable conditions. We include a variety of measures to also look at the degree of trust in the process, the level of engagement (are we engaging more people), and the degree to which there is any learning in the process. These measures help us track the degree to which the work is penetrating the complexity (are we learning so that we are become less uncertain; are we engaging so we are expanding reach, etc.). The metrics are usually identified in an iterative process involving those who authorize the process (often senior government officials in my case, given what we work on) and mid level people, as well as end users (whose views are central to the construction of the problem).December 12, 2014 at 7:48 am #272
Partners in specific projects related to development should clearly show their interests and their other commitments early in the undertaking. External partners’ interests are usually connected with the political image and the commitments with budget execution. So success rides on a rather quantitative institutional road (works and activities done) whilst the affected communities have to journey through the fear of change and the cultural and emotional burden of the felt problems. The definition of success also has to state, or relate to, to what extent partners are accountable to the concerned issues beyond their own institutional audit responsibilities.December 13, 2014 at 6:04 pm #284
To go to another question in the original post – how do we measure progress in a DDD process… The main point here, for me as I think about DDD, is not just to track the usual ‘results required under the results framework’ of the development project (or as jgcervantes mentions above, the quantitative institutional road of activities completed), but more importantly understanding and helping the actual journey through the process of change of those working and living through it. So, this means not counting activities completed and checking them off the list/matrix, but capturing outcomes – those that were in the original plan, yes, but also those that were unexpected and that seem to have been an important part of the reform process. Progress still matters, whatever form it comes in, and it should be captured to unpack the steps and changes through a process. This is particularly important in complex reform projects where the path towards the end goal is sometimes unclear, and where a lot of learning from what was achieved, and how it was achieved, is required so as to best adapt and iterate through the process. There is a richness in understanding some granularity of how the achievements were actually reached – not just the list of results themselves – who did what, and why. And that learning from the ‘how’ can help other teams/sectors/ projects/countries.
We don’t know yet what we don’t know when beginning a project, so we need flexibility in the way we measure progress. One approach I’ve worked with for the past couple of years to help with this was Outcome Harvesting, which is part of a broader outcome mapping learning community (OMLC). If interested you can check it out here: http://www.outcomemapping.ca and here: http://www.outcomemapping.ca/download.php?file=/resource/files/wilsongrau_en_Outome%20Harvesting%20Brief_revised%20Nov%202013.pdfDecember 13, 2014 at 6:21 pm #285
On reflection, this Outcome Harvesting approach also helps partly to speak to Eva’s comment on allowing the implementers themselves to define success… By capturing what they deemed a necessary and important outcome (but which hadn’t been pre-specified in the project results agreement), we’re giving more control of the steering wheel to the implementers to drive towards what they consider as necessary for success – and giving recognition of this, rather than saying the project is ‘stuck’ because the implementers are not meeting the pre-agreed results indicators.December 14, 2014 at 7:14 pm #286
I think there is scope for thinking more broadly about ‘success’ – a focus on measuring outputs/outcomes/impacts implies that the time to start thinking about ‘success’ is when implementing and/or evaluating a particular activity. But what about aiming for success in designing? My wish for DDD is that we spend a lot more time and thinking (and possibly money) on getting the questions right, including establishing what we already know. And measures of success (and this speaks also to the accountability question) could include the level of resonance the questions we seek to address have with policy makers, communities and others.December 15, 2014 at 1:34 am #287
Very much agreed with Devpacific. Thinking we need humility (do we really “do” development? Don’t think so…) and empathy (with those who really do).December 24, 2014 at 5:34 pm #310
Eva, I liked your contribution
This is an important question. In addition to “How do we define success?” we should also ask ourselves “Who defines success?” Because control over the definition of success means power and as long as we (the external development actors) insist on being the ones to define success we are not willing to hand over power to those who own the problem.
One but though: I think all of us should stop using the “we” unless the person that uses “we” refers to all on this forum. So Eva – and all – it may sound trivial, but I think it is not, as long as some of us use “we” and then have to specify that “the external development actors” are meant (or any other sub group in this forum for that matter) psychologically speaking, there is something wrong! – Peter realizing how easy it is to use “we” in a mistaken way.
You must be logged in to reply to this topic.