Generate and integrate evidence
Our research with ALNAP (Obrecht and Warner, 2016) found that generating and using evidence was a key factor for successful innovation. It will be one of, if not the, main factor that will show whether what you are working on is going to have an impact. It will also be one of the key determinants as to whether others will use and fund the innovation.
But while evidence-based approaches to humanitarian innovation are understood to be important, most people struggle to generate and use evidence—strategically, explicitly, and systematically—toward the development of innovative humanitarian solutions.
Without consideration of evidence, you risk of creating solutions that are, at best, ill-suited to users, have wasted time and money to develop, are inconsequential in addressing needs or, at worst, may lead to unintentional harm. It is therefore critical that you build a process for generating and collecting evidence throughout the innovation cycle.
Understand the role of evidence and how it can be applied
There are three key concepts to help you better understand the relationship between evidence and innovation:
First, evidence is not the same thing as information. Whereas information can be understood as the facts and details learned about something, through study or experience, evidence is information that can be used to either justify or deny a hypothesis or claim (Miller and Rudnick, 2012).
Second, the role of evidence varies throughout the innovation lifecycle. It’s not limited to carrying out an assessment at the at the beginning of a project, and an evaluation at the end of a pilot. We can think of the application of evidence as a resource for the following types of activities:
- Exploring and explaining problems (Recognition)
- Generating insights to guide the development of new ideas (Invention)
- Validating and stress-testing initial ideas, and fine-turning functionality of viable ideas (Invention)
- Demonstrating impact of real-world solutions (Pilot)
- Documenting, reflecting on, and sharing learning with the wider community (Scale)
- Supporting and articulating strategic decision-making at key stages and pivot points throughout the innovation process. (all stages)
Third, using evidence in support of innovation requires a process. Using evidence for innovation requires identifying relevant information and a process for moving that knowledge into action.
Demonstrate improvement and learning
Ultimately, any innovation must demonstrate that it performs better than existing solutions. In order to do this, you must understand the problem and the effectiveness of existing solutions and of your solution, and you must be able to compare the two. You must also analyse and document your process for learning and improvement.
The problem
Too many times there is only a surface-level understanding of a problem when people start trying to develop solutions for it. The stronger your understanding of the problem and the better the evidence you have regarding the problem, the more likely you are to find or develop a relevant, impactful solution. In order to prove that there is a problem, the more quantitative and qualitative evidence that can be generated toward this aim, the better. The Guide explores this in the section on Recognition.
Existing solutions
How are the current practices and solutions for the problem working? Do you have evidence of their performance and impact? This can often be extremely tricky to find. There are a number of ways that this can be explored, and we will look at some of them in the Search, Adapt and Pilot sections of this guide. The more quantitative and robust qualitative evidence that can be generated to prove the performance and impact, of existing solutions, the easier it will be to carry out a comparative analysis.
You may also find that there isn’t currently a full solution to the problem, but people are using ‘workarounds’, for example, women not using latrines between dusk and dawn because access is unsafe. In this instance you need to find a way of using the evidence you have gathered on the problem to show the impact of not only the problem, but also to use as a basis for assessing the performance and impact of these ‘workarounds.’ What are the social and health implications of this approach? Is the performance of these ‘workarounds’ so poor that your hypothesis that a new solution is required is proven?
“The biggest tipping point for us has been the formal research that demonstrates that what we’re doing is better that any other options in the field in terms of trauma therapy. You can use innovation to attract people, but for people to really get on board, you need evidence.” Darcy Attaman, Make Music Matter (interview)
Your solution
If you have adapted or developed a solution, your innovation will need to be put through a rigorous process of evidence building. This will require at the bare minimum a strong DME/MEAL (Design, Monitoring and Evaluation/Monitoring, Evaluation, Accountability and Learning) process, but should have a further research component to it as well.
This should include a baseline assessment, effective monitoring and an evaluation. We would also recommend a learning process and research be part of your whole approach. It is important to understand that if you are developing and implementing an innovation in the real world, you are effectively carrying out a social science experiment, and therefore the evidence-building processes you establish should support this experiment happening in an ethical and evidence-based manner.
For all your evidence building you need to ensure that you are clear on what you are trying to prove or disprove/refute at each stage of the development of your solution. Don’t collect data and information unless you have a defined purpose for it.
Comparison between your solution and other solutions
In order for your innovation to be deemed a success, it will require you to assess the performance and impact of your solution on the problem, vis-à-vis the performance and impact of other solutions to the problem. This should be carried out through a comparative analysis. By carrying this out, you are trying to prove whether or not your solution is better than other solutions to the problem.
It is worth remembering that ‘success’ can be defined in different ways, for example, your innovation may have slightly worse performance on the problem than other solutions, but it might be ten times cheaper. Depending on what the problem is, this might be a success because the economy gains outweigh the effectiveness loss. However, you will have to factor in variations that may be present in any data (e.g. is the data from the same context?, for the same age group?, using the same measures?). It is important, but difficult, to make sure you are comparing like with like.
The process
Innovation journeys often require you to develop new ways of doing things. Your innovation itself will generate the need to develop and test operating procedures, process maps, technical specifications etc. Documenting how this was done, and what ways worked and didn’t enables you to understand not only the performance and impact of your innovation, but also for the process you use. This will enable you to make a judgement on what are the key aspects of your process that you would seek to replicate, and which should you stop applying. Also, documenting the processes around implementing your innovation is critical for scaling your innovation.
Sell your idea: the psychology of evidence
To convince others to invest in your idea – whether they are gatekeepers, users, donors, partners or supporters – you will need to provide a strong evidence base. However, you also need to know how to present evidence in order to make your case successfully.
Some people and organisations will need to see statistical quantitative data; think of this as ‘head’ evidence. Others need compelling stories, from case studies or media coverage; think of this as ‘heart’ evidence.
When building up your evidence base, make sure that you are getting the right balance between the two types of evidence – ensuring that you have “key facts” to provide statistical evidence of impact, as well as good stories to create an emotional ad psychological connection.
Further resources
There are few evidence-building tools, approaches and frameworks specifically designed for humanitarian innovation. Outlined below are some useful resources that have been developed for this field, which we will continue to add to as new resources become available.
Mobilising evidence
We recommend reading the following documents to learn about the kinds of information that make for good evidence (ie, what is relevant and what is not) and the application of information as a resource for problem-solving (ie, the explicit and formalized process for moving knowledge into action.)
Miller, R. & Rudnick, L. (2012) A Framework Document for Evidence-based Design, UNIDIR.
Building evidence through monitoring and evaluation
Read the following papers from ALNAP for some of the first evaluative concepts and practices for humanitarian innovation (types of evaluative inquiries, purposes and applications, and focus areas; key milestones, and strategies for monitoring progress and impact).
Warner, A. (2017) Monitoring Humanitarian Innovation, HIF-ALNAP Working Paper.