When working in fragile contexts, it is often difficult to carry out traditional project monitoring and evaluation (M&E) activities due to security concerns or a lack of infrastructure (→ digital infrastructure). In these situations digital tools offer numerous alternatives, which can be used to collect data and contact target groups, even in areas that are difficult to access. They also allow you to document the expedient use of funds by donor organisations. But remote monitoring activities can be highly complex and very context sensitive. The following is therefore only intended to offer guidance.
DO NO HARM: While this principle obviously applies to all contexts, it is particularly important in fragile situations. The DO NO HARM principle should be taken into account at the outset of all project planning and integrated into ongoing monitoring. Unintended negative impacts caused by project activities that may jeopardise the project objective or result in dangerous situations for target groups and local partner organisations should be avoided at all cost.
- Greater range: The proliferation of mobile phones makes it possible to include ‘hidden populations’ (i.e. groups that are difficult or costly to reach using conventional M&E) in project evaluations that use digital approaches.
- Simpler analysis: Digital analytical tools help improve the evaluation and formatting of data (e.g. using graphics). Digital solutions can make it much easier and less time-consuming to use complex statistical analytical methods that can help produce more evidence.
- Participation, empowerment and ownership: Digital systems can be used to include more people’s perspectives, making survey results much more representative. They also promote greater transparency in the collection and monitoring of data and boost the acceptance of results and recommendations compiled on this basis.
- Cost-benefit ratio: Initial digital solutions for data collection that have already been deployed in DC (by the World Bank, among others) have been shown to be more cost-effective, even when collecting large volumes of data. When collecting data however, you must establish the following:
- What do I want to achieve with the data and how do they fit in with my project objective?
- How do I relate the data sources with each other (data mining)? For instance, if mobile communications data can be correlated with age-related or income-related data, what can be derived from this correlation and how can these findings be used to achieve the project objective?
- Quick iteration: In digital projects, individual data collection cycles can usually be completed in under 24 hours. Results are also available practically in real time, meaning teams can quickly adjust activities to ensure they better achieve the project objectives.
- Not a magic cure: Digital systems are only one instrument in the M&E toolbox. If they are to be really effective, they must be ‘mainstreamed’ in the project cycle. To this end, roles, processes and the inclusion of partners also have to be coordinated so that digital systems can unleash their true impact.
- Training needs: Digital projects in DC require a minimum level of technical understanding of what is often complicated subject matter (e.g. → mobile communications technology, digital data collection, data science). In existing teams, this understanding is often lacking (e-skills, e-literacy).
Clarify at the outset
It is essential to determine the most appropriate communications channel/s from the very start even in cases where DC/IC projects already use digital M&E. What is the best medium for reaching the target groups in question and what uses of media could cause negative, unintended impacts?
In many cases, after careful investigation, the decision is made not to use internet- and smartphone-based systems (smartphones), because reliable internet → access is limited in rural areas and is only available to a few users. Smartphones and data plans are too expensive for many users. Text messagebased and Interactive Voice Response (IVR) systems are therefore often the best choice, as they work on every kind of mobile phone, regardless of the device’s age and whether it is internet-enabled (Text messages).
(Potential) best pactices
- Digital as a cross-cutting issue in the project cycle: Digital M&E is a cross-cutting task involving all members of the extended team, including partner organisations. As such, it should not be outsourced. Instead, you should promote capacity development.
- Keep it simple: The digital readiness of the target group in the DC/IC target regions is diverse. Therefore, make sure to choose the technologies that target groups are actually able to use.
- Cost-free and incentivised participation: Participating in the evaluation should come at no cost to the target group. At the same time, incentives are required to recruit as many participants as possible for surveys. Be careful, though. Incentive systems can motivate participants to make false statements. Potential local partners acting as intermediaries need to be made aware of this.
- Customise approach used to reflect literacy levels and local languages: For target groups with a low level of literacy, choose voice-based solutions. For those deemed to have functional reading and writing skills, opt for text messages or messenger services.
- In all cases: Translate questionnaires into the most important local languages.
- Reduce complexity: Work with short questionnaires.
- Check data quality at an early stage: Incorrect data entry or allocation is to be expected. Raw data must be manually checked in the early stages of digital M&E projects, at least randomly. To ensure that such checks are carried out regularly, it is a good idea to establish control mechanisms to validate data.
- Opt-in and opt-out: Ensure data protection and comply with statutory spam regulations, security regulations and prohibited technologies and applications and data privacy legislation of project countries. Document consent to participate in surveys and put in place an easily accessible opt-out function that can be used at any time. Users should have control of their data at all times.