Expression of Interest: ECHO SHIP Endline Evaluation

1 PROJECT DESCRIPTIONThe proposed action will scale up emergency humanitarian assistance that prevents loss of life from the combined effects of armed conflict and natural disasters in Somalia. The actions aim to increase the availability, access, and utilization of essential health, nutrition, WASH, and Food Security services, reaching 223,964 of the most vulnerable internally displaced people (IDPs), returnees, and host community households in Somalia through Integrated Area-Based Programming.Specifically, the action will contribute to reduced morbidity and mortality by providing life-saving, integrated Health, Nutrition, and WASH services, as well as multi-purpose cash assistance, to the most vulnerable and conflict-affected populations in Afmadow, Jamaame, Dhobley, Buale, Jilib, Dinsoor, Erigavo, and Bardhere Districts.The proposed action will result in:1. Enhanced access to responsive, preventative, and curative health care services.2. Improved nutrition status for at-risk under-five children, pregnant and lactating women.3. Improved access to public health-strengthening WASH products and services for at-risk health facilities.4. Access to timely, gender-responsive Multipurpose Cash Assistance (MPCA), for vulnerable households to meet urgent essential needs enhanced.5. Life-saving emergency crisis contingency capacities in displacement and conflict-affected communities strengthened.World Vision Germany, together with implementing partners World Vision Somalia, Oxfam Somalia, Oxfam Novib, ZAMZAM Foundation, GREDO, and WASDA, will collaborate with the Ministry of Health at the State and Federal Levels, the Somalia Disaster Management Agency, the State Ministry of Humanitarian and Disaster Management, and other humanitarian and development agencies in delivering life-saving assistance.1.1 Purpose and Intended UseThe purpose of the endline evaluation is to measure the Action’s results at completion, verify progress against the logframe indicators, assess program quality and outcomes, and generate evidence, learning, and recommendations for the consortium, communities, and DG ECHO reporting. 1.1.1 Intended Users• World Vision Somalia Program (WVSOM)• Oxfam Somalia Country Office• Implementing partners: ZAMZAM Foundation, GREDO, WASDA• DG ECHO and relevant coordination structures (clusters/authorities where applicable)• Community stakeholders and committees (for validation and feedback loops)• Relevant government ministries (MoH) 1.1.2 Intended Uses of Findings• Accountability: Provide credible evidence on outputs/outcomes and explain what worked and what did not.• Learning: Identify best practices, innovations, and constraints (access, supply chain, coordination, digital systems, referrals, MPCA delivery).• Decision‑making: Inform close‑out, handover of assets/systems, and design of follow‑on actions.• Donor reporting: Support DG ECHO final reporting and results communication. 1.2 Evaluation Objectives1. Measure end line values for outcome/SO indicators and verify key result indicators at end line (including ECHO mandatory indicators).2. Assess relevance/appropriateness of the integrated response for the needs, context, and evolving risks in targeted districts.3. Assess effectiveness and quality of delivery across Health, Nutrition, WASH, MPCA, and crisis modifier/contingency activities, including referrals and layering.4. Assess coverage, equity, and inclusion, including for women, children under five, pregnant and lactating women (PLW), and persons with disabilities.5. Assess Accountability to Affected People (AAP), protection mainstreaming, safety, and safeguarding in delivery and community feedback systems.6. Document key lessons learned and provide practical recommendations for consortium partners and stakeholders.2 EVALUATION SCOPE2.1 TimeframeThe evaluation covers the whole implementation period from 01 Nov 2024 through the endline (March 2026). 2.2 Geographic CoverageThe evaluation will cover a purposive and/or statistically representative sample across the eight project districts: Afmadow, Jamaame, Dhobley, Buale, Jilib, Dinsoor, Erigavo, and Bardhere Districts. The final selection of sites will consider access, security, and operational feasibility. 2.3 Thematic CoverageThe evaluation will cover all results and relevant cross‑cutting components:• Result 1: Enhanced access to responsive, preventative, and curative health care services.• Result 2: Improved nutrition status for at‑risk under-five children and pregnant and lactating women.• Result 3: Improved access to public health‑strengthening WASH products and gender‑sensitive services in at‑risk health facilities and communities.• Result 4: Access to timely, gender‑responsive MPCA, enabling vulnerable households to meet urgent essential needs, was enhanced.• Result 5: Life‑saving emergency crisis contingency plans and response capacities strengthened (crisis modifier/anticipatory action).3 EVALUATION QUESTIONS A. Relevance and Appropriateness• To what extent did the Action address priority needs and risk drivers among targeted IDPs, returnees, and host communities, including women, children under 5, PLW, and persons with disabilities?• How appropriate was the integrated and sequenced approach (Health/Nutrition as entry point with WASH and MPCA layering) to the local context, access constraints, and market conditions? B. Effectiveness and Outcomes• Health: To what extent were primary health consultations, community case management, skilled deliveries, SGBV clinical support and referral services delivered and of acceptable quality?• Nutrition: To what extent were screening, admissions, treatment, and referrals achieved, and did performance align with standards/targets (e.g., recovery rate)?• WASH: To what extent did health‑facility WASH improvements and community WASH response (Erigavo) improve access, practices, and infection prevention and control conditions?• MPCA: To what extent did MPCA improve households’ ability to meet essential needs(MEB/expenditure outcomes) and reduce negative coping strategies?• Crisis modifier/DRR: How effectively did contingency/anticipatory mechanisms function (triggers, timeliness, reach, satisfaction, and coordination)? C. Coverage, Equity, and Inclusion• Were services equitably accessed across sex, age, disability, and displacement status? What barriers (distance, cost, safety, cultural norms, accessibility) remained?• To what extent did the Action reduce barriers and mitigate risks for vulnerable groups, including PWD accessibility within facilities and gender‑sensitive WASH and cash delivery? D. Accountability, Protection, Mainstreaming, and Safeguarding• To what extent did people perceive that assistance was delivered safely, accessibly, accountably, and participatorily, and what corrective actions were implemented during the Action?• How effective were complaints and feedback mechanisms (hotlines/IVR, helpdesks, suggestion boxes, community meetings) in resolving issues and closing feedback loops?• Were protection risks identified and mitigated (including PSEA, GBV referral pathways, aid diversion risks, and data protection safeguards)? E. Efficiency and Coordination/Coherence• How efficient were delivery mechanisms and partner coordination arrangements (supply chain, registration/biometrics, digital systems, referrals, TPM/remote monitoring where relevant)?• How coherent was the Action with cluster standards, government strategies, and other programmes (e.g., UNICEF/WFP referral pathways and coordination structures)? F. Learning and Recommendations• What innovations and good practices should be sustained or scaled (e.g., digital medical records, Clean Clinic/WASH‑FIT, vulnerability scoring, interoperability, anticipatory action triggers)?• What priority recommendations should inform close‑out, handover, and future programming?4 KEY PROJECT INDICATORS TO MEASURE/VERIFY (Minimum Set)The end line evaluation should measure/verify (at minimum) the following outcome/SO indicators and selected results indicators (as applicable):• Crude Mortality Rate (CMR) – measured through survey and/or validated secondary sources where feasible.• Severe Acute Malnutrition (SAM) recovery rate – verified from treatment facility records.• Percentage of target population with adequate WASH services and hygiene practices – using household survey/KAP and observation aligned to SPHERE.• Percentage of beneficiaries reporting assistance delivered in a safe, accessible, accountable, and participatory manner – using the ECHO recommended tool and disaggregation.• Percentage of households without crisis and emergency Livelihood Coping Strategies (LCS).• Percentage of persons effectively referred to other specialized services or assistance providers.5 EVALUATION DESIGN AND METHODOLOGYThe evaluation will use a mixed‑methods design that integrates quantitative and qualitative methods and triangulates findings across multiple data sources. 5.1 Quantitative Components• Household survey: modules aligned to key outcome indicators including Livelihood Coping Strategies (LCS), Minimum Expenditure Basket (MEB)/expenditure, WASH access and hygiene practices, and perceptions of safe/accountable delivery (ECHO process indicator).• Health facility/client exit interviews (where feasible): satisfaction, perceived quality, safety, accessibility, referral experience, and barriers.• Routine data verification: extraction and verification of DHIS2 / facility registers, CMAM/OTP registers, referral logs, MPCA payment records, market monitoring, and PDM datasets. 5.2 Qualitative Components• Focus Group Discussions (FGDs) with women caregivers/PLW, men, adolescents/youth, where relevant, and persons with disabilities/caregivers.• Key Informant Interviews (KIIs) with MoH/DMOs, facility in‑charges, CHWs/FHWs, WASH committees, cash agents/mobile money providers (where appropriate), local leaders, and consortium staff.• Participatory validation: community feedback sessions to share preliminary findings and validate interpretations. 5.3 Sampling Strategy (Evaluator to Finalize)The evaluation team will propose a statistically appropriate sampling design aligned with the baseline (where available) and the Action’s geographic and beneficiary stratifications. Sampling should allow robust estimation of key proportions at the consortium level and, where feasible, by major geographic strata. The evaluator should consider oversampling vulnerable groups (women‑headed households, U5/PLW households, households with members with disabilities) and MPCA recipients to support subgroup analysis.5.4 Data Quality AssuranceAll data quality control measures will be strictly observed throughout the survey. This will include:• Enumerator training and pilot testing of tools.• Daily review of submissions, logic checks, and rapid feedback to teams.• Back‑checks and spot‑checks; supervision plans by district.• Triangulation across survey, qualitative evidence, and routine datasets.• Transparent documentation of limitations due to access/security constraints.• Review and translation of study tools into local languages to ensure clarity and cultural appropriateness.• Standardization of training, including pre-testing and familiarization of enumerators with local terminology.• Verification of evidence against established parameters.• Use of GPS functionality in KoboCollect to geo-reference data.• Regular supervision of field teams and continuous data cleaning. 5.5 Ethical Consideration:The following ethical principles will be strictly observed throughout the survey process:• Respect for individual rights and privacy, ensuring participants’ personal information isprotected.• Voluntary participation, with participants retaining the right to withdraw from the survey, either partially or completely, at any stage.• Informed consent, ensuring participants understand the purpose of the survey and are aware of any potential risks, with avoidance of deception.• Confidentiality and anonymity, safeguarding the data provided by participants and ensuring that identifiable information is not disclosed.• Sensitivity to participant reactions, ensuring that data collection methods do not cause discomfort or distress.• Minimizing harm, considering the potential effects of data analysis and reporting on participants.• Professional conduct and objectivity of surveyors, ensuring impartiality, integrity, and adherence to research ethics throughout the study.5.6 LimitationsSecurity in several targeted districts is volatile, with potential disruptions that may restrict enumerators’ access to specific field locations, potentially affecting the quality of the evaluation. To mitigate this risk, the survey will be conducted in close collaboration with local line ministry staff from inception to completion. Their involvement will ensure adequate supervision of field activities and enable daily updates to sector teams, maintaining the integrity and continuity of the data collection process.6 DELIVERABLESThe final evaluation Report:• Findings should be disaggregated by Sectors and also provide the cumulative results by district and project.• Recommendations need to be supported by a specific set of findings.• Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions.• Findings should be specific, concise, and supported by strong quantitative or qualitative evidence.• Include recommendations for improved future programming and implementation• Clean Datasets & Codebook: De‑identified datasets; codebook; transcripts/notes de‑identified.• Includes photos and quotes from key stakeholders such as beneficiaries and project facilities during FGDs & KII• Including, but not limited to, sections on context, sampling, methodology, findings (including a table showing end-results per indicator/ activity• Disaggregation of beneficiaries - Male/Female, IDPs/Host community levels at a minimum.7 RESPONSIBILITYThe evaluation will be conducted externally by a consultant in collaboration with the World Vision M&E team. A qualified evaluation consultant will be competitively recruited to perform a high-quality end-line evaluation of the project. The external evaluators will not have been involved in project implementation, ensuring that the evaluation is entirely impartial and independent from the design, field data collection, supervision, report writing, and related project processes. This impartiality enhances the credibility of the evaluation, prevents biased assessments, and ensures that data analysis, reporting, and recommendations are objective. Independence also strengthens the evaluation's validity and minimizes potential conflicts of interest that could arise if project implementers assessed the performance of their own interventions.The ECHO MEAL TWG team will provide technical and operational support to the evaluator. The ECHO MEAL Manager will over…