External evaluation of ICI’s Community Development Programme


Download External evaluation of ICI’s Community Development Programme


Preview text

External evaluation of ICI’s Community Development Programme
Executive Summary
March 2020

Prepared by: Bureau of Integrated Rural Development (BIRD), Kwame Nkrumah University of Science and Technology, Kumasi, Ghana.
This report was prepared by: Dr Albert Arhin, Prof. Paul Sarfo-Mensah, Cheikh Tidiabne Sarr, Dr Ernestina Fredua Antoh, Dr Olivia Agbenyega, Kwaku ObengOkrah and Kuwornu Messa Amenyo.

External Evaluation of ICI’s Community Development Programme 2015-2018

1

CONTENTS
Introduction................................................................................................................................................... 3 Methodology................................................................................................................................................. 3
Data Analysis ..........................................................................................................................................4 Summary of Findings ...............................................................................................................................4 Recommendations ..................................................................................................................................... 7
How could ICI improve the design and implementation of future community development projects?...................................................................................................................... 7 What could communities do to continue to combat child labour? ........................... 8 What could local authorities do to improve the protection of children?...............8

External Evaluation of ICI’s Community Development Programme 2015-2018

2

Introduction
Since 2015, the International Cocoa Initiative (ICI) has been implementing its ‘core’ community development programme in 75 cocoa-growing communities, 46 in Côte d’Ivoire and 29 in Ghana. The programme set up and supported community groups to put in place a range of activities focused on child protection, education, women’s empowerment and income-generation. This external evaluation was commissioned by the International Cocoa Initiative (ICI) in January 2019. The main objective of the evaluation was to assess to what extent has progress been made since 2015 in ensuring that cocoa-growing communities are more protective of children and their rights?
Although the initial scope of the evaluation included an estimation of the programme’s impact on child labour prevalence, the consultants were not able address this question in the time available. The impact on child labour has been analysed and presented separately in another report.
Methodology
The evaluation used a mixed methods approach that comprised of two principal elements: quantitative data analysis from datasets related to the programme; and qualitative data collected by the consultant:
a) Sources of quantitative data include: (i) Community-level assessments conducted using ICI’s Protective Cocoa Community Framework (PCCF) tool in all ICI-assisted communities in 2015, 2016, 2017 and 2018; (ii) Community-level assessments conducted using ICI’s Protective Cocoa Community Framework (PCCF) tool in control communities in 2015 and 2018; (iii) Datasets from ICI’s Monitoring tool, providing annual records of activities in each assisted community from 2015 to 2018; (iv) Knowledge Attitudes and Practices survey data collected from a sample of community members in assisted communities in 2016, 2017 and 2018; (v) Data from Women’s Empowerment surveys in assisted communities 2017 and 2018; (vi) Child labour prevalence survey data for ICI-assisted and control communities, collected in January 2019; and (vii) Follow-up surveys administered to the participants and recipients of education interventions, collected in January 2019.
b) Qualitative data sources include 11 key informant interviews and 45 focus group discussions (involving 390 participants) held in Ghana, and 12 key informant interviews and 40 focus group discussions (involving 600 participants) held in Cote d’Ivoire. In total, nearly 1000 participants shared their perspectives on the project. The key informant interviews were held with relevant stakeholders at the community and district levels including community chiefs, elders, representatives from local authorities and ICI project staff. The focus group discussions were held with community members who had participated in the project activities in some way. They include members of Community Child Protection Committees (CCPCs), income-generating activity (IGA) groups, savings and loans groups (CFREC), community service groups (CSGs), child members of Child Protection Clubs (CPCs), selected children who were not part of the CPCs, school management committees (SMCs) and parent-teacher associations (PTAs).

External Evaluation of ICI’s Community Development Programme 2015-2018

3

Data Analysis
To determine the progress, effectiveness, and by extension the impact of the ICI interventions, the methodological approach used to compare changes between 2015 and 2018 in the ICI-assisted communities with those occurring in ‘control communities’ during the same period. The statistical approach taken to derive average treatment effects of the ICI core programme is the difference-indifferences (DiD) estimator, which is typically used to estimate the effect of a specific intervention or treatment by comparing the changes in outcomes over time between a population that is enrolled in a program (the intervention group) and a population that is not (the control group). Qualitative data were analysed to identify patterns, narratives and themes around the various subjects of interest.
Summary of Findings
The set-up, activeness and performance of Community Child Protection Committees (CCPCs)
• The programme contributed significantly to the set-up of CCPCs—key actors in ICI’s community development approach. Overall, the proportion of ICI-assisted communities with Community Child Protection Committees (CCPCs) increased from 8% to 99% between 2015-18, while in the control communities it increased from 14% to 18%. The difference between ICIassisted communities and control communities was statistically significant overall (80 percentage points), as well as in Ghana (90 percentage points) and in Cote d’Ivoire (84 percentage points).
• CCPCs in the ICI-assisted communities were more active than their counterparts in the control communities, where they existed. For example, the evaluation found that the share of CCPCs holding meetings at least once a quarter was 55 percentage points higher in the ICI-assisted communities than in control communities, a difference which is statistically significant.
• Assisted communities were more likely to have rules and regulations to support children. Between 2015-2018, the proportion of ICI-assisted communities with rules and regulations to protect children increased from 18% to 87%, while in the control communities it increased from 9% to 43%. The 36-percentage point difference is statistically significant.
• Across all the communities visited, participants in focus group discussions consistently reported that the CCPCs are performing their roles: undertaking awareness raising activities on child labour, facilitating the development of community action plans and identifying and supporting vulnerable children.
The programme’s role in empowering beneficiary communities • The programme was effective in building communities’ capacity to approach local authorities for development assistance. The share of ICIassisted communities with the capacity to approach local authorities for assistance for development increased from 49% to 77%, while in control communities the proportion decreased from 49% to 33%. The 44percentage point difference is statistically significant. • Assisted communities were better able to mobilise resources for community development. The share of the ICI-assisted communities reporting the ability to mobilise resources for community development increased from 47% to 71%, while in the control communities it increased

External Evaluation of ICI’s Community Development Programme 2015-2018

4

from 40% to 44%. The 19-percentage point difference is statistically significant. • The programme was effective in facilitating resource mobilization to benefit children. The proportion of ICI-assisted communities with the ability to mobilize resources to benefit children increased from 47% to 87%, while in the control communities it decreased from 31% to 28%. The 43-percentage point difference is statistically significant. • During focus group discussions with the various community groups (such as CCPCs, CFREC, CSGs), participants stressed that Income Generation Activities (IGAs) played an important role in enhancing the commitment and motivation of members and promoting the sustainability of these groups.
The programme’s role in empowering women • The evaluation found that income generating activities had made a modest contribution to improving women’s livelihoods. In Côte d’Ivoire 42% of women surveyed in 2017 (n=701) perceived either a high or a very high increase in their income as a result of the IGA. In Ghana, 59% of respondents interviewed in 2018 perceived either a high or a very high increase in their income as a result of the IGA. • Findings from multiple sources of qualitative data suggest that participation in the various women’s groups improved social interactions and solidarity among women, both in Ghana and Côte d’Ivoire. Several respondents reported that they now feel more confident and emboldened to speak and express their opinions at meetings. However, socio-cultural factors often continue to constrain women’s ability to participate in decision-making.
The programme’s role in improving access to quality education Côte d’Ivoire:
• Between 2015 and 2018, average gross enrolment rates increased in ICIassisted communities compared to control communities at all levels of schooling, except kindergarten.
• At primary school level, average gross enrolment increased from 70% to 92% in ICI-assisted communities, while in control communities it decreased from 86% to 84%. The difference is not statistically significant.
• At Junior High School level, average gross enrolment increased from 7% to 21% in ICI-assisted communities, while in control communities it increased from 60% to 73%. The 20-percentage point difference is statistically significant. The difference for girls (from 6.1% to 58% in assisted communities compared to 59 to 63% in control) is also significant, but not for boys (8 to 22% in assisted communities compared to 60 to 80% in control communities).
• At Senior High school level, average gross enrolment stayed at 3% in ICIassisted communities but decreased from 28% to 20% in control communities. The 6-percentage point difference is statistically significant overall and for both girls and boys.
• Compared to control communities, there was no statistically significant difference in the distances travelled to educational facilities at any level.
• Compared to control communities, there was no statistically significant difference in the proportion of ICI-assisted communities with a school

External Evaluation of ICI’s Community Development Programme 2015-2018

5

canteen, access to Government school feeding programmes or toilet facilities. • Compared to control communities, there was no statistically significant difference in the pupil to teacher ratio in ICI-assisted communities. • The proportion of ICI-assisted communities with a School Management Committee stayed at 93%, while in control communities it decreased from 100% to 80%. The 21-percentage point difference is statistically significant. • The remediation or support components of the ICI core programme were perceived by beneficiaries (parents and pupils) to be helpful - children are reportedly able to attend school more regularly compared to before receiving the items. Adult literacy classes for women made participants ‘feel more independent’ due to reading and writing skills gained.

Ghana: • • • • • • • •
• •

Between 2015 and 2018, average gross enrolment rates increased in ICIassisted communities compared to control communities at all levels of schooling, except kindergarten. No data was available for technical/vocational schools in control communities. At primary school level, average gross enrolment increased from 84% to 99% in ICI-assisted communities, while in control communities it decreased from 105% to 42%. The 80-percentage point difference is statistically significant overall, and for both girls and boys.
At Junior High School level, average gross enrolment increased from 80% to 86% in ICI-assisted communities, while in control communities it decreased from 70% to 32%. The difference is not statistically significant overall or for boys but is significant for girls. At Senior High School level, average gross enrolment increased from 59% to 73% in ICI-assisted communities, while in control communities it decreased from 30% to 10%. The difference is statistically significant overall and for both boys and girls.
Compared to control communities, there was no statistically significant difference in the distances travelled to educational facilities at any level.
The proportion of ICI-assisted communities with a school canteen increased from 25% to 52%, while in control communities it decreased from 22% to 10%. The 40-percentage point difference is statistically significant. Compared to control communities, no significant differences were observed for the proportion of communities with access to Government school feeding programmes, nor for the proportion of communities with school toilets. Compared to control communities, the pupil to teacher ratio increased significantly in ICI-assisted communities. Qualitative data suggests that the refurbishment of school facilities in some communities made them more appealing, attracting children from neighbouring communities as well as from the ICI-assisted communities. The proportion of ICI-assisted communities with a School Management Committee stayed at 88%, while in control communities it decreased from 100% to 65%. The 35-percentage point difference is statistically significant.
The remediation or support components of the ICI core programme were perceived by beneficiaries (parents and pupils) to be helpful. For example, children are reportedly able to attend school more regularly compared to

External Evaluation of ICI’s Community Development Programme 2015-2018

6

before receiving the items. Adult literacy classes for women made participants ‘feel more independent’ due to reading and writing skills gained.
Relevance, efficiency and sustainability of the programme • Qualitative interviews showed that communities have positive views and perceptions about the relevance of ICI work. There is unanimity in the responses provided by participants in the various focus group discussions and key informant interviews that the main components of the project (child protection, community and women’s empowerment and education support) have helped communities to become more protective of their children and minimized involvement of children in activities such as carriage of heavy loads, and use of dangerous tools. • In both Ghana and Côte d’Ivoire, responses obtained from staff and community members suggest that the ICI core programme deployed ‘adequate’ human, material and financial resources for the implementation of the programme—though staff recruitment in some regions (e.g. Soubre in Cote d’Ivoire) was more delayed than expected. • In both countries, feelings about the sustainability of the programme was mixed. On the one hand, communities feel enthusiastic about the activities implemented. On the other hand, all the ICI-assisted communities expressed the need for more time for ICI to be with them to fully mature.
Recommendations
How could ICI improve the design and implementation of future community development projects?
1. Consider rolling out community savings groups (CFREC or VSLA) in Ghana, as well as in Côte d’Ivoire. This is because we found CFREC to be successfully contributing to women’s economic empowerment and members reported many social benefits.
2. Improve community participation in the procurement of project equipment, specify items carefully, and ensure clear communication with communities throughout the procurement process. In some communities, participants reported that they had not been consulted about the model or type of items purchased – for example mills, knapsack sprays and tricycles – resulting in the procurement of equipment they perceived as inappropriate or poor quality. The evaluation team observed that some equipment was idling or had broken down, confirming this challenge. More careful specification of items prior to tender could help avoid such problems in the future.
3. Improve data collection tools and information management systems to ensure that all data are properly cleaned and checked in a more systematic way. This would allow for more efficient analysis and reporting, as well as facilitating the identification of challenges during project implementation.
4. Consider complementary approaches, beyond awareness-raising sessions, to effectively change knowledge, attitudes and practices on child labour. Despite the awareness raising activities conducted by CCPCs, knowledge, attitudes and practices still require further improvement.

External Evaluation of ICI’s Community Development Programme 2015-2018

7

5. Consider conducting monitoring visits to assisted communities to observe progress and support in an advisory capacity. Communities reported that they would like ICI to continue to pay occasional visits now that the project has ended, so that they could continue to benefit from the advice as they continue project activities themselves.
6. Plan additional ways of improving the mobility of the child protection committee members, for example through the provision of motorbikes. This would help CCPC members to reach all homes in a community, even those in more isolated locations that are harder to reach
7. Consider providing training to CCPCs in resource mobilization with other external actors to facilitate the implementation of actions that decrease children’s vulnerability. This is because despite community’s ability to approach local authorities, many actions in the Community Action Plans do not get implemented as local authorities lack enough funds.
What could communities do to continue to combat child labour?
1. CCPCs should continue to identify children without birth certificates and collaborate with authorities to ensure they can acquire them. This is a particular challenge in Côte d’Ivoire, where children cannot enter secondary school without birth certificates.
2. Community groups, such as savings groups (CFREC) and Community Service Groups should continue to use part of the revenue from income generating activities to support children, for example donating funds or produce to school feeding programmes. Although it is important to note that this is not a substitute to other sources of funding.
3. Community leadership should encourage community to pay Community Service Groups after using their services helping these groups to stay afloat so that the community can continue to have access to affordable adult labour.
What could local authorities do to improve the protection of children?
1. Provide more housing and other resources for teachers. This would contribute to reducing absenteeism and increase regular attendance to schools.
2. Local authorities should continue to be responsive to community requests for assistance, including facilitation of projects to improve the availability accessible roads, water and basic social infrastructure.

External Evaluation of ICI’s Community Development Programme 2015-2018

8

Preparing to load PDF file. please wait...

0 of 0
100%
External evaluation of ICI’s Community Development Programme