GEF COUNTRY PORTFOLIO EVALUATION:

(Prepared by the GEF Evaluation Office)

Global Environment Facility


Recommended Council Decision

The Council, having reviewed Document GEF/ME/C.31/5, GEF Country Portfolio Evaluation: Samoa (1992-2007) and document GEF/ME/C.31/6, Management Response to the GEF Country Portfolio Evaluation: Samoa, takes note of the recommendations of the evaluation and the management response and requests the Secretariat to take into account Samoa’s experience with the GEF in its further development of the proposed GEF-Pacific Alliance for Sustainability, including the following:

- the need for support for establishing an environmental framework in national policies, laws and regulations, and where this has been achieved, the need for support for implementation of the framework so as to achieve global environmental benefits;


- recognition of the importance of marine resources and resilience to climate change to sustain global environmental benefits;


- recognition of the high transaction costs in the region; and


- the need for involving more GEF Agencies in the region, as well as harmonization with recipient countries and other donors.


The Council welcomes the response of Samoa to the Evaluation and invites the Secretariat to takes this into account when preparing the programmatic alliance.


Table of Contents
1. Background............................................................................................................... 1
2. Conclusions............................................................................................................... 2
Relevance of GEF Support................................................................................ 2
Results of the Portfolio...................................................................................... 4
Efficiency........................................................................................................... 6
3. Recommendations..................................................................................................... 9
Recommendations to the GEF Council.............................................................. 9
Recommendations to the Government of Samoa............................................... 10

EXECUTIVE SUMMARY
1. Samoa has been the recipient of GEF financial support since the Pilot Phase of the GEF through two regional projects which set the stage for GEF interventions in Samoa (and the Pacific region), creating a partnership between GEF, UNDP, the South Pacific Regional Environmental Programme (SPREP) and the Samoan Ministry of Natural Resources and Environment (MNRE), which has lasted until today.
2. The evaluation of the GEF support to Samoa took place between January and April 2007 following the Standard Terms of Reference for the GEF Country Portfolio Evaluation developed by the GEF Evaluation Office in October 2006. It was conducted by a team of staff from the Evaluation Office and members of the Pacific Environmental Consultants, Ltd (PECL). The objective of this evaluation is to provide the GEF Council with an assessment of how the GEF is implemented in Samoa. It reports on results from projects and assesses how these projects are linked to national environmental and sustainable development strategies as well as to the GEF mandate of generating global environmental benefits within its focal areas.
3. The focus of the evaluation is a portfolio of 18 projects funded by the GEF during the period from 1992 to December 2006 with an estimated investment of $7 million. Eight are national projects (6 Enabling Activities and 2 Medium-Sized Projects), 7 are regional (projects in which Samoa participates as a member of the Pacific Island States) and 3 global which have national components in Samoa. All focal areas are represented in this cohort of projects and although 80% of projects are implemented through UNDP, the World Bank and UNEP have also had experience with projects in Samoa.
4. The evaluation reaches the following conclusions:
(1) GEF support has been relevant to the Samoa Development Strategy (SDS) and national environmental policies.

(2) All GEF funded projects are highly relevant to the GEF mandate and focal areas but slow follow-up support from government sources could jeopardize the sustainability of results.

(3) Enabling activities have supported Samoa in building the foundations for its environmental frameworks and strategies which are necessary conditions for generating global environmental benefits.

(4) Completed projects have achieved concrete on-the ground results but reporting on results has limitations because of poor quality of final evaluations and limited baselines.


(5) Samoa has improved its efficiency to access GEF funding, but there are still some obstacles.
(6) Most GEF Agencies have not been engaged in Samoa mainly because of the high transaction cost and limited understanding of GEF objectives and procedures.

5. The findings and conclusions of the evaluation lead to the following recommendation to the GEF Council:
(1) The proposed programmatic approach for the Pacific SIDS should take into account Samoa’s experience.


6. Furthermore, the following recommendations have been brought to the attention of the Government of Samoa:
(2) Environmental concerns as cross-cutting issue need to become visible in the Samoa Development Strategy.
(3) Increased participation by other stakeholders (ministries, civil society, and private sector) in implementing GEF supported projects will increase national capacity.

1 BACKGROUND
1. Samoa has been the recipient of GEF financial support since the Pilot Phase of the GEF when Samoa participated in two regional projects, one on biodiversity and the other one in climate change. These two projects set the stage for GEF interventions in Samoa (and the Pacific region), creating a partnership between GEF, UNDP, the South Pacific Regional Environmental Programme (SPREP) and the Samoan Ministry of Natural Resources and Environment (MNRE), which has lasted until today.
2. The evaluation of the GEF support to Samoa took place between January and April 2007 following the Standard Terms of Reference for the GEF Country Portfolio Evaluation developed by the GEF Evaluation Office in October 2006. It was conducted by a team of staff from the Evaluation Office and members of the Pacific Environmental Consultants, Ltd (PECL). The objectives of this evaluation are to provide the GEF Council with an assessment of how the GEF is implemented in Samoa. It reports on results from projects and assesses how these projects are linked to national environmental and sustainable development strategies as well as to the GEF mandate of generating global environmental benefits within its focal areas. Samoa was selected through a random selection process among all countries in the Asia and Pacific region but also as a representative of two very relevant groups of countries for the GEF: Small Islands Development States (SIDS) of the Pacific and Least Developed Countries (LDCs). During the evaluation process it was evident that although Samoa shares common problems regarding accessing and implementing GEF-funded projects with countries in these groups (such as limited capacity, high transaction cost of doing business and high vulnerability) not all lessons from this case can be transferred to other countries in those groups. When appropriate, the evaluation recognizes the diversity of the countries in the Pacific.
3. The evaluation explores three key questions for the GEF and Samoa:
(1) Is the GEF support relevant to the Strategy for the Development of Samoa (SDS) 2005-2007, the national development needs and challenges as well as the action plans for the GEF’s focal areas and the GEF mandate, objectives, policies and focal areas programs and strategies?
(2) Is GEF support efficient as indicated by the time, effort and money it takes to develop and implement GEF projects; any particular issues related to regional projects; and synergies and partnerships between GEF projects and between GEF and government agencies as well as other GEF stakeholders? and
(3) What are the results of completed projects, aggregated at the focal area and country levels?


4. The focus of the evaluation is a portfolio of 18 projects funded by the GEF during the period from 1992 to December 2006 with an estimated investment of $7 million.1 Eight are national projects (6 Enabling Activities and 2 Medium-Sized Projects), 7 are regional (projects in which Samoa participates as a member of the Pacific Island States) and 3 global which have national components in Samoa. All focal areas are represented in this cohort of projects and although 80% of projects are implemented through UNDP, the World Bank and UNEP have also had experience with projects in Samoa. The following figure and table depict the distribution of projects across focal areas, project status, geographic scope and GEF modalities of projects.

1 All dollars cited in this report are current US dollars unless otherwise noted.
Figure 1. Amount of GEF funding for all GEF activities in Samoa according to their status (completed or under implementation)

Table 1. Number of GEF supported activities according to GEF modality: Enabling Activity (EA), Medium Size Projects (MSP) and Full Size Projects (FSP)

EA
MSP
FSP
Total
National
6
2
0
8
Regional
2
1
4
7
Global (*)
0
0
3
3

8
3
7
18
(*) includes SGP






2 CONCLUSIONS

5. Based on the analysis of information and evidence collected through this evaluation, the following conclusions can be reached about the relevance, efficiency and results of the GEF support to Samoa in the last 15 years.

Relevance of the GEF Support

Conclusion 1: GEF support has been relevant to the Samoa Development Strategy (SDS) and national environmental policies.

Detailed Findings

GEF support has direct linkages to the key outcomes of the Samoa Development Strategy.

6. The GEF has supported key outcomes for the protection/conservation of biodiversity, protection of water catchment, increased awareness about potential climate change impacts, importance of ozone depleting substances, community based natural resources management and community development. The GEF Enabling Activities coincided with the development of the national policies relating to Samoa’s National Environmental Management Strategy. For example, the Biodiversity Policy was developed in conjunction with the National Biodiversity Strategic Action Plan (NBSAP) and the Watershed Management Policy and Water Resources Policy took advantage of the International Waters project to advance their implementation. Furthermore, it was found that the GEF had targeted national priorities established under Samoa’s environmental policies. This consistency of GEF support with national priorities has helped develop a strong ownership of GEF activities.

GEF is the main source of external financial assistance to Samoa’s environmental protection and conservation needs.

7. The GEF contributes about 60% of total external funding to the environment sector in the country. Samoa thus, has a high level of dependency on GEF financing to meet its needs and it is expected that this dependency situation will continue and perhaps increase in future.

GEF modalities of support have been appropriate to the state of Samoa’s development.

8. The modalities supported so far (i.e., PDFs, SGP, EAs, MSP and regional projects) are relevant and appropriate for Samoa’s capacity, knowledge base, existing environmental frameworks and type of environmental issues. The availability of PDF funding has been highly valued by different GEF stakeholders as this funding makes it possible to devote the time and resources needed to achieve a thorough understanding of the issues and modalities of intervention in preparation for a project (i.e., stakeholder consultations, improve existing capacity by hiring expertise to prepare project documents and follow GEF guidelines). The country has received support to fulfill the reporting requirements from the conventions where such reporting is eligible for GEF support. With the exception of land degradation and the National Capacity Self Assessment which are near completion, all other Enabling Activities have been completed. Regional approaches were found appropriate when dealing with transboundary issues and the Small Grants Programme appropriate for providing NGOs and community groups with a transparent access to GEF support.

Conclusion 2: All GEF funded projects are highly relevant to the GEF mandate and focal areas but slow follow-up support from government sources could jeopardize the sustainability of results.

Detailed Findings

All GEF-funded projects were developed and approved on the basis of their relevance to the GEF mandate and focal areas strategies.

9. GEF projects have focused on biodiversity, climate change, land degradation or international waters. Enabling activities have concentrated largely on capacity building.

The sustainability of projects results could be jeopardized

10. For example, while the Marine Protected Areas project and South Pacific Biodiversity Conservation Project (SPBCP) had met their objectives through the establishment of four community-based and community-managed protected areas in Samoa all these protected areas have suffered from inadequate financial follow up support from the government since the end of GEF funding. Another example is the Marine Protected Areas, with very ambitious objectives, which established high costs of services that the government could not sustain once the GEF funding ended.

Results of the Portfolio

Conclusion 3: Enabling activities have supported Samoa in building the foundations for its environmental frameworks and strategies which are necessary conditions for generating global environmental benefits.

Detailed Findings

GEF support achieved its greatest results in the area of policy and strategy development.

11. Samoa has completed the necessary national plans, policies and legislation relating to the environment such as National Biodiversity Strategic Action Plan, National Action Plan for Adaptation, land degradation National Action Plan and POPs National Implementation Plan. Furthermore, as a systematic approach to addressing the environmental issues in Samoa, the country through the Ministry of Natural Resources and Environment has focused over the last 15 years of GEF support on building its capacity. The Ministry’s capacity has been built by developing the necessary strategies and supporting other relevant stakeholders so all of these actors are able to adequately implement the plans developed. The Ministry has contracted a “Consultant” as a staff to deal with GEF matters. In 1992 the Ministry had only 5 staff dedicated to environmental work and now it has grown to over 100 staff dealing with the entire spectrum of environmental issues. Staff that had managed GEF projects now completed has been retained within the Ministry sustaining the lessons learned from previous experiences.

Enabling Activities in climate change have supported strategies and frameworks

12. A good example of enabling activities supporting strategies and frameworks comes from the climate change enabling activities. They have contributed to increased public awareness about greenhouse gases and ozone depletion substances, as well as natural disasters and their potential impacts on the environment and people. Priorities identified in the National Action Plan for Adaptation are beginning to be implemented and mainstreamed into investments including the Coastal Infrastructure Management Plans and Coastal Emergency Recovery Project projects funded as credits from the World Bank. Furthermore, the Draft National Energy Plan has made notable progress in promoting the use of renewable energy with pilots on solar energy and coconut oil underway, and planning for more hydro schemes in the island of Savaii.
13. All of these actions are necessary conditions for impacts to be able to emerge. By supporting the establishment of these policies, strategies and framework the GEF support has contributed to building a strong foundation for Samoa to make a useful contribution to international efforts to protect the global environment.

Conclusion 4: Completed projects have achieved concrete on-the ground results but reporting on results has limitations because of poor quality of final evaluations and limited baselines.

Detailed Findings

GEF support in the biodiversity focal area enabled the conservation and sustainable management of forest and marine ecosystems.

14. GEF projects facilitated the participation of more than 20 village communities within critical forest and mangrove ecosystems on the island of Upolu in resource conservation and management and helped build local capacity for the effective planning and management of Samoa’s environment. The Marine Protected Areas project initiated bans on commercial scuba fishing within these protected areas which the Ministry of Agriculture and Fisheries subsequently expanded through a national ban of this fishing practice throughout the country and adopted by about 50 communities. The project also imposed bans on the commercial harvesting of sea turtles within the protected areas thereby supporting regional and international efforts to protect these endangered marine animals.
15. The Species Conservation Component of the South Pacific Biodiversity Conservation project initiated efforts for the conservation of marine mammals and turtles in the region, which in Samoa led to the ban on the commercial harvesting of sea turtle at the national level. Since 1995 when the First Pacific Year of the Sea Turtle campaign was launched, populations of sea turtles have increased in Samoa both at the nesting beaches and from sightings by fishermen and divers.
16. Some anecdotal information on impacts was found regarding marine ecosystems and the interventions through the Marine Protected Areas project and the SGP activities. The Marine Protected Areas project collected baselines that later on helped in showing increases in the fish population in the last few years. In the two marine communities visited by the evaluation and supported by SGP activities reported improvements to coral health and fish populations. In particular, it was reported by one of the communities that fishermen from neighboring villages are coming to their no-take zone, illegally, because the fish population is better.

Evaluating the impacts of GEF funded initiatives is not straightforward.
17. Often the type of information generated by project evaluations is largely limited to reports on outcomes, and does not contain impacts on environmental conditions. The absence of information on project impacts is also attributed to the fact that evaluations were conducted before intended project impacts could be detected or have had time to emerge. In fact, it has been suggested that project impacts often can not be detected until well after the projects have ended. Many GEF funded projects in Samoa have been completed only over the last 2 or 3 years.

Other results on the ground have been achieved through the replication of approaches, processes and lessons

18. Further to results on the ground, other results were achieved from the replication of approaches, processes and lessons coming out from the experience of a number of GEF-funded projects into new GEF initiatives and other development assistance programs in Samoa. For example, the community-based conservation approach supported by the South Pacific Biodiversity Conservation project in Samoa was replicated at a larger scale by the Samoa Marine Protected Areas project. The consultative and participatory processes that were important features of both initial regional projects (South Pacific Biodiversity Conservation and the International Waters projects) have been accepted as best practice for all other environment initiatives in Samoa especially given the customary nature of land and natural resource ownership that exists. Village bans on the use of certain types of fishing gear and practices in Marine Protected Areas have been adopted by around 50 other communities in Samoa and the ban on the commercial harvesting of sea turtles under the MPA MSP complemented the government’s own efforts to protect migratory species and marine mammals in Samoan waters.

Efficiency

Conclusion 5: Samoa has improved its efficiency to access GEF funding, but there are still some obstacles.

Detailed Findings

Samoa has improved its efficiency to access GEF funding…

• The Ministry of Natural Resources and Environment has improved its capacity by retaining expertise within its staff, hiring a GEF “Consultant” as a staff dedicated to coordinate all GEF activities and expanding its mandate to cover most environmental issues.
• All enabling activities have produced action plans and strategies that are ready for implementation
• The GEF SGP has been recognized as an efficient mechanism for delivery support to local communities and for local communities to access the GEF. Support from the SGP is already helping increase the visibility of the GEF throughout Samoa. Its flexibility and easy access by village communities and NGOs enables the SGP to respond effectively to country priorities at the local/community level. The small amounts of funds involved are easily absorbed by the limited capacity of local communities and the small community-based projects supports are more manageable and their outcomes can be easily sustained by local groups. These features are often not present in medium sized and full sized projects that are usually more difficult to sustain after donor funding has ended.
• Samoa has implemented projects using most of available GEF modalities from enabling activities, medium size projects, projects approved under “umbrella” global projects (which have a national component), regional projects and SGP.
• The government’s willingness to reach out to other GEF Agencies in addition to UNDP to implement the action plans and strategies with potential generation of global environmental benefits.
• Sharing lessons coming from GEF projects within and outside the country.


…but there are still some obstacles

• The GEF project cycle has too many steps, it is too long, and costly. Samoa’s experience with the implementation of GEF supports the findings of the Joint Evaluation on the GEF Activity Cycle and Modalities. Furthermore and consistent with the findings of other evaluations from the GEF Evaluation Office, the foremost issue facing this type of analysis was the absence of project information. The recently completed evaluation on the GEF Activity Cycle provided the most accurate information but was not always applicable since that evaluation did not collect information for enabling activities, which are half of the GEF support activities in Samoa. In general, the GEF still does not properly and systematically compile and conduct quality control of project data (for example, project cycle dates, status and finances). Uncertainties about where projects are within the project cycle are still common among national proponents.
• Lengthy delays between project preparation and actual start up hinders implementation. There are some variations in the time it takes to prepare and implement GEF projects in Samoa according to modality. Enabling Activities take between 3 to 6 months to prepare (from PDF approval to project approval) and then 3 to 4 years for implementation, which is longer than the GEF expectation of 18 months. The regional full sized projects (most of them including 14 countries) have taken between 6 months to 2 year to prepare and up to 10 years for implementation. For example the South Pacific Biodiversity Conservation project took less than 8 months to design but it took almost 16 months for UNDP and GEF to approve the design (making the design and project document acceptable to the GEF). The more than 2 years preparation, “wait and see period,” created negative feedback, reduced the readiness of the project for start up and reduced the willingness and enthusiasm of participants.
• The implementation of the RAF has created additional uncertainties, particularly about the fate of projects in previous pipelines.
• An additional ingredient to the issues from the Joint Evaluation is that harmonization has not taken place among all players working in the environment sector although two of the main donors, AusAid and NZAid, are now attempting to harmonize their contributions to Samoa. In the case of the environment, AusAid has taken the leadership. The different systems (those of GEF, its Agencies and the Government) have different requirements for project preparation, monitoring and reporting. For example, a project implemented through SPREP would have different reporting requirements for GEF, UNDP, SPREP and national governments, removing the limited capacity for implementation to reporting.
• Most relevant government agencies have not prepared and implemented GEF. This is affecting the full capacity of Samoa to reach access of the GEF. Although many government agencies have participated in the implementation of GEF projects only the Ministry of Natural Resources and Environment has been the executing agency responsible for implementing GEF projects. To date, NGOs and community-based organizations only participate in the SGP mainly due to their lack of capacity to implement MSP and FSP and limited cofinancing.


Conclusion 6: Most GEF Agencies have not been engaged in Samoa mainly because of the high transaction cost and limited understanding of GEF objectives and procedures.

Detailed Findings

The leading GEF Agency in Samoa is UNDP.

19. Why only UNDP? Stakeholders indicated that the main reason is that UNDP has an office in Samoa and recently placement of a UNDP-GEF Adviser based in this office. The World Bank and Asian Development Bank have extensive portfolios of loans, currently active and combined US$70 million of investments and US$10 million of technical assistance. Furthermore, both banks are working in areas very relevant to the GEF, for example, cyclone recovery, infrastructure improvement along coastal areas, power sector improvement, sanitation and drainage and small business development. None of these loans have included GEF co-financing. FAO also has an important technical assistance program with the government of Samoa with no plans to include GEF.

Most relevant GEF Agencies have now a presence in the Pacific region.

20. Most relevant GEF Agencies have now established (or are planning to expand) their presence in the region: UNDP plans to increase the number of national office in the region and has brought from its regional office in Bangkok a GEF person to the Samoa regional office, ADB has an officer in Fiji, World Bank has an office in Syndey, UNEP is bringing an additional person to be located within SPREP, and FAO has a regional office in Samoa.

High transaction cost and limited knowledge on the GEF are the main reasons but they are others.

• high transaction cost of developing stand alone GEF activities in the Pacific (i.e., high cost of airline tickets within and coming into the region; long time to travel; limited local network of consultants requiring outside consultants to come in to the region; etc.);
• lack of awareness and knowledge about the GEF, not fully understanding the potential of GEF objectives and their complementarity with their regular activities
• lack of internal communications within Agencies about the possibilities of GEF and GEF procedures
• complexity of accessing GEF funds and lengthy project preparation (out of phased with the Agencies own project cycles).
• limited GEF resources available in Samoa which makes the investment less cost-effective.


3. RECOMMENDATIONS

Recommendations to the GEF Council

Recommendation 1: The proposed programmatic approach for the Pacific SIDS should take into account Samoa’s experience.

21. The lessons from Samoa’s experience with the GEF should be taken into account when developing the proposed regional programmatic approach for PIC SIDS for implementation in GEF4. Although no two countries are alike, situations are different and diversity of this region should be recognized, nevertheless, Samoa shares common problems with the rest of the Pacific Island States: limited capacity, high transaction cost of doing business, and high vulnerability and fragile ecosystems. The following bullets present the key lessons coming from the GEF experience in Samoa:
- Focus of GEF support to Pacific SIDS should be to assist countries to establish the foundation for policies and strategies and develop action plans, frameworks and priorities, primarily through enabling activities. When the foundation and priorities have been established, as is the case of Samoa, then the focus of the GEF support should be on the implementation of these priorities and action plans that are will generate global benefits.
- There should be recognition of comparative advantages of the different GEF stakeholders (national, regional, and global). In particular, there should be a clear discussion and agreement of the roles and responsibilities of the GEF Secretariat, Council, SPREP, UNDP, other GEF agencies and bilateral donors in this programmatic approach. The GEF is a major player in the environment sector of the region but it is not the only one.
- There should be enough flexibility to recognize the different capacities of the different PICs. A one size fit all approach should not be proposed.
- The case of Samoa confirms that high transaction costs are important in the Pacific region and should be taken into account. There are ways to reduce these costs specially when GEF activities are considered within regular programs of GEF Agencies already working in the region, as part of their regular programs and activities. GEF stand alone projects should not be encouraged so some of these transaction costs can be reduced.
- Harmonization needs to be strengthened across GEF stakeholders. The experience of NZAid and AusAid should be reviewed and recognized as a possible way forward.
- GEF, in partnership with STAP and PIC SIDS, should more specifically identify the global environmental benefits in Samoa and the Pacific. Two areas that are still not clear across the GEF system are: global benefits of marine resources and defining the role of the GEF on adaptation to climate change impacts.


Recommendations to the Government of Samoa
Recommendation 2: Environmental concerns as cross-cutting issue need to become visible in the Samoa Development Strategy.
22. Although environmental concerns have been well integrated into the many sectors and policy areas, the environment is not identified as a particular priority or sector in the Samoa Development Strategy. Instead environment is considered as a cross-cutting issue. The lack of clarity regarding environment concerns has cause confusion among external partners of Samoa when it comes to financial support. The sector is not recognized as a priority so donors do not prioritize it for support.

Recommendation 3: Increased participation by other stakeholders (ministries, civil society, and private sector) in implementing GEF supported projects will increase national capacity.

23. It has been recognized by the evaluation that the Ministry of Natural Resources and Environment’s capacity to develop and implement GEF projects has increased considerably in the last few years. The implementation of the national priorities and action plans developed from GEF support involve activities in many sectors of the country’s development strategy. The Ministry alone can not implement all of these plans. It is recommended that the Ministry reaches out to other sectors of Samoa, both within government and civil society, to assist in the implementation and increase the country’s capacity to access and implement those plans. For example, the Ministry could develop a proactive plan for public awareness and capacity building on GEF issues and to create the demand for GEF funding in those other sectors.