Nigeria: Midterm Evaluation of the Contraceptive Logistics System

Publication date: 2005

Nigeria: Midterm Evaluation of the Contraceptive Logistics System Federal Ministry of Health of Nigeria Nigeria: Midterm Evaluation of the Contraceptive Logistics System Briton Bieze Lea Teclemariam Timothy O‘Hearn Federal Ministry of Health of Nigeria 2005 DELIVER DELIVER (a six-year, worldwide, technical assistance support contract) is funded by the U.S. Agency for International Development (USAID). Implemented by John Snow, Inc. (JSI), (contract no. HRN-C-00-00-00010-00) and subcontractors (Manoff Group, Program for Appropriate Technology in Health [PATH], and Social Sectors Development Strategies, Inc.), DELIVER strengthens the supply chains of health and family planning programs in developing countries to ensure the availability of critical health products for customers. DELIVER also provides technical management of USAID‘s central contraceptive management information system. This document does not necessarily represent the views or opinions of USAID. It may be reproduced if credit is given to John Snow, Inc./DELIVER. Recommended Citation Bieze, Briton, Lea Teclemariam, and Timothy O‘Hearn. 2005. Nigeria: Midterm Evaluation of the Contraceptive Logistics System. Arlington, Va.: John Snow, Inc./DELIVER for the U.S. Agency for International Development. Abstract Since 2002, the Nigerian Federal Ministry of Health (FMOH) has been working to revitalize its contraceptive logistics system so it can improve product availability to clients and can move toward reproductive health commodity security. As an initial step, a baseline logistics system assessment was conducted in June 2002 to identify strengths and weaknesses of the system. In March 2005, a follow-on evaluation of the system was conducted to measure progress made since 2002, as well as to gather baseline data for five new USAID- supported focus states. Two data collection instruments, the Logistics System Assessment Tool (LSAT) and the Logistics Indicators Assessment Tool (LIAT), were used to gather qualitative and quantitative information during this assessment. A number of strengths were identified, including a significant increase in product availability for nearly all methods and an increase in the percentage of facilities updating their stock cards. However, many important weaknesses were also found, including a dearth of logistics system management tools in the facilities; a failure to order additional products, especially at the store level; and stock levels that are below minimum levels. Generally, the system has not been fully implemented as designed. As a result, the increased stock levels will not last because facilities are not using the system to maintain stock levels above minimum levels and to reorder additional products. Over the past several years, the reproductive health logistics system in Nigeria has seen some important improvements. However, if Nigeria is to avoid product shortages in the future, the contraceptive logistics management system (CLMS) must be fully implemented. DELIVER John Snow, Inc. 1616 North Fort Myer Drive, 11th Floor Arlington, VA 22209 USA Phone: 703-528-7474 Fax: 703-528-7480 Email: deliver_project@jsi.com Internet: deliver.jsi.com Contents Acronyms .vii Acknowledgments . ix Executive Summary . 1 Product Availability . 2 Logistics System Performance. 2 Recommendations. 3 Background. 5 Overview of the Redesigned Logistics System. 5 Objectives of the Assessment . 7 Assessment Methodology . 7 Composition of Assessment Teams . 7 Sampling Methodology. 7 Description of the Sampling Plan . 8 Limitations of the Survey. 8 Findings . 9 2005 Data. 9 Logistics System Performance. 12 Selected Findings by State . 15 2002 and 2005 Comparison Data. 16 Conclusions and Recommendations . 21 Conclusions. 21 Recommendations. 23 Figures 1. Availability of Contraceptive Methods on Day of Visit (based on physical inventory). 10 2. Percentage of Stores and SDPs with Stock below the Minimum Stock Level . 12 C.1. Percentage of Facilities with Mix of Methods Available on Day of Visit. 38 iii Tables 1. Stockouts during the Previous Six Months . 11 2. Percentage of Facilities with Stock Cards and Accuracy of Information Recorded. 13 3. Percentage of Stores and SDPs That Adhere to the Storage Guidelines. 15 4. Comparison of Availability of Contraceptives on the Day of Visit between 2002 and 2005 . 17 5. Percentage of Facilities Trained in Logistics Functions, 2002œ2005 Comparison. 17 6. Percentage of Facilities with Stock Card Available, 2002œ2005. 18 7. Percentage of Facilities with Updated Stock Card, Comparison 2002œ2005 . 18 8. Percentage of Facilities That Received Supervision Visits in the Past Six Months, Comparison, 2002œ2005. 19 9. Percentage of Facilities Adhering to Storage Guidelines, Comparison, 2002œ2005 . 19 A.1. Facility List . 27 B.1. Team Composition. 35 C.1. Stores and SDPs Managing Contraceptives at Time of Visit . 37 C.2. Percentage of Facilities Adhering to CLMS Guidelines. 39 C.3. Supervision and Reporting by State. 40 C.4. Availability of Contraceptive Products in Public SDPs by State on Day of Visit (based on physical inventory). 41 C.5. Number of Facilities with Product in Stock. 42 E.1. Storage Conditions. 62 E.2. Stock Status. 65 E.3. Comparison of Quantity Ordered and Quantity Received . 66 E.4. Order Fill Rate–Only for State and LGA Stores. 68 Appendices A . 27 B. 33 C. 37 D. 41 E . 49 iv Acronyms CHEWs community health extension workers CLMS contraceptive logistics management system DCDPA Department of Community Development and Population Activities ECP emergency contraceptive pill FCT Federal Capital Territory FEFO first-to-expire, first-out FMOH Federal Ministry of Health FP family planning IUCD intrauterine contraceptive device LGA local government area LIAT Logistics Indicators Assessment Tool LMIS logistics management information system LSAT Logistics System Assessment Tool M&E monitoring and evaluation MCH maternal and child health NGO nongovernmental organization NPHCDA National Primary Health Care Development Agency PPFN Planned Parenthood Federation of Nigeria RH reproductive health RH/FP reproductive health/family planning RHCS reproductive health commodity security SDP service delivery point SFH Society for Family Health SMOH State Ministry of Health UNFPA United Nations Population Fund USAID United States Agency for International Development v vi Acknowledgments The assessment team sincerely thanks the Federal Ministry of Health (FMOH) of Nigeria, United States Agency for International Development (USAID)/Nigeria, and USAID/Washington for their continuous assistance and support, without which this activity could never have been completed. Many people in those organizations deserve special thanks, including Dr. Nike Adeyemi, Director of the FMOH Department of Community Development and Population Activities (DCDPA), and Akua Kwateng-Addo and Bunmi Dosumu of USAID/Nigeria. Their assistance and support was instrumental in the assessment. Appreciation also goes to all organizations and individuals in Nigeria who provided insights into the operations of the contraceptive logistics system through interviews or participation in assessment activities. Many of the findings and analysis in this report are based on those inputs. The United Nations Population Fund (UNFPA) deserves special acknowledgment for being a valuable partner in strengthening the contraceptive logistics management system. Finally, we acknowledge and thank the 32 members of the assessment teams who carried out the field work with dedication and provided the results for the assessment and final report: Lawrence Anyanwu, Elizabeth Attah, Victoria Akinrolabu, Greg Osuba, Esther Ladipo, Esther N. Yiga, Olusunde Oluseyi, Shigudu Koche, Y. Y. Abdullahi, Hajia Yaya Tijjani, Nafisat Lara Yakubu, Rufai Ibrahim, Pauline Aribisala, Aishat Lawan, Yunusa Bala, Zakari Zakariya, Musa Odiniya, Salamatu Suleiman, Suleman Manu, Dr. Taiwo Avbayeru, Hanidu Mosun, Teleoia Kajeoro, Tayo Olugbemi, Ralph Olayele, Ojediran Mojoyinola, Ester Fadeleu, Greg Izuwa, Charity Nnamani, Ukagwu Ngozi, Dr. Bose Adeniran, Dr. W. Imongan, and U. Aliu. An assessment of this magnitude involves inputs and contributions from so many people that it is impossible to acknowledge all by name, and we would unwittingly omit many if we tried to do so. Many other organizations and individuals contributed to the assessment in different ways, and we are sincerely grateful for their extensive assistance. vii viii Executive Summary The Nigerian Federal Ministry of Health (FMOH), in collaboration with other stakeholders working in reproductive health (RH), continues to strive toward achieving contraceptive security for its people. In support of this goal, a baseline assessment of the contraceptive logistics management system (CLMS) was conducted in 2002. This assessment was the first step toward improving contraceptive availability and strengthening RH logistics functions in the public sector. In response to the findings from the 2002 assessment, the CLMS was redesigned and the National Strategic Plan for Reproductive Health Commodity Security was drafted, with key support from USAID, United Nations Population Fund (UNFPA), the POLICY Project, and the DELIVER project. As part of the nationwide roll-out of the redesigned CLMS from 2003 to 2004, the logistics staff was trained on the new system, new forms and handbooks were distributed, and seed stock was distributed to state and local government area (LGA) stores and service delivery points (SDPs). In February 2005œMarch 2005, the FMOH, the National Primary Heath Care Development Agency (NPHCDA), and DELIVER conducted a second assessment of the CLMS. The primary objectives of the assessment were to accomplish the following: ° Evaluate the progress made toward increased product availability and improved logistics practices since the 2002 baseline assessment. ° Provide baseline information on key logistics performance indicators, including product availability in the five USAID-supported focus states. ° Furnish the FMOH with a midterm evaluation of the logistics components of the national RH commodity security strategy. The assessment teams used a quantitative tool (Logistics Indicator Assessment Tool or LIAT), which was developed by DELIVER, to measure the performance of the CLMS, including the quality and flow of information through the system; the commodity availability at all levels of the system; the way of ordering and issuing; the system for reporting; the manner of monitoring and supervision; and the storage conditions. Additionally, a qualitative data collection instrument (Logistics System Assessment Tool, or LSAT) was used to assess central-level components, such as the organizational structure and financing that had an effect on the system. The assessment took place in nine states: Bauchi, Edo, Enugu, Oyo, Sokoto, Kano, Lagos, Nassarawa, and the Federal Capital Territory (FCT). The assessment teams visited 199 facilities (41 stores and 158 SDPs) in the nine states over the course of two weeks. The assessment included only the 182 (38 stores and 144 SDPs) that were found to be managing contraceptive methods. Five of the nine states were visited in both the 2002 and 2005 assessments; the same facilities were visited in 2005 as had been in 2002, when possible. Both studies assessed 79 facilities; 10 were stores and 69 were SDPs. Seven contraceptive methods being managed in 2002 were also being managed in 2005. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 1 This report presents detailed results from the assessment. Strengths of the system include an increase in availability of contraceptives in many facilities, as well as better management of key logistics data. Some states in particular, such as the FCT and Nassarawa, are for the most part using the system as it was designed. However, significant weaknesses remain in the system and could jeopardize the successes reached since 2002, including failure to order additional supplies when needed and poor reporting rates. In addition, training and supervision were substandard. In response to the identified weaknesses, this report describes specific recommendations for stsrengthening the contraceptive logistics system. Product Availability A positive accomplishment over the past three years has been the increase in availability of nearly all contraceptive methods in both stores and SDPs. Well over half of all facilities in 2005 had most of the methods in stock, with Depo-Provera® the most widely available and with Norplant® and Lo-Femenal the least widely available in the facilities. Compared to the baseline assessment, there was a significant increase in the availability of the following four products: Noristerat, Exluton, Depo-Provera, and male condoms. However, the increase in the availability of contraceptive stock could be more a reflection of the continued availability of seed stock than the result of success of the CLMS. Despite increased availability, a long duration of stockouts and a high percentage of facilities that are stocked below the established minimum stock level actually signal important weaknesses in the system. Products stocked out at stores were stocked out for an average of at least two months, with three out of the nine products stocked out for the full six months. SDPs were generally not stocked out for as long a time as in stores. However, even with this improved performance, more than half of SDPs were stocked below minimum levels for most products. Stockouts of a long duration and stocks below minimum levels indicate a serious problem with the CLMS–the inability of facilities to reorder additional stock and to order at the right time. Those issues, if not adequately addressed, will lead to stockouts in the near future. Logistics System Performance From 2002 to 2005, the performance of the logistics system improved in some areas but not in others. The percentage of facilities with stock cards that were available and had been updated increased during the three years. The percentage of facilities maintaining stock cards increased for every product except for intrauterine contraceptive device (IUCDs), and the percentage of facilities updating their stock cards increased for every method. This finding suggests that facilities are adhering better to some of the stockkeeping guidelines of the CLMS. Record keeping and reporting remain significant weaknesses of the logistics system, however. Less than 50 percent of facilities maintained most of the required CLMS records, and even fewer sent the reports to the higher levels. The percentage of trained staff members remained unchanged between 2002 and 2005. Although this finding may be partly because of the wording of the questions in the two assessments, other reasons include the presence of (1) individuals who have nonlogistics responsibilities but who are trained in CLMS; (2) staff members who are trained in CLMS and then reassigned to other tasks or facilities and (3) staff members who are trained in CLMS and who retired from their jobs. There was also no change in the percentage of supervisors conducting supervisory visits from 2002 to 2005. Supervisors lacked knowledge, supervision forms, and funding to conduct such visits effectively. However, on a more positive note, most stores that conducted supervision visits 2 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System at all did so during the previous three months, which indicates that the visits that did occur were made on a timely basis. The percentage of facilities adhering to storage guidelines increased for almost every guideline for proper storage between 2002 and 2005. Those improved storage practices suggest that stored products were lasting longer and that there was a reduced risk of storing or dispensing expired products. Recommendations Although there was a dramatic increase in product availability during the previous three years, the LIAT results exposed several significant weaknesses of the CLMS. To address those issues, this report proposes feasible and specific recommendations. Most of the identified weaknesses were problems related to implementing and understanding the CLMS, difficulties in adhering to CLMS guidelines, and concerns about the low demand for family planning services. The following are selected recommendations: Develop on-the-job training (OJT) approaches. OJT could be used to reinforce CLMS concepts and to strengthen skills on how to correctly complete the logistics management information system (LMIS) forms, to adhere to distribution and reorder schedules and procedures, and to comply with storage guidelines. Additional CLMS training would also be beneficial for facility staff members in states with a high staff turnover. Increase the frequency and improve the quality of supervision visits. Supervision visits are a key opportunity for state and LGA reproductive health/family planning (RH/FP) coordinators to reinforce CLMS concepts and to bolster the skills of service providers. Regular visits would also help facility staff members to communicate problems to their supervisors and for supervisors to impart practical information on inventory control and logistics management information to the facility staff. Intensify advocacy activities. Advocacy for the CLMS would raise the awareness of state ministries of health and of local government health departments about the importance of ensuring continuous contraceptive availability. Local funding of reproductive health and family planning activities should improve the long-term sustainability of service and product provision. At the community level, advocacy campaigns for increased patronage of public sector sites would create an awareness about the availability of contraceptives in SDPs and should, therefore, stimulate demand for family planning and other RH products and services. Support behavior change communication activities for family planning and reproductive health. Through partnerships with organizations that have expertise in behavior change communication strategies, the public sector‘s family planning program should become more widely known and accepted by communities and individuals. Addressing this problem would also help relieve some of the financial burden on those states whose cost-recovery systems are not functioning well because of low consumption of contraceptives by clients. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 3 4 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Background The FMOH and its partners have recognized that sound logistics systems will ensure the continuous availability of RH commodities. As a result, they have increased attention and resources to strengthen the CLMS. This effort has been guided by the National Policy on Population for Sustainable Development, which was revised in January 2004. The overall goal of the national policy is to achieve the following: • Improve the quality of life and the standard of living for the Nigerian people. • Expand access to and coverage of RH services and then improve the quality of those services. • Strengthen and expand a comprehensive family planning and fertility management program to ensure that all couples or individuals who want contraceptives have access to a reasonable range of methods at affordable prices. • Strengthen and improve safe motherhood programs to reduce maternal mortality and morbidity and to enhance the health of women. In 2003, to support the achievement of these objectives, the FMOH and its partners developed a National Strategic Plan for Reproductive Health Commodity Security. The six components of this strategy include policy, finance, logistics, service delivery, demand, and coordination. Establishing a well-functioning CLMS was central to accomplishing the logistics component, and it was the main focus of the government‘s family planning effort from 2002 to 2005. The effort to strengthen the CLMS began with a baseline assessment. In 2002, the FMOH/Department of Community Development and Population Activities (DCDPA)–in collaboration with John Snow, Inc. (JSI)/DELIVER, United States Agency for International Development (USAID), and the United Nations Population Fund (UNFPA)–conducted a logistics assessment to provide key baseline indicators on the performance of the contraceptive supply chain at all levels. The assessment provided program planners with information to design interventions to improve the CLMS and to measure progress toward reproductive health commodity security (RHCS) over time. Overview of the Redesigned Logistics System As a follow-up activity to the baseline assessment, DCDPA and its partners organized a logistics system redesign workshop to improve the effectiveness and efficiency of the CLMS. The workshop resulted in five major outcomes. First, to shorten the pipeline, the zonal tier of warehouses was eliminated. Second, standard operating procedures were developed and introduced to provide guidance to storekeepers and service providers. Third, new logistics forms were developed and introduced at all levels of the system. Fourth, cost recovery was introduced to generate funding for resupply, supervision, transportation, and–at local levels–incentives. Fifth, reproductive health and family planning logistics officers and service providers were trained in all 36 states and the Federal Capital Territory (FCT), at all levels of the system. The CLMS National Handbook, which was developed as part of the redesign, covers seven primary topics: (1) forecasting and procurement, (2) inventory management, (3) clearing and storage, (4) transportation and distribution, (5) logistics information management system (LMIS), (6) cost recovery, and (7) logistics system monitoring and supervision. The handbook and other Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 5 CLMS management tools and contraceptive seed stock kits were distributed during the national rollout. The forecasting and procurement elements of the CLMS are the responsibility of the FMOH at the central level. The system prepares forecasts on an annual basis using dispensed-to-client data from the SDPs and distribution data from the central and state stores. The FMOH works with UNFPA to finalize its procurement plans; UNFPA organizes funding through CIDA donations and its thematic trust funds and then uses its procurement system to bring contraceptives to the country. Under the system, inventory management uses defined minimum stock levels and fixed ordering periods. The system is structured so that facilities order from the immediate next higher level according to the established ordering frequency (SDPs order from the LGAs, LGAs order from the states, and states order from the central warehouse). The central level is responsible for the clearing and storage of RH commodities, as well as for transit and custom clearance when contraceptives arrive in Nigeria. Commodities are then stored in the central contraceptive warehouse in Lagos. The transportation and distribution of commodities are implemented according to a distribution calendar at all levels of the system. The LMIS component of the system collects data on daily consumption, stock on hand, and distribution activity (for stores only); it reports to the next higher level of the system. LMIS information is used to make key management decisions and, at the same time, to improve customer service. The cost-recovery scheme is a significant component of the CLMS redesign. A price structure was developed and the system was designed to operate on a cash-and-carry basis. Below the central level, the cost-recovery system operates like a contraceptive revolving fund, using funds earned from contraceptive sales to purchase future supplies and margins to cover other costs, such as transportation and supervision. While the system primarily serves the public health sector facilities, it also provides contraceptives to selected central- and state-level, not-for-profit, nongovernmental organizations (NGOs), such as the Planned Parenthood Federation of the Nigeria (PPFN). 6 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Objectives of the Assessment The main audiences for this effort were national planners and managers, donor agencies, and NGOs. This assessment serves particularly to assist the FMOH, UNFPA, USAID, and key stakeholders to evaluate all aspects of the CLMS. The primary objectives of the assessment were to accomplish the following: 1. Evaluate the progress made toward increased product availability and improved logistics practices since the 2002 baseline assessment. 2. Provide baseline information on key logistics performance indicators, including product availability, in the five USAID-supported focus states. 3. Furnish the FMOH with a midterm evaluation of the logistics components of the RHCS‘s five-year strategy. Assessment Methodology The primary tools used for the assessment were the Logistics Indicators Assessment Tool (LIAT) and the Logistics Systems Assessment Tool (LSAT). The study assessed the performance of the logistics system that manages key RH commodities, the knowledge and understanding of the system by individuals at each level of the system, the cost-recovery system, and the availability of RH commodities. The assessment collected quantitative and qualitative data on the performance of the CLMS. To collect information from all levels of facilities in the system, including the Central Medical Stores (CMS), the assessment also looked at specific activities, such as ordering and issuing, reporting, monitoring and supervision, and storage conditions. The LSAT, a qualitative data collection tool, was used to assess the CMS and to guide discussions with assessment team leaders when they reviewed their LIAT field reports. At the central store, an assessment was conducted on warehouse stock status, procurement planning, receiving process, state LMIS reports, and other key warehouse issues. A review of the central store‘s warehouse personnel structure was also completed. After the data collection phase of the assessment, the LSAT was used to review each site assessment field report. The field reports included information such as the name, location, and type of site visited; observations by team members during the site visits; and various problems that the teams experienced with the survey tools, facility location, facility personnel, or transportation and logistics. Composition of Assessment Teams The nine assessment teams comprised individuals from the FMOH, State Ministry of Health (MOH), National Primary Health Care Development Agency (NPHCDA), and COMPASS project. One individual from each organization was a member of each team, with FMOH staff members serving as team leaders. In addition, COMPASS M&E officers were assigned to teams in the five USAID focus states (Bauchi, the FCT, Kano, Lagos, and Nassarawa). See appendix 2 for a complete list of LIAT assessment team members. Sampling Methodology The selection of states was guided by the decision to evaluate the effect of the redesigned CLMS in the five baseline states where the LIAT had previously been used. For those five states, the Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 7 same LGAs and the same SDPs within those LGAs were selected as those found in the initial round in 2002 assessment. The five baseline states were Bauchi, Edo, Enugu, Oyo, and Sokoto. Four USAID focal states were also selected to establish baseline indicators for contraceptive availability and logistics system performance. The four states were Kano, Lagos, Nasarawa, and the Federal Capital Territory (FCT). The nine states provide a general geographic regional representation for the country by including at least one state from each of Nigeria‘s six geopolitical zones. From a list of the 51 LGAs that were involved with the USAID-supported COMPASS Project, five LGAs were selected randomly in four USAID focal states. In each of the LGAs, five SDPs were randomly selected from the full list of SDPs. If any LGA had five or fewer SDPs, all SDPs were included in the sample. Using the information provided on the SDP listings and confirmed by the state FP coordinators, SDPs were classified as urban/semi-urban, or rural. After a random selection of one SDP each from those two strata within the state, three additional SDPs were randomly selected from the remaining SDPs. Description of the Sampling Plan The sampling plan selected for this study was designed to assess the performance of the redesigned CLMS. It was, desirable to have a reasonably representative picture of CLMS performance at the central, state, LGA, and SDP levels. It was also desirable to capture programmatic and geographic variation in the sampling process. The assessment team intentionally oversampled the previous baseline states and facilities and the USAID-supported focal states so it could achieve the subsidiary objectives of comparison with the earlier assessment and could establish baseline data for USAID. This approach, while not yielding a true probability sample, nevertheless provided a reasonable overview of contraceptive logistics system performance longitudinally from 2002 to 2005 and as a broad cross-section in 2005. Please see appendix 1 for a complete listing of states, including the numbers of selected LGAs and SDPs assigned to each of the nine assessment teams. Limitations of the Survey • Most of the NPHCDA zonal technical officers who were part of the assessment team were new to the system and did not participate during the countrywide CLMS training. • The second week of the assessment coincided with the National Immunization Day Campaign, which made it difficult to reach some of the providers. • Data were missing in some questionnaires, which decreased the power of the analysis for some indicators. • Only a few questions remained the same in both the baseline and midterm survey tools. Therefore, direct comparisons could be made on only a few indicators. 8 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Findings The LIAT tool was used to assess the facilities in nine states on their implementation of the CLMS guidelines. Data were gathered on seven key areas: contraceptive availability, LMIS, inventory control, cost recovery, supervision, transportation, and storage conditions. The first set of results, presented below, provides a general overview of the performance of the CLMS in March 2005. The overview is followed by a brief comparison of results at the state level; a more comprehensive review of each state, which includes both qualitative and quantitative findings, can be found in appendices 3 and 4. The second section provides a comparison of selected indicators for facilities that were visited in both 2002 and 2005. 2005 Data The 2005 LIAT study included 199 facilities; 41 were stores and 158 were SDPs. Of the 199 facilities assessed, only the 182 that were managing contraceptives (38 stores and 144 SDPs) were included in the analysis. Contraceptive Availability To assess availability of nine different contraceptive methods, each survey team conducted a physical inventory of both stores and SDPs. Two other products were measured during the inventory: Neo-Sampoon and Postinor. However, those methods are being phased out of the system and were not included in the analysis. All facilities are expected to manage the nine contraceptive methods; therefore, the percentages are based on all facilities (38 stores and 144 SDPs). The exception is IUCD and Norplant, which all facilities are not expected to manage. Only facilities that reported managing those two methods are included in the denominator. Depo-Provera was the most widely available method, with 74 percent of stores and 80 percent of SDPs having it in stock on the day of visit. Male condoms and Noristerat were also available in the majority of stores, with 67 percent of SDPs and 70 percent of stores having both methods in stock. Of those facilities managing the IUCD, 80 percent of stores and 74 percent of SDPs had it in stock. Norplant was in stock in the fewest number of facilities among those facilities managing it (43 percent of stores and 46 percent of SDPs); the same was true for Lo-Femenal in stores (32 percent) and Microgynon in SDPs (40 percent). See figure 1. It is positive to note that, with the exception of the weak stock status of the three products mentioned above, well over half of all the facilities had all methods in stock. For most methods, the percentage was more than 60 percent. However, because the ultimate goal is to achieve 100 percent availability of all methods in 100 percent of facilities, much room for improvement remains. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 9 Fe ma le co nd om Ma le co nd om De po -P ro ve ra ® Ex lut on Lo -Fe me na l Mi cr og yn on No ris ter at IU CD * No rp lan t** Figure 1. Availability of Contraceptive Methods on Day of Visit (based on physical inventory) 0 10 20 30 40 50 60 70 80 90 100 Pe rc en t o f F ac ili tie s Stores SDPs *n = 30 for stores; n = 73 for SDPs **n = 7 for stores; n = 26 for SDPsMethod of Contraception Facilities were also assessed on whether they had stocked out any time during the past six months, how many times they had stocked out, and for how many days or months. This information is useful in determining whether facilities stock out on a chronic or intermittent basis. It is important to note that those percentages are based on stock cards and not on physical inventory. Therefore, the survey team had to rely on the quality and quantity of the facilities‘ record-keeping practices. As table 1 shows, the number of facilities reporting stockouts during the previous six months largely reflects the number that were stocked out on the day of the visit. As said previously, Depo-Provera was the most available commodity during the past six months (87 percent of stores and 86 percent of SDPs), followed by Exluton with nearly 65 percent of stores and SDPs having stock over the previous six months. Norplant was a particularly difficult method to keep in stock in stores, as 80 percent of them had been stocked out in the recent six months. The duration of stockouts is a significant problem, especially for the stores. For three methods– female condoms, Lo-Femenal, and Noristerat–the stores that stocked out did not have these methods for the full six months. All other methods in stores were stocked out an average of at least three months. On average, SDPs were not stocked out for as many days as stores. 10 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Table 1. Stockouts during the Previous Six Months Contraceptives Among Facilities Reporting Stockouts in the Past 6 Months Percentage of Stores Average Number of Days of Stockouts Percentage of SDPs Average Number of Days of Stockouts Female condom 10 180 25 95 Male condom 19 104 16 134 Depo-Provera® 13 123 14 69 Exluton 37 77 33 85 Lo-Femenal 29 180 20 93 Microgynon 23 68 33 94 Noristerat 13 180 27 59 IUCD 12 135 20 73 Norplant 80 120 11 missing Approximately 35 percent of facilities reported stocking out of commodities at the time of resupply. SDPs stocked out more often than stores, at 37 percent and 26 percent, respectively. In SDPs, Noristerat was stocked out the most often; Exluton was the next most frequent, followed by Depo-Provera. The two most common reasons for stockout were the failure of facilities to receive a resupply of a product and unexpectedly high demand for a product. Additionally, 30 percent of both SDPs and stores reported a usual overstock of contraceptives at the time of resupply. The male condom was the contraceptive that was the most commonly overstocked commodity, followed by the female condom, Microgynon, and the IUCD. Facilities order according to the established frequency when stock of any of the contraceptive methods falls below the minimum stock level quantity. The minimum stock level is based on monthly consumption rates for every facility; it is the minimum amount that the facility should have before reordering. Figure 2 shows that a large percentage of facilities did not order at the appropriate time, which helps to explain the long duration of stockouts. The average duration of stockouts in sites experiencing a stockout was greater than 59 days, indicating that those sites do not seek resupply regularly, as required by the CLMS guidelines. More than 30 percent of SDPs were stocked below the minimum level for all methods. For seven of the products, more than 50 percent of facilities were stocked below the minimum level. Noristerat and IUDs were stocked below the minimum level at approximately three-fourths of the facilities. This failure to reorder at a minimum stock level puts SDPs at a high risk of eventual stock out. A higher percentage of stores had quantities of product that were above the minimum level compared to the SDPs. Despite this better performance, however, more than 40 percent of the stores assessed did not keep the male condoms, Lo-Femenal, and Exluton according to their minimum stock level. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 11 Figure 2. Percentage of Stores and SDPs with Stock below the Minimum Stock Level 0 10 20 30 40 50 60 70 80 90 100 Fe ma le c on do m Ma le c on do m De po -P ro ve ra ® Ex lut on Lo -Fe me na l Mi cr og yn on No ris ter at IU CD No rp lan t Method of Contraception Pe rc en t o f F ac ili tie s Stores SDPs Logistics System Performance Logistics Management Information System Several new management tools were introduced during the CLMS training, including the CLMS Handbook, which includes job aids on how to complete logistics management forms. Although the forms were distributed during training, only half of both stores and SDPs had the forms required to collect data on consumption, stock on hand, and sales and stock management. Additionally, almost all of the stores and nearly three-fourths of SDPs had personnel trained on the CLMS. Given the poor availability of forms, it is not surprising that only 37 percent of facilities had the requisition and issue forms needed for ordering, and 41 percent had the cost-recovery records for the past six months. More than three-fifths of SDPs had the daily consumption record; about half the stores had tally cards to record the stock movement. Reporting was generally poor. About 15 percent of stores reported sending the store activity and distribution reports to the higher level. Major reasons for nonreporting included not having the forms (57 percent) and not knowing how to complete the forms (17 percent). Only four stores sent the store distribution reports to the higher level, and only one store sent in the supervision reports. However, 81 percent of stores reported being trained to complete the store activity, distribution, and supervision reports. All 38 stores reported that their SDPs are required to send them CLMS reports. The number of facilities reporting to a given store may be as few as two to as many as 106, with most stores serving six dependent facilities. Within the sample, the 34 stores assessed reported that only 72 of their 466 dependent facilities (15 percent) had sent in all of their CLMS forms during the past six months (see table 2). However, four stores reported receiving 100 percent reporting from their SDPs. 12 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Most facilities had records, but the accuracy of the data recorded was low. Accuracy was determined by comparing the number in stock on the stock card with the physical inventory. This comparison is a particularly strict measurement because a stock card with as little as one number difference from the physical inventory is considered inaccurate. Two-thirds of SDPs and just over 60 percent of stores had stock cards for at least one product. More than half of all facilities had accurate information on their stock cards for at least one product; however, only 10 percent of SDPs and 5 percent of stores that kept stock cards had accurate information on all of their products. Table 2. Percentage of Facilities with Stock Cards and Accuracy of Information Recorded Type of Facility Number of Facilities Percentage of Facilities (%) With Stock Card for at Least One Product With Accurate Information on Stock Card for at Least One Product With Accurate Information on Stock Card for Products, among Those That Had a Stock Card Store 38 61 63 5 SDP 144 67 56 10 Inventory Control Knowledge about the procedures and frequency of ordering is crucial to preventing stockouts, and it is an important element of the proper administration of the CLMS. According to CLMS guidelines, facilities should order according to the established frequency as soon as any commodity reaches its minimum stock level quantity. Fifty percent of stores reported that they had never ordered products during the previous six months, and just over one-third ordered according to the set frequency. Thirty percent of SDPs had never ordered products. However, of the ones that did order, more than 50 percent ordered every two months, as mandated in the guidelines. Of those who were trained on CLMS, 95 percent reported receiving training on how to calculate the minimum stock level. However, of those facilities that have ordered in the past six months, only half ordered according to their minimum stock level (n = 91). Facilities are instructed to place emergency orders if products fall below their minimum stock levels outside their regular order period. Of those facilities that ordered in the past six months, only 17 percent of stores and 19 percent of SDPs had to place an emergency order. The result remains that facilities are not ordering or placing emergency orders according to the CLMS guidelines. Because substantially higher percentages had stock on hand below minimum stock levels, it is clear that many facilities do not resupply their stock according to CLMS guidelines. Cost Recovery Financial record keeping and management were inadequate in most of the facilities. Less than half of the facilities‘ ledger balances matched the total income from commodity sales. Only 34 percent of the states had opened a separate account for contraceptives, but, on a positive note, most had been able to get approval to withdraw from the account to resupply contraceptives. Almost all facilities (90 percent) reported using CLMS funds strictly for the CLMS. However, only half of those facilities reported using the margins. Of those that knew how to use the margins according to the CLMS guidelines, only half had a ledger or cash book that showed the use of the margin according to those guidelines. Aside from not knowing how to use the margins, people Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 13 reported that the most common reasons for not using them included not having the necessary forms or CLMS handbook, not ordering new stock, or not having received seed stock. Supervision To ensure the proper execution of the CLMS, stores should make regular supervisory visits to the facilities they supply. However, only 55 percent of the stores had conducted such a visit. Of the stores that had conducted a visit, nearly 90 percent had conducted their visits within the past three months. Since the start of CLMS, in mid-2003, 65 percent of stores had conducted three or fewer visits, while 30 percent had conducted four or more visits. Although the stores that conducted visits did so fairly regularly, only 20 percent had the supervision reports on file. Transportation Sixty percent of stores responded that they collected contraceptives for their facilities; of those, 61 percent used public transportation to collect the products, and 16 percent used private vehicles. Ninety percent of SDPs reported collecting their contraceptives; similar to the stores, 64 percent of those used public transportation and 19 percent used private vehicles. Storage Conditions Storage areas were visually inspected by the survey teams to assess whether they were meeting the CLMS‘s recommended guidelines. SDPs were assessed on the first 12 guidelines listed in the table, while stores were assessed on all 17. A large percentage of the facilities stored their commodities according to the guidelines. For example, 91 percent of stores and 97 percent of SDPs had products in good condition; nearly all facilities had products stored away from direct sunlight, water, and humidity. The same percentage of facilities had their products stored at the appropriate temperature. However, only 61 percent of stores and 57 percent of SDPs managed commodities according to FEFO (first-to- expire, first-out), which is a critical procedure so facilities can avoid issuing expired commodities. Storage space was the major problem in most of the stores, as was evident in the arrangement of products. Table 3 shows all of the facilities that adhered to the different storage guidelines. 14 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Table 3. Percentage of Stores and SDPs That Adhere to the Storage Guidelines Storage Condition Description Store (%) SDP (%) 1. Products arranged with identification labels and expiry dates and/or manufacturing dates visible 64 54 2. Products organized for FEFO counting and general management 61 57 3. Cartons and products in good condition, not crushed as a result of mishandling, and not wet or cracked because of heat and radiation 91 97 4. Damaged or expired products or both that were separated from good products and removed 66 73 5. Products protected from direct sunlight 97 99 6. Cartons and products protected from water and humidity 94 97 7. Storage free from harmful insects and rodents 79 85 8. Storage area secured with a lock and key 94 92 9. Products stored at the appropriate temperature 100 98 10. Roof in good condition 91 96 11. Storeroom maintained in good condition 73 77 12. Space sufficient for existing products and expansion 67 69 13. Products stored separately from insecticides and chemicals 81 NA 14. Products stacked at least 10 cm (4 inches) off the floor 87 NA 15. Products stacked at least 30 cm (1 foot) away from the walls and other stacks 63 NA 16. Products are stacked no more than 2.5 meters (8 feet) high 97 NA 17. Fire safety equipment available and accessible 9 NA Selected Findings by State State-specific differences emerged when the data at the state level were evaluated. For product availability, Nassarawa, the FCT, and Enugu appear to have contraceptives stocked in a higher percentage of its facilities compared to other states in the study. In contrast, Bauchi and Kano reported the smallest percentage of facilities with available product. For example, although Enugu had among the highest percentage of SDPs with female and male condoms, Bauchi had the lowest percentage of availability for most products: Depo-Provera, Exluton, and Microgynon. And although state stores tended to be better stocked than LGA stores, no product reached availability in more than 80 percent of stores in any state. Facilities in the FCT, Nassarawa, and Sokoto have adhered best to CLMS guidelines in areas such as training, record keeping, reporting, and ordering. More than 90 percent of facilities in those three states were trained in CLMS, and more than 80 percent had all the CLMS forms. More than 80 percent of facilities in those states also had the Requisition and Voucher Forms for the previous six months, and 90 percent or more facilities in the FCT and Nassarawa had placed orders during the previous six months. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 15 Bauchi, Edo, and Kano, however, are states that had low percentages of providers who were trained on the CLMS and low percentages of providers who were not strictly adhering to the CLMS guidelines. Only 24 percent of facilities in Bauchi had received CLMS training at the time of the survey, the lowest of any state. Not surprisingly, only 16 percent of facilities in Bauchi had the CLMS forms, and no stores had submitted the store distribution reports. Only 19 percent of facilities in Kano had the CLMS forms available, despite having 82 percent of facilities trained. A comprehensive report of state-by-state quantitative and qualitative findings can be found in appendices 3 and 4. 2002 and 2005 Comparison Data Five states were visited in both the 2002 and 2005 LIATs: Bauchi, Edo, Enugu, Oyo, and Sokoto, allowing a mid-term evaluation in these areas. The same facilities were visited in all states, if possible, and comparisons were made between relevant facilities. There were 89 facilities that were assessed in both studies (10 stores and 79 SDPs). However, 10 SDPs that reported that they did not manage contraceptives in 2005 were excluded from the analysis. There were seven contraceptive methods that were managed in 2002 that were also managed in 2005 at the time of the assessment: Lo-Femenal, Microgynon, Noristerat, Exluton, Depo-Provera, male condoms, and IUCD. Because the LIAT tool underwent several revisions since 2002, only a few questions remained the same in both survey tools. Data from those questions were directly compared. The McNemar test was used to test statistical significances for those questions. The goal of the analysis was to determine if significant changes over time had taken place in availability of stock, availability and accuracy of stock cards, completion of supervisory visits, and training of facility personnel on logistics. Contraceptive Availability Between 2002 and 2005, positive changes were made in the availability of all contraceptive methods, except IUDs. Statistically significant increases occurred in the availability of four products: Noristerat, Exluton, Depo-Provera, and male condoms (see table 4). Though falling short of statistical significance, there was a 15 percentage point increase in Microgynon availability and a 6 percent increase in the availability of Lo-Femenal from 2002 to 2005. Because one important goal of a logistics system is to ensure availability of a product, this finding is important. 16 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System 2005 Table 4. Comparison of Availability of Contraceptives on the Day of Visit between 2002 and Method Percentage of Facilities with Product in Stock on Day of Visit n= Significance 2002 2005 Lo-Femenal 27 33 55 .690 Microgynon 35 50 34 .359 Noristerat 25 56 55 .003 Exluton 6 63 16 .004 Depo-Provera 56 75 61 .036 Male Condoms 5 73 22 .000 IUCD 77 71 31 .754 Logistics System Performance Logistics Management Information System. In 2002, facilities were asked if they were trained in four specific logistics functions: ordering supplies, receiving supplies, conducting inventory management, and supervising (see table 5). If they answered yes to training for all four functions, they were considered to be trained in logistics management. In the 2005 assessment, facilities were asked whether they had received training in CLMS, which included those same logistics skills. These two questions were compared and no difference was found. This finding is surprising, considering the significant CLMS training effort during the past two years. However, because the questions in the two surveys (2002 and 2005) were worded differently, the results are not strictly comparable. Table 5. Percentage of Facilities Trained in Logistics Functions, 2002œ2005 Comparison 2002 2005 n= Significance Percentage of facilities that are trained in all logistics functions (trained in CLMS for 2005) 58 57 67 1.0 In the redesigned CLMS, facilities were instructed to maintain stock cards for every contraceptive method they manage. The percentage of facilities maintaining stock cards rose substantially for every product but IUCD (see table 6). For male condoms, this increase reached a level of statistical significance at the .01 level. The increase in the number of facilities completing stock cards suggests that facilities are adhering to this aspect of the CLMS guidelines. With increased data availability, facilities will be able to monitor their own supplies and to prevent their facilities from stocking out. It is also crucial for a well-functioning LMIS. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 17 Table 6. Percentage of Facilities with Stock Card Available, 2002œ2005 Method Percentage of Facilities with Stock Card Available n= Significance 2002 2005 Lo-Femenal 33 50 18 .508 Microgynon 25 42 12 .625 Noristerat 41 59 22 .344 Exluton 20 50 10 .250 Depo-Provera 27 45 11 .500 Male Condoms 15 69 13 .016 IUCD 0 0 2 N/A Properly maintaining stock cards for products requires regularly updating the cards. Once again, the number of facilities updating stock cards increased between 2002 and 2005 for all methods (see table 7). The increased change in the percentage of facilities updating stock cards was statistically significant (P < 0.1) for Noristerat, although there were also dramatic increases for Exluton, Depo-Provera, and male condoms. Table 7. Percentage of Facilities with Updated Stock Card, Comparison, 2002œ2005 Method Percentage of Facilities with Stock Card Updated n= Significance 2002 2005 Lo-Femenal 31 69 13 .180 Microgynon 33 56 9 .625 Noristerat 28 72 18 .021 Exluton 13 63 8 .125 Depo-Provera 14 71 7 .125 Male condoms 20 70 10 .125 IUCD 0 50 2 N/A Supervision. To ensure the correct functioning of the CLMS, it is imperative to have regular supervision . Unfortunately, the number of facilities either receiving supervisory visits or conducting visits (stores only) generally remained constant during the three-year interval between studies (see table 8). Few facilities at all levels in all states reported either conducting or receiving supervisory visits. 18 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Table 8. Percentage of Facilities That Received Supervision Visits in the Past Six Months, Comparison, 2002œ2005 2002 2005 n= Significance Facilities 44 48 54 .839 Stores 83 83 6 1.0 Storage Conditions. Storage conditions improved for almost every storage guideline assessed. The increase in number of facilities adhering to those guidelines reached statistical significance (P < 0.1) for three types of conditions (see table 9). The improvements are a strong indicator that facilities have made the necessary enhancements to their storage practices as outlined in the CLMS Handbook. Better storage practices signify that the quality of products is being maintained and that risks of storing products past their expiry date or dispensing expired products is reduced. Table 9. Percentage of Facilities Adhering to Storage Guidelines, Comparison, 2002œ2005 Storage Condition Description 2002 2005 n= Significance 1. Products arranged with visible identification labels and expiry dates or manufacturing dates or both 7 12 28 .267 2. Products organized for FEFO counting and general management 10 14 31 .481 3. Cartons and products in good condition, not crushed as a result of mishandling, and not wet or cracked because of heat or radiation 27 34 38 .092 4. Damaged or expired products or both separated from good products and removed 21 29 41 .134 5. Products protected from direct sunlight 42 43 45 1.0 6. Cartons and products protected from water and humidity 36 38 41 .625 7. Storage free from harmful insects and rodents 35 35 45 1.0 8. Storage area secured with a lock and key 37 43 47 .146 9. Products stored at the appropriate temperature 37 39 46 .791 10. Roof in good condition - - - 11. Storeroom maintained in good condition 14 25 33 .013 12. Space sufficient for existing products and expansion 39 43 46 .289 13. Products stored separately from insecticides and chemicals 16 25 33 .049 Central Warehouse. The Central Contraceptive Warehouse in Oshodi, Lagos, was also assessed at the time of the LIAT survey. Contraceptives are delivered to state stores upon request. The Central Contraceptive Warehouse staff includes a manager, a deputy manager, two storekeepers, and two security guards. The storekeepers have not been trained on issuing contraceptives. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 19 Procurement planning is conducted in collaboration with UNFPA/Nigeria using the information from annual forecasts. The procurement planning is based, for the most part, on issues data, and it is further adjusted using the warehouse manager and the head of CLMS‘s discretion on past issues. Though medium-term forecasts are updated yearly, long-term (three to five years) procurement plans are not developed. On average, it takes about nine months between ordering and receiving from the central warehouse. After the orders are placed to UNFPA headquarters through the UNFPA/Nigeria office, UNFPA/Nigeria receives notification on shipments. Upon the shipments‘ arrival, documents are sent to agents to handle the clearing of the goods, and contraceptives are brought to the central warehouse. The warehouse is notified after the shipments arrive in-country. In some cases, the warehouse manager is not informed about shipment schedules, which is caused by the lack of coordination among all the parties involved. Occasionally, it results in shipments arriving after the warehouse is near stockout. Availability of most contraceptives in the central warehouse has been good. The central warehouse issues products according to the requested orders, even though requisition and issue forms are often not properly filled out. This finding confirms that most facilities in the system do not order according to their minimum stock level. So far, the distribution of contraceptives has been based on receipt of orders because a distribution schedule for the states has not yet been developed. The central warehouse does not have a transportation system, and it relies mainly on UNFPA or PPFN trucks to deliver contraceptives. Not all records in the warehouse were properly maintained; however, tally cards were updated regularly. An inventory for all the CLMS forms and seed stock kits delivered to the state stores exists but was not available during the visit. Although records of checks for the purchase of contraceptives were available, revenues and expenditures were not being recorded in the cash book. Reporting and analysis of state reports at the central level has remained weak. Reports generated by the warehouse, such as the store activity report, have not been submitted to the Director of DCDPA. Similarly, reporting from states has been low; only five stores had sent their activity report, and those reports were kept at the central warehouse. The manager recognized the lack of information to analyze the information contained in the reports, but the manager plans to submit the reports to senior DCDPA management. The condition of the warehouse was generally good, and contraceptives were properly arranged. In addition, expired products were separated from the inventory, and contraceptives were arranged according to FEFO. Cartons and boxes were stacked appropriately, and, although the current space adequately accommodated existing products, it would not be sufficient for future expansion. Though the warehouse was well ventilated (windows), it has been without electricity for the past year and was not protected against temperature fluctuations. Measures to protect the warehouse against other hazards, such as fire, had been taken but the available fire equipment had not been tested since 1992. The central warehouse has never received a supervision visit; as a result, it has not correctly followed the guidelines stipulated in the handbook. 20 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Conclusions and Recommendations Conclusions • From 2002 to 2005, availability of stock has increased for nearly every method. The 2005 study shows that more than 60 percent of facilities have contraceptives available for most contraceptive methods. This finding is significant because the ultimate goal of a family planning logistics system is to ensure the timely availability of contraception for those who seek it. However, this finding is tempered by the consideration that seed stock was still available in many facilities and could account for the high availability. As seed stock supplies decrease, facilities will need to be resupplied with contraceptives to prevent stockouts. Despite improvement over the 2002 stock availability, approximately one-third of the facilities still did not have the required contraceptives. The principal reason for nonavailability was the failure of higher logistics system levels to distribute seed stocks to lower levels. In certain instances, when staff members were reassigned, they took the seed stock with them to their new facilities. Lower-level staff members also do not properly use the margin generated from cost-recovery funds to cover their cost of transportation to high- level facilities to collect their contraceptives. • When facilities stock out of products, they are stocked out for long periods of time (two to six months). In general, stores stock out for longer periods of time than SDPs. This finding primarily suggests that facilities are not ordering new supplies on a timely basis, although, in certain instances, stores might not be filling new orders. • A large number of facilities, particularly SDPs, are stocked below minimum levels. Because of these low stock levels, the majority of SDPs are at a serious risk for a stockout in the near future. Additionally, about half of those facilities that had placed an order in the previous six months did not order according to their minimum stock levels. Relevant staff members at facilities, especially SDPs, clearly do not understand the ordering and minimum stock guidelines described in the CLMS handbook. They have not been trained to calculate minimum stock levels, they did not understand the training they received, or there was no refresher training or follow-up supervision to reinforce their knowledge and skills. • The upkeep of stock cards has increased from 2002 to 2005. On average, twice as many facilities are updating their stock cards in 2005 compared to 2002. This increase in available data is useful for facilities when determining the amount of stock they need to order to keep their supplies above their minimum stock level. However, accuracy is still a serious problem with only a minority of facilities having registered accurate information on all of their stock cards in 2005. The facilities will need to improve the accuracy of their data to secure appropriate resupply. • Record keeping and reporting are low in facilities. Less than half the facilities fill out most of the required CLMS records, and even fewer send reports to the higher levels. Without accurate consumption data from the SDP level, the FMOH and its partners will be unable to forecast and procure suitable quantities of contraceptives. In addition, without accurate record keeping of revenue generated from the sale of contraceptives, margins are not accounted for or used for supervision, transportation, incentives, and resupply. • There was no change in the percentage of facilities trained in logistics between 2002 and 2005. The 2005 survey shows that slightly more than half the facility staff members were Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 21 trained on the CLMS, essentially the same percentage of staff members trained in key logistics functions in 2002. The percentages are not strictly comparable because the 2005 questionnaire referred to CLMS training and the 2002 questionnaire referred to four key logistics functions. However, the relatively low 2005 percentage of staff members trained in CLMS is disappointing. Reasons include staff members trained in CLMS who were not assigned CLMS responsibilities, reassignment of trained staff members to other facilities or duties, or retirement of trained staff members. Commonly, there is no transfer of knowledge and skills to new staff members who fill positions. Additionally, a criticism of the CLMS training is that it was classroom-based and didactic, and it did not transmit practical information needed for effective inventory and information management that store keepers and service providers required in their everyday work. • CLMS forms and CLMS handbooks are not present in a significant percentage of facilities. Only around three-fifths of stores and one-half of SDPs have the handbook. Without the necessary forms and handbooks, facilities do not have the tools to correctly manage their inventory or to order new supplies. Facilities do not have the handbook and forms for various reasons; they were never issued the material, they were issued the material but misplaced it, or their staff members were transferred and took the material with them. Whatever the reason, most personnel knew about the existence of the material but did not secure it on their own. In addition, supervisors were not making their supervision rounds and, therefore, did not know about or did not rectify this lack of material. • Just over half of all stores in 2005 conducted supervisory visits during the previous six months. There is no significant difference between the percentage of stores conducting supervisory visits in 2002 and 2005. Supervision is an important element of the CLMS, because it facilitates communication and support between levels, as well as provides opportunities for on-the-job training. The principal reason for the lack of supervision is a lack of money. Supervisors do not properly use the cost-recovery fund margin to make the visits, the cost-recovery remittances are not high enough to cover the cost of supervision, or state and local governments do not provide funding for staff members to go on supervision visits. • Availability of product and logistics system performance varied greatly by state. For example, stores and SDPs in Bauchi consistently had a low percentage of facilities with product in stock. Bauchi also has the smallest percentage of facilities with the CLMS Handbook and forms, and one of the lowest order rates. Conversely, Enugu SDPs and Nassarawa stores had high percentages of facilities with stock available. At the same time, those states also had relatively high percentages of SDPs that had the necessary CLMS forms. Many weaknesses with the system will, therefore, have to be addressed state by state. Recommendations The performance of the CLMS relies heavily on the strengths and skills of logistics managers and facility staff members to manage the system, the continuous availability of commodities, and the availability of data at all levels of the system for decision making. Following are proposed recommendations to improve the system based on the empirical results of the assessment. Data from the LMIS are not submitted to the central level. • In the short term, the FMOH should ask the state RH/FP coordinator to be responsible for photocopying LGA and SDP requisition and issue forms (RIFs) to attach to the state RIF, and to submit those forms every four months. This information can be used to track consumption and to make forecasts for future supplies of commodities. The FMOH should also assess the 22 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System additional cost of printing the SDP RIFs in quadruplicate and the LGA RIFs in triplicate to ensure that there is a copy for each level. • In the medium to long term, if feasible, print the SDP RIFs in quadruplicate and the LGA RIFs in triplicate. LMIS forms are not completed correctly or transmitted, reorder schedules are not respected, and storage conditions are not supporting FEFO. These complex problems are related, in some cases, to staff turnover; in other cases, to a failure to receive the necessary materials; and in still others, to generally poor communication between system levels. • Develop alternative training strategies that decentralize training, such as on-the-job training (OJT), which is supported by appropriate materials and training of state-level supervisors and trainers. ° Use OJT to reinforce all aspects of system rollout training, particularly when - completing LMIS forms, - submitting RIFs to next higher level, - maintaining proper storage conditions, - reinforcing FEFO, or - collecting supplies on time. • Advocate for state-level support, as well as donor support, for the development of OJT. • Have the FMOH work with state RH/FP coordinators to prepare supervision and OJT action plans and budgets to submit to the state MOH for annual funding allocation. • Remember that in states where staff turnover is highest, the FMOH may need to conduct a limited amount of primary training for state, LGA, SDP, or all three types of personnel. Handbooks and forms are not distributed or are not available in sufficient quantity. • The FMOH should identify states where the LMIS materials (forms, handbooks) have not been distributed to LGAs and SDPs. • The FMOH should meet with the state RH/FP coordinators and with other relevant officials in the most affected states to ensure proper replication and distribution of LMIS materials. Stock card records do not match physical inventory. • Noting that there were frequent discrepancies between stock cards and physical inventory, the state RH/FP coordinators need to reinforce the physical inventory schedule, as defined in the CLMS Handbook during supervision visits and training activities. It is very difficult to measure system performance indicators (e.g., reporting rates) precisely without knowing exactly how many sites are offering family planning and their geographic distribution. • Create a map of family planning services by requesting that the state RH/FP coordinator submit a list of government SDPs in their respective states and LGAs that are offering family planning. • Develop a tracking sheet for the state RH/FP coordinators so they can monitor LGAs and SDPs that are (and are not) submitting their RIFs. Routine supervision visits are not taking place. This complex problem may be the result of any one or a combination of factors, including competence of the supervisor to perform the Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 23 supervisory tasks, motivation, funding, vehicle or transport availability, and political and material support from the state. In the short term, the FMOH/DCPDA needs to develop messages and schedule immediate advocacy visits to relevant officials of the most poorly performing states and to all the states over the next 12 months. • The FMOH/DCPDA should work through the RH Commodity Security Committee to coordinate support from the FMOH, UNFPA, and the USAID projects for advocacy activities. Contraceptive seed stocks have not been distributed in some states. • The Federal MOH needs to identify all states that have not yet distributed seed stock for immediate intervention. • The Federal MOH should request that those states notify their LGAs immediately to collect seed stocks. • The state MOH should inform the SDPs when seed stock is available for pickup from the LGA, the state stores, or both. Although most state stores have stock for most of the contraceptives, stockouts still occur at the SDPs and LGA stores in all states. • State stores should inform the LGAs and SDPs by circular that contraceptives are available for pickup at the state store. Sites fail to set up a separate account to manage the contraceptive cost-recovery funds. The ability to account for cost-recovery revenue and to subsequently access those funds for contraceptive commodity transportation and to account for supervision activities is strongly associated with establishing a separate account to manage those funds, as specified in the CLMS Handbook. • Work through advocacy, supervision, and OJT activities to encourage LGAs and SDPs to open and to maintain separate accounts for contraceptive cost-recovery funds. Revenue from the cost-recovery system is insufficient to support supervision and commodity transportation in states that have low product turnover. • Identify alternate resources for contraceptive commodity transportation and supervision in states with insufficient cost-recovery revenue. • Identify and work with constituencies interested in RH and FP to promote increased demand and consumption of contraceptives. • Develop and implement communication strategies in each state to promote family planning in a contextually appropriate way, as well as to increase product turnover and revenue to a sustainable level. There is insufficient physical storage space in some facilities. Some LGA stores fail to respect key storage conditions, especially fire safety, proper space between cartons and walls, and adequate lighting. Insufficient storage and handling space impedes the practice of FEFO. • The state MOH should assess storage needs within the state and should identify resources to improve the quality of storage and to increase storage capacity, as required for family planning products. 24 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System • Because there are large quantities of material other than family planning commodities in the stores, including broken furniture and expired products, it would be advisable to dejunk the LGA stores. The political and social environment in some states does not support family planning service delivery; in some states, the majority of SDPs do not conduct family planning activities. Despite supply-side and logistics problems, there is real demand for particular methods–even in the northern part of the country: IUDs in Sokoto and injectables in Zamfara are two examples. If one notes that changing local attitudes and practices toward family planning is beyond the scope of a logistics project, this topic can still be addressed as a critical issue within the broader context of commodity security. • Kano and Bauchi are two USAID focus states, and the USAID-financed COMPASS project is working at the grass roots level in those two states. We recommend that their communications strategists develop a plan to empower local populations to advocate to their local governments for such services and to request from the National Family Planning Program the logistics that support them. • In the long term, after the supply chains are functioning and have ensured supplies to the SDP level, states such as Bauchi and Kano may want to reactivate their system of Community Health Extension Workers (CHEWs) to encourage demand at the community level. However, unless there is a functioning system to resupply the CHEWs, community-based distribution is unlikely to succeed. • One possibility would be to identify at least one currently functioning LGA in either or both of the states and to set up a model program that would include community-based contraceptive distribution. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 25 26 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Appendix A Table A.1. Facility List States LGA SDP State Store LGA Store Alkaleri General Hospital Alkaleri Town Maternity Alkaleri Gar Maternity Gokaru Maternity Yalwan Duguru Maternity Yankari Game Reserve Clinic LGA Store Isawa Maternity Clinic Giade Kurba Maternity Town Maternity Giade BAUCHI Zabi Maternity Giade Kirfi LGA Store Bara Maternity Kirfi Primary Health Center Sharifuri PHC Maternity Tubule Maternity Wanka Maternity Bauchi LGA Store Fed Low Cost Maternity Clinic Kofar Ran Urban Maternity Kofar Wase FP Clinic State House of Assembly Clinic Yalwa Domiciliary Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 27 BAUCHI Tafawa Balewa LGA Store Bununu Maternity General Hospital Tafawa Balewa Gital Maternity Lere Maternity Tafawa Balewa Maternity Zwall Maternity Oredo LGA Store New Benin Health Center Oredo PHC, Sapele Road Urban Health Centre LGA Store PHC Abudu Orhiomwon PHC Evbobehighae, Ugo-Ogan Road PHC Igbekkhue, Health Center Road PHC Urhonigbe, 3 Freedon Street Primary Health Center Adanako Road LGA Store PHC Ake PHC Ekpoma Owan East PHC Ihievbe PHH Uoka EDO Referral Centre Clinic Urban Health Centre Esan West LGA Store PHC Illeh PHC Uhiele PHC Ujogba PHC Ukhun 28 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System EDO Uhunmode LGA Store PHC Ehor PHC Igieduma PHC Oke PHC Orhua PHC Ugha LGA Store Parklane Sp. Hospital Enugu North Railway Ind. Clinic U.N.T.H. LGA Store ENUGU Udeno Obollo-Afor Health Centre Ogbede Health Centre Nkanu West LGA Store Health Centre, Agbani LGA Store Enugu East Abakpa Pry Health Centre State Store Abaji Hospital Yaba Health Clinic Abaji Abaji General Hospital Abaji Maternity Clinic FEDERAL CAPITAL Family Health Clinic, Garki TERRITORY Municipal Area Council Gwarinpa Hospital Karu Dispensary Gowon Barracks Clinic Wuse Hospital Kuje Kuje Health Center Robuchi Hospital Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 29 FEDERAL CAPITAL Bwari Lower Usman Dan Clinic Bwari Health Center RE Bwari General Hospital TERRITORY Mpape Health Center Kwali Kwali Health Center Kwali General Hospital State Store LGA Store Jakra Marmara KANO Kano Municipal Sharada Nuhu Bamalli Yakasi LGA Store Kura Kura General Hospital Unguwar Gabas LGA Store Sir Sunusi Nassarawa Gwagukarwa Abdullahi Wase Tsamiyar Boka Unguwa Uku KANO Gwarzo LGA Store Gwarzo General Hospital Dala LGA Store Kurna Clinic Dala Orthopaedic Waziri Gidado Dala MCH Sheik Jidda 30 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System LAGOS State Store Ijebu Lekki LGA Store Ibeju PHC Orimadu PHC Awoyaya PHC Lekki PHC Ikorodu LGA Store Ikorodu General Hospital Igbobo PHC Ijede PHC Ipakodo PHC Emmanuel PHC LGA Store Harvey Road Health Center Atto PHC Lagos Mainland E/B Health Center Abule Nla PHC Ondo Street West PHC LGA Store Alves PHC Mushin Isolo PHC Palm Avenue PHC Kajola PHC Ayantuga PHC State Store LGA Store Akwanga PHC NASSARAWA Akwanga Andaha PHC Akwanga South Gudi PHC PHC Akwanga College of Education Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 31 NASSARAWA Dona LGA Store Doma Town PHC Bosco Road Doma General Hospital LGA Store Karu Karu Masaka Koroduma Gitata Uke General Hospital LGA Store Keana Giza PHC NASSARAWA Kaderako PHC DH Keana NPHC Keffi LGA Store A/Waje PHC State Store LGA Store Elekaara Health Post General Hospital, Fiditi Afijio General Hospital, Ilora PHC Fiditi PHC Ilora PHC Oke Boda, Awe OYO Oriire LGA Store General Hospital, Ikoyi-lle PHC Lluju PHC Youth Friendly Clinic, Tenure Youth Friendly RH Clinic, Ikoyi Lle Ibarapa East LGA Store FP Clinic Eruwa PHC Eruwa PHC Oke Imale Lanlate 32 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System OYO Ibadan South LGA Store FP Clinic, R/road, Ibadan General Hospital, Lanlate Maternal and Child Clinic, Ibadan NRC Hospital, Ibadan PHC Alashinloye PHC Odo ona Ibadan State FP Clinic, Ibadan LGA Store Ogbomoso South ljeru PHC PHC/Youth Friendly Clinic, Ilogbo State Store LGA Store Assara Dispensary Gwadabawa Gigane Dispensary Gwadabawa Rural Health Center Meli Dispensary Ragandan Dispensary LGA Store Arkilla Clinic SOKOTO Wamakko Bakin Kusu Dispensary Farfaru Basic Health Clinic Kontagora Clinic UTUTH Sokoto North LGA Store Assada Dispensary Helele Clinic Kofor Rimi Clinic Market Clinic Women and Children Welfare Clinic Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 33 SOKOTO Wurno LGA Store Achida Upgraded Dispensary Alkamu Community Dispensary General Hospital, Wurno Marnowa Dispensary Wurno Town Dispensary Shagari LGA Store Aggur Dispensary Kainji Upgraded Dispensary Kanbara Upgraded Dispensary Lambara Upgraded Dispensary PHC Shagari 34 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Appendix B Table B.1. Team Composition Teams Team 1 Team 2 Team 3 Team 4 Team 5 Team 6 Team 7 Team 8 Team 9 State Covered FCT Nassarawa Bauchi Kano Sokoto Lagos Oyo Enugu Edo FMOH Lawrence Anyanwu Esther Ladipo Y. Y. Abdullahi Pauline Aribisala Musa Odiniya Taiwo Avbayeru Ralph Olayele Greg Izuwa Bose Adeniran SMOH Elizabeth Attah Esther N. Yiga Hajia Yaya Tijjani Aishat Lawan Salamatu Suleiman Hanidu Mosun Ojediran Mojoyinola Charity Nnamani Dr. W. I. Imongan NPHCDA Victoria Akinrolabu Olusunde Oluseyi Nafisat Lara Yakubu Yunusa Bala Suleman Manu Teleoia Kajeoro Ester Fadeleu Ukagwu Ngozi U. Aliu COMPASS M&E Officers Greg Osuba Shigudu Koche Rufai Ibrahim Zakari Zakariya Tayo Olugbemi Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 35 36 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Appendix C Quantitative Findings by State To determine the strengths and weaknesses of the system at the state level, assessment teams analyzed the data state-by-state. At this level, national and state family planning officials have a snapshot of the contraceptive logistics management system (CLMS) in each state. The CLMS can then be used to focus attention and design improvements. The assessment teams visited 17 facilities that are not managing contraceptives. Most of the facilities are in Sokoto, where two stores and 11 service delivery points (SDPs) were visited; the facilities are not providing family planning services. The remaining four are in Bauchi with one store, the Federal Capital Territory (FCT) with two SDPs, and Kano with one SDP. Table C.1 lists the number of stores and SDPs in each state that were managing contraceptives at the time of the visit. Table C.1. Stores and SDPs Managing Contraceptives at Time of Visit State Store SDP Bauchi 2 19 Edo 6 14 Enugu 3 6 FCT 1 12 Kano 6 16 Lagos 6 24 Nassarawa 5 16 Oyo 5 22 Sokoto 4 15 Total 38 144 Product availability varies widely state by state. Bauchi was consistently among the lowest percentage of SDPs with product in stock. In fact, it had the lowest percentage of facilities for every product except Lo-Femenal and Noristerat. In contrast, Enugu had among the highest percentages of SDPs with product in stock. Female condoms, male condoms, Depo-Provera, Exluton, and Microgynon appear to be well stocked. However, it is important to note that only six SDPs from Enugu were included in the assessment. Product availability was also assessed at the local government area (LGA) and state store level. Depo-Provera, female and male condoms, and Noristerat were available in the most number of facilities, although no product reached availability in 80 percent or more of stores. Norplant, followed by Lo-Femenal, were available in the fewest number of facilities. Only three stores (only in Oyo‘s LGA stores) had any stock of Norplant. Two LGA stores (in Oyo and Nassarawa) succeeded in ensuring availability of products for nearly all methods and in most of its facilities. The LGA stores in Bauchi and Kano had particular difficulty maintaining stock for all methods in its facilities. State stores were generally better stocked in the different methods and in more facilities. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 37 Nas sar awa Enu gu FCT Oyo Edo Lag os Sok oto Kan o Bau chi Male condoms were most available in all facilities in Enugu (100 percent), Nassarawa (95 percent), and the FCT (85 percent). They were least available in Kano (32 percent), Bauchi (33 percent), and Sokoto (58 percent). Pills (either Microgynon or Lo-Femenal) were most widely stocked in Nassarawa, the FCT, and Enugu (100 percent, 92 percent, and 89 percent, respectively). They were least available in Bauchi (38 percent), Kano (50 percent), and Lagos (63 percent). Injectables (either Depo-Provera or Noristerat) were the most available product in all facilities; they were 100 percent available in four states: Edo, Enugu, the FCT, and Nassarawa. They were least available in Bauchi, Kano, and Sokoto, although more than 50 percent of the facilities had injectables in stock. Offering all the product categories above is important for providing clients with a choice of contraceptive method, which helps to increase demand and use. Figure C.1 shows the states that are stocked with male condoms and pills (either brand) and injectables (either brand). Enugu, the FCT, and Nassarawa provide a mix of methods in more than 80 percent of their facilities. Bauchi and Kano have the lowest percentage of facilities, at about 10 percent and 23 percent, respectively. Figure C.1. Percentage of Facilities with Mix of Methods Available on Day of Visit 0 10 20 30 40 50 60 70 80 90 100 Pe rc en t ( % ) States Most states had a large percentage of facilities trained in CLMS; only Bauchi state did not reach 50 percent. More than 80 percent of facilities in the FCT, Nassarawa, and Sokoto have all the CLMS forms available. However, less than 30 percent of facilities in three states (Bauchi, Kano, and Oyo) have all the forms available. More than 80 percent of facilities in the FCT, Nassarawa, and Sokoto also have the requisition and voucher records for the past six months; no facilities in Kano, and only 5 percent and 10 percent of sites in Bauchi and Edo, respectively, have such records. 38 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Most of the stores are not reporting as expected. For example, none of the stores in five states have sent the store distribution reports to the higher level in the past six months. The FCT is the only area with 100 percent reporting; however, there is only one store there. Only 8 percent of facilities in the FCT, 10 percent in Nassarawa, and 20 percent in Edo reported never having ordered resupply quantities in the past six months, which is among the best of any states. In contrast, 67 percent of facilities in Bauchi and almost 50 percent in Sokoto have never ordered contraceptives in the past six months. See table C.2. Table C.2. Percentage of Facilities Adhering to CLMS Guidelines Percentage of Facilities State Trained in CLMS (n = 182) CLMS Forms Available (n = 179) Never Ordered in Past 6 Months (n = 172) Requisition and Voucher Records for Past 6 Months (n = 170) Send Store Distribution Reports up (n = 39) Bauchi 24 16 67 5 0 Edo 100 70 20 10 0 Enugu 100 67 44 63 0 Oyo 67 26 37 33 0 Sokoto 90 90 47 80 25 FCT 92 85 8 82 100 Kano 82 19 32 0 0 Lagos 53 73 39 25 14 Nassarawa 95 95 10 90 20 More than half the facilities visited in six states have never received supervisory visits in the previous six months. Nassarawa, Oyo, and the FCT appear to be receiving more supervisory visits, with only 5 percent, 23 percent, and 36 percent of facilities, respectively, not receiving visits. Stores were asked whether they had conducted supervisory visits during the previous six months. For the most part, Kano and Edo stores (83 percent and 67 percent, respectively) are not conducting visits, while only 20œ25 percent of stores in Oyo, Nassarawa, and Sokoto have not conducted visits. All facilities were asked if they are regularly overstocked in any methods and whether they regularly stocked out of any methods. Only 5 percent of facilities in Nassarawa and 7 percent in Sokoto were regularly overstocked in any contraceptives. However, 85 percent of facilities in Bauchi reported having too much stock in some methods. Less than one-quarter of facilities in five states regularly stocked out of any method. In contrast, many Bauchi facilities (71 percent) reported regularly stocking out of contraceptives. See table C.3. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 39 Table C.3. Supervision and Inventory by State Percentage of Facilities State Never Received Supervision Visit (n = 176) Never Conducted Supervision Visit (n = 37) Overstocks Regularly (n = 150) Stocks out Regularly (n = 158) Bauchi 52 Missing 85 71 Edo 85 67 44 45 Enugu 50 33 13 13 Oyo 23 20 19 15 Sokoto 50 25 7 21 FCT 36 Missing 42 25 Kano 81 83 33 42 Lagos 63 50 32 50 Nassarawa 5 20 5 25 Wide differences among states are also seen when looking at the cost-recovery system. The FCT and Nassarawa facilities have opened the largest percentage of separate accounts for contraceptives (83 percent and 70 percent, respectively), while less than one-quarter of facilities in six states have opened such accounts. Among facilities, 100 percent in Nassarawa and 83 percent in the FCT have used the margins they set up for the cost-recovery system. The percentage of facilities using the margins in most other states is between 29 percent and 63 percent; however, no facilities in Bauchi have used the margins. Keeping records of the margins usage is minimal in most states. Five states have less than 25 percent of facilities with ledgers for margins usage. Nassarawa and the FCT were among the best record keepers, at 64 percent and 60 percent, respectively. 40 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Table C.4. Availability of Contraceptive Products in Public SDPs by State on Day of Visit (based on physical inventory) State Number of Facilities by State Percent of Facilities with Contraceptive in Stock, among All SDPs Providing Family Planning Services Number of Facilities Managing IUCD Percent of Facilities with IUCD in Stock* Number of Facilities Managing Norplant Percent of Facilities with Norplant in Stock** Female Condom Male Condom Depo- Provera® Exluton Lo- Femenal Micro- gynon Noristerat Bauchi 19 11 26 42 0 37 5 53 10 50 14 36 Edo 14 79 79 100 36 36 57 86 2 50 0 N/A Enugu 6 83 100 100 83 17 83 67 6 83 3 67 Oyo 22 68 77 77 59 64 59 36 19 74 7 71 Sokoto 15 60 60 73 60 47 40 67 3 100 0 N/A FCT 12 33 83 100 75 92 67 100 10 100 1 0 Kano 16 19 38 69 31 63 19 63 7 57 0 N/A Lagos 24 63 75 88 63 63 8 67 15 80 1 0 Nassarawa 16 81 94 94 88 100 75 94 1 0 0 N/A Total SDP 144 73 26 * Among those managing IUCD (n = 73) **Among those managing Norplant (n = 26) Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 41 Table C.5. Number of Facilities with Product in Stock State Type of Site No. of Facilities Number of Facilities with Product in Stock Female Condoms Male Condoms Depo- Provera® Exluton IUDs Lo- Femenal Micro- gynon Noristerat Norplant Bauchi LGA Store 1 0 1 0 0 1 0 0 0 0 State Store 1 0 1 0 1 1 0 1 0 0 Edo LGA Store 5 5 3 5 2 5 0 5 5 0 State Store 1 1 1 1 0 1 0 1 1 0 Enugu LGA Store 3 3 3 3 2 1 0 3 3 0 State Store 0 0 0 0 0 0 0 0 0 0 Oyo LGA Store 5 5 4 5 3 4 1 3 4 3 State Store 0 0 0 0 0 0 0 0 0 0 Sokoto LGA Store 3 0 1 0 1 1 1 0 1 0 State Store 1 1 1 1 1 1 1 1 0 0 FCT LGA Store 0 0 0 0 0 0 0 0 0 0 State Store 1 1 1 1 1 1 1 1 1 0 Kano LGA Store 5 0 0 0 0 0 0 0 0 0 State Store 1 1 1 1 1 1 1 1 1 0 Lagos LGA Store 5 5 5 5 4 5 1 2 5 0 State Store 1 1 1 1 0 1 1 1 1 0 Nassarawa LGA Store 4 3 4 4 4 0 4 4 4 0 State Store 1 1 1 1 1 1 1 1 1 0 Total 38 28 28 30 21 24 12 24 27 3 42 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Appendix D Qualitative Findings by State Bauchi Two stores and 19 service delivery points (SDPs) were included in the assessment of Bauchi. Although training of the local government areas (LGAs) and SDPs has taken place, the state reproductive health/family planning (RH/FP) coordinator has not distributed the seed stock kits that were provided after the training. Those commodities have been stored in poor condition since their arrival in Bauchi. In the five LGAs visited, only one of the stores was managing contraceptives. Product availability in Bauchi was found to be among the lowest of all the states visited. For example, only 26 percent of SDPs in Bauchi had male condoms in stock on the day of the visit, and no SDPs had Exluton in stock. Additionally, only 10 percent of facilities offered a mix of different contraceptive methods, including condoms, oral pills, and injectables. Many Bauchi facilities reported being either overstocked (85 percent) or stocking out (71 percent) of some contraceptive methods. The assessment team also found that a proper handover has not been done with the incoming RH/FP coordinator. Although the former RH/FP coordinator was trained, she did not practice any of the CLMS guidelines (record keeping, storage, inventory management, etc.). The new coordinator is now learning about the system from her participation in this assessment. As a first priority, commodities have been moved to another store. Many of the facilities visited have purchased contraceptives from the Society for Family Health (SFH), from the social marketing program, or from private pharmacies. This arrangement is the result of continuing the former VISION project scheme of providing SDPs with socially marketed contraceptives before implementing CLMS, coupled with the unavailability of the public sector contraceptives (seed stock not distributed to SDPs). Besides those sources, several nongovernmental organizations (NGOs) are present in the state, and they provide supplies of contraceptives to the public SDPs. However, distribution of such supplies is very irregular. Most facilities stressed the importance of providing a continuous supply of contraceptives and training about the system. In most cases, when the provider trained on CLMS is absent because he or she is being transferred to another facility or is leaving the system, the knowledge on CLMS and the skills of filling out the CLMS forms are not transferred to the replacement. The lack of knowledge transfer was also evident during the survey interviews, because the colleague working with the person managing the commodities often could not answer questions about the CLMS. Most of the facilities do not have records, and the assessment team found it difficult to report consumption, stock status, and general stock management. In general, facilities in Bauchi have not been implementing the system. Only 24 percent of facilities have received training; out of the 19 SDPs, three of the trained providers have been transferred. Additionally, a majority of facilities in Bauchi (67 percent) have not ordered more contraceptives in the past six months. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 43 Edo Six stores and 14 SDPs were visited during the survey, all of which are managing contraceptives. Compared to other states, facilities in Edo state had an average amount of availability for female and male condoms, Depo-Provera, Microgynon, and Noristerat. Exluton and Lo-Femenal, however, were not as readily available. Approximately 60 percent of facilities in Edo had a mix of methods available on the day of the visit. Almost all facilities appreciated the continuous availability of commodities; some have reported a slight increase in client flow resulting from the affordable prices. Despite finding 100 percent of surveyed facilities trained in CLMS, 20 percent had not ordered supplies in the past six months, although most were below the minimum stock level quantity for two to four of their products. Additionally, only 10 percent of facilities had the requisition and issue forms (RIFs) on file for the past six months. The state store has not placed orders since the implementation of the system, and RIFs for the state were not accessible to the store until January 2005. For this reason, the state store has been using cost-recovery records instead of the RIF. The store has not been reporting to the central level, and none of the facilities issued by the state store have reported in the past six months. Although total sales match the funds available at the store, there was no ledger or cash book for tracking revenues and expenditures. There was very little supervision from the central level in 2004; this lack has resulted in the ineffective implementation of the system. Among facilities, 85 percent reported that they never received supervision visits, and 67 percent of stores reported that they have never conducted supervision visits. However, the state FP coordinator recognizes that there is a need for adequate supervision from the central level to the state and, likewise, from the state store to lower-level facilities. The FP coordinator reported that LGA RH/FP supervisors have a poor understanding of the system. The newly recruited storekeepers and providers in the state need retraining. For example, providers in two facilities were transferred and, when they left, took the seed stock and the CLMS forms. There is no system of following up or any way to trace the providers who have been transferred; the assessment team found it impossible to locate the previous two providers. Management tools for the different levels in implementing the system have not been appropriately distributed. Store activity reports or cost-recovery records were not found in the LGA stores. No RIFs for the SDPs were found except in the two LGAs trained by FMOH logistics officers. In brief, LGA RH/FP supervisors have not been implementing the system, and supervision in the state has been very weak at all levels in Edo. Cost recovery was being practiced according to the price structure proposed in the system in all of the facilities visited except in the one facility where injectables were being sold at almost six times the actual price. Enugu With only three stores and six SDPs visited, Enugu has the smallest sample of facilities in the survey. Availability was found to be very high in those few facilities; however, most of the facilities had stock in most contraceptives (Lo-Femenal was the only exception at 36 percent). Enugu also had the second highest percentage of facilities, at close to 90 percent, with a mix of methods available. With so few facilities, however, it is difficult to know if there was some sampling bias with the results. 44 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System CLMS forms were available in 67 percent of the facilities; the assessment team observed that the forms were not always properly distributed. For example, in one of the LGAs visited, where the SDP is located in the same vicinity as the LGA store, there were no CLMS forms, and the SDP was using the old form (daily activity report). Another facility was using photocopied forms. The state store is among the 67 percent that had the forms available, but the store had not ordered because of the manner of implementing the system. That unclear procedure makes it difficult for a new person to easily acquire the level of knowledge needed to calculate the minimum stock level. For the most part, records were updated, and the forms were completed accurately. In total, nine LGA stores and five SDPs were supposed to report to the state stores, but only two SDPs had ordered and submitted their RIFs. One of the facilities with a high client flow had problems accessing the CLMS funds because the hospital management is tightly controlling the sales of commodities. As a result, providers are not filling out the cost-recovery records, and they do not handle financial transactions. Additionally, it takes up to one week for the release of money; this lag may delay the reordering of commodities. In general, accounting procedures prescribed in the CLMS handbook are not being followed. Federal Capital Territory Twelve SDPs and only one store that were managing contraceptives from the FCT were included in the assessment. The territory facilities implemented the system better than most of the other states. Product availability was generally high (above 75 percent for most products)–the state store had contraceptives in stock for every product except Norplant. Most FCT facilities (85 percent) also provided a mix of methods to its clients. Records in the store were available to be verified, but records of monthly totals on the tally cards did not match the figures on the cost-recovery records. The total issued for the past six months could not be ascertained; information on the tally cards covered only the period starting from December 2004. More than 85 percent of facilities reported being trained in CLMS and having the CLMS forms. In fact, more than 80 percent of facilities also have the RIFs for the past six months. Only 8 percent of facilities had never ordered in the past six months. Although supervisory visits are occurring more often in the FCT than many other states, 37 percent of facilities are still not receiving such visits. The council area FP supervisors (equivalent to LGA stores), where they exist (Kuje and Kwali), are not in liaison with the SDPs, and the SDPs are collecting commodities directly from the state store. However, some of the SDPs have very low consumption, which generates inadequate margins to cover transportation costs to the state store. Kano Six stores and 16 SDPs were visited during the assessment, but the assessment team found that the system has not been implemented in the state, despite reporting that 82 percent of facilities have been trained. Product availability was generally low, ranging from 20 to 70 percent for the different products; just over 20 percent of facilities had a mix of methods available. The state store, the LGA stores, and the SDPs have not been following the procedures of the redesign system, including ordering and issuing, record keeping, cost-recovery management, and supervision. The state FP coordinator reported that the state store did not receive CLMS forms except for tally cards, and only 19 percent of facilities overall reported having the CLMS forms. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 45 The assessment team observed that there were no RIFs for any of the levels. As a result, the few SDPs that were ordering were making their requisition on a piece of paper. The state store has not ordered since the beginning of CLMS, and, therefore, has never calculated the minimum stock level. The store reported being fully stocked as a reason for not ordering, but commodities have not been distributed to the lower levels. This reality is supported by the survey findings, which shows Kano facilities as having among the lowest percentage of product availability among the nine states. No records were kept in the store, and none of the facilities from the lower level were reporting to the state store. More than 80 percent of facilities have not received supervision visits in the past six months, and 67 percent of stores reported not having conducted any in the same time period. The state FP coordinator has reported that she conducted supervisory visits to SDPs but did not use the supervision checklist. The assessment team observed that about 75 percent of the funds generated from the cost recovery were kept in the store. The facility has not opened an account for contraceptives because of the fear that the money may be used by the State Ministry for other purposes. Part of the fund was spent on SDP training on CLMS in January 2005. No cash book or ledger showed the use of margin, according to the specified terms in the handbook. In general, there was a poor accountability and management of the cost-recovery funds in the store. Storage in the store was mediocre. Possibly because of its location within the Sheik Muhammed Jidda General Hospital, commodities were cluttered in among hospital equipment, beds, mattresses, and so forth. The LGA level is not active in Kano; no commodities were kept at this level. Knowledge of the redesigned CLMS is generally very low at all levels in the state. Providers in charge, mostly community health extension workers (CHEWs), have not been trained in CLMS. In addition, some SDPs did not receive the seed stock to start implementing the system, and they have used their money to buy as many products as they can afford. Rotary International was supplying a few SDPs. Lagos Six stores and 24 SDPs were assessed in Lagos state. The LGA stores were well stocked in all products except for Lo-Femenal, Microgynon, and Norplant. The state store had everything in stock except for Exluton. Between 63 and 88 percent of SDPs were stocked in most products. Sixty percent of facilities stocked at least one injectable method, one contraceptive pill method, and male condoms. However, only 8 percent of SDPs had Microgynon in stock. Although the CLMS system became functional in August 2004, the creation of additional LGAs in the state and the subsequent random transfer of staff members have made the implementation of the system challenging. Additionally, while the state has made efforts to fully implement the system, distribution of CLMS forms was not done concurrently with the distribution of seed stock kits. At the time of the assessment, 73 percent of facilities had CLMS forms available, but only 53 percent had been trained in CLMS. The state store has been ordering quarterly; however, funds generated from the cost recovery have not been adequate for the resupply of some products, according to the minimum stock level quantities. The store has supervised 3 of the 20 LGAs in the state but has yet to send the reports to the higher level. There has also been no reporting to the state store from the lower-level facilities. 46 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System The assessment team observed that the store was kept clean and that boxes were properly organized. However, cartons were not stored on shelves or pallets. The team found that the space for the commodities was too small. Most sites visited were not fully operating according to the system as a result of several factors. As mentioned earlier, the main reasons for the lack of implementation of CLMS was the reassignment of providers and the creation of additional facilities in the newly formed LGAs without employing nurses or midwives. Furthermore, under the redesigned system, the decision to wait until providers exhausted their original stock to start them with the seed stock has slowed the start of activities. Nassarawa At the time of the survey, Nassarawa was one of the states that had most successfully implemented the system. Five stores and 16 SDPs were visited, and well over 75 percent of facilities had most products available. However, it was discovered that the seed stocks were not distributed to the LGA stores until February 2005, so this delay could possibly explain the high availability. More than 10 percent of facilities reported never ordering more supplies in the previous six months. Although the FP coordinator reported ordering every four months, an order was not made in the 12 months since the seed stock kits were supplied. The store reported being fully stocked as a reason for not ordering but, as previously mentioned, seed stocks to the LGA stores were distributed only in February 2005. In addition, distribution was still ongoing during this visit, and some LGA stores were still expected to collect their seed stock kits. The state store has been filling out the reports for the past three-quarters, and 90 percent of facilities had the RIFs for the previous six months. The FP coordinator reported that only a few providers were trained and that not all the LGAs in the state were covered. However, 95 percent of facilities visited during this assessment reported being trained in CLMS. Almost 100 percent of facilities had the CLMS forms available. Despite the high percentage of facilities trained in CLMS, however, many providers had only little knowledge of calculating the minimum stock level. Calculations on the cost-recovery records were not matching the sales figures from the daily consumption records. In addition, many found filling out the forms and the number of forms a challenge. Most providers mentioned the affordability of the contraceptives and viewed the margin as a motivation for providers. They also acknowledged that awareness about family planning in their communities is low and that there is need for support to increase consumption. All the SDPs visited emphasized that availability of contraceptives is in itself a motivation, and they suggested that regular availability of commodities be maintained to ensure continuity of work. The assessment team reported that the distribution of the same number of products to all SDPs in the seed stock kits has left some SDPs overstocked with some slow-moving products. In one particular LGA, SDPs within the area have transferred products to ensure the use of those products before expiration. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 47 Oyo The five stores and 22 SDPs visited in the assessment had average availability of products; more than 60 percent had a mix of at least one injectable, one contraceptive pill, and male condoms. However, Oyo performed relatively poorly in other aspects of the CLMS. Though the state store had tally cards and RIFs, it did not have the remaining CLMS forms. Consequently, cost-recovery records and consumption monitoring forms were not readily available at any of the levels. In fact, only 26 percent of facilities had CLMS forms available. In general, record keeping in all of the facilities visited was poor because of the low knowledge level and the absence of the necessary management tools. For example, the state store had not been filling out and sending its activity report for the previous six months. During the same period, less than half of the 60 facilities from the lower level have reported to the store. Although the store reported that supervision visits were conducted, there were no reports on file. The store also had major discrepancies between the records on the tally cards and the physical inventory for Depo-Provera and Lo-Femenal. Even though the state store‘s performance has not been superior, the state FP coordinator highlighted three major benefits of the system: it motivates providers by providing incentives as a reward for their hard work; it allows easy organization of work in terms of the specified frequency for reporting and ordering; and, through meetings, it strengthens the relationship between the state and LGAs coordinators. LGA supervisors and service providers trained on CLMS have reported knowledge in calculating the minimum stock level, and ordering and calculating their margin. They have also expressed that their challenge was the absence of the relevant CLMS forms. However, the assessment team observed that both LGA RH/FP coordinators and service providers had difficulty determining the resupply and margin value. In most of the SDPs, the total sales fund was used to purchase commodities without removing the margin and the incentive. It was not possible to assess financial management in almost all facilities because facilities did not have ledgers or cash books to show sales and expenditures. Sokoto Many facilities that were visited during the assessment could not be included because they were not providing family planning services. Four stores and 15 SDPs were assessed, but two stores and 11 SDPs could not be included because of the lack of family planning services. Not surprisingly, the facilities in Sokoto that were providing those services have had some difficulties implementing the system. The state store has not ordered since the implementation of the system, and it is storing the LGA seed stock kits. The state store issues to SDPs at the prices specified for SDPs. The family planning coordinator has stated that there are advocacy issues with the LGA policymakers that need to be resolved before commodities can be issued to LGA stores. SDPs are supplied from the LGA seed stocks in the state store. Most LGA stores in the state are inactive, and some have no space to keep commodities. In some LGAs with a family planning manager, the clinic will function as a stand-alone clinic and does not supply commodities to other SDPs in the local government. The state store has all the CLMS forms and has distributed to some of the SDPs visited by the team. The family planning coordinator reported that there were not an adequate number of handbooks, CLMS forms, and seed stock kits during the training, although 90 percent of facilities reported receiving training. However, the selection of service providers to be trained on CLMS was done randomly; a good number of attendees at the training were not providing family 48 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System planning services. Additionally, some facilities expressed concern that some staff members trained on CLMS were being randomly replaced or transferred by the local government. In most of the facilities, there was a high demand for Noristerat and Depo-Provera, and yet service providers were not aware that they can reorder from the state store. Providers do not understand how to fill out the forms, and they have to place orders for any of their products that are below their minimum stock level. Most of the providers did not understand how to calculate the minimum stock level. In addition, some of the clinics were being supplied by Ipas, and providers were confused about the source of the commodities and the reporting mechanisms. Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System │ 49 50 │ Nigeria: Midterm Evaluation of the Contraceptive Logistics Management System Appendix E Logistics Indicators Assessment Tool (LIAT) Logistics Indicators Assessment Tool (LIAT) │ 51 Interviewer‘s Guide Facility Identification Record the name of the facility and location. Using the codes provided for each question, place all other responses in the boxes on the right. Information about Interview Record the date that the interview took place, and list the names of the interviewers. Introduction Use the text here to guide your introduction of the survey to facility staff members. Questions 01 to 06 Receive permission to conduct the interview and to record information regarding the interviewee. Use of Comments Column Use the Comments Column as needed to clarify responses. This usage applies especially to questions in which the interviewer is instructed to verify a response by the interviewee. As guidance for comments regarding verification of forms use the following: a) Facility didn‘t have forms. b) Forms were complete and accurate. c) Forms were complete but inaccurate. d) Forms were incomplete. (This usage applies to questions numbered 105, 107, 201, 202, 203, 204, 205, 206, 303, 305. 307, 309, 402, and 507). Questions 101 to 118 Record responses by clearly circling either the number or letter that corresponds to the interviewee‘s response. Questions with letters may have multiple responses; questions with numbers have only a single response. Table E.1: Storage Conditions Record observations on the main storage area (even if it is a cabinet) by responding to storage conditions 1 to 12 for every facility visited. For large storage areas that require stacking of multiple boxes, continue to complete storage conditions 13 to 17. Table E.2: Stock Status Above the table, record the maximum months of stock, minimum months of stock, and order interval. If the interviewee does not know those facts, mark DK (don‘t know) as the response. To fill in the cells, follow the instructions above the table. Table E.3: Data Quality Complete the table for all or for a selection of products. Table E.4: Forecast Accuracy Complete the table for all or for a selection of products. End Interview Ask the interviewee or interviewees if they want to ask you any questions. Thank them for their time and cooperation. 52 │ Logistics Indicators Assessment Tool (LIAT) ____________________________________________ Facility Services and Infrastructure Facility Identification Name of the Facility _____________________________ Facility Location State _________________________________________ State . LGA _________________________________________ City/Town _____________________________________ LGA . Contact Information: Address ______________________________________ _____________________________________________ Phone ________________________________________ e-mail ________________________________________ Code of the Facility . Facility Type: (1 = Store; 2 = SDP) Facility Code. If SDP, mark type of facility: (1 = Tertiary Hospital; 2 = Secondary Hospital; 3 = PHC; 4 = Other ____________ . Electricity . . Store Type. SDP Type . If Store, mark level: (1 = Central; 2 = State; 3 = LGA) Is there electricity in this facility? (0 = no; 1 = yes) . Is there a water supply at this facility? (0 = no; 1 = yes) Water……………………………………. ……… Phone……………………………………. Is there a phone in this facility? (0 = no; 1 = yes) ……………. Information about Interview DAY/ MONTH/ YEAR Date: _______________________________________ Interviewer: __________________________________ Additional Comments Regarding This Clinic: Logistics Indicators Assessment Tool (LIAT) │ 53 Introduction Introduce all team members, and ask facility representatives to introduce themselves. Explain the objectives of this survey: Good day. My name is ________________. My colleagues and I are representing the federal, the state Ministry of Health, and the National Primary Health Care Development Agency. We are conducting a survey regarding the Contraceptive Logistics Management System. We are looking at the availability of contraceptives, how you order and receive those products, what your level of understanding is about the CLMS forms, and what the status is of the cost-recovery system. We are visiting selected health facilities throughout the country; this facility was randomly selected to be in the survey. The primary objectives of the survey are to collect current information on logistics system performance and stock status of contraceptives. The results of this national survey will provide important information to make decisions about the current system and to make changes to promote improvements where needed. The survey is being conducted to measure changes in the logistics system since the redesign of the system. This meeting is an assessment of the CLMS and not a staff performance review. If we may, we would like to ask you a few questions, to count the contraceptives you have in stock today, and to observe the general storage conditions. Do you have any questions? No. Question Code Classification Go To 01. Can we continue? Yes………………………………….…1 No………………………………….….0 ÍSTOP 02. Name and title of person interviewed for this section? ___________________________ ___________________________ ___________________________ 03. Number of years and months you have worked at this facility? Years: ______ Months: ________ 04. Have you been trained on CLMS? Yes………………………………….…1 No…………………………………….0 05. If yes, where? During the national CLMS training……………………………….1 On-the-job training ……………….…2 On-the-job (self-learning) ….……….3 Other (specify)……………….………8 06. Who is the principal person responsible for managing contraceptives at this facility? Family planning/reproductive health coordinator………………………….…1 Nurse/Midwife…………….…………2 CHEW………………………………….3 Medical Officer………………….….4 Pharmacy Technician ………….….5 Store Manager……………….….….6 Other (specify) _________________8 54 │ Logistics Indicators Assessment Tool (LIAT) First, ask the following questions of the in-charge or acting facility manager. After asking all of the questions, visit the warehouse, storeroom, or storage area where the contraceptives listed are managed. If you are referred to another staff member for the stocktaking exercise, introduce the survey goals and objectives as you did during the introduction. Ask the interviewee to bring all of the records for contraceptives and the CLMS handbook. I. Ordering and Issuing No. Questions Code Classification Go To, and Comments 101. 102. Does this facility manage contraceptives? Do you expect to issue contraceptives? Yes………………….……….1 No.……………………………0 Yes………………….……….1 No.……………………………0 ÍGo To 104 ÍSTOP 103. When do you expect to issue contraceptives? _______________________ ÍSTOP 104. 105. 106. 107. Do you have a copy of the CLMS handbook? Do you have all the CLMS forms you need to manage contraceptives? Where do you get the requisition and issue forms you need for ordering? How often do you order from the higher level? (Circle whichever applies.) Yes………………….……….1 No.……………………………0 Ask to see CLMS handbook, and mark here if verified. Yes………………….……….1 No.……………………………0 Ask to see CLMS forms, and mark here if verified. _______ From the central store…….…1 From the state store……….…2 From the LGA store……….3 Other (specify) ________________________8 Never……………………….….0 Every 2 months…………….1 Every 3 months…………….….3 Every 4 months…………….….4 Annually……………………….5 Other(specify) ____________8 Ask to see RIFs for the past 12 months, and mark here if verified. __________ ____________ Í112 108. Do you order according to your minimum stock level? Yes……………….……….……1 No….…….…….0 Logistics Indicators Assessment Tool (LIAT) │ 55 109. Who calculates the minimum stock level? The State Store……………….1 The LGA Store………….2 The Service Delivery Point….3 Don‘t know………………….….9 Í112 110. Were you trained to calculate the minimum stock level? Yes ………………….…….…1 No …………………………….0 Í112 111. How did you learn to calculate your minimum stock level? Never learned………………….0 During the national CLMS training………………………….1 On-the-job training ……………2 On-the-job (self-learning) ………………………………….3 Other (specify)…………………8 112. Have you ordered from the next higher level in the past 6 months? Yes………………………….…1 No…………………………….2 Í114 113 If you have not ordered from the higher level in the past 6 months, what are the reasons? Fully stocked……………….….1 Transportation problems……………………….2 Other……………………………8 Í115 114. How many emergency orders have you placed in the past 6 months? None………………………….0 1……………………………….1 2……………………………….2 3……………………………….3 More than 3………………….4 115. How are contraceptives transported to your facility? (Circle all that apply.) Higher level delivers………….1 This facility collects………….2 Other (specify)____________8 116. What type of transportation is most often used? Facility vehicle ……………….1 Public transportation ……….2 Private vehicle ……………….3 Motorcycle …………………….4 Bicycle ……………………….5 On foot ……………………….6 Other (specify) ____________8 117. On average, approximately how long does it take between ordering and receiving products? Upon presentation of the RIF…………………………….1 Less than 2 weeks…………….2 2 weeks to 1 month………….3 Between 1 and 2 months…….4 More than 2 months ………….5 118. Have you developed a distribution schedule for the facilities you issue to? (Stores only) Yes …………………………….1 No ………………………………0 N/A…………………………….9 56 │ Logistics Indicators Assessment Tool (LIAT) II. Record Keeping 201. Do you have the daily consumption record for the past 6 months? (SDPs only) Yes …………………………….….….1 No….……………………………….…0 Ask to see the daily consumption records for the past 6 months, and mark here if verified. __________ 202. Do you have the tally card for each contraceptive for the past 6 months? (Stores only) Yes ………………………………….…1 No…………………………………….0 Ask to see the tally cards for the past 6 months, and mark here if verified. __________ 203. Do you have the requisition and issue vouchers for the past 6 months? Yes ……………………………….……1 No…………………………………….0 Ask to see the requisition and issue vouchers for the past 6 months, and mark here if verified. _________ 204. Do you have the requisition and issue vouchers for the facilities you have issued to during the past 6 months? (Stores only) Yes …………………………….…….1 No ………………………….…….……0 Ask to see the requisition and issue vouchers for the past 6 months, and mark here if verified. __________ 205. Do you have the cost-recovery record for the past 6 months? Yes……….……………………….….1 No ………………………………….0 Check if the monthly total from the DCR or the tally card matches the entry on the cost-recovery record, and mark here if verified. ______ 206. Do you have the store activity report for the past 6 months? (Stores only) Yes……….………………………….1 No ………………………………….0 Ask to see the store activity reports for the past 6 months, and mark here if verified. ______________ III. Reporting Ask questions 301œ309 if assessing a store. If assessing an SDP, skip to question 401. 301. Do you send the store activity reports to the higher level? Yes……….………………………….1 No ………………………………….0 Í303 Logistics Indicators Assessment Tool (LIAT) │ 57 302. Why not? Do not have forms………………….1 Do not know how to fill out the form…………………………………….2 No funds for posting………………….3 Other………………………………….8 303. About how often do you send the store activity reports to the higher level? Every 3 months……………….………1 Every 4 months…………………….…2 Every 6 months…………………….3 Other (specify)_________________8 Ask to see store activity reports for the past 12 months, and mark here if verified. __________ 304. Do you send the store distribution report to the higher level? Yes……….……………………….….1 No ……………………………….….0 305. How often do you send the store distribution reports to the higher level? (Circle all that apply.) Never……………………………….…0 Every 3 months………………….….1 Every 4 months………………….….2 Every 6 months………………….….3 Other……………………………….…8 Ask to see distribution reports for the lasts 12 months, and mark here if verified. _______ 306. Do you send the supervision reports to the higher level? Yes……….……………………….….1 No ………………………………….0 307. How often do you send the supervision reports to the higher level? Never……………………………….…0 Every 4 months………………….….1 Every 6 months………………….….2 Every year……………………….….3 Other……………………………….…8 Ask to see supervision reports for the last 12 months, and mark here if verified. ____________ 308. How many facilities are supposed to send CMS reports to this facility? ___________________ 309. How many facilities submitted all required CLMS reports for the past 6 months? ___________________ Ask to see the reports, and mark here if verified. ___________ 310. Were you trained to complete the store activity report? Yes……….……………………….….1 No ……………………………….….0 58 │ Logistics Indicators Assessment Tool (LIAT) 311. Were you trained to complete the store distribution report? Yes……….……………………….….1 No ……………………………….….0 312. Were you trained to complete the supervision report? Yes……….……………….………….1 No …………………………….…….0 IV. Management of Cost-Recovery Funds 401. Does the ledger balance match the total income from commodity sales? Yes……………………………….……1 No ……………………………….….….0 402. Have you opened a separate account for contraceptives? Yes……………………………….……1 No ………………………………….….0 Check the cash or ledger to verify that total sales matches incomes and expenditures for the past 6 months, and mark here if verified. __________ 403. Does approval to withdraw from the account for the re-supply of contraceptives pose a problem? Yes………………………………….…1 No ………………………………….….0 Í405 404. In what way has this been a problem? ______________________________ 405. Have any CLMS funds been used for other programs Yes…………………………….….……1 No ………………………………….….0 406. Do you know the use of the margins as described in the CLMS handbook? Yes………………………………….…1 No ………………………………….….0 407. Has this facility been able to use the margin or margins as described in the CLMS handbook? Yes……………………………….….…1 No……………………………………….0 Í409 408. Give reasons this facility has not used the margins as described in the CLMS handbook? ____________________________ 409. Does the ledger or cash book show the use of the margin according to the specified terms and items in the CLMS handbook? Yes……………………….……………1 No ………………………………………0 Logistics Indicators Assessment Tool (LIAT) │ 59 V. Monitoring and Supervision 501. When did you receive your most recent supervision visit? Never received………………………….0 Within the last 4 months ……….…….1 Within the last 6 months ……….…….2 More than 6 months ago ………….…3 Other (specify) __________________8 Í503 502. During your last supervision visit, which of the following were checked: a. Stock cards b. All CLMS reports c. Removal of expired stock Yes………………………………….….1 No …………………………………….0 Yes………………………………….….1 No ………………………………….….0 Yes………………………………….….1 No ………………………………….….0 Not applicable…………………….……8 Stores Only (not for SDPs) 503. Have you conducted supervisory visits? Yes…………………………………….1 No …………………………………….0 Í505 504. Why have you not conducted such visits? ________________________________ _____________________________ ÍSection Vl (601) 505. When did you conduct the most recent visit? Within the past month ……………….…1 Within the past 3 months ………….…2 Within the past 6 months ……….……3 More than 6 months ago……………….4 Other (specify) __________________ 8 506 How many supervision visits have you conducted since the implementation of CLMS? None…………………………….……….0 One………………………………………1 Two………………………………………2 Three…………………………………….3 4 or more………………………………. 4 ÍSection VI (601) 507. Do you have all your supervision visit reports on file? Yes………………………………….….1 No ………………………………….….0 Ask to see the supervision reports, tick here if verified __________ 60 │ Logistics Indicators Assessment Tool (LIAT) VI. Additional questions 601. Are there some contraceptives that you usually stock out of before resupply? Yes………………………………….….….1 No ………………………………….…….0 Í604 602. List the commodities that you stock out of most frequently (up to 3 products). 1. _____________________________ 2. _____________________________ 3. _____________________________ 603. What do you consider to be the main reason you stock out of these commodities? Higher level didn‘t send products……….1 We did not go pick up the products…….2 We did not request the correct amount…3 Transportation is unavailable…………….4 Unexpectedly high demand…………….5 Other (specify)____________________8 604. Do you usually have an overstock of some contraceptives before resupply? Yes……………………………………….1 No ……………………………………….0 ÍGo to tables 605. List the commodities you usually overstock of most frequently (up to 3 products). 1. _____________________________ 2. _____________________________ 3. _____________________________ Thank you for you time and information. You have been very helpful. Our remaining questions will require looking at products in the storeroom and speaking with the person who oversees the store. Logistics Indicators Assessment Tool (LIAT) │ 61 Table E.1. Storage Conditions Items 1œ12 should be assessed for all facilities for products that are ready to be issued or distributed to clients. Place a checkmark in the appropriate column according to your visual inspection of the storage facility; note any relevant observations in the comments column. To qualify as —yes,“ all products and cartons must meet the criteria for each item. No Description No Yes Comments 01. Products are arranged so that identification labels and expiry dates or manufacturing dates or both are visible. 02. Products are stored and organized in a manner accessible for first-to-expire, first-out (FEFO) counting and general management. 03. Cartons and products are in good condition, not crushed because of mishandling. If cartons are open, determine if products are wet or cracked as a result of heat or radiation (fluorescent lights in the case of condoms; cartons right-side up for all products). 04. The facility makes it a practice to separate damaged or expired products from usable products, and it removes them from the inventory. 05. Products are protected from direct sunlight at all times of the day and during all seasons. 06. Cartons and products are protected from water and humidity during all seasons. 07. Storage area is visually free from harmful insects and rodents. (Check the storage area for traces of rodents [droppings] or insects].) 08. Storage area is secured with a lock and key, but it is accessible during normal working hours. 09. If storekeeper is absent, does another staff person have access to the key? 10. Products are stored at the appropriate temperature. 11. Roof is maintained in good condition to avoid sunlight and water penetration. 12. Storeroom is maintained in good condition (clean, all trash removed, sturdy shelves, organized boxes). 13. The current space and organization is sufficient for existing products and reasonable expansion (i.e., receipt of expected product deliveries for foreseeable future). 62 │ Logistics Indicators Assessment Tool (LIAT) The additional standards below can be applied to any facility large enough to require stacking of multiple boxes. No. Description No Yes Comments 14. Products are stacked at least 10 cm off the floor. 15. Products are stacked at least 30 cm away from the walls and other stacks. 16. Products are stacked no more than 2.5 meters high. 17. Fire safety equipment is available and accessible (any item identified as being used to promote fire safety should be considered). 18. Products are stored separately from insecticides and chemicals. Additional guidelines for specific questions: Item 2: In noting proper product arrangement, consider the shelf life of the different products. Item 3: Check cartons to determine if they are smashed because of mishandling. Also, examine the conditions of the products inside opened or damaged cartons to see if they are wet, cracked open because of heat or radiation (e.g., for condoms, because of fluorescent lights), or crushed. Item 4: Conduct the discarding of damaged or expired products according to the facility‘s procedures (this step may differ from one facility to another). Specify whether procedures exist, and note what they are. Item 7: It is important to check the storage area for traces of rodents (droppings) or insects harmful to the products. Item 8: This sentence refers to either a warehouse secured with a lock or to a cabinet in a clinic with a key. Item 16: Fire safety equipment does not have to meet international standards. Consider any item identified as being used to promote fire safety (e.g., water bucket, sand). Do not consider empty or expired fire extinguishers as valid fire safety equipment. Logistics Indicators Assessment Tool (LIAT) │ 63 Background for Table E.2. Stock Status (August 15, 2004–February 15, 2005, and the day of visit) Column: 1. Name of all authorized products that will be counted 2. Unit of count for the product Note: Columns 1 and 2 should be filled out before questionnaires are printed for the survey. 3. Whether or not the product is managed at this facility, answer Y for yes or N for no. Note that at certain levels for some products, all facilities should manage the product. In such cases, this column should be marked Y. 4. Check whether the stock card is available; answer Y for yes or N for no. 5. Check whether the stock card had been updated within the past 30 days; answer Y for yes or N for no. Note: If the stock card was last updated with the balance of 0 and the facility has not received any resupply, consider the stock card up-to-date. 6. Record the balance on the stock card. 7. Record if the facility has had any stockout of the product during the most recent six full months before the survey; answer Y for yes or N for no. 8. Record how many times the product stocked out during the most recent full six months before the survey according to stock cards, if available, or to a key informant if not. Note source information. 9. Record the total number of days the product was stocked out during the most recent full six months before the survey. 10. Record the quantity of product dispensed to users or issued from the storeroom during the most recent six months before the survey. Note: If the answer to column 4 is N, record NA in this column. 11. Record the quantity of product in the storeroom. 12. Record whether the facility is experiencing a stockout of the product on the day of the visit, according to the physical inventory; answer Y for yes or N for no. 14. Record the quantity of expired products. Count all expired products on the day of the visit. If some products are near their expiration date (within one week), note in the comments section. Minimum months of stock: _________________ Order interval: ___________________________ Note: For any product that experienced a stockout in the past six months (including the day of the visit), please note reasons (by product). 64 │ Logistics Indicators Assessment Tool (LIAT) Table E.2. Stock Status Note: For any product that experienced a stockout in the past 6 months (including the day of visit), please note reasons (by product). Product Units of Count Managed by the Facility? (Y/N) Stock Card/DCR Available? (Y/N) Stock Card/DCR Updated? (Y/N) Balance on Stock Card/DCR Stockout most recent 6 Months (Y/N) Number of Times you Stockout Total Number of Days Total issued (most recent 6 months) Physical inventory Stockout Today? (Y/N) Quantity of Expired or Unusable 1 2 3 4 5 6 7 8 9 10 11 12 13 Condom (female) Piece Condom (male) Piece Depo Provera Vial Exluton/ Ovrette Cycle IUCD Piece Lo-femenal Cycle Microgynon Cycle Neo- sampoon 20/tube Noristerat Amp Norplant Set Postinor Set of 2 tabs Gloves Pair Syringe Unit Additional Comments Regarding Stock Status: Logistics Indicators Assessment Tool (LIAT) │ 65 Background for Table E.3. Comparison of Quantity Ordered and Quantity Received Column: 1. List the same products as in table E.1, or use a sample of those products. (Note: Do this step before finalizing the questionnaire and making photocopies.) 2. Enter the quantity ordered for the last order period for which products should have been received (i.e., don‘t include open orders whose expected receipt date has not arrived). 3. Enter the date the order was placed. 4. Enter the quantity received in the last order. 5. Enter the date the order was received. 6. Note comments. Table E.3. Comparison of Quantity Ordered and Quantity Received Method/Brand/ Product Quantity Ordered for Last Order Period Date Order Placed Quantity Received in Last Order/Procurement Date Order Received Comments 1 2 3 4 5 6 Condom (female) Condom (male) Depo Provera Exluton/ Ovrette IUCD Lo-femenal Microgynon Neo-sampoon Noristerat Norplant 66 │ Logistics Indicators Assessment Tool (LIAT) Method/Brand/ Product Quantity Ordered for Last Order Period Date Order Placed Quantity Received in Last Order/Procurement Date Order Received Comments Postinor Gloves Syringe Logistics Indicators Assessment Tool (LIAT) │ 67 Background for Table E.4. Order Fill Rate—Only for State and LGA Stores Instructions 1. Fill in all authorized products of interest in column 1. (Note: Do this before finalizing the questionnaire and making photocopies.) 2. Obtain order forms received by this warehouse during the periods before the beginning month of the current survey (e.g., if the current survey runs from September to October, obtain order forms for the months of June, July, and August). Obtain forms corresponding to each lower- level facility to be visited during the survey, and complete a separate table for each lower-level facility. 3. Obtain issues records that correspond to each order, if not shown on the order forms. 4. In the appropriate space at the top of each table, write in the name of the lower-level facility that made an order to this issuing facility during the three months in question. 5. Under each ordering facility, enter the quantity that was ordered by the lower level and the amount that was supplied or issued by this facility. This information is used to calculate the line order fill rate. The total order fill rate can be calculated later by determining the percentage of facilities in which quantity supplied was equal to the quantity ordered for all listed products. 6. Record any notes or comments about why orders weren‘t filled in their entirety. 7. Use as many pages as needed to collect data for all facilities to be visited during the assessment. 8. Use December 31, 2004, for the end of the final period (period 3). Table E.4. Order Fill Rate–Only for State and LGA Stores Period 1 Period 2 Period 3 Name of Ordering Facility Product Quantity Ordered Quantity Supplied Quantity Ordered Quantity Supplied Quantity Ordered Quantity Supplied 68 │ Logistics Indicators Assessment Tool (LIAT) Period 1 Period 2 Period 3 Name of Ordering Facility Product Quantity Ordered Quantity Supplied Quantity Ordered Quantity Supplied Quantity Ordered Quantity Supplied Logistics Indicators Assessment Tool (LIAT) │ 69 Period 1 Period 2 Period 3 Name of Ordering Facility Product Quantity Ordered Quantity Supplied Quantity Ordered Quantity Supplied Quantity Ordered Quantity Supplied Comments, Notes, Reasons for Underfilled Orders 70 │ Logistics Indicators Assessment Tool (LIAT) Ask the person or people whom you interviewed if they want to ask you any questions or to give you any information they believe could be helpful for improving the logistics system. Comments or general observations on products management: Ask for spontaneous responses (enter in box above). If none, see suggested probes or questions to encourage responses (enter above): Ask how they feel the system is working? a. What is working best? b. What is not working? c. What suggestions they may have for improvements? d. Have you seen an improvement in product availability over the past two years? e. How has the cost-recovery scheme affected the management of the CLMS? Thank the person or people who talked with you. Reiterate how they have helped the program achieve its objectives, and assure them that the results will be used to develop improvements in logistics system performance. Notes or comments by the interviewer: Logistics Indicators Assessment Tool (LIAT) │ 71 << /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /All /Binding /Left /CalGrayProfile (Dot Gain 20%) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Warning /CompatibilityLevel 1.2 /CompressObjects /Tags /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJDFFile false /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.0000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments false /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Preserve /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 300 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 300 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False /Description << /CHS <FEFF4f7f75288fd94e9b8bbe5b9a521b5efa7684002000500044004600206587686353ef901a8fc7684c976262535370673a548c002000700072006f006f00660065007200208fdb884c9ad88d2891cf62535370300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002> /CHT <FEFF4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef653ef5728684c9762537088686a5f548c002000700072006f006f00660065007200204e0a73725f979ad854c18cea7684521753706548679c300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002> /DAN <FEFF004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002000740069006c0020006b00760061006c00690074006500740073007500640073006b007200690076006e0069006e006700200065006c006c006500720020006b006f007200720065006b007400750072006c00e60073006e0069006e0067002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /DEU <FEFF00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f00630068007700650072007400690067006500200044007200750063006b006500200061007500660020004400650073006b0074006f0070002d0044007200750063006b00650072006e00200075006e0064002000500072006f006f0066002d00470065007200e400740065006e002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /ESP <FEFF005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f0073002000640065002000410064006f0062006500200050004400460020007000610072006100200063006f006e00730065006700750069007200200069006d0070007200650073006900f3006e002000640065002000630061006c006900640061006400200065006e00200069006d0070007200650073006f0072006100730020006400650020006500730063007200690074006f00720069006f00200079002000680065007200720061006d00690065006e00740061007300200064006500200063006f00720072006500630063006900f3006e002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /FRA <FEFF005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f007500720020006400650073002000e90070007200650075007600650073002000650074002000640065007300200069006d007000720065007300730069006f006e00730020006400650020006800610075007400650020007100750061006c0069007400e90020007300750072002000640065007300200069006d007000720069006d0061006e0074006500730020006400650020006200750072006500610075002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e> /ITA <FEFF005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f006200650020005000440046002000700065007200200075006e00610020007300740061006d007000610020006400690020007100750061006c0069007400e00020007300750020007300740061006d00700061006e0074006900200065002000700072006f006f0066006500720020006400650073006b0074006f0070002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /JPN <FEFF9ad854c18cea51fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e30593002537052376642306e753b8cea3092670059279650306b4fdd306430533068304c3067304d307e3059300230c730b930af30c830c330d730d730ea30f330bf3067306e53705237307e305f306f30d730eb30fc30d57528306b9069305730663044307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e30593002> /KOR <FEFFc7740020c124c815c7440020c0acc6a9d558c5ec0020b370c2a4d06cd0d10020d504b9b0d1300020bc0f0020ad50c815ae30c5d0c11c0020ace0d488c9c8b85c0020c778c1c4d560002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e> /NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken voor kwaliteitsafdrukken op desktopprinters en proofers. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.) /NOR <FEFF004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200066006f00720020007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c00690074006500740020007000e500200062006f007200640073006b0072006900760065007200200065006c006c00650072002000700072006f006f006600650072002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e> /PTB <FEFF005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020007000610072006100200069006d0070007200650073007300f5006500730020006400650020007100750061006c0069006400610064006500200065006d00200069006d00700072006500730073006f0072006100730020006400650073006b0074006f00700020006500200064006900730070006f00730069007400690076006f0073002000640065002000700072006f00760061002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e> /SUO <FEFF004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f0074002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a00610020006c0061006100640075006b006100730074006100200074007900f6007000f60079007400e400740075006c006f0073007400750073007400610020006a00610020007600650064006f007300740075007300740061002000760061007200740065006e002e00200020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e> /SVE <FEFF0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740020006600f600720020006b00760061006c00690074006500740073007500740073006b0072006900660074006500720020007000e5002000760061006e006c00690067006100200073006b0072006900760061007200650020006f006300680020006600f600720020006b006f007200720065006b007400750072002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e> /ENU (Use these settings to create Adobe PDF documents for quality printing on desktop printers and proofers. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.) >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /ConvertColors /NoConversion /DestinationProfileName () /DestinationProfileSelector /NA /Downsample16BitImages true /FlattenerPreset << /PresetSelector /MediumResolution >> /FormElements false /GenerateStructure true /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /NA /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /LeaveUntagged /UseDocumentBleed false >> ] >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice

View the publication