Accurately forecasting contraceptive needs: levels, trends and determinants

Publication date: 2007

ACCURATELY FORECASTING CONTRACEPTIVE NEEDS: LEVELS, TRENDS, AND DETERMINANTS AUGUST 2007 This publication was produced for review by the U.S. Agency for International Development. It was prepared by the USAID | DELIVER PROJECT, Task Order 1. ACCURATELY FORECASTING CONTRACEPTIVE NEEDS: LEVELS, TRENDS, AND DETERMINANTS The authors' views expressed in this publication do not necessarily reflect the views of the U.S. Agency for International Development or the United States Government. USAID | DELIVER PROJECT, Task Order 1 The USAID | DELIVER PROJECT, Task Order 1, is funded by the U.S. Agency for International Development under contract no. GPO-1-01-06-0007-00, beginning September 29, 2006. Task Order 1 is implemented by John Snow, Inc., in collaboration with PATH, Crown Agents Consultancy, Inc., Abt Associates, Fuel Logistics Group (Pty) Ltd., UPS Supply Chain Solutions, Family Health International, The Manoff Group, and 3i Infotech. The project improves essential health commodity supply chains by strengthening logistics management information systems, streamlining distribution systems, identifying financial resources for procurement and supply chain operation, and enhancing forecasting and procurement planning. The project also encourages policymakers and donors to support logistics as a critical factor in the overall success of their health care mandates. Recommended Citation Karim, Ali Mehryar. 2007. Accurately Forecasting Contraceptive Needs: Levels, Trends, and Determinants. Arlington, Va.: USAID | DELIVER PROJECT, Task Order 1. Abstract Information on the expected accuracy of the contraceptive forecasting processes is useful for family planning supply chain managers to efficiently plan and procure contraceptive commodities and maintain uninterrupted supplies to meet clients’ needs. This study examines the accuracy of the contraceptive forecasting processes of 81 family planning programs in 30 developing countries using time-series records between 1994 and 2005 on past contraceptive consumption and projected needs. Forecast accuracy is defined as the absolute percentage difference between the actual and projected quantity of a contraceptive dispensed. Analysis of 1,586 one-year-ahead contraceptive forecasts indicates that the expected median absolute percent error for one-year-ahead contraceptive forecasts for public sector family planning programs is about 25 percent. Multiple regression analysis indicates that the forecast accuracy of public sector programs has been improving over time, which is partly attributable to improvement of the family planning logistics management information system performance and to the use of forecasting software. USAID | DELIVER PROJECT John Snow, Inc. 1616 Fort Myer Drive, 11th Floor Arlington, VA 22209 USA Phone: 703-528-7474 Fax: 703-528-7480 E-mail: deliver_project@jsi.com Internet: deliver.jsi.com CONTENTS Background.1 Contraceptive Forecasting Methods .1 Data .2 Explanatory Variables and Related Hypotheses.2 Measurements .3 Analysis.5 Results .7 Discussion.14 References.17 : Trends and Differentials.19 Appendix 1 Table A1: Trend in the Median Absolute Percent Forecast Error by Country (1995–2004).19 Table A2: Differentials in the Subsample Characteristics (1995, 1999, and 2000).22 Figures 1. Trend in the median absolute percent forecast error for public sector programs, all methods, 1995–2004 .9 2. Trend in the Direction of the Forecast Error for Public Sector Programs, All Methods (1995–2004); Average = ±25 Percent Difference Between Forecasted and Actual Consumption of a Contraceptive .10 3. Forecast Accuracy by Logistics Management Information System Performance Index Score, All Methods (1995, 1999, and 2000).11 4. Trend in the Percentage of Sample That Used PipeLine for Forecasting (1995–2004) .11 Tables 1. Description of the Items Used to Construct the Logistics Management Information System (LMIS) Performance Index among 26 Family Planning Programs in 14 Countries (1995, 1999, and 2000) .5 2. Percentage Distribution of the Sample by Selected Characteristics (n = 1,586).7 3. Trend in the Median Absolute Percent Forecast Error by Selected Sample Characteristics (1995–2004).8 4. Regression Models Predicting Median Absolute Forecast Accuracy (1995–2004) .12 5. Regression Models Predicting the Propensity of the Forecasting Accuracy Being Within Average (1995–2004) .13 iii iv BACKGROUND Forecasting contraceptive commodity needs in the short term is one of the vital functions of family planning logistics systems in order to obtain adequate financing to procure and maintain uninterrupted contraceptive supplies to meet client needs. Adequate funding for the increasing demand for contraceptives is a growing concern for governments/donors of most developing countries—which is complicated by the competing need for financing for other public health commodities (John Snow, Inc., [JSI] and Futures Group 2003; Sharma and Dayaratna 2005). In such circumstances, accurately forecasting contraceptive needs is vital if scarce public health resources are to be used efficiently and contribute to contraceptive security1. The expected accuracy of the forecasting processes in developing-country settings is not well documented—this information would help improve forecasts and efficiently finance and procure contraceptive commodities. To reduce the information gap, this study examines the levels, trends, and determinants of the accuracy of contraceptive forecasting processes in developing-country settings using data from NEWVERN, an information system containing yearly data on past distribution, future needs, and procurement and shipment plans of contraceptives for worldwide family planning programs supported by the U.S. Agency for International Development (USAID) under the DELIVER project. CONTRACEPTIVE FORECASTING METHODS Accuracy of forecasting using statistical methods depends on the availability of good-quality data and the number and accuracy of the assumptions associated with it. In developing-country settings, especially where the quality and reliability of logistics data is often inadequate, contraceptive forecasts are ideally obtained by combining two or more methods, using expert judgment (Family Planning Logistics Management [FPLM] 2000). The methods for forecasting depend on the different data sources—primarily logistics data (the actual amount dispensed to end users) of contraceptives; service statistic (data on clients and visits); demographic data; and distribution capacity of the program. Forecast accuracy based on past consumption is considered the best method because very few assumptions are associated with it—the main assumption being that the future consumption will follow historical trends. Service statistics-based forecasts are less likely to be accurate than logistics data-based forecasts because, in addition to assuming that the future family planning service use pattern follows the past trend, it assumes that all clients receive the expected quantity of supplies during their visit to a service delivery point. Demographic data-based forecasts are likely to be less accurate than historical data-based forecasts because they require many assumptions. Demographic data-based forecasts assume that the program will reach its demographic goal for the end of the forecast year; additional assumptions are then applied to obtain the annual contraceptive requirements per user and the expected change in method and source mix. A small deviation from any of these assumptions has a significant impact on the forecast. Excluding new or rapidly expanding programs, distribution capacity-based forecasts are seldom used as a stand-alone method to determine contraceptive needs in the short term. Nevertheless, distribution capacity-based forecasts are carried out routinely to assess whether the program will be able to manage the quantity of commodities forecast using other methods. To get the final forecast estimate, the quality and assumptions associated with data from different sources are assessed, and the final estimate is obtained using the best judgment of program managers and experts (for details on contraceptive forecasting methodologies, see FPLM 2000). Adjusting forecasts subjectively incorporates the influence of changes in the external environment and programmatic policies on contraceptive demand. 1 Contraceptive security exists when every person is able to choose, obtain, and use quality contraceptives whenever he or she needs them (Sharma and Dayaratna 2005). 1 DATA Each year, to plan for procurement, for every family planning program receiving USAID support for commodities, a Contraceptive Procurement Table is prepared for each of the required contraceptive products by brand. The year for which procurement plans are made based on the forecast, is referred to as the Contraceptive Procurement Table planning year; the year preceding the Contraceptive Procurement Table planning year is called the Contraceptive Procurement Table year. The Contraceptive Procurement Table for a contraceptive also reports the actual annual consumption during the two years prior to the Contraceptive Procurement Table year. All Contraceptive Procurement Tables since 1987 have been archived in a management information system, NEWVERN, which allows assess to forecast accuracy by comparing the forecast consumption with the actual. Since brand names for condoms change frequently over time, two-years-apart Contraceptive Procurement Tables for condom are matched based on product rather than brand name to avoid unnecessary exclusion of observations. Annual contraceptive forecasts for the past decade—between 1995 and 2004— contained in the Contraceptive Procurement Tables between 1994 and 2003 are assessed for accuracy. The analysis is restricted to 1,586 Contraceptive Procurement Tables from 30 countries where John Snow, Inc. (JSI), provided technical assistance to improve family planning programs’ supply chain. The sample is also restricted by the ability to link two-years-apart Contraceptive Procurement Tables to get the forecast accuracy measure. Reasons for the inability to link between two-years-apart Contraceptive Procurement Table include (1) the product or the program is phased out, (2) the program is no longer a USAID-supported program, (3) the product brand name changed between the two Contraceptive Procurement Table years, and (4) some of the 2005 Contraceptive Procurement Tables were not completed during the time of this analysis. About 56 percent of the forecasts in the NEWVERN from the reference period could be validated for accuracy. EXPLANATORY VARIABLES AND RELATED HYPOTHESES In addition to methodological issues, the accuracy of forecasting contraceptives is influenced by many other factors inside and outside the family planning logistics management systems. Data on past consumption are routinely available at the service delivery points, but obtaining these data at the central level for forecasting and other logistics decision-making purposes requires an information system—the logistics management information system (LMIS). The LMIS also provides data on the usable stock level of commodities, including losses and adjustments2 at all warehouses and service delivery points of a program, which are essential for logistics decision making. These logistics data elements are not generally considered to be program performance indicators, so they are not usually collected by routine health management information systems. Therefore, establishing an LMIS becomes essential for efficiently managing the supply chain of public health commodities. Programs with well-functioning LMISs are expected to provide historical data on consumption, which would allow for more accurate forecasting (FPLM 2000; JSI/DELIVER 2004). Accordingly, the functional level of the family planning logistics management information system is included as an explanatory variable for forecast accuracy. Using PipeLine software for forecasting contraceptive needs is expected to improve forecasting accuracy by avoiding mathematical errors and allowing for more frequent forecasting. Increasing the frequency of forecasting for short-term consumption allows for adjusting and improving longer-term forecasting (FPLM 2000; Wilson 1995). Accordingly, using PipeLine to prepare Contraceptive Procurement Tables is included in the analysis as an indicator variable. 2 Losses are the quantity of stock removed from the supply chain for any reason other than consumption by clients, for example, due to expiration, theft, damage, and so on (for details on logistics management information system, see JSI/DELIVER 2004: 26). 2 JSI’s technical assistance to improve the family planning logistics management systems of the programs for which Contraceptive Procurement Tables are prepared varied over time and between programs. Nevertheless, we expect that technical assistance, with the Contraceptive Procurement Table preparation exercise, will improve the logistics systems, including forecasting and procurement capacity, over time. Accordingly, the forecast period or year will become one of the important explanatory variables for this study. Although JSI prepares Contraceptive Procurement Tables for all USAID-supported programs in focus countries, its major clients for receiving technical assistance to improve the supply chain management systems for health and family planning commodities are public sector programs; as such, forecast accuracy is expected to improve primarily for public sector family planning programs. Selection of the contraceptive mix and brand that the system delivers reflects client preferences for family planning methods, so it may affect contraceptive consumption. To capture heterogeneity due to method mix, contraceptive products are categorized into five broad method categories: there are 10 products for condom, 20 products for pill, four products for injectable, and a single product for intrauterine device (IUD) and for implant. The NEWVERN database tracks the quantity of a contraceptive product shipped to the respective programs for which Contraceptive Procurement Tables are prepared. Along with the projected consumption, the Contraceptive Procurement Tables record the required quantity of a particular contraceptive to be procured. The required quantity proposed is based on the rate of forecasted consumption, available stock on hand, expected losses and adjustments, expected lead time in procuring contraceptives, and distribution capacity of the supply chain. Unexpected delays and inadequate quantity of shipments of a contraceptive may lead to stockouts, which could reduce the level of expected consumption and lead to forecast error. Accordingly, the variable adequate shipment is included as an explanatory variable. For any given year, if a program received 75 percent3 or more of the proposed required quantity of a product, then shipment for that product is considered to be adequate; if a program receives less than 75 percent, it is considered to be inadequate. The selection of explanatory variables for forecast accuracy for this study is limited by data availability. Important explanatory variables for contraceptive forecast accuracy may have remained unmeasured. Forecasting methodology is expected to influence forecast accuracy. However, the forecasting methodology for each of the Contraceptive Procurement Tables could not be meaningfully categorized because logistics advisors combined different methods of forecasting to obtain the final forecast based on their best judgment. Other unmeasured explanatory variables included unexpected policy changes and environment and other exogenous factors influencing the supply or demand side of a family planning program. MEASUREMENTS FORECAST ACCURACY Let us suppose that At indicates actual quantity of a contraceptive distributed at time t, and Ft indicates the forecast of At. Then forecast accuracy or forecast error is et = At – Ft. Summarizing forecast accuracy using the mean (or median) error is not very useful because the positive and negative errors cancel each other out4. Therefore, this study defines forecast accuracy using the absolute value of the forecast error (i.e., |et|). To be able to compare forecast accuracy over time and across countries and programs, the forecast accuracy is expressed as a percentage and referred to as the absolute percent forecast error (APE)—that is, the absolute percent difference between the actual and projected quantity given by 100 × |et| / At. Although there are other measures for comparing forecast accuracy—for example, the relative absolute forecast error and the cumulative relative absolute forecast error (for details, see Armstrong and Collopy 1992; Armstrong and Green 2005; or Hyndman and Koehler 2006)—for the sake of simplicity in interpreting forecast accuracy, the 3 The cutoff point, 75 percent, is based on expert opinion. 4 The mean or the median forecast error is mainly useful to assess whether forecasts are over- (indicated by negative value) or under-estimating (indicated by positive value) the actual consumption and its magnitude. 3 last two measures are not used. Forecast accuracy, forecast error, and APE are used synonymously in the rest of the text unless mentioned otherwise. LOGISTICS MANAGEMENT INFORMATION SYSTEM For the purpose of monitoring and evaluation, JSI and the Centers for Disease Control and Prevention (CDC) developed the Composite Indicators for Contraceptive Logistics Management, a tool to quantify the functional level of logistics systems for family planning programs in developing-country settings (JSI/FPLM and the Centers for Disease Control and Prevention [JSI/CDC] 1999; Karim 2005). The Composite Indicators for Contraceptive Logistics Management, based on the different functions of the logistics systems, uses 23 items to obtain information on eight aspects of family planning logistics systems: use of a logistics management information system (LMIS), forecasting, procurement, warehousing, distribution, organization and staffing, policy, and adaptability. To collect information using the tool, logistics advisors conduct in-depth interviews with family planning program policymakers and managers, who are knowledgeable about their country’s logistics systems. Interviews are conducted through workshops or group discussions. To prevent any strong leadership from dominating group discussions, supervisors and subordinates are grouped separately; workshop facilitators specifically look for group dynamics such as the bandwagon effect5 by encouraging all participants to provide their own opinions. The Composite Indicators for Contraceptive Logistics Management has two scores for each item: one for performance and one for sustainability. The scores for each of the items are recorded using Likert-type scales, ranging between zero and four or zero and two. This scoring method gives higher weight to items considered relatively more important. The score for each item is recorded based on consensus among the key informants. JSI used this tool to measure the performance of contraceptive logistics systems in 1995, 1999, and 2000 in 28 countries, 17 of which are measured at two separate points. This analysis uses only the performance score and not the sustainability score because (1) the latter is less likely to influence short-term programmatic outcomes, such as forecasting accuracy, and (2) logistics advisors have been especially critical of the measurement error of the sustainability component due to its subjectivity (Karim 2005; Gelfeld 2000). The scores of the four items on LMIS performance are aggregated to create a Logistics Management Information System Performance Index that measures the performance or functionality of the reference program’s LMIS. The minimum and maximum possible score of the index ranged from 0 to 12, and a comparatively higher score indicates that the program has a comparatively better LMIS performance. The Cronbach’s reliability coefficient6 of the index is 0.7. A dose-response relationship is expected between the functional level of LMIS and the forecast error—that is, a comparatively high score of the Logistics Management Information System Performance Index is expected to be associated with comparatively high forecast accuracy. Table 1 describes the items used to construct the Logistics Management Information System Performance Index. Information on the functional level of the LMIS of the family planning programs is limited to 26 programs in 14 countries for forecast years 1995, 1999, and 2000, thereby restricting the sample size for analyzing the influence of LMIS performance on forecast accuracy to 206. 5 The bandwagon effect is defined as people often doing or believing things because many other people do or believe the same. 6 Cronbach’s reliability coefficient, a measure of the consistency or reliability of a scale or index, is defined as the square of the correlation between the scale or index and the included variables. Alpha values of 0.70 or higher are usually desirable for acceptable reliability (StataCorp 2005a). 4 Table 1: Description of the Items Used to Construct the Logistics Management Information System (LMIS) Performance Index among 26 Family Planning Programs in 14 Countries (1995, 1999, and 2000) Item Maximum Score Mean Score 1995 n = 22 1999 n = 12 2000 n = 13 Program has basic elements of LMIS. 4 2.0 3.3 3.0 LMIS information is used in management decision making. 4 1.7 3.0 2.7 LMIS information is fed back to all lower levels in the distribution system. 2 0.6 1.1 1.0 Commodities data are validated by cross-checking with other data sources. 2 0.6 1.5 1.0 Total score for the LMIS capacity index 12 5.2 8.8 7.7 The Logistics Management Information System Performance Index may be criticized for the subjectivity of respondents, inter-rater reliability, and variance of the quality and source of data (Gelfeld 2000; Karim 2005). Part of the subjectivity and reliability of the Contraceptive Logistics System Performance Index is improved by focusing only on the performance score of the items. Correction for measurement error of the index is discussed further in the methodology section. ANALYSIS To compare forecast accuracy over time and across programs, the central tendency of the absolute percent forecast error is described using the median, rather than the mean, because of its higher reliability and robustness to outliers (Armstrong and Collopy 1992). The presence of outliers is not infrequently expected for several reasons that may or may not relate to the forecasting methodology, including (1) unforeseen inclusion or exclusion of another contraceptive product or service delivery system in the market that significantly decreased or increased use of the reference product under study, leading to a larger than expected forecast error; (2) the methodology for estimating actual use may have changed over time—for example, the source of data for estimating actual consumption may have changed over time, which could result in larger than expected forecast error; and (3) in some cases, the lack of adequate consumption data could have introduced a larger than expected forecast error. Some of these reasons may not necessarily produce outliers; rather, they may partly explain the observed forecast error. First, the crude or the bivariate association of the forecast accuracy with the selected explanatory variables is assessed using naïve median regression models—that is, median regression models with single predictors7. Next, more complex models are used to address the threat to validity of the crude models. The major issues of the naïve/crude models included (1) measured and unmeasured heterogeneity (i.e., confounding), (2) measurement error of the dependent and independent variables of interest, and (3) the non-independence of repeated observations over time.8 Accurately estimating the forecast accuracy— that is, the dependent variable—depended on the reliability and validity of data on actual consumption, which, for a few programs, due to lack of logistics data, is also an estimate (Wilson 1995). 7 A median regression (also known as the quantile regression) model is very similar to ordinary regression. However, instead of getting the line that minimizes the sum of the squares of the residuals, as in ordinary regression, the median regression model finds the line that minimizes the sum of the absolute residuals (StataCorp 2005b). 8 Commonly used statistical tests assume that all observations are independent of each other. 5 All of the explanatory variables are added to the naïve model to control for confounders. Country-level, fixed-effects median regression models are used to correct for the portion of the measurement error on the dependent (i.e., forecast accuracy) as well as on the independent variables (i.e., specifically the Logistics Management Information System Performance Index) that remained constant over time (Schultz 1993, 1994). In other words, the fixed-effects model controlled for country-level unobserved confounders and the measurement error of the explanatory variables that remained constant over time. For example, the portion of measurement error of the forecast accuracy measure due to the error in estimating the actual use would be eliminated by the proposed regression model if the errors are similar between two points. The common error between the forecast years would cancel each other out. The time-varying unobserved confounders and measurement errors of the independent variables that are related to the forecast accuracy remained as a threat to the validity of this study. A time-varying measurement error of forecast accuracy could occur if the subjective nature of the LMIS performance raters changed over time. For example, if the raters knew that the forecast accuracy is improving or declining during the analysis period, and they scored the performance of the LMIS accordingly—which is unlikely. The median regression models are estimated to predict forecast accuracy using bsqreg command implemented by StataCorp (2005c). The country-level, fixed-effects model is obtained by including dummy variables for countries (Wooldridge 2003; Koenker 2004). The standard errors for the model are obtained by the nonparametric bootstrap method—which is robust to the complications of estimating standard errors when observations are non-independent (Efron and Tibshirani 1993; Gould 1992). The bootstrap method is also implemented for getting the standard errors of the naïve models, and the fixed-effects model also accounted for the non-independence of the observations (Hsiao 1986). Other than for estimating the variation in forecast error over time, the indicator variable for the forecast period included in the model accounted for the secular change in forecast accuracy due to all other factors not accounted for by the observed variables. The variation in forecast accuracy over time is assessed using the following equation for the median regression model: MdAPEit = xit β +υ i + ε it .(1) Where, MdAPE is the median absolute percent error for country i for the period t; x is the vector of explanatory variables; υ is the country-level fixed-effect (i.e., country-level unobserved determinants of forecast accuracy that remain constant over time); and ε indicates unexplained determinants of forecast accuracy that change over time. The major assumption of the model is that ε is not correlated with x. The effect of the Logistics Management Information System Performance Index on forecast accuracy is analyzed separately because of the smaller sample available (i.e., only 206 cases) for the index. Accordingly, equation 1 is estimated with and without the variable measuring LMIS performance. The differential in the subsample characteristics used for the analysis of the effect of LMIS performance on forecast accuracy is also assessed to understand the possible bias due to non-random sample selection. By definition, the MdAPE does not indicate whether the forecast for a contraceptive over- or underestimated actual consumption. Therefore, to identify the extent to which the projected use overestimated or underestimated actual consumption, the percentage difference between the projected and actual consumption is categorized into three groups: (1) the projected use for a contraceptive that underestimated consumption by more than 25 percent; (2) the projected use that is within ±25 percent of the actual use (referred to as the average in the tables and figures); and (3) the projected use that overestimated consumption by more than 25 percent. The 25 percent cutoff point for the average forecast error is used because that level is considered acceptable for one-year-ahead forecasts for such commodities by U.S. commercial standards (Frazelle 1999; Wilson 1995). Logit models analogous to the median regression models are estimated to predict the propensity of average forecasting to validate the findings of the median regression models. Results from the median regression models are preferable as they allowed quantification of the magnitude of the forecast error. 6 RESULTS DESCRIPTIVE ANALYSIS Table 2 displays the characteristics of the sample. The sample is more or less evenly distributed over the analysis period. About half the sample is from the Africa region (AFR), which represents 33 family planning programs from 14 countries; 37 percent is from the Latin America and the Caribbean region (LAC), representing 41 programs from 10 countries; and the remaining 13 percent is from Asia and the Near East region (ANE), representing seven programs from six countries.9 Table 2: Percentage Distribution of the Sample by Selected Characteristics (n = 1,586) Forecast period Method Program type 1995 12.8 Condom 21.4 Public sector 56.4 1996 10.7 Oral pill 34.9 Other 43.6 1997 10.5 Injectable 18.6 Adequate shipment (received 75%+ of plan) 1998 9.7 IUD 17.5 1999 7.8 Implant 7.6 No 31.2 2000 10.3 Region Yes 68.8 2001 10.3 Africa 50.6 PipeLine used 2002 10.3 ANE 12.6 No 49.1 2003 9.5 LAC 36.8 Yes 50.9 2004 8.1 Oral pill represents about one-third of the sample; condom, injectable, and IUD each represents more or less one-fifth of the sample; and implant represents the rest of the sample (8 percent). The majority (56 percent) of the forecasts are for public sector programs. The funding source for the projected contraceptive needs is mainly USAID (81 percent), followed by the United Nations Fund for Population Activities (UNFPA), Department for International Development (DFID), and International Planned Parenthood Federation (23, 6, and 5 percent, respectively; analysis not shown). For about two-thirds of the sample, the quantity of the contraceptive the program received is adequate. PipeLine software is used to prepare about half of the Contraceptive Procurement Tables included for this analysis. The median absolute percent error (MdAPE) of the pooled data over the analysis period is 31 percent (see table 3). The 25th percentile of the pooled forecast error is 13 percent, the 75th percentile is 67 percent (analysis not shown). About 5 percent of the forecast error is above 300 percent and classified as outliers; graphic eyeball assessment of the outliers revealed that they are primarily associated with forecasts for a relatively smaller quantity of a contraceptive (analysis not shown). 9 Countries included Burkina Faso, Cameroon, Ghana, Guinea, Madagascar, Malawi, Mali, Mozambique, Rwanda, Senegal, Tanzania, Togo, Uganda, and Zimbabwe from Africa; Bolivia, Dominican Republic, Ecuador, El Salvador, Guatemala, Haiti, Honduras, Nicaragua, Paraguay, and Peru from LAC; and, Bangladesh, Egypt, Jordan, Morocco, Nepal, and Philippines from ANE. 7 CRUDE ANALYSIS: POOLED The crude association between forecast accuracy and the explanatory variables is assessed using naïve median regression models after pooling the data over the analysis period (see table 3). The explanatory variables over which the forecast accuracy significantly varies are marked with asterisks and appear under the column labeled Pooled in table 3. The pooled MdAPE varies significantly by contraceptive method. The lowest forecast error is observed for oral pill (25 percent), followed by forecast error for condom (30 percent), injectable (32 percent), IUD (37 percent), and implant (46 percent). Table 3: Trend in the Median Absolute Percent Forecast Error by Selected Sample Characteristics (1995–2004) Characteristics Pooled 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Method *** Condom 30.0 34.2 33.5 35.4 21.4 32.5 26.3 17.6 23.2 37.3 30.0 ** (339) (54) (42) (38) (35) (26) (32) (31) (30) (29) (22) Oral pill 25.0 28.8 26.0 21.0 26.2 25.4 28.3 23.2 21.3 25.5 23.7 (553) (72) (56) (53) (50) (44) (58) (61) (59) (53) (47) Injectable 32.0 58.4 58.3 24.6 33.0 19.3 44.7 20.4 23.8 25.0 30.3 *** (295) (23) (27) (32) (30) (24) (32) (31) (36) (33) (27) IUD 37.2 27.9 41.2 40.3 35.8 25.2 65.5 47.6 43.3 29.3 30.3 (278) (42) (34) (32) (28) (21) (25) (26) (24) (24) (22) Implant 45.8 150.9 105.9 38.2 39.4 39.6 71.8 39.3 25.0 40.0 48.9 (121) (12) (10) (12) (11) (9) (16) (15) (15) (11) (10) Program ** Public sector 28.6 33.9 37.5 30.0 29.8 21.6 33.7 21.4 24.9 24.4 25.0 *** (894) (107) (88) (91) (81) (79) (84) (94) (93) (90) (87) Other 34.2 38.3 33.8 29.1 27.6 38.9 44.6 33.3 29.1 34.5 50.9 (692) (96) (81) (76) (73) (45) (79) (70) (71) (60) (41) Shipment adequacy *** No 44.7 53.8 30.8 41.3 59.7 33.1 60.0 40.7 51.4 44.4 41.3 (495) (60) (52) (59) (48) (40) (64) (41) (52) (38) (41) Yes 27.3 32.0 37.5 28.6 24.1 25.0 32.3 22.9 21.2 25.7 25.0 *** (1091) (143) (117) (108) (106) (84) (99) (123) (112) (112) (87) Quantity projected *** 100,000+ 24.7 28.6 31.6 26.5 22.7 23.8 27.4 19.6 22.3 29.3 25.5 (846) (103) (86) (88) (81) (72) (79) (94) (86) (83) (74) <100,000 38.6 43.3 42.9 32.0 36.1 37.6 48.0 51.5 29.9 29.5 40.2 (740) (100) (83) (79) (73) (52) (84) (70) (78) (67) (54) PipeLine used ** No 33.3 34.3 36.9 28.6 27.9 33.8 36.4 35.6 22.2 (779) (203) (169) (150) (102) (67) (53) (20) (15) (0) (0) Yes 28.8 73.6 34.5 16.8 39.3 23.9 26.5 29.4 30.0 (807) (0) (0) (17) (52) (57) (110) (144) (149) (150) (128) ** Total 30.9 34.3 36.9 29.5 29.6 26.5 38.7 24.5 26.1 29.4 30.0 (1,586) (203) (169) (167) (154) (124) (163) (164) (164) (150) (128) Note : Sample sizes are given in parentheses; *p<.1; **p<.05; ***p<.01. 8 The pooled median forecast error also varies by program type. As expected, the median forecast error for public sector programs is lower (29 percent) than those for other programs (34.2 percent). Because the outliers are mainly observed for forecasts for relatively low quantities of a contraceptive, the variation of the forecast error by the quantity of a contraceptive forecasted is also assessed. As expected, the median forecast error is higher when the forecasted quantity of a contraceptive is less than 100,000 units. The relationship between the forecast error and adequate shipment is significant. The median pooled forecast error is 27 percent when the shipment is adequate and 45 percent when it is not adequate. Although inadequate shipment as planned may have caused lower consumption, leading to greater discrepancy between projected and actual use,10 it is possible that the Contraceptive Procurement Table advisors know that the program would order less than what it proposed; therefore, to ensure enough supply, they deliberately proposed a higher quantity to ship, which led to higher forecast error. Use of PipeLine to prepare Contraceptive Procurement Tables is associated with lower forecast error, as expected. The median forecast error of the pooled analysis shows that forecast error is 29 percent when PipeLine is used, compared to 33 percent when it is not used. CRUDE ANALYSIS: VARIATION OVER TIME Table 3 also gives the median forecast error for each year between 1995 and 2004 by selected explanatory variables. The variation of the forecast error over time by the categories of the independent variables is assessed using naïve median regression models (with forecast years as dummy variables). Statistically significant change of the median forecast accuracy over time is marked by asterisks at the row ends of table 3. The crude analysis of the full sample indicates that there is no significant variation of the forecast error over time. However, when the variation of forecast accuracy is assessed according to the explanatory variables, statistically significant variation of the median forecast error is observed for condom, injectable, and public sector programs; when contraceptive shipments to the program are adequate; and when PipeLine is used for forecasting (see table 3). The variation of the forecast accuracy over the analysis period for public sector programs appears to have an improving trend, as expected. Between 1995 and 1998, the median forecast error for public sector programs was 30 percent or above and declined to 25 percent or below between 2001 and 2004. A naïve regression model with forecast period as a linear predictor for median forecast error indicates that declining trend of the forecast error among public sector programs is significant (see figure 1); on average, the forecast error for the public sector program improved by about 1.2 percentage point per year, from 34 percent in 1995 to 25 percent in 2004. The unusually high forecast error (about 34 percent) for public sector programs observed in 2000 is due primarily to an increase in forecast error Figure 1: Trend in the median absolute percent forecast error for public sector programs, all methods, 1995–2004 20 25 30 35 40 M ed ia n fo re ca st e rr or (% ) 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Median error = 32.99 - 1.15x (Trend) 10 Inadequate receipt of shipment for a given year is not likely to reduce consumption substantially for the same year because of the buffer stock already present in the pipeline, which within a country consists of central and peripheral warehouses and storages at the service delivery points. Most family planning programs have at least nine to 12 months of contraceptive stock within the in- country pipeline. 9 30 38 32 28 38 34 21 43 36 21 46 33 13 57 30 14 45 40 12 59 30 12 52 37 12 53 34 10 53 37 0 20 40 60 80 10 0 pe rc en t 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Under Average Over for implant, IUD, and injectable for that year. Next, the direction of the forecast error is Figure 2: Trend in the Direction of the Forecast assessed. Almost half (about 45 percent) of Error for Public Sector Programs, All Methods all forecasts during the analysis period are (1995–2004); Average = ±25 Percent Difference within average forecasting—that is, the Between Forecasted and Actual Consumption of forecast error is within ±25 percent of actual a Contraceptive consumption. The pooled data indicate that the likelihood of the forecasts to overestimate actual consumption is more than twice what it is to underestimate it. During the analysis period, 38 percent of the forecasts overestimated consumption by more than 25 percent; while only 17 percent of the forecasts underestimated consumption by less than 25 percent. The trend in the direction of the forecast error for the full sample did not show any significant variation over time (analysis not shown). However, the trend in propensity for average forecasting improved over time for public sector programs (see figure 2), which is consistent with the declining trend of median forecast error observed in the crude analysis. The propensity of average forecasting increased significantly, from 38 percent in 1995 to 53 percent in 2004. The increase in average forecasting among the public sector programs is due primarily to the declining trend of under-forecasting from 30 percent in 1995 to 10 percent in 2004. The forecast accuracy by country is given in table A1 in the appendix. CRUDE ANALYSIS: SUBSAMPLE As indicated earlier, a subsample of the study is analyzed to assess the influence of LMIS performance on forecast accuracy. First, the differentials of the distribution of the explanatory and the outcome variables by sample selection are assessed to see the expected sample selection bias. The comparison is restricted to periods for which data on performance of logistics management information systems are available (i.e., during 1995, 1999, and 2000; see table A2 in the appendix). The analysis indicates that the selected sample is more likely to be public sector programs, which represents nearly two-thirds (73 percent) of the selected sample, while it represents less than half (42 percent) of the sample not selected. Such a selection bias is expected because selection is dictated by the availability of the Composite Indicators for Logistics Management data, which are mainly available for public sector programs, the major clients for JSI’s supply chain management improvement project. Although the median forecast error of the selected sample is lower (28 percent) than of those that are not selected (36 percent)—reflecting the better forecast accuracy of the public sector programs that dominated the sample—the difference is not statistically significant with the critical value for alpha error set at 5 percent. It is interesting to note that the likelihood of using PipeLine to conduct forecasting is higher in the selected sample (54 percent) than it is in those not selected (20 percent). Table 1 indicates that the Logistics Management Information System Performance Index score increased by nearly twofold, from 5.2 in 1995 to 8.8 in 1999, and then decreased slightly, to 7.7 in 2000. Nevertheless, the decrease of the index score between 1999 and 2000 is not statistically significant (p>.05), while the change of the index score between 1995 and 1999 or between 1995 and 2000 is significant (p<.01). Whether programs with higher scores for the index had better forecast accuracy is assessed next. Figure 3 shows the median forecast error by the observed score for the Logistics Systems Management Information Performance Index. As expected, the crude analysis indicates that the median forecast error is lower when the Logistics 10 Management Information System Performance Index score is comparatively high. The median forecast error is 50 percent or higher when the score for the index is less than two, and the forecast error declines to 8 percent when the score for the index is maximum (i.e., 12). However, analysis in figure 3 does not account for changes in the covariates over time and the repeated measures. ADJUSTED ANALYSIS: FULL SAMPLE Multiple regression models are used to obtain the adjusted influence of the explanatory variables on forecast accuracy that minimized the different sources of bias. The Logistics Management Information System Performance Index is dropped from the full Figure 3: Forecast Accuracy by Logistics sample regression analyses, then, using the rest of the explanatory variables, a country-level, Management Information System Performance fixed-effects median regression model is Index Score, All Methods (1995, 1999, and 2000) estimated (see model 1 in table 4). Consistent with the crude analysis, model 1 indicates that oral pill, program type, shipment adequacy, and quantity projected are the significant correlates of forecast accuracy, as observed in the bivariate analysis. Forecast accuracy for oral pill is greater than it is for condom; the accuracy is better for public sector family planning program than it is for other programs; forecast error is lower when shipments are adequate than it is when shipments are not adequate; and relatively small projected quantity of a contraceptive is likely to be associated with relatively large forecast error. However, the effects of forecast period and PipeLine are not statistically significant in model 1 (of table 4). The use of PipeLine to prepare Contraceptive Procurement Tables increased in a linear trend from 0 percent in 1995 to 10 percent in 1997 and reached 100 percent in 2003 (see figure 4), indicating nearly absolute correlation between the use of PipeLine and the last four forecast periods (i.e., 2001 to 2004). Therefore, it is likely that PipeLine use and forecast period in model 1 are colinear—that is, use of PipeLine software explained the portion of the variation of the forecast error that is also explained by the Figure 4: Trend in the Percentage of Sample forecast period. That Used PipeLine for Forecasting (1995–2004) 0 50 10 0 15 0 M ed ia n fo re ca st e rr or (% ) 0 2 4 6 8 10 12 LMIS score Median forecast error vs. LMIS performance score To assess whether PipeLine use is colinear with the forecast period in model 1, the model is re-estimated, dropping forecast period to get model 2 in table 4. As expected, the effect of PipeLine appeared significant in model 2 when forecast period is dropped; PipeLine use decreased the median forecast error by a modest 5.6 percentage points. Next, the validity of the declining trend of the forecast error among public sector programs observed from the crude analysis is assessed. For this purpose, a country-level, fixed-effect median regression model is estimated, with forecast period specified as a linear trend and the sample restricted among the public sector 0 0 10 34 46 67 88 91 100 100 0 20 40 60 80 10 0 pe rc en t 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 11 programs (see model 3 in table 4). PipeLine is not included in model 3 due to its expected colinearity with the trend effect. The Logistics Management Information System Performance Index is also omitted from the model to include all forecast periods. After controlling for variation due to method, shipment adequacy, quantity projected, and country-level differences, Model 3 indicates that median absolute percent forecast error for public sector programs declined steadily over the analysis period by an average of 1.1 percentage points each year. Table 4: Regression Models Predicting Median Absolute Forecast Accuracy (1995–2004) Independent Variable Coef. (s.e.) Coef. (s.e.) Coef. (s.e.) Coef. (s.e.) Model 1 (n = 1,586) Model 2 (n = 1,586) Model 3 (n = 894) Model 4 (n = 206) Forecast period 1.23 (9, 1539) 1996 2.410 (3.864) 1997 -6.624 (4.258) 1998 -3.910 (4.813) 1999 -9.518 (4.995) * 2000 -2.430 (6.567) 2001 -7.851 (5.477) 2002 -8.708 (5.193) * 2003 -5.382 (5.441) 2004 -7.882 (5.688) Trend -1.103 (0.457) ** -0.357 (1.739) LMIS performance index -3.240 (1.567) ** PipeLine used Yes (No) 0.536 (4.017) -3.978 (1.914) ** Method 2.07 (4, 1539) * 1.62 (4, 1548) 1.00 (4, 862) 0.93 (4, 183) Oral pill (condom) -5.387 (2.551) ** -5.597 (2.658) ** -4.102 (2.714) -2.082 (6.385) Injectable -0.023 (2.914) -1.313 (2.783) -0.811 (3.214) 13.379 (8.989) IUD 0.549 (3.459) -1.198 (3.449) -0.173 (4.004) -1.700 (9.883) Implant 4.808 (6.599) 3.158 (6.454) 1.950 (7.114) 3.164 (25.237) Program type Other (Public sector) 5.664 (2.698) ** 4.069 (2.374) * 12.437 (9.333) Shipment adequacy Yes (No) -18.238 (3.178) *** -19.065 (3.118) *** -15.470 (3.733) *** -13.361 (7.919) * Quantity projected Less (More) 12.593 (2.700) *** 13.471 (2.861) *** 12.480 (2.902) *** 8.518 (8.065) Country 3.57 (29, 1539) *** 3.93 (29, 1548) *** 5.44 (24, 862) *** 1.68 (13, 183) * Constant 56.751 (6.721) *** 55.188 (6.862) *** 41.987 (37.291) 41.781 (16.664) ** Note: Reference category for the qualitative independent variables is in parentheses. The coefficients are estimated using median regression models. Wald tests with degrees of freedom are reported for the qualitative predictors with more than two categories. The coefficients of the country effects are not presented; *p<.1; **p<.05; ***p<.01. ADJUSTED ANALYSIS: SUBSAMPLE The bivariate relationship between LMIS performance and forecast accuracy found earlier is further assessed using regression analysis. The median regression model is used for this purpose (see model 4 in table 4). The variable indicating PipeLine use is omitted because of its expected colinearity with the survey period. Model 4 in table 4 indicates that the observed relationship between forecast accuracy and Logistics Management Information System Performance Index remains significant, even after controlling for variation of the forecast error due to trend, method, shipment accuracy, country, and program type. For each point of 12 increase in Logistics Management Information Performance Index score, the median forecast error declines by 3.2 percentage points. Therefore, it is expected that a substantial reduction (about 16.8 percentage points on average11) of the forecast error can be achieved if a perfect score for Logistics Management Information System Performance Index can be obtained. The findings from median regression models 1, 2, 3, and 4 in table 4 are compared with their analogous country-level, fixed-effects logistic regression models 1, 2, 3, and 4 in table 5, respectively, for the purpose of validation. The logit models predict the propensity of the forecasted quantity of contraceptive to be within ±25 percent of actual consumption. A largely similar conclusion is derived from the logit models. As with the median regression models, the logit models indicated that oral pill, shipment adequacy, quantity projected, PipeLine use, and Logistics Management Information System Performance Index are the significant correlates of forecast accuracy. Although program type is not a significant predictor of average forecasting in the logit models, the trend effect for the public sector program is significant in model 3 of table 5, which supports indicating improvement of forecasting accuracy among the public sector programs. Therefore, the conclusions derived from the logit models validate the conclusions from the median regression models. Table 5: Regression Models Predicting the Propensity of the Forecasting Accuracy Being Within Average (1995–2004) Independent variable Coef. (s.e.) Coef. (s.e.) Coef. (s.e.) Coef. (s.e.) Model 1 (n = 1,586) Model 2 (n = 1,586) Model 3 (n = 894) Model 4 (n = 194) Forecast period 5.76 1996 -0.078 (0.225) 1997 0.144 (0.228) 1998 0.162 (0.242) 1999 0.269 (0.270) 2000 0.016 (0.287) 2001 0.321 (0.310) 2002 0.431 (0.313) 2003 0.108 (0.336) 2004 0.146 (0.330) Trend 0.111 (0.027) *** 0.058 (0.107) LMIS performance score 0.173 (0.098) * PipeLine used Yes (No) 0.157 (0.219) 0.303 (0.123) ** Method 17.67 *** 17.64 *** 8.10 * 5.38 Oral pill (Condom) 0.535 (0.157) *** 0.539 (0.156) *** 0.594 (0.222) *** 0.616 (0.469) Injectable 0.261 (0.185) 0.270 (0.184) 0.308 (0.244) -0.257 (0.545) IUD 0.154 (0.223) 0.163 (0.222) 0.291 (0.305) 0.711 (0.627) Implant -0.071 (0.285) -0.054 (0.284) 0.330 (0.358) 0.032 (0.884) Program type Other (Public sector) -0.178 (0.137) -0.162 (0.136) -0.536 (0.548) Shipment adequacy Yes (No) 0.715 (0.122) *** 0.716 (0.121) *** 0.804 (0.172) *** 0.764 (0.380) ** Quantity projected Less (More) -0.678 (0.153) *** -0.684 (0.152) *** -0.830 (0.214) *** -0.593 (0.455) Note: Reference category for the qualitative independent variables is in parentheses. The coefficients are estimated using country- level, fixed-effects logistics regression models. Chi square statistics are reported for the qualitative predictors with more than two 11 The average Logistics Management Information System Performance Index score of the subsample is 6.8, which can potentially reach 12—resulting in 18.5 percentage-points ([12 - 6.8] × 3.5) reduction in the median absolute forecast error. 13 categories. About 12 observations from the subsample for model 4 are dropped because the values for average forecasting remained unchanged over the analysis period; *p<.1; **p<.05; ***p<.01. DISCUSSION This study examined the level, trend, and determinants of contraceptive forecasting accuracy of 81 family planning programs in 30 countries from 1995 to 2004. The variation of forecast accuracy by forecast year, PipeLine use, method, program type, and shipment adequacy is assessed using country-level, fixed-effects median regression model to minimize the sources of bias. The findings of the median regression models are validated from the findings from their analogous logistics regression models. The analysis found that forecast accuracy improved over time, primarily for public sector family planning programs. When forecast accuracy deviated from the average, the tendency of the projected use is to overestimate rather than underestimate actual consumption. It is interesting to note that forecasted estimates for oral pill are found to be the most accurate of all methods. It is widely acknowledged within the logistics and supply chain management field that accurate forecasting is one of the most important prerequisites necessary to ensure continuous availability of supplies at minimum cost. This applies to contraceptive supply as well as to industrial manufacturing. The current practice of combining two or more methods to forecast contraceptive requirements for family planning programs in developing-country setting is consistent with best practices in forecasting methods documented by Armstrong (2005) and Sanders (2005). The accuracy of the forecasting methods practiced in international family planning settings observed recently for public sector programs (about 25 percent or less pooled over the past four years) is acceptable, according to the U.S. commercial standard for one-year-ahead forecasting— indicating that best standards in forecasting contraceptives can be achieved in developing-country settings. Improvement of the forecast accuracy for contraceptive commodities is observed primarily for public sector family planning programs, which are the major clients of the USAID-supported family planning supply chain management improvement project implemented by JSI. Nevertheless, the net impact of the supply chain improvement project on forecast accuracy could not be isolated. The improvement of the forecast accuracy among the public sector family planning programs observed in this study could also be due partly to program maturity, which is independent of JSI’s supply chain improvement project. The reason for comparatively higher forecast error among non-public sector programs is probably due to the lack of use of actual consumption data for forecasting contraceptive needs. The use of forecasting software improved forecast accuracy, possibly because it allowed frequent monitoring of short-term consumption of a contraceptive, providing the means to adjust and improve the longer-term forecasts. PipeLine use is currently universal among Contraceptive Procurement Table clients; therefore, the maximum impact of PipeLine to improve forecast accuracy—which is moderate—has mostly been achieved. The improvement in LMIS performance substantially improved forecast accuracy. Nevertheless, the low average Logistics Management Information System Performance Index scores for the selected programs indicate that there is opportunity for further improvement, which would substantially improve forecast accuracy. Therefore, to optimize contraceptive financing and procurement to ensure contraceptive security, further investment to improve family planning logistics system is required. The sample for analysis of the effect of LMIS performance on forecast accuracy is not random; it depended on the availability of the Composite Indicators for Logistics Management data, limiting the external generalizability of the findings. Since the subsample analysis mainly included the public sector programs, the findings may not be generalized to other programs. Nevertheless, there is no compelling reason to believe that including more or different family planning programs would yield significantly dissimilar results. The validity of the construct measuring the performance of the LMIS may be questioned. Nevertheless, the following arguments support the index’s validity: (1) the items for measuring the construct are designed by a team of logisticians who are experienced in family planning supply chain management in developing-country 14 settings; (2) data on the items are obtained carefully to avoid bias due to group dynamics; (3) the analytic methodology—that is, the country-level, fixed-effect models—partly accounted for the remaining measurement error associated with the index; and (4) the relationship between the Logistics Management Information System Performance Index and forecast accuracy measure is in the expected direction. A few issues remained inconclusive and require further investigation. Forecasts for relatively smaller quantities of a contraceptive are associated with larger error, which is independent of method, program type, and country. Method type accounted for differentials in forecast accuracy between IUD and condom, which are also associated with quantity projected—condom requirement generally is much higher than it is for IUD or injectable. Similarly, program type and country-level effect accounted for differentials in the forecast accuracy due to program size and capacity, which are also associated with quantity required. Also, the causal relationship between shipment accuracy and forecast accuracy is not conclusive, because both indicators are measured at the same point. Further analysis showed that overestimating consumption is twice as likely when shipment is inadequate (59 percent) than when it is adequate (29 percent). If inadequate shipment is causing lower consumption, demonstrated by the overestimation of consumption, it should be monitored closely to ensure the contraceptive security of the country. 15 16 REFERENCES Armstrong, J. Scott. 2005. “The Forecasting Canon: Nine Generalizations to Improve Forecast Accuracy.” Foresight, 1(1): 29–35. Armstrong, J. Scott, and F. Collopy. 1992. “Error Measures for Generalizing about Forecasting Methods: Empirical Comparisons.” International Journal of Forecasting, 8: 69–80. Armstrong, J. Scott, and Kesten C. Green. 2005. “Demand Forecasting: Evidence-based Methods.” Working paper. Wharton University of Pennsylvania. Accessed December 19, 2005, from http://marketing.wharton.upenn.edu/ideas/pdf/Armstrong/DemandForcasting.pdf Davis, H., and C. Hart. 2002. “Increasing Program Impact by Ensuring Product Availability.” Global Health Link, 114: 7, 24. Efron, B., and R. Tibshirani. 1993. An Introduction to Bootstrap. New York: Chapman & Hill. Family Planning Logistics Management (FPLM). 2000. The Contraceptive Forecasting Handbook for Family Planning and HIV Prevention Programs. Arlington, Va.: FPLM/John Snow, Inc., for the U.S. Agency for International Development. Frazelle, Edward H. 1999. Logistics Performance Cost and Value Measures. Cleveland, Ohio: Logistics Management Center, Penton Institute. Gelfeld, Dana. 2000. Summary of Results: The Composite indicators for Contraceptive Logistics Management Under FPLM III, Working paper. Arlington, Va.: FPLM/John Snow, Inc., for the U.S. Agency for International Development. Gould, W.W. 1992. sg11.1: “Quantile Regression with Bootstrapped Standard Errors.” Stata Technical Bulletin, 9: 19–21. Hsiao, Cheng. 1986. Analysis of Panel Data. Cambridge: Cambridge University Press. Hyndman, Rob J., and Anne B. Koehler. 2006. “Another Look at Measures of Forecast Accuracy.” International Journal of Forecasting, 22(4): 679-688. John Snow, Inc. (JSI)/Family Planning Logistics Management (FPLM) and Centers for Disease Control and Prevention (CDC). 1999. Composite Indicators for Contraceptive Logistics Management. Arlington, Va.: FPLM, for the U.S. Agency for International Development. (Compiled by the EVALUATION project and John Snow, Inc./FPLM and the Centers for Disease Control and Prevention.) John Snow Inc. (JSI)/DELIVER. 2004. The Logistics Handbook: A Practical Guide for Supply Chain Managers in Family Planning and Health Programs. Arlington, Va.: JSI/DELIVER, for the U.S. Agency for International Development. John Snow, Inc. (JSI)/DELIVER and Futures Group/POLICY Project. 2003. Contraceptive Security Index 2003: A Tool for Priority Setting and Planning. Arlington, Va.: John Snow, Inc./DELIVER. Karim, A. M. 2005. “The Influence of Family Planning Logistics Systems on Contraceptive Use.” Presented at the Population Association of America annual meeting, 2005, Philadelphia, Pa. 17 Koenker, Roger. 2004. “Quantile regression for longitudinal data.” Working paper. Urbana- Champaign, Ill.: University of Illinois at Urbana-Champaign. Sanders, Nada Ed. 2005. “Special Feature: When and How Should Statistical Forecasts Be Judgmentally Adjusted?” Foresight, 1(1): 5–23. Schultz, T. Paul. 1993. “Sources of Fertility Decline in Modern Economic Growth: Is Aggregate Evidence on the Demographic Transition Credible?” Edited by M. R. Rosenzweig and O. Stark, Handbook of Population and Family Economics. _______ . 1994. “Human Capital, Family Planning, and Their Effects on Population Growth.” American Economic Review, 84(2): 255–260. Sharma, S., and V. Dayaratna. 2005. “Creating Conditions for Greater Private Sector Participation in Achieving Contraceptive Security.” Health Policy, 71(3): 347–357. StataCorp. 2005a. Stata Base Reference Manual, Volume 1, Release 9. A Stata Press Publication. College Station, Tex.: StataCorp LP. ________. 2005b. Stata Base Reference Manual, Volume 2, Release 9. A Stata Press Publication. College Station, Tex.: StataCorp LP. ________. 2005c. Stata Statistical Software: Release 9. College Station, Tex.: StataCorp LP. Wilson, Edward. 1995. “1993 Annual Accuracy Analysis of FPLM Contraceptive Forecasting. Working paper. Family Planning Logistics Management (FPLM).” Arlington, Va.: John Snow, Inc./FPLM. Wooldridge, Jeffrey M. 2003. Introductory Econometrics: A Modern Approach, 2nd Ed. Mason, Ohio: Thomson, South-Western. 18 APPENDIX 1 TRENDS AND DIFFERENTIALS Table A1: Trend in the Median Absolute Percent Forecast Error by Country (1995–2004) Region/Country Pooled 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Africa & Near East Bangladesh 33.2 28.9 122.2 36.0 27.8 48.4 35.0 (14) (0) (0) (0) (0) (2) (1) (2) (3) (3) (3) Egypt 25.2 24.5 29.7 37.8 41.6 24.6 8.5 13.8 (26) (4) (4) (4) (4) (4) (4) (2) (0) (0) (0) Jordan 17.7 40.9 28.6 22.1 21.2 11.1 5.6 14.3 (43) (0) (0) (0) (6) (7) (8) (6) (6) (5) (5) Morocco 23.8 22.2 72.3 21.9 9.1 (22) (6) (6) (5) (5) (0) (0) (0) (0) (0) (0) Nepal 17.7 16.1 45.1 27.3 15.5 17.5 4.6 20.8 12.2 (67) (10) (8) (9) (10) (10) (5) (10) (5) (0) (0) Bolivia 25.2 24.6 14.1 37.8 52.5 91.0 42.3 41.0 9.1 27.7 19.5 (69) (13) (18) (12) (2) (1) (7) (6) (3) (3) (4) Dominican Republic 42.0 42.7 42.9 40.4 36.5 (52) (13) (13) (14) (12) (0) (0) (0) (0) (0) (0) Ecuador 65.0 107.3 39.0 33.8 40.9 92.1 95.9 (37) (12) (0) (4) (0) (5) (4) (0) (6) (0) (6) El Salvador 39.5 113.2 70.9 31.1 24.0 38.5 (49) (14) (12) (0) (0) (0) (0) (5) (0) (12) (6) Guatemala 33.3 48.0 47.6 34.2 28.4 14.4 22.7 28.6 61.6 38.9 57.3 (138) (13) (13) (13) (14) (14) (19) (14) (12) (19) (7) Haiti 28.6 72.8 66.9 22.1 15.9 19.4 16.4 (37) (6) (6) (7) (0) (6) (0) (6) (6) (0) (0) Honduras 23.0 24.6 81.8 7.6 48.5 16.1 (48) (15) (7) (0) (14) (0) (0) (0) (0) (4) (8) Nicaragua 30.0 37.3 27.2 22.7 33.8 27.4 16.5 50.0 45.6 29.8 (77) (7) (0) (8) (9) (6) (11) (7) (13) (5) (11) Paraguay 39.5 40.4 50.9 39.4 (14) (0) (3) (4) (7) (0) (0) (0) (0) (0) (0) Peru 40.5 59.1 34.0 27.8 31.6 19.6 70.0 84.8 (63) (14) (14) (14) (0) (5) (0) (5) (7) (0) (4) Philippines 12.8 26.4 21.3 14.7 9.2 9.8 4.8 14.1 (27) (3) (4) (4) (4) (4) (0) (4) (0) (4) (0) Latin America and the Caribbean region 19 Continued 20 Table A1 continued … Region/Countr Poole y d 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Africa Burkina Faso 42.9 (24) (0) (0) (0) (0) (0) 117.2 (6) 141.5 (6) 52.2 (6) (0) 18.3 (6) Cameroon 47.4 199.3 76.2 17.5 19.3 49.7 (60) (0) (0) (0) (0) (0) (14) (15) (15) (8) (8) Ghana 36.7 24.4 23.0 25.0 54.0 51.5 62.5 36.4 24.0 20.7 46.2 (140) (9) (11) (9) (18) (14) (17) (18) (19) (12) (13) Guinea 45.4 56.1 43.3 93.0 37.1 (20) (0) (0) (0) (0) (5) (5) (5) (0) (5) (0) Madagascar 50.7 60.2 55.8 55.4 35.5 7.9 (35) (9) (10) (6) (5) (0) (0) (0) (0) (0) (5) Malawi 34.2 37.2 37.5 200.0 133.3 9.5 24.2 20.4 21.5 28.8 32.8 (54) (5) (5) (4) (6) (4) (6) (6) (6) (6) (6) Mali 46.4 27.3 10.8 58.2 39.3 197.5 56.4 31.5 63.9 100.0 (50) (13) (4) (6) (4) (3) (6) (2) (3) (9) (0) Mozambique 29.6 29.6 (4) (0) (0) (0) (0) (0) (0) (0) (0) (4) (0) Rwanda 21.0 21.0 (8) (0) (0) (0) (0) (0) (0) (0) (0) (0) (8) Senegal 27.4 28.6 29.6 24.7 8.3 54.9 48.0 16.3 26.5 21.8 14.1 (156) (15) (17) (19) (19) (14) (20) (9) (15) (14) (14) Tanzania 25.8 50.0 33.6 29.6 18.2 20.1 46.3 10.3 47.4 (61) (7) (6) (6) (8) (9) (9) (9) (7) Togo 35.6 61.3 33.3 22.2 39.4 (60) (0) (0) (0) (0) (0) (15) (13) (15) (17) (0) Uganda 39.2 10.9 46.9 121.9 26.2 22.4 26.4 25.3 172.7 23.7 (76) (8) (11) (9) (6) (9) (9) (10) (7) (7) Zimbabwe 21.0 18.2 19.9 11.3 21.7 34.1 22.1 35.0 17.3 21.5 (55) (7) (8) (8) (6) (6) (6) (5) (5) (4) (0) Total 30.9 34.3 36.9 29.5 29.6 26.5 38.7 24.5 26.1 29.4 30.0 (1586) (203) (169) (167) (154) (124) (163) (164) (164) (150) (128) Sample sizes are given in parenthesis; *p<.1; **p<.05; ***p<.01. 21 Table A2: Differentials in the Subsample Characteristics (1995, 1999, and 2000) Sample Selected Variable No Yes Total Forecast period 1995 41.6 41.3 41.4 1999 26.8 23.3 25.3 2000 31.7 35.4 33.3 Method Condom 23.6 21.8 22.9 Oral pill 37.0 33.5 35.5 Injectable 13.4 19.9 16.1 IUD 17.6 18.5 18.0 Implant 8.5 6.3 7.6 Program type* Public sector 41.9 73.3 55.1 Other 58.1 26.7 44.9 Adequate shipment No 34.5 32.0 33.5 Yes 65.5 68.0 66.5 Quantity projected 100,000+ 50.7 53.4 51.8 <100,000 49.3 46.6 48.2 PipeLine used* No 80.3 46.1 65.9 Yes 19.7 53.9 34.1 Median forecast error 35.6 27.6 33.8 Sample size 284 206 490 *Significant variation in the distribution of a variable by sample selection is indicated by an asterisk. 22 23 For more information, please visit deliver.jsi.com. USAID | DELIVER PROJECT John Snow, Inc. 1616 Fort Myer Drive, 11th Floor Arlington, VA 22209 USA Phone: 703-528-7474 Fax: 703-528-7480 Email: deliver_project@jsi.com Internet: deliver.jsi.com << /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /All /Binding /Left /CalGrayProfile (Dot Gain 20%) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Warning /CompatibilityLevel 1.4 /CompressObjects /Tags /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJDFFile false /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.0000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments false /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Preserve /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 300 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 300 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False /Description << /CHS <FEFF4f7f75288fd94e9b8bbe5b9a521b5efa7684002000500044004600206587686353ef901a8fc7684c976262535370673a548c002000700072006f006f00660065007200208fdb884c9ad88d2891cf62535370300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002> /CHT <FEFF4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef653ef5728684c9762537088686a5f548c002000700072006f006f00660065007200204e0a73725f979ad854c18cea7684521753706548679c300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002> /DAN <FEFF004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002000740069006c0020006b00760061006c00690074006500740073007500640073006b007200690076006e0069006e006700200065006c006c006500720020006b006f007200720065006b007400750072006c00e60073006e0069006e0067002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /DEU <FEFF00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200076006f006e002000640065006e0065006e002000530069006500200068006f00630068007700650072007400690067006500200044007200750063006b006500200061007500660020004400650073006b0074006f0070002d0044007200750063006b00650072006e00200075006e0064002000500072006f006f0066002d00470065007200e400740065006e002000650072007a0065007500670065006e0020006d00f60063006800740065006e002e002000450072007300740065006c006c007400650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000410064006f00620065002000520065006100640065007200200035002e00300020006f0064006500720020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /ESP <FEFF005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f0073002000640065002000410064006f0062006500200050004400460020007000610072006100200063006f006e00730065006700750069007200200069006d0070007200650073006900f3006e002000640065002000630061006c006900640061006400200065006e00200069006d0070007200650073006f0072006100730020006400650020006500730063007200690074006f00720069006f00200079002000680065007200720061006d00690065006e00740061007300200064006500200063006f00720072006500630063006900f3006e002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /FRA <FEFF005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f00620065002000500044004600200070006f007500720020006400650073002000e90070007200650075007600650073002000650074002000640065007300200069006d007000720065007300730069006f006e00730020006400650020006800610075007400650020007100750061006c0069007400e90020007300750072002000640065007300200069006d007000720069006d0061006e0074006500730020006400650020006200750072006500610075002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e> /ITA <FEFF005500740069006c0069007a007a006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e00740069002000410064006f006200650020005000440046002000700065007200200075006e00610020007300740061006d007000610020006400690020007100750061006c0069007400e00020007300750020007300740061006d00700061006e0074006900200065002000700072006f006f0066006500720020006400650073006b0074006f0070002e0020004900200064006f00630075006d0065006e007400690020005000440046002000630072006500610074006900200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000410064006f00620065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /JPN <FEFF9ad854c18cea51fa529b7528002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e30593002537052376642306e753b8cea3092670059279650306b4fdd306430533068304c3067304d307e3059300230c730b930af30c830c330d730d730ea30f330bf3067306e53705237307e305f306f30d730eb30fc30d57528306b9069305730663044307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e30593002> /KOR <FEFFc7740020c124c815c7440020c0acc6a9d558c5ec0020b370c2a4d06cd0d10020d504b9b0d1300020bc0f0020ad50c815ae30c5d0c11c0020ace0d488c9c8b85c0020c778c1c4d560002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e> /NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken voor kwaliteitsafdrukken op desktopprinters en proofers. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.) /NOR <FEFF004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200066006f00720020007500740073006b00720069006600740020006100760020006800f800790020006b00760061006c00690074006500740020007000e500200062006f007200640073006b0072006900760065007200200065006c006c00650072002000700072006f006f006600650072002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006500720065002e> /PTB <FEFF005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f0062006500200050004400460020007000610072006100200069006d0070007200650073007300f5006500730020006400650020007100750061006c0069006400610064006500200065006d00200069006d00700072006500730073006f0072006100730020006400650073006b0074006f00700020006500200064006900730070006f00730069007400690076006f0073002000640065002000700072006f00760061002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e> /SUO <FEFF004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f0074002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a00610020006c0061006100640075006b006100730074006100200074007900f6007000f60079007400e400740075006c006f0073007400750073007400610020006a00610020007600650064006f007300740075007300740061002000760061007200740065006e002e00200020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e> /SVE <FEFF0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740020006600f600720020006b00760061006c00690074006500740073007500740073006b0072006900660074006500720020007000e5002000760061006e006c00690067006100200073006b0072006900760061007200650020006f006300680020006600f600720020006b006f007200720065006b007400750072002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e> /ENU (Use these settings to create Adobe PDF documents for quality printing on desktop printers and proofers. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.) >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /ConvertColors /NoConversion /DestinationProfileName () /DestinationProfileSelector /NA /Downsample16BitImages true /FlattenerPreset << /PresetSelector /MediumResolution >> /FormElements false /GenerateStructure true /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /NA /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /LeaveUntagged /UseDocumentBleed false >> ] >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice

View the publication