Barriers to Implementing Contraceptive Security Policies in Romania

Publication date: 2002

POLICY Barriers to Implementing Contraceptive Security Policies in Romania By Varuni Dayaratna Imelda Zosa-Feranil Alin Stanescu Luminita Marcu Cassandra Butu January 2002 POLICY is a five-year project funded by the U.S. Agency for International Development under Contract No. HRN-C-00-00-00006-00, beginning July 7, 2000. The project is implemented by The Futures Group International in collaboration with Research Triangle Institute (RTI) and The Centre for Development and Population Activities (CEDPA). 2 Barriers to Implementing Contraceptive Security Policies in Romania Varuni Dayaratna Imelda Zosa-Feranil Alin Stanescu Luminita Marcu Cassandra Butu POLICY Project January 2002 3 Table of Contents Introduction. 1 Background . 1 Policy and Program Context. 2 Policy Implementation: An Assessment. 4 Key Findings. 6 Conclusions. 20 Annex . 1 References. 2 1 Barriers to Implementing Contraceptive Security Policies in Romania Introduction At the start of the new century, Romania faces slow economic growth, rising poverty levels, and low health status, among other challenges. However, certain positive changes have occurred, notably improvements in reproductive health (RH). The last few years witnessed more favorable policies for family planning (FP)/RH. In August 2000, the government approved policies that paved the way for its groundbreaking approach to contraceptive security. These policies defined contraceptive security in terms of government financing for contraceptive commodities, targeting free public sector contraceptives to vulnerable segments of the population, establishing revolving funds to enable judets (districts) to purchase and sell contraceptives locally to nontargeted clients, and ensuring access especially in rural areas. The concepts and policies underlying this approach to contraceptive security, and the structures and operational program guidelines that have been put in place to support the initiatives, are new to Romania. As such, it would be unrealistic to expect that this complex initiative could be implemented without experiencing problems and setbacks. Identifying and understanding such problems constitutes a first step toward addressing them and improving the program. With this in mind, the Ministry of Health and Family (MOHF), in collaboration with the POLICY Project, undertook an assessment in March–May 2001 of how well the new national contraceptive security policies and laws were being translated into action at the service delivery level. The main objective of the analysis was to identify and inform policymakers about • Barriers to the new policies and other issues that hinder the achievement of contraceptive security; and • Underlying policies, regulations, and norms that are at the root of these operational barriers. The information is intended to serve as the basis for multisectoral dialogue and discussion directed at developing policy recommendations to reduce operational barriers and facilitate the achievement of contraceptive security. Results of the assessment are presented in this paper. The paper starts with a socioeconomic and RH background and policy and program context. It then highlights operational constraints and other findings from the assessment using questions aimed at stimulating dialogue on policy barriers and their implications. These discussions are followed by policy options that might be considered in addressing the various operational barriers. The concluding section summarizes the key policy concerns and identifies what the authors believe are the most critical issues and policy options. Background Socioeconomic Difficulties After decades of economic inefficiency during the socialist regime, Romania’s restructuring following the 1989 revolution proved extremely difficult. The economy experienced negative growth rates during the 1990–1999 decade (World Bank, 2001). In 2000, more than 40 percent of Romania’s population of 22 million lived in poverty.1 Romania ranked as one of Central and Eastern European countries with the highest magnitude and severity of poverty (Tesliuc and Pop, 2000). Romania has programs aimed at alleviating poverty: social insurance (e.g., unemployment, pension, maternity, and sickness benefits); entitlements (e.g., child allowances); and social 1 The government’s White Book report uses 44 percent for 2000, while the Research Institute for Quality of Life (RIQL) estimated poverty at 41 percent in 1999. 2 assistance for people in need (however, coverage is low). In 1998, only 7 percent whose consumption was below the poverty threshold benefited from social assistance programs (Tesliuc and Pop, 2000). Reproductive Health Romania’s RH problems are interrelated. The country’s maternal mortality ratio (41) ranked among the highest in Europe during the 1990s, and was four times higher than ratios in Italy and France. The highest maternal mortality ratios in Romania were among women with less than a secondary education. Even toward the end of the decade, one-half of all maternal deaths were due to abortion complications (Ghetau, 1997). The lifetime abortion rate per woman was 3.4 in 1993, but declined to 2.2 in 1999.2 In 1999, two-thirds (64%) of women in union used contraception, and women on average had only 1.3 children. Modern contraceptive prevalence, however, was only 30 percent. Four modern methods dominated: condoms, pills, IUDs, and injectables. Modern method use was lowest among women living in rural areas, who had three or more children, less than secondary education, or low socioeconomic status. Traditional method prevalence was higher (34%). Abstinence and withdrawal, however, registered the highest failure rates for both years. 3 In 1999, 60 percent of all women in union desired no more children. Still, most unwanted pregnancies end in abortion, which persists as the population’s primary method of controlling fertility. Unmet need for modern methods approximated 30 percent, or nearly 1.5 million women aged 15–44.4 More than 80 percent with unmet need belonged to low and middle socioeconomic groups. Consequently, these groups accounted for almost 85 percent of induced abortions reported between 1994 and 1999. Policy and Program Context A repressive pronatalist policy that banned modern contraception until December 1989 gave rise to a heavy reliance on abortion as the primary method of controlling fertility. Reliance on abortion persisted even after the policy was repealed following the 1989 revolution. To ensure an alternative to abortion, the Ministry of Health (MOH), beginning in the early 1990s, established 241 urban-based FP clinics.5 These clinics provided FP service provision by general practitioners (GPs), a marked shift away from relying solely on Ob–Gyns. However, the GPs were still required to undergo substantial training. GPs at the FP clinics became the primary providers of contraceptives procured by the MOH using funds from a World Bank loan. These contraceptives were procured by UNFPA through international bidding and were sold at low prices to clients.6 Although the government, through the MOH, initiated and organized a family planning services network since the early 1990s, Romania did not have an explicit FP/RH policy until the late 1990s. Only during the last few years, with health reform underway, did Romania approve a number of policies aimed at increasing access to FP/RH. • In 1997, the Romanian Parliament approved the Social Health Insurance Law that gave the National Health Insurance House (NHIH) direct control over the financing of health services throughout the country (Law 145/197). • In 1999, the government approved the benefits package for health insurance implementation (Order 143/792). 2 1993 and 1999 Romania Reproductive Health Surveys (RRHS). 3 Data for 1999 from the 1999 RRHS. 4 The Centers for Disease Control (CDC) definition of unmet need includes traditional users and nonusers at risk of an unintended pregnancy. This is used in several Eastern European countries. 5 NGOs and pharmacies are two other major sources of FP information, education, and communication (IEC), services, and commodities in Romania. Please refer to Sharma et al. (2001) for a discussion on FP service providers. 6 Population Services International (PSI) estimated low-priced contraceptives to be about 10,000 Romanian lei (ROL) (or about $1) in 1998. 3 • In 2000, revised health insurance norms made the FP clinics part of the health insurance system, and gave approval for family doctors at the primary care level to provide basic FP services (i.e., counseling, consultation, and prescription) “without risk.” The policy eliminated stringent competency requirements for FP provision by family doctors under the previous norms (Order 921/1.765).7 Policy champions from the public sector and the NGOs of the Reproductive Health Coalition contributed significantly to the foregoing policy changes affecting FP/RH services.8 They also played key roles in government approval of policies toward contraceptive security. Under health reform, the MOH retained authority over policies, standards, planning, program development, and evaluation. The MOH also coordinates and oversees national health programs that include health promotion, maternal and child health (MCH), and FP. Predominantly preventive, these national programs include commodity support to public health-related activities of service providers under the social health insurance system, being the primary source of health care in the country. Public sector contraceptive supplies, earlier procured under the last tranche of the World Bank loan, were reported to be dwindling by late 1998, resulting in stockouts at UNIFARM, the national drug procurement agency (MSH, 1999). Certain FP clinics resorted to purchasing FP commodities from private suppliers. Advocacy and policy dialogue between the government and other stakeholders pointed to the need for policy action to avert a crisis.9 A high-level Consultative Committee on RH Financing highlighted the low levels of government spending for FP compared to curative RH services like abortion.10 The RH coalition also conducted women’s health forums, fairs, and caravans in 1998 and 1999 to generate political and popular support for access to RH services and low- cost contraceptives, especially in rural areas. Severe state budget cutbacks, resulting in reduced funding for health programs, precipitated the recognition among FP/RH decision makers of the need to target scarce public sector contraceptives only to disadvantaged sectors of the population. In August 2000, the prime minister and ministers of Health and Minister of Finance signed a government order that authorized, among other actions, state budget allocations for contraceptives and approved the following policies (Order No. 730): 1. Distribution of free contraceptives to selected social categories: unemployed women, students, persons under social protection, and those with minimum or without income. 2. Establishment of revolving funds for the local purchase of contraceptives by hospitals for sale to nontargeted sectors of the population. The proceeds of these sales are to be deposited in revolving funds, for future contraceptive purchases. 3. Distribution of contraceptives by family doctors in rural areas without FP clinics. 7 However, health insurance covers only services, not commodities. 8 These advocacy events were conducted with assistance from POLICY. 9 USAID through POLICY supported research, advocacy, and policy dialogue to ensure resources for contraceptives. 10 POLICY supported the analysis conducted by Marinescu and Cakir (2000) as well as the attendance of the high- level Committee to the RH Finance Conference held in Austria in June 2000. 4 To guide local government agencies and service providers in implementing the Government Order, the MOH developed and disseminated technical norms that expounded on the two mechanisms for financing, procuring and distributing contraceptives (MOH, 2000). These norms identified the national and local entities that would be involved in the government’s contraceptive security initiative. Policy Implementation: An Assessment Contraceptive security initiatives in Romania are relatively new and problems have emerged during initial implementation. The next section (Key Findings) discusses several barriers to implementation, which came to light during the assessment conducted in March–May 2001. Methodology. The assessment used various data sources: data provided by the MOHF and Panait Sarbu National FP Reference Center; questionnaires administered to respondents from local health authorities, hospital finance offices, FP center/clinic staff, and family doctors; and exit interviews of clients at FP centers/clinics. The interviews with health personnel and clients were conducted in Cluj, Iasi, Constanta, and Mures judets, which rank among the largest districts in Romania. Thirteen percent of women of reproductive age live in these four judets. Service providers included those belonging to 16 FP centers/clinics and two trained family doctors who are providing free MOH-funded contraceptives. Exit interviews were conducted with 95 women who obtained contraceptives from these FP providers. Data from the Institute for National Statistics (2000) were used to describe the socioeconomic characteristics of judets. The assessment also makes use of the two other complementary studies: Sharma et al.’s (2001) contraceptive market segmentation analysis and Erhan’s (2001) review of public sector financing for contraceptives. Flow of Funds and Commodities The new contraceptive security policies have given rise to two financing and commodity flows (Figure 1) that originate within the MOHF. Free Contraceptives The first of these flows pertains to free contraceptives. Panait Sarbu National FP Reference Center, located in Bucharest, centrally procures contraceptives for free distribution, using money set aside by the MOHF. The centrally procured contraceptives are then delivered to each of the 41 judets or districts. Either Panait Sarbu or the primary FP clinic in each judet receives the contraceptives, which are then distributed to individual FP clinics and/or family doctors. These providers in turn make the contraceptives available free-of-charge to eligible clients. Locally Purchased Contraceptives The second flow of funds and commodities depicted in the diagram applies to locally purchased contraceptives. The MOHF releases special funds earmarked for the local purchase of contraceptives to Local Health Authorities (LHA) in each judet. These funds are in turn distributed to individual hospitals within the judet to which FP clinics are attached. The hospital serves as a repository and handler of the special funds allocated to the judet because FP clinics are prohibited by law to hold money. Each hospital, usually in conjunction with the FP clinic, uses the funds to purchase contraceptives from local suppliers. The FP clinic then sells the locally purchased contraceptives to those clients who are not eligible for free contraceptives. The proceeds from these sales are returned to the hospital to replenish its revolving fund for contraceptives. 5 Figure 1 Flow of Resources for Public Sector–Subsidized Contraceptive Commodities Funds to judets Free supplies to FP clinics/providers Commodities to FP clinics Targeted clients MOHF State budget Central purchase Local hospitals (purchase) Nontargeted clients FP centers 6 Key Findings This section discusses several barriers in implementing Romania’s contraceptive security policies. They pertain to the adequacy of resources, the efficiency of procurement, the effectiveness of targeting policies, the sustainability of the revolving fund, and access in rural areas. We approach each barrier by posing a question; for example, “Are the supplies of free contraceptives sufficient to meet the needs of target groups?” The evidence and discussion that follow provide answers to each question and specify barriers that affect policy implementation. Each discussion ends with a list of policy options that might be considered in addressing the problem or barrier at hand. These options are drawn from discussions with FP/RH stakeholders in Romania as well as from studies and lessons learned on the experiences in other countries. The policy options are not mutually exclusive and they are not ranked in order of importance. Many of these options may well be combined in order to address specific barriers more fully. Free Contraceptives 1. Are the supplies of free contraceptives sufficient to meet the needs of targeted clients? National Perspective In 2000, the MOHF allocated $122,00011 to Panait Sarbu for central contraceptive procurement. This allocation was based on MSH forecasts prepared in 1999 on contraceptive requirements based on 1993 prevalence rates. Using a portion of these funds, Panait Sarbu purchased a total of 94,480 cycles of pills (Microgynon, Triquilar, and Marvelon) between August and the end of the year.12 Instead of a single large procurement, several purchases were made during the five-month period, with 499 million ROL, or $22,000, as the largest purchase,13 due to the more bureaucratic approval requirements associated with purchases of or above 500 million ROL or financial procedures involving funds from the state budget (Erhan, 2001). Large procurements are also not possible at present because government finance regulations involve monthly budgetary approval and fund releases (Erhan, 2001). These factors in turn contribute to the lack of continuity in contraceptive supplies and raise questions on the government’s ability to reach targeted populations. In 2001, the MOHF allocated $137,000, which was in turn used by Panait Sarbu to procure free contraceptives in the next seven months (January 1–July 31, 2001).14 This money was used to purchase 130,993 cycles of pills, which included three pill brands (Microgynon, Triquilar, and Rigevidon) and 143 boxes of injectables (Megestrone). Data from a recent market segmentation analysis (Sharma et al., 2001) can be used to demonstrate that contraceptives available for free distribution are insufficient to meet the needs of various target groups. One such target group comprises the poor, designated by the government to comprise the lowest 40 percent of the population. As Table 1 shows, more than 2 million women between ages 15–44 belong to the lowest 40 percent of socioeconomic groupings. Of these women, 297,000, or 15 percent, currently use modern methods of contraception—more than 121,000 (41%) use pills, 69,000 (23%) use IUDs, and 58,000 (19%) use condoms, while the rest use other modern methods such as injectables.15 11 Equivalent to 2.8 billion ROL (using an exchange rate of 23,000 ROL to $1). Note: the total value of Panait Sarbu’s purchase was 2 billion ROL. It is not clear how the balance of 800 million ROL was used. 12 Part of the $122,000 was used for management and administrative costs. 13 At an exchange rate of 23,000 ROL to $1. 14 Equivalent to 4 billion ROL (using an exchange rate of 29,160 ROL to $1). A certain portion, however, was used for administrative and management costs. 15 Estimated using information on contraceptive prevalence rates and method mix from the 1999 RRHS. 7 Table 1 Estimated Number of women 15–44 Using Modern Methods, or with Unmet Need by Specific Socioeconomic Group?16 Socioeconomic grouping Modern contraceptive methods Estimated number of current users of modern methods Estimated number with unmet need Lowest 40 percent All modern 297,000 798,000 (rural poor, urban poor, Pill 121,000 lower middle) IUD 69,000 2,040,000 women Condom 58,000 Other modern 48,000 Middle 20 percent All modern 183,600 337,000 1,020,000 Pill 84,000 IUD 40,000 Condom 40,000 Other modern 18,000 Top 40 percent All modern 651,000 543,000 (wealthy rural, upper Pill 169,000 middle urban, and urban IUD 127,000 rich) Condom 173,000 2,040,000 women Other modern 182,000 Total, all groups All modern 1,132,000 1,678,000 Pill 375,000 IUD 237,000 Condom 272,000 Other modern 248,000 Source: Estimates based on Sharma et al. (2001) based on the 1999 RRHS. From August 2000–July 2001, the MOHF procured 225,473 pill cycles. Assuming 12 cycles per woman for this 12-month period, these cycles would have been sufficient to meet the needs of only 18,789 poor women—or only a little more than 15 percent of the estimated 121,000 current pill users among the poor.17 This is an optimistic scenario, which assumes that there is no wastage and loss and that the free supplies are perfectly targeted only to the poor. If the reference is narrowed only to the 30,000 poor pill users who report using the public sector for pill supplies, 18 potential coverage would have been 62 percent.19 These figures do not even take into account potential 16 See the Annex for a more detailed table (Table A1) with disaggregated data for seven socioeconomic groups (rural poor, urban poor, lower middle, middle, wealth rural, upper middle urban, and urban rich), as calculated by Sharma et. al (2001). 17 Typically, in well-established logistics management systems, contraceptives are procured for 12–15 months, to avoid the delays, lags, and extra expenses that can results from more short-term procurements. Such 12-month procurements are considered standard practice. In Romania, however, central procurement of contraceptives by the Panait Sarbu has been occurring on a monthly basis. To avoid bureaucratic approval requirements, the value of a single procurement never exceeded 500 million ROL (or $22,000). 18 Estimated through the POLICY market segmentation analysis using the 1999 RRHS data. 19 Some students fall into the low-income categories, and thus part of the targeted poor population. Other students, however, belong to higher income categories. Efforts were made to use the 1999 RRHS to estimate the student population (or develop some proxy measure) in order to determine the contraceptive market segmentation among students. Unfortunately, this was not possible with the existing data. Some estimation of contraceptive behavior among the student population is one major issue for future policy research. 8 demand among nearly 800,000 poor women with unmet need, or demand among other target groups who are also eligible for free contraceptives. A picture of excess demand emerges for the student population as well.20 In 2000, female students alone numbered around 750,000.21 Even if all free contraceptives in 2000–2001 had been distributed to students using pills, the 225,473 cycles would only have met a fraction of their demand. Judet-level Response to Inadequate Supplies of Free Contraceptives At the judet-level, between September 2000 and March 2001, FP reference centers received three shipments of free contraceptives from Panait Sarbu. Table 2 reports on the actual magnitude of each shipment. Following each shipment, contraceptives were distributed to FP clinics. In Cluj, the 620 pill supplies were distributed equally among 11 doctors: one each at eight FP clinics and three at the reference center. Hence, in October/November 2000, each FP clinic received 56 cycles of each pill type for a two-month period. The reference center received three times this quantity. Staff at the Cluj FP reference center claimed that the free contraceptives were sufficient only to meet 10 percent of the needs of target clientele. The FP clinic at the Cluj Student Hospital, where all clients, per Romania’s contraceptive security policies, are eligible for free contraceptives, stocked out of free contraceptives in a matter of days. In order to make supplies last longer and spread the benefit of the free contraceptives, personnel at both the student clinic and the Spitatul Municipal Clinic opted to give each eligible client just one, rather than three, cycle of pills. The experiences in other FP clinics in Cluj, Iasi, Constanta, and Mures were similar, however with some variations. In general, FP doctors and nurses claimed that the contraceptives they received from Panait Sarbu were insufficient to meet demand. In Iasi, however, where the FP reference center received a shipment of free contraceptives from USAID, the lack of contraceptives was less apparent. Health personnel at the Elena Doamna Reference Center in Iasi did state, however, that they would have stocked out of free contraceptives had USAID supplements been unavailable. In Mures, when stocks of centrally procured contraceptives ran out, health authorities opted to use the remaining money from a UNIFARM revolving fund to locally purchase contraceptives and provide them free-of-charge to eligible clients.22 This, however, was a one-time solution. 20 This includes students from high schools, vocational apprenticeship and post-high schools, and higher education institutions. 21 Estimated using data from the Institute for National Statistics (2000). 22 UNIFARM was the central procurement agency that handled contraceptives through UNFPA using funds from the World Bank health loan. 9 Table 2 Contraceptive Shipments Received by Each FP Reference Center from Panait Sarbu 2000 2001 Judet and Contraceptive type S O N D J F M Total (7 months) Cluj Microgynon pill 620 303 170 1,093 Triquilar pill 620 303 186 Megestrone inj. 800 800 Constanta Microgynon pill 620 303 170 1,093 Triquilar pill 620 303 186 1,109 Megestrone inj. 100 100 Iasi Microgynon pill 620 303 300 1,223 Triquilar pill 303 400 703 Megestrone inj. 800 800 Mures Microgynon pill 620 303 300 1,223 Triquilar pill 620 303 400 1,323 Note: Iasi also reported receiving Megestrone in April. This is not included in the table because information from the other areas was obtained only for the September–March period. Policy options to improve the supply of free contraceptives to targeted clients: ™ Increase MOHF resources available for free contraceptives. ™ Reexamine target groups with an eye toward scaling back the range of potential recipients of free contraceptives to include only the neediest groups. ™ Negotiate with the NHIH for the inclusion of basic/generic contraceptive brands in the list of compensated drugs to help ensure access for insured low-income clients. ™ Exempt the central procurement organization from budgetary ceilings to enable it to purchase large numbers of contraceptives for free distribution. Bulk purchases generally tend to be less expensive. ™ In order to ensure that available resources stretch further, procure low-cost WHO/UNFPA certified generic brands instead of more expensive name brands.23 ™ Procure cheap, but reliable, products through a social marketing agency or negotiate with other international organizations like International Planned Parenthood Federation (IPPF), which procures contraceptives for Society for Education, Contraception, and Sexuality (SECS), the local IPPF affiliate. This, however, is not a long-term solution. 2. Does the Panait Sarbu’s brand procurement policy maximize limited resources? In 2000, Panait Sarbu purchased three oral contraceptive brands from Schering and Organon for free distribution: Triquilar, Microgynon, and Marvelon. Both Microgynon and Triquilar were each priced at 20,112 ROL (or $0.87) per cycle.24 Less expensive alternatives to these two brands included their generic formularies (that at present are not registered in Romania) and low-priced UNFPA-approved brands such as Rigevidon, both of which can be purchased for about one-third of the Triquilar and Microgynon cost. 23 For example, Microgynon’s generic formulary can be purchased at about one-third of the brand name price. 24 At 23,000 ROL to $1. 10 For example, in September 2000, Panait Sarbu used approximately 500 million ROL ($22,727) to purchase nearly 25,000 cycles of Microgynon pills. If instead Panait Sarbu had used the same amount of money to purchase Rigevidon, which sells for as low as $0.25–0.30 when purchased in bulk, it could have tripled the total quantity of free contraceptives available for distribution and, thereby, reached more people in target groups. This analysis also holds true for generic formularies. Policy options to optimize use of Panait Sarbu’s resources for procurement of contraceptives: ™ Explore less expensive but reliable brand name alternatives to Triqular and Microgynon in order to serve larger segments of the target population. ™ Negotiate with pharmaceutical companies for generic formularies of Triquilar and Microgynon. ™ Procure less expensive, but reliable, products through a social marketing agency or negotiate with other international organizations, such as IPPF, which procures contraceptives for SECS, the local IPPF affiliate. ™ Ensure an open bidding process for central procurement. 3. Does the allocation of free contraceptives across and within judets reflect local needs? The distribution of free contraceptives across judets by Panait Sarbu does not seem to be based on the socioeconomic conditions and needs within each judet. For example, for September/October 2000, Cluj, Constanta, and Mures each received 1,240 pill cycles for free distribution. Iasi received only 620 cycles (see Table 2). However, as Table 3 shows, the poverty and abortion rates, as well as the student population, in Iasi far exceeded those in the other three judets.25 The criteria on which free contraceptives are distributed to FP clinics within each locale were also not need-based. In Cluj, following the example set by the Panait Sarbu, the FP reference center distributed the free contraceptives equally to FP doctors within the judet. In Mures, unlike in the other judets, the Local Health Authority (LHA) is involved with the reference center in determining the quantities of contraceptives that will be sent to each FP clinic. The decision, however, is not based on need because, in the words of the LHA Inspector General, the small supply of free contraceptives does not allow the luxury of need-based allocation. The larger reference center at Targu Mures received more contraceptives, and the remainder was divided equally among the judet’s four FP clinics. Constanta received as many contraceptives as the other judets, even though for several months, two of the three FP clinics within the judet were either closed or without FP providers. 25 USAID donated contraceptives recently became available in Iasi for free distribution. 11 Table 3 Socioeconomic Characteristics in Four Judets Judet Women ages 15–44 Abortion rate per 1000 females 15–44, 1998 Estimated number of female students (high school and university), 200026 Subregional poverty rate 1997 (%)27 Iasi 184,722 81.7 43,197 41 Constanta 179,454 34.4 26,827 33 Cluj 160,793 42.2 41,109 28 Mures 132,523 52.6 15,615 32 Note: All data (except on poverty) are from the Institute for National Statistics (2000). Policy options for need-based allocation of free contraceptives at judet-level: ™ Focus distribution of free contraceptives only in specific areas, for example, in rural areas where most of the poor reside or economically depressed judets or regions. ™ Develop allocation criteria that are based on magnitude of unmet need, general socioeconomic characteristics and concentration of poverty groups within each judet. Such a strategy would only be viable when supplies are available in quantities sufficient to meet need. 4. Are service providers effectively targeting free contraceptives to eligible groups? According to service providers, a client has to provide documentation verifying eligibility before receiving free contraceptives. Eligible categories include students, the unemployed, people with low income or no income, and those receiving social protection allowance. With few exceptions, service providers in all four judets stated that a client claiming eligibility, however bearing no proof of that status, would not receive free contraceptives. The data available from exit interviews paint a different picture for income-based targeting. Forty-four of the 96 clients (46%) interviewed in 18 FP service delivery points categorized themselves as having no income, or a level of income below minimum wage. Service providers asked only 16 of these 44 clients for verification of income status. More disturbing, however, is the finding that only 13, or 30 percent, of these clients with low or no income received free contraceptives. The rest paid for their pills. Fifty percent of low-income clients who paid for contraceptives were not even asked for documents about income status by the service provider. There are three possible explanations for this: (1) service providers are unaware of the targeting policy; (2) service providers are simply not making an attempt to target free contraceptives to low-income groups; or (3) when free contraceptives have stocked out, service providers opt not to ask clients for eligibility verification because there are no free products to distribute.28 This last explanation is the most probable. Low-income and unemployed clients may be unable to verify their eligibility status. Anecdotal evidence suggests that certification for income- and unemployment-based eligibility is difficult to obtain. The requisite papers must be obtained from the mayor’s office and include the mayor’s stamp and signature. Conversely, all students possess identification cards that verify their status as students and, hence, are able to obtain free contraceptives more easily while supplies last. Figure 2 shows the breakdown of women who received free contraceptives by eligible category for several FP clinics. 26 The estimated numbers of female students at the university level in each judet was assumed to be one-half of the reported total number of university students (the university data were not disaggregated by sex). These figures may be underestimated, as national figures reported more female than male university students. 27 Poverty rates are from UNDP (1999). 28 These explanations were not probed in the interviews with providers. 12 Figure 2 Students Are the Primary Recipients of Free Contraceptives 0 20 40 60 80 100 Clu j R ef Ct r Tu rda , C luj Efo rie N ord , C on t. Sig his oa ra, M ure s Re f C tr, Mu res Co po u M ed . C tr, Ias i Tg . F rum os , Ia si Student Low-income/unemployed With the exception of two clinics, Turda in Cluj and Tg. Frumos in Iasi, the other five clinics depicted in the graph distribute more than 60 percent of their stock of free contraceptives to students. These service statistics suggest that students, regardless of income status, benefit disproportionately from the current targeting strategy, whereas a much smaller proportion of the free contraceptives go to low-income and unemployed clients. Abortions in Romania, which are closely related to unmet need, are concentrated among low-income women, particularly in rural areas and those over 20 years of age. Many students do not fall into these categories. Although students are the predominant recipients of free contraceptives, data from the exit interviews show that even students often must pay for their contraceptives. Seventy-five percent of the 27 students who were interviewed actually paid for their contraceptives, even though all of them were able to verify their student status. Again, the reasons could be varied. 13 Policy options for effective targeting to eligible clients: ™ Ensure that service providers at all levels are aware of and adhere to targeting policies and criteria. ™ Examine the option of income-based targeting within the student population. ™ Address the discrepancy between availability of free contraceptives and the demand among target groups by either increasing resources for free contraceptives or scaling back the targeting strategy. Resources may be increased at the national and/or local level. 5. Should government monies be spent on procuring methods other than oral contraceptives? Currently, MOHF funds for free contraceptives are devoted primarily to purchasing oral contraceptives. In Romania, 28 percent of women who use modern methods use pills, 33 percent use condoms, and 21 percent use IUDs (1999 RRHS). Table 1 above shows that an estimated 116,000 poor women currently use IUDs. Given that government resources available for contraceptive commodities in Romania are scarce, it is reasonable to argue that they should be spent on the most cost-effective methods. One indicator used to measure the relative cost-effectiveness of alternative contraceptive methods is couple-years of protection (CYP). This concept helps compare the cost-effectiveness of, for example, condoms versus IUDs by showing the cost of providing one year of protection via each method. As Figure 3 shows, IUDs are among the most cost-effective methods of contraception.29 Other methods, a little more costly, do more to help women space births. Oral contraceptives are among the most cost-effective supply methods (Dayaratna et. al., 2000). Figure 3 Cost per CYP, for Selected Contraceptive Methods, US$ $0 $5 $10 $15 $20 $25 IUD Pill Condom Source: Stover and Heaton (1997). On the basis of cost-effectiveness, the purchase of IUDs and oral contraceptives would constitute a good use of government resources. In most cases, IUDs eliminate the need for frequent visits to health 29 As labor costs increase, IUD costs may exceed the cost of pills. Additionally, the relative cost-effectiveness of IUDs decreases if the method is used for less than eight years. 14 facilities, which is a great benefit for rural residents who generally must travel far to access health care. At present, however, the government of Romania does not provide free IUDs to eligible clientele. Although, condoms constitute a less cost-effective supply method of contraception than pills, the government’s decision whether to include condoms in the set of contraceptive provided free-of-charge needs to take into consideration the state of the HIV/AIDS pandemic and the prevalence of sexually transmitted diseases in Romania. Policy options to expand procurement of contraceptives beyond pills: ™ Revise central procurement strategy to include IUDs. ™ Identify and procure other modern contraceptive methods that would be cost-effective in Romania. ™ Identify and implement a cost-effective method mix that responds to client needs. 6. Are family doctors receiving free contraceptives for their clients? Despite the existence of a law that allows family doctors to distribute contraceptives, and a program of basic training that increases their competency, family doctors generally do not receive contraceptives for free distribution. Health personnel at FP reference centers in the four judets stated that they only distributed free contraceptives from Panait Sarbu to FP clinics. Reasons for not sending free contraceptives to family doctors varied, including insufficient contraceptives, resistance to the new law because of the perception that family doctors are not sufficiently competent to be given the responsibility, and uncertainty about the legality of the practice. Most family doctors lack training in FP counseling and service delivery. To date, this training, which certifies them competent to fulfill the tasks associated with the distribution of free contraceptives, is available only to family doctors in USAID and UNFPA pilot areas, constituting only about 10 percent of Romania’s family doctors. Some providers expressed concern that the financial law prohibits government entities, such as hospitals and FP centers from distributing drugs, including contraceptives, to private family doctors. Such a law would run counter to the more recent contraceptive security policy that allows family doctors to distribute free contraceptives, especially in rural areas. Despite the alleged existence of prohibitions in the financial law, it is important to note that government entities do distribute drugs for rickets and anemia to private family doctors. Hence, the legal framework, the policy initiatives, and actual practice are at odds with each other. It is also important to note that there are already practices and systems in place that allow family doctors to be involved in the distribution of free contraceptives. In Iasi and Constanta, family doctors who have been trained in FP received free contraceptives for distribution to their FP clients. These were contraceptives donated directly by USAID and not those purchased with MOH funds. Such practices could be replicated in other sites once legal restrictions, training issues, and supply shortfalls are addressed. 15 Policy options to ensure that family doctors receive free contraceptives for their eligible clients: ™ Revise existing legislation, including the financial law, to permit the distribution of essential drugs, particularly contraceptive, by family doctors. ™ Provide necessary training on family planning counseling and service delivery to all family doctors willing to distribute free contraceptives. ™ Delineate roles and relationships among family doctors, FP clinic staff and Ob–Gyns in FP service, and commodity provision. ™ Advocate to personnel at FP centers/clinics about programs to improve family doctors’ competency and how FP centers/clinics and family doctors can be linked together to ensure access and quality of care. Locally Procured Contraceptives 1. Are judets using special MOH funds efficiently? Trade-off Between Providing High-end Brands and Serving Low-income Clients Judet hospitals and FP clinics use the limited money they receive from the MOH and their own revolving funds to purchase a range of different pill types that fall into a wide spectrum of prices. This is a historically accepted practice and is justified on the grounds that it ensures client choice. In some cases, a clinic will order as many as 10 different brands of pills, but as few as five cycles of some brands. The prices of these brands range from less than $1 to more than $6 per cycle. Table 4 demonstrates local purchase patterns for FP clinics in Constanta, Mures, Iasi, and Cluj. Table 4 Brands/Quantities of Pills Purchased in February–March 2001 (a) Sighisoara–Mures (b) Eforie Nord–Constanta Brand name Price in ROL30 Quantity Brand name Price in ROL Quantity Climen 197,675 ($6.80) 5 Diane 35 162,894 ($5.60) 5 Diane 35 154,791 ($5.32) 10 Logest 125,566 ($4.32) 10 Femoden 139,868 ($4.81) 5 Femoden 119,521 ($4.11) 5 Cycloprogynova 136,842 ($4.71) 5 Novynette 83,621 ($2.88) 5 Logest 135,666 ($4.66) 50 Desorelle 66,739 ($2.29) 5 Mercilon 125,986 ($4.33) 50 Microgynon 57,224 ($1.97) 20 Novynette 109,382 ($3.76) 5 Rigevidon 29,152 ($1.00) 50 Triregol 39,288 ($1.35) 62 Postinor 26,332 ($0.91) 5 30 Prices include value-added tax (VAT). 16 (c) Pascani–Iasi (d) Cluj Brand name Price in ROL31 Quantity Brand name Price in ROL32 Quantity Desorelle 71,000 ($2.63) 10 Diane 35 162,900 ($6.03) 10 Triquilar 53,491 ($1.98) 60 Cycloprogynova 136,800 ($5.07) 20 Triquilar 53,491 ($1.98) 40 Cycloprogynon 134,600 ($4.99) 10 Microgynon 52,860 ($1.96) 60 Mercilon 117,800 ($4.36) 10 Microgynon 51,000 ($1.89) 30 Mercilon 117,800 ($4.36) 5 Novynette 109,000 ($4.04) 10 Marvelon 88,300 ($3.27) 30 Microgynon 62,800 ($2.33) 50 Microgynon 63,500 ($2.35) 50 Triregol 37,200 ($1.38) 210 Rigevidon 28,000 ($1.04) 400 If these clinics opt to use the money in their revolving fund to purchase only low-priced contraceptives (such as Triregol, Rigevidon, Anteovin, and Excluton) or a combination of low- priced formularies and one popular middle-priced brand, such as Microgynon, instead of the range they currently purchase, they would be able to purchase larger quantities of contraceptives and thereby serve more clients in need. For example, as Table 5 demonstrates, the municipal hospital in Constanta could serve more than three times as many clients if it purchased only Rigevidon ($1 a cycle), rather than 11 different brands that range in price from 28,583 to 162,895 ROL ($1 and $6). Alternatively, in the interest of client choice, if the hospital used its revolving fund to purchase Rigevidon and a more expensive popular brand such as Micrigynon, it would still be able to serve twice as many clients. 31 Prices include VAT and are based on an exchange rate of 27,000 ROL to $1. 32 Prices include VAT and are based on an exchange rate of 27,000 ROL to $1. 17 Table 5 Number of Pill Brands, Price Range, Actual, and Estimated Number of Cycles for Selected Clinics in Four Judets Judet/Clinic # of brands Price range in ROL Actual number of pill cycles # of pill cycles if only the cheapest brands were purchased # of pill cycles if funds were divided equally between Microgynon/ Triregol/Desorelle and cheapest brand purchased Cluj Judet Reference Ctr 10 28,000-162,000 1,463 3,256 2,358* Diagnostic & Treatment 9 23,430-176,755 1,113 2,857 1,961++ Turda MH 7 28,000-194,000 134 424 286 + Student Hospital 11 28,031-146,360 1,103 1,844 1,361* Campia Turzii TH 8 37,200-162,900 805 1,306 941* Constanta Municipal Hospital 11 28,583-162,895 375 1,274 927* Eforie Nord 8 26,322-162,894 165 299 218* Iasi Elena Doamna 3 67,587-173,320 94 191 327 * Pascani 3 51,000-71,000 200 207 296* Mures Sighisoara 8 39,288-197,675 239 668 803 ** +Desorelle and Rigevidon ++Anteovin and Microgynon *Microgynon and Rigevidon **Rigevidon and Triregol It might be argued that clients who are willing to pay as much as $5 or $6 for these brands could be just as well served in the private sector, thus enabling the public sector to serve a greater number of clients who do not belong to target groups and cannot afford to pay pharmacy prices for their contraceptives. Pharmacy prices tend to generally be 25–30 percent more than FP clinic prices. Policy options to improve the efficiency of local brand procurement: ™ Use the revolving funds to purchase only low-priced contraceptives while relying on nongovernment sources including pharmacies to provide a wider array of choices to suit the needs of those who can afford to pay. This entails developing a collaborative effort among the government, NGO and commercial sectors in meeting total contraceptive needs. ™ Use a large portion of the revolving funds to purchase low-priced contraceptives and the remainder for higher priced commodities to ensure choice within public facilities for those who can afford to pay. ™ Create awareness among local procurement managers and decision makers about the problems created by the practice of buying too many pill types and expensive brands. ™ Assess the feasibility of service providers receiving the more expensive contraceptive brands on a consignment basis from commercial sources to ensure choice. ™ Provide local procurement authorities with a list of the lowest prevailing prices of various contraceptive products to guide local authorities in negotiating contraceptive purchases with local suppliers. This price list can be provided by the MOHF as often as possible, perhaps every three months. 18 Lost Opportunities for Economies of Scale Using direct transfers of special funds from the MOH, each individual hospital (with FP clinics attached) in Constanta, Cluj, and Mures locally purchases its own contraceptives. Hence, in Cluj, where there are 10 FP clinics, each hospital purchases contraceptives from its own set of local suppliers. As shown in Table 5, a clinic/hospital may purchase only 5–10 cycles of a given pill brand every two months. This does not constitute a bulk purchase and, hence, does not merit discounted bulk prices. In Cluj, hospitals paid around 62,000 ROL per cycle for purchasing between 10 to 100 cycles of Microgynon. Are these the best prices that a judet can negotiate for the purchase of these pill types? It might be argued that the judet as a whole would be better able to negotiate a lower unit price with the supplier if it were to centralize local contraceptive procurement within the judet. Panait Sarbu provides evidence of the benefits of bulk purchase. In contrast to local hospitals that paid between 50,000–65,000 ROL per cycle for Microgynon, Panait Sarbu obtained 25,000 cycles of the same oral contraceptive for only 20,112 ROL per cycle. Policy options to improve economies of scale in procurement: ™ Centralize procurement at the central level (Panait Sarbu). ™ Centralize procurement at the judet level. ™ Form inter-judet agreements where neighboring judets pool their resources to jointly purchase contraceptives from pharmaceutical companies at better prices. ™ Develop commodity information and distribution channels between clinics and central judet authorities. 2. Is the revolving fund sustainable? The contraceptives that FP clinics/hospital purchase using special MOHF funds are sold to clients who are not eligible for free contraceptives. The proceeds from the sales are used to maintain a revolving fund for local purchase of contraceptives. The fund sits in a hospital account and receives daily or weekly remittances from the family planning clinics. Hospitals use the funds to purchase additional contraceptives as required by each clinic. Locally procured contraceptives are sold at cost to clients – there is no profit margin; nor is there an adjustment for inflation, even though the acquisition price of contraceptives increase each month because of inflation.33 Table 6 shows the percentage increments in the acquisition prices of oral contraceptives between September/October 2000 and February/March 2001 for two clinics: one in Constanta and the other in Cluj. Table 6 Increments in Acquisition Prices of Locally Purchased Contraceptives Price increase (%) Contraceptive Brand Constanta Cluj Microgynon NA 14 Rigevidon 45 11 Diane 35 34 9 Logest NA 4 33 The acquisition price refers to the price that hospitals pay to purchase contraceptives from local suppliers. 19 Increments in the acquisition prices of contraceptives over a three-month time period are significant, particularly in the Constanta clinic. Despite this, the prices that users at family planning clinics pay for contraceptives exactly equal the acquisition price and do not reflect the replacement price of these contraceptives. As a result, the purchasing power of the revolving fund diminishes on a monthly, if not more frequent, basis. Two questions arise: (1) Why don’t FP clinics simply adjust the sale prices for inflation and other factors; and (2) Are there other mechanisms in place to stem the erosion of the revolving fund’s purchasing power? Why don’t FP clinics simply adjust the sale prices for inflation and other factors? Respondents often cited that the Romanian Financial Law prohibits government facilities from charging clients any mark up above acquisition cost, even if it is simply to adjust for inflationary pressures, for any product or service. In a country where inflation amounted to 40 percent in 2000 and there are no similar prohibitions on commercial suppliers of commodities, this leads to the inevitable erosion of the purchasing power of revenue. Unless there are other mechanisms in place to stem this erosion, the purchasing power of the revolving fund will gradually decline to zero. Are there other mechanisms in place to stem the erosion of the revolving fund’s purchasing power? The special funds that judet-level hospitals received from the MOHF was not a one-time infusion of money into the hospital account. Recognizing the pricing restrictions on FP clinics, the MOHF periodically adds to the fund “to adjust it for inflation” by sending new monies to the Local Health Authority (LHA) in each judet. For example, in September 2000, the hospital linked to the family planning reference center in Cluj received six million ROL to start the revolving fund. Soon thereafter, in November, the hospital received another six million ROL to replenish the fund. The revolving fund as it operates now is not sustainable without constant cash infusions from the MOHF. Given that the funds set aside by the MOHF for free contraceptives for target groups is grossly insufficient, the question arises: is replenishment of a revolving fund that currently serves clients who are well able to afford expensive contraceptive brands a justified and efficient use of scarce public sector resources? Instead, should a more realistic pricing mechanism be put in place by which each FP clinic is able to sustain its revolving fund independent of MOHF support? Policy options to improve the sustainability of the revolving fund: ™ Revise financial law to create an exemption that allows public sector entities to charge clients prices that are adjusted for inflation. This would allow facilities to charge clients prices that reflect the replacement price of contraceptives. ™ Ensure that revenues generated through the sale of contraceptives are used only to replenish the revolving fund. Issue clear guidelines about the use of the revolving fund. 3. Are family doctors receiving contraceptive supplies for sale to their clientele? Respondents from the local health authority, FP centers, and FP clinics argue that, legally, family doctors, unlike pharmacies, cannot sell drugs, including contraceptives. Family doctors also receive payment for the services they provide through the national health insurance system, which does not cover the provision or sale of contraceptives. Hence, family doctors are not included in the present system of local contraceptive procurement and sale. Again, as in the case of free contraceptives, this precludes rural 20 users from accessing subsidized contraceptives. Instead, rural women who wish to contracept must rely on pharmacies at which contraceptives cost 25–30 percent more than at FP clinics. Moreover, pharmacies in rural areas are few and far between. While the negative impact to rural users of not being able to receive subsidized or free contraceptives from family doctors is apparent, it is also necessary to view this situation from a doctor’s perspective. Would a private doctor be willing to provide a service that is not lucrative for him or her? Within this context, some family doctors who were interviewed expressed a desire for their FP services to be covered through fees for service under the health insurance system. Policy options to ensure that family doctors receive locally procured contraceptives: ™ Review and revise any legal restrictions that prevent family doctors from selling locally procured contraceptives. ™ Provide necessary training on FP counseling and service delivery to all family doctors willing to distribute contraceptives. ™ Advocate to family doctors about their critical role in reaching poor rural women with unmet need for modern contraception. ™ Delineate roles and relationships among family doctors, FP clinic staff, and Ob–Gyns to facilitate the full participation of family doctors in contraceptive distribution. Conclusions The assessment shows that key elements of contraceptive security—the policy framework, government financing, and the commitment of the MOHF leadership as well as service providers to undertake targeting—are already present in Romania. However, problems and barriers limit the effectiveness of contraceptive security efforts. The various barriers to implementing Romania’s contraceptive security initiative can be consolidated into five broad policy issues that pertain to • Adequacy of resources; • Effectiveness of targeting (based on geographic and individual socioeconomic status); • Efficiency of procurement (at the national and local level); • Sustainability of revolving funds; and • Access in rural areas. The paper presents various policy options that can be considered in addressing these issues and related barriers. The policy options need to be assessed in terms of their feasibility and financing implications. However, the authors believe that certain policy steps are critical. Centrally procured contraceptives do get to FP clinics in various parts of the country and FP clinics distribute these free commodities to targeted clients, particularly students. However, government- funded supplies are limited, thus inadequate to meet the total demand among these target groups. Few low-income clients receive free contraceptives and the primary beneficiaries are students, regardless of their income status. Addressing these problems in the long-term requires increasing MOHF resources for free contraceptives to cover all target populations, reducing the target groups that are eligible to receive free contraceptives, or pursuing a strategy that incorporates both these options. However, regardless of which option is chosen, the central procurement process can be greatly improved and made more efficient 21 by using scarce resources to purchase cheaper (generic) contraceptive brands, thus making available resources stretch further. The latter policy recommendation also applies to local contraceptive procurement. If the government chooses to continue the current revolving fund scheme, specific policies to ensure its sustainability need to be put in place. Such policies should require that judet entities buy primarily low- priced contraceptives, leaving the provision of more expensive brands to the commercial sector. Moreover, restrictions on charging clients prices that are adjusted for inflation should be removed or loosened to ensure the viability of the fund. In the face of scarce resources, inadequate supplies, and a large target population, the government’s current policy of supporting two mechanisms of funding contraceptive supplies needs to be reexamined. It is evident that the public sector funds available for the procurement of free contraceptives is grossly inadequate to meet the needs of target groups. Then, is the infusion of monies into revolving funds, most of which are used to buy more expensive contraceptive brands, justified? Certain stakeholders have called for the government to focus only on free contraceptive provision—whether procured centrally or locally- to the most vulnerable segments of the population. The decision on who constitutes “vulnerable groups” is one that must be reached through dialogue and consensus among stakeholders. In such a scenario, those who do not fall into groups eligible for government subsidies need to be served by alternative providers in the NGO and commercial sectors. Finally, the issue of access in rural areas must be addressed. Significant access to contraceptive services and commodities in these areas can only be assured through the primary care level, especially the family doctors. While the August 2000 law mandated that these doctors can indeed distribute the government-procured contraceptives, much still needs to be done to make this policy a reality. Ensuring that family doctors play a role in contraceptive security initiatives requires providing them with adequate training, a working logistics and management system, and more importantly, a continuing supply of contraceptives. Annex Table A1: Number of women 15–44 in union in specific socioeconomic groups using modern contraceptive methods and with unmet need Socioeconomic grouping Modern contraceptive methods Estimated number of current users of modern methods Estimated number with unmet need Rural poor Total 84,000 337,000 (16.4% of total or Pill 27,000 835,380) IUD 22,000 Condom 12,000 Other modern 23,000 Urban poor Total 30,000 83,000 (3.6% of total or Pill 10,000 184,620) IUD 7,000 Condom 6,000 Other modern 6,000 Lower middle Total 184,000 378,000 (20% of total or Pill 84,000 1,020,000) IUD 40,000 Condom 40,000 Other modern 18,000 Middle Total 255,000 337,000 (20% of total or Pill 74,000 1,020,000) IUD 61,000 Condom 66,000 Other modern 54,000 Wealthy rural Total 30,000 45,000 (2.6% of total or Pill 7,000 131,580) IUD 6,000 Condom 8,000 Other modern 9,000 Upper middle urban Total 282,000 246,000 (18.4% of total or Pill 70,000 938,400) IUD 53,000 Condom 73,000 Other modern 84,000 Urban rich Total 340,000 252,000 (19% of total or Pill 92,000 970,020) IUD 68,000 Condom 92,000 Other modern 88,000 2 References Centers for Disease Control, Romanian Association of Public Health and Health Management and National Commission of Statistics. 2001. Reproductive Health Survey: Romania, 1999. Dayaratna, Varuni, William Winfrey, William McGreevey, Karen Hardee, Janet Smith, Elizabeth Mumford, Jeffrey Sine, and Ruth Berg. 2000. “Reproductive Health Interventions: Which Ones Work and How Much do they Cost?” Occasional Paper Series No. 5. Washington, D.C.: The Futures Group International, POLICY Project. Erhan, Eugenia. 2001. Public Sector Funding of Contraceptives in Romania. POLICY Project. Ghetau, Vasile. 1997. Maternal Mortality and Abortion in Romania, 1989–1996. UNFPA. Government of Romania. 1999. Health Insurance Methodological Norms (Order 143/792). Official Monitor, September 7. Government of Romania. 2000. Health Insurance Methodological Norms (Order 921/1.765). Official Monitor, July 25. Government of Romania. 2000. Order for the approved structure, indicators and funds referring to financing of National Public Health Programs for 2000 (Order No. 730). Official Monitor, Part I, No. 422, September 1. Institute for National Statistics. 2000. Romania Statistical Yearbook: 1998–2000. Management Sciences for Health (MSH). 1999. Options for Organizing the Financing, Procurement and Distribution and Management of Contraceptives in Romania’s Public Sector. MSH. Marinescu, Luisa and Volkan Cakir. 2000. Financing Reproductive Health in Romania. POLICY Project. Ministry of Health (MOH). 2000. Technical Norms to Apply MOH Program No. 12 for National Protection for Women and Children’s Health and Family Planning. Research Institute for the Quality of Life and UNDP. 2001. Poverty in Romania, Causes, Anti-poverty Policies, and Recommendations for Action. Sharma, Suneeta, William Winfrey, and Mona Marin. 2001. Operationalizing Contraceptive Security Polices in Romania: A Market Segmentation Analysis. Washington, D.C.: POLICY Project. Stover, John and Laura Heaton. 1997. FamPlan (Version 4): A Computer Program for Projecting Family Planning Requirements. SPECTRUM System of Policy Models. POLICY Project. Tesliuc, Cornelia Mihaela and Lucian Pop. 2000. Poverty, Inequality, and Social Protection. UNDP. 1999. National Human Development Report: Romania 1999. UNDP. 2001. National Human Development Report: Romania 2000: The Implications of the Process of EU Accession for Human Development. World Bank. 2001. World Development Indicators, 2000/2001.

View the publication