UNICEF & WHO & UNFPA Guidelines for Monitoring the Availability and Use of Obstetric Services

Publication date: 1997

Guidelines for Monitoring the Availability and Use of Obstetric Services UNICEF WHO UNFPA August 1997 First Edition: October 1992 Second Edition: October 1997 Copyright © 1997 United Nations Children's Fund 3 UN Plaza New York, NY, 10017 USA E-mail: pubdoc@unicef.org Web site: www.unicef.org ISBN: 92-806-3198-5 This document has not been formally edited to publications standard. It does not necessarily reflect the policies of UNFPA, UNICEF, or WHO and the views expressed are solely the responsibility of the authors. The designations employed and the presentation of the material do not imply on the part of the above organizations the legal status of any country or territory, or its authorities, or the delimitations of its borders. The document may be freely reviewed abstracted, reproduced and translated, in part or in whole, provided it is not for sale or used for any commercial purpose. 1 Center for Population and Family Health, Columbia University. 2 UNICEF. 3 International Planned Parenthood Federation in New York. iv ACKNOWLEDGEMENTS These Guidelines were written by Deborah Maine1, Tessa M. Wardlaw2, Victoria M. Ward3, James McCarthy1, Amanda Birnbaum1, Murat Z. Akalin1 and Jennifer E. Brown1. A great many people have helped in the preparation of these Guidelines. First of all, we wish to thank our colleagues at UNICEF who not only presented us with this challenging assignment but made invaluable contributions: France Donnay, Gareth Jones, Jon Rohde, Monica Sharma, Leila Bisharat, Ranjit Atapattu, Siddharth Nirupam, Alexandra Yuster, Tewabech Bishaw and a number of others who commented on various drafts. We would also like to thank colleagues at the WHO in Geneva, including Tomris Turmen, Susan Holck, Carla AbouZahr, Robert Johnson and Godfrey Walker. Valuable assistance was also provided by Patricia Stephenson; Judith Fortney of Family Health International; Christopher C. Ekwempu of Zaria, Nigeria; Renu Varma of Uttar Pradesh, India; Paul Sengeh of Bo, Sierra Leone; and Borbor Kandeh of Freetown, Sierra Leone. Finally, as always our colleagues at Columbia University were helpful and supportive, particularly Allan Rosenfield, Angela Kamara, Therese McGinn, Linda Cushman, Hadi El Tahir, Joe Wray, Maxine Kuroda, Graciela Salvador, Laura Sanders, Annamaria Cerulli, Ana Pagan, Deborah Morton and Katrina Karkazis. v CONTENTS ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iii LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 METHODOLOGICAL ISSUES IN MEASURING MATERNAL MORTALITY 2.1. Frequency of Maternal Deaths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 2.2. Under-reporting of Maternal Deaths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 2.3. Misreporting of Maternal Deaths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 INDICATORS OF EFFORTS TO REDUCE MATERNAL MORTALITY 3.1. Indicators of Impact. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 3.1.1. Maternal mortality ratio (deaths per 100,000 live births). . . . . . . . . .15 3.1.2. Maternal mortality rate (deaths per 100,000 women aged 15–49 per year) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 3.1.3. Lifetime risk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 3.1.4. Proportion of all deaths among women of reproductive age. . . . . . .19 3.2. Indicators of Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 3.2.1. Essential obstetric care (EOC) coverage. . . . . . . . . . . . . . . . . . . . . . .23 3.2.2. Performance of EOC facilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36 COLLECTING DATA FOR PROCESS INDICATORS 4.1. Types of Data Required. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 4.2. Preparation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43 4.2.1. Determine the number of areas to be studied. . . . . . . . . . . . . . . . . . .43 4.2.2. Random selection of areas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 4.2.3. Determine a nationally uniform 12-month period to be studied. . . .45 4.3. Form 1: List All Possible EOC Facilities/Providers in Study Area. . . . . . .45 4.3.1. Determine the number of EOC facilities to be reviewed. . . . . . . . . . .46 4.3.2. Random selection of facilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47 4.4. Form 2: Conduct Site Visits to Assess EOC Actually Being Provided . . . .48 4.4.1. Notes on data collection using Form 2 (Worksheets 2a and 2b). . . . .49 4.5. Form 3: Summarize Findings for Basic and Comprehensive EOC Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 4.6. Form 4: Calculate Indicators for the Area. . . . . . . . . . . . . . . . . . . . . . . . . .50 4.7. Form 5: Calculate Indicators for the Country. . . . . . . . . . . . . . . . . . . . . . .51 4.8. Notes for Area-level Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51 4.8.1. Levels of functioning among facilities. . . . . . . . . . . . . . . . . . . . . . . . .51 4.8.2. Time availability of services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52 4.8.3. Geographical distribution of services within areas. . . . . . . . . . . . . . .52 vi 4.8.4. Differences between public-sector and private-sector facilities. . . . .52 4.8.5. Quality of care at facilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52 4.8.6. Quality of facility records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53 INTERPRETING THE FINDINGS 5.1. General Notes on Interpretation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73 5.1.1. ‘Minimum’ versus ‘optimum’ levels. . . . . . . . . . . . . . . . . . . . . . . . . .73 5.1.2. Generalizability of results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74 5.1.3. Incomplete or poor data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74 5.2. Interpreting the indicators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77 5.2.1. Amount of EOC services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77 5.2.2. Geographical distribution of EOC services. . . . . . . . . . . . . . . . . . . . .77 5.2.3. Proportion of all births in Basic and Comprehensive EOC facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77 5.2.4. Met need for EOC: Proportion of women estimated to have obstetric complications who are treated in EOC facilities. . . . . . . . . . . . . . . .78 5.2.5. Caesarean sections as a percentage of all births. . . . . . . . . . . . . . . . .78 5.2.6. Case fatality rates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81 APPENDIX A: Collecting Data for Impact Indicators. . . . . . . . . . . . . . . . . . . . . . . . . . . .89 APPENDIX B: Process Measures Not Recommended for Monitoring Maternal Mortality Programmes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97 APPENDIX C: Random Number Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100 APPENDIX D: Sample Register Head. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .101 vii LIST OF FIGURES Figure 1 Maternal deaths per 100,000 live births, Matlab, Bangladesh, 1987-1985 Figure 2 Maternal mortality ratios using the direct household survey: 95% confidence intervals and point estimates Figure 3 Maternal mortality ratios using the sisterhood method: 95% confidence intervals and point estimates Figure 4 Per cent of maternal deaths occurring in hospitals (data from population studies) Figure 5 Per cent of maternal deaths missing from official records Figure 6 Maternal deaths per 100,000 live births (maternal mortality ratios), subnational and national studies, 1976-1985 Figure 7 Maternal deaths per 100,000 women of reproductive age per year (maternal mortality rates), subnational and national studies, 1976-1985. Figure 8 Lifetime risk of maternal death at varying maternal mortality ratios and pregnancies per women Figure 9 Maternal deaths as a percentage of all deaths among women of reproductive age, subnational and national studies, 1976-1985 Figure 10 Reproductive, maternal and contraceptive mortality in Egypt (1981-1983), Indonesia (1980-1981) and the United States (1975) Figure 11 Indicators and minimum acceptable levels Figure 12 Signal functions used to identify Basic and Comprehensive EOC Figure 13 Estimated average interval from onset to death for major obstetric complications, in the absence of medical intervention Figure 14 Per cent of expected obstetric complications reaching first referral units, in 10 districts of India, 1993 Figure 15 Case fatality rates: Deaths per 100 complicated obstetric admissions or deliveries in EOC facilities Figure 16 Types of data used to construct process indicators Figure 17 User’s guide to the data collection forms 1 EXECUTIVE SUMMARY These Guidelines have been issued in response to the challenge put forward by the 1990 World Summit for Children. In the World Summit for Children Declaration and Plan of Action, one of the seven major goals is the reduction of maternal mortality by half between 1990 and the year 2000. This goal, originally formulated at the 1987 Safe Motherhood Conference in Nairobi, was re-emphasized at the 1994 International Conference on Population and Development in Cairo and the 1995 Fourth World Conference on Women in Beijing. The World Summit for Children Plan of Action states: Each country should establish appropriate mechanisms for the regular and timely collection, analysis and publication of data required to monitor relevant social indicators . which record the progress being made towards the goals set forth in this Plan of Action and corresponding national plans of action .[paragraph 34(v)]. While the primary responsibility for monitoring progress towards the goal lies with the government of each country, United Nations agencies, in cooperation with other national and international organizations, have a critical role to play in supporting countries in this process. These Guidelines discuss two approaches to monitoring progress. The conventional approach is to monitor the level of maternal mortality using such indicators as maternal mortality rates and ratios. In theory, repeated measurements over time would be used to monitor trends in maternal mortality. In most developing countries, where no comprehensive vital registration is available, this approach, however, has a number of serious drawbacks — both technical and substantive. Measuring maternal mortality is extremely difficult and costly. Even innovative methodologies that have been developed to estimate maternal mortality present difficulties. For example, the ‘sisterhood’ method provides information for a period of approximately 10 years before the survey. Thus, the information gathered does not reflect the current situation or progress made in the last decade. These Guidelines propose an alternative approach based on monitoring the processes, or interventions, aimed at reducing maternal mortality. There are several distinct advantages to this approach. First, it avoids the substantial expense involved in generating maternal mortality estimates, which in many cases may not be accurate, or which may reflect a situation 10 or more years in the past. Second, process indicators can provide information essential for guiding policies and programmes. One of the critical pathways to reducing maternal mortality is improving the accessibility, utilization and quality of services for the treatment of complications during pregnancy and childbirth. The evidence shows that at least 15 per cent of all pregnant women develop sudden serious complications and require life-saving access to quality obstetric services. These Guidelines therefore present a series 2 of process indicators that assess the availability, use and quality of obstetric services and provide guidance on data collection and interpretation. This series of process indicators was initially proposed in the first edition of these Guidelines [UNICEF, 1992]. In the second edition, the indicators have been revised so that they benefit from further work done in the field, and data collection forms have been added, as well as new sections on local level monitoring and interpretation of findings. The central purpose of monitoring is to gather data that are useful for guiding policies and programmes. Using process indicators will help programme planners identify priority issues and interventions. Furthermore, regular monitoring of process indicators will alert managers to areas of the programme that need to be strengthened. 3 1 INTRODUCTION In 1987, the International Conference on Safe Motherhood was held in Nairobi, Kenya. This was the first time that the attention of the international health community was clearly focused on deaths of women due to complications arising during pregnancy or childbirth. A second important moment was the 1990 World Summit for Children, sponsored by the United Nations and organized by UNICEF. In the World Summit for Children Declaration and Plan of Action, one of the seven major goals is the reduction of maternal mortality by half between 1990 and the year 2000. This goal was re- emphasized at the 1994 International Conference on Population and Development, held in Cairo, and the 1995 Fourth World Conference on Women, in Beijing. Monitoring of progress towards the reduction of maternal mortality will require reliable, timely and internationally comparable data. Such data are also needed to form a basis for policy and programme development, implementation, monitoring and evaluation. However, significant gaps remain in the available information, and data systems need to be strengthened in most countries. Moreover, there needs to be more and better interaction between the people who provide information and those who use it. While the primary responsibility for monitoring progress towards the goal lies with the government of each country, United Nations agencies have a critical role to play in supporting countries in this process, in cooperation with other national and international organizations. These Guidelines discuss two approaches to monitoring progress. The conventional approach is to monitor the level of maternal mortality using such indicators as maternal mortality rates and ratios. In theory, repeated measurements over time would be used to monitor trends in maternal mortality. This approach (i.e., monitoring the impact of programmes) would directly measure progress in achieving the goal of reducing maternal mortality by half by the year 2000. However, in most developing countries, where no comprehensive vital registration is available, this presents a number of serious drawbacks — both technical and substantive. Measuring maternal mortality is extremely difficult and costly. Even innovative methodologies that have been developed to estimate maternal mortality present difficulties. For example, the ‘sisterhood’ method generally provides information for a period of approximately 10 years before the survey. Thus, the information gathered does not reflect the current situation and therefore cannot be used to measure progress made during the last decade. These Guidelines propose an alternative approach which consists of monitoring interventions aimed at reducing maternal mortality. One of the critical pathways to reducing maternal mortality is improving the accessibility, utilization and quality of services for the treatment of complications during pregnancy and childbirth. At least 15 per cent of all pregnant women develop serious 4 complications that are often unpredictable and require life-saving access to quality obstetric services. However, there are virtually no data on the proportion of women with access to such care. One indicator that tries to capture such access is the proportion of pregnant women who deliver with the assistance of a skilled birth attendant. However, this information can best be collected using special surveys. By contrast, the indicators proposed in these Guidelines can be collected and analysed at the facility level. These Guidelines focus on improving the ability of the health system to respond to women’s needs for care in case of complications. This series of process indicators was initially proposed in the first edition of these Guidelines [UNICEF, 1992]. Since then, a number of groups have discussed and adapted some of these indicators [WHO, 1994a; WHO, 1994b; Reproductive Health Indicators Working Group, 1995; The World Bank, 1995; UNFPA, 1997]. In this edition, the indicators have been revised to reflect further work done in the field. This edition also includes new data collection forms, as well as new sections on local level monitoring and interpretation of findings. Chapter 2 describes the specific features of maternal mortality that make it difficult to study. Compared to other demographic events, such as births or infant deaths, maternal deaths occur relatively infrequently. In addition, those that do occur often go unrecorded or, if recorded, are not correctly classified as maternal deaths. Chapter 3 presents the two types of indicators that can be used to monitor changes in maternal mortality — impact and process indicators. Because impact indicators are all based on the identification of maternal deaths, which are difficult to identify, the use of process indicators is a crucial component in monitoring progress in reducing maternal deaths. Specific process indicators to assess availability, use and quality of essential obstetric care services are described. For the purpose of monitoring, a short list of ‘signal functions’ is used to measure the care being provided to women with obstetric complications. These ‘signal functions’ do not cover the full range of functions which constitute essential obstetric care [WHO, 1995], nor do they address other aspects of maternal and newborn health such as sexually transmitted diseases, prevention of complications or care of the newborn. Chapter 4 highlights the practical aspects of gathering data needed to calculate the process indicators, all of which can be generated from facility-based records. Data collection forms are included, as well as recommendations on how to calculate the indicators. Chapter 5 provides guidance on the interpretations of the findings and discusses conclusions that may be drawn from the indicators, and their implications for policies and programmes. The appendices include details on methodological issues involved in collecting data for impact indicators. The central purpose of monitoring is to gather data that are useful for guiding policies and programmes. Using process indicators will help programme planners identify priority interventions and areas. Furthermore, regular monitoring of process indicators will alert managers to areas of the programme that need to be strengthened. A key principle underlying the identification of these 5 indicators is that they are useful for case or programme management at the level at which the data are collected. Thus, they serve to strengthen national capacity for data-led decision-making. b Assuming a birthrate of 40 per 1,000 population and an average household size of five. 6 2 METH ODOLOGICAL ISSUES IN MEASURING MATERNAL MORTALITY There are ways in which maternal mortality research is different — both in kind and in degree — from research in more established fields, such as child survival and family planning. These differences shape the types of research on maternal mortality that are appropriate, feasible or even possible in a given circumstance. Some of the most important factors affecting research on maternal mortality are discussed below: the frequency of deaths; under-reporting of deaths; and misreporting of the cause of death. 2.1. Frequency of Maternal Deaths Pregnancy and childbirth are the leading cause of death among women in many developing countries. However, deaths of young adults are relatively rare events. Therefore, in a given geographical area and period of time there may not be a great many maternal deaths, as the following example shows: The Matlab project in Bangladesh is probably the largest and most intensive population research study in the developing world. During 1984–1986, births and deaths among more than 21,000 women of reproductive age were recorded. Maternal mortality is high, with more than 400 maternal deaths per 100,000 live births [Fauveau et al., 1991]. Nevertheless, during those three years only 40 maternal deaths were recorded. The relative infrequency of maternal deaths in a short period (such as 1–2 years) has important consequences for monitoring maternal mortality. If the study population or sample is too small, the number of deaths will not be large enough to yield reliable, stable estimates. This can be illustrated using the Matlab data. Maternal mortality is high there, but the relatively small number of maternal deaths each year makes the rates appear to jump around. This makes interpretation, especially interpretation of trends over time, difficult. Figure 1 illustrates this point. The infrequency of maternal deaths means that large populations need to be studied, which is very costly. For example, to document a maternal mortality ratio of 400, and be fairly certain that your estimate is reasonably correct (e.g., within 20 per cent) would require a sample size of 50,000 births, or 200,000 householdsb. When the desired margin of error is reduced to 10 per cent, the sample size requirement jumps to 800,000 households. Because of the large sample size requirements, such survey methods are limited in their ability to detect statistically significant changes in maternal mortality over time. Figure 2 shows the maternal c Confidence intervals around original measurement were calculated using the standard error reported by Kwast et al. Standard errors under the two scenarios were calculated using the method presented by Fleiss, 1981. The standard errors under the two scenarios are underestimates; they do not take into account the additional variation related to the survey design and field conditions. The widths of the resulting confidence intervals are thus underestimated as well. d Confidence intervals for sisterhood method calculated using the methodology presented by Hanley et al., 1996. 7 mortality ratio obtained from a direct survey of 32,215 households (which identified 9,315 pregnancies) in Addis Ababa, Ethiopia [Kwast et al., 1985]c. It shows the 95 per cent confidence interval around this estimate, and the 95 per cent confidence intervals under two other scenarios: a 50 per cent decline in maternal mortality and a 25 per cent decline. In order to be able to say with reasonable certainty that an observed decline is not simply due to chance fluctuation, the confidence intervals of the original estimate and the observed decline must not overlap. As can be seen in Figure 2, the confidence intervals of both the 50 per cent decline and the 25 per cent decline overlap with the original estimate. Thus, this survey would be incapable of detecting even a 50 per cent reduction in maternal mortality. New methods of estimating maternal mortality, such as the ‘sisterhood’ method, are more efficient and do not require sample sizes as large as those of conventional household surveys. But they are still limited in their ability to detect substantial changes over time. Figure 3 shows the same exercise, using data from the original sisterhood study in the Gambia [Graham et al., 1989], which included 2,163 respondentsd. In this case, a 50 per cent decline in maternal mortality would be detectable, but a 25 per cent decline would not. The sisterhood method has other limitations. The most important of these is that it produces an estimate of maternal mortality that refers to a period of time approximately 12 years before the survey. By increasing the sample size, the most this time lag can be reduced is to about 6 years before the survey [Hanley et al., 1996]. Such estimates, therefore, are not useful for monitoring changes in response to programmes being implemented now. (Appendix A contains further details on survey methods for measuring maternal mortality.) 2.2. Under-reporting of Maternal Deaths Registration of births and deaths (i.e., ‘vital registration’) is taken for granted in industrialized countries. In these countries, and in a few developing countries, nearly all deaths are reported to the government. This is not the case, however, in most developing countries. Figure 2. Maternal mortality ratios using the direct household survey: 95% confidence intervals (C.I.) and point estimates MMR (deaths/100,000 live births) 758 374 391 175 548 302 Actual estimate* 50% decline 25% decline 0 200 400 600 800 *Data from the Addis Ababa study [Kwast et al., 1985]. Sample size equals 32,215 households; 45 maternal deaths identified. Estimate refers to the two-year period before the survey. 95% C.I. 425 Point estimate 566 283 Figure 3. Maternal mortality ratios using the sisterhood method: 95% confidence intervals (C.I.) and point estimates MMR (deaths/100,000 live births) 1,243 767 674 332 961 547 Actual estimate* 50% decline 25% decline 0 200 400 600 800 1,000 1,200 1,400 95% C.I. *Data from the Gambia study [Graham et al., 1989]. Sample size equals 2,163 respondents; 91 maternal deaths identified. Estimate refers to a time period centred approximately 12 years before the survey. 503 754 1,005 Point estimate 11 One of the reasons why it is difficult (if not impossible) to register deaths in developing countries is that most deaths do not take place in health facilities, where health personnel would be required to report them. Many people (especially poor people) die at home or on their way to the hospital. Their deaths are not recorded. Figure 4 shows the proportions of maternal deaths that took place in hospital in population-based studies. Even where records are available, under-reporting may be a serious problem because of poor record-keeping, etc. 2.3. Misreporting of Maternal Deaths The term ‘misreporting’, as used here, means that the death was reported, but the death was incorrectly classified — i.e., not recorded as a maternal death. According to the Tenth International Classification of Diseases, a maternal death is defined as “the death of a woman while pregnant or within 42 days of termination of pregnancy, irrespective of the duration and the site of the pregnancy, from any cause related to or aggravated by the pregnancy or its management, but not from accidental or incidental causes” [Fortney, 1990]. Thus, in order to classify a death correctly as a maternal death, it is necessary to know not only that the woman died, but also that the cause and timing of death meet the specifications. In contrast, to correctly classify infant mortality, it is necessary only to know the age of the child who has died. The more complicated definition of maternal mortality thus facilitates misreporting. This has the same consequences for maternal mortality statistics as not reporting the death at all, i.e., underestimation. Some women die before they (or their relatives, who report the death) know that they are pregnant. In addition, some women who eventually die of obstetric complications survive the 42-day period. Although such deaths are due to obstetric causes, they are not classified as maternal deaths using the International Classification of Diseases [Fortney, 1990]. This source of error is less important in developing than in developed countries. For example, in England and Wales in 1982–1984, 23 per cent of maternal deaths occurred between 42 and 365 days after the end of the pregnancy [Turnbull, 1989]. One reason for this is that women who are going to die can be kept alive longer with sophisticated medical technology. In Jamaica, only 4 per cent of maternal deaths took place between 42 and 365 days after the end of the pregnancy [Walker, 1986]. Determining the medical cause of death is much more difficult than determining timing of death. The fact that a death was maternal is often not noted for various reasons, both intentional and unintentional. Intentional misreporting of maternal deaths is common when the death is due to complications of illicitly induced abortion. In many societies, abortion-related deaths are concealed to protect the reputation of the woman or her family. In some countries, legal action is taken against people who perform abortions and/or against women who obtain them, if they survive. Thus, fear of legal prosecution is also a cause of the misreporting of maternal deaths. Unintentional misreporting of maternal deaths is very common. Frequently, women die of obstetric complications in emergency wards or medical wards, as opposed to maternity wards. Consequently, 12 the obstetric origin of the bleeding or infection may not be noted. In some cases the health professional attending the woman knows that the death was related to pregnancy, but this information is neither requested nor recorded. Such mistakes lead to massive underestimation, even in developed countries. Figure 5 presents the results of some studies in which special efforts were made to identify misclassified maternal deaths. In England and Wales, for example, 22 per cent of maternal deaths were missing from official reports during 1982–1984 [Turnbull et al., 1989]. In the city of Sa˜o Paulo in Brazil, more than half of the maternal deaths were not officially reported in 1986 [Laurenti, 1993]. Campbell and Graham [1990] provide a thorough review of this topic. e In a true rate, the numerator is drawn from the denominator, and time is an essential component. In other words, a rate measures the speed with which people in the denominator become part of the numerator. Obviously women who die are not a sub-group of live births. Also, time is not an integral feature of this statistic. The maternal mortality ratio, therefore, measures the risk, not the rate. f The use of 100,000 live births as a denominator makes this measure somewhat imprecise. Some women (especially those who experience unsafe abortions) are at risk of dying a maternal death without ever having a live birth. Therefore, a more precise maternal mortality ratio would be the number of maternal deaths per 100,000 pregnancies. However, data on number of pregnancies, even in countries with good vital statistics systems, are difficult to obtain. 15 3 INDICATORS OF EFFORTS TO REDUCE MATERNAL MORTALITY A maternal death is not just a discrete event but rather the culmination of a process. Therefore, the identification and recording of maternal deaths is far from a simple task, and monitoring efforts to reduce maternal mortality cannot rely solely on counting changes in deaths. Monitoring must include information on the processes or pathways that culminate in a maternal death. Thus, in the following discussion of indicators of efforts to reduce maternal mortality, we discuss both indicators of impact (i.e., of changes in deaths) and indicators of process (i.e., of changes in those activities or circumstances that are known to contribute to maternal death). 3.1. Indicators of Impact The impact of a programme to reduce maternal deaths is determined by measuring changes in the level of maternal mortality. There are a number of ways to describe this level, some of which are discussed below. 3.1.1. Maternal mortality ratio (deaths per 100,000 live births) This statistic has traditionally been called the maternal mortality ‘rate’. Technically, it is not a rate, but a ratio.e A variety of experts are now advocating the use of the term ‘ratio’, not only for technical reasons, but because the true maternal mortality rate is also an informative statistic, and we need to be able to refer to it [Fortney, 1987a]. Until the usage of these terms is settled, one should be certain to specify which definition is being used. The maternal mortality ratio (maternal deaths per 100,000 live births) measures the risk of maternal death among pregnant or recently pregnant women.f Figure 6 shows maternal mortality ratios in a variety of countries. 16 3.1.2. Maternal mortality rate (deaths per 100,000 women aged 15–49 per year) This is the true maternal mortality rate. Although less often cited than the ratio, it is important because it measures the impact of maternal deaths on the population of women as a whole, not just on pregnant women. This statistic is affected by two forces: (1) the risk of death among pregnant women; and (2) the proportion of women who become pregnant each year. Consequently, the maternal mortality rate can be lowered either by making childbirth safer or by reducing the fertility rate in the population. Figure 7 shows maternal mortality rates in several countries. 3.1.3. Lifetime risk A woman's risk of maternal death accumulates over her reproductive lifetime. Every time she becomes pregnant she runs the risk again. This is different from the risk of infant mortality, to which each person is exposed only once. Additionally, mortality and fertility, in general, vary together (i.e., women in countries with high maternal mortality tend to have high fertility). Consequently, the lifetime risk reflects the true discrepancy in the risk of maternal death between developed and developing countries. Although this statistic came into use only recently, it is perhaps the most eloquent of all. Lifetime risk, like the maternal mortality rate, is influenced by both the risks associated with an average pregnancy and the number of times the average woman gives birth. However, the maternal mortality rate shows the effect of these forces on a particular population of women. Lifetime risk, on the other hand, reflects the effect of these forces on an average woman's risk of dying a maternal death. Figure 8 shows how lifetime risk is influenced by both mortality and fertility. In terms of lifetime risk of maternal death, halving the number of pregnancies has the same effect as halving the risk of dying per pregnancy (the maternal mortality ratio). 19 Figure 8. Lifetime risk of maternal death at varying maternal mortality ratios (MMRs) and pregnancies per woman MMR = 500 MMR = 250 1 pregnancy 1 in 200 1 in 400 2 pregnancies 1 in 100 1 in 200 4 pregnancies 1 in 50 1 in 100 8 pregnancies 1 in 25 1 in 50 3.1.4. Proportion of all deaths among women of reproductive age In countries where maternal mortality is high, so is death from many other causes (e.g., infectious diseases). It is conceivable, therefore, that maternal deaths might account for similar proportions of deaths among women of childbearing age in developing and developed countries. Figure 9 shows that this is far from the case. Maternal deaths often account for more than one quarter of deaths among women in developing countries. By contrast, such deaths represent less than 1 per cent of deaths among women in developed countries. Actually, maternal deaths are part of a larger category called ‘reproductive mortality’, which includes both maternal deaths and deaths due to the side effects of contraceptive methods [Beral, 1979]. In developed countries, almost all deaths from obstetric complications are prevented and most women use some form of contraception. As a result, even though deaths from contraceptive side effects are extremely rare, they make up a large proportion of reproductive deaths in developed countries. In other words, contraceptive deaths make up a large proportion of the small number of reproductive deaths in developed countries. In developing countries, the situation is the reverse, as Figure 10 shows. Reproductive deaths are quite common, but almost all of them are due to complications of pregnancy and delivery, rather than to contraceptive side effects. 21 Figure 10. Reproductive, maternal and contraceptive mortality in Egypt (1981–1983), Indonesia (1980–1981) and the United States (1975)* Egypt Indonesia United States Reproductive deaths per 100,000 women aged 15–44 46** 70** 2 Maternal deaths per 100,000 live births 190 718 13 Per cent of all deaths of women aged 15–44 due to reproduction23 23 2 Per cent of all reproductive deaths due to contraception 2 1 47 * The Centers for Disease Control and Prevention report that no more recent analysis exists for the United States. ** Married women only. Source: Fortney et al., 1986. 3.2. Indicators of Process People are accustomed to hearing and thinking about maternal mortality rates and ratios, or about the actual number of deaths (e.g., 585,000 worldwide per year). Furthermore, international and national goals for the reduction of maternal mortality imply that we know or can learn current maternal mortality rates and/or ratios, and that we can monitor changes in these indicators. There is, however, another way to measure progress in the reduction of maternal mortality, and that is by using process indicators. It is important to note that process indicators are not poor substitutes for impact indicators. Process indicators, in fact, provide a great deal of extremely useful information that impact indicators do not. Consider, for example, a medium-size developing country with a weak vital statistics system. In order to determine the current maternal mortality rate or ratio in such a country, it would be necessary to do a series of special studies to provide a stable estimate of the current level of maternal mortality. This would be both expensive and difficult to do. Once these studies were completed and analysed, one would have information on the frequency of maternal death, but not on the status of activities needed to prevent it. In addition, suppose that one were able to do enough special studies to chart changes in maternal mortality over time. If there were an unexpected change, one would not be able to explain it simply by using rates and ratios. One would need information on events that affect maternal mortality, such as changes in the ability of the health system to provide timely treatment to women with obstetric complications. 22 This situation is not unique to maternal mortality. It is analogous to the situation in the field of infant and child health. For example, immunization during infancy can prevent certain life-threatening diseases, such as measles and pertussis. Deaths from measles are more common than deaths from obstetric complications, but they are nevertheless difficult to count. Therefore, efforts to evaluate progress usually focus on the process (proportion of children who have been immunized), rather than on the impact (e.g., death rates due to measles). A great deal of work has been done on monitoring the effect of family planning programmes by examining both impact and process indicators. Programmes designed to reduce maternal deaths by reducing pregnancies can easily adopt some of the well-developed methods for evaluating family planning programmes. However, the use of process indicators to evaluate efforts to improve the treatment of obstetric complications is not well established. Therefore, the following discussion of process indicators focuses on the prevention of maternal deaths by ensuring prompt, adequate treatment of obstetric complications. A note on terminology: Because the purpose of this document is to facilitate the monitoring of programmes, it is necessary to identify a short list of ‘signal functions’ with which to measure the care provided for obstetric complications in a given setting. In this document, we use the term ‘essential obstetric care’ (EOC) to refer to the short list of services that can save the lives of the majority of women with obstetric complications. (See Figure 12.) Two levels of care are defined: Basic and Comprehensive EOC. The main difference is the provision of Caesarean sections and blood transfusions in Comprehensive EOC facilities. The list of signal functions is, by definition, not comprehensive. It does not include every service that ought to be provided to women with complicated pregnancies. This list is intended for monitoring activities, not for designing programmes. Other documents may be consulted for content of services that should be provided to women with complicated or problem pregnancies and to pregnant women in general [WHO, 1994a]. 23 In the sections below we describe a series of process indicators with which national progress in the prevention of maternal death can be monitored. The order in which they are discussed reflects a rough order of priority. If women are to receive prompt adequate treatment for complications, then facilities for providing essential obstetric care (EOC) must ! exist; ! be distributed in a useful fashion; ! be used by women; and ! be used by women who really need them. All of these issues can be subsumed under the heading of coverage. Adequate coverage does not suggest that all births should take place in health facilities. It does mean that all pregnant women need access to functioning EOC facilities, in case they need them. Once coverage is established, then questions of performance must be addressed. After all, many women die in hospitals. Some of them die because they were not admitted until their condition was critical. Many others, however, die because they did not receive timely treatment or because the treatment they received was inadequate. Figure 11 presents a series of process indicators that address the issues of EOC coverage and quality mentioned above. Beside each indicator there is a ‘minimum acceptable level’. (Note that the indicator using data on Caesarean sections has both a minimum and a maximum.) These acceptable levels are, of necessity, approximate. They are based on the best data, estimates and assumptions currently available. They may be adapted in light of individual countries’ circumstances. These levels can be used to guide programmes as well as to monitor them. In a given developing country, the people responsible for reducing maternal deaths can start at the top of this list of indicators and work down. When they reach an indicator for which the country does not meet the minimum acceptable level, appropriate interventions are needed. For example, if a particular country meets the minimum acceptable levels for the amount and distribution of EOC facilities, but not for utilization, then interventions are needed to improve women's use of EOC services. 3.2.1. Essential obstetric care (EOC) coverage A. Amount of EOC services The first in this series of process measures is the existence of sufficient essential obstetric care services. For purposes of monitoring, it is best to use a short list of clearly defined ‘signal functions’ to represent EOC. A number of authors have estimated the proportion of pregnant women who develop serious complications to be at least 15 per cent [Hibbard, 1978; Hartfield, 1980]. Furthermore, a Technical g This list has been revised since the first edition. 24 Working Group assembled by WHO agreed to use 15 per cent as the minimum proportion of pregnant women who require medical care in order to avoid death or disability [WHO, 1994b]. How many EOC facilities are required to treat complications? That, of course, depends on the size and capabilities of the facilities. One could count only facilities where all of the EOC procedures are performed. This would, however, impose an unnecessarily strict standard. Moreover, it would impart the wrong message by implying that only hospitals are useful in reducing maternal mortality. It is this mistaken impression that has given some policy makers the idea that reducing maternal deaths means building new hospitals and supplying them with sophisticated equipment and specialist physicians. This is not necessary in many places. In fact, one of the most promising interventions is the upgrading of health centres and other small facilities so that they can provide basic essential obstetric care. This would require providing some, but not all, EOC services at such facilities; thus both Basic and Comprehensive EOC facilities are preventing maternal deaths. In essence, the difference between Basic EOC and Comprehensive EOC is the capacity to give blood and perform surgery (e.g., Caesarean section). It is important that the distinction be made on the basis of how facilities are actually functioning, and not on how they are supposed to function. The importance of this distinction is illustrated by the results of a field test of this indicator in selected facilities in Bangladesh: District hospitals in Bangladesh are supposed to provide Comprehensive EOC services. However, when 20 district hospitals were surveyed in 1993, 6 — a full 30 per cent — were found to be functioning as Basic, not Comprehensive, EOC facilities. Thana Health Complexes (THCs) are supposed to provide Basic EOC. Yet a review of 25 THCs found that 10 (40 per cent) were not, despite the fact that each facility employed a full-time medical officer [Mostafa and Ali Haque, 1993]. In order to assess which level of care an EOC facility is actually providing, it is helpful to select a few important EOC functions to identify both Basic and Comprehensive EOC. These are not intended to serve as a complete list of services that should be provided at a Basic or a Comprehensive EOC facility.g Rather, they are ‘signal functions’ that can be used for classification and monitoring. For these purposes, the procedures used to identify Basic and Comprehensive EOC are shown in Figure 12. 25 Figure 11. Indicators and minimum acceptable levels Indicator Minimum acceptable level Amount of essential obstetric care (EOC): Basic EOC facilities Comprehensive EOC facilities For every 500,000 population, there should be: At least 4 Basic EOC facilities. At least 1 Comprehensive EOC facility. Geographical distribution of EOC facilities Minimum level for amount of EOC services is met in subnational areas. Proportion of all births in Basic and Comprehensive EOC facilities At least 15% of all births in the population take place in either Basic or Comprehensive EOC facilities. Met need for EOC: Proportion of women estimated to have complications who are treated in EOC facilities At least 100% of women estimated to have obstetric complications are treated in EOC facilities. Caesarean sections as a percentage of all birthsAs a proportion of all births in the population, Caesarean sections account for not less than 5% nor more than 15%. Case fatality rate The case fatality rate among women with obstetric complications in EOC facilities is less than 1%. 26 Figure 12. Signal functions used to identify Basic and Comprehensive EOC Basic EOC services Comprehensive EOC services (1) Administer parenteral* antibiotics (1–6) All of those included in Basic EOC (2) Administer parenteral oxytocic drugs(7) Perform surgery (Caesarean section) (3) Administer parenteral anticonvulsants for pre-eclampsia and eclampsia (8) Perform blood transfusion (4) Perform manual removal of placenta (5) Perform removal of retained products (e.g., manual vacuum aspiration) (6) Perform assisted vaginal delivery A Basic EOC facility is one that is performing all of functions 1–6. A Comprehensive EOC facility is one that is performing all of functions 1–8. *Parenteral administration of drugs means by injection or intravenous infusion (‘drip’). A Basic EOC facility is one that is performing all six of the functions listed above. This does not mean that other functions are not important. But for the purposes of monitoring, these six functions are considered sufficient to identify the kind of facility that can perform most, but not all, EOC activities. Of course, at a Comprehensive EOC facility, the ability to perform surgery entails a number of other capabilities, e.g., administering anaesthesia. For the sake of simplicity, these are not listed in the definition of Comprehensive EOC. A health centre that provides Basic EOC can prevent many maternal deaths. For some conditions (e.g., some cases of post-partum haemorrhage), these services would be sufficient. For other complications (e.g., obstructed labour), more complicated treatment is required. Even then, first aid can save lives because the woman’s condition can be stabilized before she is referred. Often the journey takes many hours, during which her condition could deteriorate. For example, a woman with obstructed labour may not be able to be treated in a health centre that provides only Basic EOC. She h A Technical Working Group convened by WHO endorsed a similar list; however, that list distinguishes between health centres and hospitals rather than between Basic and Comprehensive EOC facilities. As noted earlier, for monitoring purposes, we emphasize the actual — not potential — levels of functioning of facilities. i The WHO Technical Working Group also adopted these estimates. 27 Minimum acceptable level: For every 500,000 people, there should be: 1 facility providing Comprehensive EOC; and 4 facilities providing Basic EOC. needs a Caesarean section. Her chances of surviving the Caesarean section are, however, greatly improved if she does not arrive at the hospital dehydrated and infected. Administration of intravenous fluids and antibiotics at the health centre is extremely helpful, especially in cases where the trip to the hospital is long. There are, of course, many health facilities that perform some, but not all, of the Basic EOC functions listed above. These facilities are undoubtedly useful, and may well avert some maternal deaths. Such facilities should definitely be included in, for example, an in-depth study of a district. For national monitoring, however, it is neither feasible nor useful to have fine distinctions and many categories. Consequently, for the present purposes, only facilities currently providing all the signal functions in either the Basic or Comprehensive EOC lists are included.h What should be the minimum acceptable levels for these different kinds of care? A reasonable (even conservative) estimate of the minimum proportion of pregnant women who will require a Caesarean section is 5 per cent [Nordberg, 1984]. Since we are assuming that about 15 per cent of pregnant women will develop serious complications, then we can estimate that one third (5/15 = .33) of women with complications will require treatment in a Comprehensive EOC facility.i Theoretically, all women who need Basic EOC could be treated in a single facility. This kind of arrangement, however, would ignore problems of access. Even in a city, one facility may not be easily accessible to everyone. The minimum acceptable level for this indicator has been defined in relation to population rather than births because most health planning is done in relation to population. However, if it is judged more appropriate to assess the adequacy of EOC services in relation to births, the comparable minimum j For example, auxiliary midwives at rural health posts could save lives with injectable oxytocin or ergometrine. Among the low-tech, village-based methods proposed for preventing deaths from haemorrhage are the following: ‘rub up a contraction’ by massaging the fundus of the uterus; perform external bimanual compression of the uterus; promote uterine contractions by nipple stimulation, including putting the baby to the breast. There is some controversy about the efficacy of some of these procedures, particularly the last one. More research on this topic is needed. 28 acceptable level would be one Comprehensive EOC facility and four Basic EOC facilities for every 20,000 annual births. B. Geographical distribution of EOC facilities If enough EOC facilities exist, then the next step is to see if they are appropriately located, i.e., near the women who need them. There are a number of possible ways to measure the distribution of facilities. A telling indicator of access to EOC is time. Time is crucial to the survival of women with complications. Figure 13 shows the estimated average time interval from onset to death for the major obstetric complications. The most salient feature of these estimates is that for most complications the average time is 12 hours or more. The exception to this is post-partum haemorrhage, which can kill a woman in less than one hour. Haemorrhage is, however, one of the few major obstetric complications for which first aid could be provided at peripheral health facilitiesj [Kamara, 1990; WHO, 1989]. Figure 13. Estimated average interval from onset to death for major obstetric complications, in the absence of medical intervention Complication Hours Days Haemorrhage Post-partum Antepartum 2 12 Ruptured uterus 1 Eclampsia 2 Obstructed labour 3 Infection 6 Source: Maine et al., 1987. 29 It might be possible to establish a reasonable standard for the availability of services: e.g., to have Basic EOC available within three hours' travel of most women and Comprehensive EOC available within 12 hours. Unfortunately, determining what proportion of the population live in particular areas is a very cumbersome business. Furthermore, collecting and analysing the data necessary to do this would consume a disproportionate amount of time and resources. Nevertheless, the distribution of services is too important to ignore. It is not uncommon to find an excess of services clustered around the main cities and large parts of the population in more rural areas virtually without services. In this analysis, it is relevant to consider private and religious as well as government facilities. In some countries, private facilities provide an important proportion of EOC procedures, and should be included when doing an inventory of facilities. An efficient way of checking on the distribution of EOC services throughout the country is to calculate the amount of EOC services available in areas smaller than the country as a whole — the smaller the better. Even an analysis at the state or regional level will often point out important discrepancies. Therefore, the minimum acceptable level for distribution of EOC services is the same as that for the amount of EOC services, but applied to smaller geographical areas. Monitoring of this indicator would thus involve dividing the country into geographical areas based on existing divisions or population. The numbers of EOC facilities in these areas would then provide a better indication of the distribution of facilities. For example, a situation analysis in Syria in 1982 determined that the amount and distribution of facilities providing maternity care were both insufficient. Hospitals — many of which provided Comprehensive EOC — were disproportionately located in one city, Damascus. Furthermore, only 31 per cent of all ‘health centres for basic services’ provided MCH services; and these facilities were also clustered in Damascus city and in the capitals of the other Syrian governorates. In El-Hassakeh, for instance, the population level required at least 5–6 Basic EOC facilities; however, only one existed in the capital, and none elsewhere [Fathalla, 1983]. C. Proportion of all births in Basic and Comprehensive EOC facilities Following the series of questions posed earlier, the next question is whether women are using the EOC facilities. The idea here is not to recommend that all women deliver in hospitals. In many developing countries the health system could not cope with the added patient load. Furthermore, if a woman is having a normal delivery, then she may well be better off at home. The question is what happens when she develops a complication? The proportion of all births that take place in an EOC facility serves as a crude indicator of utilization of EOC facilities. We estimate that 15 per cent of pregnant women develop an obstetric complication serious enough to require medical care. Thus, if the number of women receiving care in an EOC facility is not at least 15 per cent of all women giving birth in the population, then it is certain that 30 Minimum acceptable level: At least 15 per cent of all births in the population take place in either a Basic or a Comprehensive EOC facility. some proportion of obstetric complications are going untreated. In that case, the efforts of the national maternal care programme should be directed towards seeing why the existing facilities are not being utilized by women who need medical care. Of course, what is most important is that EOC facilities are used by the women who really need them for life-saving obstetric care. In other words, if all of the women in the population who develop obstetric complications receive EOC services, the proportion of the need for EOC that is being met in the population is 100 per cent. Estimating ‘met need’ is therefore actually a more precise way to monitor progress towards reducing maternal mortality. However, in many places the data needed to estimate met need are not yet available. But data for calculating the present indicator — the proportion of all births in Basic and Comprehensive EOC facilities — are often available. Using this indicator should therefore be seen as a preliminary step leading up to the calculation of met need. Efforts to improve utilization can include a great variety of activities, depending on what factors are discouraging use. For example, if a needs assessment shows that people lack basic information about obstetric complications, then a community education programme would be in order. The precise shape of this programme would be determined by the local circumstances, but it should be aimed at the people who influence the decision to seek care, such as traditional birth attendants, women of reproductive age, husbands, mothers-in-law. If transportation from the village to the EOC facility is a major problem, efforts can be made to mobilize the community to coordinate the use of existing vehicles. If poor roads are a barrier to care, one could work with the local government to improve them. If shortages of supplies make people feel that going to the hospital is not worth the trouble, then solutions to this problem should be sought. D. Met need for EOC: Proportion of all women with complications who are treated in EOC facilities Of course, just because 15 per cent of births take place in EOC facilities does not mean that women with complications are receiving care. It might be that most of the births in the EOC facilities are normal deliveries. In that case, the women with complications would still be outside the EOC facilities and not receiving treatment. This indicator, therefore, is a more refined measure of the utilization of EOC services because it takes into account the type of activities occurring in the EOC facilities. k During committee meetings of the USAID-funded Evaluation Project, the met need indicator proposed in the first edition of these Guidelines was refined, and the definition of a complicated case changed somewhat. 31 Minimum acceptable level: The proportion of all women with obstetric complications who are treated in Basic or Comprehensive EOC facilities is at least 100 per cent. For the purposes of monitoring, a complicated case is defined as one that has any of the following diagnoses:k Working definition of a complicated case: ! Haemorrhage: antepartum or post-partum ! Prolonged/obstructed labour ! Post-partum sepsis ! Complications of abortion ! Pre-eclampsia/eclampsia ! Ectopic pregnancy ! Ruptured uterus Note: If a woman has more than one of these complications, the most immediately life- threatening one should be selected. While the met need for EOC may be used as a gauge of the level of EOC activity in an area, it cannot describe what needs to be done. If the proportion of need being met is low, it is not possible to distinguish from this statistic alone where the problem lies. It may lie in the availability, accessibility or quality of care being provided, or it may lie in utilization of the services or, most probably, both. Further investigation is then required. 32 Early experience with this indicator in Bangladesh and India shows that in countries where maternal mortality is a major problem, the proportion of the need for EOC that is being met will be low — e.g., under 20 per cent in many areas. Figure 14 shows met need for EOC in 10 districts of India. In some places — chiefly in developed countries — the proportion of women with complications managed in EOC facilities may be greater than 15 per cent of births — indicating that more than 100 per cent of the estimated need is met. One reason that this could happen is that, in reality, more than 15 per cent of pregnant women in the population develop these obstetric complications. Preliminary results from several research studies now under way indicate that this may be the case [Fortney, 1995]. This is especially likely where the incidence of unsafe abortion is very high, because this would substantially increase the proportion of women in that population who develop a complication. Overdiagnosis of complications, which is seen especially in parts of Eastern Europe, could also cause this ratio to be greater than 100 per cent, since it would make the numerator artificially high. It seems unlikely that ‘double-counting’ of women who are admitted to more than one facility, or who are admitted to the same facility more than once during a pregnancy, will seriously bias the results. In any case, if they did, the effect would be to give a more positive impression of the health system than it merits, rather than an unfairly negative impression. 34 E. Caesarean sections as a proportion of all births An indicator of whether EOC facilities are, in fact, providing life-saving obstetric services is the number of Caesarean sections as a proportion of all births. In many facilities in developing countries, not all EOC procedures are recorded. Of all the procedures used to treat the major obstetric complications, Caesarean sections are the easiest to study. This can be done using existing data, such as operating theatre log books, which are often the most complete records available. Annual reports and operating theatre records were examined for 10 rural hospitals in Kenya, Tanzania, southern Sudan, and Ethiopia for the years 1979–1981 [Nordberg, 1984]. It was assumed that Caesarean section is necessary in 5 per cent of deliveries. In order to meet the needs of women in the catchment area of those hospitals, 200–250 Caesarean sections annually per 100,000 population would have been necessary. The data showed, however, that only about one tenth that number were performed. In 1992, a UNICEF study in India summarized available data to determine the percentage of all births that were Caesarean sections in three districts: Barabanki, Uttar Pradesh; Bhilwara, Rajasthan; and Raisen, Madhya Pradesh. While it was estimated that at least 5 per cent of all births would have needed to be Caesarean sections, the data showed that Caesarean section rates in the three districts were well under that level [Nirupam, 1992]. The use of the proportion of births that are Caesarean sections as an indicator is somewhat controversial because the procedure is sometimes overutilized. While this operation can be convenient and lucrative for physicians, it is dangerous and expensive for their patients. Of the countries where excessive use of Caesarean sections has been documented, most are industrialized countries — but not all. A study was done of levels of Caesarean section in hospitals in 14 countries during the mid-1980s. Of all the countries studied, those with the highest proportion of births by Caesarean section were Brazil (32 per cent) and the United States (19 per cent). In only two countries did Caesarean sections account for less than 10 per cent of hospital deliveries: Japan and Czechoslovakia (7 per cent each). Between 10 and 13 per cent of births were Caesarean sections in most of the countries, including Scotland, Denmark, Spain, Sweden, Greece, England and Wales, and New Zealand [Notzon, 1990]. Overuse of Caesarean sections should be discouraged for many reasons. First of all, Caesarean section (like any major surgery) carries a substantial risk of injury and even death for the patient. This risk needs to be weighed against the potential benefits of the surgery. In the case of obstructed labour, the benefits definitely outweigh the risks. Without a Caesarean section most women with this 35 Minimum and maximum acceptable levels: As a proportion of all births in the population, Caesarean sections should account for not less than 5 nor more than 15 per cent. condition will either die or be severely maimed [WHO, 1991]. For women who do not have such severe complications, however, the risks often outweigh the benefits [Fortney, 1987a]. In setting acceptable levels for Caesarean sections, it seems appropriate to have both a minimum and a maximum. Five per cent of all births in the population is a relatively conservative lower limit. For the upper limit, 15 per cent seems reasonable. It is slightly higher than the level in most developed countries, but less than the level in those countries known to have problems with excessive use of this procedure. These minimum and maximum levels have been adopted for global use by the Technical Working Group assembled by WHO [WHO, 1994b]. Even specifying both a maximum and a minimum acceptable level for Caesarean sections does not prevent this important procedure from being misused. For example, a particular country might have 8 per cent of all births being accomplished by Caesarean section, which is well within the acceptable levels. Nevertheless, it might be that in the large cities, half of all women have Caesarean sections, while in rural areas the proportion is well under 5 per cent. Some people might consider the fact that the indicator can conceal abuse of Caesarean section a reason not to use this particular criterion at all. Other approaches, however, are possible. One is to look more closely at the data. For example, the proportion of births that are by Caesarean section could (and, perhaps, should) be analysed by subnational areas. As was true of data on the amount of EOC services, the smaller the unit of analysis, the more likely one is to be able to detect important discrepancies. Even so, aggregated data can still conceal important discrepancies. For example, even in a poor region, an unacceptably high proportion of private patients may be having Caesarean sections, but this may be masked in averages if there are low levels among the public service patients. Consequently, the ultimate responsibility for seeing that Caesarean sections are performed only when necessary lies with clinicians. The chief of obstetrics in a hospital should review the indications for Caesareans that are being done. Training and supervision by senior physicians can also be important in maintaining standards. National societies of obstetricians and gynaecologists should set standards and discourage excessive use of this procedure. Also, consumer groups can raise the awareness of both the public and the medical community. 36 While data on Caesarean sections need to be interpreted with caution, they do have the advantage of being available where information on complications is not. In some situations, it will be necessary to use the proportion of births that are Caesarean sections as a proxy for met need for EOC while countries begin to gather information on complications. Certainly, if the national or regional data show that less than 5 per cent of births are by Caesarean section, this means that some women with life-threatening complications are not receiving necessary care. 3.2.2. Performance of EOC facilities The previous sections have focused on coverage of the population by EOC services. If a country meets all of these criteria, then one can say that (1) a reasonable number of EOC facilities exist; (2) they are reasonably well distributed within the country; (3) they are serving a reasonable proportion of women; (4) they are serving the kinds of women who need them most (i.e., women with obstetric complications); and (5) they are actually providing life-saving obstetric services such as Caesarean sections. Having determined that the country has an acceptable level of EOC, the next issue is the quality of the services provided. Quality of care is the subject of a growing and complex literature. In the present context, we will use relatively crude indicators of performance. Of course, it would be valuable to the national programme to supplement this information with other kinds, such as information gained from in-depth analyses (e.g., case reviews of deaths) and from qualitative studies. A. Case fatality rates A case fatality rate (CFR) is the number of deaths from the condition of interest, divided by the number of people with that condition. In this context, the term means the number of maternal deaths among women with obstetric complications in the health facility being studied. Ideally, one would calculate a separate cause-specific CFR for each of the major causes of maternal death. However, the number of maternal deaths in a given facility is usually too small to allow a stable CFR to be determined for each complication. Therefore, in most facilities only one CFR will be calculated. This indicator of performance has not been frequently used, even though it is relatively easy to calculate. A search of several literature collections and computerized databases found almost no articles containing information on overall case fatality rates. (Most of the studies that are published refer to specific complications.) The available data, presented in Figure 15, indicate that there is a wide gulf between case fatality rates in developed countries and those in developing countries. 37 Figure 15. Case fatality rates: Deaths per 100 complicated obstetric admissions or deliveries in EOC facilities Location Year Deaths per 100 admissions Nigeria Enugu 1983 3.3 1988 3.2 Ota 1983–1986 5.0 1987–1990 8.0 Zaria 1983 3.7 1985 2.5 1988 4.0 Ghana Kumasi 1981 1.9 1989 1.2 Sierra Leone Bo 1987–1989 2.0 United States hospitals* 1970 0.05 1978 0.03 *Uses deliveries with an obstetric complication listed as denominator. Note: All numerators contain at least 18 cases. Sources: PMM Network, 1995; Petitti et al., 1982. Deaths among women with complications in West African hospitals in the late 1980s ranged from a low of 1.2 per cent in Kumasi, Ghana, to a high of 8.0 per cent in Ota, Nigeria [PMM Network, 1995]. In contrast, a study of 654 US hospitals showed a case fatality rate of 0.05 per cent of complicated deliveries even in 1970. By 1978, the rate had declined even further, to 0.03 per cent [Petitti et al., 1982]. l The WHO Technical Working Group decided that this indicator is promising but required further study. 38 Maximum acceptable level: The case fatality rate among women with obstetric complications in EOC facilities should not exceed 1 per cent. Given these data, it seems that 1 per cent is a reasonable maximum acceptable level. It falls in the large gap between the rates from Africa and those from the United States. Since 1 per cent is the maximum acceptable case fatality rate, even countries meeting this level should strive to reduce the rate to less than 1 per cent.l However, in some situations, circumstances beyond the control of the hospital management may make it difficult to achieve a CFR below 1 per cent. The important objective here is the effort to progressively reduce the CFR. The case fatality rate can be calculated by any facility that meets three conditions: obstetric complications cases are treated there; maternal deaths may take place there; and there are adequate records on both of these kinds of events. Case fatality rates for Basic EOC facilities are difficult to interpret because women at risk of death may be referred to Comprehensive EOC facilities. Therefore, for monitoring purposes, case fatality rates should be calculated only for Comprehensive EOC facilities. As we gain experience in gathering and interpreting case fatality information from a variety of settings in developing countries, we will see whether certain limitations should be suggested when comparing CFRs from different institutions or settings. For example, it may not be valid to compare CFRs from district and teaching hospitals, since women with the most serious complications may be referred to the teaching hospital at the last moment, where they die. This would lower the CFR at the district hospital and raise it at the teaching hospital. One simple way to expose such patterns is to analyse data from various kinds of facilities (or from different areas) separately before combining them. Also, in addition to calculating averages, it can be very informative to put the data on bar charts or ‘scattergrams’, either creating a separate graph for each category of facility or using different colours for different categories in the same graph. It is true, however, that the CFR can be high even when the facility is functioning well — e.g., when many women in need of EOC arrive in very poor condition. One way to disentangle the components of the CFR is to gather information on other indicators of quality of care. For example, the time interval from admission-to-treatment can be analysed (either for all complications or for a subset, such as prolonged/obstructed labour). Although there is only a little experience with this statistic, data from West Africa show that, in general, facilities with long waiting periods for treatment of obstetric complications also have relatively high CFRs [PMM Network, 1995]. 39 A somewhat more complicated, but very informative, exercise is to gather information about the condition of the women on admission (e.g., pulse, blood pressure and temperature). This would also help disentangle the effect of patients' condition on arrival from that of the quality of care. Of course, CFRs do not take into account deaths outside the health system. This does not affect the validity of this indicator, because we are using it only to give us a sense of the performance of the EOC facility. If the coverage indicators show that EOC services are well distributed and well utilized, and CFRs are low, then it is safe to say that the maternity care system in the country is working fairly well. If the CFR is high, then further studies should be done to investigate why. Such studies need not, however, be part of national monitoring. 40 4 COLLECTING DATA FOR PROCESS INDICATORS In this edition of the monitoring Guidelines, detailed information on collecting data for impact indicators has been moved to Appendix A because, in most situations, impact indicators present serious methodological challenges to monitoring progress in reducing maternal mortality. Briefly, the main reasons for this are the following: ! Many of the countries with high maternal mortality do not have vital registration systems that can provide adequate data on maternal deaths. ! Even in countries with complete reporting of deaths, maternal deaths are often misclassified. ! The most cost-effective of the survey methods for obtaining information on maternal deaths (the ‘sisterhood’ method) provides estimates for a point in time 6 to 12 years before the study. Consequently, countries that are now conducting sisterhood studies of maternal mortality (e.g., as part of a Demographic and Health Survey) will need to wait at least that long before they can gather new data for trend analysis. ! Impact data do not provide information that can be used to guide programmes. They provide no information on which aspects of a programme are going well and which need improvement. For these and other reasons, the emphasis here is on process indicators. 4.1. Types of Data Required The use of process indicators in this field is not new. However, the Guidelines propose a ‘new generation’ of indicators that focus specifically on availability, utilization, and quality of EOC — factors causally related to maternal deaths. The more familiar process indicators, on the other hand, measure factors that may be important to women's health but are not causally related to maternal death. Appendix B discusses some of those indicators. In order to construct the process indicators proposed in this document (presented in section 3.2), three kinds of data are needed: population data, data on birthrates and health facility data. Figure 16 shows how the process indicators are composed of such data. 41 Figure 16. Types of data used to construct process indicators Type of data Indicator 1 Number of EOC facilities per 500,000 population Indicator 2 Geographical distribution of EOC facilities Indicator 3 Proportion of all births in EOC facilities Indicator 4 Met need for EOC: Proportion of women with complications treated in EOC facilities Indicator 5 Caesarean sections as a per cent of all births Indicator 6 Case fatality rate Population size X X Birthrate X X X Health facility data EOC ‘signal functions’ X X Number of births X Number of complicated cases X X Number of Caesarean sections X Number of maternal deaths X Note: EOC = essential obstetric care. 42 Information on population and on birthrates is available in most countries. Gathering information on health facilities, however, will be more difficult in some situations. Fortunately, the task is simplified by the fact that only those facilities providing essential obstetric care need to be counted for the present purposes. The names used to refer to such facilities will vary from place to place. In some countries, ‘health centres’ might provide services that would qualify as Basic EOC. In other countries, ‘maternities’ might be more likely to perform Basic EOC functions. There will, of course, be variation within countries as well. For example, health centres may be better staffed and equipped in some areas than in others. The emphasis here is on the EOC services that a facility is actually providing, rather than on what it is supposed to be able to provide. Recently, several checklists have been developed that can be helpful in assessing the type and level of care that can be provided by different health facilities [WHO, 1994b; Sloan et al., 1995]. However, while a checklist contains information on whether the facility is (theoretically) capable of providing certain services, it cannot gather information on whether the services are actually being provided. For example, in numerous countries, medical students are required to work for the government for 2–3 years after they graduate. They are usually posted to places where more senior physicians do not want to work. These young physicians have had little special training in obstetrics, and they receive little supervision at their posts. Consequently, some of them do not perform manual removal of the placenta or Caesarean section, perhaps for fear of harming the patient. A checklist would show that there is a physician present in the facility, but not whether he or she is actually performing such life- saving procedures. These Guidelines contain forms for collecting data on EOC services that a facility is actually providing. This chapter lays out the steps to collect the data needed for the process indicators. Figure 17 (located at the end of the chapter) provides a summary of these steps, each of which is discussed in detail below. In addition, suggestions are provided about additional data that can be of use in area- level monitoring. Sample data collection forms are included and discussed in each section. 43 4.2. Preparation Most of the data for calculating these indicators will be collected in facilities. In a relatively small country, visiting every hospital should not be too difficult, but in a large country it might not be possible. Visiting every health centre that may provide essential obstetric care would be difficult even in some small countries. Therefore, in most countries, a subset of EOC facilities will need to be selected for review. We hope that in a few years the kinds of information required for these process indicators will be routinely reported to ministries of health, in which case the data from all facilities would already be compiled and available. The steps described in this section and the next will help to identify a set of facilities that gives a reasonably accurate picture of the situation, while at the same time not requiring an unreasonable amount of work. In countries where financial and human resources are tightly constrained, the approach described below will suffice to yield informative data about the maternity care system. Ensuring that the facilities selected for review provide a fairly accurate picture of the situation depends largely on avoiding two major pitfalls: systematic bias and the effects of chance variation. Systematic bias can occur when conscious or unconscious factors affect selection of facilities for study. For example, the people selecting the facilities might want to present the situation in the most favourable light possible, or they might select facilities that are easily accessible (e.g., on a paved road or near a large town). In either case, the data collected might give an overly favourable impression. The effects of chance are, of course, unpredictable, but they do tend to diminish as the number of facilities studied increases. The selection process will be done in two stages: selecting areas of the country for study and then selecting facilities within these areas. Sections 4.2.1 and 4.2.2 present a guide for selecting areas for study, which will be done at the national level. The selection of facilities within those areas will be done at the area level and is addressed in Sections 4.3.1 and 4.3.2. 4.2.1. Determine the number of areas to be studied Consider a level smaller than ‘national’. The term for this administrative level will vary by country — e.g., state, province — and will be referred to here as an ‘area’. The following guidelines should be used to determine whether or not to study all areas of a country: m In a few countries where the administrative units of the ‘province’ or ‘state’ are exceptionally large, it may be necessary to select sub-areas for study. Again, as a rough guideline, if an area has more than 100 hospitals (public and private), sub-areas may be selected, and the number of sub-areas studied should represent at least 30 per cent of the total. For the purposes of the forms, each sub-area should be considered an ‘area.’ Professional help from a statistician should be sought in obtaining national estimates in countries where sub-areas are selected. 44 If a country has 100 or fewer hospitals (public and private), then study all areas. If a country has more than 100 hospitals (public and private), then a subset of areas may be selected for study. Select as many areas as possible, but the number selected should be at least 30 per cent of the total number of areas in the country.m If selecting a subset of areas, the aim should be to study as many areas as possible, without compromising the quality of the data collected. For example, if there are 21 areas in country W, 10 might be selected for study. Fewer may be studied if resources are scarce, but the proportion selected should not be less than 30 per cent, or a minimum of seven areas. 4.2.2. Random selection of areas In order to avoid bias, described above, the basis for selection of areas within each type must be random. The procedure for random selection is outlined below. Step 1. Make a list of all areas in the country. The list should be in alphabetical order, to minimize the possibility of bias. Step 2. Assign each area a consecutive number, starting with the number 1 for the first area on the list. Step 3. Calculate the ‘sampling interval’. The sampling interval will tell you to select every nth area, once the first area has been selected at random. Use the following formula: total number of areas in the country Sampling interval = divided by number of areas selected In country W there are a total of 21 areas, of which 10 are to be selected for study, giving a sampling interval of 2 (21/10 = 2.1). 45 Note: Sampling intervals should be rounded to the nearest whole number. If, for example, it had been decided that only 15 of the 21 areas would be studied, the sampling interval would be 1.4, which would therefore round down to 1 — an indication that either fewer areas should be selected for study or all areas should be included in the sample. Step 4. Identify the first area to be included in the sample by generating a random number that is less than or equal to the sampling interval but greater than zero. This can be done using a random number table (Appendix C). To use the table, look away from the page and touch it with the point of a pencil. The digit closest to where the pencil touches the page is the random number. If the digit is less than or equal to the sampling interval and greater than zero, use it; if not, read from left to right until a digit that satisfies this condition is reached. This number will be the first area selected. For country W, the sampling interval is 2. Using the random number table, our pencil point falls on the digit 7, at row 21, column 33. This is larger than our sampling interval, so we read left to right, passing the digits 0, 4 and 6, until we come to 2. Thus, area #2 on the list will be the first area selected. Step 5. Identify all other areas to be included in the sample by adding the sampling interval to the number that located the first area and continue to select areas until the desired number has been reached. Since the first selected area is #2 on the list of areas, the next one would be 2 plus 2, or #4, and the next #6, and so on, until 10 areas have been selected. 4.2.3. Determine a nationally uniform 12-month period to be studied The data collected from facilities will be retrospective, but the 12-month period selected should be a recent one, to help ensure that the data will still be available. For comparability of data, it is important that all data collected throughout the country be from the same 12-month period. The decision about which period to use should be made at the national level, and should then be entered at the top of Form 2 before the form is duplicated for use. This will ensure that data collection at all facilities will focus on the same time period. The 12-month period may be either a calendar year (e.g., January 1, 1994–December 31, 1994) or any other 12-month period (e.g., June 1, 1994–May 31, 1995). Once areas for study have been selected, Forms 1–4, including all worksheets, should be duplicated and a complete set sent to the person coordinating the research in each area. 4.3. Form 1: List All Possible EOC Facilities/Providers in Study Area The first step in gathering the required data is to make a list of all the facilities/providers within the study area that may be providing EOC services — either Basic or Comprehensive — as defined by 46 the signal functions (Section 3.2). In some circumstances, trained health care providers may provide EOC services outside of health facilities — e.g., midwives delivering babies in women’s homes. These providers will generally be affiliated with a health facility. In this case, special care should be taken to see that their records (e.g., the midwives’ registers) are examined for information when completing Form 2. If such providers are not operating out of a facility, then they should be listed as though they were. The objective is to include the full range of EOC providers in the data collection. A facility/provider that may be providing EOC services is one that is either: (1) on the Ministry of Health's list of hospitals or facilities/providers that should be providing at least Basic EOC; (2) on a list of private hospitals or facilities/providers that might be providing at least Basic EOC; or (3) known to the area Medical Officer as possibly providing at least Basic EOC. This list should be as complete as possible so that no likely provider of EOC is overlooked; however, care should be taken to avoid double-counting. Worksheets 1a–b can be used for this purpose. The worksheets should be used to list all of the different facilities/providers — hospitals, maternities, health centres, clinics, trained midwives working at the village level and other types — that may be providing Basic or Comprehensive EOC in the area. Since each worksheet has enough space to list 17 facilities/providers, it is likely that copies of each worksheet will have to be made and the lists of each type of facility/provider will be several pages long. Form 1 summarizes the numbers of facilities/providers listed on Worksheets 1a–b. 4.3.1. Determine the number of EOC facilities to be reviewed In a relatively small area, visiting every hospital may be feasible, while in larger areas it will not. Even in small areas, it will often be difficult to visit every lower level facility that might be providing Basic EOC. Thus, within most areas, a subset of EOC facilities must be selected for review. In order to avoid bias, this second stage of selection should also be done randomly. The criteria below can be used in deciding whether to study all facilities or to select a subset for review: Possible Comprehensive EOC facilities: If there are 25 or fewer, study all of them. If there are more than 25, a subset may be selected for study. Select as many as possible, but the number should represent at least 30 per cent, and should not be smaller than 20. 47 Possible Basic EOC facilities: If there are 100 or fewer, study all of them. If there are more than 100, a subset may be selected for study. Select as many as possible, but the number should represent at least 30 per cent. In area X, there are not too many possible Comprehensive EOC facilities — 48. Although the number is greater than 25, it is decided that it is feasible to visit all of them. However, there are 390 possible Basic EOC facilities, and it would be too difficult and costly to visit all of them, so a subset of these facilities must be selected for review. If a subset of either type of facility is to be selected, the number to be visited must be decided. As described above, this number should be as large as possible in order to minimize the effects of chance variation, and should be at least 30 per cent of all facilities of each type. In determining the number of facilities to visit, it is important to strike a good balance between the number of facilities and the quality of the data that will be collected from them. In other words, the number of facilities selected should be as large as possible while still allowing for careful data collection at each facility. In area X, it is decided that all 48 possible Comprehensive EOC facilities will be visited, and that 40 per cent of possible Basic EOC facilities will be selected for review. Thus, 156 (.4 x 390) possible Basic EOC facilities will be selected. 4.3.2. Random selection of facilities Once the number of facilities to be visited has been decided, the next step is to select the actual facilities for review. To minimize the chance of bias, this should be done randomly, in a procedure similar to that followed for selecting areas. If all possible Comprehensive and all possible Basic EOC facilities will be visited, this step will not be necessary. If a subset of both types of facilities will be selected, the random selection procedure should be carried out separately for each. The procedure is outlined below. The random selection will be done using all copies of Worksheets 1a and/or 1b that have been filled out for the geographical area in question. Step 1. Assign each facility a consecutive number. (Note: In order to minimize the possibility of bias, it is best to have facilities listed in alphabetical order before numbering them.) Step 2. Calculate the sampling interval. The sampling interval will tell you to select every nth facility, once the first facility has been selected at random. Use the following formula: 48 number of facilities in the area Sampling interval = divided by number of facilities to be selected In area X there are a total of 390 possible Basic EOC facilities, of which 156 are to be selected for study, giving a sampling interval of about 3 (390/156 = 2.5). Note: Sampling intervals are rounded to the nearest whole number. Step 3. Identify the first facility to be included in the sample by generating a random number that is less than or equal to the sampling interval but greater than zero. This can be done using a random number table (Appendix C). To use the table, look away from the page and touch it with the point of a pencil. The digit closest to where the pencil touches the page is the random number. If the digit is less than or equal to the sampling interval and greater than zero, use it; if not, read from left to right until a digit that satisfies this condition is reached. This number will be the first facility selected. For possible Basic EOC facilities of area X, the sampling interval is 3. Using the random number table, our pencil point falls on the digit 4, at row 15, column 22. This is larger than our sampling interval, so we read left to right, passing the digits 0, 7 and 5, until we come to 1. Thus, facility #1 on the list of possible Basic EOC facilities will be the first area selected. Step 4. Identify all other facilities to be studied by adding the sampling interval to the number that located the first facility. Continue to select facilities until the desired number has been reached. If you come to the end of a list in the selection process, continue on back to the beginning of the list, but do not count those facilities that have already been selected. Since the first selected facility is #1 on the list, the next one would be 1 plus 3, or #4, and the next #7, and so on. Facility #388 will be the 129th facility selected, and facility #3 will be the 130th (since #1 has already been selected and should not be counted in the second pass through). Every third facility will continue to be selected in this way until all 156 have been selected. Once the facilities to be reviewed have been selected, site visits to gather data at each of these facilities can begin. 4.4. Form 2: Conduct Site Visits to Assess EOC Actually Being Provided A copy of Form 2 and Worksheets 2a and 2b should be used at each facility to record the type and amount of services provided. The information compiled on the form will enable research staff to determine whether a given facility is actually providing EOC services and, if it is, whether these are Basic or Comprehensive. The same forms also elicit information needed to assess EOC coverage and 49 performance. Except for data on population size and birthrate, all the information needed to construct the process indicators is contained in Form 2 and Worksheets 2a and 2b. Note: There has been some discussion of the fact that the minimum acceptable level for Caesarean sections might be somewhat lower than 5 per cent if it included only operations performed for maternal indications. Unfortunately, in developing countries it is often very difficult to obtain this information from records. 4.4.1. Notes on data collection using Form 2 (Worksheets 2a and 2b) A. Collecting data on complicated cases Worksheet 2a should be used in conjunction with Form 2 for recording complications. Depending on the size of each facility and the quality of its records, it may be too difficult to collect the necessary information for the entire year. Therefore, the worksheet presents two other plans, to be used when necessary. ! Plan 1 should be followed whenever possible. This entails completing the grid on Worksheet 2a to record the number of each type of complication at the facility during each of the 12 months being studied. ! Plan 2 can be followed if the facility's patient volume is so large that gathering this information for all 12 months would be too time-consuming (e.g., if there are more than 100 admissions to the obstetric ward per month). This plan uses a sample of four months distributed throughout the year, and then multiplies by three to estimate the total number of complications for the year. ! Plan 3 should be followed only if the records at the facility do not contain the information needed to follow Plan 1 or 2. Plan 3 entails using commonly available information — the total number of deliveries in the facility and the number of ‘normal’ deliveries. The number of ‘normal’ deliveries in the study period is subtracted from the number of total deliveries, which yields the number of ‘non- normal’ deliveries. This number is then multiplied by a correction factor (1.25), and the resulting number is a proxy for the number of women with obstetric complications. The correction factor is applied because the number of non-normal deliveries is likely to underestimate the number of women with major obstetric complications admitted to the facility. The number of non-normal deliveries will fail to include women admitted for at least three of the major obstetric complications: post-partum and antepartum haemorrhage, post-partum sepsis and complications of induced abortion. On the other hand, non-normal deliveries will include a certain number of complications that are not among those being used here to define a complicated case (e.g., non-obstetric illnesses occurring during pregnancy or post-partum). Depending on how hospital records are kept, non-normal deliveries may also include events such as multiple births or even deliveries done with episiotomy. Thus, Plan 3 is likely to produce a liberal estimate of the number of women with complications receiving treatment at a facility. 50 B. Collecting data on maternal deaths Worksheet 2b is used in conjunction with Form 2 for recording maternal deaths. To ensure that all maternal deaths that occurred in the facility during the 12-month period are recorded, all relevant sources of information should be investigated, including (but not limited to) maternity ward death registers, morgue record books and emergency room records. While only those maternal deaths due to the direct obstetric complications specified earlier will be used in calculating case fatality rates, other maternal deaths discovered in these investigations may still be informative to facility managers. 4.5. Form 3: Summarize Findings for Basic and Comprehensive EOC Facilities After copies of Form 2 have been used to gather data from EOC facilities, the forms should be collected and sorted into three groups, based on the findings in Box B (‘Facility's Actual EOC Status’) at the top of the first page: ! facilities actually providing Comprehensive EOC; ! facilities actually providing Basic EOC; and ! facilities not providing EOC. The next step should be to summarize these findings for the area. Form 3 is used for this purpose. The form has two parts. Part A — a straightforward summary of the data collected from facilities — should be used only if all possible Basic and Comprehensive facilities in the area were visited (that is, no selection of facilities was done). Worksheets 3a–b will help in creating this summary. Part B of Form 3 should be used if some facilities were not visited. Because it uses data from areas where a subset of all facilities were selected for study, an intermediate step is necessary to convert the data collected into estimates for all facilities in the area. Worksheet 3c (in addition to Worksheets 3a–b) should be used for this intermediate step. Thus, for each area included in the study, one copy of Form 3 will be filled out, using either Part A or Part B. 4.6. Form 4: Calculate Indicators for the Area Once the findings from site visits have been summarized, Form 4 can be used to calculate the indicators for the area. This form lays out the steps for using the information summarized in Form 3. A summary checklist to determine whether or not each indictor meets acceptable levels is part of this form. While, ultimately, data from facilities will be aggregated to calculate the indicators for the whole country, the area-level indicators provide important information. First, they provide useful 51 information for setting programme priorities at the area level. An entire set of completed Forms 1–4 should be maintained at the area level for this purpose. Second, these indicators will allow for comparisons across study areas at the national level. Using the information obtained for each study area, national-level researchers can examine the differences in EOC coverage, utilization and performance in different areas of the country. This, in turn, may have important implications for policy and programming priorities. 4.7. Form 5: Calculate Indicators for the Country In order to calculate the process indicators for the country as a whole, the national-level researchers will need to collect copies of all Forms 1–4 (including worksheets) from each of the study areas. All the information needed for this final step — calculating the indicators for the country — is summarized on Form 5 and Worksheets 5a–c. Worksheets 5a–c are designed to organize the data needed to calculate the indicators for the country. Worksheet 5a summarizes information on amount of EOC facilities for all areas studied. Worksheet 5b does the same for deliveries, complications and Caesarean sections. Worksheet 5c summarizes obstetric complications and deaths in Comprehensive EOC facilities studies, for the calculation of CFR. Finally, the indicators for the country as a whole are determined on Form 5. As with Form 4 for the calculation of indicators at the area level, a summary checklist for whether each indictor meets or does not meet acceptable levels is provided. Once the indicators have been calculated, the important last step is interpretation. Chapter 5 is a guide to interpreting the findings. This chapter begins with several general notes on interpretation of process indicators. Then interpretation of each of the indicators is addressed. 4.8. Notes for Area-level Monitoring Area officials and planners may be interested in a greater level of detail than is required for national monitoring. Therefore, during the site visits to facilities, it may be useful to add some questions. This may be done by attaching an extra sheet to Form 2 (EOC facility review). A discussion of some types of additional data that might be of interest follows. It is important, however, that all the data required for the calculation of the indicators be collected in a uniform manner for the whole country. So, while questions may be added to Form 2, none of the existing questions should be modified or deleted. 4.8.1. Levels of functioning among facilities For monitoring purposes, it is crucial that only facilities that are fully functioning as Basic or Comprehensive EOC facilities (i.e., facilities that are performing the signal functions in Figure 12, page 28) are included in the analysis. However, area planners may also be interested in knowing how many facilities in the area are close to being able to function as Basic or Comprehensive EOC facilities. It may be desirable, therefore, to keep a separate record of such facilities. This would be especially useful if the analysis of the process indicators reveals a shortage of EOC facilities. In that 52 case, information about which facilities are already close to providing EOC can be used when planning which facilities to upgrade. 4.8.2. Time availability of services Another factor that area officials may wish to examine is the availability of EOC at those facilities that are already fully functioning. For example, a question that asks about the hours per day and days per week that the procedures defined as signal functions are actually available might be added to the facility review form (Form 2). Since obstetric complications are unpredictable, it is important that women have access to life-saving EOC procedures around the clock. Analyses of local patterns in EOC availability may show that EOC coverage is actually lower than the number of facilities would imply. In such cases, expanding the hours when services are available is strongly recommended. 4.8.3. Geographical distribution of services within areas The geographical distribution of EOC facilities also affects the accessibility of EOC services. While the total number of facilities in the area may meet or exceed the minimum acceptable level, there may be smaller geographical regions that have too few, or no, EOC facilities. At the area level, therefore, identifying the location of facilities on a map may be desirable. By mapping Basic and Comprehensive EOC facilities, planners can identify local areas where women do not have access to EOC, either because facilities do not exist or because existing facilities are not accessible (e.g., because of poor or non-existent roads and bridges, etc.). 4.8.4. Differences between public-sector and private-sector facilities While no distinction is made between public and private sector at the national level, area-level planners may be interested in examining differences between the two types of facilities. Such differences can have important implications for programming. For example, one might want to know the proportions of women with complications who are receiving EOC in public versus private facilities, or which types of facilities are performing more EOC signal functions. One might also examine differences in case fatality rates in hospitals by type of facility. 4.8.5. Quality of care at facilities As discussed earlier, case fatality rates provide a crude indicator of the level of performance at EOC facilities. Area researchers or administrators may therefore wish to collect additional information to gain more insight into the quality of care provided at local facilities. One approach is to collect data on the interval between the time a woman is admitted to an EOC facility and the time she actually receives treatment. (This information corresponds to the third delay in the three-delays model described in Chapter 2.) Studies in Comprehensive EOC facilities in West Africa used both chart reviews and prospective data collection to obtain information on this interval. By looking at trends in the admission-to-treatment interval, and by investigating the factors 53 contributing to delays in individual facilities, the researchers were able to identify specific problem areas that could be addressed to improve quality of care [PMM Network, 1995]. In-depth case reviews or audits of both maternal deaths and ‘near misses’ can also provide valuable information about quality of care. Case reviews and audits also have the advantage of identifying problem areas within facilities and suggesting possible remedies. WHO is developing guidelines on case reviews and audits. 4.8.6. Quality of facility records Area-level officials should examine the method by which the number of women with complications is derived in the facility review forms (Worksheet 2a). The worksheet offers three plans for arriving at this number (see discussion in Section 4.4.1.); Plan 3 is to be used only when Plans 1 and 2 are not feasible because the facility lacks records on women with complications. The estimate of women with complications obtained using Plan 3 is the difference between total deliveries and ‘normal’ deliveries, multiplied by a correction factor of 1.25. This produces a crude estimate. Area-level officials should target facilities where Plan 3 was used for improvements in record-keeping. If, in the data collection process, it is found that many facilities in an area required the use of Plan 3, a broad effort to put improved record-keeping systems in place (e.g., workshops, revised registers) should be considered. Even where Plan 1 or Plan 2 is used, it is likely that some facilities are actually treating more women with obstetric complications than their records indicate. On the facility review form (Form 2), question 14 asks the reviewer to give an informed opinion about the completeness of the facility's records. Area-level officials may be interested in examining the responses to this question for facilities in their area. If it appears that records are incomplete in a number of facilities, it may be desirable to hold a workshop on facility record-keeping. Having complete and accurate records will facilitate future monitoring efforts. 54 Figure 17. User's guide to the data collection forms Form # Level of action Actions required Use None National National National Select areas for study, if necessary. Determine nationally uniform 12-month period to be studied and enter on Form 2. Duplicate Forms 1–4 (with worksheets) and distribute to areas selected. Text 4.2. 1 Area* Area List all possible facilities providing EOC in area. If necessary, select facilities to be visited. Worksheets 1a–b Text 4.3. 2 Local Conduct site visits to facilities. Worksheets 2a–b Text 4.4. 3 Area Area Area If a sample of facilities was visited, count: ! Possible Basic EOC facilities visited ! Possible Comprehensive EOC facilities visited Separate facilities into three groups: ! Actual Comprehensive EOC facilities ! Actual Basic EOC facilities ! Not EOC Summarize findings from Basic and Comprehensive EOC facilities. Forms 1 & 2 Worksheet 3c Form 2 Worksheets 3a–c Text 4.5. 4 Area Area Calculate indicators for area. Interpret. Form 3 & Worksheet 3b Text 4.6. Text Chapter 5 5 National National National Collect completed Forms 1–4 (with worksheets) from all study areas. Calculate indicators for entire country. Interpret. Forms 3 & 4, Worksheet 3b Text 4.7. Text Chapter 5 55 * ‘Area’ refers to the administrative level in the country being used for monitoring — e.g., state, province. 56 You will need to complete Worksheets 1a–b BEFORE filling in the totals below. FORM 1FORM 1 LIST OF POSSIBLE ESSENTIAL OBSTETRIC CARE (EOC) FACILITIESLIST OF POSSIBLE ESSENTIAL OBSTETRIC CARE (EOC) FACILITIES 1. Name of area: 2. Population size of area: 3. Sources of information: (list additional sources on separate sheet) 4. Form completed by: Name: Title: 5. Form completed on: Date: / / / / 1.1. Total number of possible BASIC EOC facilitiesTotal number of possible BASIC EOC facilities == (Add sheet totals from all copies of Worksheet 1a.)(Add sheet totals from all copies of Worksheet 1a.) 2.2. Total number of possible COMPREHENSIVE EOC facilitiesTotal number of possible COMPREHENSIVE EOC facilities = (Add sheet totals from all copies of Worksheet 1b.)(Add sheet totals from all copies of Worksheet 1b.) BasicBasic EOC includes the following procedures: Parenteral administration of medications (antibiotics, oxytocics, sedatives); manual removal of placenta; removal of retained products; and assisted vaginal delivery (vacuum extraction, forceps). ComprehensiveComprehensive EOC includes all of the procedures of Basic EOC plus surgery (Caesarean section, curettage, hysterectomy) and blood transfusion. 57 WORKSHEET 1aWORKSHEET 1a LIST OF FACILITIES WHERE LIST OF FACILITIES WHERE BASICBASIC EOC MIGHT BE PERFORMEDEOC MIGHT BE PERFORMED Area: Area: This worksheet should be used to list all facilities in the area that might be providing BasicBasic EOC. Possible Comprehensive EOC facilities should be listed on Worksheet 1b. Do not list any facility twice.Do not list any facility twice. BasicBasic EOC includes the following procedures: Parenteral administration of medications (antibiotics, oxytocics, sedatives); manual removal of placenta; removal of retained products; and assisted vaginal delivery (vacuum extraction, forceps). Facility nameFacility name LocationLocation Government/PrivateGovernment/Private Sheet total of facilities where Basic EOC might be performed = 58 WORKSHEET 1bWORKSHEET 1b LIST OF FACILITIES WHERE LIST OF FACILITIES WHERE COMPREHENSIVECOMPREHENSIVE EOC MIGHT BE PERFORMEDEOC MIGHT BE PERFORMED Area: Area: This worksheet should be used to list all facilities in the area that might be providing ComprehensiveComprehensive EOC. Possible Basic EOC facilities should be listed on Worksheet 1a. Do not list any facility twice.Do not list any facility twice. ComprehensiveComprehensive EOC includes all of the procedures of Basic EOC plus surgery (Caesarean section, curettage, hysterectomy) and blood transfusion. Facility nameFacility name LocationLocation Government/PrivateGovernment/Private Sheet total of facilities where Comprehensive EOC might be performed = 59 FORM 2FORM 2 ESSENTIAL OBSTETRIC CARE (EOC) FACILITY REVIEWESSENTIAL OBSTETRIC CARE (EOC) FACILITY REVIEW < 12-month period under review: through = Box A: Facility's Box A: Facility's PossiblePossible EOC Status EOC Status To be done at areaarea level before completion of this form. Circle ONE (Use W.S. 1a-b)Circle ONE (Use W.S. 1a-b) Box B: Facility's Box B: Facility's ActualActual EOC Status EOC Status To be done at facilityfacility level after completion of this form. Circle ONE (Use Q11 Box)Circle ONE (Use Q11 Box) Comprehensive EOCComprehensive EOC Basic EOCBasic EOC Comprehensive EOCComprehensive EOC Basic EOCBasic EOC Not EOCNot EOC 1. Name of facility: 2. Location of facility: 3. Contact information: ! If no data at all are availableIf no data at all are available at this facility, check here: (Skip to last page and sign.) 4. Type of facility: (a) Hospital (b) Maternity (c) Health centre (Check one) (d) Clinic (e) Other (specify) 5. Type of operating agency: (a) Government (b) Private (Check one) 6. Total deliveries during 12-month period 7. Normal deliveries during 12-month period 8. Caesarean sections during 12-month period Complete Worksheets 2a and 2b and enter a total for each of the following items 9. Complicated obstetric cases* during 12-month period (* fill in from Line 9b, Worksheet 2a) Check one (see Worksheet 2a) Plan 1 Plan 2 Plan 3 10. Direct obstetric deaths from selected causes** during 12-month period (** fill in from Line 8, TOTAL, Worksheet 2b) 60 FORM 2FORM 2 (continued)(continued) Check Yes or No for each of the following items (a-h) Box: DeterminationBox: Determination of EOC statusof EOC status (Use Q11. Check only ONE.) 11. Were the following services11. Were the following services performed at least once during the performed at least once during the last 3 months? last 3 months? Yes No ! If ALLALL of 11a–h = Yes, check: (a) Parenteral antibiotics COMPREHENSIVE EOC (b) Parenteral oxytocics (c) Parenteral sedatives/anticonvulsants ! If ALLALL of 11a–f = Yes ANDAND (d) Manual removal of placenta 11g OROR 11h = No, check: (e) Removal of retained products BASIC EOC (f) Assisted vaginal delivery (g) Blood transfusion ! If ANYANY of 11a–f = No, check: (h) Caesarean section NOTNOT EOC 12. What sources of data were used to complete this form?12. What sources of data were used to complete this form? (e.g., maternity ward register, delivery book, general admissions register, etc.) Quality of information:Quality of information: 13. In your informed opinion (from talking to staff, seeing the record system, etc.), what proportion of the complications treated in this facility are recorded on this form? (check one) None Some Most All 14. Date of review: 15. Reviewed by: Name: Title: 61 Facility: Facility: Period: Period: to to WORKSHEET 2aWORKSHEET 2a COMPLICATED OBSTETRIC CASES DURING 12-MONTH PERIODCOMPLICATED OBSTETRIC CASES DURING 12-MONTH PERIOD Indicate with a check which plan is being used (use only one): PLAN 1: PLAN 1: TO BE FOLLOWED WHENEVER POSSIBLETO BE FOLLOWED WHENEVER POSSIBLE < Enter the number of each type of complicated case treated each month during the 12-month period using the grid below. PLAN 2: PLAN 2: TO BE FOLLOWED WHEN IT IS NOT FEASIBLE TO RECORD TO BE FOLLOWED WHEN IT IS NOT FEASIBLE TO RECORD ALLALL COMPLICATIONS COMPLICATIONS (i.e., when this would be too much work) < Enter the number of each type of complicated case treated during the four months underlined — i.e., months 1, 4, 7 & 10. PLAN 3: PLAN 3: TO BE FOLLOWED ONLY WHEN DATA ON COMPLICATIONS ARE NOT AVAILABLE AT THE FACILITYTO BE FOLLOWED ONLY WHEN DATA ON COMPLICATIONS ARE NOT AVAILABLE AT THE FACILITY < Enter the number of deliveries and the number of ‘normal’ deliveries for the 12-month period and skip to question 9 below. Complication If more than one, use the most life-threatening. Month (write in month above each number) 1 2 3 4 5 6 7 8 9 10 11 12 1. Haemorrhage (ante or post-partum) 2. Prolonged/obstructed labour 3. Post-partum sepsis 4. Complications of abortion 5. Pre-eclampsia/eclampsia 6. Ectopic pregnancy 7. Ruptured uterus 8. Monthly totals8. Monthly totals 9. TOTAL COMPLICATED OBSTETRIC CASES 9. TOTAL COMPLICATED OBSTETRIC CASES (Complete only ONE of the boxes below.)(Complete only ONE of the boxes below.) PLAN 1 9b. Sum of monthly totals (Q8, columns 1-12) = PLAN 2 9a. Sum of monthly totals (Q8, columns 1,4,7,10) = 9b. [Q9a] × 3 = PLAN 3 9a. (All deliveries) ! (‘Normal’ deliveries) = 9b. [Q9a] × 1.25* = *Correction factor 62 Facility: Facility: Period: Period: to to WORKSHEET 2bWORKSHEET 2b MATERNAL DEATHS DURING 12-MONTH PERIODMATERNAL DEATHS DURING 12-MONTH PERIOD Use this worksheet to record maternal deaths, by cause, in this facility during the 12-month period covered. When transferring information to Form 2, be sure to use the total direct obstetric deaths from Line 8. Cause of maternal death Month (write in month above each number) If more than one, use the most life-threatening cause 1 2 3 4 5 6 7 8 9 10 11 12 Total 1. Haemorrhage (ante or post-partum) 2. Prolonged/obstructed labour 3. Post-partum sepsis 4. Complications of abortion 5. Pre-eclampsia/eclampsia 6. Ectopic pregnancy 7. Ruptured uterus 8. Total direct obstetric deaths from8. Total direct obstetric deaths from selected causes selected causes (not including Other) (Sum of Questions 1-7)(Sum of Questions 1-7) ** 9. Other (all other causes) 10. Total maternal deaths ** UseUse this total in completing Form 2, Question 9. this total in completing Form 2, Question 9. The case fatality rate (CFR) will be calculated by dividing the number of deaths by the number of complicated cases. To keep the numerator and denominator of the CFR comparable, the deaths used in this calculation are restricted to only those due to the causes used to define a complicated case. 63 FORM 3FORM 3 SUMMARY OF DATA FROM EOC FACILITIES IN AREASUMMARY OF DATA FROM EOC FACILITIES IN AREA This form summarizes all the facilities' data that have been collected on all copies of Form 2 within the area. One copy of this form should be completed for each area. 1. Name of area: 2. Population in area: 3. Birthrate in area: 4. Estimated annual births in area (Q2 × Q3): Complete Complete eithereither Part A Part A oror Part B below. The other part will be left blank. Part B below. The other part will be left blank. If ALLALL facilities in area were visited, complete PART A ONLYPART A ONLY. If a SUBSETSUBSET of facilities in area were selected, complete PART B ONLYPART B ONLY. PART APART A Use Worksheets 3a–b to complete the table below. Column 1 Basic EOC facilities Column 2 Comprehensive EOC facilities Column 3 Total (Col 1 + Col 2) 5. Number of facilities providing EOC (W.S. 3a, Q2) (W.S. 3b, Q2) 6. Number of deliveries in 12-month period (W.S. 3a, Q1a) (W.S. 3b, Q1a) 7. Number of complicated cases treated in 12-month period (W.S. 3a, Q1b) (W.S. 3b, Q1b) 8. Number of Caesarean sections in 12-month period (W.S. 3a, Q1c) (W.S. 3b, Q1c) PART BPART B Complete Worksheets 3a–c. Then use Worksheet 3c to complete the table below. Column 1 Basic EOC facilities Column 2 Comprehensive EOC facilities Column 3 Total (Col 1 + Col 2) 5. Number of facilities providing EOC (W.S. 3c, Q4) (W.S. 3c, Q11) 6. Number of deliveries in 12-month period (W.S. 3c, Q5) (W.S. 3c, Q12) 7. Number of complicated cases treated in 12-month period (W.S. 3c, Q6) (W.S. 3c, Q13) 8. Number of Caesarean sections in 12-month period (W.S. 3c, Q7) (W.S. 3c, Q14) 64 WORKSHEET 3aWORKSHEET 3a SUMMARY OF SUMMARY OF BASICBASIC EOC FACILITY REVIEWS EOC FACILITY REVIEWS Area: Area: This worksheet summarizes all BASICBASIC EOC facilities' data collected on all copies of Form 2. Use Box B at top of Form 2 (‘Facility's EOC Status: Actual’) to identify Basic EOC facilities. Attach additional sheets if necessary. Column 1 Column 2 Column 3 Column 4 Facility Number of deliveries (Form 2, Q6) Number of complicated cases (Form 2, Q9) Number of Caesarean sections (Form 2, Q8) 1. Column totals*1. Column totals* 1a. 1b. 1c. 2. 2. Total number* of BASICBASIC EOC facilities listed in Column 1 = *If more than one sheet was used, add sheet totals to get overall total. 65 WORKSHEET 3bWORKSHEET 3b SUMMARY OF SUMMARY OF COMPREHENSIVECOMPREHENSIVE EOC FACILITY REVIEWS EOC FACILITY REVIEWS Area: Area: This worksheet summarizes all COMPREHENSIVECOMPREHENSIVE EOC facilities' data collected on all copies of Form 2. Use Box B at top of Form 2 (‘Facility's EOC Status: Actual’) to identify Comprehensive EOC facilities. Attach additional sheets if necessary. Column 1 Column 2 Column 3 Column 4 Column 5 Column 6 Facility Number of deliveries (Form 2, Q6) Number of complicated cases (Form 2, Q9) Number of Caesarean sections (Form 2, Q8) Number of direct obstetric deaths (from selected causes) (Form 2, Q10) Facility case fatality rate (CFR) Direct obstetric deaths x 100 Complicated cases (Column 5) ÷ (Column 3) × 100 1. Column totals*1. Column totals* 1a. 1b. 1c. 1d. 2. 2. Total number* of COMPREHENSIVECOMPREHENSIVE EOC facilities listed in Column 1 = *If more than one sheet was used, add sheet totals to get overall total. 66 Area: Area: WORKSHEET 3cWORKSHEET 3c AREA-WIDE ESTIMATES OF EOCAREA-WIDE ESTIMATES OF EOC This worksheet converts the data from the subset of facilities that were selected for site visits into estimates for the entire area. If allall possible Basic and Comprehensive EOC facilities in the area were visited then there is no need to complete this worksheet. BASIC EOC FACILITIES:BASIC EOC FACILITIES: Use Forms 1 and 2 to complete the box below. 1. Number of possible BasicBasic EOC facilities visited (Use all copies of Form 2, Box A at top of p. 1.) 2. Number of possible BasicBasic EOC facilities in area (Form 1, Q1) 3. Proportion of facilities for which data were collected (Q1 ÷ Q2) Use Worksheet 3a to calculate the following estimates for BasicBasic EOC facilities in the area. Total from facilities visited ÷÷ Proportion of Basic EOC facilities visited (Q3 above) == EstimateEstimate for areafor area 4. Estimated number of Basic EOC facilities (W.S. 3a, Q2) ÷ = 5. Estimated number of deliveries in 12-month period (W.S. 3a, Q1a) ÷ = 6. Estimated number of complicated cases treated in 12-month period (W.S. 3a, Q1b) ÷ = 7. Estimated number of Caesarean sections in 12- month period (W.S. 3a, Q1c) ÷ = COMPREHENSIVE EOC FACILITIES:COMPREHENSIVE EOC FACILITIES: Use Forms 1 and 2 to complete the box below. 8. Number of possible ComprehensiveComprehensive EOC facilities visited (Use all copies of Form 2, Box A at top of p. 1.) 9. Number of possible ComprehensiveComprehensive EOC facilities in area (Form 1, Q2) 10. Proportion of facilities for which data were collected (Q1 ÷ Q2) Use Worksheet 3b to calculate the following estimates for ComprehensiveComprehensive EOC facilities in the area. Total from facilities visited ÷÷ Proportion of Comprehensive EOC facilities visited (Q10 above) == EstimateEstimate for areafor area 11. Estimated number of Comprehensive EOC facilities (W.S. 3b, Q2) ÷ = 12. Estimated number of deliveries in 12-month period (W.S. 3b, Q1a) ÷ = 13. Estimated number of complicated cases treated in 12-month period (W.S. 3b, Q1b) ÷ = 14. Estimated number of Caesarean sections in 12- month period (W.S. 3b, Q1c) ÷ = 67 IS ACCEPTABLE LEVEL MET? Minimum acceptable level = 4 per 500,000 population Met Not met Minimum acceptable level = 1 per 500,000 population Met Not met Minimum acceptable level = 15% Met Not met Area: Area: FORM 4FORM 4 CALCULATION OF INDICATORS FOR THE AREACALCULATION OF INDICATORS FOR THE AREA Use Form 3 to calculate the indicators below. INDICATOR #1: AMOUNT OF EOC SERVICESINDICATOR #1: AMOUNT OF EOC SERVICES Total Basic EOC facilities in area (Form 3, Q5, col. 1) Population in area (Form 3, Q2) Indicator #1aIndicator #1a Number of Basic EOC facilitiesNumber of Basic EOC facilities per 500,000 populationper 500,000 population (( ÷ )) × 500,000 = Total Comprehensive EOC facilities in area (Form 3, Q5, col. 2) Population in area (Form 3, Q2) Indicator #1bIndicator #1b Number of Comprehensive EOCNumber of Comprehensive EOC facilities per 500,000 populationfacilities per 500,000 population (( ÷ )) × 500,000 = INDICATOR #2: DISTRIBUTION OF EOC FACILITIESINDICATOR #2: DISTRIBUTION OF EOC FACILITIES Note: This indicator is generally intended for use at the national level. In large areas (e.g. with millions of inhabitants), it is reasonable to calculate the distribution of EOC facilities for sub-areas. This may be done by repeating the steps above (in Indicator #1), and then calculating the percentage of sub-areas meeting the minimum acceptable levels. The minimum acceptable level for this indicator is 100 per cent. INDICATOR #3: PROPORTION OF ALL BIRTHS IN BASIC AND COMPREHENSIVE EOC FACILITIESINDICATOR #3: PROPORTION OF ALL BIRTHS IN BASIC AND COMPREHENSIVE EOC FACILITIES Total deliveries in all EOC facilities in area (Form 3, Q6, col.3) Total annual births in area (Form 3, Q4) Indicator #3Indicator #3 Proportion of all births in Basic andProportion of all births in Basic and Comprehensive EOC facilitiesComprehensive EOC facilities ÷ = × 100 = % 68 IS ACCEPTABLE LEVEL MET? Minimum acceptable level = 100% Met Not met Minimum acceptable level = 5% Maximum acceptable level = 15% Met Not met Maximum acceptable level = 1% Met Not met FORM 4FORM 4 (continued)(continued) INDICATOR #4: MET NEED FOR EOCINDICATOR #4: MET NEED FOR EOC Total complicated cases in all EOC facilities (Form 3, Q7, col. 3) Total annual births in area (Form 3, Q4) Indicator #4Indicator #4 Proportion of women estimated Proportion of women estimated to have complications who areto have complications who are treated in EOC facilitiestreated in EOC facilities ÷(( × .15* ))= × 100 = % * Births are multiplied by .15 to estimate total complications in the population. INDICATOR #5: Caesarean SECTIONS AS A PROPORTION OF ALL BIRTHSINDICATOR #5: Caesarean SECTIONS AS A PROPORTION OF ALL BIRTHS Total Caesarean sections in all EOC facilities (Form 3, Q8, col. 3) Total annual births in area (Form 3, Q4) Indicator #5Indicator #5 Caesarean sectionsCaesarean sections as a proportion of all birthsas a proportion of all births ÷ = × 100 = % INDICATOR #6: CASE FATALITY RATEINDICATOR #6: CASE FATALITY RATE Total direct obstetric deaths (from selected causes) in all Comprehensive EOC facilities studied (W.S. 3b, Q1d) Total complicated cases in all Comprehensive EOC facilities studied (W.S. 3b, Q1b) Indicator #6Indicator #6 Case fatality rateCase fatality rate ÷ = × 100 = % CFR bar chart for area:CFR bar chart for area: Create a bar chart for the area to show the CFRs for each Comprehensive EOC facility studied. The horizontal axis should be labelled with the facility names and the vertical axis ‘CFR (%)’. Use Worksheet 3b to obtain CFRs for each facility. 69 IS ACCEPTABLE LEVEL MET? Minimum acceptable level = 4 per 500,000 population Met Not met Minimum acceptable level = 1 per 500,000 population Met Not met Minimum acceptable level = 100% of areas have the minimum acceptable numbers of Basic and Comprehensive EOC facilities Met Not met Minimum acceptable level = 15% Met Not met FORM 5FORM 5 CALCULATION OF INDICATORS FOR THE COUNTRYCALCULATION OF INDICATORS FOR THE COUNTRY Complete worksheets 5a-c before calculating the indicators below. INDICATOR #1: AMOUNT OF EOC SERVICESINDICATOR #1: AMOUNT OF EOC SERVICES Total Basic EOC facilities (W.S. 5a, Q1a) Total population (W.S. 5a, Q1c) Indicator #1aIndicator #1a Number of Basic EOC facilitiesNumber of Basic EOC facilities per 500,000 populationper 500,000 population (( ÷ )) × 500,000 = Total Comprehensive EOC facilities (W.S. 5a, Q1b) Total population (W.S. 5a, Q1c) Indicator #1bIndicator #1b Number of Comprehensive EOCNumber of Comprehensive EOC facilities per 500,000 populationfacilities per 500,000 population (( ÷ )) × 500,000 = INDICATOR #2: DISTRIBUTION OF EOC FACILITIESINDICATOR #2: DISTRIBUTION OF EOC FACILITIES Number of areas meeting minimum levels for both Basic and Comprehensive EOC (W.S. 5a, Q1d) Number of areas (W.S. 5a, Q2) Indicator #2Indicator #2 Proportion of areas with theProportion of areas with the minimum acceptable number of Basicminimum acceptable number of Basic and Comprehensive EOC facilitiesand Comprehensive EOC facilities ÷ = × 100 = % INDICATOR #3: PROPORTION OF ALL BIRTHS IN BASIC AND COMPREHENSIVE EOC FACILITIESINDICATOR #3: PROPORTION OF ALL BIRTHS IN BASIC AND COMPREHENSIVE EOC FACILITIES Total deliveries in all EOC facilities (W.S. 5b, Q1a) Total annual births in all areas (W.S. 5b, Q1d) Indicator #3Indicator #3 Proportion of all births in Basic andProportion of all births in Basic and Comprehensive EOC facilitiesComprehensive EOC facilities ÷ = × 100 = % 70 IS ACCEPTABLE LEVEL MET? Minimum acceptable level = 100% Met Not met Minimum acceptable level = 5% Maximum acceptable level = 15% Met Not met Maximum acceptable level = 1% Met Not met FORM 5FORM 5 (continued)(continued) INDICATOR #4: MET NEED FOR EOCINDICATOR #4: MET NEED FOR EOC Total complicated cases in all EOC facilities (W.S. 5b, Q1b) Total annual births in all areas (W.S. 5b, Q1d) Indicator #4Indicator #4 Proportion of women estimated Proportion of women estimated to have complications who areto have complications who are treated in EOC facilitiestreated in EOC facilities ÷( × .15* )= × 100 = % *Births are multiplied by .15 to estimate total complications in the population. INDICATOR #5: Caesarean SECTIONS AS A PROPORTION OF ALL BIRTHSINDICATOR #5: Caesarean SECTIONS AS A PROPORTION OF ALL BIRTHS Total Caesarean section in all EOC facilities (W.S. 5b, Q1c) Total annual births in all areas (W.S. 5b, Q1d) Indicator #5Indicator #5 Caesarean sectionsCaesarean sections as a proportion of all birthsas a proportion of all births ÷ = × 100 = % INDICATOR #6: CASE FATALITY RATEINDICATOR #6: CASE FATALITY RATE Total direct obstetric deaths (from selected causes) in all Comprehensive EOC facilities studied (W.S. 5c, Q1b) Total complicated cases in all Comprehensive EOC facilities studied (W.S. 5c, Q1a) Indicator #6Indicator #6 Case fatality rateCase fatality rate ÷ = × 100 = % CFR Scattergram for country:CFR Scattergram for country: Create a scattergram for the country to show the CFRs in each Comprehensive EOC facility studied, grouped by area. The horizontal axis should be labelled ‘Area’ and the vertical axis ‘CFR (%)’. Use all copies of Worksheet 3b to obtain CFRs. For each area, plot the CFR of all facilities and the aggregate CFR for that area. 71 WORKSHEET 5aWORKSHEET 5a AMOUNT OF EOC SERVICESAMOUNT OF EOC SERVICES Use copies of Form 3 and Form 4 for all areas studied to complete the following table. Attach additional sheets if necessary. Name of area Number of Basic EOC facilities in area (Form 3, Q5, col. 1) Number of Comprehensive EOC facilities in area (Form 3, Q5, col. 2) Population in area (Form 3, Q2) Are minimum levels for both Basic and Comprehensive EOC met? (If YES, place check in column.) (Form 4, Indicators #1a & #1b) 1. Column totals*1. Column totals* 1a. 1b. 1c. 1d. *If more than one sheet is used, add sheet totals to get overall column total. 2. 2. Number of areas = 72 WORKSHEET 5bWORKSHEET 5b DELIVERIES, COMPLICATIONS & CAESAREAN SECTIONSDELIVERIES, COMPLICATIONS & CAESAREAN SECTIONS Use copies of Form 3 for all areas studied to complete the following table. Attach additional sheets if necessary. Name of area Total deliveries in all EOC facilities in area (Form 3, Q6, col. 3) Total complicated cases treated in all EOC facilities in area (Form 3, Q7, col. 3) Total Caesarean sections in all EOC facilities in area (Form 3, Q8, col. 3) Total annual births in area (Form 3, Q4) 1. Column totals*1. Column totals* 1a. 1b. 1c. 1d. *If more than one sheet is used, add sheet totals to get overall column total. 73 WORKSHEET 5cWORKSHEET 5c OBSTETRIC COMPLICATIONS AND DEATHS IN COMPREHENSIVE EOC FACILITIESOBSTETRIC COMPLICATIONS AND DEATHS IN COMPREHENSIVE EOC FACILITIES Use all copies of Worksheet 3b to complete the following table. Attach additional sheets if necessary. Name of area Number of complicated obstetric cases in Comprehensive EOC facilities studied (W.S. 3b, Q1b) Number of direct obstetric deaths (due to selected causes) in Comprehensive EOC facilities studied (W.S. 3b, Q1d) 1. Column totals*1. Column totals* 1a. 1b. *If more than one sheet is used, add sheet totals to get overall column total. 74 5 INTERPRETING THE FINDINGS Having calculated the indicators, the next step is to interpret the findings. This chapter first presents some general notes on interpretation and then addresses each of the indicators. One of the strengths of using these process indicators is that each one provides guidance in determining priorities for programmes. 5.1. General Notes on Interpretation Some interpretation issues are relevant to most of the process indicators. These include: distinguishing between ‘minimum’ and ‘optimum’ levels, assessing the generalizability of results, and working with incomplete or poor data. These points are discussed below. 5.1.1. ‘Minimum’ versus ‘optimum’ levels One important distinction that applies to most of the indicators is the distinction between ‘minimum’ and ‘optimum’ levels. By necessity, the minimum acceptable levels proposed in this book are approximations. Therefore, if the minimum acceptable level is met for a particular indicator, this does not imply that the optimum level has been reached. For instance, one key assumption in setting the minimum acceptable levels is that approximately 15 per cent of pregnant women will experience serious obstetric complications. If in fact this is an underestimate — as recent studies indicate it may be — then the minimum acceptable levels proposed here may be underestimates as well [Koblinsky, 1995; Bhatia and Cleland, 1995]. However, since it would be extremely difficult and costly to collect national and local data on the incidence of obstetric complications, it is reasonable to assume (based on the evidence presented in Section 3.2.1) that a country meeting the minimum acceptable level for each indicator has a strong programme for reducing maternal mortality. In comparing the findings to the minimum acceptable levels, a good rule is that when the actual level meets or exceeds the minimum acceptable level, it is probable that the need for EOC is being reasonably well met. Nevertheless, even if the minimum acceptable level for an indicator is met on the national level, there may be problems in particular areas. On the other hand, when the level falls below the minimum acceptable level, one can conclude that the need for EOC is not being met in most areas of the country. The general principle here is that favourable findings, while reassuring, do not justify complacency. Unfavourable findings, on the other hand, clearly indicate the need for action. 75 5.1.2. Generalizability of results In countries where subsets of areas and/or facilities are selected for study, another concern about interpreting data is the generalizability of the findings. In sections 4.2–4.4, which discuss selection of facilities for study, the selection process had two steps — selection of areas for study, and, within these areas, selection of facilities for study. However, if it turns out that the information is not useful for generalization, it may nevertheless be useful for managing or evaluating health services in the area. For example, supposing that the possible EOC facilities selected for study were not randomly selected and were therefore much more likely to be located on a major road than a randomly selected group would have been. While it may not be possible to generalize from these data, they may show that some hospitals are not providing such life- saving services as Caesarean sections, even though government standards indicate that they should. This information, by itself, can be used to direct efforts to reduce maternal deaths. Furthermore, even if one knows that data are biased, they may still be useful if the direction of the bias is known. For instance, in the example given above, it may be possible to say with reasonable certainly that hospitals far from major roads are less likely (rather than more likely) than hospitals on the major roads to perform Caesarean sections. Therefore, one could cautiously say that the estimate derived from the biased sample presents an unrealistically favourable picture and that the situation is probably worse than the data indicate. 5.1.3. Incomplete or poor data The routine maternity record system in many countries does not make it easy to gather data on obstetric complications. Often, the staff in a facility have fallen out of the habit of filling in some of the columns of the maternity register or the admissions register. This is a management issue and is relatively easy to correct. A more difficult problem is that in many countries the maternity register does not have a column for ‘reason for admission’ or ‘complications’. And yet complications are a key event. Without them, all deliveries would have good outcomes. On the other hand, there are often register columns devoted to uncommon events, such as multiple births. In Appendix D, a sample register format is presented that includes columns for all the information needed to calculate the indicators, as well as some other statistics that are of interest primarily at the local level. For example, recording time of admission is useful for studying the interval between admission and emergency Caesarean section [PMM Network, 1995]. Thus, it is likely that incomplete or poor records will be encountered when gathering data for these indicators — at least the first time. (As periodic collection of these data becomes part of routine programme monitoring, record-keeping should probably improve as well.) The question is, what to do when problems are encountered? 76 First of all, it is important to remember that poor records will bias the findings in one direction — undercounting events taking place in facilities. Therefore, when interpreting the data, one can discuss the possible effect of undercounting. In many situations, the level of EOC being provided is so low that, even allowing for substantial undercounting (e.g., 100 per cent), the meaning of the findings does not change very much. For example, if the records show that only 6 per cent of the need for EOC is met in an area, and one assumes that the true proportion is twice as high, that is still only 12 per cent. This change does not alter the clear implications for programmes. A 1992 study in three districts in India found that met need for EOC ranged from a low of 3.3 per cent to a high of only 6.5 per cent [Nirupam, 1992]. The acceptable level of met need for EOC is 100 per cent. Therefore, these data would have to be under-reported by a factor of 15 — which is highly unlikely — to be completely misrepresenting the actual situation. There are two ways in which it would be possible to overestimate, rather than underestimate, the amount of EOC being provided. The first is by underestimating the denominator — i.e., by underestimating live births. The second might arise if data on women with complications are unavailable in most facilities. In fact, as noted earlier, these are the data that are likely to be the most difficult to gather. If these data cannot be gathered from the registers, then it may be possible to obtain them by going through individual patient files. In many facilities, however, such records are incomplete or non-existent. In that case, it may not be possible to get direct information on complications the first time. For these situations, the facility review form provides a way of calculating a proxy for the number of complications. If the proxy method (Plan 3) proposed in Worksheet 2a is used in a substantial proportion of facilities, it is likely that the number of women with complications will be overestimated. (The reasons for this are discussed in Section 4.4.1.) Under these conditions, if it is found that minimum met need for EOC is not being satisfied, one can reasonably assume that the situation is probably even worse. If, on the other hand, it is found that met need exceeds 100 per cent, the conclusion is indeterminate. If the number of women with complications is overestimated, then the case fatality rate (CFR) for these facilities is likely to be underestimated. The interpretation of CFRs from such facilities follows a similar logic: If the CFR is found to be unacceptable, one can reasonably assume that the actual situation is even worse. If it is the case that a substantial proportion of facilities in an area or country lack the data required to count the number of women with complications directly, this should send a strong message to planners to improve record-keeping mechanisms for the next round of data collection, a few years hence. In the absence of information on women with complications, information on Caesarean sections (Indicator 5 — Caesarean sections as a percentage of births) may be used as a rough indication of the amount of EOC being provided. (Surgery registers are usually fairly well kept.) 77 Figure 18. Indicators and minimum acceptable levels Indicator Minimum acceptable level Amount of EOC: Basic EOC facilities Comprehensive EOC facilities For every 500,000 population, there should be: At least 4 Basic EOC facilities. At least 1 Comprehensive EOC facility. Geographical distribution of EOC facilities Minimum level for amount of EOC services is met in subnational areas. Proportion of all births in Basic and Comprehensive EOC facilities At least 15% of all births in the population take place in either Basic or Comprehensive EOC facilities. Met need for EOC: Proportion of women estimated to have complications who are treated in EOC facilities At least 100% of women estimated to have obstetric complications are treated in EOC facilities. Caesarean sections as a percentage of all births As a proportion of all births in the population, Caesarean sections account for not less than 5% nor more than 15%. Case fatality rate The case fatality rate among women with obstetric complications in EOC facilities is less than 1%. 78 5.2. Interpreting the Indicators Figure 18 below contains the six process indicators and the minimum acceptable level for each. Based on the data summarized in this figure, the following section may be used as a guide in interpretation. Additionally, ‘Emergency Obstetric Care: Measuring Availability and Monitoring Progress’, which presents findings on the process indicators for several areas in India, is a useful reference [Nirupam and Yuster, 1995]. Although the process indicators have been revised somewhat since the Nirupam and Yuster study (which is based on an early draft of these Guidelines), the article provides a good example of how to present and interpret the findings. 5.2.1. Amount of EOC services If, in the aggregate, there are not four Basic and one Comprehensive EOC facilities per 500,000 population, the overall minimum acceptable level of EOC services is not met for the country. In this case, a high priority is to bring the amount of EOC services at least up to the minimum acceptable level. This may be done in different ways — i.e., by upgrading existing facilities, building new facilities or some combination of the two. If the overall minimum acceptable level of EOC services is met — that is, if there are four Basic and one Comprehensive EOC facilities per 500,000 population — it is reasonable to conclude that, in the aggregate, there currently exists a reasonable number of EOC facilities. The next step is to look at the geographical distribution of EOC facilities. 5.2.2. Geographical distribution of EOC services In order to prevent maternal deaths, the minimum acceptable level of EOC facilities should be met not only in the aggregate, but in smaller geographical areas as well. If this is not the case in some areas, it should be a priority to increase the availability of EOC services in the underserved areas. Again, it is worth noting that meeting the minimum acceptable level of EOC services does not mean that all women necessarily have access to EOC. In very difficult terrain, for example, people may be spread over a vast area with few roads, so that more than the minimum number of EOC facilities might be needed to make them reasonably accessible to women in need. If in smaller geographical areas of the country the minimum acceptable level of EOC facilities is met, the next step is to examine how many women are using these facilities, and for what purposes. 5.2.3. Proportion of all births in Basic and Comprehensive EOC facilities If the minimum acceptable level for this indicator is not met — i.e., fewer than 15 per cent of all births in the population take place in EOC facilities — one can conclude with reasonable certainty that some women who need life-saving EOC services are not receiving them. In this case, the reasons for underutilization need to be explored and addressed. Of course, in seeking to increase utilization, 79 the emphasis should be on encouraging women with complications to use EOC facilities, and not simply on increasing the number of normal deliveries taking place in facilities. As discussed in earlier chapters, the goal is to have 100 per cent of women with obstetric complications delivering in EOC facilities, not 100 per cent of all pregnant women. On the other hand, if the minimum acceptable level is met for this indicator, it is reasonable to conclude that it is possible that many women needing EOC are delivering in EOC facilities. However, since this indicator does not provide any information about the types of deliveries taking place in EOC facilities, one cannot draw conclusions about whether it is likely that most women who need EOC are in fact receiving it. It may be that a large proportion of women delivering in facilities are those having normal deliveries. Also, there are major obstetric complications that are not usually counted among deliveries — antepartum and post-partum haemorrhage, post-partum sepsis and complications of induced abortion. This indicator provides no information about whether women with these complications are receiving EOC. 5.2.4. Met need for EOC: Proportion of women estimated to have obstetric complications who are treated in EOC facilities If the minimum acceptable level for this indicator is not met — that is, met need is less than 100 per cent — then the conclusion to be drawn is that some women with complications are not receiving the medical care they need. If the preceding indicators have all met the minimum acceptable levels and met need is less than 100 per cent, then the national priority must be to improve utilization of EOC facilities by women with complications. Depending on the individual country's situation, strategies for meeting this objective may include improving quality of care at facilities, providing community education about recognition of complications and the importance of seeking care, or other interventions. If the minimum acceptable level for this indicator is met, it is reasonable to conclude that most women who need EOC services are receiving them. Since, as discussed earlier, the true incidence of complications in the population may be greater than 15 per cent, it is possible that even if this indicator is 100 per cent, there may still be women in need of life-saving EOC services who are not receiving them. It is also for this reason that the level of met need may turn out to be greater than 100 per cent. Therefore, if a met need of more that 100 per cent is found, this should not be taken to mean that there is necessarily a problem with the data — e.g., overdiagnosis of complications. 5.2.5. Caesarean sections as a percentage of all births Because of concerns about the performance of unnecessary Caesarean sections, this indicator has both a minimum and a maximum acceptable level. If the minimum level of Caesarean sections is not met — that is, if fewer than 5 per cent of all births are Caesarean sections — one may conclude that some women who need Caesarean sections are not receiving them. The priority is then to increase the availability and performance of appropriate Caesarean sections. 80 If the maximum level of Caesarean sections is exceeded — that is, more than 15 per cent of all births are Caesarean sections — one may assume that some unnecessary Caesarean sections are being performed. Local- and facility-level monitoring should be encouraged to prevent the performance of unnecessary Caesarean sections. If the findings for this indicator are within the acceptable range — between 5 and 15 per cent of all births — one may conclude that it is possible that most women who need a Caesarean section are receiving one. As discussed in earlier chapters, this indicator does not provide information on the appropriateness of the Caesarean sections being performed. Ongoing monitoring is important to ensure that women who need Caesarean sections get them promptly and that unnecessary Caesarean sections are not common. 5.2.6. Case fatality rates The indicators discussed so far are measures of coverage and utilization of EOC at the population level. Case fatality rates (CFRs), on the other hand, are measures of EOC performance at the facility level. They may lose meaning and usefulness when aggregated. For example, an average CFR for an area or a country does not provide information on which facilities are doing well and which need improvement — the very kind of information that a health official setting priorities would want. Therefore, when interpreting CFRs, we suggest using bar charts and scattergrams so that the facility- level information is not lost. Bar charts are useful for displaying the results from a number of hospitals in one area. Scattergrams are useful for displaying the results from a number of areas, because they allow comparison of mean values while at the same time visually displaying the distribution of data points. Guidelines for creating bar charts and scattergrams are given in the CFR section of Form 4 and Form 5. Interpreting CFRs is simplest if done in several stages. As described in previous chapters, these stages include: ! creating a bar chart to compare the CFRs of individual facilities in a geographical area (Form 4); ! calculating the aggregate CFR for all Comprehensive EOC facilities studied in a geographical area (Form 4); ! creating a national scattergram to compare the means and ranges of facility CFRs of several geographical areas within the country (Form 5); and ! calculating the aggregate CFR for all Comprehensive EOC facilities studied in the country (Form 5). It is informative to compare the actual CFR to the maximum acceptable level — 1 per cent — at each stage. 81 As discussed in earlier chapters, if the aggregate CFR is at an acceptable level and EOC coverage (indicators 1–5) meets the minimum acceptable levels, one can reasonably conclude that the country's maternity care system is functioning well to prevent maternal deaths. If, however, the aggregate CFR is at an acceptable level and EOC coverage and/or utilization are insufficient, the interpretation is quite different. In this case, the data imply that while women who deliver in EOC facilities are likely to survive, maternal deaths outside health facilities are likely to be unacceptably common. The aggregate CFR may be interpreted as a very rough indicator of quality of care in the area or country as a whole. However, this overall measure will not adequately reflect variation in CFRs among facilities, which is likely to be great. Therefore, careful attention to the CFR bar charts and scattergram is important. A facility's CFR may exceed the maximum acceptable level for many reasons. In some cases, it may in fact be that quality of care is inadequate. However, there may be other explanations — for example, long delays in reaching EOC facilities may result in poor condition on arrival, or a particular facility with a high CFR may be the end point of the local referral chain, so that women with the most serious complications are sent there. It is also important to consider the number of women being counted in the CFR. If this rate is based on a small number of women (e.g., fewer than 20), then even a single death can create a deceptively large increase. If facility CFRs are found to exceed the maximum acceptable level, more information will be needed to understand why the rates are high. As explained earlier, this stage of investigation should be done at the area level. The earlier section on area-level monitoring (Section 4.8) provides suggestions for collecting such information. 82 REFERENCES Akin-Dervisoglu, A. ‘Maternal Mortality in Turkey’. Paper presented at the World Health Organization Interregional Meeting on the Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.19). Alauddin, M. ‘Maternal Mortality in Rural Bangladesh: The Tangail District’. Studies in Family Planning; Jan./ Feb. 1986; 17(1): 13–21. Baldo, M.H. ‘Maternal Mortality in Sudan’. Paper presented at the World Health Organization Interregional Meeting on the Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.5). Beral, V. ‘Reproductive Mortality’. British Medical Journal; 1979; 2: 632–34. Bhatia, J.C. ‘Maternal Mortality in Ananthapur District, India: Preliminary Findings of a Study’. Paper presented at the World Health Organization Interregional Meeting on Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.16). Bhatia, J.C. A Study of Maternal Mortality in Ananthapur District, Andhra Pradesh, India, Tables 8 & 9. Bangalore, India: Indian Institute of Management; 1986. Bhatia, J.C., and Cleland, J. ‘Self-Reported Symptoms of Gynecological Morbidity and their Treatment in South India’. Studies in Family Planning; 1995; 26(4): 203-16. Boerma, J. T., and Mati, J. ‘Identifying Maternal Mortality through Networking: Results from Coastal Kenya’. Studies in Family Planning; Sept./Oct. 1989; 20(5): 245–53. Buescher, P. A. ‘Enhanced Maternal Mortality Surveillance North Carolina, 1988 and 1989'. Morbidity and Mortality Weekly Report; July 19, 1991; 40(28): 469–71. Campbell, O.M.R., and Graham, W.J. Measuring Maternal Mortality and Morbidity: Levels and Trends. London: Maternal and Child Health Epidemiology Unit, London School of Hygiene and Tropical Medicine; 1990. Cantrelle, P. Systems of Demographic Measurement Data Collection Systems: The Multiround Demographic Survey Method. Chapel Hill: Carolina Population Center, University of North Carolina; 1977. Chen, R.J. ‘Maternal Mortality in Shanghai, China’, World Health Organization (FHE/PMM/85.9.2). Correa, P. ‘Maternal Mortality in Senegal’. Paper presented at the World Health Organization Interregional Meeting on the Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.21). 83 Cottingham, J., and Royston, E. Obstetric Fistulae: Review of Available Information. Geneva: World Health Organization; 1991. El-Badry, M.A. ‘Higher Female than Male Mortality in Some Countries of South Asia: A Digest’. American Statistical Association Journal; 1969; 64: 1234–44. Farnot-Cardoso, U. ‘Maternal Mortality in Cuba’. Paper presented at the World Health Organization Interregional Meeting on the Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.14). Fathalla, M.F. Assignment Report: Maternal Health in Syria, December 19–29, 1982. Geneva: World Health Organization; Mar. 1983. Fauveau, V., et al. ‘Effect on Mortality of Community-Based Maternity-Care Programmes in Rural Bangladesh’. The Lancet; Nov. 9, 1991; 338: 1183–86. Fleiss, J. Statistical Methods for Rates and Proportions. New York: John Wiley and Sons; 1981: 13. Fortney, J.A. ‘Management of Breech Presentations in Developing Country Hospitals’. Tropical Doctor; Jan. 1987a; 17(1): 34–38. Fortney, J.A. ‘The Importance of Family Planning in Reducing Maternal Mortality’. Studies in Family Planning; Mar./Apr. 1987b; 18(2): 109–13. Fortney, J.A. ‘Implications of the ICD-10 Definitions Related to Death in Pregnancy Childbirth or the Puerperium’. World Health Statistics Quarterly; 1990; 43(4):246–48. Fortney, J.A. Personal communication, 1995. Fortney, J.A., et al. ‘Maternal Mortality in Indonesia and Egypt’. Paper presented at the World Health Organization Interregional Meeting on Prevention of Maternal Mortality, Nov. 11–15, 1985 (FHE/PMM/85.9.13). Fortney, J.A., et al. ‘Reproductive Mortality in Two Developing Countries’. American Journal of Public Health; 1986; 76:134–38. Graham, W., Brass, W., and Snow, R.W. ‘Estimating Maternal Mortality in Developing Countries’. The Lancet; Feb. 20, 1988; 416. Graham, W., Brass, W., and Snow, R.W. ‘Indirect Estimation of Maternal Mortality: The Sisterhood Method’. London: Centre for Population Studies, London School of Hygiene and Tropical Medicine; Apr. 1988. 84 Graham, W., Brass, W., and Snow, R.W. ‘Estimating Maternal Mortality: The Sisterhood Method’. Studies in Family Planning; May/June 1989; 20(3): 125–35. Greenwood, A.M., et al. ‘A Prospective Survey of the Outcome of Pregnancy in Rural Area of the Gambia’. Bulletin of the World Health Organization; 1987; 65(5): 635–43. Grubb, G.S., et al. ‘A Comparison of Two Cause-of-Death Classification Systems for Deaths Among Women of Reproductive Age in Menoufia, Egypt’. International Journal of Epidemiology; 17(2): 201–7. Hanley, J.A., Hagen C.A., and Shiferaw T. ‘Confidence Intervals and Sample-size Calculations for the Sisterhood Method of Estimating Maternal Mortality’. Studies in Family Planning; July/Aug 1996; 27(4):179-187. Harlap S., Kost K., Forrest J.D. Preventing Pregnancy, Protecting Health: A New Look at Birth Control Choices in the United States; New York: The Alan Guttmacher Institute; 1991. Hartfield, V.J. ‘Maternal Mortality in Nigeria Compared with Earlier International Experience’. International Journal of Gynaecology and Obstetrics; 1980; 18: 70–75. Hibbard, L.T. ‘Complications of Pregnancy and Delivery’. In Current Obstetric and Gynecological Diagnosis and Treatment, Lange, ed. Los Altos, Calif.: R.C. Benson; 1978; pp. 664-86. Indonesia. Indonesia Fertility Survey 1976: Principal Report, Vol. 1. Jakarta: Central Bureau of Statistics; 1978. Institute for the Protection of Mother and Newborn. ‘Maternal Mortality in Selected Areas of Vietnam’. Paper presented at the World Health Organization Interregional Meeting on the Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.8). International Conference on Population and Development. Report of the International Conference on Population and Development, UN Doc. A/CONF.171/13 (1994). Janowitz, B., et al. ‘Referrals by Traditional Birth Attendants in Northeast Brazil’. American Journal of Public Health; 1985; 75(7): 745–48. Kamara, Angela. ‘Health Messages and Obstetric First Aid for the Community’. Paper presented at the Seventh Operations Research Workshop of the Prevention of Maternal Mortality Network, Accra, Ghana, May 11–15, 1992. Kasongo Project Team. ‘Antenatal Screening for Fetopelvic Dystocias. A Cost-Effectiveness Approach to the Choice of Simple Indicators for Use by Auxiliary Personnel’. Journal of Tropical Medicine and Hygiene; Aug. 1984; 87(4): 173–83. 85 Kaunitz, A.M., et al. ‘Perinatal and Maternal Mortality in a Religious Group Avoiding Obstetric Care’. American Journal of Obstetrics and Gynecology; Dec. 1, 1984; 150(7): 826–31. Khan, A.R., et al. ‘Maternal Mortality in Rural Bangladesh: The Jamalpur District’. Studies in Family Planning; Jan./Feb. 1986; 17(1): 7–12. Koblinsky, M.A. ‘Beyond Maternal Mortality — Magnitude, Interrelationship, and Consequences of Women's Health, Pregnancy-Related Complications and Nutritional Status on Pregnancy Outcomes’. International Journal of Gynecology and Obstetrics; 1995; 48 Suppl.: S21-S32. Koenig, M.A., et al. ‘Maternal Mortality in Matlab, Bangladesh: 1976–85'. Studies in Family Planning; Mar./Apr. 1988; 19(2): 69–80. Koenig, M. Personal communication; 1990. Kwast B.E., et al. ‘Epidemiology of Maternal Mortality in Addis Ababa: A Community-Based Study’. Ethiopian Medical Journal; 1985; 23: 7–16. Kwast, B.E., Rochat, R.W., and Kidane-Mariam, W. ‘Maternal Mortality in Addis Ababa, Ethiopia’. Studies in Family Planning; Nov./Dec. 1986; 17(6): 288–301. Laurenti, R. ‘Maternal Mortality in Latin American Urban Areas: The Case of São Paulo, Brazil’. Bulletin of PAHO; 1993; 27(3):205-214. Loudon, I. ‘Obstetric Care, Social Class and Maternal Mortality’. British Medical Journal; Sept. 6, 1986; 293: 606–8. Loudon, I. Death in Childbirth: An International Study of Maternal Care and Maternal Mortality 1800- 1950. Oxford: Clarendon Press; 1986. Maine, D. Prevention of Maternal Mortality: Report of a WHO Interregional Meeting, Geneva, 11–15 November, 1985. Geneva: World Health Organization; 1986. Maine, D. Studying Maternal Mortality in Developing Countries: A Guidebook: Rates and Causes. Geneva: World Health Organization; 1987 (FHE/87.7). Maine, D. Safe Motherhood Programs: Options and Issues. New York: Center for Population and Family Health, Columbia University; 1991. Maine, D., Akalin, M.Z., Chakraborty, J., de Francisco, A., and Strong, M. ‘Why Did Maternal Mortality Decline in Matlab?’ Studies in Family Planning; July/Aug 1996; 27(4):179-187. McDonagh, M. ‘Is Antenatal Care Effective in Reducing Maternal Morbidity and Mortality?’ Health Policy and Planning; 1996; 11(1):1-15. 86 Mostafa, G., and Ali Haque, Y. A Review of the Emergency Obstetric Care Functions of Selected Facilities in Bangladesh, UNICEF, 1993. Mtimavalye, L.A.R., Justesen, A., and Ngwalle, E. ‘Survey on Institutional Maternal Deaths in Four Regions of Tanzania, July 1983 – December 1984: Preliminary Report’. Paper presented at the World Health Organization Interregional Meeting on the Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.20). Nirupam, S. Availability of Emergency Obstetric Care in Three Districts: Current Status and Next Steps. Government of India/UNICEF; 1992. Nirupam, S., and Yuster, E.A. ‘Emergency Obstetric Care: Measuring Availability and Monitoring Progress’. International Journal of Gynecology and Obstetrics; 1995; 50 Suppl. 2: S79-S88. Nordberg, E.M. ‘Incidence and Estimated Need of Caesarean Section, Inguinal Hernia Repair, and Operation for Strangulated Hernia in Rural Africa’. British Medical Journal; July 14, 1984; 289: 92–93. Notzon, F.C. ‘International Differences in the Use of Obstetric Interventions’. Journal of the American Medical Association; June 27, 1990; 263(24): 3286–91. Papiernik, E. ‘The Role of Obstetric Medical Care in Preventing Deaths. An Historical Perspective on European Figures Since 1751'. International Journal of Gynecology and Obstetrics; Oct. 1995; 50(Supp. 2):S73-S78. Petitti, D.A., et al. ‘In-Hospital Maternal Mortality in the United States: Time Trends and Relation to Method of Delivery’. Obstetrics and Gynecology; Jan. 1982; 59(1): 6–12. The PMM Network. ‘Situation Analyses of Emergency Obstetric Care Facilities: Examples from Eleven Sites in West Africa’. Social Science and Medicine; 1995; 40(5): 657–67. Rao, A.B. ‘Maternal Mortality in India: A Review’. Paper presented at the World Health Organization Interregional Meeting on the Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.4). Reproductive Health Indicators Working Group. Indicators for Safe Pregnancy Services (draft). Chapel Hill: The Evaluation Project, Carolina Population Center, University of North Carolina; 1994. Rodriquez, J., et al. ‘Avoidable Mortality and Maternal Mortality in Cali, Colombia’. Paper presented at the World Health Organization Interregional Meeting on Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.1). Rooney, C. Antenatal Care and Maternal Health: How Effective is it? Geneva, World Health Organization; 1992. 87 Ross, J.A., et al. Family Planning and Child Survival 100 Developing Countries. New York: Center for Population and Family Health, Columbia University; 1988. Rubin, G.L., et al. ‘The Risk of Childbearing Re-evaluated’. American Journal of Public Health; 1981; 71: 712–16. Sloan, N.C., et al. Guidelines and Instruments for a Situation Analysis of Obstetric Services. New York: The Population Council; 1995. Smith, J.C., et al. ‘An Assessment of the Incidence of Maternal Mortality in the United States’. American Journal of Public Health; Aug. 1984; 74(8): 780–83. Speckhard, M.E., et al. ‘Intensive Surveillance of Pregnancy Related Deaths in Puerto Rico, 1978–1979'. Boletín de la Asociación Medica de Puerto Rico; 1985; 77(12): 508–13. Thaddeus, S., and Maine, D. Too Far to Walk: Maternal Mortality in Context. New York: Center for Population and Family Health, Columbia University, 1990. Turnbull, A., et al. Report on Confidential Enquiries into Maternal Deaths in England and Wales, 1982–1984. London: Her Majesty's Stationery Office; 1989 (Report on Health and Social Subjects no. 34). UNICEF Planning Office. Personal communication, Aug. 1995. United Nations (UN). United Nations Population Chart 1988. New York: UN, Population Division, Department of International Economic and Social Affairs; 1988 (ST/ESA/SER.A/106/Add.1). United Nations. United Nations Demographic Yearbook. New York: UN, Department of International Economic and Social Affairs, Statistical Office; 1989. United Nations. Demographic Yearbook, 41st Edition. New York: UN; 1991. United Nations. The Health Rationale for Family Planning: Timing of Births and Child Survival, New York: UN Department for Economic and Social Information and Policy Analysis, Population Division; 1994. United Nations Fund for Population Activities. World Population Wall Chart. New York: UN; 1984. U.S. Department of Commerce, Bureau of the Census. Vital Statistics of the United States 1937, Part 1 — Natality and Mortality Data for the United States Tabulated by Place of Occurrence with Supplemental Tables for Hawaii, Puerto Rico, and The Virgin Islands. Washington, D.C.: U.S. Government Printing Office; 1939. 88 U.S. Department of Commerce, Bureau of the Census. Vital Statistics of the United States 1940, Part l — Natality and Mortality Data for the United States Tabulated by Place of Occurrence with Supplemental Tables for Hawaii, Puerto Rico, and the Virgin Islands. Washington, D.C.: U.S. Government Printing Office; 1943. U.S. Department of Health, Education and Welfare (DHEW). Vital Statistics of the United States 1950, Vol. I — Analysis and Summary Tables with Supplemental Tables for Alaska, Hawaii, Puerto Rico, and Virgin Islands. Washington, D.C.: U.S. Government Printing Office; 1954. U.S. Federal Security Agency, U.S. Public Health Service. Vital Statistics of the United States 1945, Part 1 — Natality and Mortality Data for the United States Tabulated by Place of Occurrence with Supplemental Tables for Hawaii, Puerto Rico, the Virgin Islands and Alaska. Washington, D.C.: U.S. Government Printing Office; 1947. U.S. National Center for Health Statistics (NCHS). Vital Statistics of the United States 1983: Vol.II — Mortality, Part A. Hyattsville, Md.: U.S. Department of Health and Human Services; 1987. Walker, G.J.A. The Identification of Maternal Deaths and a Review of their Causes in Jamaica: Methodological Issues. London: London School of Hygiene and Tropical Medicine; 1986. Walker, G.J.A., et al. ‘Maternal Mortality in Jamaica: A Confidential Inquiry into All Maternal Deaths in Jamaica 1981–1983'. Paper presented at the World Health Organization Interregional Meeting on Prevention of Maternal Mortality, Geneva, Nov. 11–15, 1985 (FHE/PMM/85.9.10). Walker, G.J.A., et al. ‘Maternal Mortality in Jamaica’. The Lancet; Mar. 1, 1986; 1: 486–88. Willmott Dobbie, B.M. ‘An Attempt to Estimate the True Rate of Maternal Mortality, Sixteenth to Eighteenth Centuries’. Medical History; 1982; 26: 79–90. The World Bank. World Development Report 1987. New York: Oxford University Press; 1987. World Health Organization (WHO). International Classification of Diseases Manual of the International Statistical Classification of Diseases, Injuries, and Causes of Death, Vol. 1. Geneva; 1977. World Health Organization. Prevention of Maternal Mortality: Report of a World Health Organization Interregional Meeting, November 11–15, 1985. Geneva; 1986 (FHE/86.1). World Health Organization. The Prevention and Management of Post-partum Haemorrhage: Report of a Technical Working Group, Geneva, 3–6 July, 1989. Geneva; 1989. World Health Organization. Essential Elements of Obstetric Care at First Referral Level. Geneva; 1991. World Health Organization. Mother-Baby Package: A Roadmap for Implementation in Countries. Geneva: WHO, Division of Family Health; 1993. 89 World Health Organization. Indicators to Monitor Maternal Health Goals: Report of a Technical Working Group, Geneva, 8–12 November 1993. Geneva; 1994. World Health Organization. Personal communication, C. AbouZahr, Oct. 4, 1995. World Health Organization/UNICEF. Revised 1990 Estimates of Maternal Mortality, April 1996 (WHO/FRH/MSM/96.11; UNICEF/PLN/96.1). Yadav, H. ‘Study of Maternal Deaths in Kerian (1976–1980)’. Medical Journal of Malaysia; 1982; 37: 165–69. Yan, Ren-Ying. ‘Maternal Mortality in China’. World Health Forum; 1989; 10: 327–31. Ziskin, L.Z., Gregory, M., and Kreitzer, M. ‘Improved Surveillance of Maternal Deaths’. International Journal of Gynecology and Obstetrics; 1979; 16: 281–86. 90 APPENDIX A: Collecting Data for Impact Indicators As Figure A shows, all of the impact measures discussed in Chapter 4 can be constructed from a few pieces of data: the number of maternal deaths in the population; the number of deaths among all women age 15–49; the number of births in the population; the total fertility rate; and the number of women of reproductive age. Figure A. Types of data used in constructing various indicators of impact Type of data Maternal mortality ratio Maternal mortality rate Lifetime risk Per cent of all deaths Maternal deaths in the community X X X X All deaths of women age 15–49 X Birthrate Crude birthrate X X Total fertility rate X Population Size X X No. of women age 15–49 X Information on population size and composition and on fertility patterns is available in most countries, at least at the national level. In developed countries this information can be found in existing records. In developing countries, it is usually available from surveys that have been conducted for various purposes, e.g., the Demographic and Health Surveys. Information on maternal deaths is far more difficult to obtain. Even in industrialized countries, substantial numbers of maternal deaths are missing from official records, and in developing countries, the difficulties in obtaining information on maternal deaths are even greater. A.1. Existing Data In general, it is quicker and less expensive to analyse existing data than to collect new data. Thus, before special studies are conducted, one should explore the possibility of using existing data. 91 Unfortunately, for most developing countries such data will not be useful in determining the current level of maternal mortality, nor for tracking trends in rates or ratios over time. A.1.1. Vital registration ! Government registration systems. In theory, governments gather information on all deaths (including maternal deaths) through routine reporting systems, and, in turn, report these to the United Nations. The results of this process are published in the United Nations Demographic Yearbook. Unfortunately, the areas in which reduction of maternal mortality is most urgent are the areas in which vital registration systems are the weakest, as Figure B shows. In the 1990 United Nations Demographic Yearbook, less than 6 per cent of the world's population lived in areas where maternal mortality data were reported for the most recent time period (1985–87) [Campbell and Graham, 1990]. Figure B. Countries reporting vital statistics to the United Nations, 1985–1989 Number of countries Number reporting maternal deaths Percentage reporting maternal deaths Africa 63 4 6 North America and Caribbean 37 11 30 South America 14 9 64 Asia 48 8 17 Europe 40 27 68 Source: United Nations, 1991. Even where vital registration systems are adequate, they usually do not provide an accurate picture of the level of maternal mortality. As was shown earlier, while most deaths may be reported, the fact that a given death was related to pregnancy is quite likely to be overlooked. This may account for the fact that the reported number of maternal deaths is unbelievably small in some countries. ! Other registration systems. In some developing countries, government health and vital statistics agencies have attempted to improve vital reporting systems by instituting ‘village registries’. These village registries are usually located in local health centres or are kept by village health committees. While these registries may improve the reporting of births, they are less likely to result in markedly better reporting of maternal deaths. As noted above, even where most deaths occur in health facilities, maternal deaths are often not classified as such. This is even more likely when they are reported by lay reporters. In fact, lay reporting systems have not been very successful in reporting maternal deaths [Campbell and Graham, 1990; Maine, 1987]. 92 A.1.2. Health facility records Health facility records are an important source of information on maternal deaths, provided the data are correctly analysed and interpreted. Unfortunately, this is often not the case. One appropriate use for records from hospitals and other health facilities is to provide information on the functioning and quality of health services. This kind of information is, however, used in process (not impact) indicators. In terms of impact indicators, health facility records are useful for estimating the extent to which maternal deaths are under-reported by routine reporting. For example, it is possible to obtain a list of women who died maternal deaths from the city health department, and then check the records of hospitals in the city to see if any deaths were missed. In this way, one could estimate a correction factor that could be applied to the reported level of maternal mortality. This is discussed further under the heading ‘Multiple Source Studies’. Thus, hospital data are appropriate for deriving process indicators and correction factors. However, they should not be used in constructing impact indicators — e.g., maternal mortality rates and ratios. For a further discussion of this subject, see Appendix B. A.2. Multiple Source Studies Various sources of information on maternal deaths each have their strengths and weaknesses. In general, the more sources used for a single study, the higher the proportion of deaths that are likely to be identified. Again, using multiple sources may not be practical for large studies, such as those that try to determine the level of maternal mortality in a large country. It can, however, be informative. In Jamaica six separate sources of information on deaths were searched and compared: hospital records, coroner's court records, police records, morgue records, interviews with health department staff, and death certificates. Only one of these sources (hospital in-patient records) identified as many as two thirds of the maternal deaths [Walker et al., 1985, 1986]. The deaths identified by using multiple sources raised the national maternal mortality ratio from 48 to 108 maternal deaths per 100,000 live births. In Bali, Indonesia, society is very highly organized. There is a tradition of small villages with a headman and a clearly defined population. In addition, at the time of the study in question, family planning workers regularly visited the houses of women of reproductive age. The researchers made good use of these circumstances, enlisting village headmen and family planning workers in identifying all deaths among women of reproductive age. By chance, a repeat household survey was done in a neighbouring area. By comparing the results of the two studies, the researchers concluded that they had missed about half of the deaths [Fortney et al., 1985; Fortney, 1992]. 93 In a network study in Kenya, the investigators noted that women were reluctant to discuss maternal deaths, perhaps out of fear of witchcraft [Boerma and Mati, 1989]. More than two thirds of the women interviewed said that they were not aware of any maternal deaths. Furthermore, the investigators found that traditional birth attendants were not good sources of information, since they mentioned only 19 of the 35 deaths. The local health workers mentioned only one maternal death. As noted earlier, even in the United States and the United Kingdom, studies have found substantial additional deaths by supplementing vital statistics data with other sources of information, such as death certificates [Rubin et al., 1981], confidential inquiries [Turnbull et al., 1989], and reports from hospitals, physicians and medical examiners [Ziskin et al., 1979]. Using such data, it would be possible to estimate the proportion by which the vital registration statistics underestimate the level of maternal mortality. This, however, is not usually done. It is important to note that in all of the instances cited here there was a well-functioning vital registration system to be supplemented. A.3. Retrospective Household Surveys There are a number of kinds of retrospective surveys that are used to determine levels of maternal mortality. They all use the household as the sampling unit. An interviewer visits the households selected and asks about maternal deaths. Beyond this, however, the methods vary. A.3.1. Relationship of respondent to the woman who died Maternal mortality surveys differ with respect to the relationship of the person interviewed to the woman who died. Respondents may be asked about women who have died within their household, about their sisters or about women they have known. ! Deaths in the respondent's household. Some studies in the literature have asked respondents about women living in their household who have died. One problem with this type of study is that households may dissolve when the wife/mother dies — e.g., the children may be sent to live with relatives and the husband may live elsewhere. While there have been few such studies, the example below indicates the large sample sizes necessary for such studies. In 1983, Addis Ababa, Ethiopia, was the site of a survey of maternal deaths [Kwast et al., 1986]. A three-stage cluster sample was taken. In each district, enough subdistricts were selected to make up 20 per cent of the district population. Within the subdistricts, houses were selected by systematic sampling. (All houses were already enumerated.) In an eight-week period, 43 interviewers, seven supervisors and one research assistant gathered data on 32,215 households. This provided information on 9,315 women who had been pregnant during the previous two years. This monumental effort provided information on 45 maternal deaths. The maternal mortality ratio derived was 566 maternal deaths per 100,000 live births. 94 In rural areas, a study such as this would probably be impossible. Where maternal mortality levels are lower, sample sizes would need to be even greater, or the number of deaths detected would be smaller. ! Deaths among the respondent's sisters. In the ‘sisterhood’ method, adults in the household are interviewed about their own sisters: how many sisters they had who survived to adulthood; how many of these sisters died of pregnancy-related causes [Graham et al., 1988, 1989]. These data can then be used to construct such indicators as the maternal mortality rate, ratio, lifetime risk, etc. Survey methods produce estimates of maternal mortality levels for different periods of time. Several methods produce estimates that refer to a short period of time (1–3 years) immediately before the survey. The sisterhood method, in contrast, gathers information about deaths among respondents' sisters, whenever they died. Consequently, the results of such studies usually yield an estimate of the level of maternal mortality centred around 10-12 years before the study. This estimate may or may not reflect the current level of maternal mortality in the community. Using such an estimate, one would have to wait at least 10 years in order to assess any change in maternal mortality. This presents problems in terms of monitoring the impact of activities to reduce maternal mortality. Thus, while the sisterhood method reduces the required sample size, it does so by expanding the time period to which the deaths refer, so that the rates refer to 12 or more years in the past. Efforts are being made to develop versions of the sisterhood method that would give more detail — e.g., shorter time spans, age-specific maternal mortality, etc. Such refinements may, however, increase the sample size needed to give stable estimates. The sisterhood method was first tested in the Gambia in 1987, in an area that the British Medical Research Council has been studying since 1982. The questionnaire contained only four questions on sisters and their survival. Women and men aged 15 or older were interviewed in six villages. The 2,163 interviews were conducted by six field workers over a five-day period. A total of 90 maternal deaths were identified. The lifetime risk of maternal death was calculated to be higher than one woman in 20 [Greenwood et al., 1987]. ! Deaths in the respondent's ‘network’. Another survey technique for studying levels of maternal mortality is ‘networking’ [Boerma and Mati, 1989]. With this method, people being interviewed are asked about any maternal deaths among their network of acquaintances. Those deaths that occurred within a specified geographical and time period are then investigated further. This method was tested in the Kwale area of Kenya in 1987. Questions about maternal mortality were added to a large child health survey sponsored by the Government and UNICEF. During the survey, 3,835 women in 2,900 households were interviewed. They were able to cite 345 maternal deaths, but only 35 of these met the criteria for eligibility. When divided by the estimated number of births in the area during the same period, the investigators concluded that the maternal mortality ratio was between 600 and 700 deaths per 100,000 live births [Boerma and Mati, 1989]. 95 As usual, the proportion of maternal deaths that are missed altogether is unknown when using the network method. Potential problems with constructing rates and ratios from such data include not only the danger of deaths being missed, but also of them being counted more than once. In addition, it may be difficult to specify the time period. A.3.2. Required sample size Of these survey methods, the sisterhood method is the most efficient way to identify maternal deaths. Consequently, a relatively small sample is required. Generally speaking, 3,000 to 6,000 respondents will be needed [Graham et al., 1989]. If there are two or more adults from different sibships in each household, then this sample size may be reached by visiting 1,000–3,000 households. The table below shows the sample sizes used in the surveys discussed above, as well as the number of maternal deaths reported that were eligible for study. Figure C. Comparison of Maternal Mortality Survey Methods Study method and location Number of households Number of respondents Number of maternal deaths Number of years covered by estimate Random survey: Addis Ababa, Ethiopia 32,215 9,315 45 2 Sisterhood study: Gambia — * 2,163 91 10 Network study: Kwale, Kenya 2,900 3,835 35 3 * Not applicable. Because the Gambia survey was done in the context of an existing rural population surveillance system, the researchers knew which households had eligible respondents. To a certain extent, the numbers of maternal deaths identified using the various methods probably reflect real differences in the levels of maternal mortality in the study areas. For example, in India maternal mortality is considerably higher in rural areas than in cities [Bhatia, 1985]. Therefore, it is reasonable is assume that maternal deaths really are more common in rural Gambia than in urban Ethiopia. Nevertheless, most of the difference in yield between the studies in Ethiopia and the Gambia is due to a real improvement in efficiency. 96 A.3.3. Practical considerations Clearly, the discussion of methods above has important implications for monitoring national progress in the reduction of maternal mortality. First of all, to get an idea of the national level of maternal mortality, it would be necessary either to do a series of relatively small surveys in various regions or to do a single large survey with a nationally representative sample. A series of small surveys would provide detail on variation within the population, but would be burdensome to do. A single large survey would be easier, but would not provide information useful for programme planning at the regional level unless sample sizes were even greater. The reason for this is that in order to compare, for example, the level of maternal mortality in two regions of a country, the total number of deaths would have to be divided and analysed separately. This would decrease the number of deaths in each substudy. Decreasing the number of cases increases the margin of error around the estimate, and decreases the confidence that we can have in the findings. In addition to the reliability and usefulness of the findings, there is the issue of cost. Even relatively small cluster surveys entail considerable expense, especially if they are to be done well. The issues of sample size and cost are inextricably related. All other things being equal, the larger the sample size, the greater the cost of the survey. One way to reduce the cost is by adding a few questions onto a survey that is already planned rather than planning an entirely new survey. There can, unfortunately, be problems with this approach. And there are often a variety of groups with particular interests that want to add “just a few questions.” Consequently, the people planning the survey may be reluctant (with good reason) to add more and more questions. Interviewers may be less diligent in asking (and respondents less patient in answering) questions in a long questionnaire, especially questions seemingly unrelated to the central theme of the interview. Finally, adding new and different questions may greatly increase the required sample size, since different types of indicators require different sample sizes in order to produce stable estimates. A.4. Prospective Studies Prospective studies have a decided advantage over retrospective research in terms of completeness of reporting. The reason is simple. The researchers know how many women were in the village or household at the beginning of the study. If some of them are not present later in the study, an explanation is requested. In a retrospective study, if no one mentions a woman who has died, the researchers may never know of her existence. There are a number of methods for gathering prospective data on maternal mortality. The cohort study is the most straightforward kind of prospective study. The researchers identify a group (cohort) of women and follow them for a specified period of time, identifying maternal deaths as they occur. Some cohort studies are embedded in larger research projects in which a population is under long-term observation. The Matlab study in Bangladesh is an example of this rare and expensive type of study. 97 Repeat household surveys are another method of gathering prospective data. A repeat household survey can be described as a cohort study stripped to its essential elements. In this kind of study, the researchers make a list of people of interest (e.g., women) living in each household. After a period of time (e.g., one year) the researchers return with the list and enquire about any individual who is missing. While ascertaining death is probably more accurate with a prospective than with a retrospective study, it is still likely that some deaths are not reported or are misreported, especially deaths due to clandestine abortions. A major drawback is that the relatively small number of deaths makes the estimates of maternal mortality unstable. For example, if a similar study of the same number of pregnant women in the Gambia [Greenwood, 1987] was done in 1989, and only six deaths were identified (instead of 15), one could not say with certainly whether maternal mortality had really declined or whether the difference was merely due to chance fluctuations. Another important drawback for monitoring national progress is the fact that the results are not generalizable to a large area (such as a country). 98 APPENDIX B: Process Measures Not Recommended for Monitoring Maternal Mortality Programmes B.1. ‘Hospital Maternal Mortality Rates’ Health facility data are not suitable for constructing maternal mortality rates or ratios because not all maternal deaths that occur in the population take place in facilities. In developing countries, the proportion of deaths that take place outside health facilities is often high, though generally unknown. The same is true of births, though estimates of the proportion of births taking place in health facilities can be obtained fairly easily — i.e., through population surveys. The most common misuse of hospital data is the construction of what is erroneously called the ‘hospital maternal mortality rate’. This misleading statistic has been used in dozens of articles on maternal mortality. It is usually derived by dividing the number of maternal deaths in the hospital during a given period of time by the number of live births (or total deliveries) in the same hospital during the same time period. Such statistics can never tell us about the level of maternal mortality in the community, because it is not known what proportion of deaths take place in the hospital. In addition, changes in the proportion of births that take place in hospitals will greatly affect this statistic. In Nigeria, the number of women going to teaching hospitals for normal deliveries fell sharply during the 1980s, as the economy deteriorated and hospital fees increased. Figure D illustrates how such a decline in the number of normal deliveries in a hospital will increase the ‘hospital maternal mortality rate’ even if nothing else changes in the community or the hospital. 99 Figure D. Per cent change in ‘Hospital Maternal Mortality Rate’ due to changes in births in hospital Measure Situation A Situation B Maternal deaths per year in population 100 100 Per cent of maternal deaths in hospital x 0.50 x 0.50 Number of maternal deaths in hospital =50 =50 Births per year 20,000 20,000 Per cent of births in hospital x 0.50 x 0.25 Number of births in hospital =10,000 =5,000 Number of maternal deaths in hospital 50 50 Number of births in hospital ÷ 10,000 ÷ 5,000 =0.005 =0.01 ‘Hospital Maternal Mortality Rate’ 500 1,000 ‘Hospital maternal mortality rates’ are not only uninformative about the level of maternal mortality in the community, they are not even useful as an indicator of the quality of care in the hospital. This would be true even if the proportion of births and deaths taking place in hospitals were known — a highly unlikely situation. The reason is that a crucial factor is whether the deliveries in the hospital are mostly normal or complicated. If many women go to the hospital for normal deliveries, then the hospital maternal mortality rate may be quite low. On the other hand, if women generally go to the hospital only when they are having difficulty, then the rate may be high. Neither of these circumstances gives any indication of the overall level of maternal mortality in the community or the quality of care in the hospital. They simply have to do with the distribution of various kinds of events. Consequently, comparisons of ‘hospital maternal mortality rates’ over time, between institutions, or (worse) between countries, should be absolutely avoided. A much more meaningful index of hospital functioning — the case fatality rate — was presented in section 4.2. B.2. Proportions of Women Who Are ‘Booked’ and ‘Unbooked’ There is growing agreement at the international level that the proportion of women who receive antenatal care is not a suitable indicator for monitoring progress in maternal mortality reduction [WHO, 1994b]. However, the proportion of women in the population receiving antenatal care can be used to indicate women's access to and utilization of health services. Another way that information on antenatal care has been used is in hospital studies. As is true of ‘hospital maternal mortality rates’, here too, caution must be used in drawing conclusions from hospital data about what is happening in the population at large. Typically, in hospital studies, deaths 100 are less common among ‘booked’ women (those who have received antenatal care) than among ‘unbooked’ women. Unfortunately, in such studies, potentially different patterns of hospital utilization between these two groups are seldom considered. It may well be that those who are booked are women who would choose to give birth in the hospital, and most of them will have normal deliveries. Unbooked women, in contrast, may be women who intend to deliver at home and would only go to the hospital if they were in serious danger. Thus, most unbooked women who come to the hospital do so with life-threatening complications. From the perspective of the hospital, therefore, it would appear as if unbooked women suffer more complications — and more deaths — than booked women, when this is really just an artifact of the differences in hospital utilization between these two groups. The differences in utilization may be due to a number of factors (e.g., socio-economic, geographical, cultural), but in any event, the two groups cannot be validly compared in this way. 101 APPENDIX C: Random Number Table 102 APPENDIX D: Sample Register Head It is likely that the records at some facilities will not currently have all of the data required for calculating the process indicators. Hopefully, this monitoring effort will help facility managers to perceive the need for maintaining good quality, complete records and will help them to improve record-keeping systems. The attached sample register head is provided as a model for administrators or managers to adapt for local use. The items included in this sample register should be considered the bare minimum of what is needed to calculate the process indicators. Of course, the register may be expanded to include more categories, such as discharge dates, etc. Also, many of the current column headings are broad enough to allow the recording of several different items. For instance, information about duration of pregnancy may be included in the ‘reason for admission’, column if relevant, and multiple births may be recorded in the ‘outcome: baby’ column. In some facilities, the ‘remarks’ column may be used to record financial information. 103 Patient Register Ward: Month(s) & Year No. Name of patient Usual address Age Parity Admission date/time Reason for admission Obstetric complications Treatment (include type of delivery*) Out- come: Mother Out- come: Baby Remarks *N = Normal; N.E. = Normal with Episiotomy; F.E. = Forceps with Episiotomy; M = Surgical Manipulation; C.S. = Caesarean Section; An = Any other.

View the publication