RPM Project- Rapid Pharmaceutical Management Assessment: An Indicator-Based Approach

Publication date: 1995

RAPID PHARMACEUTICAL MANAGEMENT ASSESSMENT: AN INDICATOR-BASED APPROACH July 1995 Rational Pharmaceutical Management Project Latin America and Caribbean Health Drug Management Program and Nutrition Sustainability Project Management Sciences for Health University Research Corporation 1655 North Fort Myer Drive, Suite 920 7200 Wisconsin Avenue, Suite 600 Arlington, VA 22209 USA Bethesda, MD 20814 USA (703) 524-6575 (301) 941-8490 In technical collaboration with: Regional Program on Essential Drugs Pan American Health Organization 525 23rd Street, N.W. Washington, DC 20037 USA (202) 861-3200 Table of Contents EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 A. Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 B. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 C. Development Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 D. Intended Users of this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 E. Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 I. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 A. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 B. Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 C. Cautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 II. INDICATORS DESCRIPTION FORMAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 III. DETAILED DESCRIPTIONS OF THE INDICATORS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 A. Policy, Legislation and Regulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 A.1 Existence of a national drug policy approved by the government . . . . . . . . . . . . . 17 A.2 Existence of comprehensive drug control legislation, regulations and enforcement agencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 A.3 Percentage of unregistered drug products in a sample of private sector drug retail outlets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 A.4 Type of drug registration information system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 A.5 Number of drugs registered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 A.6 Law permitting generic substitution by pharmacists . . . . . . . . . . . . . . . . . . . . . . . . 27 A.7 Practice of generic substitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 B. Formulary/Essential Drugs List and Drug Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 B.1 Number of unique drug products on National Drug Formulary List . . . . . . . . . . . . 31 B.2 Existence of an official manual, based on the National Drug Formulary List, providing basic drug information to prescribers, revised and published within the last five years . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 B.3 Percentage of MOH health facilities visited with the most current edition of an official manual based on the National Drug Formulary List . . . . . . . . . . . . . . . . . . 35 B.4 Existence of drug information centers that provide unbiased and current information to public health decision makers, health care providers and consumers . . . . . . . . . 36 C. Ministry of Health Budget and Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 C.1 MOH budget or expenditures on pharmaceuticals, US$ per capita . . . . . . . . . . . . 38 C.2 Existence of a system for recovering the cost of drugs dispensed in MOH health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 C.3 Percentage of patients who pay a charge for drugs they receive in MOH health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 C.4 Percentage of total government recurrent budget used for Ministry of Health . . . . 45 C.5 Percentage of total MOH recurrent budget allocated to pharmaceuticals . . . . . . . . 47 D. Ministry of Health Pharmaceutical Procurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 D.1 Existence of a policy limiting MOH pharmaceutical procurement to drugs on the National Drug Formulary List/National Essential Drugs List . . . . . . . . . . . . . . . . . 49 D.2 Percentage by value of MOH drugs purchased through a central procurement system50 D.3 Percentage of average international price paid for last regular procurement of a set of indicator drugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 D.4 Percentage by value of MOH drugs purchased through competitive tender . . . . . . 55 E. Ministry of Health Pharmaceutical Logistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 E.1 Weighted average percentage of inventory variation for a set of indicator drugs in MOH storage and health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 E.2 Average percentage of individual variation for a set of indicator drugs in MOH storage and health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 E.3 Average percentage of stock records that corresponds with physical counts for a set of indicator drugs in MOH storage and health facilities . . . . . . . . . . . . . . . . . . . . . . . 63 E.4 Average percentage of a set of unexpired indicator drugs available in MOH storage and health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 E.5 Average percentage of time out of stock for a set of indicator drugs in MOH storage and health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 F. Patient Access and Drug Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 F.1 Population per functional MOH health facility that dispenses drugs . . . . . . . . . . . 70 F.2 Population per licensed pharmacist or pharmacy technician in the public sector . . 72 F.3 Population per authorized prescriber in the public sector . . . . . . . . . . . . . . . . . . . . 74 F.4 Average number of drugs prescribed per curative outpatient encounter in MOH health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 F.5 Percentage of drugs prescribed by generic name in MOH health facilities . . . . . . 78 F.6 Percentage of drugs prescribed from the National Drug Formulary List in MOH health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 F.7 Percentage of outpatients prescribed injections at MOH health facilities . . . . . . . . 82 F.8 Percentage of outpatients prescribed antibiotics at MOH health facilities . . . . . . . 82 F.9 Percentage of prescribed drugs presented for dispensing that are actually dispensed in MOH health facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 G. Product Quality Assurance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 G.1 MOH drug product quality laboratory tests during the past year: (a) number of drug products tested and (b) total number of drug product quality tests performed . . . . 87 G.2 Use of WHO Certification Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 G.3 Existence of formal systems for reporting: (a) product quality complaints, and (b) adverse drug reactions (ADRs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 H. Private Sector Pharmaceutical Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 H.1 Population per licensed private sector drug retail outlet . . . . . . . . . . . . . . . . . . . . . 93 H.2 Number of licensed or registered drug retail outlets per government drug inspector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 H.3 Percentages of drug manufacturers, distributors, and drug retail outlets inspected during a one-year period . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 H.4 Total value of total private sector retail pharmaceutical sales, US$ per capita . . . . 98 H.5 Combined value of public sector pharmaceutical expenditures and private sector retail sales, US$ per capita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 H.6 Percentage of products on National Drug Formulary List that are currently manufactured or co-manufactured within the country . . . . . . . . . . . . . . . . . . . . . 102 H.7 Average of median private sector drug retail prices as a percentage of MOH acquisition prices for a set of indicator drugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 H.8 Existence of price controls for drugs in the private sector . . . . . . . . . . . . . . . . . . 106 H.9 Percentage of licensed drug retail outlets where an antibiotic was available without a prescription . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 IV. GUIDELINES FOR STUDY DESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 A. Gathering Vital Statistics and Background Information . . . . . . . . . . . . . . . . . . . . . . . . . . 113 B. Preparing an Overview of MOH Pharmaceutical Management Operations . . . . . . . . . . . 113 C. Specifying a List of Indicator Drugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 D. Designing a Survey of Health Facilities and Drug Retail Outlets . . . . . . . . . . . . . . . . . . . 115 E. Defining Approaches for Collecting Survey Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 F. Developing Data Collection Forms and Structured Interview Questionnaires . . . . . . . . . 121 G. Defining a Range of Dates for Retrospective Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 V. IMPLEMENTATION OF AN INDICATORS STUDY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 A. Work Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 B. Review of Data Collection Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 C. Logistical Arrangements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 D. Analyzing Data and Presenting Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Annex A: Sample Materials for an Overview of Pharmaceutical Management . . . . . . . . . . . . . . . . . . . . . 129 Annex B: Sample Lists of Indicator Drugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Annex C: Sample Data Collection Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Annex D: Sample Structured Interview Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Annex E: End Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 List of Tables Table 1: List of Pharmaceutical System Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Table 2: Pharmaceutical Indicators Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Table 3: Indicators to Be Measured with Survey Methods and Data Collection Forms . . . . . . . . . . . . . . . 122 Table 4: Provisional Work Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Table 5: Illustrative Training Course for Data Collectors in Health Facilities and Drug Retail Outlets . . . 128 ACKNOWLEDGMENTS This manual could not have been created, tested and revised without the assistance of a large number of interested professionals. We take pleasure in acknowledging the important contributions made by the following organizations and individuals. United States Agency for International Development Two USAID staff members deserve special mention for their roles in this activity. Nicholas Studzinski provided instrumental leadership in getting the work started through the Latin America/Caribbean Health and Nutrition Sustainability Project. Anthony Boni has provided unfailing support for continued development through the Rational Pharmaceutical Management Project. Latin America/Caribbean Health and Pan American Health Organization Nutrition Sustainability Project Thomas Bossert Kees De Joncheere Jack Galloway Enrique Fefer John Holley Adela Rosenkranz Aida Leroy Ileana Santich Harvard Medical School International Review Panel Drug Policy Research Group Dennis Ross-Degnan Kevin Armstrong Stephen Soumerai Ana Maria Arriagada Management Sciences for Health Drug Management Program James Bates Anthony Battersby Elvira Beracochea Jaime Benavente Enriqueta Clark Karin Bergstrom Vimal Dias Michael Bernhart Patricia Foreman Henk Den Besten Pradeep Goel Bimo Richard Laing Richard Blakney David Lee Roy Brooks Jeanne Madden Denis Broun Julie McFadyen Eugenie Brown Chris Olson Pascale Brudon-Jakobowicz James Rankin Patricia Bush Jean-Pierre Sallet Malcolm Butler Anthony Savelli Patrizia Carlevaro Sarah Schuler Sidney Chernenkoff Waichi Wong Rosario D'Alessio Fernando Antezana Jeffrey Ashley Paul Auxila Ola A. Ayanbeku Wilbert Bannenberg Azad Chowdhury William McGreevey Agathe Wehrli Rene Christensen Marc Mitchell Albert Wertheimer Richard Church Catherine Morlock Brian Wilkinson Mariam Claeson Gabriel Mwaluko Richard Wilson Catherine Crone Coburn Richard Nelson Alonzo Wind Peter Cross Ole Nielsen Antonio Carlos Zanini Carlos Javier Cuellar Christine Norton Gabriel Daniel Nyi Nyi Ananda Mohan Das Ronald O'Connor Jan Dik Joseph Oddis Vinod Diwan David Ofori-Adjei Lina Domingo William Oldham Graham Dukes Mead Over J.F. Dunne Agostino Paganini Edward Ellerbeck C. Pannenborg Steven Fabricant Clydette Powell Peter H.M. Fontilus Agma Prins Nadine Gasman Jonathan Quick Reginald Gipson Ralph Rack Peter Graaff David Radel Sherita Gregoire Michael Reich Hilbrand Haak Richard Roberts Anita Hardon Robert Robertson Paul Hartenberger John Rohde Jens Hasfeldt Donald Rucker Margaretha Helling-Borda Camille Saade Ned Heltzer Stephen Sacca Filiberto Hernández Lateef Salako Hans Hogerzeil Budiono Santoso Peter Huff-Rousselle Philippa Saunders Margaret Hume Gordon Schiff William Jansen Miriam Schneidman Stephen Jarrett David Sencer Keith Johnson Diana Silimperi Kumud Kafle Barry Smith Noreen Kerr James Socknat Robert Kim-Farley Steven Solter Rosalyn King A.O.O. Sorungbe Ved Kumar Göran Sterky Calvin Kunin Yutaka Suzuki Jean-Louis Lamboray E. Tarimo Karen Lashman-Hall Youssef Tawfik Laurence Laumonier-Ickx Göran Tomson Ernst Lauridsen James Tulloch Philip Lee Johannes Van Haperen Barry Levy Alexander van Ommen A.F.B. Mabadeje Marcelo Vernengo Pierre Mandl Taryn Vian Patricio Marquez Ruday Wabitsh John Martin Gill Walt Amos Massele Krisantha Weerasuriya EXECUTIVE SUMMARY A. Purpose This manual presents an indicator-based approach for rapidly assessing pharmaceutical management systems and programs. Such assessments have a number of potential applications, including: � Defining the status of the pharmaceutical system, including strengths and weaknesses, for managers and donors � Designing and planning interventions � Defining budget or resource requirements � Monitoring changes in systems and the impact of interventions � Comparing the performance of different systems, programs, or countries Rapid assessments may be indicated when the existing management information system is unable to track the performance of the pharmaceutical system, or when urgent drug problems are evident. The assessment should result in an analysis of why the problems that exist have come to pass, which problems can be solved, and what interventions are feasible in terms of cost-effectiveness and sustainability. B. Scope The manual presents a set of 46 indicators of performance, grouped under eight topics of pharmaceutical management, which are listed below. � Policy, Legislation and Regulation � Formulary/Essential Drug Lists and Drug Information � Ministry of Health Budget and Finance � Ministry of Health Pharmaceutical Procurement � Ministry of Health Pharmaceutical Logistics � Patient Access and Drug Utilization � Product Quality Assurance � Private Sector Pharmaceutical Activity Each topic is covered by a subset of indicators. Thirty-four of the indicators are quantitative, that is, expressed as numbers. Twelve are qualitative, in that they describe the presence or absence of a policy or management system, and in some cases, the degree of implementation. Carrying out an assessment that includes all 46 indicators will provide a reasonably complete overview of pharmaceutical systems operations in a given country. It is also valid to use the subsets of indicators selectively, according to need. This approach could be used for focused assessments based on, for example, the indicators for logistics or the indicators for drug utilization. C. Development Process In developing this manual, our overall objective has been to produce a concise set of proposed indicators for use in the context of a rapid assessment method that can be carried out in a reasonable period of time by non-specialists. To be useful over the long run, it is necessary that each of the indicators in the set meet the following criteria. 2 Rapid Pharmaceutical Management Assessment � Importance - Each indicator must reflect an important dimension of performance. � Measurability - Indicators must be measurable, within constraints of time and variable quality and availability of data. � Reliability - Each indicator must be reliable over time and with different observers. � Validity - Each indicator must allow a consistent and clear interpretation and have a similar meaning across different environments. To meet these criteria, we began by soliciting recommendations from a panel of over 150 professionals working internationally in drug management, who subsequently produced a preliminary list of over 200 items. From this list, we distilled a list of 33 indicators for field testing. Next, teams of international and local experts carried out assessments in 11 countries, including Guatemala, Ecuador, El Salvador, Nicaragua, Jamaica, the Organization of Eastern Caribbean States, Ghana, Mozambique, Nepal, Cambodia and Russia. Based on the outcomes of these field tests, we made revisions, added new indicators and produced the current set of 46 indicators presented in this manual. The work was done in close collaboration with PAHO, with a view to harmonizing this set of indicators with data collected by PAHO in a survey of Central American countries. D. Intended Users of this Manual Depending on the purpose and context of assessment activities, likely users of this manual include: � Ministry of Health (MOH) decision makers, health planners, health economists, donor representatives, or experts responsible for project design and evaluation activities � System managers at the national, regional or local levels wishing to measure the performance of specific systems or activities, such as drug registration, product selection, procurement, stock record keeping or prescribing � Social scientists, and health project or facility managers who are interested in operational research and management tools Other potential "users" of this manual are "system participants" who might benefit from the specific findings that an indicator-based assessment produces. For example, indicators measuring the performance of stock record keeping systems or prescribing practices can be usefully reported to staff at warehouses or clinical facilities to give them an idea of how their own performance compares with that of others. Users have basically three options for gathering data for the indicators study: � Structured Assessment - Field visits by a dedicated team that makes use of structured survey instruments � Limited Assessment - Interviews, document review using less formal assessment methods � Self-Assessment - Informal questionnaires and working groups to determine the nature and extent of the problems Executive Summary 3 E. Contents The manual is designed to take users step-by-step through the assessment process, beginning with introducing the concept of indicator-based assessments, and ending with producing an assessment report. There are five chapters in the manual: I. Introduction: Provides background on the history of indicator-based assessments and the development of this manual. II. Indicators Description Format: Defines the standard format for presenting the indicators. III. Detailed Descriptions of the Indicators: Explains, for each indicator, the rationale, definition, data collection, computation, and sample presentations of results. IV. Guidelines for Study Design: Provides details for survey design and data collection activities. V. Implementation of an Indicators Study: Covers the practical problems of carrying out an assessment, including logistical arrangements and recruiting and training data collectors. I. INTRODUCTION This manual is the result of an ongoing effort to develop and test a rapid assessment method based on a set of process and outcome indicators that can be used to monitor and compare significant aspects of pharmaceutical systems. The work has been done under the auspices of two projects financed by the United States Agency for International Development (USAID). Initial development and field testing of 33 indicators, and various methods for data collection, were done by a working group led by the Management Sciences for Health Drug Management Program and the Harvard Drug Policy Research Group, with support and guidance from the Latin America/Caribbean Health and Nutrition Sustainability Project (LA/C-HNS), a USAID-funded project managed by University Research Corporation. Input into the first draft list of indicators was also provided by the Ghana and Indonesia Core Groups of the International Network for Rational Use of Drugs (INRUD). Additional field tests, and revisions of both methods and the manual itself, have been accomplished through country pharmaceutical sector assessments performed by the USAID-financed Rational Pharmaceutical Management Project (RPM), which is composed of cooperative agreements with Management Sciences for Health (MSH) and the United States Pharmacopeial Convention, Inc. (USP). As a result, the original set of indicators has been expanded to 46. This set is given as Table 1 (page 10). As of December 1994, data have been collected in nine countries, and the results are presented in Table 2 (page 12). The method has subsequently been applied in Cambodia and Russia, but the data from these assessments have not yet been incorporated into the comparison matrix. As a review of these two tables shows, the indicators are grouped under eight topics of drug management: � Policy, Legislation and Regulation � Formulary/Essential Drug Lists and Drug Information � Ministry of Health Budget and Finance � Ministry of Health Pharmaceutical Procurement � Ministry of Health Pharmaceutical Logistics � Patient Access and Drug Utilization � Product Quality Assurance � Private Sector Pharmaceutical Activity In reviewing Table 2, it is important to note that, as the indicator set has undergone testing and revision, a number of indicators have evolved. In general, this evolutionary process has been in the direction of more specificity and focus, based on the data actually available. One consequence of this is that, in some cases, indicator data collected in earlier tests are not complete enough for purposes of expressing the revised indicators. Where this has occurred, the decision has been made to not include the incomplete data in the table. Although this produces gaps in the presentation, it maintains the quality and integrity of the data set. A. Background Pharmaceutical supply systems in many developing countries have severe problems, including ineffective procedures in selection, poor quality control, and economically inefficient procurement. This has been documented in numerous reports and publications. Nonetheless, there has been no standard method for1 measuring the effectiveness of these systems. The LA/C-HNS and RPM work described in this manual builds on and complements other activities directed at developing indicators for pharmaceutical systems and sectors. 6 Rapid Pharmaceutical Management Assessment Indicators have been used for years in "developed" countries to measure aspects of pharmaceutical systems. Notable examples of organizations that have used indicators in the United States include the U.S. Public Health Service, Division of Health Care Delivery, the Joint Commission on Accreditation of Healthcare Organizations, and the American Society of Hospital Pharmacists. The focus of the indicators developed by these organizations is relatively specific to U.S. institutions. The World Health Organization, and in particular the WHO Action Programme on Essential Drugs, have recognized the need for formal indicators that would facilitate comparative analyses of developing country pharmaceutical systems. In 1988, WHO published the World Drug Situation, which presented an impressive amount of useful information, organized into an indicator format. These indicators were, in2 most cases, not presented in a quantitative format, but this work stimulated widespread interest in the concept of pharmaceutical system indicators. In the fall of 1994, the WHO Action Programme on Essential Drugs issued a draft manual, entitled Model Indicators for Monitoring National Drug Policies, which proposes a set of 31 background indicators, 50 structural indicators, 38 process indicators, and 10 outcome indicators. These indicators are intended for3 self use by developing countries to monitor their pharmaceutical systems. The WHO Action Programme on Essential Drugs worked with a group including the Harvard School of Public Health and the Centre de Recherches et d'Etudes pour le Développement de la Santé (CREDES - Paris, France) to develop these indicators. Field testing is planned for 1995. The Australian government has recently adapted the WHO indicator format to develop its own set of drug policy indicators for use in Australia. This set has 41 process indicators, 27 impact indicators and 64 outcome indicators. In this model, process indicators are equivalent to WHO structural indicators, and impact indicators are similar to WHO process indicators. Another group active in developing and testing indicators is the International Network for Rational Use of Drugs (INRUD). INRUD is a network promoting rational drug use in developing countries, involving 10 African and Asian member countries. It is coordinated by MSH, with additional technical support provided by the Harvard Drug Policy Research Group, the Karolinska Institute, the University of Newcastle, and the WHO Action Programme on Essential Drugs. Network sponsors include Danida, WHO, USAID, SIDA, and the Pew Charitable Trusts. INRUD developed and field tested a set of indicators related to drug use review in primary care health facilities. The INRUD drug use indicators have been adopted by WHO as the standard methodology for assessing drug use, and published as the Action Programme on Essential Drugs' manual How to Investigate Drug Use in Health Facilities. These indicators also form the core of drug use5 assessment in this manual. Introduction 7 In addition, the Pan American Health Organization (PAHO) sponsored a multi-country project to develop and test indicators which could be used to measure progress of Essential Drugs Programs in Central America. The data from this work were collated and assembled by the PAHO Essential Drugs Advisor for the subregion. In October 1993 LA/C-HNS, RPM and PAHO technical staff met for the purpose of standardizing terminology and methods. The indicators in this manual were modified according to the recommendations made at this meeting, and, in light of the more advanced stage of development and testing of the MSH indicators and their compatibility with PAHO's previous effort, the Regional Essential Drugs Program discontinued its field work in this area. Subsequently, in August 1994, PAHO technical staff responsible for PAHO/WHO Essential Drugs Programs in Latin America met in Lima, Peru, and agreed that the revised RPM and LA/C-HNS indicators would be used, as required, for purposes of program development and evaluation. PAHO will be collaborating in workshops and other activities to promote the use of the indicators by the managers of national Essential Drugs Programs. Discussions have also taken place with the WHO Action Programme on Essential Drugs to explore the possibility of harmonizing the terms and methods used in this manual with those used in Model Indicators for Monitoring National Drug Policies. The two methods are complementary rather than competitive, in that they have different approaches to data collection. The RPM/LA/C-HNS approach is primarily a rapid assessment method, while the WHO approach relies on self reporting by country officials. It is expected that steps toward harmonization will occur after the WHO Model Indicators are field tested. B. Objectives The goal of the LA/C-HNS and RPM indicators development process is to develop an indicator-based rapid assessment methodology that can be used by developing country managers, international agencies, and donors to monitor pharmaceutical systems on a regular basis. This means that the indicator set and assessment methodology should be as concise as possible, so that the data can be collected in a reasonable time frame. There are four general criteria for useful indicators, which are listed below. � Importance - Each indicator must reflect an important dimension of performance. Certain data may be readily and consistently available, but say nothing important about the system performance. � Measurability - Each indicator must be measurable, within existing constraints of time and variable quality and availability of source data. � Reliability - Each indicator must be reliable over time and with different observers. If one observer reports a certain result from a set of data, it is expected that a second observer will report the same result. � Validity - Each indicator must allow a consistent and clear interpretation and have a similar meaning across different environments. 8 Rapid Pharmaceutical Management Assessment It is important to recognize that, although the term "indicators" is used for convenience, the indicators proposed are not yet indicators in the truest sense, but rather they are standardized measurements of a local pharmaceutical system. There is not enough data available yet to be able to establish normative ranges for any of the measures, or to determine which of the measures have sufficient diagnostic worth to be regarded as standard indicators. For example, we may be able to say that a country has a national formulary with 400 discrete drug products, but we are unable to say if this is too many or too few items for a national formulary list, or indeed whether a national formulary list is a necessary feature of a well functioning pharmaceutical system. As more data are gathered using consistent methods, it should become possible to determine to what extent these proposed standard measurements meet the criteria of importance, measurability, reliability, and validity, and to begin to establish normal ranges of performance for each of the indicators. C. Cautions It is important to keep in mind that this list of indicators, like any other, has limitations. When interpreting the data presented in Table 2 (page 12), or indicator data which users of this manual may collect themselves, four important cautions should be observed: � First, when collecting indicator data, it is essential to systematically collect descriptive background information. Such information is often required for usefully interpreting individual indicators. For example, consider indicator G.3, "Existence of a formal system for reporting product quality complaints." The indicator may be recorded as limited, indicating that such a system formally exists but does not function regularly in practice. Specificity can be added by providing background information, as in the following illustrative example: In country Y, the Drug Regulatory Authority is mandated by the health code to monitor and take action on product quality complaints. The status of this system is at best limited, because no standard reporting forms exist, and only one complaint was received during the twelve months preceding the assessment. There was no evidence of follow up action. � Second, to place the pharmaceutical indicator data in context, additional information about the local situation may be required. The indicators described here are not intended to stand alone, but rather to be used in conjunction with compilations of social and health indicators such as the World Bank World Development Report. � Third, it should not be assumed that the indicator data must be collected on an all-or-nothing basis. For example, if for some reason the indicators in section H, "Private Sector Pharmaceutical Activity" cannot be collected, it is still worthwhile to collect the indicators covering the public sector system. In fact, one recent application of the indicator set, carried out in Cambodia, demonstrated that selected indicators could be drawn from various sections of the overall set to meet the needs of local decision makers. � Fourth, not all of the indicators may be reliably compared from country to country and region to region. Indicators for drug prices, drug availability, stock record keeping and drug use may be usefully compared across a spectrum of countries when supplemented with appropriate background information. Financial indicators, however, should be compared with great caution because public and private sector financial contexts are so complex and vary so significantly from country to country, and region to region. Table 1: List of Pharmaceutical System Indicators A. POLICY, LEGISLATION AND REGULATION 1. Existence of a national drug policy approved by the government 2. Existence of comprehensive drug control legislation, regulations and enforcement agencies 3. Percentage of unregistered drug products in a sample of private sector drug retail outlets 4. Type of drug registration information system 5. Number of drugs registered 6. Law permitting generic substitution by pharmacists 7. Practice of generic substitution B. FORMULARY/ESSENTIAL DRUGS LIST AND DRUG INFORMATION 1. Number of unique drug products on the National Drug Formulary List 2. Existence of an official manual, based on the National Drug Formulary List, providing basic drug information to prescribers, revised and published within the last five years 3. Percentage of MOH health facilities visited with the most current edition of an official manual based on the National Drug Formulary List 4. Existence of drug information centers that provide unbiased and current information to public health decision makers, health care providers and consumers C. MINISTRY OF HEALTH BUDGET AND FINANCE 1. MOH budget or expenditures on pharmaceuticals, US$ per capita 2. Existence of a system for recovering the cost of drugs dispensed in MOH health facilities 3. Percentage of patients who pay a charge for drugs they receive in MOH health facilities 4. Percentage of total government recurrent budget used for Ministry of Health 5. Percentage of total MOH recurrent budget allocated to pharmaceuticals D. MINISTRY OF HEALTH PHARMACEUTICAL PROCUREMENT 1. Existence of a policy limiting MOH pharmaceutical procurement to drugs on the National Drug Formulary List 2. Percentage by value of MOH drugs purchased through a central procurement system 3. Percentage of average international price paid for last regular procurement of a set of indicator drugs 4. Percentage by value of MOH drugs purchased through competitive tender E. MINISTRY OF HEALTH PHARMACEUTICAL LOGISTICS 1. Weighted average percentage of inventory variation for a set of indicator drugs in MOH storage and health facilities 2. Average percentage of individual variation for a set of indicator drugs in MOH storage and health facilities 3. Average percentage of stock records that corresponds with physical counts for a set of indicator drugs in MOH storage and health facilities 4. Average percentage of a set of unexpired indicator drugs available in MOH storage and health facilities 5. Average percentage of time out of stock for a set of indicator drugs in MOH storage and health facilities 10 Rapid Pharmaceutical Management Assessment F. PATIENT ACCESS AND DRUG UTILIZATION 1. Population per functional MOH health facility that dispenses drugs 2. Population per licensed pharmacist or pharmacy technician in the public sector 3. Population per authorized prescriber in the public sector 4. Average number of drugs prescribed per curative outpatient encounter in MOH health facilities 5. Percentage of drugs prescribed by generic name in MOH health facilities 6. Percentage of drugs prescribed from the National Drug Formulary List in MOH health facilities 7. Percentage of outpatients prescribed injections at MOH health facilities 8. Percentage of outpatients prescribed antibiotics at MOH health facilities 9. Percentage of prescribed drugs presented for dispensing that are actually dispensed in MOH health facilities G. PRODUCT QUALITY ASSURANCE 1. MOH drug product quality laboratory tests during the past year: (a) number of drug products tested, and (b) total number of drug product quality tests performed 2. Use of WHO Certification Scheme 3. Existence of formal systems for reporting: (a) product quality complaints, and (b) adverse drug reactions (ADRs) H. PRIVATE SECTOR PHARMACEUTICAL ACTIVITY 1. Population per licensed private sector drug retail outlet 2. Number of licensed or registered drug retail outlets per government drug inspector 3. Percentages of drug manufacturers, distributors, and drug retail outlets inspected during a one- year period 4. Total value of private sector retail pharmaceutical sales, US$ per capita 5. Combined value of public sector pharmaceutical expenditures and private sector retail sales, US$ per capita 6. Percentage of products on National Drug Formulary List that are currently manufactured or co- manufactured within the country 7. Average of median private sector drug retail prices as a percentage of MOH acquisition prices for a set of indicator drugs 8. Existence of price controls for drugs in the private sector 9. Percentage of licensed drug retail outlets where an antibiotic was available without a prescription TABLE 2: PHARMACEUTICAL INDICATORS SET: COMPARISON OF SELECTED COUNTRIES NEPALOECS (B)JAMAICANICARAGUA (A)GUATEMALAEL SALVADORECUADORECUADORGHANAMOZAMBIQUE 1993199319921994199219931994199219931993 A. POLICY, LEGISLATION AND REGULATION NoNoNoYesNoYesYesNoNo1. Existence of a national drug policy approved by the government YesNoYesNoYesYesYesNo2. Existence of comprehensive drug control legislation, regulations and enforcement agencies 100%21%7.3%23%N/A0%N/AN/A3. % of unregistered drug products in a sample of private sector drug retail outlets ManualNoneManualComputerizedManualMixedMixedManualNone4. Type of drug registration information system 11,000+7,00619,7001,5745. Number of drugs registered No No NoNoNoNoNo No No N/A6. Law permitting generic substitution by pharmacists 7. Practice of generic substitution B. FORMULARY/ESSENTIAL DRUGS LIST AND DRUG INFORMATION 2613881,0102344282844384382223831. Number of unique drug products on National Drug Formulary List YesYesYesNoYesYesYesYesNoNo2. Existence of an official manual, based on the NDFL, providing basic drug information to prescribers, revised and published within last 5 years N/A100%N/A7%0%0%70%25% 45%100%3. % of MOH health facilities visited with the most current edition of an official manual based on the NDFL 4. Existence of drug information centers that provide unbiased and current information to public health NoNoYesYesYesNoNoNoNo decision makers, health care providers and consumers C. MINISTRY OF HEALTH BUDGET AND FINANCE $5.50$1.98$1.13$3.93$4.96$0.25$0.09$0.46 (C)$ 0.621. MOH budget or expenditures on pharmaceuticals, US$ per capita YesYesYesYesYesYesYesYesYes2. Existence of a system for recovering the costs of drugs dispensed in MOH health facilities N/A0%N/AN/A3. % of patients who pay a charge for drugs they receive in MOH health facilities 4%12%3%15%15%8%14%6%4. % of total government recurrent budget used for MOH 6%8%16%26%4%1%No budget2%5. % of total MOH recurrent budget allocated to pharmaceuticals All dollar amounts are in U.S. dollars. N/A indicates that information was not available despite attempts to collect it. Blanks indicate that these indicators were not part of the original studies. Grey shading indicates that the indicator is new or has been changed since the assessment, as a result of field tests. For some indicators, data is available from information already collected. (A) Nicaragua results are based on data collected for the USAID-funded Decentralized Health Services Project. They include two regional/intermediate medical stores and 20 health facilities. These results may or may not represent the country as a whole. (B) OECS is the Organization of Eastern Caribbean States. The countries studied included: Dominica, Grenada, Montserrat, St. Kitts and Nevis, St. Lucia, and St. Vincent and the Grenadines. (C) Ghana CMS purchases only; there were also substantial direct purchases by regional stores and health facilities. D. MINISTRY OF HEALTH PHARMACEUTICAL PROCUREMENT YesYesYesYesYesYesYesYesYesYes1. Existence of a policy limiting MOH pharmaceutical procurement to drugs on NDFL N/A100%80%27%80%<50%<50% N/A100%2. % by value of MOH drugs purchased through a central procurement system 63%147%145%164-371%114%161%79%76-205%3. % of average international price paid for last regular procurement of a set of indicator drugs 50%100%95%10%80%28%45%45% (D)91%4. % by value of MOH drugs purchased through competitive tender E. MINISTRY OF HEALTH PHARMACEUTICAL LOGISTICS 48%0%124%Tally:CMS1. Weighted average % of inventory variation for a set 8% 3% RMS of indicator drugs in MOH storage and health facilities 8%11% HF 19% 5%0%3%15%159% Ledger:CMS 5%5% RMS 38%16% HF Tally:CMS2. Average % of individual variation for a set of indicator RMS drugs in MOH storage and health facilities HF Ledger:CMS RMS HF 0% Tally:CMS3. Average % of stock records that corresponds with RMS physical counts for a set of indicator drugs HF in MOH storage and health facilities 14%Ledger:CMS 23% RMS HF 79% 100%93%94%93%100%67% CMS4. Average % of a set of unexpired indicator drugs available 58% 89%81%72%87%87%80% RMS in MOH storage and health facilities 95%72%60%78%46%38%60%59% HF 15% 27%32%23%79%8% CMS5. Average % of time out of stock for a set of indicator drugs 21%7% RMS in MOH storage and health facilities 13%12%11% HF All dollar amounts are in U.S. dollars. N/A indicates that information was not available despite attempts to collect it. Blanks indicate that these indicators were not part of the original studies. Grey shading indicates that the indicator is new or has been changed since the assessment, as a result of field tests. For some indicators, data is available from information already collected. (D) 87% of the Ghana central procurement was done competitively; about 52% of drugs purchased by regional stores came through the central agency. Thus, 45% of the MOH drugs were purchased competitively. INRUD Avg. (E)F. PATIENT ACCESS AND DRUG UTILIZATION 15,6003,9455,8558,6228,52914,4306,3076,31035,25313,7981. Population per functional MOH health facility that dispenses drugs 2. Population per licensed pharmacist or pharmacy technician in the public sector 3. Population per authorized prescriber in the public sector 2.12.12.02.42.11.42.22.01.34.32.44. Average # of drugs prescribed per curative outpatient encounter in MOH health facilities 67%44%49%40%86%71%72%39%37%59%99%5. % of drugs prescribed by generic name in MOH health facilities 6. % of drugs prescribed from the NDFL in MOH health facilities 25%5%2%4%10%13%7%19%17%56%20%7. % of outpatients prescribed injections at MOH health facilities 43%43%39%30%34%27%32%42%27%47%52%8. % of outpatients prescribed antibiotics at MOH health facilities 77%83%84%92%61%86%77%9. % of prescribed drugs presented for dispensing that are actually dispensed in MOH health facilities G. PRODUCT QUALITY ASSURANCE 1. MOH drug product quality laboratory tests during the past year: N/A430N/AN/A N/A<10 N/A (a) number of drug products tested 960N/A03,042N/AN/AN/AN/A (b) total number of drug product quality tests performed N/ANoNoNoNoYesYesNo2. Use of WHO Certification Scheme 3. Existence of formal systems for reporting: NoneFunctionalNoneNoneLimitedNoneNoneNoneNoneNone (a) product quality complaints NoYesNoNoYesYesNoNoNo (b) adverse drug reactions (ADRs) H. PRIVATE SECTOR PHARMACEUTICAL ACTIVITY 8,1789,7204,8054,8353,5693,4193,438285,1561. Population per licensed private sector drug retail outlet 1 Grenada inspect. (F)63947no inspectors13262192. Number of licensed or registered drug retail outlets per government drug inspector 3. %s of drug manufacturers, distributors, and drug retail outlets inspected during a one-year period N/A$10.29$10.98$11.09$18.98$12.84N/A$ 0.084. Total value of private sector retail pharmaceutical sales, US$ per capita N/A$12.27$14.91$16.05$19.23$12.93N/A$ 0.855. Combined value of public sector pharmaceutical expenditures & private sector retail sales, US$ per capita 7%0%15-20%71%50%50%70%1%6. % of products on NDFL which are currently manufactured or co-manufactured within the country 7. Average of median private sector drug retail prices as a % of MOH acquisition prices for a set of indicator drugs NoYesYesYesYesNo8. Existence of price controls for drugs in the private sector N/A100%100%95%100%85%9. % of licensed drug retail outlets where an antibiotic was available without a prescription All dollar amounts are in U.S. dollars. N/A indicates that information was not available despite attempts to collect it. Blanks indicate that these indicators were not part of the original studies. Grey shading indicates that the indicator is new or has been changed since the assessment, as a result of field tests. For some indicators, data is available from information already collected. (E) INRUD Average is based on data collected by the International Network for Rational Use of Drugs in eleven countries from 1989 to 1992. (F) Grenada has one inspector. The other OECS countries do not have any inspectors. About the Indicators Description Format 13 II. INDICATORS DESCRIPTION FORMAT Chapter 3 presents detailed descriptions of each of the 46 proposed indicators, grouped under the eight topics identified in chapter 1. Each description follows exactly the same format, which is summarized below. Indicator data can be collected at three different levels of the health care system. Each indicator in the descriptions that follow is coded according to the level at which it is measured, with the code appearing after the indicator title. The level codes used are: C Central level - under direct supervision of the central government R Regional or district level - acts as the intermediary; provides supplies to the health facilities and not directly to patients F Health facility level - provides direct care to the patient population Indicator Name: The name of the indicator along with the different system levels that may be examined (for example, C/R/F signals that the indicator may be applied at the central, regional and health facility levels). Rationale: The reason that the indicator is important. Definition: The meaning of the indicator, and the terms used to describe this indicator. Data Collection: The most likely source(s) of information are summarized in a table indicating where the data are to be collected, who to ask for assistance, and what documents and records to review. Brief discussions of methods and issues related to data collection. Citations of the data collection forms to be used, if any. Data for 18 indicators are collected using eight different forms. There is a discussion of how to develop the required forms in chapter 4, and examples given in Annex C. Computation & Example: Computations, if any, which are needed, accompanied by an example using illustrative data. Presentation: Brief example of how results may be presented. Notes: Suggestions for additional information or discussion required to put the indicator in proper context, or to provide more detail. 14 Rapid Pharmaceutical Management Assessment III. DETAILED DESCRIPTIONS OF THE INDICATORS A. Policy, Legislation and Regulation A.1 Existence of a national drug policy approved by the government (C)* Rationale: National drug legislation should be a guide to action in three areas: providing quality pharmaceuticals to the public, regulating control of pharmaceuticals, and supporting cost-effective management. To be effective, health policies should be based on reliable information. When approved by the appropriate authorities, the government explicitly commits itself to implement and enforce the policy. The health needs and priorities of the population change; therefore, policies need to be reviewed to ensure that they respond to changing needs. This indicator will assess the existence or absence of a document containing the national drug policy (NDP), as recommended by WHO. This is a measure of the government's political commitment to improving the pharmaceutical management in both the public and private sectors. Definition: A national drug policy is a written document, officially approved by the national government, and used as the basis for current policy in the pharmaceutical sector. It has guidelines pertaining to three areas: � Control of import, export, manufacture, and pricing of drugs, and of distribution, supply, storage and sale � Authority for regulation of labelling, information and advertising, drug registration, scheduling of controlled substances, imposition of fees and price controls � Drug control administration: organization and function, prescribing and dispensing restrictions, mechanisms of appeals against decisions Data Collection: Where to go Who to ask What to get MOH Senior administrative official, Document(s) containing the Chief Pharmacist National Drug Policy Health Policy and/or Planning Officer in charge Office Drug Regulatory Authority Officer in charge of registry Central Government Registry or Officer in charge National Archives Government official newspaper or Information Officer Record of published dissemination gazette of the NDP This national drug policy should be identified through interviews with key officials and a search of records. A copy should be made available to the assessment team. See page 15 for description of codes.* 16 Rapid Pharmaceutical Management Assessment Computation & Example: Record the existence of a national drug policy according to the following categories: � Yes, WHO, if a national drug policy along WHO guidelines exists (include year approved or revised) � Yes, not WHO, if a national drug policy exists, but does not follow WHO guidelines (include year approved or revised) � In process, if a WHO-style national drug policy is in the process of development (include expected date of completion) � No, if there is no national drug policy Presentation: In country X, the national drug policy, which follows WHO guidelines, is described within the "Política Nacional del Sector Salud," approved by the National Health Council in 1988 and revised in October 1990. Thus country X is rated yes, WHO. The policy addresses issues regarding the supply of drugs and the control of importation and local manufacturing, but does not include the registration of drugs for sale in the country. A copy of the policy is attached in the annex. Notes: The ideal way to present data for indicators such as this one is to record the rating plus annotation, such as dates. For example: Yes WHO, 1989. However, for relatively complex tables which compare data from several countries, such as Table 2 (page 12), this would result in a visually crowded presentation that would be difficult to read. In such cases it is best to present just the rating. When presenting data for just one country, however, it is more feasible to present both rating and annotation. Detailed Descriptions of the Indicators 17 A.2 Existence of comprehensive drug control legislation, regulations and enforcement agencies (C) Rationale: Drug control legislation and regulations, and functioning agencies that enforce them, are measures of a government's capacity to implement beneficial policies and practices in pharmaceutical management. If a government does not allocate financial resources or staff to the agencies charged with enforcing legislation and regulation, this suggests a situation where plans for pharmaceutical improvement will exist on paper but are not implemented in reality. Definition: Comprehensive drug control legislation is defined as written, government-approved legislation and regulation applying to: drug manufacturing, registration, licensing, distribution, prescribing, and sales practices, as well as regulations assigning official responsibility for enforcement of these rules. A functioning drug regulation enforcement agency is one that has sufficient staff and funding to carry out policy mandates. Reports of its activities should be available, and discussions with its staff will indicate the extent of their activities. Data Collection: Where to go Who to ask What to get MOH Pharmaceutical Services Legislation and regulation Director/Chief/Secretary documents, name of agency/persons in charge of enforcing them Drug Regulatory Authority Officer in charge of registry Ministry of Industry Officer in charge of importation Legislation documents, registers, and control, registry of retail and reports of inspections pharmacies, and of protection of local industry Pharmacists Association and Association Executive, Director or Verification of awareness of drug Pharmacy Owners Association Secretary control legislation and compliance Member pharmacists with the laws Through document review and interviews with the Chief Pharmacist in the MOH, the Head of the Drug Regulatory Authority, or other similar agency, determine if legislation exists in areas listed below, and which agency has been assigned responsibility for enforcing these regulations. If possible, check with the pharmacy association about awareness of and enforcement of these laws. 18 Rapid Pharmaceutical Management Assessment Computation & Example: Record the existence of drug control legislation, regulations and enforcement agencies for drug manufacturing, drug registration, drug distribution, authorization to prescribe drugs, authorization to sell drugs and drug marketing and promotion as follows: � Yes, if drug control legislation, regulations and enforcement agencies all exist (include relevant dates and titles of legislation, regulations or agency names) � Partial, if some of these elements are in place but others are not � No, if there is no drug control legislation, regulation, or enforcement agency Presentation: In country A, the existence of components of drug control legislation was as follows: Legislative Area Legislation in Place Enforcing Agency in Place Drug manufacturing Yes, Food & Drug Act, 1989 Yes, Pharmaceutical Supply Division, MOH (PSD/MOH) Drug registration Yes, Food & Drug Act, 1989 Yes, PSD/MOH Drug distribution Yes, Food & Drug Act, 1989 Yes, PSD/MOH Authorization to prescribe drugs Yes, Medicines Act, 1989 Yes, National Medical Association Authorization to sell drugs Yes, Medicines Act, 1989 Yes, National Pharmacy Council Drug marketing and promotion Yes, Medicines Act, 1989 YES, PSD/MOH Detailed Descriptions of the Indicators 19 A.3 Percentage of unregistered drug products in a sample of private sector drug retail outlets (C) Rationale: The percentage of unregistered drugs available for sale is a measure of the degree to which a government enforces its drug control policies and protects its people from products of uncertified quality and effect. High percentages of unregistered products indicate a regulatory agency that does not function effectively. This may be due to such factors as insufficiency of staff, financial resources or political will. Definition: This indicator measures how completely drug registration regulations are being implemented. A product is considered officially registered when the drug is listed on an official register of products approved for sale or distribution in the country. Drugs not listed on this official register are considered to be unregistered. The indicator applies to all pharmaceutical drug products identified in national legislation as requiring registration, both legend (prescription required) and non-legend. Products classified as foodstuffs, diagnostics and medical supplies are usually excluded. Vitamins may or may not be included as pharmaceutical drug products. Note: If drug registration is not required by law, this indicator is not relevant. Data Collection: Where to go Who to ask What to get MOH Pharmaceutical Services Director/ List of registered drugs, how often Chief Pharmacist it is updated, purged, date of last entry, waiting list to be registered Drug Regulatory Authority Drug Registration Inspector Review of drug registration information system, reports of sale of unregistered drugs, list of registered drugs Private drug retail outlets Pharmacist/Dispenser Check products on shelf for number of unregistered drugs out of 200 selected brand names Select a sample of 20 retail drug sales outlets. At each site select from the shelves 10 products, and record the complete name (both brand and generic), and name of manufacturer and/or supplier listed on the product label. If the law requires a product license number on the package, record this number from sample products. This gives a total sample of 200 products. If working with more than one data collector, duplication can be avoided by assigning each data collector an exclusive range of letters of the alphabet, with instructions to select products within the assigned range. For example, if there are three data collectors (Joe, Mary, and Bob), then Joe selects only products starting with the letters A-H, Mary selects products starting with the letters I-Q, and Bob selects products starting with letters R-Z. Prior to the start of the assessment, all the data collectors should be instructed to carry out this part of the work at the same time. 20 Rapid Pharmaceutical Management Assessment See the "Drug Registration Data Form" in Annex C, page 145. Computation & Example: Compare the sample of 200 product names collected with the most current list or database of registered products. Check to see if any of the names in the sample are not registered. Record the number of unregistered products. Compute the percentage of unregistered sample products by dividing the number of unregistered products by the number of products sampled; multiply this quotient by 100 to obtain the percentage. % of Unregistered = Number of Unregistered Drug Products x 100 Drug Products Number of Drug Products Sampled % of Unregistered = 14 x 100 = 7% Drug Products 200 Presentation: In country A, 7% of the drug products, or 14 out of a sample of 200, selected during July 1993 were found to be unregistered. Detailed Descriptions of the Indicators 21 A.4 Type of drug registration information system (C) Rationale: The type and characteristics of the drug registration system a country has in place is a measure of the government's ability to collect and retrieve information on drug registration and use it to monitor and evaluate the degree to which policies and regulations are enforced. Computerized systems, when functional, are far more efficient in organizing and recovering information than manual systems. The degree of usefulness of the registration system is measured in terms of the speed, type and amount of information that it can generate to support decision-making and action. Definition: A drug registration system should record pharmaceutical product information supplied for registration, including the name and contact information of the company which registered the drug, the name of the manufacturing company (if different from the company registering), the country where the product was manufactured, when it was registered, and complete product description (which includes therapeutic effect, dosage, side effects, and shelf life). A drug registration system should not be static. Manufacturers and distributors should periodically register their drugs; therefore, regular reviews ensure that drug information is up-to-date. Note: If drug registration is not required by law, this indicator is not relevant. Data Collection: Where to go Who to ask What to get MOH Pharmaceutical Services A demonstration of how the Director/Chief/Secretary registration system works, information (brand names, strengths and dosages, and companies that registered the drugs) about two or three common drugs selected by INN or generic name Drug Regulatory Authority Officer in charge of registry Ministry of Industry Officer in charge of importation Description of registration system and control, and protection of local industry Interview personnel who work directly with the drug registration system. Ask them to show how the system works. To determine whether the system is manual or computerized, ask the following questions for two or three common drug products: � How many manufacturers or suppliers have registered the drug under the generic name or a brand name? � What are the strengths and dosage forms available? � Which companies have registered the drug? 22 Rapid Pharmaceutical Management Assessment Watch to see if this information is obtained through consultation of files and ledgers (indicating a manual system) or through use of a computer. In some cases, the respondent may consult both manual files and a computerized data base in order to answer the questions; this is termed a mixed system. Computation & Example: Describe the drug registration system as follows: � Manual (include date established) � Computerized (include date established) � Mixed (include dates for establishment of the manual system, and introduction of computerized elements) � None, if no system is established Presentation: In country A, the drug registration system is fully computerized. The system was established in 1989. The primary source document is the "Registration Application Form" submitted by the party wishing to register the product. One staff member enters the information from approved applications into the system. The system is menu- driven and designed to produce reports by a range of variables including brand and generic names, number of active ingredients, dosage forms, routes of administration, manufacturers and country of origin. Detailed Descriptions of the Indicators 23 A.5 Number of drugs registered (C) Rationale: The number of drugs registered is a rough measure of the functionality of a drug registration system and the degree to which the regulatory authority enforces drug control regulations. A relatively large number of registered products (in excess of 10,000) may indicate that (a) there is no effective effort to make sure only safe and effective products are registered, (b) products are not licensed for a specific time period, and in fact, many of the registered products may not really be marketed, or (c) the system is not being kept up-to-date, and products with expired licenses are not being purged from the records. Definition: A drug registration system, either manual or computerized, should be able to produce an accurate figure for the total number of drugs registered for import and sale in a country. In order to usefully interpret this indicator it is necessary to gather background information on the contents of drug policies and regulations, as well as the degree to which staff are able to keep the registration information system current and up-to-date. Data Collection: Where to go Who to ask What to get MOH Pharmaceutical Services Determine terms of registration, Director/Chief/Secretary number of drugs registered, frequency of license checks Drug Regulatory Authority Officer in charge of registry Number of registered drugs, date of last check, number of drugs delisted Ministry of Industry Officer in charge of importation Number of patented or registered and control and of protection of drugs, duration of registration local industry The information for this indicator may be collected at the time when interviews are held to determine the type of drug registration information system. Ask for the total number of drugs registered for sale in the country, and verify the source of information and the method of calculating the number of registered products. Computation & Example: Record the number of registered drugs provided by drug registration staff, obtained from the registration information system or other sources, as well as the source itself. Presentation: In country A, there are 7,006 registered products, of which 5,898 are brand named and 1,108 are generically named, according to a computerized report produced by staff in the drug registration process upon request. Registration is renewed every five years. If the registration is not renewed, the product is delisted and manufacturer, health facilities and retailers are advised accordingly. Registration checks for expiry are carried out monthly. 24 Rapid Pharmaceutical Management Assessment A.6 Law permitting generic substitution by pharmacists (C) Rationale: A generic product is any drug product unprotected by patent. Any manufacturer is therefore entitled to produce it. In the marketplace, drug products sold under generic name are almost always less expensive than brand name products. Generic substitution is the dispensing of a product, with the same active ingredients(s) in the same dosage form and strength, in place of the drug that is prescribed by brand name. The existence of a law or regulation that allows pharmacists to make generic substitutions is a means of promoting the use of relatively low cost, generically named products. Definition: The written law or regulation must explicitly permit qualified pharmacists, or those practicing in designated settings, to substitute generic drug products for prescribed brand name products, only when the substituted drug is chemically equivalent to the one prescribed. Data Collection: Where to go Who to ask What to get MOH Pharmaceutical Services Document(s) that regulate generic Director/Chief/Secretary prescribing or dispensing, views on level of compliance with regulationBoard of Pharmacy Chairman Pharmacy Owners Association, Executive Director, Secretary Pharmacists Association and/or members Use document review and interviews to determine the existence of a law or written regulation that allows generic drug products to be dispensed in place of brand name products. Note which settings are covered by this law or regulation. This indicator scores only the existence of such law(s) or regulation(s), not whether substitution of generic for proprietary drugs is actually practiced. A copy of such a document should be made available to the assessment team and/or as a reference in the MOH. Computation & Example: Record the existence of a law permitting generic substitution by pharmacists as follows: � Yes public and private, if a written law or regulation permitting generic substitution in both the public and the private sector (pharmacies) exists (record name of law or regulation and year of enactment) � Yes public, if a written law or regulation permitting generic substitution only in public settings exists (record name of law or regulation and year of enactment) � Yes private, if a written law or regulation permitting generic substitution only in the private sector (pharmacies) exists (record name of law or regulation and year of enactment) � No, if such a law or regulation does not exist � Illegal, if generic substitution is explicitly prohibited in both public and private sectors Detailed Descriptions of the Indicators 25 Presentation: In country Y, Congressional Decree #85 was approved in 1991, amending the Public Health Code, and permitting pharmacists practicing in MOH facilities to substitute generic equivalents unless specifically indicated otherwise on the prescription. 26 Rapid Pharmaceutical Management Assessment A.7 Practice of generic substitution (C/R/F) Rationale: While in many countries generic substitution is not explicitly permitted by law, it does often occur. This indicator measures the practical acceptance of this important principle of cost reduction. Definition: This indicator is defined as the prevalence of generic substitution practices, as described by either: (a) reports on this behavior by informed sources, or (b) observation based on simulated purchases. Data Collection: Where to go Who to ask What to get MOH health facility Pharmacy Dispenser or Pharmacist Record of generic substitution behavior, or views on practice of generic substitution; direct observation through simulated purchase 20 drug retail outlets Pharmacy Dispenser or Pharmacist Data to substantiate this indicator may be qualitative, gathered through interviews with dispensers; or quantitative, gathered through simulated purchase surveys. The obvious limitation of interviews is that most developing countries do not have laws explicitly permitting generic substitution, and informants may perceive risk in acknowledging the practice. The result will be under-reporting of this behavior. When interviews are used, it is suggested that questions be posed to both key informants at the central and regional levels, and to dispensers at the health facility or retail pharmacy level. This will provide a cross-check for assessing the probable validity of responses. The most objective way to measure this indicator is to carry out a sample survey using the simulated purchase method. This is described in detail in chapter 4, section E, "Defining Approaches for Collecting Survey Data." To summarize, this requires a data collector, who pretends to be a patient, to visit a sample of dispensing sites, present a prescription for a brand name product, and ask the dispenser if it is possible to substitute a cheaper product for the one prescribed. Willingness to do this may suggest prevalence of generic substitution. In making this determination, however, only those cases wherein the substituted item is chemically equivalent to the one prescribed should be counted. For cases in which a different chemical compound is substituted for the prescribed product, the term which applies is "therapeutic substitution." The simulated purchase method is best suited for retail settings because the dispensers there normally have several options for making substitutions, and this is often not the case in MOH facilities. In practice, therefore, it may be most feasible to gather data in MOH settings through interview, and gather data in retail settings through simulated purchase. See the "Generic Substitution and Sale of Antibiotics Data Form" in Annex C, page 149. Detailed Descriptions of the Indicators 27 Computation & Example: This indicator is recorded as a percentage, calculated as the number of dispensers who either: (a) state that they practice generic substitution; or (b) are observed to practice generic substitution, divided by the total number of dispensers in a sample, and multiplied by 100. � For data collected through interview, the formula is: % of Dispensers Acknowledging = Number of Acknowledging Practice x 100 Generic Substitution Total Number Interviewed % of Dispensers Acknowledging = 8 x 100 = 40% Generic Substitution 20 � For data collected through simulated purchase surveys, the formula is: % of Dispensers Observed Practicing Generic Substitution = Number Making a Generic Substitution x 100 Total Number Purchases % of Dispensers Observed = 12 x 100 = 60% Practicing Generic Substitution 20 Presentation: In country C, 40%, or 8 dispensers interviewed at a sample of 20 MOH clinical facilities, stated that they make generic substitutions when filling prescriptions. At the same time, 60%, or 12 drug sellers from a sample of 20 retail pharmacies, made valid generic substitutions, for brand name products, while 15%, or 3 drug sellers, substituted nonequivalent products. 28 Rapid Pharmaceutical Management Assessment B. Formulary/Essential Drugs List and Drug Information B.1 Number of unique drug products on National Drug Formulary List (C) Rationale: The number of unique drug products on a National Drug Formulary List (NDFL) or National Essential Drugs List (NEDL) is one measure of a country's commitment to rational resource allocation and to containing drug costs by using only essential and cost-effective products in the health care system. Sometimes, however, the term "Essential Drugs List" refers only to a list of products authorized for use in primary health care facilities. It is important, therefore, to understand how the terms related to essential drug lists are used before assuming that they meet the test of being equivalent to an NDFL. Definition: The term NDFL refers to a listing of all the unique drug products approved for medical practice in MOH facilities in a particular country. Sometimes the NDFL will appear in a manual which contains a description for each product on the list. In countries where the MOH uses the term "National Essential Drug List," this is often the equivalent of an NDFL. Data Collection: Where to go Who to ask What to get MOH Director of Pharmaceutical and/or NDFL and/or NEDL Medical Supplies Services Central Medical Stores Officer in charge/Director/Mgr. Determine whether a NDFL/NEDL exists. If so, study investigators must obtain copies to assess the number of drug products it contains. In order to make an accurate assessment, it is necessary to specify criteria for counting products containing the same active ingredient(s). Products that are counted as the same item include: � Brand name products that are chemically equivalent to generic products of the same strength and dose form appearing on the list. For example, Bactrim 400/80 mg tablets and co-trimoxazole 400/80 mg tablets are counted as the same product. � Tablets and capsules of the same product appearing in the same strength. For example, ampicillin 500 mg tablets and ampicillin 500 mg capsules are counted as one product. � Fixed combination drug products, no matter how many chemicals they contain. For example, a combination product, containing ergometrine and caffeine, is counted as one drug product. Detailed Descriptions of the Indicators 29 Products that are counted as different items include: � Different strengths of the same chemical entity. For example, tetracycline 250 mg capsules and tetracycline 500 mg capsules are counted as two products. � Dosage forms for different routes of administration. For example, tablets and capsules (oral), suppositories (rectal), and injectable (IM/IV/SC) should each be counted as different drug products for a particular drug product. � Different dosage forms for the same route of administration, such as tablets and suspensions. For example, ampicillin 500 mg tablets and ampicillin 50 mg/ml suspension are counted as two different drug products. Computation & Example: The indicator is recorded as the total number of unique drug products on the National Drug Formulary List. Record the year of the most recent edition of the published NDFL. If no NDFL exists, this indicator would be recorded as none. Presentation: Country A has a National Drug Formulary List with a total of 230 unique drug products listed. It was revised in 1993. There is also an Essential Drugs List for primary health care facilities with 35 unique drug products listed. 30 Rapid Pharmaceutical Management Assessment B.2 Existence of an official manual, based on the National Drug Formulary List, providing basic drug information to prescribers, revised and published within the last five years (C) Rationale: The existence of a government produced or sanctioned manual with accurate, unbiased, and reasonably current information for prescribers concerning the drugs on the National Drug Formulary List is a measure of the official awareness of the need for drug information to promote rational use of the drugs available. Definition: To qualify as an official manual for purposes of this indicator, a document must be intended as a clinical reference for health care providers and present the following information on drug products: pharmacology, chemical components, accepted indications, contraindications, side effects, and recommended dosages. Any document which provides the information summarized above for the sub-set of drugs authorized for use in a given type of clinical facility meets the definition of manual for this indicator. Such documents might take the form of formulary manuals (FM), which use the drugs on the list as the point of departure, and are organized by therapeutic categories of products. The manual could also be organized as standard treatment guidelines (STG), which use health problems as the point of departure, and discuss the use of NDFL drugs to treat these problems. In many countries, the drugs presented in the NDFL are classified according to level of use, with the greatest number of drugs authorized for use in hospitals with specialist physicians and the least number authorized for use in facilities staffed by paramedical workers. Accordingly, there may be FMs or STGs which cover subsets of the overall NDFL and which are aimed at particular groups of health care providers, such as paramedical staff working in rural health posts. Data Collection: Where to go Who to ask What to get MOH Director of Pharmaceutical and/or Most recent copy of manual Medical Supplies Services, Director of Health Services Such a manual must officially exist for this indicator to be meaningful. If so, obtain the most recent copy of the manual that has been prepared to provide impartial information about the drugs on the National Drug Formulary List. Evaluate whether or not the information in the manual meets all the following criteria, specified in the definition above: � The document is intended as a clinical reference for health care providers � The document presents therapeutic effects, chemical components, indications, contraindications, side effects, and recommended dosages for drug products Detailed Descriptions of the Indicators 31 Computation & Example: Record the existence of an official manual, based on the NDFL, providing information to prescribers, and revised within the last five years, as follows: � Yes, if either a formulary manual, or standard treatment guidelines, which meet the definitions given above, exists and has been revised and published within the last five years. Record the type of manual (FM or STG) and the year in which the manual was published. � No, if: - A manual exists, but has not been revised within the past five years. - A manual is under production and incomplete. - A complete manual exists as an approved draft, but has not been printed or circulated. Presentation: In country Y, a national formulary manual exists; it was revised in June 1992. The manual is intended for use by physicians, nurses and medical and nursing students. It contains information on indications, dosages and contraindications for 38 drugs. It lacks information on side effects or pediatric dosages. 32 Rapid Pharmaceutical Management Assessment B.3 Percentage of MOH health facilities visited with the most current edition of an official manual based on the National Drug Formulary List (F) Rationale: The degree of distribution of a current edition of a manual providing unbiased information on products of the NDFL is a measure of the effort actually being made to effectively promote appropriate use of the drugs on the list. Definition: This indicator measures the presence of the current edition of an official formulary manual in a given sample of health facilities. Data Collection: Where to go Who to ask What to get 20 MOH health facilities Officer in charge Most recent copy of manual A manual, or manuals, covering products on the NDFL/NEDL, must exist in order for this indicator to be meaningful. As noted in the discussion of indicator B.2, such a document may take the form of either formulary manual or standard treatment guidelines. Data for this indicator are collected by survey of a sample of 20 health facilities. At each site, staff are asked to produce a copy of a document appropriate to their type of facility. If a recent manual (revised within the past five years) does not exist (that is, the answer to indicator B.2 is No) then, if possible, record the presence of the most current edition of any manual which meets the definition. See the "Inventory Data Form" in Annex C, page 153. Computation & Example: This indicator is a percentage. It is computed as the number of facilities at which an official manual is found, divided by the total number of facilities in the sample; multiply this quotient by 100, to convert the decimal to a percentage. % of Facilities with = Number of Facilities with Official Manual x 100 Official Manual Number of Facilities in Sample % of Facilities with = 5 x 100 = 25% Official Manual 20 Presentation: An indicator study carried out in country A revealed that in only 25% of health facilities, or five health facilities out of a sample of 20 surveyed, could staff produce a copy of the 1990 edition of the Standard Treatment Guidelines. Detailed Descriptions of the Indicators 33 B.4 Existence of drug information centers that provide unbiased and current information to public health decision makers, health care providers and consumers (C) Rationale: The existence of one or more drug information centers that provide unbiased and current information is a measure of the degree of access that exists to the information required for rational selection, procurement, and use of drugs. Definition: A drug information center should maintain and provide current information for policy makers, prescribers, pharmacists and consumers on such topics as: indications, contraindications, dosage, adverse reactions, costs and sources, storage requirements, drug quality/purity, and actions to take for inadvertent administration and overdoses. The center may or may not be government operated, but to qualify according to this indicator, it should be freely accessible to health care providers of the system being surveyed. In some countries, there are information centers which exist only to provide information on drug availability, that is, where drugs may be purchased, or otherwise obtained. There may also be centers operated by groups or associations which are intended only for use by members. Cases such as these do not meet the definition for this indicator. Data Collection: Where to go Who to ask What to get MOH Drug Regulatory Authority Director/Manager Information on the existence of the MOH Dept. of Pharmacy Services Chief Pharmacist center and a description according to criteria listed below MOH Information, Education, and Health Educator Communication Department Schools of Medicine or Pharmacy Professors Drug information center Director Information on services offered, utilization, resources, location, staffing and budget Interview key informants to identify the drug information centers. Follow up with a brief visit to each site. Collect the following information: � Location of the center and its affiliation (i.e., university, MOH, etc.) � Principal users (i.e. Drug Regulatory Authority drug registration staff, hospital staff, medical students or the general public) � Services offered and frequency of use � Documented demand/quantity of services provided � Types of equipment and information resources available, scope and currency � Numbers and qualifications of staff � Source of funding 34 Rapid Pharmaceutical Management Assessment Computation & Example: This indicator is recorded as follows: � Yes, if one or more drug information centers that meet the definition for this indicator exist; record the number of such centers and describe any existing centers by providing the information stated above � No, if no drug information center exists Presentation: In country Q, there is one drug information center. It is located in the College of Pharmacy, and serves the University, the MOH, and the Board of Pharmacy. The center responds to requests and irregularly publishes a newsletter on new drugs. This center is staffed by one pharmacist, who has no formal training in providing drug information. The center has a typewriter but no computer. The information resources consist of twelve texts; the newest is five years old. Funds for the center are obtained from the MOH, the College of Pharmacy and WHO. Funds for resource acquisition are budgeted, but never spent. No records are kept to document the numbers and types of inquiries received. Detailed Descriptions of the Indicators 35 C. Ministry of Health Budget and Finance C.1 MOH budget or expenditures on pharmaceuticals, US$ per capita (C) Rationale: The MOH's per capita budget or expenditures on drugs is a measure of the adequacy of financing for pharmaceuticals in the public sector. Government financing in many developing countries has decreased in recent years due to inflation and increase in population. Definition: MOH per capita pharmaceutical expenditures are defined as the total amount of money (in US$ at the current rate of exchange) spent on pharmaceuticals by all MOH sources (national, regional and local budgets combined) for the most recent financial year, per individual in the population. MOH budget is defined as the budget for pharmaceutical purchases which was approved, not the requested budget. In many countries, only the approved budget figures will be readily available, but whenever possible, it is preferable to use expenditure data for calculating this indicator. Data Collection: Where to go Who to ask What to get MOH/MOF Budget Director, Health Planner Budget and/or expenditures by budget category for most recent financial year(s), World Bank reports Donors and NGOs Health Financing Specialist, Logistics Specialist Bureau of Statistics Census Director Most recent population census figure, population projections for intercensal period In most countries, the Ministry of Finance (MOF) publishes an annual report on budgets and expenditures. This source of information should be combined with interviews with relevant government officials and examination of historical documents to obtain a complete-as-possible accounting of total budget or expenditures on pharmaceuticals in the previous fiscal year. Valuation of donated products should be included, if their value was accounted for in the forward planning budget. This estimate must include budget or expenditures from all public sources at the central, regional, and local levels. Finally, obtain a reasonably reliable estimate of the current national population from the Bureau of Statistics or some other official source. In collecting this data, keep the following points in mind: � Although the indicator is for the budget or expenditures for the most recent financial year, if possible, the data should be collected for the preceding three to five years. This would give an idea of trends in financing. � In some countries, there are as many as three budget figures: amount requested, amount sent forward, and amount approved. It is crucial to use the amount approved rather than either of the other two amounts. 36 Rapid Pharmaceutical Management Assessment � The value of donated drugs should be included in the indicator only if the value of donations was included as a line item in forward budget planning or actual expenditures. The practice on this point varies from country to country. � If reliable current population estimates are not available locally, there are two options. The first is to obtain the last census figure, note the date, and calculate an estimated current population based on estimated annual population growth rates. The second option is to use the figures from the latest edition of the World Bank World Development Report. Computation & Example: The indicator is computed by dividing the total value of the MOH drug budget or expenditures for the most recent fiscal year (expressed in US$ at the current rate of exchange) by the national population. Be sure to stipulate whether the value represents actual expenditure or budget, and record the fiscal year on which the data are based. MOH Drug Budget = Total of MOH Drug Budget Per Capita National Population MOH Drug Expenditure = Total of MOH Drug Expenditure Per Capita National Population MOH Drug Expenditure = $2,850,000 = $0.77 Per Capita 3,694,000 Presentation: In country A, both budget and actual expenditure data for the most recent financial year were available. These data showed a pharmaceutical budget for fiscal year 1993 of $3,000,000, with actual expenditures being $2,850,000. In the same year, the MOH received $1,050,000 worth of donated drugs, including $500,000 from UNICEF, $500,000 from USAID and $50,000 from WHO. In accordance with local accounting norms, none of the donated drugs figured in budget projections or expenditure estimates. The most current population estimate was from a census taken in 1990, which gave the figure 3,430,000. The MOH Family Planning Division estimates annual population growth at 2.5%. This gives an estimated population of 3,694,000 in 1993. Taking into account the norms listed in the Data Collection section, the correct financial figure to use is $2,850,000 (that is, expenditure is preferred to budget, and donations are excluded), and the MOH drug expenditure per capita was US$ 0.77 in 1993. Detailed Descriptions of the Indicators 37 C.2 Existence of a system for recovering the cost of drugs dispensed in MOH health facilities (C) Rationale: In most countries the funds available through government budgets and donors are not sufficient to meet rising demands for drugs. The existence of a system (or systems) to recover costs of drugs from consumers is a measure of a country's recognition that regular budgetary means require supplementation in order to bridge the gap between supply and demand. Definition: For purposes of this indicator, a cost recovery system is defined as any system which supports drug supply costs by charging patients for all or part of the drugs dispensed to them. Included within this definition are: � Systems that charge patients prices intended to recover 100% of acquisition, storage, distribution and dispensing costs � Systems that charge patients prices intended to recover only part of these cost, for example 50%; such systems are sometimes called cost sharing systems � Systems that charge patients a flat fee, regardless of the types of drugs dispensed or their costs, when all revenues are allocated to buying new drugs; these systems are also sometimes called cost sharing systems Not included in the definition of pharmaceutical cost recovery, for purposes of this indicator, are schemes that charge patients for medical consultations, with no stipulation that the revenues be used for buying new drugs. This exclusion holds even in cases where some of the revenues may be used to purchase drugs. Data Collection: Where to go Who to ask What to get MOH Health Finance Director Document that describes the cost recovery program, its objectives, management and accounting systems, etc. MOF Revenue Collection/ Planning Records of cost recovery revenues Director collected by MOH facilities NGO/Donors Director or Project Director Cost recovery program activities, plan and records Gather information for verifying the existence of cost recovery systems through interviews with key informants. In addition, attempt to gather a range of background information as available: � Coverage of the cost recovery system, expressed as the percentage of all facilities wherein cost recovery takes place � Financial objectives, such as full or partial cost recovery, recovery of replacement cost, etc. � Percentage of costs actually recovered � Types of fee charged and basis for fee scheme 38 Rapid Pharmaceutical Management Assessment � Exemption systems which excuse indigent or chronically ill patients from all or part of customary charges � The year(s) for which all such data collected apply � Management system and accountability for fees collected: records, reports, joint facility-community bank accounts, etc. Computation & Example: Record this indicator as follows: � Yes, if one or more cost recovery systems exits, meeting the definitions listed above � No, if at least one scheme does not exist Presentation: In country A, cost recovery schemes operate in all of the MOH's 36 national and regional hospitals, but not in the 122 health centers. There is no documentation of financial objectives concerning average percentage of costs to be recovered, but it is clear that the objective is partial cost recovery, or cost sharing, and not total cost recovery. Patients are charged flat fees for each different drug product that they receive, but there is a two-tiered fee structure: 10 pesos for ordinary drugs and 20 pesos for expensive drugs. No figures are available for the total amounts of money collected nationally. Data for the Western Region indicate that for fiscal year 1992, total revenues from drug sales at the six hospitals were P 9,000,000. This represents 25% of the cost of the region's hospital drug purchases, and 15% of MOH's total drug purchases for the region. Notes: This indicator has been reduced in scope as a result of field tests of this manual. Originally, the indicator covered such points as average prices charged and revenues collected as a percentage of MOH drug expenditures. In practice, these data were difficult to obtain in most settings without major study efforts, which were not feasible in the context of the overall indicator assessment. Nevertheless, it is strongly felt that attempts should be made to obtain the types of information listed above, as part of filling in the background for this indicator. Remember: This indicator only measures a country's recognition of the problem of inadequate finances for drug supply. By itself, it does not tell how active are that country's attempts at using cost recovery to solve the problem. The more background data that is collected, the more usefully this indicator can be interpreted. Although the indicator does not measure cost recovery systems which are not related to drugs, it is useful to note in the report information on other types of cost sharing/cost recovery programs, including the nature of the charge, amounts charged, scope of the program and results achieved. Detailed Descriptions of the Indicators 39 C.3 Percentage of patients who pay a charge for drugs they receive in MOH health facilities (F) Rationale: There is often substantial variation between stated cost recovery policies and actual practice. Ministries of Health may have stated policies of charging for drugs, but in practice do not enforce these policies in health care facilities. Taking a measure of the percentage of patients who actually make some payment gives an indication of whether cost recovery systems exist on paper only, or whether they operate in practice, and how much is actually recovered. This will determine the success of the cost recovery policy in achieving its objectives. Definition: The definition of patient who pays a charge is any observed patient encounter in which the patient pays a charge (no matter how large or small) specifically related to drugs which are dispensed. Data Collection: Where to go Who to ask What to get 20 MOH health facilities Pharmacist/Dispenser Number of patients that paid a charge out of 30 observations or records Data for this indicator are collected retrospectively, from dispensing or accounting records, or prospectively as follows: In a sample of 20 health facilities, review records or observe drug dispensing for 30 patients at each facility. This gives a sample of 600 contacts. At each site, record the number of patients who pay a charge explicitly for the drugs they receive, and the number that do not. If a fee is paid for medical services, which includes consultation fees as well as drugs, then these data should not be used for this indicator. See the "Charge for Drugs Tally Form" in Annex C, page 157. Computation & Example: The percentage of patients paying a charge is calculated by dividing the total number of patients paying a charge (for pharmaceuticals only), by the total number of patients observed, and multiplying by 100. % of Patients who Pay = Number of Patients Paying a Charge for Drugs x 100 A Charge For Drugs Number of Patients Observed % of Patients Who Pay = 252 x 100 = 42% A Charge for Drugs 600 Record the number of health facilities examined and the time period in which the data were collected. Presentation: In country Q, drug dispensing was observed at a sample of 20 health care facilities. Out of 600 patients observed, 42% paid a charge for the drugs they received during November 1991. 40 Rapid Pharmaceutical Management Assessment C.4 Percentage of total government recurrent budget used for Ministry of Health (C) Rationale: The percentage of total government recurrent budget used for the Ministry of Health is a measure of the relative priority which the government gives to public health. The MOH percentage depends on the government's policies in other sectors, such as education, social services, industry subsidies, defense forces, macroeconomic adjustment policies, debt servicing, etc. The impact of decisions in these other sectors will be reflected in the overall state of health of the population as a consequence of budgetary cuts. Trends in the previous years will help show whether or not the health sector has been favored by governmental policies. Definition: The recurrent budget is the amount of money allocated for routine operating costs. It normally excludes capital costs (and in some cases, excludes donations). This indicator is expressed as the percentage of the total government recurrent budget which is provided as a recurrent budget to the MOH, in the most recent fiscal year for which information is available. Data Collection: Where to go Who to ask What to get MOH Permanent Secretary, Health Budget and expenditure reports for Finance Director, Chief most recent financial year(s), Administrative Officer a copy of the most recent budget/expenditure books MOF Planning Director Normally these data are collected from the most recent report on recurrent expenditures published by the Ministry of Finance. If such a report has not been published, contact senior officials at the MOH and/or the Ministry of Finance to obtain necessary information. If the information is only estimated, stipulate this when recording the data. Be sure to avoid confusion between amounts requested and amounts approved. The amount approved is the relevant number. If information is available for actual expenditures, record information for both allocated budget and expenditures. Attempt to determine whether or not debt servicing is included in the budget figures being used. If so, this should be excluded before making calculations. Computation & Example: The indicator is recorded as a percentage, calculated as total Ministry of Health recurrent budget divided by total government recurrent budget, and multiplied by 100. % of Total Government Recurrent Budget = Ministry of Health Recurrent Budget x 100 for MOH Government Recurrent Budget % of Total Government Recurrent Budget = 8,190,000 x 100 = 6% for MOH 136,500,000 Detailed Descriptions of the Indicators 41 Record the year on which data are based. Presentation: In 1992, country A devoted 6.0% of its total recurrent budget to the Ministry of Health. 42 Rapid Pharmaceutical Management Assessment C.5 Percentage of total MOH recurrent budget allocated to pharmaceuticals (C) Rationale: The percentage of total Ministry of Health recurrent budget used for pharmaceuticals is a measure of the degree of priority given to meeting demand for drugs, and also measures the impact on the system of pharmaceutical expense. Definition: The recurrent MOH budget has been defined in the previous indicator. The amount used for pharmaceuticals is that amount allocated for the purchase of pharmaceuticals in the same budget year. As is the case with the previous indicator, actual expenditures are preferable to budget allocations, if the information is available. It is important that all figures used for deriving the indicator be of the same type, that is, either expenditure or budget. The two should not be mixed within one calculation. Data Collection: Where to go Who to ask What to get MOH Chief Administrative Officer, Pharmaceutical budget allocation Permanent Secretary, Planning and expenditures for most recent Director, Finance Director, financial year(s) Director of Pharmaceutical and/or Medical Supplies Service MOF Planning Director Normally, this data is collected from the most recent report on recurrent expenditures which is published by the Ministry of Finance. If such a report has not been published, contact senior officials at the MOH and/or the Ministry of Finance to obtain necessary information. If the information is only estimated, stipulate this when recording the data. Be sure to avoid confusion between amounts requested and amounts approved. The amount approved is the relevant number. If information is available for actual expenditures, use this information for comparison with allocations. In some countries, it may be difficult to quantify the drug budget or expenditures because expendable medical supplies and drugs are included together in the same figure. In such cases, the total amount should be recorded, and the assessment team should obtain the most reliable estimates available concerning the relationship between drugs and other supplies. The values of donated drugs should be excluded from this indicator, unless they are included as line items in forward budget planning or actual expenditures. Computation & Example: The indicator is recorded as a percentage, calculated as the MOH recurrent budget for pharmaceuticals divided by total Ministry of Health Budget, and multiplied by 100. % of MOH Recurrent Budget Used for = MOH Recurrent Budget Used for Pharmaceuticals x 100 Pharmaceuticals Total MOH Recurrent Budget % of MOH Recurrent Budget Used for = 2,850,000 x 100 = 35% Detailed Descriptions of the Indicators 43 Pharmaceuticals 8,190,000 Record the year on which data are based. Presentation: In the 1993 fiscal year, country A devoted 35% of its Ministry of Health recurrent budget to pharmaceuticals. 44 Rapid Pharmaceutical Management Assessment D. Ministry of Health Pharmaceutical Procurement D.1 Existence of a policy limiting MOH pharmaceutical procurement to drugs on the National Drug Formulary List/National Essential Drugs List (C) Rationale: The existence of a policy limiting MOH pharmaceutical procurement to products on the NDFL/NEDL is a measure of a country's recognition of the need to contain costs by restricting supplies to those items that have been authorized. Definition: A written official policy should explicitly restrict MOH procurement to items included on the government-approved national drug list (NDFL/NEDL). Data Collection: Where to go Who to ask What to get MOH Procurement Office Procurement Officer Copy of the written policy CMS Chief of CMS Obtain a copy of a written MOH procurement policy related to the restricted purchase of products on the National Drug Formulary List. The indicator measures only whether such a policy exists, not whether it is actually followed. Computation & Example: The indicator is recorded as follows: � Yes, if only drugs on the NDFL/NEDL are eligible for MOH procurement � No, if an NDFL/NEDL does not exist, or if procurement is not restricted to drugs on the list Presentation: In country X, an NEDL exists, but the MOH has no requirement that procurements be limited to products on this list. Data show that in 1993 MOH expenditures on drugs from the NEDL accounted for 62% of total drug expenditures, while out-of-formulary drugs accounted for 38%. Detailed Descriptions of the Indicators 45 D.2 Percentage by value of MOH drugs purchased through a central procurement system (C) Rationale: Centralized procurement systems, when operated efficiently, have significant potential for containing drug costs by taking advantage of competitive forces in the market and by achieving economies of scale. The presence of centralized procurement systems is a measure of recognition of this potential. Definition: The "central system" is an officially mandated procurement mechanism via a centralized agency, authority, or ministry office. Data Collection: Where to go Who to ask What to get MOH Procurement Office or other Procurement Officer Procurement reports for most government or parastatal agency Tender Board Secretary recent financial year(s), invoices (Disease Control Programs or Chief Pharmacist from suppliers, tender documents facilities, Regional or Provincial governments) Central Medical Stores Chief of CMS Assess whether a centralized procurement system exists, and assess the approximate percentage by value of MOH drugs procured via this system. Data are obtained from procurement records for the most recent year for which they are available. If data on total procurement are not reliable, or in situations where extensive local facility purchases occur, it may be necessary to rely on estimates gathered through interviews. It should be documented, however, when such a method is used. The following data are needed: � Total value of drugs procured by or for MOH; if more than one agency or institution purchases drugs, provide separate value purchased by each facility or each type of facility � Value of drugs purchased by central procurement system If no central procurement system exists, the indicator should be scored as 0%. Computation & Example: Record the percentage by value of drugs purchased through the central procurement system, computed by dividing the monetary value of drugs centrally procured by the total monetary value of drugs purchased in the MOH, and multiplied by 100. % by Value of MOH = Value of Drugs Centrally Procured x 100 Drugs Purchased Total Value of Drugs Purchased by MOH through Central Procurement System 46 Rapid Pharmaceutical Management Assessment % by Value of MOH = 2,280,000 x 100 = 80% Drugs Purchased 2,850,000 through Central Procurement System Record the year on which data are based. Presentation: In country A, central drug procurement is managed by the Central Medical Stores. National and regional hospitals also have separate budgets for emergency procurements. In 1992, 80% by value of total drug purchases was made centrally by the CMS, and a total of 20% by the individual hospitals. Notes: In some countries each state or region may have a separate central procurement system. In others, health facilities may purchase independently, but there may be a central group purchasing scheme serving one group of facilities. If there are significant variations in procurement practices from region to region, or at different levels of the system, it is important to document these differences, and calculate estimated values for central procurement separately for the regions or facility types. Detailed Descriptions of the Indicators 47 D.3 Percentage of average international price paid for last regular procurement of a set of indicator drugs (C/R/F) Rationale: The cost of drugs procured as a percentage of average international prices is a measure of the efficiency of procurement systems. This indicator will help determine the potential savings that could be achieved if the procurement practices are improved, and in this way support changes in the pharmaceutical supply system. Definition: Average international price is the average free on board (FOB) price from a set of international suppliers. One source of price information is the MSH International Drug Price Indicator Guide. The last regular procurement price refers to the cost, insurance, freight (CIF) price paid during the last regular procurement. Data Collection: Where to go Who to ask What to get MOH-Procurement Unit Officer in charge of pharmaceutical List of most recent prices paid for purchases a set of indicator drugs Central Medical Store Manager or Reception Officer Regional government Manager administration or Medical Store Health facilities Pharmacist or Procurement Officer Tender documents, supplier invoices A set of indicator drugs for which this indicator will be measured must be developed as one of the preparatory activities for the indicator study (see chapter 4, "Guidelines for Study Design," section C, "Specifying a List of Indicator Drugs"). Examples of indicator drug lists that have been used for assessments in different countries are found in Annex B. Information on CIF prices paid by the MOH for the indicator drugs should apply to the last regular procurement. Any more recent ad hoc or emergency procurements that may have taken place should be compared separately to international prices. The average international prices for the indicator drugs may be determined by reference to average international unit prices in the MSH International Drug Price Indicator Guide. The average price in this guide is FOB, and should be6 adjusted upward by 20% to reflect average shipping and insurance costs. Specify the source of international prices and the year of both data sets. If all purchases are not done by one central agency, compile information separately by type of institution, and compute the percentage of international price for each type of purchasing institution (e.g., Regional Medical Stores, hospitals, health centers, etc). Note the date of the most recent regular drug procurement. When making calculations, it may be necessary to convert prices paid in local currencies into U.S. dollars. It is important to use the exchange rates in effect at the time the purchases were made, and to use the edition of the Price Guide which corresponds with the year in which purchases were made. See the "International Price Comparison Form" in Annex C, page 173. 48 Rapid Pharmaceutical Management Assessment Computation & Example: The indicator should be presented as an average of the percentages of international prices for the set of indicator drugs. If data are collected from different levels of the system, a separate average should be calculated for each level. The computation involves two steps: � First, the percentages are calculated for each of the indicator drugs by dividing the purchase cost of the comparison unit (e.g., tablet, milliliter, etc.) at the last regular procurement by the average international price of that unit and multiplying the result by 100. % of Average = Comparison Unit Price x 100 International Price Average International Unit Price � Second, the average percentage for all indicator drugs is calculated by summing their percentages and dividing by the total number on the list. Average % of All = Sum of Percentage of All Indicator Drugs Indicator Drugs Total Number of Indicator Drugs For purposes of illustrating the computation of the result at the CMS, assume an indicator list of three products: Product Comparison Unit Price International Unit Price* Adjusted Average Tetracycline 250 MG CAP 0.0207 0.0163 ORS 200 ML PKT 0.0677 0.0578 Paracetamol SYR 24 MG/ML 0.0070 0.0051 1. The first step is to calculate the percentage for each product. For tetracycline, the first product on the list, this is done as follows: % of Average = 0.0207 x 100 = 127% International Price 0.0163 Using the data in the table, the percentages for ORS and paracetamol are calculated as 117% and 137%, respectively. 2. Next, the average percentage for all three products is calculated as follows: Average % of All = 127 + 117 + 137 = 127% Indicator Drugs 3 * The figures in this column have been adjusted to reflect estimated CIF prices. Detailed Descriptions of the Indicators 49 Presentation: In country C, comparisons of drug purchase prices with average international prices were made at both the Central Medical Store and at a sample of one national and three regional hospitals. In 1992 the CMS paid 127% of the average international price, while the hospitals paid 206%. Notes: Where there is a wide range in the average percentages for a list of indicator drugs, this can be reflected in the report by also calculating the standard deviation from the mean for the list. The standard deviation from the mean is a way of indicating an average amount by which the individual percentages are different than the average percentages. All major computer spreadsheet programs have a function which automatically calculates standard deviation. 50 Rapid Pharmaceutical Management Assessment D.4 Percentage by value of MOH drugs purchased through competitive tender (C/R/F) Rationale: Competitive tenders are among the best ways to minimize the costs of drug purchases. Even when central purchase mechanisms are in place, they may not result in economical purchases unless competitive tendering is used. The percentage of total purchases made using this method is a measure of commitment and ability to manage purchases in a cost efficient way. Definition: To be considered a formal competitive tender process, pharmaceuticals must be purchased on the basis of sealed bids submitted in response to government requests to supply drugs. The tender process may be open, such as international competitive bidding (ICB), or closed, and with or without pre-qualification of suppliers. Data Collection: Where to go Who to ask What to get MOH-Procurement Unit Officer in charge of pharmaceutical Total value of drugs purchased, purchases total value of drugs purchased through competitive tender Central Medical Store Manager or Procurement Officer Health facilities (if procurement is Procurement Officer/Facility decentralized) accountant For this indicator to be relevant, the MOH must use competitive tenders. Collect data on the total value of all drugs purchased during the last fiscal year, and on the value of drugs purchased through formal competitive tenders. Information may be available from a central purchasing agency. As for D.2, if reliable data is not available, estimates may be used, providing that the source and reliability of estimates are documented. Computation & Example: The indicator is a percentage, calculated by dividing the total value of drugs purchased through competitive tender (in the previous fiscal year) by the total value of all drugs purchased that year, and multiplying the result by 100. % of Drugs Purchased = Total Value of Drugs Purchased through Competitive Tender x 100 through Competitive Total Value of All Drugs Purchased Tender % of Drugs Purchased = $1,710,000 x 100 = 60% through Competitive $2,850,000 Tender Presentation: In country A, in the Financial Year 1990/91, 60% of the drugs (by value) procured by the MOH were purchased through competitive tender. Notes: As for D.2, it may be necessary to compile percentages of competitive procurement separately for different states, regions, or type of health facility. Detailed Descriptions of the Indicators 51 E. Ministry of Health Pharmaceutical Logistics E.1 Weighted average percentage of inventory variation for a set of indicator drugs in MOH storage and health facilities (C/R/F) Rationale: Stock record keeping systems that are significantly inaccurate are of limited use for monitoring the status of inventory, estimating future needs, and for controlling leakage and wastage of stock. Weighted average percentage of inventory variation measures the degree to which stock record keeping systems reflect the real status of drugs in stock. Definition: The weighted average percentage of inventory variation is the weighted average percentage difference between recorded stock levels and actual physical counts for a set of indicator drugs. This is also known as average piece variation. As a measure, it indicates the overall correspondence between records and real stock levels. Data Collection: Where to go Who to ask What to get Central Medical Store Inventory Officer/Storekeeper Most accurate records of current stock levels for each indicator drug, issues and receipts not entered; method to record stocks; physical count of unexpired stock levels Regional Medical Store Manager 20 MOH health facilities Dispenser/Pharmacist/Storekeeper This indicator is based on the list of 25 to 50 indicator drugs used to treat common health problems (see chapter 4 "Guidelines for Study Design," section C "Specifying a List of Indicator Drugs," and Annex B, which gives examples of indicator drug lists). Visit the CMS (and at least one regional store if they exist in this system), and a sample of 20 health facilities. At each site, carry out the following procedure: � Ask staff to produce the most accurate records of current stock level for each of the indicator drugs. Ask them to produce their files for any recent (i.e., within the past month) issues or receipt vouchers which have not been entered in their stock level records. If unentered issues and receipts can be adjusted on the spot, this should be done, and the recorded totals should reflect the adjustment. If the records are badly behind (e.g., several weeks or months of issues or receipts have not been entered), use the actual recorded total, and make a note in the report that records are not current. � Take note of the means used to produce these estimates (computerized system, manual ledgers, bin cards). If bin cards exist, and if they were not used to produce the best estimates, obtain a second set of data from the bin cards. If there are both manual and computerized records, obtain data from both systems. 52 Rapid Pharmaceutical Management Assessment � Finally, carry out a physical count of the unexpired stock levels for these drugs, and record the number of units for each indicator drug in stock. The expired units should not be counted. It may be possible to collect three sets of data for a list of indicator drugs at each site in a sample. These sets would include: physical count, stock levels noted in the computerized records or manual ledgers, and stock levels taken by tally from bin cards. This means that this indicator is calculated twice, that is, one calculation based on comparison of the physical count and computer/ledger results, and another based on physical count and the tally or bin card results. If either ledgers or bin cards do not exist, or if one of these systems is in total disarray, it may be possible to make only one comparison. If there is difficulty in collecting data, the comparison between physical count and ledger results should have priority. For the most ambitious assessment, this indicator would be calculated for both computerized stock records/ledgers and bin cards at the CMS, at the sample Regional or District Medical Stores, and at a sample of health facilities. Whether this is possible for all of these sites, or just some of them, will depend on the state of the record keeping systems. It is important, however, to record the following information for all data collected and all indicators computed: the site visited, and which record keeping systems have been assessed, that is, computerized systems, ledgers or bin cards. See the "Inventory Data Form" in Annex C, page 153. Computation & Example: To calculate the weighted average percentage of inventory variation, carry out the following steps: � First, write down the sum of the adjusted totals of all units of indicator drugs recorded in the computer/ledger and/or on the bin cards. � Second, record the total of all units of indicator drugs verified by physical count. � Third, subtract the physical count from the recorded count, and remove any negative signs, thereby creating an absolute value. � Fourth, divide the result by the total of the physical count, and multiply this quotient by 100 to get the weighted average percentage difference. Weighted Average = Recorded Count - Physical Count x 100 % of Inventory Physical Count Variation Detailed Descriptions of the Indicators 53 For purposes of illustrating the computation of this indicator, assume an indicator drug list of three products: Product Record Physical Count Tetracycline 250 MG CAP 1000 900 ORS 200 ML PKT 400 450 Paracetamol SYR 100 ML BOT 350 100 Total 1750 1450 The weighted average percentage of inventory variation is calculated as follows: 1. Add up the total number of units for each product shown in the record: 1000 + 400 + 350 = 1750 2. Add up the total number of units verified by physical count: 900 + 450 + 100 = 1450 3. Subtract the physical count from the recorded amount: 1750 - 1450 = 300 4. Calculate the weighted average percentage of inventory variation: Weighted Average % = 300 x 100 = 21% of Inventory Variation 1450 Presentation: After adjusting for issues and receipts not yet entered in the records, at the Central Medical Store in country B, the average percentage of inventory variation between the computerized record keeping system and the physical count was calculated to be 21%. 54 Rapid Pharmaceutical Management Assessment E.2 Average percentage of individual variation for a set of indicator drugs in MOH storage and health facilities (C/R/F) Rationale: As noted in the previous indicator, stock record keeping systems that are significantly inaccurate are of limited use for monitoring the status of inventory and for controlling leakage and wastage of stock. Average percentage of individual variation measures the degree to which stock record keeping systems reflect the real status of drugs in stock. As a measure, it indicates the magnitude of discrepancy between records and the real stock levels of individual items. Definition: The average percentage of individual variation is the weighted average of the absolute differences between recorded stock levels and physical counts for the same list of indicator drugs. Data Collection: Where to go Who to ask What to get Central Medical Store Inventory Officer/Storekeeper Most accurate records of current stock levels for each indicator drug, issues and receipts not entered, method to record stocks, physical count of unexpired stock levels Regional Medical Store Manager 20 MOH health facilities Dispenser/Pharmacist/Storekeeper This indicator is based on the list of 25 to 50 indicator drugs used to treat common health problems (see chapter 4 "Guidelines for Study Design," section C "Specifying a List of Indicator Drugs," and Annex B, which gives examples of indicator drug lists). It is calculated based on the same data that was collected for the previous indicator, E.1. Use this data and proceed to calculate the weighted average percentage of individual variation. As was the case with indicator E.1, if there are multiple record systems, the indicator should be calculated for all functioning systems. The data should be presented in separate tables for each type of facility in the sample (CMS, RMS and peripheral health facilities). See the "Inventory Data Form" in Annex C, page 153. Computation & Example: To calculate the average percentage of individual variation, carry out the following steps: � For each drug on the indicator list, determine the absolute value of individual variation, as follows: Subtract the physical count from the recorded quantity (ledger or bin cards). Record that result as an absolute value by removing any negative signs. All results should be expressed as positive numbers. Absolute Value of = Recorded Quantity - Physical Count Variation Detailed Descriptions of the Indicators 55 � Calculate the percentage of variation for each indicator drug as follows: Divide the absolute variation by the recorded quantity, and multiply that quotient by 100 to get the percentage of individual variation. % of Individual = Absolute Value of Variation x 100 Variation Recorded Quantity � Determine the average percentage of individual variations by summing all of the percentages of individual variation and dividing by the total number of percentages of variation calculated. Average % of = Sum of Percentages of Individual Variations Individual Total Number of Percentages Calculated Variation Once again, this indicator may be calculated for both computerized stock records/ledgers and bin cards at the CMS, at a sample of Regional or District Medical Stores, and at a sample of health facilities. Whether this is possible for all of these sites, or just some of them, will depend on the state of the record keeping systems. It is important, however, to record the following information for all data collected and all indicators computed: the site visited, and which record keeping systems have been assessed, that is, computerized systems, ledgers or bin cards. Assume the same indicator drug list as for the previous indicator, E.1, for purposes of illustrating the computation of results: Product Record Count Tetracycline 250 MG CAP 1000 900 ORS 200 ML PKT 400 450 Paracetamol SYR 100 ML BOT 350 100 1. To determine the weighted average percentage of individual variation, the first step is to calculate the percentages of individual variation. For tetracycline, the first product on the list, this is done as follows: Absolute Value of Variation: 1000 - 900 = 100 2. Next, percentage of individual variation is calculated: % of Individual = 100 x 100 = 10% Variation 1000 56 Rapid Pharmaceutical Management Assessment The percentage of individual variation for ORS is calculated as: 1. 400 - 450 = -50 = 50 2. 50 x 100 = 12.5% 400 The percentage of individual variation for paracetamol is: 1. 350 - 100 = 250 2. 250 x 100 = 71.4% 350 3. Finally, the average percentage of individual variation for all three products is calculated as follows: Average % of = 10 + 12.5 + 71.4 = 31.3% Individual 3 Variation Presentation: After adjusting for issue tickets not yet entered in the records, at the Central Medical Store in country B, the average percentage of individual variation for the set of indicator drugs was calculated as 31.3%. Detailed Descriptions of the Indicators 57 E.3 Average percentage of stock records that corresponds with physical counts for a set of indicator drugs in MOH storage and health facilities (C/R/F) Rationale: The average percentage of stock records that corresponds with physical counts is useful for clarifying the quality of the stock record keeping system in cases where average variations and variance are skewed by a small number of items. Definition: This is the average percentage of in-stock indicator drug inventory records which corresponds exactly with physical stock count for a set of indicator drugs. Data Collection: Where to go Who to ask What to get Central Medical Store Inventory Officer/Storekeeper Most accurate records of current stock levels for each indicator drug, issues and receipts not entered, method to record stocks, physical count of unexpired stock levels Regional Medical Store Manager 20 MOH health facilities Dispenser/Pharmacist/Storekeeper This indicator is based on the list of 25 to 50 indicator drugs used to treat common health problems (see chapter 4 "Guidelines for Study Design," section C "Specifying a List of Indicator Drugs," and Annex B, which gives examples of indicator drug lists). Visit the CMS (and at least one regional store if they exist in this system), and a sample of 20 health facilities. At each site, carry out the following procedure: � Ask staff to produce the most accurate records of current stock level for each of the indicator drugs. Ask them to produce their records for any recent issues or receipts which have not been entered in their stock level records. � Take note of the means used to produce these estimates (computerized system, manual ledgers, bin cards). If bin cards exist, and if they were not used to produce the best estimates, obtain a second set of data based on bin cards. � Finally, carry out a physical count of the unexpired stock levels for these drugs, and record the number of units for each indicator drug in stock. The expired units should not be counted. Indicator drugs which are not normally stocked by the facility should be excluded. See the "Inventory Data Form" in Annex C, page 153. Computation & Example: � For the set of indicator drugs calculate the percentage of records checked which corresponds exactly with the physical counts according to the tally and the ledger. To do this, divide the number of records for which no discrepancy was found by the total number of records checked, and multiply this result by 100. 58 Rapid Pharmaceutical Management Assessment % of Stock Records Corresponding with Physical Counts = Number of Stock Records with No Discrepancies x 100 Total Number of Records Examined � Present the data in separate tables for each type of facility in the sample (CMS, RMS, or peripheral health facilities). For the sample of health facilities, the indicator is calculated as the average of the facility-specific averages: Average % of Stock Records Corresponding with Physical Counts = Sum of Average % for Each Facility Total Number of Facilities in Sample For purposes of illustrating this computation, assume an indicator drug list of three products: Product Record Count Tetracycline 250 MG CAP 10000 10000 ORS 200 ML PKT 1000 990 Paracetamol SYR 100 ML BOT 88 87 To calculate the percentage of stock records that corresponds exactly with physical counts, carry out the following steps: For one health facility, using the indicator drug list above: 1. The number of records examined = 3 2. The number of records with no discrepancy = 1 % of Stock Records Corresponding = 1 x 100 = 33% with Physical Counts 3 For a sample of 20 health facilities, for which the sum of percentages of stock records that correspond exactly with physical counts is 600%, the average percentage of indicator drugs in stock is calculated as: Average % of Stock Records Corresponding = 600% = 30% with Physical Counts 20 Detailed Descriptions of the Indicators 59 Presentation: After adjusting for issue tickets not yet entered in the records at the Central Medical Store in country Q, the percentage of records for 25 indicator drugs that corresponded exactly with physical counts was 33%. The average percentage of health facility records that corresponded exactly with physical counts was 30%, with the range among facilities from 10% to 60%. 60 Rapid Pharmaceutical Management Assessment E.4 Average percentage of a set of unexpired indicator drugs available in MOH storage and health facilities (C/R/F) Rationale: The availability of indicator drugs is perhaps the single most important indicator of this entire set. This indicator measures a procurement and distribution system's ultimate effectiveness in fulfilling its basic mission, that is, providing drugs at health facilities. Definition: A drug is defined as available if even one unit of unexpired product is in stock. Since expired drugs are inappropriate for use in almost all situations, they are not counted as stock available for use. Data Collection: Where to go Who to ask What to get Central Medical Store Inventory Officer/Storekeeper Inventory records and stock count for indicator drugs Regional Medical Store Manager/Storekeeper 20 MOH health facilities Dispenser/Pharmacist/Storekeeper This indicator is based on the list, developed by study organizers, of indicator drugs used to treat common health problems (see chapter 4 "Guidelines for Study Design," section C "Specifying a List of Indicator Drugs," and Annex B, which gives examples of indicator drug lists). First, in consultation with staff at the CMS, RMS, and local health facilities, determine which of these products are normally stocked at each level. The figure for drugs normally stocked becomes the denominator in calculations. Then, assess whether each of the normally stocked drugs is available. If any of each of the indicator drugs is unexpired and available, record that item as "present" even if it is likely to be out of stock very soon. If all stock for a product on the list is expired, record 0. Do not worry about stock levels for this indicator. See the "Inventory Data Form" in Annex C, page 153. Computation & Example: � The indicator is recorded as a percentage, calculated by dividing the number of specified products found in stock by the total number of drugs for which availability was assessed, and multiplying by 100. % of Indicator Drug = Number of Indicator Drugs with Unexpired Stock x 100 Availability Total Number of Indicator Drugs Normally Stocked � Present the data in separate tables for each type of facility (CMS, RMS and peripheral health facilities) visited. For the sample of health facilities, the indicator is calculated as the average of the facility-specific averages: Average % of Indicator = Sum of Average % for Each Facility Drug Availability Total Number of Facilities in Sample Detailed Descriptions of the Indicators 61 To calculate the average percentage of indicator drug availability for the sample of health facilities, carry out the following steps: 1. For one health facility with 11 unexpired indicator drugs in stock, from a list of 19 indicator drugs normally stocked, the calculation is: % of Indicator Drug = 11 x 100 = 58% Availability 19 2. For a sample of 20 health facilities, for which the sum of percentages of indicator drugs in stock is 960%, the average percentage of indicator drugs in stock is calculated as: Average % of Indicator = 960% = 48% Drug Availability 20 Presentation: In a survey of 20 health facilities, where 19 indicator products were confirmed to be normally stocked, an average of 48% of the listed products was found in stock. The range among facilities was 25% to 85%, with the lower end of the range being associated with more peripheral health facilities. The facility-specific averages are listed below. � Regional medical stores - 85% � District hospitals - 64% � Health centers and posts - 48% 62 Rapid Pharmaceutical Management Assessment E.5 Average percentage of time out of stock for a set of indicator drugs in MOH storage and health facilities (C/R/F) Rationale: The percentage of time out of stock for a set of indicator drugs gives a measure of a procurement and distribution system's capacity to maintain a constant supply of drugs. Definition: Time out of stock, or stockout time, is defined as the number of days that a product was not present in a warehouse or health facility over a recent twelve month period (usually the 12 months preceding the one during which the assessment takes place). To be considered a stockout, there must have been none of an unexpired drug in stock. If even small quantities of an unexpired drug were present, the drug should be counted as in stock. Percentage of time out of stock is defined as the percentage of days during a 12 month period that a drug has been out of stock (based on inventory records). Data Collection: Where to go Who to ask What to get Central Medical Store Inventory Officer/Storekeeper Drugs that are normally stocked from the list of indicator drugs, number of days these normally stocked drugs were out of stock during the 12 months prior to assessment or previous year Regional Medical Store Manager 20 MOH health facilities Dispenser/Pharmacist/Storekeeper This indicator is based on the list, developed by study organizers, of 25 to 50 indicator drugs used to treat common health problems (see chapter 4 "Guidelines for Study Design," section C "Specifying a List of Indicator Drugs," and Annex B, which gives examples of indicator drug lists). In order to determine stockout duration, it is necessary that there be a reasonably accurate inventory recording system (computer/ledger/bin cards) in place. As in the previous indicator, the first step is to consult with staff at each facility and determine which of the products are normally stocked. It is the number of drugs normally stocked that will be used in calculations. To determine average stockout duration, identify which of the normally stocked drugs were out of stock during the last year, and then determine for how many days the product was out of stock during that time. Ideally, this should be determined for the 12 months prior to the month in which the visit occurs. The critical issue is that the same 12-month period should be used for all health facilities and warehouses visited. See the "Stockout Data Form" in Annex C, page 161. Detailed Descriptions of the Indicators 63 Computation & Example: Enter the historical stock data into a table, recording the names of the indicator drugs, and the number of days of stockout in the previous year. To compute this indicator, carry out the following steps: � First, for each indicator drug on the table, record the number of days out of stock for each of the last 12 months. Then sum the total numbers of days out of stock over the past 12 months for all drugs. � Second, to record this indicator, compute the average percentage of time that all indicator drugs were out of stock, within the 12 month period, by adding all the stockout days for all drugs, dividing by 365 times the number of drugs, and multiplying by 100. Average % of Time that Indicator Drugs were Out of Stock = Total Number of Stockout Days for All Indicator Drugs x 100 365 x Total Number of Indicator Drugs Normally Stocked Present this data in tables, and report averages for each type of facility visited (CMS, RMS, and peripheral health facilities). For purposes of illustrating the computation, assume an indicator drug list of three products: Product Total Days Out of Stock Tetracycline 250 MG CAP 36 ORS 200 ML PKT 64 Paracetamol SYR 100 ML BOT 123 Assume that in a CMS, all three of these indicator drugs are normally stocked. Average % of Time that Indicator Drugs were Out of Stock = 36 + 64 + 123 x 100 = 20% 365 x 3 Presentation: In country C, over a 12-month period, the indicator drugs were out of stock an average of 20% of the time at the Central Medical Stores. In the Regional Medical Store, the indicator drugs were out of stock an average of 30% of the time. In the sample of health clinics, the indicator drugs were out of stock an average of 40% of the time. 64 Rapid Pharmaceutical Management Assessment F. Patient Access and Drug Utilization F.1 Population per functional MOH health facility that dispenses drugs (C) Rationale: The ratio of population to functional health facilities that dispense drugs provides a rough measure of access to and availability of pharmaceutical services. The information provided by this indicator will allow health planners and evaluators to assess and monitor strategies to improve distribution and access to pharmaceuticals. Definition: Any fixed, functional health facility that dispenses drugs to inpatients or outpatients should be included in the total count. This might include hospitals, health centers and rural health posts. It should not include vaccination posts which do not normally dispense drugs. Data Collection: Where to go Who to ask What to get MOH Chief Pharmacist, Director of Most recent list of facilities Health Services Bureau of Statistics Census Director Most recent population census figure, population projections for intercensal period Obtain lists of all functional MOH health facilities that meet the above definition of facilities that dispense drugs. If facilities that are managed completely, or partly, by NGOs, are considered part of the overall MOH system, they should be counted as well. The population estimate should be as current as possible. If an estimate for the current year is not available locally, then the options are the same as those explained in the Data Collection section of indicator C.1. That is, either use the estimate provided by the most recent edition of the World Bank's World Development Report, or build an estimate using a recent census figure and applying the cumulative annual rate of population growth to bring it up to the current year. Computation & Example: This indicator is a ratio, calculated by dividing the total national population by the number of functional MOH health facilities which dispense drugs. Ratio = National Population Number of Functional MOH Health Facilities which Dispense Drugs Ratio = 13,000,000 = 30,660 424 Presentation: In country C, the total number of MOH facilities is reported as 424. This includes national, regional and district hospitals, health posts, and sub health posts, all of which stock and dispense drugs. With a population of 13,000,000 this gives an average of 30,600 persons per health facility that dispenses drugs. Detailed Descriptions of the Indicators 65 F.2 Population per licensed pharmacist or pharmacy technician in the public sector (C) Rationale: The unavailability of technically skilled human resources can be an important constraint in developing countries. This indicator provides a rough measure of access to skilled pharmacy personnel. Health care systems with a high ratio of population per pharmacist and trained pharmacy technician should include in their human resource management plan the recruitment, training and development of this important resource in order to bring about improvements in service delivery. Definition: For purposes of this indicator, pharmacist is defined as a person holding a university degree in pharmacy, and pharmacy technician is defined as a person who has completed formal course work leading to a certificate or diploma in pharmacy technology. Only these personnel who work full or part-time in the health care system which is surveyed should be counted. Data Collection: Where to go Who to ask What to get MOH Chief Pharmacist, Director of Latest count or registry of Personnel or Human Resources pharmacists and pharmacy technicians in the health care systemDrug Regulatory Authority Officer in charge of professional /Professional Associations and technical registrations School of Pharmacy Faculty members Ministry of Planning/Personnel Human Resource Planner Management Use interviews with key informants at any of the sites listed above to obtain lists, tables or other documentation. Sometimes this type of information is available in gazettes or other compendia. If possible, obtain breakdowns of numbers of pharmacists and pharmacy technicians working in the public, NGO and private sectors. It is important that the estimates be as current as possible. Cite the names and dates of sources. Computation & Example: This indicator is a ratio obtained by dividing the total national population by the combined number of licensed pharmacists and pharmacy technicians who were full or part-time employees in the public sector health system. Ratio = National Population Pharmacists + Pharmacy Technicians Ratio = 13,000,000 = 146,067 21 + 68 66 Rapid Pharmaceutical Management Assessment Presentation: In country C, information on the numbers of licensed pharmacists and pharmacy technicians was obtained from the Human Resources Division of the Ministry of Planning. Breakdowns for both the public and private sector are given in the table below for the year 1993. Sector Pharmacists Pharmacy Technicians Public (full or part-time) Ministry of Health 14 46 Social Security 7 22 Agency Subtotal 21 68 Private (only) Manufacturers 19 32 Distributors 3 0 Retail 2 41 Subtotal 24 73 Totals: 45 141 There were 21 pharmacists and 68 pharmacy technicians who worked full or part-time for the MOH in 1993. The estimated population for that year was 13,000,000, which gives a ratio of 146,067 people per licensed pharmacist or pharmacy technician in the MOH health system. Detailed Descriptions of the Indicators 67 F.3 Population per authorized prescriber in the public sector (C) Rationale: Adequate numbers of technically qualified staff who are authorized to prescribe medicines are essential to a sound health care system. This indicator provides a rough measure of access to health care providers in the public sector who are authorized to prescribe drugs. Definition: For purposes of this indicator, an authorized prescriber is a practitioner with formal training in primary and/or curative health care, who is legally authorized to prescribe drugs. Physicians, registered nurses, auxiliary nurses and nurse mid-wives are examples of categories of professionals who are authorized to prescribe drugs in many countries. Exclude traditional healers and community based volunteers, even if they have had training and prescribe selected drugs. Data Collection: Where to go Who to ask What to get MOH Director of Personnel or Human Latest count or registry of Resources physicians, professional nurses and auxiliary nurses and other paramedical staff (community health workers, health inspectors, health extension officers, etc.) who are authorized to prescribe drugs Drug Regulatory Authority Officer in charge of professional /Professional Associations and technical registrations Ministry of Planning/Personnel Human Resource Planner Management Use interviews with key informants at any of the sites listed to obtain lists, tables or other documentation. Sometimes this type of information is available in gazettes or other compendia. The indicator is for the health care system being surveyed. If possible, obtain breakdowns for numbers of authorized prescribers working in the public, NGO and private sectors. Keep in mind that one difficulty in interpreting such breakdowns is that, in many countries, practitioners may work in both the public and private sectors. It is important that the estimates be as current as possible. Cite names and dates of sources. Computation & Example: This indicator is a ratio obtained by dividing the total national population by the combined number of authorized prescribers. Ratio = National Population Combined Total of Authorized Prescribers in the Health Care System In country C, information on the numbers of authorized prescribers was obtained by first consulting the National Health Code. According to that document, there are three legally established categories of health care providers: physicians, registered nurses and auxiliary nurses. Only the physicians may prescribe the full range of ethical and over-the-counter products. Prescribing practices of nurses and auxiliary nurses are covered by an amendment to the Code, which authorizes them to prescribe selected drugs on the National Drug Formulary List when working in MOH facilities, or under 68 Rapid Pharmaceutical Management Assessment the direct supervision of a private physician. It was only possible to obtain numbers of physicians practicing in both the public and private sectors. Information on the two categories of nurses was available only for the public sector, though there are thought to be relatively few of them working in the private sector. Sector Physicians Nurses Nurses Auxiliary Public (full or part-time) Ministry of Health 209 42 612 Social Security Agency 62 15 65 University Hospital 42 20 32 Subtotal 313 77 709 Private (only) 74 N/A N/A Totals: 387 77 709 Ratio = 13,000,000 = 11,829 313 + 77 + 709 Presentation: In country C, the population for 1993 was estimated at 13,000,000. Based on the information available, this gives a ratio of 11,829 people per authorized prescriber in the MOH system. Detailed Descriptions of the Indicators 69 F.4 Average number of drugs prescribed per curative outpatient encounter in MOH health facilities (F) Rationale: This indicator attempts to describe prescriber behavior, since too high (or too low) an average number prescribed can indicate poor prescribing practices. These practices can be explained by lack of pharmaceutical information and education, or by chronic lack of supplies that force health workers to prescribe what is available. Additional information is necessary to explain the cause of the prescribing practices observed. Definition: This indicator measures the average number of drugs prescribed per outpatient per curative encounter. Each drug written separately is counted as a separate drug prescribed. Encounters include only visits by patients seeking curative care. Data Collection: Where to go Who to ask What to get 20 MOH health facilities Medical Records Officer/Health Determine the number of drugs Facility Manager/Pharmacist prescribed for a sample of 30 patients per facility, by consulting daily registers, patient records, prescription slips, or through observation Select a sample of 30 outpatient encounters at each health facility, and count the number of drugs prescribed to each patient. These data may be gathered either retrospectively from medical records, or prospectively from observation. (See description of sampling methods in chapter 4, section E, "Defining Approaches for Collecting Survey Data.") Include only patients seeking curative care. To count drugs in a uniform way in some settings, guidelines must be established for data collectors on how to count ambiguous prescribing practices. Examples of these include locally compounded multi-drug combination therapies, or certain combined oral/injection therapies, such as a standard treatment of injection followed by oral administration. The methods for collecting these indicators are also described in the manual How to Investigate Drug Use in Health Facilities, published by WHO. 7 See the "Drug Use Data Form" in Annex C, page 165. Computation & Example: For each health facility in the sample, the indicator is recorded as an average, calculated by dividing the number of different drug products prescribed by the total number of curative outpatient encounters surveyed. Remember to count any curative encounters in which no drug was prescribed. The overall indicator is an average of these facility-specific averages. Along with this overall average, provide the range of highest and lowest facility averages. If different levels of the system are visited (i.e., hospitals, health centers and health posts), tabulate results separately and summarize with averages for each type of facility. Express the results as the number of drugs prescribed per encounter. 70 Rapid Pharmaceutical Management Assessment Average Number of Drugs Prescribed Per Curative = Total # of Drugs Prescribed in All Encounters Outpatient Encounter Total # of Curative Encounters Studied in All Facilities � For example, assume that the data collected for 30 patient encounters at one health facility showed that 51 drugs had been prescribed. Results for that one facility are calculated as follows: Average Number of Drugs Prescribed Per Curative = 51 = 1.7 per encounter Outpatient Encounter 30 � For a sample of 20 health centers surveyed, a total of 1800 drugs were prescribed for 600 patient encounters. The average number of drugs per encounter, for health facilities, is calculated as follows: Average Number of Drugs Prescribed Per Curative = 1800 = 3 drugs per encounter Outpatient Encounter 600 Presentation: An indicator survey covering a sample of 20 health centers in country A found that patients received an average of 3 prescribed drugs per curative encounter. The range among health facilities was 0.9 to 3.8 drugs per facility. Detailed Descriptions of the Indicators 71 F.5 Percentage of drugs prescribed by generic name in MOH health facilities (F) Rationale: If health care providers prescribe by generic names instead of brand names, it is easier to control drug costs in the health care system, because generic substitution is not needed. Definition: This indicator measures the percentage of drugs which are prescribed using their internationally recognized generic names, as identified in the WHO list of international nonproprietary names (INN). The availability of generically named8 drugs in the market, and the information available to prescribers, will influence the pattern observed. Although the INN is used as the "official" generic name for this indicator, in some countries, generic drugs may be available under INN and other variations such as USAN or BAN. For example, the antihistamine chlorphenamine (INN) may be marked as chlorpheniramine (USAN). Or, only the USAN or BAN generic product may be marketed. In such situations, any of the common generic names may be counted for purposes of this indicator. Data Collection: Where to go Who to ask What to get 20 MOH health facilities Medical Records Officer/Health Determine the number of generic Facility Manager drugs and total of all drugs prescribed for a sample of 30 patients per facility, by consulting daily registers, patient records, prescription slips, or through observation Study organizers must develop a list of (or an explicit way of defining) the specific product names to be included as generic drugs; usually the generic names of drugs are identified on the National Drug Formulary List. Select a sample of 30 patient encounters at each health facility. These data may be gathered either retrospectively, from medical records, or prospectively, from observation of patient encounters. Note the way each drug prescribed in the sample is recorded. (See chapter 4, "Guidelines for Study Design," section E, "Defining Approaches for Collecting Survey Data"). Include only patients seeking curative care. Data collectors must be able to observe the actual names used to describe the drugs prescribed, as opposed to having access only to the names of the products dispensed. Thus, the dispensing ledger (if there is one) may not be an accurate source for this indicator. See the "Drug Use Data Form" in Annex C, page 165. 72 Rapid Pharmaceutical Management Assessment Computation & Example: For each facility in the sample, the indicator is recorded as a percentage, computed by dividing the number of drugs prescribed by generic name by the total number of drugs prescribed, and multiplying this quotient by 100. The overall indicator is an average of these facility-specific percentages. Along with this average, provide range figures. % of Drugs = Total Number of Drugs Prescribed by Generic Name x 100 Prescribed by Total Number of All Drugs Prescribed Generic Name If different levels of the system are included, summarize the results in separate tables for each type of health facility. � For example, results for one health facility are calculated as follows: % of Drugs = 41 x 100 = 80% Prescribed by 51 Generic Name � A total of 900 drugs were prescribed at 15 health posts; 324 were prescribed by generic name. The average for this sample would be calculated as follows: % of Drugs = 324 x 100 = 36% Prescribed by 900 Generic Name for All Facilities Presentation: In 15 health centers in country A, an average of 36% of drugs was prescribed by generic name, while in 5 hospitals in the same country, 43% of drugs were prescribed using their generic names. The rate among facilities varied from 15% to 80%. Detailed Descriptions of the Indicators 73 F.6 Percentage of drugs prescribed from the National Drug Formulary List in MOH health facilities (F) Rationale: In well-managed pharmaceutical systems, health care providers are trained to confine prescribing to those drugs on the NDFL/NEDL which are authorized for the type of facility in which they work; compliance with the NDFL/NEDL is an essential part of cost control. This indicator provides a measure of the extent to which this objective is achieved. Definition: This indicator measures the percentage of drugs prescribed, by either brand or generic name, which are listed in the current NDFL/NEDL. Data Collection: Where to go Who to ask What to get 20 MOH health facilities Medical Records Officer/Health Determine the number of drugs Facility Manager/Pharmacist prescribed that are in the NDFL/NEDL, and total number of drugs prescribed in a sample of 30 patients per facility, by consulting daily registers, patient records, prescription slips, or through observation Before the study, organizers should obtain a copy of the NDFL/NEDL, and if possible, assemble an index of brand and generic names for products on the list. Select a sample of 30 patient encounters from each health facility. These data may be gathered either retrospectively, from medical records, or prospectively, from observation of patient encounters. Record the names of the drugs prescribed exactly the way they are written. (See chapter 4, "Guidelines for Study Design," section E, "Defining Approaches for Collecting Survey Data"). Include only patients seeking curative care. Data collectors must be able to observe the actual names used to describe the drugs prescribed, as opposed to having access only to the names of products dispensed. For drugs written by brand name it may be necessary to determine the generic equivalent, before determining whether or not the product is contained in the NDFL/NEDL. This may be done by consulting relevant indexes. The services of a local pharmacist will also be useful here. See the "Drug Use Data Form" in Annex C, page 165. Computation & Example: For each health facility in the sample, the indicator is recorded as a percentage, computed by dividing the number of drugs prescribed which are contained in the NDFL/NEDL by the total number of drugs prescribed, and multiplying by 100. The overall indicator is an average of these facility-specific percentages. Along with this average, provide range figures. 74 Rapid Pharmaceutical Management Assessment % of Drugs Prescribed = Total Number of Drugs Prescribed from NDFL/NEDL From the NDFL/NEDL Total Number of Drugs Prescribed If different levels of the system are included, summarize the results in separate tables for each type of health facility. � For example, results from one health facility are calculated as follows: % of Drugs Prescribed = 36 x 100 = 80% From the NDFL/NEDL 45 � If, for 20 health facilities, data for a sample of 600 patient encounters showed that a total of 2,280 drugs were prescribed, and that 1664 of these were prescribed by generic name, then the average for all facilities would be calculated as follows: % of Drugs Prescribed = 1,664 x 100 = 73% From the NDFL/NEDL 2,280 for All Facilities Presentation: An indicator survey covering a sample of twenty health facilities in country A found that 73% of all drugs prescribed were contained in the NEDL. The range was from 62% to 94%. Detailed Descriptions of the Indicators 75 F.7 Percentage of outpatients prescribed injections at MOH health facilities (F) F.8 Percentage of outpatients prescribed antibiotics at MOH health facilities (F) Rationale: The increasing incidence of AIDS and hepatitis B provides a good reason to assess the extent of injectable therapies and promote their rational use. Injections are essential for some purposes, but overuse, which is common, wastes scarce resources and exposes patients to risks for adverse reactions and disease. Both of these outcomes are less likely with oral therapies. Antibiotics and injections are costly therapies and are frequently overused. Antibiotic resistance to common infections has rendered some formerly useful drugs ineffective. This is partly caused by indiscriminate, empirical and uninformed prescribing practices and other forms of overuse. This is especially serious when national capacity for laboratory monitoring of antimicrobial sensitivity is limited or nonexistent. Definition: Injectable drugs are those given intravenously or intramuscularly. For purposes of this indicator, count only injectable drugs prescribed for curative encounters. Do not count EPI vaccinations. All antibacterials, penicillins, anti-infective dermatologicals, ophthalmic anti-infectives, and anti-diarrheal drugs containing antibiotics should be included as antibiotics. Metronidazole and anti-tuberculosis drugs (except streptomycin) would not be considered antibiotics for this indicator. The indicator measures the percentage of outpatient curative encounters for which these therapies are prescribed. The indicator result would be misleading if inpatient and outpatient encounters were mixed. Data Collection: Where to go Who to ask What to get 20 MOH health facilities Medical Records Officer/Health Determine the number of Facility Manager/Pharmacist outpatients who were prescribed an antibiotic, and number who were prescribed an injection for a sample of 30 patients by consulting daily registers, patient records, prescription slips, or through observation Before the study, organizers should develop a list of which medications are to be counted as antibiotics to be used as a reference by data collectors. Select a sample of 30 patient encounters from each facility. These data may be gathered either retrospectively, from medical records, or prospectively, from observation. Count the number of injections and/or antibiotics prescribed for each patient. (See description of sampling methods in chapter 4, section E, "Defining Approaches for Collecting Survey Data.") Include only outpatients seeking curative care. Count separately the number of patients who are prescribed one or more antibiotics, or one or more injections. If a patient receives two or more antibiotics, this counts as one instance for this purpose. If a patient receives two or more injections, this counts as one instance for the indicator. 76 Rapid Pharmaceutical Management Assessment It is extremely important to avoid, or at least be aware of, factors in the data collecting environment which would bias results. For example, if the patient encounter sample covered one or more days at facilities when most patients who attended were diabetics coming for insulin, the number of injections observed in a sample would be artificially high. Similar problems would occur if the sample was weighted with days when a psychiatric clinic was held and a high proportion of patients came for maintenance antipsychotic injections. See the "Drug Use Data Form" in Annex C, page 165. Computation & Example: For each facility in a sample, both indicators are recorded as percentages, computed by dividing the number of patient encounters during which an antibiotic is prescribed/an injection is given, by the total number of patient encounters surveyed, and multiplying by 100. The overall indicators are the averages of these facility-specific percentages. Along with this average, provide range figures. % of Patients = Total Number of Patients Prescribed Injections x 100 Prescribed Injections Total Number of Patient Encounters Surveyed % of Patients = Total Number of Patients Prescribed Antibiotics x 100 Prescribed Antibiotics Total Number of Patient Encounters Surveyed � For example, results from one health facility are calculated as follows: % of Patients = 4 x 100 = 13% Prescribed Injections 30 % of Patients = 7 x 100 = 23% Prescribed Antibiotics 30 � If, for 20 health facilities surveyed, data for a sample of 600 patient encounters showed that a total 120 patients received injections, then the average for all facilities would be calculated as follows: % of Patients = 120 x 100 = 20% Prescribed Injections 600 for All Facilities � For the same 20 health facilities, the same data also showed that 102 patients received antibiotics. Therefore, the average for all facilities would be calculated as follows: % of Patients = 102 x 100 = 17% Prescribed Antibiotics 600 for All Facilities Detailed Descriptions of the Indicators 77 Presentation: In a survey of 20 health facilities in country Z, an injection was prescribed during 20% of all outpatient encounters, with a range of 0% to 55% among facilities. For the same sample, 17% of encounters resulted in prescription of antibiotics, with a range of 0% to 57% among facilities. 78 Rapid Pharmaceutical Management Assessment F.9 Percentage of prescribed drugs presented for dispensing that are actually dispensed in MOH health facilities (F) Rationale: This indicator measures the ability of a sample of health facilities to meet the pharmaceutical needs of their users. It is also a measure of the availability of drugs in the health facilities. Definition: Drugs that are actually dispensed are defined as drugs that are prescribed and dispensed from the health facility. In this indicator, it is based only on the prescriptions presented for dispensing. Date Collection: Where to go Who to ask What to get 20 MOH health facilities Medical Records Officer/Health Number of drugs dispensed and Facility Manager total number of drugs that were prescribed in a sample of 30 dispensing encounters at each facility At each of the 20 health facilities, examine 30 dispensing encounters in a given month. Prior to the start of the assessment, all the data collectors should agree on which month or set time period to study these 30 encounters at each health facility. If records document which drugs were not dispensed, this indicator can be collected retrospectively. Otherwise, observe 30 dispensing encounters. (See chapter 4, section E, "Defining Approaches for Collecting Survey Data.") Record the number of prescribed drugs presented for dispensing, and then record the number of drugs actually dispensed from these 30 encounters. Each prescribed drug should be counted separately. If any portion of the prescribed amount is dispensed, count the drug as dispensed. See the Drug Use Data Form" in Annex C, page 165. Computation & Example: For each health facility in the sample, indicators are recorded as percentages, computed by dividing the number of drugs actually dispensed by the total number of prescribed drugs which were presented for dispensing, and multiplying this quotient by 100. The overall indicator is an average of these facility-specific percentages. Along with this average, provide the range figures. % of Prescribed Drugs that are Dispensed = Number of Drugs Actually Dispensed x 100 Number of Prescribed Drugs Presented for Dispensing Detailed Descriptions of the Indicators 79 � The results for one health facility are calculated as follows: % of Prescribed Drugs that are Dispensed = 35 x 100 = 78% 45 � If, for 20 health facilities surveyed, data for a sample of 600 patient encounters showed that 438 prescribed drugs were actually dispensed, then the average for all facilities would be calculated as follows: Average % of Prescribed Drugs = 438 x 100 = 73% that are Dispensed for All Facilities 600 Presentation: In country C, for a sample of 20 health facilities, an average of 73% of prescribed drugs presented for dispensing were actually dispensed, with a range from 50% to 96% among facilities. 80 Rapid Pharmaceutical Management Assessment G. Product Quality Assurance G.1 MOH drug product quality laboratory tests during the past year: (a) number of drug products tested and (b) total number of drug product quality tests performed (C) Rationale: This indicator measures how actively the Ministry of Health pursues drug product quality assurance (QA) testing. Typically, Ministries of Health have laboratory analyses conducted on drug samples on such occasions as: receipt of bids from potential suppliers, arrival of new stock at the central warehouse, receipt of complaints concerning specific products, or random sampling in retail pharmacies. Definition: This indicator measures the number of separate drug products which were submitted by MOH facilities for testing within a recent 12 month period, and the total number of tests for quality which were performed. Data Collection: Where to go Who to ask What to get Drug Regulatory Authority or Officer in charge of quality control Information on the existence of Quality Control Testing and drug testing laboratory services drug product testing program, its Laboratory description and performance Use interviews with key informants to determine and record: � Whether or not the MOH has an active drug product testing program � Names and affiliations of the laboratories which actually perform the tests � Occasions on which testing is carried out � Number of drug products submitted for testing, from where and why � Number of drug products actually tested this past year � Number of tests carried out in the past year, with results of tests � The year these tests were performed � Information at the drug testing laboratory concerning staff level and capacity, equipment, record systems, and access to reagents, reference standards and technical information Computation & Example: This indicator is expressed as: � Number of drug products tested by the MOH for drug quality. � Total number of tests carried out by the MOH for drug quality. For example, drugs X and Y were submitted to the MOH for testing. Four separate tests were done on X and only two on Y. Therefore, two drug products were tested and a total of six tests were performed. In addition, present any information which permits breaking down the total number of tests by such criteria as: number initially presented for testing, reasons for testing, numbers of positive and negative results, and locations of testing. Detailed Descriptions of the Indicators 81 Presentation: Six drugs purchased by the MOH in country Q were submitted for testing to the Regional Drug Testing Laboratory in 1993. At the time of the survey, results had been obtained for three of these items. One (ampicillin) was found to be substandard. A total of eight tests were performed on these three drugs. 82 Rapid Pharmaceutical Management Assessment G.2 Use of WHO Certification Scheme (C) Rationale: Developing countries with inadequate pharmaceutical testing facilities may be unable to carry out quality control analyses. The WHO Certification Scheme on the Quality of Pharmaceutical Products Moving in International Commerce is one mechanism to help assure the quality of pharmaceutical products manufactured or purchased in a country. Full participation of the exporting country in the Scheme provides assurance9 that the drug has been authorized to be sold in the exporting country, that the manufacturing laboratory has been inspected and complied with good manufacturing practices, and facilitates the exchange of drug information between governments. Definition: This indicator measures the degree to which the WHO Certification Scheme on the Quality of Pharmaceutical Products Moving in International Commerce is used by importing countries for drug registration and for public sector procurement, and by exporting countries to certify quality of exported products. Data Collection: Where to go Who to ask What to get MOH Director of Pharmaceutical Evidence of membership to and Services use of WHO Scheme to assure quality of pharmaceuticals in registration, procurement, and/or certification for exports Central Procurement Agency Director Drug Regulatory Authority Officers in charge of registration and of certifying good manufacturing practices There are four ways for a country to participate in the WHO Certification Scheme. A country may: 1. Issue and use certification for drug registration 2. Issue certification for exports 3. Use certification issued by other countries for drug registration 4. Uses certification by other countries for public sector procurement Visit whatever sites are required for determining whether the country participates in each of these components. Also determine if the country is using the current version of the Scheme, and obtain data on the specific number of instances of use for purposes of procurement, registration and certification of exports. Verbal assurances of participation in and active use of the Certification Scheme do not necessarily mean that the Scheme is being used. The study team should ask for documents which verify use. Detailed Descriptions of the Indicators 83 Computation & Example: Record participation in the Certification Scheme as follows: � Total, if the country is a member and uses the Scheme and for all of the registration, export, and procurement functions � Partial, if the country is a member and actively uses the Scheme for at least one of the registration, export, and procurement functions � Limited, if the country is an official member of the Scheme, but there is limited or no evidence that it is active for specific purposes of registration, exports, or procurement � None, if the country does not participate in the Scheme Presentation: Country A is a partial participant in the WHO Certification Scheme of Pharmaceutical Products Moving in International Commerce. Review of relevant records showed that in 1993, there were six import certifications for drugs purchased by the MOH Central Medical Store. There is no evidence that the Scheme is used for product registration or export certification. 84 Rapid Pharmaceutical Management Assessment G.3 Existence of formal systems for reporting: (a) product quality complaints, and (b) adverse drug reactions (ADRs) (C) Rationale: Product quality assurance is most readily assured by a combination of purchases from reliable suppliers and functioning mechanisms through which health care providers report potentially defective products and potential adverse reactions. Effective follow- up on such reports is also critical. This indicator measures a government's ability to monitor drug product quality problems, as well as drug-induced disease. Definition: Formal mechanisms designed for reporting product quality complaints and adverse drug reactions in patients are programs which have been enacted by legislation, written regulation, or through official notices from the responsible government agency. To be considered functioning, the offices responsible for administering these systems must be able to produce standard forms used for reporting quality complaints or ADRs. In addition, there must be evidence that follow-up investigations occur when reports are submitted. Data Collection: Where to go Who to ask What to get Drug Regulatory Authority Quality Control Officer Format for reporting drug quality complaints and ADRs, number of complaints and ADRs reported during previous year, action taken and sample of reporting forms MOH Chief Pharmacist Adverse Drug Reaction ADR monitoring officer Monitoring Center or Unit Determine, through interviews and review of documents on hand at the office(s) responsible for pharmaceutical quality control and adverse drug reactions, how reports are intended to be processed. Obtain copies of reporting forms. Determine how many product quality complaints or ADR reports have been received during the last year. If it is verified that reports have been received, ask for evidence of any follow-up that took place. Computation & Example: Each part of this indicator should be scored according to the following criteria: � Functional, if a formal reporting system is mandated, there are standard reporting forms, there is evidence that reports are submitted, and there is evidence that follow-up investigations take place � Limited, if a reporting system is mandated, but no standard reporting forms exist, or even if standard reporting forms do exist, reports are rarely or never submitted, and there is no evidence of follow-up investigations � Nonexistent, if there is no reporting system in place Detailed Descriptions of the Indicators 85 Presentation: In country Y, the Drug Regulatory Authority is mandated by the Health Code to monitor and take action on product quality complaints. The status of this system is at best limited, because no standard reporting forms exist, and only one complaint was received during the 12 months preceding the assessment. There was no evidence of follow-up action. For adverse drug reactions the situation is similar, with even less evidence of activity. There is no legal mandate for reporting ADRs, but staff at the Drug Regulatory Authority do acknowledge responsibility in this area. There is no standard reporting form, nor is there evidence that reports have ever been submitted. Therefore, the most appropriate rating for an ADR reporting system appears to be nonexistent at the national level. There are, however, two hospitals in the capital city that do have their own ADR reporting systems, both using regular reporting forms: the University Teaching Hospital and the Adventist Hospital. At the University Hospital, an average of two ADRs were reported monthly over the twelve months preceding the assessment, and at the Adventist Hospital a monthly average of one ADR was reported over the same period. 86 Rapid Pharmaceutical Management Assessment H. Private Sector Pharmaceutical Activity H.1 Population per licensed private sector drug retail outlet (C) Rationale: The private pharmaceutical sector is the primary source of medicines consumed in most countries. The extent and impact of this sector on the health status of the population is usually not well documented. This indicator provides one measure of the coverage of the private sector drug system. Definition: A private sector drug retail outlet is any private drug outlet, pharmacy, or formally organized store that is authorized by the Drug Regulatory Authority to sell ethical and/or over-the-counter pharmaceutical products to the public. Data Collection: Where to go Who to ask What to get MOH Chief Pharmacist List of all licensed private sector drug retail outlets Pharmacy Board Members Drug Regulatory Authority Director Bureau of Statistics Census Director Most recent population census figure/population projections for intercensal period Obtain a list or a count of all licensed private sector drug retail outlets. Obtain official information about the most recent estimate of national population. If the most recent local estimates are not current, extrapolate from the last census figure, the year it was done, and the population growth rate per year (or use the most recent edition of the World Development Report). Computation & Example: This indicator is a ratio, obtained by dividing the national population by the number of licensed private sector drug retail outlets. Express results in terms of persons per drug retail outlet. Population per Licensed = National Population Private Sector Drug Number of Licensed Private Sector Drug Retail Outlets Retail Outlet Population per Licensed = 10,000,000 = 25,000 Private Sector Drug 400 Retail Outlet Presentation: In country X, a total of 400 licensed private sector drug retail outlets were in operation during the year 1993. With a total population of 10,000,000 persons that year, this yields an average of 25,000 persons per licensed private sector drug retail outlet. Detailed Descriptions of the Indicators 87 H.2 Number of licensed or registered drug retail outlets per government drug inspector (C) Rationale: Governments need to ensure that the private pharmaceutical sector is an effective partner in sustaining and improving the health status of the population. Active enforcement of the legislation and regulations require adequate numbers of trained inspectors. The number of licensed/registered drug retail outlets per government drug inspector provides an estimate of the government's capacity to monitor the formal drug retail market. Definition: Only pharmacies and other formally organized stores that are licensed to sell drugs to the public should be counted. Government drug inspectors are those whose primary or secondary responsibility is inspecting drug manufacturers, drug importers, drug warehouses, private pharmacies, and other drug retail outlets. This indicator, however, concerns only drug retail outlets. Data Collection: Where to go Who to ask What to get MOH Director of Pharmaceutical List of all licensed private sector Services drug retail outlets, and total number of staff that are officially counted as drug inspectorsPharmacy Board Members Drug Regulatory Authority Director Obtain the most recent information regarding the number of licensed and/or registered drug retail outlets in a given year. Define the categories of government staff that are officially counted as drug inspectors, and obtain an estimate of the number of such personnel that are currently employed by the government in that given year. Computation & Example: This indicator is calculated by dividing the number of licensed drug retail outlets by the number of actual drug inspectors. Express the results in terms of outlets per inspector. Number of Licensed Drug Retail = Number of Licensed Drug Retail Outlets Outlets per Drug Inspector Number of Drug Inspectors Number of Licensed Drug Retail = 900 = 75 Outlets per Drug Inspector 12 Presentation: In country X, the Department of Drug Administration has 12 staff members whose full or part time responsibilities include inspecting drug retail outlets of all categories. There are 900 licensed pharmacies. This yields an average of 75 licensed drug retail outlets per inspector. 88 Rapid Pharmaceutical Management Assessment H.3 Percentages of drug manufacturers, distributors, and drug retail outlets inspected during a one-year period (C) Rationale: The degree of coverage for inspections of drug manufacturers, distributors, and drug retail outlets provides a rough measure of the enforcement of drug legislation and regulations. Definition: For this indicator, drug manufacturers are those companies that are licensed to manufacture raw materials, formulate from imported or manufactured raw materials, repackage finished dosage forms and strengths, or any combination thereof. Distributors include companies that distribute pharmaceuticals, as well as other products. See H.1 for a definition of drug retail outlets. Data Collection: Where to go Who to ask What to get MOH Director of Pharmaceutical List of all manufacturers, Services distributors and licensed private sector drug retail outlets, and total number of sites inspected in each category in 12 months prior to the assessment or during the last year Pharmacy Board Members Drug Regulatory Authority Director Interview key informants and review records to determine: � Numbers of manufacturers, distributors and retailers � Numbers of sites in each category that received one or more inspection visits during either the 12-month period preceding the assessment, or the most recent one-year period for which data are available Computation & Example: For each category, calculate the percentage of total sites receiving one or more inspection visits in the past year, as follows: % of Sites = Number of Sites Inspected x 100 Inspected Total Number of Sites Detailed Descriptions of the Indicators 89 The Office of Inspection provided information summarized in the table below concerning sites inspected during 1993. Number Percent Type of Site Number Inspected Inspected Manufacturer 6 4 67% Distributor 14 6 43% Drug Retail Outlet 900 84 9% The example below illustrates the calculation for the drug retail outlets: % of Sites = 84 x 100 = 9% Inspected 900 Presentation: During a one year period in country X, the percentage of sites inspected, by category, was as follows: 9% of drug retail outlets, 43% of distributors, and 67% of manufacturers. 90 Rapid Pharmaceutical Management Assessment H.4 Total value of total private sector retail pharmaceutical sales, US$ per capita (C) Rationale: The value per capita of private sector retail pharmaceutical sales is one measure of the value of national resources allocated to pharmaceuticals. To allow cross-national comparisons, the value is expressed in US$ per capita. Definition: Private sector retail pharmaceutical sales are those sales that are made to the public through private drug retail outlets. It excludes sales from distributors to public sector hospitals and outpatient health facilities. Data Collection: Where to go Who to ask What to get Pharmaceutical Manufacturers President and/or members Total value of retail sales in US$ Association for most recent calendar or fiscal year Retail Trade Association President and/or members Customs or Internal Revenue Registry Director Agency Ministry of Finance or Trade Planning Director Bureau of Statistics Census Director Most recent population estimate, international data sources such as IMS International Data After ascertaining what information sources are available, study organizers must determine the most reliable method of obtaining total private sector retail pharmaceutical sales in a recent fiscal or calendar year. Often, the most reliable source for such information is the local association which represents pharmaceutical manufacturers. Sometimes information will be available from the Ministry of Health or the Ministry of Trade (or its equivalent). Obtain a reasonably accurate official estimate of the national population (as described in other indicators) during this fiscal year. Be sure not to double count wholesale and retail sales. If sales data were provided in local currency, and there have been significant changes in the conversion rates, select and report on a method for averaging the exchange rate. Obtain data for the same time period used for MOH pharmaceutical expenditures. Computation & Example: The indicator is calculated as the total value of private sector retail pharmaceutical sales, in U.S. dollars, divided by the national population. Private Sector Pharmaceutical Sales = Total Value of Private Sector Retail Drug Sales Per Capita National Population Private Sector Pharmaceutical Sales = $58,800,000 = $4.20 Per Capita 14,000,000 Detailed Descriptions of the Indicators 91 Presentation: In country B, the value of the private pharmaceutical sector sales was approximately $4.20 worth of drugs per capita during 1993. 92 Rapid Pharmaceutical Management Assessment H.5 Combined value of public sector pharmaceutical expenditures and private sector retail sales, US$ per capita (C) Rationale: The combined value of public sector drug purchases and private sector retail pharmaceutical sales is one measure of the value of total national resources allocated to pharmaceuticals. This indicator complements the previous one. Viewed together, they provide a basis for comparing the relative weights of the MOH and private sector in the market. Definition: Total public sector per capita purchases are defined as the total amount of money (in US$ at the current rate of exchange) spent on purchasing pharmaceuticals by all public sector systems (national, regional and local budgets combined) for the most recent financial year, per individual in the population. The total private sector market is estimated by the value of total pharmaceutical retail (rather than wholesale) sales as described in the previous indicator. The periods (and exchange rates) used for private and public markets should be the same. This value should not include donations for (or from) NGOs and/or PVOs. However, major NGO health systems which purchase drugs in the local market for distribution through their own health systems should be included, if possible. Data Collection: Where to go Who to ask What to get MOH and other major public Finance or Budget Director Total public sector purchases and sector health systems total value of retail sales in US$ for most recent calendar or fiscal yearPharmaceutical Manufacturers President and/or members Association/Retail Trade Association College of Pharmacy President and/or members Customs or Internal Revenue Registry Director Agency Ministry of Finance or Trade Planning Director International data sources, such as IMS International Data Bureau of Statistics Census Director Most recent population estimate Two major components of this indicator have been collected previously in two other indicators (C.1 and H.4). In order to be able to combine them for this indicator, however, the MOH expenditures and private sector sales figures must be calculated for the same defined period of time. If there are other major public sector health systems, such as Social Security, these estimated or actual purchases must be added to the MOH figure for the same period of time. Detailed Descriptions of the Indicators 93 Computation & Example: For the same time period, adjust the public and private sector components, add together, then divide by the estimated national population. Combined Public Drug = Public Sector Purchases + Private Sector Sales Purchases and Private National Population Sector Sales Per Capita OR add the values of the indicators C.1 and H.4, assuming the data was collected over the same period of time for both, and there are no other major public sector health systems. Combined Public Drug Purchases and Private Sector Sales Per Capita = $18,000,000 (MOH) + $8,000,000 (Social Security) + $58,800,000(Private) = $6.06 14,000,000 Presentation: In country B, in the previous calendar year, the total value of the drug market was US$ 6.06 per capita ($1.86 by the public sector and $4.20 in the private sector). 94 Rapid Pharmaceutical Management Assessment H.6 Percentage of products on National Drug Formulary List that are currently manufactured or co-manufactured within the country (C) Rationale: The percentage of products on the NDFL/NEDL which are currently manufactured or co-manufactured within the country measures a country's self sufficiency for supplying the most essential pharmaceutical products. Definition: For the indicator to be relevant, there must be a National Drug Formulary List in existence, as well as a local drug industry. Drugs from this list are counted as manufactured in-country if they are manufactured either from raw materials or intermediate chemical constituents; drugs which are simply repackaged or relabelled are not counted. Data Collection: Where to go Who to ask What to get MOH Pharmaceutical Services Director Total number of drugs in NDFL/NEDL Local Pharmaceutical President/Members List of drugs manufactured from Manufacturers Association raw materials or intermediate chemical constituents Determine the number of pharmaceutical products listed on the National Drug Formulary List. For each of these products, determine if one or more local manufacturers currently manufactures the product in-country and markets it in either generic or proprietary form. If a product is manufactured by one or more manufacturers, it is recorded as being produced in the country. Computation & Example: This indicator is recorded as a percentage, calculated by dividing the number of drugs produced in-country, in a given year, on the National Drug Formulary List by the total number of unique, generic drug products on the list, and multiplying by 100. % NDFL Drugs Manufactured In-country = Number of Drugs on NDFL that are Manufactured In-country x 100 Number of Drugs on NDFL % NDFL Drugs = 120 x 100 = 42% Manufactured 284 In-country Presentation: In 1992, 42% of the drugs on the National Drug Formulary List of country D were manufactured in-country by one or more of the six local producers. Detailed Descriptions of the Indicators 95 H.7 Average of median private sector drug retail prices as a percentage of MOH acquisition prices for a set of indicator drugs (C) Rationale: The average percentage of median indicator drug prices in private sector drug retail outlets provides a measure of the cost-effectiveness of operating in-house retail pharmacy services in MOH health facilities. Cost-effectiveness increases to the degree that retail prices exceed MOH acquisition prices. Definition: The average of median private sector retail prices is the average of the median retail prices for a list of 25 to 50 tracer drugs, based on data collected at a sample of 20 drug retail outlets. These data can be collected as part of the sample survey covering both MOH health facilities and drug retail outlets that is required for covering a number of other indicators. (See chapter 4, "Guidelines for Study Design.") The average MOH acquisition prices are the average of CIF prices paid for the same indicator drugs in the most recent MOH procurement. Data Collection: Where to go Who to ask What to get MOH Procurement Unit or CMS Officer in charge Most recent regular CIF price paid for list of tracer drugs Sample of drug retail outlets Pharmacy owner /Dispenser Retail price for list of tracer drugs /Pharmacist To collect prices from retail outlets, begin with the list of tracer drugs. Visit the sample of pharmacies or other drug retail outlets, and at each site obtain the current selling price for each of the indicator drugs. If an item is not stocked, skip that drug and go to the next. If more than one brand is stocked, use the price of the least expensive product. Select the median price for each indicator drug from the aggregate of prices collected from all sites visited. [The median (or middle-most), is used instead of the mean (or average) retail price to avoid bias caused by outlying high or low prices for a given drug product.] To determine the median retail price for each product, examine the complete list of 20 prices obtained from all sites, arranged in ascending order, and pick the middle two numbers (10 and 11). Add these two numbers and divide by two to obtain the median. If the list contains an odd number of items, simply select the middle-most number as the median. See the following examples: Ex. 2,3,4,5,6 Median is 4 Ex. 2,3,4,5,6,7 Median: 4 + 5 = 9 ÷ 2 = 4.5 The MOH acquisition price is the CIF price paid for the indicator drugs for the most recent regular (non-emergency) procurement. This has already been discussed in indicator D.3. See the "Price Comparison Data Form" in Annex C, page 169. 96 Rapid Pharmaceutical Management Assessment Computation & Example: Using the median retail prices, this indicator is calculated as follows: � For each indicator drug, divide the median retail unit price by the MOH acquisition unit price, and multiply by 100. % of MOH Acquisition Price = Median Retail Unit Price x 100 MOH Acquisition Unit Price � For the entire list of indicator drugs, add up the results of the above calculation done for each product, then divide by the total number of indicator drugs. Average % of MOH Acquisition Price = Sum of % of MOH Acquisition Prices for all Indicator Drugs Total Number of Indicator Drugs For example, for purposes of illustrating this result, assume an indicator drug list of three products. Median Retail MOH Acquisition Product Unit Price Unit Price Tetracycline 250 MG CAP 0.112 per CAP 0.014 per CAP ORS 200 ML PKT 0.36 per PKT 0.04 per PKT Paracetamol SYR 100 ML BOT 4.30 per ML 0.43 per ML 1. To arrive at the average percentage of MOH acquisition price, the first step is to calculate the percentage of MOH acquisition price for each product. For tetracycline, the first product on the list, this is done as follows: % of MOH Acquisition Price = 0.112 x 100 = 800% 0.014 2. Using the data provided in the table, the percentages of MOH acquisition prices for ORS and paracetamol syrup are calculated as 900% and 1000%, respectively. 3. Finally, the average percentage of MOH acquisition price for all three products is calculated as follows: Average % of MOH Acquisition Price = 800 + 900 + 1000 = 900% 3 Presentation: In country Q, retail prices of 25 indicator drugs were found to be, on average, 900% of MOH acquisition prices, based on retail data collected in July 1993 and MOH acquisition prices paid in March 1993. Detailed Descriptions of the Indicators 97 H.8 Existence of price controls for drugs in the private sector (C) Rationale: Governments often attempt to influence the level of profit the private sector can obtain from drug sales. This indicator will record whether there are policies and regulations controlling the prices of drugs in the private sector. Definition: Price controls are regulations which govern markups, margins, sales prices or profits for drugs at the manufacturer, importer, wholesale distributor and/or retail sales level. Data Collection: Where to go Who to ask What to get MOH Director of Pharmaceutical Copy of legislation or regulations Services controlling prices of drugs in the private sector, list of prices, description of measures to monitor prices charged, sanctions to violators, and number of sites where price checks were carried out in the last 12 months MOF Price Control Authority Manufacturers, importers, Company Managers wholesalers Retail outlets Owners Professional and industry President associations Interviews and document review are used to determine the presence of price controls at manufacturer, wholesale and retail levels. The type and extent of controls at each level should be obtained. If feasible, price lists from manufacturers and wholesale and retail outlets should be used in combination with interviews, to determine whether controls are enforced. During interviews, request information about programs in place to monitor compliance with price controls, and sanctions which are applied to violators. If possible, obtain quantitative information as to t

View the publication