WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations

Publication date: 2007

WHO/TCM/2007.2 WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations Guide for coordinators and data collectors December 2007 It is important to know… ƒ if people have access to essential medicines; ƒ if they are getting medicines that are safe, effective and of good quality; and ƒ if these medicines are being properly used. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors WHO/TCM/2007.2 © World Health Organization 2007 All rights reserved. Publications of the World Health Organization can be obtained from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel: +41 22 791 3264; fax: +41 22 791 4857; email: bookorders@who.int). Requests for permission to reproduce or translate WHO publications – whether for sale or for noncommercial distribution – should be addressed to WHO Press, at the above address (fax: +41 22 791 4806; email: permissions@who.int). The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. The World Health Organization does not warrant that the information contained in this publication is complete and correct and shall not be liable for any damages incurred as a result of its use. - iii - Contents Abbreviations.vii Preface . ix Acknowledgements . xi Getting started .1 Pharmaceutical indicators for monitoring and assessment.5 The need for pharmaceutical sector indicators .5 Who can use the results.5 Indicator-based monitoring strategies .7 Level I core indicators.9 Pharmaceutical components in Level I indicators.9 Level II facility-based core indicators .11 Survey design and data collection (Level II - facility survey).13 Selecting geographical areas.13 Selecting public health facilities .14 Making random selections.15 Selecting private drug outlets.16 Selecting warehouses.16 Sampling patients for data collection .16 Retrospective sampling.16 Prospective sampling .18 Summary of Level II indicators measured by facility .19 Preparing the survey (Level II - facility survey).21 Coordination and survey coordinator.21 Tailoring the survey forms to country situations by choosing key medicines, selecting tracer conditions and identifying treatment protocols.22 Selecting basket of key medicines.23 Identifying medicines to be considered as injections and antibiotics.24 Identifying medicines of choice to treat pneumonia in adults and children.24 Identify standard criteria for adequate labelling and patient knowledge.25 Identifying unit price of medicines to treat outpatient adults and children .25 Instructions for obtaining global and regional drug prices [paid by the patient or paid by the facility] (Survey Forms 2 and 11).25 - iv - Training data collectors.27 Selecting data collectors .27 Training components.27 Training tools needed .28 Training slides introduced.28 The field test.29 Reviewing the field test and calculations .30 The survey .31 Notifying facilities to be visited .31 Travel schedule and logistics.31 Data collectors.32 Supervision of data collection .33 Data processing, analysis and reporting: what to do with the results .35 Computation of Level II facility indicators.35 Quality of data and information .35 Analysis and interpretation of Level I and Level II facility indicators .36 Limitations of the Level II facility survey.38 Indicator measure for Level II facility indicators .38 Performance standards for Level II facility indicators .39 Written report .39 Presentation and discussion of results and related issues.43 Format of result presentation .44 Annexes .47 Annex 1: Level I questionnaire .49 Annex 2: Level II indicators .71 Annex 2a: Technical descriptions of Level II indicators.71 Annex 2b: Issues, technical details, survey design and sampling for Level II indicators.85 Annex 3: Coordinator checklist .89 Annex 4: Example of budget allocation.95 Annex 5: Guide on how to use the training slides (Level II indicators) .97 Annex 6: Training slides for data collection .103 Annex 7: Level II Survey Forms 1—17 .121 - v - Annexes in the CD Annex 1: Level I questionnaire Annex 3: Coordinator checklist Annex 4: Example of budget allocation Annex 6a: Slides for training data collectors Annex 6b: Slides for training survey coordinators Annex 6c: Slides for policy-makers Annex 7: Level II Survey Forms 1-17 Annex 7a: Regional List for prices of drugs (supplementary to SF2 & SF 11) Annex 8: Summary Forms Annex 9: Sample report Annex 10: Letter to facilities List of boxes Box 1 Summary list of indicators and corresponding survey form used to collect the data . 12 Box 2 Summary of survey areas and sites . 14 Box 3 Selecting facilities from 5 geographical areas . 15 Box 4 Randomly selecting public health facilities from the capital city. 15 Box 5 Simple method for selecting 30 patients from the general outpatient list . 17 Box 6 Retrospective sampling: Selecting 30 patients from general outpatient records covering 12 months (365 days). 17 Box 7 Items on Survey Forms to predefine at national level . 22 Box 8 Sample training schedule. 27 Box 9 Sample letter to facilities . 31 Box 10 Checklist for data collectors. 32 Box 11 Error in sampling . 86 Box 12 Overview of the core indicator pyramid (Slide 7) . 106 List of figures Figure 1 Levels of core indicators . 1 Figure 2 Use of median and percentiles to describe the dispersion of measurement. 38 Figure 3 % of patients receiving antibiotics (distribution of facility results) . 39 Figure 4 Presenting national level data graphically. 45 - vi - List of tables Table 1 Study facilities, indicators and forms . 19 Table 2 Model list of key medicines. 24 Table 3 Suggested report outline. 41 Abbreviations - vii - Abbreviations AFRO* WHO Regional Office for Africa AMRO* WHO Regional Office for the Americas ARI Acute respiratory infection DTC Drug and therapeutics committee EC European Community EDM WHO Essential Drugs and Medicines Policy Department EML Essential medicines list EMRO* WHO Regional Office for the Eastern Mediterranean EURO* WHO Regional Office for Europe FIP International Pharmaceutical Federation HAI Health Action International HIV/AIDS Human immunodeficiency virus/Acquired immunodeficiency syndrome INN International Nonproprietary Name INRUD International Network for the Rational Use of Drugs MOH Ministry of Health MSH Management Sciences for Health NAF/ENSP/Fiocruz Center for Pharmaceutical Policies / Sérgio Arouca National School of Public Health / Oswaldo Cruz Foundation (Núcleo de Assistência Farmacêutica/ Escola Nacional de Saúde Pública Sérgio Arouca/ Fundação Oswaldo Cruz) NGO Nongovernmental organization NMP National medicines policy ORS Oral rehydration salts OTC Over-the-counter RDU Rational drug use SEARO* WHO Regional Office for South-East Asia SF Survey forms STG Standard treatment guidelines TM/CAM Traditional medicine and complementary and alternative medicines TRIPS Trade-Related Aspects of Intellectual Property Rights UNICEF United Nations Children's Fund WHO World Health Organization WPRO* WHO Regional Office for the Western Pacific WTO World Trade Organization WHO regional offices were used to group countries for purposes of regional data summary and analysis. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - viii - Preface - ix - Preface The complexity of the pharmaceutical sector, with multiple and cross-cutting factors that can influence access to and rational use of quality medicines, makes it is extremely important to have a systematic method for assessing the pharmaceutical situation at country, regional and global levels. Pharmaceutical sector assessment, monitoring and evaluation aim to answer the following vital questions: do people have access to essential medicines? Are people getting medicines that are safe, effective and of good quality? Are these medicines being used properly? Currently we have insufficient information to answer such questions, but as part of its commitment to assisting countries to improve access to and use of quality medicines, the World Health Organization (WHO) has been developing tools to monitor and evaluate pharmaceutical situations. Assessment, monitoring and evaluation underpin evidence-based policy development and strategic planning, and therefore every aspect of WHO’s activities in pharmaceuticals in general and essential medicines in particular. These processes can be complex for a number of reasons: it is difficult to establish a sustainable system of regular assessment, monitoring and evaluation; resources are not consistently allocated to these tasks and there is limited advocacy for them. Furthermore, many efforts to develop tools have been exhaustive, but impractical and in the past, most tools included indicators that were difficult to collect, especially on a regular basis. A number of indicator-based tools now exist. Indicators for Monitoring National Drug Policies,1 a WHO manual, includes approximately 120 indicators covering structure, process and outcomes of various national medicines policy (NMP) components. Another set of indicators, developed by Management Sciences for Health (MSH),2 focuses on rapid assessment of strengths and weaknesses in the pharmaceutical sector. Another WHO manual, How to Investigate Drug Use in Health Facilities,3 has been used extensively in many countries. The WHO/HAI Medicine Pricing Project,4 is also developing indicators to assess medicines regulations and access to pharmaceuticals. Development of the set of indicators referred to in this document started in 1999 when the first Level I global survey was done. Another survey was undertaken in 2003 and the intention is to repeat surveys every four years. The Level I core indicators consist of a questionnaire used to rapidly assess existing structures and processes in a national pharmaceutical system. Along with this, a set of core indicators - Level II facility indicators - was developed, which require systematic surveys and data gathering at health care facilities. The development process has benefited from countries doing baseline assessments, and feedback from trainers and experts who have used the survey forms. In 2002, the WHO 1 Brudon P, Rainhorn JD, Reich M. Indicators for monitoring national drug policies. Geneva, World Health Organization, 1994. 2 Rapid pharmaceutical assessment - An indicator-based approach. Washington DC, Management Sciences for Health, 1995. 3 How to investigate drug use in health facilities. Geneva, World Health Organization, 1993. WHO/DAP/93.1. 4 Medicine prices: a new approach to measurement. Geneva, World Health Organization and Health Action International, 2003. WHO/EDM/PAR/2003.2. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - x - Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations evolved, together with the Level I indicators. In February 2003, the first group of countries to have used the Level II facility survey tool, and the technical experts and consultants involved in the development process met in Geneva. They discussed the results of country surveys and further improved the tool based on experiences from the field. By June 2003 a working draft was developed. In July 2006, a meeting of experts discussed the status of WHO indicator-based pharmaceutical assessment and monitoring approaches. Several recommendations for future improvements of the Level I and Level II facility survey tools were made. These recommendations are included in the report, WHO Expert Meeting on Pharmaceutical Indicators, Monitoring and Assessment 10-12 July 2006, Geneva Switzerland. The WHO Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations is intended as a useful tool for researchers, policy-makers, planners and others who need to use standardized measurement tools to gather data and other information. The tools presented here have already been used for several years at global and country levels. In addition, the operational package can be used by international agencies and donors, by professional groups and nongovernmental organizations (NGOs). Countries can use the Level I questionnaire as a checklist to illustrate sectoral structures, strategies and approaches. Countries can also use selected forms from the Level II facility survey in their routine monitoring. The results can help them to focus their strategies, advocacy plans and information campaigns. Acknowledgements - xi - Acknowledgements Development of the tools included in this document has been ongoing for several years. The operational package has been used as a working draft since 2002 under the former WHO Department of Essential Drugs and Other Medicines (EDM). The current document is an initiative of the Department of Technical Cooperation for Essential Drugs and Traditional Medicine, with support from the Department of Medicines Policy and Standards (PSM). Precious Matsoso, Germán Velásquez and Gilles Forte provided the leadership and support culminating in finalization of this document. Jonathan Quick guided the initial development and evolution of the WHO medicines indicator process and the assessment of pharmaceutical situations in countries. Hans Hogerzeil has also supported the development of the operational package. Dennis Ross-Degnan has provided technical advice on the document from its inception to completion as well as assisted in developing some chapters. The development of the tools presented in this document benefited from the contributions and expertise of various individuals and institutions: In February 2003, representatives from the first group of countries who used the Level II facility survey tool and the technical experts and consultants involved in the development process met in Geneva: Timotheo Badoy (Philippines); Majid Cheraghali (the Islamic Republic of Iran); Norma Duarte (Guatemala); Ilko Getov (Bulgaria); Jean- Marc Guimier (MSH); Alec Irwin (Millennium Development Project); Martin Auton, Simona Chorliet and Indro Mattei (consultants); Dennis Ross-Degnan (Harvard University); and from WHO, Ogori Taylor (WHO Nigeria), Bakuti Shengelia, Asghari Gholamreza, Guitelle Baghdadi, Diane Whitney, Eshetu Wondemagegnehu, Helen Tata, Edelisa Carandang, Gilles Forte, Germán Velásquez and Jonathan Quick. In July 2006, another meeting of experts was held in Geneva. The recommendations and proceedings of the meeting are recorded in the report, WHO Expert Meeting on Pharmaceutical Indicators, Monitoring and Assessment. Participants were: John Chalker (INRUD-MSH); Dennis Ross-Degnan (Harvard Ubniversity); Simona Chorliet (Mali); Trudie Gerrits (University of Amsterdam); Vera Lucia Luiza (Brazil NAF/ENSP/Fiocruz); Helene Moller (UNICEF); Patrick Mubangizi (HAI-Africa); Javier Penalvier Herrero (Spain); Ray Skinner (Solomon Islands); Klara Tisocki (University of Kuwait); Mary Justin (United Republic of Tanzania); Marsha Macatti- Yamba (United Republic of Tanzania); Albert Wertheimer, (International Pharmaceutical Federation); Zafar Mirza (WHO, Regional Office for the Eastern Mediterranean - EMRO); Nelly Marin (WHO Regional Office for the Americas - AMRO); Ogori Taylor (WHO, Nigeria); Bekele Tefera Jembere (WHO, Ethiopia). The following are from WHO headquarters, Geneva - Precious Matsoso, Gilles Forte, Clive Ondari, Valerio Reggi, Richard Laing, Marthe Everard, Sue Hill, Kathy Holloway, Sophie Logez, Jörg Hetzke, Edelisa Carandang, Helen Tata, Martin Auton, Alain Prat, Shanti Noriega-Minichiello, David Wood and Paul Verboom. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - xii - The development of this document, specifically the Level II Facility survey tool, would not have been possible without the contributions of researchers, data collectors and survey coordinators from nearly 50 countries who have undertaken the Level II facility survey. The WHO Regional Pharmaceutical Advisers /Acting Advisers: Jean-Marie Trapsida (WHO Regional Office for Africa - AFRO); Jorge Bermudez, Nelly Marin, Rosario D’Alessio; Christophe Rerat (WHO AMRO); Zafar Mirza, Mohammed Bin Shahna (WHO EMRO); Kees de Joncheere (WHO Regional Office for Europe - EURO); Krisantha Weerasuriya (WHO Regional Office for South-East Asia - SEARO); and Budiono Santoso (WHO Regional Office for the Western Pacific - WPRO) have organized and facilitated country surveys with WHO country offices and National Programme Officers working on essential medicines issues. Several people have contributed at different stages of development and finalization of the main document, the survey forms - Indro Mattei, Simona Chorliet, Martin Auton, Andy Gray, Kathy Holloway, Marthe Everard, Sue Hill, Christophe Rerat, Klara Tisocki, Anita Wagner, Catherine Vialle-Valentin and Richard Laing. John Snow Inc. provided comments in the early stage of development. The facility survey tool has also benefited from feedback in the Drug Policy Issues courses held in Mumbai and Beirut. Diane Whitney assisted in the development of the June 2003 working draft. Lalit Dwivedi assisted in reformatting the working draft. The group at NAF/ENSP/Fiocruz, Brazil, led by Vera Lucia Luiza, Thiago Azeredo, Isabel Cristina Martins Emmerick and Maria Auxiliadora Oliveira assisted in the completion and finalization of the main document, and survey forms. They also helped in completing the summary forms and various sets of training slides. Claude Da Re did the layout and supervised publication. Edelisa D. Carandang coordinated and supervised the development and implementation of the Level I and Level II indicator tools, and the development of this document. The European Community (EC) supported the printing of this publication as well as some country and regional training and country surveys. The Department for International Development (DFID), UK, also provided financial support for some country training and surveys. The content of this publication does not necessarily reflect the official opinion of the EC and DFID. Getting started - 1 - Level I Core structure & process indicators Level II Core outcome/impact indicators Access to essential medicines Rational use of medicines Level III In-depth assessments of specific components of the pharmaceutical sector � Pricing � Traditional medicines � HIV/AIDS � Regulatory capacity � Drug supply mgt. � TRIPS Systematic health facility and household surveys Key informant questionnaire sent to countries Getting started Monitoring and assessing the pharmaceutical sector are vital to determine if the key pharmaceutical objectives are met: people have access to essential medicines, these medicines are safe, effective and of good quality, and they are used correctly. A systematic method to assess and monitor the impact of strategies and activities will provide information on issues and gaps, all-important input in the development of health policies. The WHO process for pharmaceutical monitoring and assessment uses a hierarchical approach with three groups of indicators: Level I, Level II and Level III. This provides a standard methodology to follow progress over time and to compare situations in different facilities, districts and countries. Figure 1: Levels of core indicators Level I indicators provide a rapid means of obtaining information on the existing infrastructure and key processes of each component of the pharmaceutical sector. The indicators are assessed by using a short questionnaire completed at national level at a regular time interval (four years), allowing measurement of trends and comparisons of the situation over a period of time. The results provide a range of descriptions of WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 2 - existing structures and processes, and can illustrate the country's capacity to implement policy in specific areas of the pharmaceutical sector - areas which are listed in the next section. The WHO database is updated using data and other information from the Level I survey. Analyses are also used in reports, publications and as references when giving country data. In addition, results can serve as a checklist for countries, and as a basis for doing rapid assessment of the structures and process of their pharmaceutical sector. Level II health facility indicators provide systematic data to measure outcomes on access (affordability and availability of key medicines and geographical accessibility of dispensing facilities) and rational use of quality medicines, including some indication of the quality of medicines at health facilities and pharmacies. Data on these indicators are collected through systematic surveys of public health facilities, public and private pharmacies and public warehouses. The processes involved are detailed in this package. The results of country surveys can be used to indicate the extent to which the objectives set by the pharmaceutical sector - specifically the government and the national medicines policy - have been achieved. The results show the areas and gaps that should be addressed and which strategy can be prioritized for facilities, districts and countries. Global comparison can also be used to establish norms in access, use and, to some degree, quality of medicines made available from health facilities. Subsequent sections of this document focus on the technical aspects and logistics of doing a Level II facility indicator survey. The Level I survey and Level II facility survey do not measure access to and use of medicines from the patient/consumer perspective. Only household surveys can provide population-based information about how pharmaceutical policies affect the well-being of individuals. A working document on household surveys has been developed and is being piloted together with Level II facility surveys in some countries. The methodology assesses whether and how people access medicines, how they use them, how much they pay for them, and how out-of-pocket payments for medicines impact on household finances. Level I and Level II surveys (facility- and population-based surveys) should be done as a way to scope comprehensive pharmaceutical situations — they can be used to establish baseline data and to measure the impact of the strategies implemented. They can also be used to establish trends, particularly the Level I survey which can track the global pharmaceutical situation regularly to measure trends. Information gathered specifically from Level I and II indicators is useful to reassess strategies, priorities and strengthen pharmaceutical system components, and to synchronize programmes and policies. Policy-makers and managers will have a clear picture of national and institutional problems. International agencies and donors can focus on priority areas where the best impact can be achieved. Professional groups and NGOs can focus advocacy and information campaigns. Information can be shared via databases and web pages. Level III indicators are a more detailed and expanded list of indicators covering key components and areas such as those elaborated in several indicator documents in medicine pricing, medicine supply management, HIV/AIDS, TRIPS, rational drug use (RDU) and regulatory capacity assessment. Countries can use any of these set of indicators as baseline assessment and follow-up studies depending on needs and capabilities. Getting started - 3 - The tools necessary to gather most of this information have been brought together in this document. This operational package provides the following: • A Guide for Coordinators and Data Collectors of the Level II Facility Survey detailing operational procedures to carry out the indicator survey, with step-by-step procedures on administrative preparation (budget, training plan and schedule) and technical requirements (training and field-testing, surveying, analysis and reporting). Training slides are also provided. • The annexes contain the technical descriptions of Level II facility indicators and the sampling process. Survey forms are included, and graphs and tables can be generated from the analysis template. • The Level I questionnaire, which is sent to countries once every four years to update global pharmaceutical data, is included in the annexes. It can also serve as a rapid assessment and checklist for countries to check current the pharmaceutical structure and processes of their national pharmaceutical systems. • A diskette that contains Level II facility survey forms, summary forms and training slides. At least 40 countries have already undertaken Level II facility surveys. The current version of the operational package has benefited considerably from experiences in these countries. The indicators have also been used in international/regional courses and meetings to gather data from health facilities and pharmacies. Experience has shown that the survey can be completed without investing large resources in terms of time, people or money. Experiences with using these tools have proven that regular monitoring is not difficult (Level I survey) and can be done in a cost-efficient manner (Level I, II facility). This experience encourages allotting a portion of country support budgets, project grants and MOH budget for monitoring and assessment. WHO published Using Indicators to Measure Country Pharmaceutical Situations: Fact Book on Level I and Level II Monitoring Indicators5 using information from surveys conducted with the above tools. This Fact Book gives the results of the assessment of Level I indicators conducted in 2003 and of Level II indicator surveys conducted between 2002 and 2004. 5 WHO, Harvard Medical School and Harvard Pilgrim Health. Using indicators to measure country pharmaceutical situations — Fact book on WHO Level I and II monitoring indicators. Geneva, World Health Organization, 2006. WHO/TCM/2006.2. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 4 - Pharmaceutical indicators for monitoring and assessment - 5 - Pharmaceutical indicators for monitoring and assessment The need for pharmaceutical sector indicators Given the complexity of the pharmaceutical sector, a systematic method of gathering data is very important for assessing access, quality and rational use of medicines. There are multiple, cross-cutting factors that can influence access and rational use of quality medicines, and a variety of strategies that countries can adopt and implement to improve their pharmaceutical situations. Indicators have been developed for monitoring national medicines policies (NMPs) that enable systematic assessment, evaluation and monitoring of the formulation and implementation of pharmaceutical policies and programmes. These can be used to: • assess country capacity, such as available infrastructure, logistics and human resources to support the pharmaceutical sector and implement NMPs; • monitor the implementation of NMPs; • measure the impact of implementation strategies; and • evaluate progress towards identified objectives. Who can use the results? All stakeholders in the pharmaceutical sector can use indicator-based assessment of the pharmaceutical situation to inform priorities and set targets. They can also use regular monitoring of the sector through indicator-based studies to assess strengths and weaknesses of strategies to improve the provision of pharmaceuticals. Indicators provide policy-makers and managers with a clear picture of national and institutional problems. Policy-makers and managers can refer to study results when developing strategies to strengthen the pharmaceutical sector. Results can also be used to synchronize policies. For instance: • Low access, as measured by availability and affordability of essential medicines, could indicate that policies on health and medicines financing should be reviewed. Economic policies may be focused on joining the global economy without adequate consideration of the implications on pricing, affordability and availability of important medicines. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 6 - • The presence on the market of a large number of substandard medicine products could indicate the need to assess various policies and systems, including licensing and inspection of manufacturers, quality assurance or product registration. International agencies and donors can use the results of indicator-based studies to identify where their activities will achieve the greatest impact. Professional groups and nongovernmental organizations (NGOs) can use the results to focus their advocacy and information campaigns. Standard indicators allow informative comparisons among countries. For example an indicator-based assessment in 12 countries revealed:6 • Most of the countries surveyed have relevant structures in place, however, “it is easier to create a structure than to make it work”. • Most of the countries have a medicines regulatory authority with a mandate to register medicines and inspect manufacturer and retail outlets, however, the enforcement of regulations is often weak. • In most of the countries, public financing for medicines is limited. Countries may use the information as follows: • Advocacy to disseminate and to share the results with national stakeholders, politicians, civil society, acadaemia and key partners (awareness issue). Comparison with other countries at a similar level of economic development, within the region or globally can also be made. • Sharing the information through national workshops, by posting the information on website links and through national brochures. These are also good ways to elicit feedback from stakeholders. • Analysing the country situation allows gaps, probable causes of problems and unmet objectives to be identified. This in turn facilitates the identification of appropriate strategies. • Reporting by topic (access, quality, safety and rational use of medicines), and doing the analysis by type of indicator (structure, process and output) will also guide countries to share sub-regional experiences, through informative comparisons. • Guiding policy-makers in developing and planning national strategy, including budget planning, reallocation of funding and human resource management. which are key cross-cutting factors that can influence access to medicines. • Providing the national medicines regulatory authorities with adequate information for the enforcement of regulations and to assess the level of regulatory authority capability. 6 Comparative analysis of national drug policies. Workshop report. Geneva, World Health Organization, 1997. WHO/DAP/97.6. Pharmaceutical indicators for monitoring and assessment - 7 - • Information and data can be used to: o formulate/revise the NMP. Regular monitoring of implementation and evaluation of progress towards strategic objectives is important in aligning medicines policy to changes in the political, economic and financial environment. o assess the quality of governance, such as systems and mechanisms that would safeguard against vulnerability to corruption. o encourage and facilitate cooperation between the different main players in a country to achieve common strategic goals. Indicator-based monitoring strategies Monitoring NMPs is a complex task. While important, it is difficult to establish a sustainable system of regular monitoring. Resources are not consistently allocated to this task and there is limited advocacy for a culture of monitoring. Further, many efforts to develop monitoring tools have been exhaustive, but impractical. In the past, most monitoring tools included indicators that were difficult to collect, especially if this was done regularly. A number of indicator-based monitoring tools now exist. Indicators for Monitoring National Drug Policies,7 a WHO manual, includes approximately 120 indicators covering structure, process and outcomes of various NMP components. Several countries have used it to monitor and evaluate their pharmaceutical situations. Another set of indicators, developed by Management Sciences for Health,8 focuses on rapid assessment of strengths and weaknesses in the pharmaceutical sector. The WHO manual How to Investigate Drug Use in Health Facilities9 has been used extensively in many countries. This document, WHO Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations. Guide for Coordinators and Data Collectors, was developed to provide a practical indicator-based tool that can be implemented regularly without investing large amounts of human or financial resources. This package relies on a hierarchical approach to monitoring built around three groups of core indicators. The core indicators are easy to collect using standardized methodologies, small samples of data and simple survey techniques. These core indicators systematically measure the most important information needed to gain a comprehensive picture of the pharmaceutical situation in a country. 7 Brudon P, Rainhorn JD, Reich M. Indicators for monitoring national drug policies. Geneva, World Health Organization, 1994. 8 Rapid pharmaceutical assessment – An indicator-based approach. Washington DC, Management Sciences for Health, 1995. 9 How to investigate drug use in health facilities. Geneva, World Health Organization, 1993. WHO/DAP/93.1. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 8 - Grouping monitoring indicators by level (Level I and Level II) has the following advantages: • It offers flexibility to those interested in information on the country pharmaceutical situation: o rapid assessment of key pharmaceutical components; o monitoring outcome and achievement of key objectives of the pharmaceutical policy; and o in-depth assessment of specific system components. • It provides practical methods for regularly monitoring NMPs and their components. • It encourages regular reporting and exchange of pharmaceutical information among facilities, districts, regions, government and nongovernmental agencies as well as international organizations. Level I indicators provide a rapid means of obtaining information on the existing infrastructure and key processes of each component of the pharmaceutical sector. These indicators are assessed using a short questionnaire completed at the national level. Level II indicators provide systematic data on access and rational use of quality medicines through facility-based surveys. Systematic survey processes to collect data on these indicators are contained in this package. A population-based survey has been developed as part of the current process and is discussed in a separate document, Manual for the Household Survey to Measure Access and Use of Medicines. Level I core indicators - 9 - Level I core indicators The Level I core indicators are used to assess existing structures and processes in a national pharmaceutical system. They provide a method to rapidly assess the implementation of NMPs and their components. For Level I indicators, a knowledgeable informant can coordinate the gathering of information. Most data will be available within the Ministry of Health, although data on intellectual property rights protection may require consultation with the responsible ministry. Data collection does not require field surveys and information can easily be updated periodically, for example every two years. Data from Level I indicators can be used to array the achievements and weaknesses of individual pharmaceutical systems and to illustrate common sectoral strategies and approaches. Many WHO Member States have submitted data from the Level I questionnaire to EDM. The WHO MedNet can be consulted to compare results over time and among countries. Comparisons among countries can be particularly interesting and convincing for policy-makers. The questionnaire on Level I indicators is included as Annex 1. The indicators in this questionnaire are summarized below. Pharmaceutical components in Level I indicators National Medicines Policy (NMP) - An NMP document that covers the public and private sectors, a written implementation plan and the integration of medicine and health policies provide a basic framework to organize and improve the pharmaceutical system. They also assist in coordinating the functions and strategies of each component as they are being implemented. Regular monitoring helps to inform the NMP and its implementation. Regulatory system (Regulatory authority, Marketing authorization, Licensing, Regulatory inspection, Control of narcotics and stupifiants, Quality control, Pharmacovigilance, Counterfeit medicines, Dispensing and prescribing, Promotion and advertising) - Legislation/regulations on medicine manufacturing, promotion and advertising, sales, distribution, dispensing and prescribing must be in place. A medicines regulatory authority should be able to efficiently regulate these activities through registration of products, licensing and inspection of manufacturers, importers and pharmacies, control of counterfeiting, control of narcotics and stupifiants and monitoring of adverse medicine reactions. Legislation directed at generic prescribing, dispensing and substitution can help increase access to essential medicines in both the private and public sectors. Quality control of pharmaceuticals should cover all activities to ensure patients receive safe, efficacious and high-quality medicines. There should be a medicines quality control laboratory to test medicines prior to registration and at various points in the distribution system. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 10 - Medicines supply system - Access and availability of essential medicines, especially at public sector facilities, are affected by how medicines are purchased and distributed and how medicines are managed in the health system. Medicines financing - Access and availability are also affected by how much money the government can allocate to medicines, pricing policies, financing schemes (such as insurance programmes and user fees) and medicine donations. Production and trade - Activities ranging from repackaging to formulation of products to developing new medicines are important in assessing the pharmaceutical sector. Implementing TRIPS flexibilities in public health can increase access to medicines. Rational use of medicines - Medicines policies can often have greater impact with effective use of strategies to improve the prescribing and dispensing practices of health workers. Key strategies include standard treatment guidelines, curricula and continuing education programmes on essential medicines concepts, medicines information centres and public education campaigns. Level II facility based core indicators - 11 - Level II facility-based core indicators The Level II facility core outcome indicators support the Level I structure and process indicators by providing specific data about important pharmaceutical outcomes. This set of indicators requires field surveys. For data to be collected accurately and reliably, attention must be paid to appropriate survey design, sampling and data gathering techniques. In selecting the core outcome indicators, consideration was given to the need to obtain the most relevant information from as limited a data collection process as possible. There are 17 survey forms to be completed during a survey (Annex 7). These indicators measure the degree of attainment of strategic pharmaceutical objectives: improved access, quality and rational use. Access is measured in terms of the availability and affordability of essential medicines, especially to the poor and in the public sector. Measuring the actual quality of medicines by testing samples can be expensive. Instead, the presence of expired medicines on pharmacy shelves and the adequacy of handling and conservation conditions of medicines are used as indicators of quality. Finally, rational use is measured by examining prescribing and dispensing habits and the implementation of key strategies such as standard treatment guidelines (STGs) and essential medicines lists (EMLs). Level II indicators are measured in public health facilities, private drug outlets, and in warehouses supplying the public sector.10 Surveys of 30 public health facilities and their dispensaries are used to gather information about availability of essential medicines, medicine prices, stockout duration, the adequacy of conservation conditions, affordability, geographical accessibility, prescribing and dispensing habits, and presence of guidelines. A similar survey of 5 warehouses supplying the public sector also examines availability, stockout duration, and adequacy of conservation conditions. Surveys of 30 private drug outlets assess availability, affordability, geographical accessibility and medicine prices. Survey Forms 1—17 have been developed to obtain data from survey sites. Box 1 summarizes the Level II indicators and lists the corresponding survey forms. The indicators are described in more detail in Annex 2a. Information on data collection and calculation can also be found on the respective survey forms. The sampling framework and survey methodology are described below. Summary Forms 1—4 provide a simple method of combining facility results to obtain national indicators (a spreadsheet is supplied with this material). 10 For the purposes of the Level II survey package, a private drug outlet is a permanent retailer selling medicines, whether a pharmacy, drug seller, drug store, or chemical seller; a warehouse is a central, regional, or district warehouse supplying the public sector; and a public health facility dispensary or public health facility pharmacy refers to the medicines dispensing area of the public health facility whether a pharmacist is present or not. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 12 - Box 1 Summary list of indicators and corresponding survey form used to collect the data Indicator Survey Form Access 1 Availability of key medicines in public health facility dispensaries, private drug outlets and warehouses supplying the public sector 1, 10, 15 2 % of prescribed medicines dispensed or administered to patients at public health facility dispensaries 6 3 Average stockout duration in public health facility dispensaries and warehouses supplying the public sector 4, 16 4 Adequate record keeping in public health facility dispensaries and warehouses supplying the public sector 4, 16 5 Affordability of treatment for adults and children under 5 years of age at public health facility dispensaries and private drug outlets 3, 12 6 Price of key medicines in public health facility dispensaries and private drug outlets 2, 11 7 Price of paediatric medicines in public health facility dispensaries and private drug outlets 2, 11 8 Average cost of medicines at public health facilities and private drug outlets 6, 14 9 Geographical accessibility of public health facility dispensaries and private drug outlets 6, 14 Quality 1 % medicines expired in public health facility dispensaries, private drug outlets and warehouses supplying the public sector 1, 10, 15 2 Adequacy of conservation conditions and handling of medicines in public health facility dispensaries and warehouses supplying the public sector 5, 13, 17 Rational use of medicines 1 % medicines adequately labelled at public health facility dispensaries and private drug outlets 6, 14 2 % patients knowing how to take medicines at public health facility dispensaries and private drug outlets 6, 14 3 Average number of medicines per prescription at public health facility dispensaries and public health facilities 6, 7 4 % patients prescribed antibiotics in public health facilities 7 5 % patients prescribed injections in public health facilities 7 6 % prescribed medicines on the essential medicines list at public health facilities 7 7 % medicines prescribed by generic name (INN) at public health facilities 7 8 Availability of standard treatment guidelines at public health facilities 8 9 Availability of essential medicines list at public health facilities 8 10 % tracer cases treated according to recommended treatment protocol/guide at public health facilities 9 11 % prescription medicines bought with no prescription 14 Other information 1 % of facilities that comply with the law (presence of a pharmacist) Sections A, C 2 % facilities with pharmacist, nurse, pharmacy aide/health assistant or untrained staff dispensing Sections A, C 3 % facilities with doctor, nurse, trained health worker/health aide prescribing Section B 4 % facilities with prescriber trained in RDU Section B Level II facility based core indicators - 13 - Survey design and data collection (Level II - facility survey) The survey of Level II indicators is a very important part of monitoring the pharmaceutical sector because these indicators measure the outcome and impact of pharmaceutical programmes in a country. The Level II core outcome indicator surveys are designed to obtain relevant information from as simple a data collection process as possible. There are 17 survey forms to be completed (Annex 7). Surveys of public health facilities, private drug outlets/pharmacies, and warehouses supplying the public sector are required. Small samples of data and simple survey techniques will be used to collect the quantitative and qualitative information needed for these indicators. In order to estimate indicators accurately and reliably, it is important to follow specific procedures for drawing samples and gathering data, to reduce bias and so that the study population is more representative of the reference population. Larger samples are more costly but give more precise results. Sample size is therefore a balance between what is desirable and what is feasible. The best sample size will be the smallest one that will result in estimates with the desired degree of precision. Sampling bias and error in sampling are discussed in Annex 2b. For data to be collected accurately and reliably, attention must be paid to appropriate sampling and data gathering techniques. Adequate preparation is needed and those who will gather the data must be trained. Key pointers are discussed below. Selecting the geographical areas The survey should be conducted in five geographical areas. A geographical area can be a district, municipality, or province, see Box 2. Selection can occur before the training or during the training with the participation of the data collectors. If appropriate data are available, select the largest or capital city, the most rural or lowest income-generating area, and randomly select three other geographical areas from the remaining regions. If data are not available, select the capital city and randomly select four other areas from the remaining regions. In selecting the five geographical areas, the urban/rural population split can be taken into consideration at this level of sampling or at the next level (facility sampling) depending upon the size and make-up of the geographical areas. In some countries, these areas may be predominantly urban, predominantly rural or they may be mixed depending upon the size of the geographical area sample unit. Most countries have official statistics on the degree of urbanization. However, it may be more practical to take the urban/rural split into consideration when sampling facilities within a geographical area. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 14 - In a number of countries major logistical constraints, such as transport, time, budget and security concerns necessitate excluding some parts of the country from the data collection process. When reporting results of the study, it is important to explicitly state any parts of the country that were excluded from the sampling in order to acknowledge possible bias. There may also be difficulties in countries with large nomadic or mobile populations where there are very few fixed facilities that are operational or used throughout the year. Often in such areas facilities only function in some seasons or there are mobile facilities. Efforts should be made to include these regions if possible. Excluding them may exclude a part of the population for whom access to health care is difficult. The degree of significance in terms of the national picture will depend upon the proportion of the population that such groups represent. Any access difficulties this group has cannot be identified separately unless they represent the whole geographical area. Box 2 Summary of survey areas and sites • Identify five geographical areas o 1 should be the largest or capital city o 1 should be among the lowest income-generating areas o 3 others should be randomly selected Selecting public health facilities To collect most of the indicators for access, quality and rational use, a total of 30 public health facilities need to be selected from the five geographical areas identified (see Box 3). These public health facilities should cater to general outpatients and they should have a pharmacy or a medicine-dispensing area. Within each geographical area, choose six public health facilities providing outpatient services. One of the selected public health facilities should be the main (biggest) public hospital in the area. Of the other five facilities, at least one should be a primary/rural health centre or lowest-level public health facility. The remaining ones should be chosen randomly from all middle- level primary and other public health facilities. The names and affiliations of all facilities in a region can usually be determined at the central level, so this sampling can be done centrally. A number of facilities can be identified as replacements if the facility selected in sampling is not available, has too few patients, or has totally inadequate records. If replacement facilities are used, data collectors should note the reason for not surveying the first facility selected. If resources allow, countries may choose to add surveys of private, nongovernmental or mission health facilities to the core package. If the country situation is such that at least 25% of primary health care is delivered by facilities other than public health facilities — such as missions, NGO, social security, etc. — the public health facility forms can be adapted for use at these facilities as well. Any surveys done at other facilities should be in addition to the surveys carried out at public health facilities. Likewise, drug outlets/pharmacies/dispensaries connected to these facilities can be included in the survey. The private pharmacy/drug outlet survey forms can be adapted for use in these drug outlets. Level II facility based core indicators - 15 - Box 3 Selecting facilities from 5 geographical areas Public health facilities - Each facility should cater to general outpatients and have a medicine-dispensing area. Total for country: 30 public health facilities Per geographical area: 6 public health facilities 1. The main or biggest public hospital in the area 2. One primary/rural health centre or lowest-level public health facility 3. Four middle-level public health facilities, randomly selected Private drug outlets Total for country: 30 private drug outlets Per geographical area: 6 private drug outlets 1. Select the private drug outlet closest to each public health facility included in the survey Central/regional/district warehouses Total for country: 5 warehouses Per geographical area: 1 warehouse Making random selections Box 4 provides an example illustrating the process of randomly selecting public health facilities from the capital city. The same process is used to select public health facilities in each of the five geographical areas included in the survey. Box 4 Randomly selecting public health facilities from the capital city Step 1: Select the first facility. From a list of all public health facilities in the capital city, select out the main or biggest public hospital. Step 2: Select the second facility. Identify all of the primary health centres or lowest-level public health facilities and randomly select one. Step 3: Number the remaining middle-level primary and public health facilities that are not hospitals or lowest-level public health facilities. Step 4: Calculate the sampling interval. For example, if there are 303 middle-level health facilities in the capital city and 4 are to be chosen, the sampling interval is calculated by dividing the total number of facilities by the number to be selected: 303 ÷ 4 = 75.75 Step 5: Identify the third, fourth, fifth and sixth facilities • Choose a random whole number between 1 and 75.75, for instance 35. • The third facility to be chosen will be the one numbered 35. • Add the sampling interval to the randomly chosen number: 75.75 + 35 = 110.75 = 111 (always round up). The fourth facility is the one numbered 111. • Continue until all 6 facilities from the capital city have been chosen, i.e. the fifth facility is: 75.75 + 110.75 = 186.5 = 187; the sixth facility is: 75.75 + 186.5 = 262.25 = 263 WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 16 - Selecting private drug outlets Select the closest private drug outlet to each public health facility surveyed (see Box 4). It may be possible to do this selection centrally from data from the ministry/department of health, the chief pharmacist and/or the national pharmacy association/council. However, information may be outdated and you may find that the “map and terrain” do not match. Data collectors may need to select the private drug outlets to be surveyed after arriving in the field. Depending on the country situation, other types of drug outlets may be surveyed as well. If another sector of drug outlets (other than private drug outlets and dispensaries/pharmacies connected to public health facilities) provides at least 25% of primary health care medicines, the private pharmacy/drug outlet forms can be adapted for use in these drug outlets. Surveys of other drug outlets should be done in addition to the surveys completed at public health facility pharmacies/dispensaries and private drug outlets. Selecting warehouses Select one central/regional/district warehouse in each geographical area for inclusion in the survey (see Box 3). Sampling patients for data collection To measure the rational medicine use indicators, general outpatient encounters from health facilities should be used. The patients to be sampled should be restricted to general illness encounters, representing a mix of health problems and ages. The indicators from the facility survey have limitations when applied to well-child visits, pre- and post-natal visits, specialist consultations, or even separate clinics for adults and paediatric cases because treatment practices are different and results may be difficult to interpret. The survey will include both retrospective and prospective sampling. In some cases only one approach will be possible, but elsewhere, there will be a choice. Prospective sampling can introduce bias due to seasons, variations in staffing, inconsistencies in the supply cycle etc. Retrospective sampling The survey should retrospectively sample the previous 12-month period. There are several ways of sampling. Random selection of patients is illustrated in Box 5. Level II facility based core indicators - 17 - Box 5 Simple method for selecting 30 patients from the general outpatient list If there are 5000 patients treated during the entire period and the sample size is 30: • Calculate the sampling interval by dividing the total number of patient encounters on the general outpatient list for the year by the number of patients to be selected. For example, if there are 5000 patient encounters on the general outpatient list covering the previous 12 months and 30 patient encounters are to be selected: 5000 ÷ 30 = 166.6. Sampling interval is 166.6. (5000 ÷ 30 = 166.6 is the sampling interval) • Select the first patient using a table of random numbers or just point to one patient at random. This is the 1st patient, add 166.6 to get the next patient and so on until 30 patients is reached. Another method of sampling is by the following chronological sampling method described in Box 6. Box 6 Retrospective sampling: Selecting 30 patients from general outpatient records covering 12 months (365 days) A. Chronological sampling method • Calculate the sampling interval by dividing the number of days covered by the outpatient list by the number of patients to be selected: 365 ÷ 30 = 12.2 days. • Number each day covered by the outpatient list. • Randomly select one patient encounter between day 1 and day 12.2. • Each subsequent encounter is selected by adding the sampling interval (12.2) to the previous total and rounding up. In other words, select one patient encounter from the first day that was randomly selected (example 1st patient was selected from day 3, skip the sampling interval, and select one patient from day 15 (3 + 12.2 = 15.2 16 always round up). Second patient is selected from day 16 Third encounter = 15.2 + 12.2 = 27.4 = 28th day Fourth encounter = 27.4 + 12.2 = 39.6 =40th day Fifth encounter = 39.6 + 12.2 =51.8 = 52nd day To choose a patient encounter from the patient list for each selected day, pick a random number between 0.0 and 1.0, multiply this number by the number of patient encounters on the list, and round upwards. For example, if you choose 0.4 and there are 18 patients on the list for the selected day: 0.4 x 18 = 7.2 = 8 (always round up). Review the 8th patient encounter when completing the survey forms provided the patient meets the inclusion criteria. If the patient does not meet the inclusion criteria, select the next patient on the list who does. Repeat this process, selecting one patient encounter from each selected day until 30 patient encounters have been reviewed. Patients for Survey Forms 7 and 9 are selected retrospectively. For retrospective sampling the following steps should be undertaken: WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 18 - • Confirm the availability and accessibility of medical records. Possible sources of retrospective prescribing data include clinic registers, treatment logbooks, patient/family files and retained prescription forms. The data collector must know where to locate these records in the facilities and how to use them. • Identify the study period to be covered. Because of seasonal differences, variations in staffing, inconsistencies in the supply cycle etc., the survey should, as far as possible, cover the 12-month period prior to the date of data collection. If records during this period cannot be found or do not meet the sampling requirements, the study period can be shortened, making sure that there are no missing records during the period covered. This can be done by checking that all the months and days are represented in the record. • Select patient encounters and extract data Survey Form 7 From general outpatient treatment records, select 30 patients with any diagnosis seen during the last 12 months. Use either the simple or chronological sampling method described in Boxes 5 and 6. If records are not available, 30 patients may be selected from current treatment records provided there are sufficient records to randomly select the patients. Prospective sampling can also be used. Survey Form 9 From general outpatient records or under-5 ledgers, select: • 10 patients under 5 years of age seen during the last 12 months for diarrhoea, • 10 patients under 5 years of age seen during the last 12 months for pneumonia, and • 10 patients of any age seen during the last 12 months for acute respiratory tract infection (ARI). If the selected patient encounter does not meet the inclusion criteria (i.e. does not have the appropriate diagnosis) select the next patient on the list who does. Prospective sampling11 Prospective sampling is used in Survey Form 6. Interview 30 patients leaving the dispensing area/pharmacy or leaving the facility after they have been treated and received medicines to see how many of the prescribed medicines were dispensed, if the medicines are adequately labelled (the label should contain the medicine's name and how it should be taken), if the patients know how to take their medicines (patient knows dosage and duration of all dispensed medicines), and how much the patient paid out-of-pocket for medicines and in non-diagnostic fees. Patients may be interviewed consecutively or as convenient. It is important to know how to identify the patients to be included and to construct a system that will allow patients to be interviewed without disrupting the normal activities of the facility. Planning the data collection is also important. Throughout the day, facilities tend to have peak times when collecting enough interviews will be easy and low periods when there will not be enough patients to carry out the surveys effectively. 11 Prospective methods for sampling encounters. In: How to investigate drug use in health facilities, pp. 63-65. Geneva, World Health Organization, 1993. Level II facility based core indicators - 19 - Summary of Level II indicators measured by facility Table 1 gives the overview of the facilities to be visited and indicators to be measured during the Level II survey (see Annex 7, for corresponding survey forms). Table 1 Study facilities, indicators and forms Indicators Public health facilities (n=30) Private drug outlets (n=30) Warehouses (n=5) Access Availability of key medicines Check availability of 15 medicines (P)* SF 1 Check availability of 15 medicines (P)* SF 10 Check availability of 15 medicines (P)* SF 15 % medicines dispensed or administered Exit interview 30 patients SF 6 Average stockout duration Review stock cards of 15 medicines (R) SF 4 Review stock cards of 15 medicines (R) SF 16 Adequate record keeping Review stock cards of 15 medicines (R) SF 4 Review stock cards of 15 medicines (R) SF 16 Affordability of treatment Check price of medicines to treat pneumonia and another condition (P) SF 3 Check price of medicines to treat pneumonia and another condition (P) SF 12 Price of key medicines Check price of 15 medicines (P) SF 2 Check price of 15 medicines (P) SF11 Price of paediatric medicines Check price of paediatric medicines (P) SF 2 Check price of paediatric medicines (P) SF11 Average cost of medicines Exit interview 30 patients SF 6 Exit interview 30 patients SF 14 Geographical accessibility of dispensing facilities Exit interview 30 patients SF 6 Exit interview 30 patients SF 14 Quality % medicines expired Check if there are expired medicines (P) SF 1 Check if there are expired medicines (P) SF 10 Check if there are expired medicines (P) SF 15 Adequacy of conservation conditions and handling of medicines Check conditions using checklist (P) SF 5 Check conditions using checklist (P) SF 13 Check conditions using checklist (P) SF 17 WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 20 - Table 1 Study facilities, indicators and forms Indicators Public health facilities (n=30) Private drug outlets (n=30) Warehouses (n=5) Rational Use of Medicines % medicines adequately labelled Exit interview 30 patients (P) SF 6 Exit interview 30 patients (P) SF 14 % patients knowing how to take medicines Exit interview 30 patients (P) SF 6 Exit interview 30 patients (P) SF 14 Average number of medicines per prescription 30 patient records (R) SF 6 & 7 % patients prescribed antibiotics 30 patient records (R) SF 7 % patients prescribed injections 30 patient records (R) SF 7 % of prescribed medicines on EML 30 patient records (R) SF 7 % medicines prescribed by generic name (INN) 30 patient records (R) SF 7 Availability of STGs Check at 30 facilities (P) SF 8 Availability of EMLs Check at 30 facilities (P) SF 8 % tracer cases treated according to recommended treatment protocol/guide 30 cases (10 each of diarrhoea, pneumonia and ARI) (R) SF 9 Prescription medicines bought with no prescription Exit interview 30 patients SF 14 Other information % of facilities that comply with the law (presence of a pharmacist) Check at 30 facilities (P) Section A Check at 30 facilities (P) Section C % facilities with pharmacist, nurse, pharmacy aide/ health assistant or untrained staff dispensing Check at 30 facilities (P) Section A Check at 30 facilities (P) Section C % facilities with doctor, nurse, trained health worker/health aide prescribing Check at 30 facilities (P) Section B % facilities with prescriber trained in RDU Check at 30 facilities (P) Section B * (R): retrospective review; (P): prospective review. Preparing the survey (Level II - facility survey) - 21 - Preparing the survey (Level II - facility survey) The survey of Level II indicators is a very important part of monitoring the pharmaceutical sector because these indicators measure the outcome and impact of pharmaceutical programmes in a country. Adequate preparation is needed and data collectors must be trained. Coordination and survey coordinator A national coordinator should be selected to take charge of the overall coordination of the Level II survey, to oversee the survey process, data analysis, reporting and presentation of results. The coordinator should be someone who is knowledgeable about the pharmaceutical sector and experienced in conducting surveys. A thorough working knowledge of the pharmaceutical sector will help to ensure that important aspects of the sector are not overlooked in planning for the Level II survey. The results of the Level I questionnaire will also inform the planning process. The coordinator has to go through the manual carefully, covering concepts of assessment, monitoring, indicators and how to carry out a systematic survey. The coordinator should also go through the sections of this manual on survey design and data collection, and on how to prepare for the training and conduct the actual survey. Any group within a health ministry/department involved in pharmaceutical activities, such as a pharmaceutical service management group, medicines regulatory authority, or national medicines policy office can coordinate the Level II survey activity. It is also possible for other groups such as NGOs, professional groups and acadaemia to initiate the activity. WHO, through its country and regional offices, and other donor agencies may also be able to provide assistance. A complete coordinator checklist is provided in Annex 3. The roles and functions of the coordinator include: • Communicating with government officials and other local agencies to gain approval for the survey and to request personnel who will do the survey; • Communicating with officials and health facilities to be visited for the field test and for the actual survey; • Selecting geographical areas and identifying facilities to be surveyed; • Allocating budget and, if necessary, requesting financial (Annex 4,) and technical support. Be sure that main components, such as data collector training, the survey itself, including transport and per diem details, analysis and dissemination of results, are covered; • Coordinating and identifying sources of information for the Level I questionnaire on structures and processes of the country pharmaceutical situation; WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 22 - • Identifying country-specific items on the survey forms, for example, the key basket of medicines, treatment guidelines; • Preparing all the necessary materials for training, the field test, and survey of Level II indicators; • Providing training and arranging the field test exercise for Level II data collectors; • Supervising the actual survey of Level II indicators; • Checking that survey forms are completed correctly and in full; • Checking the computations on the survey forms; • Coordinating completion of the summary forms, analysing results, writing of the report, and presenting/giving feedback of the results to appropriate groups. Tailoring the survey forms to the country situation through choosing key medicines, selecting tracer conditions and identifying treatment protocols In addition to selecting geographical areas and facilities to be included in the survey, there are a number of items on the survey forms – such as the basket of key medicines, treatment protocols and tracer conditions – that need to be identified at the national level. The survey forms used to collect data on Level II indicators are designed to be tailored to country situations (Box 7). Tailoring must be done at the national level and country- specific features pre-printed on the survey forms and discussed with data collectors during training. The survey forms also have several places where countries can indicate additional data to be collected without affecting the indicator outcomes. Box 7 Items on survey forms to predefine at national level Survey Form Item 1, 2, 4, 10, 11, 15, 16 Identify basket of key medicines and paediatric medicines 5, 13, 17 Review language so it fits terminology commonly used. During training ensure data collectors are using uniform criteria for evaluating items on the checklists for conservation and handling conditions 3, 12 For affordability, identify treatment of choice for pneumonia and additional condition with treatment of choice 6, 14 Define the requirements for adequate labelling on medicine dispensed and standard patient knowledge if different from those listed 7 Define what will be considered as an antibiotic, and injection Develop guidelines for data collectors on categorizing generic versus brand medicines 8 Identify the acceptable guidelines and medicines lists; obtain 1 copy of each for each survey team 9 Define 1st-line antibiotics for mild/moderate pneumonia and identify any optional conditions and medicines that will be used to measure recommended or non- recommended practices Preparing the survey (Level II - facility survey) - 23 - Selecting a basket of key medicines (Survey Forms 1, 4, 10, 15 and 16) A list of 15 key medicines used to treat common health problems must be selected to measure availability, presence of expired medicines, medicine price and stockout duration. It is important to select key medicines that are basic requirements at all levels of health care. When selecting the medicines, list the 15 most common conditions treated at primary health care level and choose medicines used to treat these conditions. The chosen medicines must be: • on the national essential medicines list • the most important therapeutically and based on national treatment guidelines or at least on the consensus of experts • the most widely used of the medicines meeting the above criteria • medicines expected to be available at all primary health care facilities at all times. The basket of key medicines can be selected systematically applying the principles above by referring to official morbidity data for adults and children, and following this step-by-step guide: 1 List the top 15—20 morbidity's for adults and children. 2 Remove from that list conditions that would not be treated as general outpatient cases at primary health care services. 3 For each remaining morbidity, assign the most important medicine that corresponds to the applicable STG. Be sure to include medicines for both adults and children. 4 The resulting list will probably be longer than 15 medicines, prioritize and reduce the list of key medicines to reflect the principles listed above. 5 Revisit the list and ensure that important medicines used for the alleviation of common symptoms and important preventive medicines have not been excluded. If they have, make appropriate substitutions in order to have a list of 15 medicines. Medicines that are known to be problematic should not be included in the basket of medicines as this will disproportionately reduce the value of this indicator. Instead, such medicines can be monitored separately. Likewise, other medicines that may be of interest, but do not meet all the principles above could be monitored separately. This enables additional data to be collected with very little effort and without adversely affecting the results. A model list of key medicines is presented in Table 2. This list can be modified based on important health problems in the country. Once the medicines have been selected, pre-print them on the survey forms. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 24 - Table 2 Model list of key medicines diarrhoea oral rehydration salts (ORS), co-trimoxazole tablets acute respiratory tract infection amoxycillin, co-trimoxazole tablets, procaine penicillin injection, paediatric paracetamol tablets malaria chloroquine tablets anaemia ferrous salt + folic acid tablets worm infestations mebendazole tablets conjunctivitis tetracycline eye ointment skin infection iodine, gentian violet or local alternative fungal skin infection benzoic acid + salicylic acid ointment pain acetylsalicylic acid or paracetamol tablets prophylactic retinol (vit. A) ferrous salt + folic acid tablets Identifying medicines to be considered as injections and antibiotics (Survey Form 7) To complete this form consistently, data collectors also need to use common criteria for identifying which of the prescribed medicines to classify as injections and antibiotics. Immunizations and injectable contraceptives should not be counted as injections. Definitions of medicines considered as antibiotics must be discussed. Antimicrobial agents are not always classified in an identical way. Indicators for antibiotic use can be sensitive to certain medicines, especially in places with high incidence of parasitic infections, such as malaria or tuberculosis. Medicines such as antiprotozoals and anthelminthics are also usually placed in a different category of antibiotics. How to classify topical antibiotics widely used in areas where trachoma, bacterial conjunctivitis and bacterial skin infection are common will also need to be considered. Identifying medicines of choice to treat pneumonia in adults and children (Survey Forms 3 and 12) One indicator of medicine affordability uses the price of treatment. The STGs to treat moderate pneumonia and another condition in adults and children who are not hospitalized should be identified. The STGs to be used must be produced by an unbiased organization. Ideally, these should be officially endorsed and used by the ministry/department of health. For adults and children, identify the medicine of choice and the recommended dosage preparation based on the guidelines. For example, from the WHO Model Formulary12 and WHO guidelines13 the following medicines and dosage preparations are recommended for pneumonia (other dosage preparations can be chosen if these are not available on the market): 12 WHO, 2006. WHO model formulary. Geneva, World Health Organization 13 WHO, 2005. Pocket book of hospital care for children. Geneva, World Health Organization. Preparing the survey (Level II - facility survey) - 25 - Adults: Amoxicillin 0.5-1g every 8 hours (capsule/tablet) Children: Amoxicillin 25 mg/kg 2 times a day The medicines of choice, their preparations, and the number of units needed to complete treatment should be pre-printed on the survey forms. If the WHO-recommended guidelines will not be used, the treatment guidelines for pneumonia and another condition must be carefully specified with input from national experts. Selected treatment guidelines must be developed by an independent group and not influenced by pharmaceutical promotion. Identify standard criteria for adequate labelling and patient knowledge (Survey Forms 6 and 14) The standard minimum criteria for adequate labelling of medicines dispensed to patients have to be set before the survey is carried out and discussed with surveyors. Minimum criteria should at least be to have the name, dosage and duration of treatment written on the package given to the patient. The patient/caregiver should at least know the appropriate dosage and duration of medicines dispensed. These criteria can be adjusted as appropriate for the survey population. Identifying unit price of medicines to treat outpatient pneumonia in adults and children (Survey Forms 2 and 11) The unit price of each medicine at the facility or the price charged to patients should be recorded, including the price of a syringe, if applicable. The lowest-priced brand or generic equivalent medicine should be used. If there are flat charges paid for each medicine given to patients, then this amount should be recorded as the price of the medicine. Indicate if medicines are given free of charge. Instructions in obtaining drug prices of global and regional drug list [paid by the patient or paid by the facility] (Survey Forms 2 and 11) Attention should be paid to ensure collection of unit prices of drugs' preparation unit, indicated in column B, e.g., if the price refers to a medicines sold in 30-tablet boxes, the value of the box should be divided by 30 to obtain the price of one tablet. All countries should obtain the prices of drugs in the global list. Countries are also encouraged to get the prices of key drugs in the regional lists (Africa, Eastern Mediterranean, South America, Central Asia, South Asia and the Western Pacific). These lists are included on the CD. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 26 - Training data collectors - 27 - Training data collectors Selecting data collectors Based on experience, the most effective data collectors are those with clinical experience such as physicians, nurses, pharmacists or paramedical staff. Health ministry/department staff and temporary employees with some health-related background and experience are possible candidates. It can also be useful to select data collectors from different parts of the country to reflect language differences and enable a more diverse sampling of geographical areas–this could also resolve any concerns data collectors may have about travelling to unfamiliar areas. In all, 10 data collectors will be needed–one team of two data collectors for each geographical area. Data collection is the most crucial part of the monitoring process. For accuracy and reliability of information and indicator measurement, the data collectors must be well trained. Training should focus on ensuring data collectors have a common understanding of the required information and know how to gather the data and complete the survey forms in a standardized fashion. Training components The training will be in three parts (Box 8). Box 8 Sample training schedule Part I: Briefing on survey forms and patient sampling (1st day - half day) Part II: Field test (2nd day - half day) Part III: Calculation, debriefing and logistical issues (3rd day) Note: All data collectors should be trained together • How to use Survey Forms 1—17. Copies of these forms must be available for discussion and use during training. • Ethical issues and confidentiality when interviewing people (professionals and users), and handling documents (administrative and clinical). • Sampling exercises using copies of actual general outpatient lists and patient records. These can be obtained from nearby facilities. It is advisable to keep all the records and materials used in training for future use. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 28 - • Exercises on how to calculate stockout duration using copies of stock cards • Field tests in public health facilities and private drug outlets to familiarize data collectors with the procedures they will use in the field. Note that facilities visited during the field test and those that will be included in the actual survey should be contacted and the permission of the facilities and appropriate officials sought in advance. Data collectors will need to be properly introduced to the facilities through an endorsement letter from an appropriate authority, e.g. the health ministry/department. Training tools needed • Slides 1—33, explained below and provided as Annex 6. • Survey Forms 1—17 (data collectors will need copies for discussion during training, for practice during the field test, and for carrying out the actual survey). • Samples of actual general outpatient lists, patient records and stock cards. • Copies of the National Medicines List. • This manual provides all the information on the indicators and the survey process. Sampling methods crucial in getting valid and accurate data are also described. Training slides introduced Several slides have been prepared and can be used by the national coordinator to train data collectors in countries. There are two other sets of slides, to train coordinators and to introduce the activity to policy-makers and other parties/stakeholders. These sets can be adapted if there are other aspects and information that need elaborating. Slide set to train data collectors - This is used by the national coordinator to train data collectors on the details of gathering data. Details of how to use the slides to train data collectors are in Annex 5 with a Powerpoint presentation in Annex 6. The trainer should review these annexes. Trainers and survey coordinators can adapt/adopt and include more information as needs are perceived during training. The slide set presented in the annex can be adjusted according to needs. Examples are: the need to elaborate further on the concepts of monitoring, evaluation and assessment; to discuss the definition of each indicator; and how to transfer data in the summary forms. Introduction Slides 1—4 introduce the activity, why monitoring is important and who can use the results. Slides 5 to 8 discuss indicators as basic tools in monitoring, pointing out that it is important to use a standardized procedure to enable comparisons. Training data collectors - 29 - Survey logistics This covers some of the logistics involved in the surveys, and adjustments and decisions to be made at the central level between the coordinator and those who will do the survey. Slide 9 explains the logistics of the survey. Slide 15 (key medicines) can be completed before or during the training. Discussion of the survey forms Slides 13-25 review Survey Forms 1-17. A complete set of Survey Forms for discussion must be provided to each of the participants. This includes practise on techniques for conducting retrospective and prospective sampling. An exercise using patient lists to select sample encounters is necessary. Data collectors should know how to get information from patient records both for general outpatients (Survey Form 7) and for cases of diarrhoea, pneumonia and ARI (Survey Form 9). Slide set for training national coordinators (file is in the CD) - is developed to train the key person who will coordinate and survey the process and who will train the data collectors in countries. Usually regional or subregional training is conducted to train coordinators from different countries. In this training the national coordinators are exposed to the principles of pharmaceutical assessment, the survey methodology and survey forms. They are also trained how to organize the survey process in their respective countries. This training equips them to train data collectors. Slide set for policy-makers (file in the CD) - This set is included for national coordinators to present the Pharmaceutical Situation Assessment Survey proposal to national policy- and decision-makers. This set includes the survey objectives and rationale as well as a guide for the main methodological decisions (geographical areas and survey sites, key medicines and tracer conditions). The field test As part of the training, two to three public health facilities with dispensaries must be identified for the field test. The data collectors should be divided into three teams and each team assigned to one of the facilities. Private drug outlets should be identified in advance or selected by the teams on the day of the field test depending on which procedure will be used during the actual survey. A notice or an agreement with facilities to be visited for the field test must be obtained before the training. Each team should have a complete set of Survey Forms 1—17 and Summary Forms 1—4. They should familiarize themselves with the sampling procedures, data-gathering processes, and appropriate completion of each form. Each data collector should practise filling in a part of each survey form and calculating the values during the field test. Any problems encountered in collecting data or completing the forms should be WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 30 - discussed during the debriefing. The coordinator should oversee the field test and review the completed forms to identify areas needing further explanation. During the field test data collectors should take note of the checklist in Box 10. Review the field test and calculations Discuss the field test. Review all the survey forms and summary forms and discuss any difficulties in collecting data or completing the forms: • Check that all computations are done properly. Each survey form must be checked for inconsistencies in data entry. • Review sampling. • Check the number of days covered by the review of stock records (Survey Forms 4 and 16). • Discuss questions arising from the field test exercise. • Ensure data collectors have a uniform approach to gathering data and completing forms. At this point the data collectors should know how to use the Survey Forms and Summary Forms. The survey - 31 - The survey Notifying facilities to be visited • Obtain permission from officials responsible for public health facilities. In some cases verbal permission may be sufficient; in others it may be necessary to send a formal letter. Box 9 provides a sample letter. • Data collectors must be properly introduced to the facility (both for the field test and actual survey) through an endorsement letter from the health ministry/department or the WHO country office. Travel schedule and logistics The national coordinator should: • Arrange the schedule and survey dates, including the necessary travel time to get to the selected geographical areas: o there should be five teams. A team of two field workers should be assigned to each geographical area; o all indicators from each public health facility and its dispensary can be collected in one and a half days, including the visit to a nearby private drug outlet and central warehouse; o each team can complete its portion of data gathering in approximately nine days. • Provide the necessary logistics: survey forms, calculator, and per diem or transport costs as applicable. Box 9 Sample letter to facilities Dear_______________________ Pharmaceuticals are important in maintaining and providing health services to the population. It is also important to know if the population has access to essential medicines of good quality and whether or not these medicines are being used properly. The name and location of facility has been selected as one of the facilities to be included in a national survey to assess the pharmaceutical situation in name of country . On date of the visit , our staff will be visiting your facility to gather some information on the availability of some key essential medicines and how these are being used. While at your facility, our staff will need to access list all areas where access is needed, i.e. the last 12 months of outpatient records, the last 12 months of stock records. Thank you for your cooperation. Sincerely, WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 32 - • Make copies of all the survey forms. The cover sheet of the survey forms' packet lists the necessary number of copies. • Indicate on the bottom of the Summary Form 1, Public health facility pharmacies/dispensaries, the names of the geographical areas included in the survey and the numbers assigned to facilities in each area. Assign the same number to the public health facility and its dispensary and to the nearby private drug outlet. Note that each survey form has a box in the upper right-hand corner. The facility number should also be recorded in this box. Data collectors On arrival at the facilities, data collectors will need to speak to the person in charge of the facility and make a brief tour in order to be able to understand patient flow, where to locate the desired patients, and the existence and location of the necessary records. It is essential that this is done before any data are collected otherwise incomplete or incorrect data might be gathered, wasting time or invalidating the results. A checklist for data collectors is provided in Box 10. Box 10 Checklist for data collectors Check if completed At the geographical area level (region, province, district) 1 Confirm that the necessary authorization from the relevant authorities has been received and check on local logistical and security issues. Obtain any necessary additional authorization letter 2 Identify the location of the facilities and the most efficient order in which to visit them At the facility level (public health facility and private drug outlet) 1 Take an initial tour of the facility and talk with key staff to understand the patient flow, and existence and location of records 2 Determine the best order to complete the survey forms and which data collector will complete which forms 3 Confirm the availability and accessibility of records: • Location of general outpatient encounter records • Patient records • Stock records 4 Decide on retrospective or prospective sampling for survey forms based on the availability and accessibility of records 5 For retrospective sampling identify the study period to be covered The survey - 33 - After completing the actual survey, data collectors should submit a brief report on the data collection process and, in particular, anything that will be important for the report writer in interpreting the results. Data collectors should also note if they had to visit more facilities in addition to the ones actually included in the survey and record the reason facilities were skipped, whether due to a lack of available records or other problems. Supervision of data collection Supervision of the data collectors should aim to ensure that the agreed procedures and methods are being followed, and the data collected are complete and of good quality. It should also solve problems such as incomplete answers and omissions. The handling and storing of data collection forms before they are processed and analysed is important as well. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 34 - Data processing, analysis and reporting: What to do with results? - 35 - Data processing, analysis and reporting: what to do with the results The coordinator should oversee the analysis of the data and the writing of the report. Guidance and a suggested report outline are provided below. Data processing should be carefully planned. Even before data collection begins, the necessary resources and a person to encode/tabulate the data and do the computations must be identified. Computation of Level II facility indicators During the survey, the coordinator must check the accuracy of data and information reported as far as possible in the field. These should be checked for completeness and consistency before making the final calculations and analysis. Data coding and entry must be double-checked. The data entered on the Survey Forms should be calculated following the instructions and formulas on each of the Survey Forms. All the calculations can easily be done manually. The completed results per facility/outlet should then be transferred onto the relevant Summary Form. For Level II facility indicators, Summary Forms 1—4 (public health facility dispensaries, public health facilities, private drug outlets, and warehouses) should be used to calculate and analyse data, make graphs and tables, and prepare reports. Summary Forms 1—4 automatically compute values, including a summary of indicators, once data are entered and can also generate several graphics automatically. The Summary Forms are included in the CD. Quality of data and information The quality of the information that can be generated depends on the quality of data. The data must be checked during the field survey and missing information completed. The data must be checked again for completeness and consistency before the final calculation and analysis. Inconsistent data that cannot be corrected must be excluded. How the data have been coded and subsequently entered must also be checked when entering them into the Summary Forms. Reporting biased data can also occur. Those involved in the survey or those providing information on Level I indicators may either be motivated to report only on the success of a programme or only the negative aspects of the existing situation. Those involved in collecting, analysing or presenting the data should bear in mind that the purpose of the exercise is to measure what is really going on in order to improve the situation. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 36 - Analysis and interpretation of Level I and Level II facility indicators Using the information from the Level I questionnaire and data from Level II facility Summary Forms, the country pharmaceutical situation can be analysed and discussed. The information in the Level I questionnaire can be used to discuss the existing pharmaceutical sector structures, and what activities and strategies have been and are being implemented in the country. The results of the Level II facility survey should be able to provide a glimpse of the probable impact or outcome in terms of meeting the overall objectives in pharmaceuticals. Thus, for analysis of outcomes and impact, Level I indicators can be discussed in the light of the results and outcome measures of the Level II indicators. The results and information from Level II core indicators can be correlated with the information on processes and structures obtained from the Level I questionnaire. The Level I questionnaire gives information on: ¾ Existence of an NMP that provides a basic framework to organize and improve the pharmaceutical system and integrates medicines and other health policies. An NMP also assists in coordinating the functions and strategies of each component of the pharmaceutical system and implementation of these strategies. ¾ Profile of the regulatory system such as legislation/regulations on medicine manufacturing, promotion and advertising, sales, distribution, dispensing and prescribing must be in place. A medicines regulatory authority should be able to efficiently regulate activities such as generic prescribing, dispensing and substitution and can help increase access to essential medicines in both the private and public sectors. Quality control of pharmaceuticals covers all activities to ensure patients receive safe, efficacious and high-quality medicines. ¾ Medicines supply systems are affected by how medicines are purchased and distributed, and hence how medicines are managed in the health system, which has a significant impact on access and availability of essential medicines, especially at public sector facilities. Access and availability are also affected by how much money the government can allocate to medicines, pricing policies and financing schemes (such as insurance programmes and user fees). ¾ How medicines are used - what structures and processes are available that can help to improve the prescribing and dispensing practices of health workers? Key strategies include STGs, curricula and continuing education programmes on essential medicines concepts, medicines information centres, and public education campaigns. The Summary Forms for Level II facility survey/indicators are grouped by point of data collection (public health service facility, public and private pharmacy/drug outlet and public warehouse). In the Summary Form, the summary indicator - Worksheet 2 presents the summary of all indicators. Outcome measures are shown by areas of achievement - access, quality and rational use of drugs. The sampling scheme of patients at each health facility and pharmacy/drug outlet limits the potential to compare one facility with another. Extending the sample to 100 patients per facility should allow such comparison. However if applicable and provided discussion is still focused on the national significance of the survey, each of the five geographical areas can be analysed and discussed. Regional comparisons may Data processing, analysis and reporting: What to do with results? - 37 - be of interest where there is especially wide variation or contrast, particularly with a group of related indicators. Regional comparisons should be done sparingly as over- emphasizing the five regions included in the study will detract from the study’s significance as a national survey. However, the characteristics of each geographical area should be described. Other measures that do not involve patient data can be compared, including affordability, availability, conservation condition, adequacy of storage and dispenser and prescriber profiles can be discussed across facilities and geographical areas. It is important to describe the general characteristics of each facility and the geographical area where they are sampled in such discussions. Data analysis of Level II facility surveys can be divided into three topics using the data from the worksheet of summary indicators: access, quality of drugs and how drugs are used. Access can be discussed in relation to the physical availability of key drugs and stockout duration, affordability and prices of medicines. It can also be discussed using data on access to health facilities. Public and private sectors can be compared. • Availability of key drugs indicates the system's ability to cater to common disease conditions. The length of stockouts can provide information on the supply logistics in relation to these key drugs. Drugs dispensed to a patient are a good measure of access, especially at public health facilities. • Affordability to treat pneumonia expresses how much a patient has to pay for treatment of an illness in terms of the number of day's wages of the lowest-paid government worker. • Prices of a selected basket of medicines can be compared across pharmacies and drug outlets, across geographical areas and can also be compared to prices in other countries. International price comparison is a good tool for advocacy for policy on drug prices. The Health Action International web site can be used: http://www.haiweb.org/medicineprices/. • Distance to health facility and cost to get there can also provide information on measure of access. These will be particularly useful when comparing facilities in urban and rural areas. The storage and handling conditions of medicines and the expired medicines on dispensing shelves can provide an indication of the quality of drugs distributed to patients. Measuring quality by testing samples is costly for this level of assessment. These indicators are measured at pharmacy dispensing and storage areas at public and private pharmacies/drug outlets as well as at public warehouses. These can also be compared. Drug use can be assessed by measuring how health workers are prescribing, such as number of drugs prescribed, magnitude of prescribing antibiotics and injections in the primary health care setting and general outpatient setting, use of drugs in the essential drugs list, generic prescribing, compliance with treatment guides and protocols. The amount of dispensing time and adequate labelling are indicators on dispensing practices that can be measured. Patients should know how to take their drugs. Inappropriate use of prescription drugs is also a measure in this survey. As mentioned earlier comparisons of the public and private sectors can be made. Comparisons can also be made with similar studies done in the past and with results from studies done in other countries. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 38 - Limitations of the Level II facility survey The survey has been designed to provide a picture of the national pharmaceutical situation in a country. The regions and facilities selected cumulatively represent the national situation. The sample sizes used are not statistically large enough to make inter-facility comparisons. For patient care indicators, for example, a minimum sample size of 100 would be necessary in order to make comparisons between facilities. This survey uses a sample size of 30. However, providing that the majority of the data are collected and the results are statistically different, comparisons between geographical regions can be made. Regional comparisons may be of interest where there is especially wide variation or contrasts, particularly with a group of related indicators. Regional comparisons should be done sparingly as not all geographical regions are represented. Over-emphasizing the five regions included in the study will detract from the study’s significance as a national survey. Indicator measure for Level II facility indicators Indicator values are computed and reported as percentages (%). They are also reported as measures of central tendency (mean and median). The two are equivalent if data are distributed normally. Mean as the average value is sensitive to outliers, weighted towards skewed value. It is the best summary of values that are normally distributed. Median the middle value is resistant to outliers, and is a good summary of any distribution. The median value (rather than the mean or average) should be used in reporting the indicator at the geographical region and at the national levels. A convincing way of presenting the dispersion of results is to use percentiles. Percentiles are points that divide all measurements in 100 equal parts. The median is the 50th percentile, i.e. the value below which 50% of measurements are positioned. As shown in Figure 2, the 25th and 75th percentiles are the boundaries of half of the values around the median, i.e. 50 % of the values are within the 25th (lower quartile) and 75th (upper quartile) percentiles. Presenting the 25th and 75th percentiles together with the median value provides a good summary of the overall spread of values and give a better summary of skewed data. Figure 2: Use of median and percentiles to describe the dispersion of measurement Data processing, analysis and reporting: What to do with results? - 39 - 0% 20% 40% 60% 80% 100% facility median value 0% 20% 40% 60% 80% 100% facility median value facility % p at ie nt s re ce iv in g an tib io tic s m ed ia n va lu e Performance standards for Level II facility indicators The target for indicators measuring the extent of adequate labelling, generic prescribing, the proportion of prescribed medicines dispensed, adherence to treatment guidelines and availability of key medicines is ideally 100%. However, internationally valid standards for other indicators, such as the average number of medicines per prescription, and the percentage use of antibiotics and injections are more complex and have not been established empirically. The optimal indicator values largely depend on disease patterns, policies and treatment guidelines, and therefore may vary from country to country and over time. Group norms can be used to some extent in target setting, with regions and facilities able to compare themselves to one another. However use of group norms must be carefully considered since the norm may be wrong, as in the figure below where the median value is almost 45%. Although not empirically tested, it is a possibility that antibiotics are over prescribed, if the value at primary health care level is > 30%. Such a result and its use have to be analysed carefully however. Moreover, in the figure below, in most facilities, more than 40% of patients are prescribed antibiotics. Figure 3: % patients receiving an antibiotic - distribution of facility results Written report A report should be prepared in order to disseminate the information obtained. This report is an important source of information and a basis for decisions on medicine policy and strategies. It should be persuasive and well prepared. The results must be discussed in a comprehensive and systematic manner, taking into account the objectives and strategies of the medicine policy. Clear recommendations should be included, based on the study findings. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 40 - After all the data have been processed and all the indicators calculated, the information should be analysed and presented in a report. Analysis can be done systematically by looking at each group of indicators and correlating the structures, processes and outcomes. Some examples are given in three key reference documents, Using Indicators to Measure Country Pharmaceutical Situations — Fact Book on WHO Level I and Level II Monitoring Indicators 14 and Indicators for Monitoring National Drug Policies15 as well as in How to Investigate Drug Use in Health Facilities”.16 The core indicators represent general measures of the current pharmaceutical situation. Reasons for improvements, deterioration or stagnation in implementation of the NMP can be deduced from the results. This will help to identify the inputs that have had a real impact and to focus attention, resources and efforts on areas requiring improvement. More detailed investigation and further analysis of a particular pharmaceutical component can be done if needed (many Level III approaches are available on the WHO Medicines website: http://www.who.int/medicinedocs/). The data and results in themselves do not suggest the implementation of specific interventions, but can be used as a good take off point to discuss problems in more detail so that a focused strategy can be identified. The more accurately data were collected the more useful they will be. This can also direct the responsible group in identifying follow-up activities that can be done. A report communicating the information obtained should be prepared by the monitoring unit. This report is an important source of information and a basis for decisions on medicine policy and strategies. It should be persuasive and well prepared. The results must be discussed in a comprehensive and systematic manner, taking into account the objectives and strategies of the medicine policy. The report should also be published. The written report should not be long. It should be objective and summarize information in tables and graphics. The results should be presented in a logical order with a logical link between findings, analysis and recommendations. Data included in the report should name the source of information, i.e. exit interview of patients, retrospective sampling of records, household interview, etc. While not necessary to include in the report, raw data should be kept for reference. See Table 3 for a suggested outline. 14 Harvard Medical School and Harvard Pilgrim Health. Using indicators to measure country pharmaceutical situations — Fact book on WHO Level I and II monitoring indicators. Geneva, World Health Organization, 2006. WHO/TCM/2006.2. 15 Brudon P, Rainhorn JD, Reich M. Indicators for monitoring national drug policies. Chapter 5, pp 46-47. Geneva, World Health Organization, 1993. WHO/EDM/PAR/99.3. 16 How to investigate drug use in health facilities, Selected drug use indicators. Chapter 4, pp 37-43. Geneva, World Health Organization, 1993. WHO/DAP/93.1. Data processing, analysis and reporting: What to do with results? - 41 - Table 3 Suggested report outline Section Notes 1 Table of contents • Include list of graphs and tables. 2 Acknowledgements 3 List of abbreviations • Sort alphabetically. 4 Executive summary • Summarize survey findings and recommendations. 5 Introduction Country profile • Basic geography (regions, provinces, districts). • Data on population. • Basic economic indicators (presented in a table). Structure of health and pharmaceutical systems (results from Level I can be used here) • Brief overview of the health situation including key health indicators (presented in a table) the structure of the health system and how the basic mechanisms of medicine distribution, financing, etc. operate. • Description of all pharmaceutical sectors: public, private, not- for-profit, NGO, informal. • Total size (or best estimate) of the pharmaceutical market, split by public and private sectors. • Inventory and overview of any previous pharmaceutical sector surveys. • Brief overview of the NMP (when approved, summary of content, achievements/constraints). 6 Study design and methodology • Brief summary including any country-specific considerations and a reference to this manual. Study purpose and indicators • Clear and realistic within the limitations of the design of the survey package. Scope and limitations of the data • General scope and limitations of the package and any additional limitations that have been introduced at the implementation level. Sampling procedure • Concise, to the point description of the process, including any exclusions from the sampling process that might limit the interpretation of the results (e.g. exclusion of a region in the sampling due to security reasons or difficulty of travel; exclusion of the not-for-profit sector). Problems experienced • In carrying out the survey especially those which might have affected the data collection and placed limitations on the interpretation of the results. Include general problems identified in carrying out the survey at the national level and reports from the data collectors (e.g. difficulties in finding sufficient patients for the exit interviews; access to private drug outlets refused). WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 42 - Table 3 Suggested report outline Section Notes 7 Results and analysis (any findings from Level I not already presented and results from Level II) • Policy • Access • Quality and safety • Rational use • Report results concisely and clearly • Use tables and graphs • Use headings and appropriate lead sentences to enable the reader to quickly find information, e.g. “62% of the 15 key medicines were found to be available in the public health facilities”. More description and discussion can then follow • The survey is primarily designed as a survey of the national situation and the report should reflect this. The report should not focus on reporting differences between the 5 geographic areas. Comparisons between the 5 areas should only be used where they are particularly interesting. Any geographic comparison should also include the national median • See Analysis above for examples of how to graphically present national results 8 Interpretation of the results • Make logical inferences based on the results of the survey and taking into account its limitations • Link appropriate access, quality and rational medicine use indicators 9 Conclusions and recommended interventions and/or next steps • Conclusions should be focused and clear • Recommendations and interventions should be realistic, limited and focus on those areas where greatest impact can be achieved. They should identify the problem to be addressed and the proposed activity to address the problem • Next steps should be as specific as possible listing: o Expected outcome o Timeline o Necessary (policy) decisions required and by whom o Who is responsible and who will carry out the recommendation o Necessary budget and other resources • Recommendations and next steps should reflect consultation with the Ministry of Health and input from the stakeholder workshop where possible 10 Annexes Completed level 1 questionnaire Summary forms • Facilities' names should not be reported as the survey was not designed for interpretation of results by facility • Original survey forms should be retained for archiving purposes. Interpretation of the coding should be retained with the forms Sampling and country-specific items • Lists of geographic areas, facilities (without coding), key medicines, definitions of adequate labelling, adequate knowledge, and other specifications to the survey forms made at the national level Data processing, analysis and reporting: What to do with results? - 43 - General notes • The report should be as concise as possible, presenting the information in a way that is understandable to a moderately informed reader. It should be easy to locate the methodology, key findings and recommendations quickly. • Tables and graphs should be used to avoid long complicated narrative descriptions. • General country, health and pharmaceutical data should be as up-to-date as possible. Generally data over 5 years old is not very informative unless part of a time-trend where more up-to-date data are also presented. Presentation and discussion of results and related issues The core indicators represent general measures of the current pharmaceutical situation. The results should be presented in meetings, seminars and other fora. Presentations should be comprehensive enough so that the audience will see how a problem in one component can affect another, but specific enough that the study’s findings can be used as a launching point for discussing the audience’s experiences and knowledge of problem areas and for identifying focused strategies to address problems. The monitoring and assessment report should be presented to all parties involved in formulating policies and implementing pharmaceutical strategies such as medicines inspectors, central medical stores, private drug outlets, and local industry. NGOs, professional organizations and academia should also know the results. The discussion should be non-judgmental, pointing out both the positive and negative findings and recommendations. The presentation focus must be suited to the type of audience. It is advisable to present the report separately to national, district/regional managers, and health facility levels. When presenting to a specific audience with a specific area of interest, provide a brief general summary of the overall results so the audience will know the entire picture and relate how a problem in one component can affect the other. The majority of the presentation, however, should be limited to issues that will be of interest and relevant to the specific audience. The presentation should: • Present the issues clearly. • Recognize what is well done and encourage further improvement in performance. • Focus on the part the audience is responsible for, can influence or is directly concerned with so the discussion, consensus and recommendations are directed to specific actions, activities and strategies. • Give enough time to discuss what did not work and potential changes that need to be implemented. Health workers should be presented with the findings and given specific feedback on how they are performing and how/where they can make improvements. They must be encouraged to participate in the discussion and give their comments. As soon as the completed forms are received, it would be useful if an individual one-page summary of data was organized and sent to each health facility. Also, if the team of data collectors is skilled enough, they could provide the individual rapid report before leaving the research area. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 44 - A meeting with key managers in the district, regional and administrative areas should also be held. Policy-makers at the national level can use the results to emphasize issues that can be addressed by a national strategy, to update current strategies, to define new ones, to reallocate resources or to adjust plans and targets. The discussion can also lead to decisions to put appropriate structures in place, to strengthen enforcement of policies and laws and to motivate/direct different levels and agencies of the health care system to take action or to improve performance. Wide dissemination of the results is important to create awareness at different levels. The results can also be discussed with other stakeholders–civil society, professional organizations and academia–that can help in revising or selecting appropriate pharmaceutical strategies and can contribute to effective implementation plans. Format of result presentation Reports always have greater impact when results are presented clearly. Tables and graphs should complement narrative description. On first examination, it might appear that there are limited ways of presenting national results for a single country graphically. However, there are a number of ways of presenting interesting and meaningful graphs. The Summary Form automatically generates several graphics. Graphics such as those in Figure 4 can be developed:. • Bar charts comparing the same indicator in different sectors, e.g. availability of the basket of medicines in public facilities, private drug outlets and warehouses. • Bar charts showing variability of facilities, e.g. variability of stockout duration in public facilities. • Comparison of a group of outcome indicators such as rational drug use, e.g. comparison of rational medicine use indicators: % of patients prescribed antibiotics, % of patients prescribed injections and % of medicines prescribed on the essential medicines list. • Presentation of the results of similar indicators in a table, e.g. prescribing of various medicines. Data processing, analysis and reporting: What to do with results? - 45 - Figure 4: Presenting national level data graphically Prescribing of drugs by diagnosis Diagnosis % prescribing of Diarrhoea Pneumonia Respiratory Tract Infections Oral rehydration salts (ORS) 76.1 Antibiotics 50.24 55.6 75.5 Antidiarrhoeals 4.76 Availability of the basket of drugs in different sectors 0 20 40 60 80 100 public health facility public sector w arehouse private drug outlet Variability of stockout duration in public facilities 0 4 8 12 16 20 24 0 1-30 31-60 61-180 >180 Duration of stock out (days) N um be r o f f ac ili tie s Comparison of rational drug use indicators 0.0 20.0 40.0 60.0 80.0 100.0 % patients receiving antibiotics % patients receiving injections % drugs on EML WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 46 - Annexes - 47 - Annexes Annex 1: Level I questionnaire .49 Annex 2: Level II indicators .71 Annex 2a: Technical descriptions of Level II indicators.71 Annex 2b: Issues, technical details, survey design and sampling for Level II indicators.85 Annex 3: Coordinator checklist .89 Annex 4: Example of budget allocation.95 Annex 5: Guide on how to use the training slides (level II indicators).97 Annex 6: Training slides for data collection .103 Annex 7: Level II Survey Forms 1—17 .121 Annexes in the CD Annex 1: Level I questionnaire Annex 3: Coordinator checklist Annex 4: Example of budget allocation Annex 6a: Slides for training data collectors Annex 6b: Slides for training survey coordinators Annex 6c: Slides for policy-makers Annex 7: Level II Survey Forms 1-17 Annex 7a: Regional List for prices of drugs (supplementary to SF2 & SF 11) Annex 8: Summary Forms Annex 9: Sample report Annex 10: Letter to facilities WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 48 - Annex 1 Level I Questionnaire - 49 - Annex 1: Level I Questionnaire Questionnaire on structures and processes of country pharmaceutical situations Country: Date (dd/mm/yyyy): Name of coordinator/principal respondent : E-mail address: Position : Postal address : Questions Responses Explanations 1. NATIONAL MEDICINES (DRUG) POLICY (NMP) Please consult the health ministry, medicines regulatory authority and/or medicine service in answering the questions in this section. 1.1. Is there a National Medicines Policy (NMP) document? If no, skip to 1.4. Yes No Don’t Know A national medicines (drug) policy document is a written expression of the government’s medium- to long- term goals and priorities for the pharmaceutical sector and the main strategies for attaining them. a) If yes, is it an official or draft document? Official Draft Don’t Know Mark “official” if the NMP document has been endorsed or officially adopted by the government otherwise mark “draft”. b) What year was it last updated? Year Indicate the year of last update whether the document is still in draft form or has been officially adopted. 1.2. Is there an NMP implementation plan that sets activities, responsibilities, budget and timeline? Yes No Don’t Know a) If yes, when was it last updated? Year 1.3. Is the NMP integrated into or included in the published/official national health policy/plan? Yes No Don’t Know The national medicines policy is considered to be integrated into the national health policy/plan if the pharmaceutical sector priorities and strategies are specified in the health plan. a) If yes, when was the national health policy/plan last updated? Year 1.4. Has a national assessment/indicator study been conducted? Yes No Don’t Know a) If yes, which topics have been studied and when was the most recent study covering each topic conducted: Overall pharmaceutical situation: Yes No DK Year Rational use/prescription audit: Yes No DK Year Access (i.e. prices, affordability and/or availability) to medicines: Yes No DK Year WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 50 - Questions Responses Explanations 1.5. Is there a code of conduct that applies to public officials and staff involved in pharmaceutical related activities or posts, such as persons working in pharmaceutical services, medicines regulation, procurement and supply of medicines and other pharmaceutical divisions of the health ministry? Yes No Don’t Know This means an officially adopted/promoted document aimed at ensuring accountability and appropriate conduct, including transparency and good governance, that applies to public/government officials and staff involved in pharmaceutical related activities or posts, such as those persons working in pharmaceutical services, medicines regulation, procurement and supply of medicines and other pharmaceutical divisions of the health ministry. 2. REGULATORY SYSTEM Please consult the medicines regulatory authority in answering the questions in this section. Specific information regarding medicines tested for quality control purposes and monitoring of adverse drug reactions may need to be obtained from the quality control laboratory or the responsible agency/department. Regulatory authority 2.1. Are there legal provisions establishing the powers and responsibilities of the medicines regulatory authority? Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which the medicines regulatory authority should be organized and operate. 2.2. Is there an existing formal medicines regulatory authority? Yes No Don’t Know This question is asking if there is a formal regulatory body with existing staff and a specific budget for conducting relevant medicines (drug) regulatory functions. Mark “no” if medicines regulatory functions, such as registration and licensing, are performed on an ad-hoc basis by an office, group or department that performs other pharmaceutical service functions, such as supply management and procurement. 2.3. What are the sources of funding for the medicines regulatory authority: Regular budget from the government: Yes No Don’t Know Fees from registration of medicines: Yes No Don’t Know Other: Yes No Don’t Know 2.4. Are there legal provisions requiring transparency and accountability and promoting a code of conduct in regulatory work? Yes No Don’t Know This question is asking whether there are legal provisions (or legislation) requiring the regulatory authority to: • Define its policies and procedures in writing and publish the written documentation • Give reasons for decisions to affected parties • Account for its conduct and actions to individuals or groups and ultimately to the public and • Follow a code of conduct in conducting its regulatory functions. Annex 1 Level I Questionnaire - 51 - Questions Responses Explanations 2.5. Is the medicines regulatory authority involved in regional/international harmonization initiatives? Yes No Don’t Know Regional/international harmonization initiatives include the International Conference on Harmonization, regional harmonization on registration of medicines, etc. Regulatory authorities are considered to be involved in these initiatives if they participate in meetings that discuss common regulatory matters with representatives of regulatory authorities from other countries, use a regional harmonized regulatory standard, etc. 2.6. Is there a medicines regulatory authority website providing publicly accessible information on any of the following: legislation, regulatory procedures, prescribing information (such as indications, contraindications, side effects, etc.), authorized companies, and/or approved medicines? Yes No Don’t Know Marketing authorization 2.7. Are there legal provisions for marketing authorization? Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which marketing authorization should be conducted. Marketing authorization is an official document issued by the medicines regulatory authority for the purpose of marketing or free distribution of a product after evaluation for safety, efficacy and quality and/or after registration of a product for marketing. 2.8. How many medicinal products have been approved to be marketed? (count total number of unique dosage forms and strengths) Number Tablets, capsules, injections, elixirs and suppositories should be counted in different strengths. For example, if Paracetamol (Brand X) 250 mg and 500 mg have been approved to be marketed, they count as two medicinal products because they are two unique strengths. Paracetamol (Brand Y) 250 mg and 500 mg are another two unique products. 2.9. Is a list of all registered products publicly accessible? Yes No Don’t Know Registered products are medicine products that have been evaluated for quality, safety and efficacy and thence authorized for marketing. In order to be publicly accessible, a list should be available on the web or to anyone contacting the responsible authority. 2.10. Is there a computerized registration system that facilitates retrieval of information on registered products? Yes No Don’t Know This refers to a systematic medicines registration process that makes the information on registered products more readily accessible and more easily updated by the responsible officials. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 52 - Questions Responses Explanations 2.11. Is the WHO Certification Scheme certificate required as part of the marketing authorization process? Yes No Don’t Know The WHO Certification Scheme on the Quality of Pharmaceutical Products Moving in International Commerce guarantees, through the issue of a WHO-type certificate by the certifying authority, the quality of pharmaceutical products entering international commerce. It is a simple administrative procedure that enables importing countries to obtain information on whether a product has been authorized to be placed on the market in the exporting country, and assure that the manufacturer has been found to comply with WHO standards of good manufacturing practices. 2.12. Is the INN used in the registration of medicines? Yes No Don’t Know An INN (international nonproprietary name) is a common, generic name selected by designated experts to identify a new pharmaceutical substance unambiguously. INNs are recommended for worldwide use, destined to be unique and public property (nonproprietary). 2.13. Is there a functional formal committee responsible for assessing applications for registration of pharmaceutical products? Yes No Don’t Know Mark “yes” if there is a committee composed of experts and specialists who review dossiers and applications for registration of medicines. Licensing 2.14. Are there legal provisions for licensing of the following: This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which manufacturers, wholesalers and distributors, and importers and exporters are subject to evaluation against a set of requirements and issued a permit to operate (licence) authorizing them to undertake specific activities. Manufacturers: Yes No Don’t Know Wholesalers or distributors: Yes No Don’t Know A wholesaler is a company that buys goods from a manufacturer or importer and sells them to retailers. The wholesaler may be an agent for one company only or deal with products from several companies. Manufacturers may also be wholesalers for their own products. In some countries, pharmacies may also have a wholesaler license. Distributors include wholesalers, retail pharmacies and medicine outlets. Importers or exporters of medicines: Yes No Don’t Know Annex 1 Level I Questionnaire - 53 - Questions Responses Explanations Regulatory inspection 2.15. Are there legal provisions to inspect premises and collect samples? Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which physical inspections of premises are conducted, including the legal procedures, guidelines and criteria for inspections. Physical inspection includes collection of samples and testing/calibrating equipment. 2.16. Are the following types of facilities inspected to check compliance with applicable requirements and are there written national guidelines/checklists for the inspection: Facilities inspected Written national guidelines/ checklists Manufacturers: Yes No DK Yes No DK Wholesalers/distributors: Yes No DK Yes No DK Importers/exporters: Yes No DK Yes No DK Retail distributors/pharmacies: Yes No DK Yes No DK A retail distributor is a company that sells goods to consumers, e.g. a pharmacy or other medicines outlet. Many low and middle income countries have at least two different types of shops in which medicines can be purchased: pharmacies with a registered pharmacist and medicines stores, chemists or medicines outlets with paramedical or lay staff. Control of narcotics and stupifiants 2.17. Are there legal provisions for the control of narcotics, psychotropic substances and precursors? Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which narcotics, psychotropic substances and precursors are controlled. 2.18. Is your country a signatory to the international conventions on the control of narcotics, psychotropic substances and precursors? Yes No Don’t Know Quality control 2.19. Is there a quality management system in place? Yes No Don’t Know This question is asking if there is an officially defined protocol for ensuring the quality of medicines, including testing of medicines to be registered, collection and testing of samples, reporting results, corrective actions to be taken when poor results are found and preventative measures to be taken to reduce future incidence of poor results. 2.20. Are medicine samples tested for the following regulatory purposes: Medicines registration: Yes No Don’t Know Post-marketing surveillance: Yes No Don’t Know Post-marketing surveillance is testing medicine samples to assess the quality of medicines that have already been licensed for public use. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 54 - Questions Responses Explanations 2.21. In which of the following laboratories are samples tested: This question is asking where medicine samples are tested. Mark “yes” if medicine samples are collected by or submitted to the listed laboratories for quality testing whether the testing is for purposes of registration, post-marketing surveillance and/or because the samples are suspected counterfeit/sub-standard medicines. Government quality control laboratory: Yes No Don’t Know Local academic institutions: Yes No Don’t Know Private laboratory: Yes No Don’t Know Mini laboratories (district, regional): Yes No Don’t Know Quality control laboratory in another country: Yes No Don’t Know 2.22. What is the total number of samples quality tested in 2006? Number This should include all samples tested whether in a quality assurance laboratory within the country or outside the country. 2.23. What is the total number of samples tested in 2006 that failed to meet quality standards? Number This should include all samples tested that failed to meet quality standards whether the testing was done in a quality assurance laboratory within the country or outside the country. 2.24. Are there regulatory procedures to ensure quality control of imported medicines? Yes No Don’t Know This question is asking if there are standard operating procedures for ensuring the quality of imported medicine, such as reviewing dossiers, product evaluation and testing of imported medicine products. This may include donated medicines. 2.25. Are there legal procedures for the recall and disposal of defective products? Yes No Don’t Know This question is asking if there are legal provisions (or legislation) and procedures under which defective products are recalled and disposed of. Defective products are those that are found to be of poor quality and/or inadequately labelled. Pharmacovigilance 2.26. Are adverse drug reactions (ADRs) monitored? Yes No Don’t Know Monitoring adverse drug reactions is the science and activities relating to the detection, assessment, understanding and prevention of adverse effects or any other medicines-related problems. a) If yes, at which of these health system levels are adverse drug reactions (ADRs) monitored: Local level: Yes No Don’t Know Mark “yes” next to local level if data are submitted to the hospital or community level for assessment and action. Regional level: Yes No Don’t Know Mark “yes” next to regional level if data are submitted to the regional or district level for assessment and action. Central level: Yes No Don’t Know Mark “yes” next to central level if data are submitted to the national level for assessment and action. Annex 1 Level I Questionnaire - 55 - Questions Responses Explanations 2.27. Does your country report ADRs to an international network or to the WHO Collaborating Centre for International Drug Monitoring? Yes No Don’t Know Mark “yes” if your country has reported cases of ADR to the WHO Collaborating Centre for International Drug Monitoring (Uppsala Monitoring Centre) or other international network at least once in the last five years. Counterfeit medicines 2.28. Are there any laws, regulations, programmes or procedures for detecting and combating counterfeit medicines? Yes No Don’t Know Programmes and procedures for detecting and combating counterfeit medicines may include active regular surveillance, official agreement with police authorities, involvement in national or international networks, etc. Counterfeit medicines are medicines, whether branded or generic, that are deliberately and fraudulently mislabelled with respect to identity and/or source or that have fake packaging. Counterfeit products may contain the correct ingredients or the wrong ingredients or may lack any, or sufficient, active ingredients. If your country has an alternative definition of counterfeit medicines, use that definition. 2.29. What sources of information are used to detect and combat counterfeit medicines: Reports from national authorities: Yes No Don’t Know “National authorities” refers to authorized groups providing data on a regular basis specifically for official use, such groups might include health facilities, professional organizations, etc. Reports from specific/ad hoc studies: Yes No Don’t Know An ad hoc study is a “one-off” study completed outside a regular system of collecting information. Reports from the pharmaceutical sector: Yes No Don’t Know “Reports from the pharmaceutical sector” primarily refers to reports from the pharmaceutical industry, whether brand or generic, local or multinational. Reports from civil society/NGOs: Yes No Don’t Know “Reports from civil society/NGOs” refers to reports produced by civil society for its own purposes rather than specifically for official use. Civil society/NGOs (nongovernmental organizations) are non-profit organizations, networks and voluntary associations including charities, community groups, faith-based organizations, professional associations, academia and trade unions. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 56 - Questions Responses Explanations Dispensing and prescribing 2.30. Are there legal provisions for the following: This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which prescribers and the practice of pharmacy are licensed. Licensing is a system that subjects all persons to evaluation against a set of requirements before they may be authorized to prescribe medicines/practice pharmacy. It may include issuing an official permit and granting authorization to prescribe medicines/practice pharmacy by either the governing authority or the body regulating the exercise of the profession. Licensing and practice of prescribers: Yes No Don’t Know Licensing and practice of pharmacy: Yes No Don’t Know 2.31. Is prescribing by generic name obligatory in the: A generic name (international non- proprietary name - INN) is a non- proprietary or approved name rather than a proprietary or brand name under which a generic medicine is marketed. If prescribing by generic name is obligatory then prescribers are required to prescribe by generic name. Public sector: Yes No Don’t Know Private sector: Yes No Don’t Know 2.32. Is generic substitution permitted at: Generic substitution is the practice of substituting a product, whether marketed under a trade name or generic name, by an equivalent product, usually a cheaper one, containing the same active ingredient at the dispensing level. Mark “yes” if either generic substitution is required or if the dispenser is allowed to make a generic substitution in at least some instances. Public pharmacies: Yes No Don’t Know Private pharmacies: Yes No Don’t Know 2.33. Are there incentives to dispense generic medicines at: Incentives may include dispensing fees or mark-ups which provide financial incentive for dispensers to dispense lower-priced generic medicines. Public pharmacies: Yes No Don’t Know Private pharmacies: Yes No Don’t Know Annex 1 Level I Questionnaire - 57 - Questions Responses Explanations Promotion and advertising 2.34. Are there provisions in the medicines legislation/regulations covering promotion and/or advertising of medicines? Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the conditions under which the promotion and/or advertisement of medicines may be conducted. Promotion and advertisement are activities that provide health workers and consumers with information about medicine products, particularly with the intent of encouraging health workers and consumers to use a particular product. 2.35. Who is responsible for regulating promotion and/or advertisement of medicines? Industry (self-regulation) only Government or national regulatory authority only Co-regulation Don’t Know Mark “industry (self-regulation) only” if the pharmaceutical industry bears responsibility for regulating itself to ensure companies’ practices of promoting and/or advertising medicines are in line with appropriate standards, whether the standards are explicitly defined or not, and government regulation is minimal. For example, in some countries companies police each other in terms of compliance to industry set guidelines. Mark “government or national authority only” if the government or national regulatory authority is responsible for enforcing written legal restrictions on the promotion and/or advertising of medicines and company (self- regulation) is minimal. Mark “co-regulation” if both industry (self-regulation) and the government or national regulatory authority contribute to regulating the promotion and/or advertisement of medicines. a) If regulated by government, do regulations include any of the following: Pre-approval for advertisements and/or promotional materials: Yes No Don’t Know Pre-approval for advertisements and/or promotional materials means official approval of advertisements and/or promotional material must be obtained before they can be used. Prohibition on advertising prescription medicines to the public: Yes No Don’t Know Guidelines on advertising of non- prescription medicines: Yes No Don’t Know Guidelines on advertising of non- prescription medicines means there may be some restrictions on advertising non- prescription medicines to the public, such as in promoting therapeutic claims. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 58 - Questions Responses Explanations 2.36. Are civil society/NGOs included in surveillance of promotion and/or advertisement of medicines? Yes No Don’t Know Mark “yes” if civil society/NGOs actively monitor and report unethical/illegal practices regarding the promotion and/or advertisement of medicines and that this involvement is recognized/requested by the medicines regulatory authority, e.g. though a written memorandum of understanding. Normal consumer/public complaints that are not part of a government request for or recognition of civil society/NGO involvement should not be counted in this question. Civil society/NGOs (nongovernmental organizations) are non-profit organizations, networks and voluntary associations including charities, community groups, faith-based organizations, professional associations, academia and trade unions. 3. MEDICINES SUPPLY SYSTEM Please consult the agency/department responsible for the procurement and supply of medicines in answering the questions in this section. 3.1. Is public sector procurement pooled at the national level (i.e. there is centralized procurement for the regions/provinces)? Yes No Don’t Know Mark “yes” if public sector procurement is centralized and medicines are procured for the entire public sector by a national procurement body even if in some instances, such as cases of stock outages, public sector facilities procure medicines through other means. 3.2. Who is responsible for public sector medicines procurement and distribution: Procurement Distribution Ministry of Health: Yes No DK Yes No DK Nongovernmental organization (NGO): Yes No DK Yes No DK Mark “yes” for nongovernmental organization (NGO) if government funds or foreign contributions are allocated to NGOs to procure or distribute medicines for the public sector. Nongovernmental organizations (NGOs) are nongovernmental, non- profit organizations, networks and voluntary associations including charities, community groups, faith- based organizations, professional associations, academia and trade unions. Private institution contracted by the government: Yes No DK Yes No DK Mark “yes” for private institution contracted by the government if the government contracts or makes an agreement with a private entity to procure or distribute medicines for the public sector, e.g. if an agreement is made with a private company to distribute medical items and supplies to public sector district warehouses and health facilities. Individual health institutions: Yes No DK Yes No DK Annex 1 Level I Questionnaire - 59 - Questions Responses Explanations 3.3. What type of tender process is used for public sector procurement and what is the percentage of the total cost for each: Percentage of total cost Competitive tender is a procedure for procuring medicines which puts a number of suppliers into competition. Purchasing is done on the basis of quotations submitted by suppliers in response to a public notice. National competitive tender: Yes No DK % National competitive tender is open to all or a limited number of local suppliers only. International competitive tender: Yes No DK % International competitive tender is open to all or a limited number of local and international suppliers though sometimes conditions give preference to either local or international suppliers. Negotiation/direct purchasing: Yes No DK % In negotiation/direct purchasing the buyer approaches one or a small number of suppliers and either buys at the quoted prices or bargains for a specific service arrangement. 3.4. Is there a tender board/committee overseeing public sector procurement? Yes No Don’t Know This question is asking if there is a board or committee responsible for overseeing public sector procurement which is composed of members who are not staff/officials of the government procurement agency. A tender is a procedure for procuring medicines by seeking quotations from suppliers in response to a public notice. a) If yes, are the key functions of the procurement office and those of the tender committee clearly separated? Yes No Don’t Know 3.5. Does public sector medicines procurement use the WHO Prequalification system? Yes No Don’t Know The WHO Prequalification Programme is a service provided by the World Health Organization (WHO) to facilitate access to medicines that meet unified standards of quality, safety and efficacy for HIV/AIDS, malaria and tuberculosis. The growing list of products (i.e. medicines) and manufacturers that have been found to meet the set requirements can be used by anyone bulk purchasing medicines, including countries. 3.6. Is public sector procurement limited to medicines on the Essential Medicines List (EML)? Yes No Don’t Know An Essential Medicines List (EML) is a government-approved selective list of medicines or national reimbursement list. Essential medicines are those that satisfy the priority health care needs of the population. They are selected with due regard to disease prevalence, evidence on efficacy and safety, and comparative cost-effectiveness. Essential medicines are intended to be available within the context of functioning health systems at all times in adequate amounts, in the appropriate dosage forms, with assured quality, and at a price the individual and the community can afford. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 60 - Questions Responses Explanations a) If yes, are there provisions for purchasing medicines not on the Essential Medicines List? Yes No Don’t Know This question is asking if public sector procurement is in general limited to medicines on the EML, are there guidelines to procure non-EML medicines (e.g. emergency medicines, high cost medicines, medicines for special conditions etc). 3.7. Did your country participate in a pooled procurement scheme with at least one other country for at least one of the last two procurement cycles? Yes No Don’t Know 4. MEDICINES FINANCING Please consult the budget/ finance division of the health ministry and/or the pharmaceutical supply group in answering the questions in this section. The hospital/health facility service and/or the national social and insurance services may also need to be consulted. 4.1. What is the total public or government expenditure for medicines in US$ for the most recent year for which data are available? US$ Year This question is asking for the total amount the government has spent on medicines, including government allotment, health ministry expenditure, donor contributions channeled through the government, etc. 4.2. Is there a national policy to provide at least some medicines free of charge (i.e. patients do not pay out-of-pocket for medicines) at public primary care facilities? Yes No Don’t Know If medicines are provided for free but patients must pay service fees, mark “yes” here. If some facilities provide medicines for free but there is not a consistent national policy that applies to all primary public health facilities, mark “no” here. If there is a national policy to provide medicines for free at primary public health facilities, but facilities are not required to abide by the policy and not all facilities provide medicines for free, mark “no” here. a) If yes, which of the following are free at public primary care All medicines: Yes No Don’t Know Malaria medicines: Yes No Don’t Know Tuberculosis medicines: Yes No Don’t Know Sexually transmitted diseases medicines: Yes No Don’t Know HIV/AIDS-related medicines: Yes No Don’t Know At least one vaccine: Yes No Don’t Know b) Which of the following types of patients receive medicines for free: Patients who cannot afford them: Yes No Don’t Know Children under 5 years of age: Yes No Don’t Know Older children: Yes No Don’t Know Mark “yes” for “older children” if children over 5 years of age receive medicines for free, regardless of the age limit, for example mark “yes” if children under 12 receive medicines for free. Pregnant women: Yes No Don’t Know Elderly persons: Yes No Don’t Know Annex 1 Level I Questionnaire - 61 - Questions Responses Explanations 4.3. Which fees are commonly charged in public primary care facilities: Registration/consultation fees: Yes No Don’t Know Registration and consultation fees are fees patients must pay for seeing a health professional for a health check-up and/or diagnosis regardless of whether or not medicines are prescribed. Dispensing fees: Yes No Don’t Know A dispensing fee is a fixed fee that pharmacies are allowed to charge per prescribed item or per prescription instead of or in addition to a percentage mark-up. The dispensing fee is paid to the dispenser and is in addition to the cost of the medicine. Both the dispensing fee and the cost of the medicine may be paid in part or whole by the patient, insurer or government. Flat fees for medicines: Yes No Don’t Know Mark “yes” for “flat fees” if either a flat fee for medicines or a flat fee per medicine item is commonly charged. A flat fee for medicines is a fee which remains the same irrespective of the number of medicines or the quantity of each medicine dispensed. Thus, for example, a patient receiving 3 medicines would pay the same as one receiving 1 medicine. Also a patient receiving 20 tablets of one medicine would pay the same as a patient receiving 100 tablets each of 2 medicines. A fee per drug item is a fee where the patient pays one set fee per each medicine irrespective of the number of units (tablets) of that medicine dispensed. Thus, for example, a patient receiving one medicine would pay US$1 and a patient receiving 2 medicines would pay US$2 and a patient receiving 3 medicines would pay US$3 and so on. However, a patient receiving 10 tablets of one medicine would pay the same as a patient receiving 100 tablets of one medicine. Flat rate co-payments for medicines: Yes No Don’t Know A flat rate co-payment is a fixed amount that a patient must pay either per medicine or per prescription to cover part of the cost of medicines, the other part being paid by an insurer or government. Percentage co-payments for medicines: Yes No Don’t Know A percentage co-payment is a fixed percentage of the cost of prescribed medicines that a patient must pay to cover part of the cost of medicines, the other part being paid by an insurer or government. The amount a patient pays will depend on the medicine and the number of units of that medicine prescribed. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 62 - Questions Responses Explanations 4.4. Is revenue from fees or the sale of medicines used to pay the salaries or supplement the income of public health personnel in the same facility? Always Frequently Occasionally Never DK Mark “yes” if any percentage of collected fees or medicines sales is used to pay salaries, expenses and/or in any way supplement the income of public health personnel in the same facility. Public sector Private sector4.5. Do prescribers dispense medicines? Always Frequently Occasionally Never DK Always Frequently Occasionally Never DK In answering this question, mark the degree of frequency doctors or other authorized prescribers dispense medicines in the public and private sectors irrespective of laws permitting or disallowing authorized prescribers to dispense medicines. 4.6. What proportion of the population has health insurance? All None Some DK All None Some DK Health insurance is any prepayment scheme for health care costs additional to but excluding subsidies funded through the health ministry budget. The purpose of questions 4.6 and 4.7 are to identify how much protection the population has against exposure to the cost of medicines at the time people are sick. This includes: Prepaid financing and Public funding through the (prepaid) health ministry budget. 4.7. Are medicines covered by health insurance? All None Some DK All None Some DK Public sector Private sector NGO4.8. Is there a policy covering medicine prices that applies to the public sector, the private sector, or non- governmental organizations? Yes No DK Yes No DK Yes No DK In some countries, NGOs, such as faith- based missions, provide non-profit or not-for-profit health services. The third column should be completed by ticking any policies applicable to this sector. Nongovernmental organizations (NGO) are nongovernmental non-profit organizations, networks and voluntary associations including charities, community groups, faith-based organizations, professional associations, academia and trade unions. a) If yes, which of the following policies covering medicine prices apply: Maximum wholesale mark-up: Yes No DK Yes No DK Yes No DK A wholesale mark-up is a certain percentage added to a purchasing price to cover the cost and profit of the wholesaler. Maximum retail mark-up: Yes No DK Yes No DK Yes No DK A retail mark-up is a certain percentage added to a purchasing price to cover the cost and profit of the retailer. Annex 1 Level I Questionnaire - 63 - Questions Responses Explanations Duty on imported raw pharmaceutical materials: Yes No DK Yes No DK Yes No DK A duty/tax on imported raw pharmaceutical materials is a fee assessed by customs or other responsible national authority on imported starting materials, reagents, intermediates, process aids, and solvents intended for use in the production of intermediates or active pharmaceutical ingredients. Duty on imported finished pharmaceutical products: Yes No DK Yes No DK Yes No DK A duty/tax on imported finished pharmaceutical products is a fee assessed by customs or other responsible national authority on medicinal products that require no further processing and are already in their final containers. 4.9. Is a national medicine prices monitoring system for retail/patient prices in place? Yes No DK Yes No DK Yes No DK A national medicine prices monitoring system for retail/patient prices is any means of regularly tracking and comparing over time retail/patient medicine prices in the public, private and/or NGO sectors. 4.10. Are there regulations mandating retail/patient medicine price information to be made publicly accessible? Yes No DK Yes No DK Yes No DK In order for retail/patient medicine price information to be considered publicly accessible, one or more of the following or similar measures should be taken: prices should be available on the web or to anyone contacting the responsible authority, prices should be periodically published in national newspapers or official publications, prices should be posted in health facilities/pharmacies, etc. 4.11. Are there official written guidelines on medicine donations that provide rules and regulations for donors and provide guidance to the public, private and/or NGO sectors on accepting and handling donated medicines? Yes No DK Yes No DK Yes No DK Countries may have differing definitions for medicine donations which may include not only products but also monetary gifts earmarked for a particular product from a named source (e.g. manufacturer, organization or other country). 5. PRODUCTION AND TRADE Please consult the medicines regulatory authority, the patent office and/or the trade ministry in answering the questions in this section. 5.1. What is the medicines production capability in the country: Mark “yes” next to each of the types of medicine production that currently occurs in your country. Research and development of new active substances: Yes No Don’t Know Production of pharmaceutical starting materials: Yes No Don’t Know Formulation from pharmaceutical starting materials: Yes No Don’t Know Repackaging of finished dosage forms: Yes No Don’t Know WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 64 - Questions Responses Explanations 5.2. Are patents granted on pharmaceutical products by the national patent office? Yes No Don’t Know A patent is an exclusive right awarded to an inventor to prevent others from making, selling, distributing, importing or using the invention, without license of authorization, for a fixed period of time, such as 20 years. This may be granted by the national patent office or, for smaller countries, a regional patent office representing several countries may grant the patent. 5.3. If your country is a member of the World Trade Organization (WTO), has national legislation been modified to implement the TRIPS Agreement? Yes No Don’t Know Country not a member of WTO Countries who are members of the World Trade Organization (WTO) should adapt national legislation to include the minimum standards of protecting and enforcing intellectual property rights, including those for patents, required under the Agreement on Trade-Related Aspects of Intellectual Property Rights (TRIPS Agreement). a) If a WTO member, has your country used the following available transitional periods to implement the TRIPS Agreement: TRIPS provides transitional periods during which countries are required to bring their national legislation and practices into conformity with its provisions. Article 65: Yes No Don’t Know Under Article 65 of the TRIPS Agreement: Transitional Arrangements: No Member shall be obliged to apply the provisions of the Agreement before the expiry of a general period of one year following the date (WIPO note 1 January 1995) of entry into force of the WTO Agreement. A developing country Member is entitled to delay for a further period of four years from the date of application, as defined in paragraph 1, of the provisions of this Agreement other than Articles 3, 4 and 5. Article 66: Yes No Don’t Know Country not an LDC Under Article 66 of the TRIPS Agreement: Least-Developed Country Members: In view of the special needs and requirements of least-developed country Members, their economic, financial and administrative constraints, and their need for flexibility to create a viable technological base, such Members shall not be required to apply the provisions of this Agreement, other than Articles 3, 4 and 5, for a period of 10 years from the date of application as defined under paragraph 1 of Article 65. The Council for TRIPS shall, upon duly motivated request by a least-developed country Member, accord extensions of this period. Doha declaration (Article 7): Yes No Don’t Know Under the Doha Declaration, the transitional period is extended until 2016. Annex 1 Level I Questionnaire - 65 - Questions Responses Explanations 5.4. Which of the following TRIPS flexibilities have been incorporated into national legislation as applies to pharmaceuticals: CBD = Currently being discussed Compulsory licensing provisions: Yes No CBD DK Compulsory licensing is when the judicial or administrative authority is allowed by law to grant a license, without permission from the holder, on various grounds of general interest (absence of working, public health, economic development, and national defense). Government use: Yes No CBD DK Government use is when the administrative authority is allowed to grant a license, without permission from the holder, for public health needs. Parallel importing provisions: Yes No CBD DK Parallel importation is importation, without the consent of the patent-holder, of a patented product marketed in another country either by the patent- holder or with the patent-holder’s consent. Parallel importation enables promotion of competition for the patented product by allowing importation of equivalent patented products marketed at lower prices in other countries. The Bolar exception: Yes No CBD DK The Bolar exception is an early working provision whereby generic pharmaceutical manufacturers are allowed to use patented inventions for the purpose of obtaining marketing approval prior to patent expiration. 6. RATIONAL USE OF MEDICINES Please consult the health ministry (hospital division), professional bodies and/or the education ministry in answering the questions in this section. 6.1. Is there a national Essential Medicines List (EML)? Yes No Don’t Know A national Essential Medicines List is a government-approved selective list of medicines or national reimbursement list from which most prescriptions should be made. Essential medicines are those that satisfy the priority health care needs of the population. They are selected with due regard to disease prevalence, evidence on efficacy and safety, and comparative cost-effectiveness. a) If yes, how many unique medicine formulations does the national EML contain? Number: Count similar formulations registered or approved as different products as one formulation, for example Brand X 500 mg Paracetamol tablets and Brand Y 500 mg Paracetamol tablets are counted as one formulation whereas Brand X 250 mg Paracetamol tablets and Brand X 500 mg Paracetamol tablets are counted as two formulations. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 66 - Questions Responses Explanations b) How many paediatric formulations are included in the: National EML: Number: Separate Paediatric EML: Number: No separate paediatric EML Some countries have a separate list of essential medicines for paediatrics. c) When was the national EML last updated? Year: Is the national EML being used in the following: Mark “yes” if the EML is currently being used. Public sector procurement: Yes No Don’t Know Public insurance reimbursement: Yes No Don’t Know Private insurance reimbursement: Yes No Don’t Know d) Is there a committee responsible for the selection of products on the national EML? Yes No Don’t Know This refers to a formally recognized committee with members of different expertise and from different agencies/organizations. National STG Hospital- level STG Primary care STG 6.2. Are the following types of standard treatment guidelines (STGs) produced by the health ministry for major conditions? Yes No DK Yes No DK Yes No DK Mark “yes” if the health ministry or similar national authority produces a collection of treatment guidelines covering prevalent/common disease conditions in the country for use at the national, hospital or primary care levels. If treatment guidelines are produced separately for each disease/condition or organ system, mark “no”. a) If yes, when were the STGs last updated? Year Year Year 6.3. Are there standard treatment guidelines for key paediatric illnesses? Yes No Don’t Know Mark “yes” if the health ministry or similar national authority produces a collection of treatment guidelines covering prevalent/common paediatric disease conditions, whether it is a stand alone document or a chapter/section of the national, hospital or primary care STG. Mark “no” if paediatric treatment guidelines are produced separately for each disease or condition. 6.4. Is there a National Medicines Formulary Manual? Yes No Don’t Know A formulary is a manual containing clinically oriented summaries of pharmacological information about selected medicines. The manual may also include administrative and regulatory information pertaining to the prescribing and dispensing of medicines. A national formulary generally concentrates on available and affordable medicines that are relevant to the treatment of diseases in a particular country. a) If yes, when was it last published/reviewed? Year: b) Does it cover only medicines on the national EML? Yes No Don’t Know Annex 1 Level I Questionnaire - 67 - Questions Responses Explanations 6.5. Are the following prescribing issues part of the basic curricula in most health training institutions for: Essential Medicines List (EML) Standard Treatment Guidelines (STGs) Problem- based pharmaco -therapy Rational presc- ribing This question is asking whether most or major government-recognized institutions training health professionals require students to be instructed in the following areas: An Essential Medicines List is a government-approved selective list of medicines or national reimbursement list from which most prescriptions should be made. Standard Treatment Guidelines are developed by the health ministry or similar national authority. They are a collection of treatment guidelines covering prevalent/common disease conditions in the country for use at the national, hospital or primary care levels. Problem-based pharmacotherapy is a problem-based practical approach to teaching prescribing. Rational prescribing, or appropriate prescribing, assures that patients receive medications appropriate to their clinical needs, in doses that meet their own individual requirements, for an adequate period of time, and at the lowest cost to them and their community. Doctors: Yes No DK Yes No DK Yes No DK Yes No DK Nurses: Yes No DK Yes No DK Yes No DK Yes No DK Pharmacists: Yes No DK Yes No DK Yes No DK Yes No DK Pharmacy assistants: Yes No DK Yes No DK Yes No DK Yes No DK Paramedical staff: Yes No DK Yes No DK Yes No DK Yes No DK 6.6. Are there obligatory, non- commercially funded continuing education programmes that include use of medicines for: A continuing education programme is a programme based on regular workshops, seminars and/or in-service training which provides all prescribers and dispensers with refresher courses on medicine issues. Doctors: Yes No Don’t Know Nurses/midwives/paramedical staff: Yes No Don’t Know Pharmacists: Yes No Don’t Know Pharmacy aides/assistants: Yes No Don’t Know WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 68 - Questions Responses Explanations 6.7. Is there a public or independently funded, nationally accessible (e.g. by phone) medicines information centre or service that provides information on demand to: A medicines information centre or service is an organization within or outside the health ministry which collects and provides objective information on medicines to health personnel and the public. Objective information should be understood as information produced by independent scientific sources without any support from the pharmaceutical industry or private firms involved in the medicines sector. The medicines information centre/service may perform additional tasks. Prescribers: Yes No Don’t Know Dispensers: Yes No Don’t Know Consumers: Yes No Don’t Know 6.8. Have there been any public education campaigns about rational medicines use in the previous two years conducted by the health ministry, a nongovernmental organization, or academia on the following topics: Mark “yes” if there has been any effort to inform the public (e.g. through radio programmes, newspaper articles/ announcements, public gatherings, etc.) about any of the listed rational medicines use issues. The focus of these efforts may include the importance of patients receiving medications appropriate to their clinical needs, the dangers of over-prescribing, the dangers of inappropriate dosing, etc. Use of antibiotics: Yes No Don’t Know Use of injections: Yes No Don’t Know Other rational medicine use topics/issues: Yes No Don’t Know 6.9. How often do the following personnel prescribe prescription-only medicines at the primary health care level in the public sector: Doctors: Always Frequently Occasionally Never DK Nurses/midwives/paramedical staff: Always Frequently Occasionally Never DK Pharmacists/pharmacy aides/assistants: Always Frequently Occasionally Never DK Personnel with <1 month formal health training: Always Frequently Occasionally Never DK Formal health training includes any government-recognized training programme on health issues. 6.10. Are there a national programme and/or multidisciplinary body, involving government, civil society and professional bodies, which monitor and promote the rational use of medicine? Yes No Don’t Know Annex 1 Level I Questionnaire - 69 - Questions Responses Explanations 6.11. Is there a mandatory requirement to organize/develop drug and therapeutics committees? Yes No Don’t Know Mark “yes” if there is a mandatory requirement for facilities/areas to organize/develop drug and therapeutics committees and positive incentives exist for those that have drug and therapeutics committees and disincentives exist for those that do not. If there is a requirement, but no incentives or disincentives mark “no” here. A drugs and therapeutics committee is a group of people established and officially approved by the health ministry and/or health facility management that promotes the safe and effective use of medicines in the area or facility under its jurisdiction. 6.12. What proportions of hospitals and regions have drug and therapeutics committees: In answering this question, please use the following approximations: All = more than 90% Most = 61-90% Half = 40-60% Few = 10-39% None = less than 10% Referral hospitals: All Most Half Few None DK Referral hospitals are tertiary care centres which include major hospitals offering a full range of services, including specialty units, and specialty hospitals dedicated to specific types of patients, e.g. paediatrics, or specific range of conditions, e.g. oncology. While referral hospitals (tertiary care centres) may provide secondary or even primary care, their main function is to provide a referral service for secondary care centres (general hospitals) in all the main subspecialties. General hospitals: All Most Half Few None DK General hospitals are secondary care centres, which usually include acute medical, surgical, paediatric and obstetric services but do not include specialist services such as oncology, cardiac or neurological surgery etc. While general hospitals may have 1-2 units providing sub-specialist tertiary care, they do not have a full range of such services, and their main function is to provide a referral service for the primary health care centres and a direct service to the population under their jurisdiction and they do not generally have specialist units. Regions/provinces: All Most Half Few None DK For regions/provinces, mark the proportion of regions/provinces with a drugs and therapeutics committee that promotes the safe and effective use of medicines in all facilities (hospital and primary health care) in the region/province. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 70 - Questions Responses Explanations 6.13. Is there a national strategy to contain antimicrobial resistance? Yes No Don’t Know A national strategy is a written plan supported/spearheaded by a national/central authority and officially endorsed by the government. 6.14. Is there a national reference laboratory to coordinate epidemiological surveillance of antimicrobial resistance? Yes No Don’t Know Mark “yes” if there is a national laboratory to which health facilities submit reports on antimicrobial resistance that monitors and reports on antimicrobial resistance in the country and coordinates and reports on any response. If there is no national laboratory, but reports on antimicrobial resistance are submitted to a regional laboratory, also mark “yes”. 6.15. Is there a funded national inter- sectoral task force to coordinate the promotion of appropriate use of antimicrobials and prevention of spread of infection? Yes No Don’t Know This question is asking if there is a formal committee or other body of representatives of public, private and NGO or mission sectors that oversees efforts to improve appropriate use of antimicrobials and prevent spread of infection. This may include developing and evaluating codes of practices at health facilities and/or education programmes for health professionals and/or consumers. 6.16. How frequently are the following types of medicines sold over the counter without any prescription: This question is asking how often antibiotics and injections which require a prescription to be dispensed are sold without a prescription, regardless of laws prohibiting such practice. Antibiotics: Always Never Frequently Occasionally DK Injections: Always Never Frequently Occasionally DK Annex 2 Level II Indicators - 71 - Annex 2: Level II Indicators Annex 2a: Technical descriptions of Level II indicators Access 1. Availability of key medicines in public health facility dispensaries, private drug outlets and warehouses supplying the public sector (Survey Forms 1, 10 and 15) Purpose To measure current availability of key medicines to treat common health problems in public health facility dispensaries, private drug outlets and warehouses. Essential medicines to treat common diseases should be available in all these facilities, especially in public sector facilities providing health services for the poor. Physical availability is a basic measure of access to essential medicines. See pages 29—30 for guidance on selecting key medicines for this study. Prerequisites List of 15 key essential medicines to treat the most common health problems. Source of data Survey of 30 public health facility dispensaries, 30 private drug outlets, and 5 warehouses. Process Go through the shelves and identify which of the listed essential medicines are available at the facility at the time of the survey. Only count in stock medicines in the facility at the time of the visit regardless of whether or not they are available at an offsite storage facility. Calculation % of key medicines available in a facility = % in stock = number of key medicines available in a facility ÷ number of key medicines reviewed x 100 National average = sum of % of key medicines available for all facilities ÷ number of facilities sampled. 2. % of prescribed medicines dispensed or administered to patients at public health facility dispensaries (Survey Form 6) Purpose To measure the degree to which facilities are able to provide needed medicines. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility dispensaries. See section on sampling. Process Interview patients leaving the dispensing area or leaving the facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Count how many chemical entities were prescribed and dispensed. Calculation % of medicines dispensed = number of medicine dispensed ÷ number of medicines prescribed x 100. National average = sum of % of medicine dispensed for all public health facility dispensaries ÷ number of public health facility dispensaries sampled. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 72 - 3. Stockout duration at public health facility dispensaries and warehouses supplying the public sector (Survey Forms 4 and 16) Purpose To measure availability over the past 12 months of key medicines to treat common health problems. An adequate logistic system ensures that essential medicines remain in stock at all times. Prerequisite List of 15 key essential medicines to treat the most common health problems Adequate medicine stock recording system (able to access stock records for the previous 12 months). Source of data Survey of 30 public health facility dispensaries and 5 warehouses. Process Using existing data on stock cards, record the number of days each key medicine is not available (when the stock is zero). A medicine is in stock if any dose of any equivalent product is available in either branded or generic form. Stockout should be measured based upon the chemical entity rather than any one brand or dosage form. The review period should, as far as possible, cover one year and never less than six months. Calculation Equivalent number of days per year each medicine is out of stock = number of days each medicine is out of stock x 365 divided by number of days covered by the review for that medicine. Average stockout duration in each facility = average number of stockout days = sum of the equivalent number of days per year all medicines are out of stock divided by number of key medicines reviewed. National average = sum of average stockout duration in all facilities number of facilities sampled. Limitations Results on this indicator will be influenced by the quality of stock records. 4. % of adequate record keeping at public health facility dispensaries and warehouses supplying the public sector (Survey Forms 4 and 16) Purpose To determine the extent to which stock records are maintained. The presence of adequately maintained and accurate stock records contributes to proper management, estimation of needs and the reorder of medicines. Prerequisite List of 15 key essential medicines to treat the most common health problems. Source of data Survey of 30 public health facility dispensaries and 5 warehouses. Process For each of the key medicines, examine the data on the stock card and identify those medicines for which there are records of quantities of receipt, issue and stock-on-hand for at least 6 months in the previous 12 months. Calculation % adequate stock records = number of incidences where there are adequate records for at least 6 months ÷ number of key medicines reviewed National average = sum of % adequate stock records for all facilities divided by number of facilities sampled. Limitations This indicator does not permit verification of whether the records are accurate. Annex 2 Level II Indicators - 73 - 5. Affordability of treatment for adults and children under 5 years of age at public health facility dispensaries and private drug outlets (Survey Forms 3 and 12) Purpose To measure affordability of basic pharmaceutical treatment as an indicator of access to essential medicines. In most developing countries, a majority of the population pays for treatment out-of-pocket. Affordability is expressed as the ratio of the cost of treating moderate pneumonia or another condition to a standard unit of measure. For this survey, the lowest daily government salary is used. Countries may also identify an optional second unit of measure (e.g. poverty line, basket of food, etc.). Prerequisite Lowest government daily wage or the lowest daily wage of any appropriate majority group. Standard treatment for moderate pneumonia and another condition (where no hospitalization is required) for adults and children. Source of data Survey of 30 public health facility dispensaries and 30 private drug outlets. Process Identify the treatment of choice (medicine/generic name, dosage form and strength) for adults and children based on standard treatment guidelines. Identify the number of units needed to complete the treatment. Identify the unit price (the price charged to patients) for the lowest priced brand or generic equivalent medicine in each pharmacy, include any applicable charges that patients would pay to receive treatment such as dispensing fees and/or syringes. If there are flat charges paid for each medicine given to patients, then this amount should be recorded as the price of the medicine. Indicate “0” if medicines are given free of charge. Calculation Total cost of treatment = number of units needed to complete the treatment x unit price. Equivalent number of day’s wages = total cost of treatment ÷ lowest daily government salary. Ratio of cost of treatment and optional standard unit of measure = total cost of treatment ÷ optional standard unit of measure. National average = sum of equivalent number of day’s wages at all facilities ÷ number of facilities sampled. National average = sum of ratio of cost of treatment and optional standard unit of measure at all facilities divided by number of facilities sampled. 6. Price variation of key medicines in public health facility dispensaries and private drug outlets (Survey Forms 2 and 11) Purpose To assess affordability based on the variation of price among pharmacies in the country. Prerequisite List of predetermined medicines to treat relevant health problems and their respective preparations. Source of data Survey of 30 public health facility dispensaries and 30 private drug outlets. Process Go through stock cards corresponding the listed medicines and take note of the most recent price per smallest dispensing unit. If there is a list of selling price, do the same. If there is not, ask a person in charge of the pharmacy. Be sure that the consulted source if the same used to charge the users. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 74 - Calculation Average minimum price of a specific medicine = sum of unit prices in each facility divided by number of facilities where the price could be recorded. 7. Average cost of medicines at public health facilities and private drug outlets (Survey Form 6 and 14) Purpose To measure average cost paid by patient for medicines at public health facilities and private drug outlets as an indicator of access to essential medicines. In most developing countries, a majority of the population pays for treatment out-of- pocket. Prerequisite Patient consent to the interview. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility pharmacies and pharmacies and 30 private drug outlets. See section on sampling . Process Interview patients leaving the dispensing area/pharmacy or leaving the facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Ask how much each patient paid out-of-pocket for the medicines received at the facility. Calculation Average cost = amount paid for medicines divided by number of patients reviewed National average = average cost for all public health facility dispensaries ÷ number of public health facility dispensaries sampled. 8. Geographical accessibility of facilities 8.1. % patients taking more than one hour to travel to the facility (Survey Forms 6 and 14) Purpose To assess geographical accessibility in terms of how long does it takes the patients to get to the pharmacy where they get their medicines. Prerequisite Patient consent to the interview. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility pharmacies and 30 private drug outlets. Process Interview patients leaving the dispensing area/pharmacy or leaving the facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Ask how long it took the patient to get to the facility. Calculation % of patients/ consumers taking more than one hour to get to the facility = number of patients/consumers who answered that it took more than one hour for them to get to the facility (category 3, > 1h) divided by number of interviewed patients/costumers. National average = % of patients/ consumers taking more than one hour to get to the facility in all public health facility dispensaries/private drug outlets divided by number of public health facility dispensaries/private drug outlets sampled. Annex 2 Level II Indicators - 75 - Limitations For some patients, the departure point may be different from their house (they may be using a health facilities near their work, for example). 8.2. Average transportation cost to the facility (Survey Forms 6 and 14) Purpose To assess geographic accessibility in terms of how much does it cost to the patients to get to the pharmacy where they get their medicines. Prerequisite Patient consent to the interview. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility pharmacies and 30 private drug outlets. Process Interview patients leaving the dispensing area/pharmacy or leaving the facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Ask how much it cost the patient to get to the pharmacy. Calculation Average cost = amount spent in transportation to get to the facilitydivided by number of patients/costumers interviewed. National average = average cost for all public health facility dispensaries/private drug outlets divided by number of public health facility dispensaries/private drug outlets sampled. Note: The average cost can be compared to the minimum government daily salary. Quality 1. Presence of expired medicines in public health facility dispensaries, private drug outlets and warehouses supplying the public sector (Survey Forms 1, 10 and 15) Purpose To determine if expired medicines are being distributed or sold. In some countries, expired medicines are distributed or medicines are allowed to go out of date on pharmacy shelves. See section on selecting key medicines for this study. Prerequisite List of 15 essential medicines to treat the most common health problems. Source of data Survey of 30 public health facility dispensaries, 30 private drug outlets and 5 warehouses. Process Go through stock on shelves and check the expiry dates of all of the generic and branded forms of each of the essential medicines. If any of the strengths has an expiry problem, the answer for that medicine should be “yes”. If expired medicines are listed and kept in a designated location in the store to be destroyed, availability of expiry should not be registered. Calculation % of expired key medicines in stock = number of key medicines with any samples beyond expiry date divided by number of key medicines in stock x 100 National average = sum of % of expired key medicines in stock at all facilities divided by number of facilities sampled. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 76 - 2. Adequacy of conservation conditions and handling of medicines in public health facility pharmacies/dispensaries, private drug outlets and central/regional/district warehouses supplying the public sector (Survey Forms 5, 13 and 17) Purpose To determine status of conservation conditions and handling of medicines in public sector facilities, both of which are factors that affect quality of medicines. Prerequisite Checklist of minimum criteria for adequate conservation conditions and handling of medicines at facilities. Source of data Survey of 30 public health facility dispensaries, 30 private drug outlets and 5 warehouses. Process Use the checklist to rate the conservation conditions and handling of medicines. Only indicate “true” if all conditions included in the statement are true. If any condition of the statement is false, indicate “false”. Calculation Conservation condition and handling of medicines = total number of “true” responses to items on the conservation condition and handling of medicines checklist ÷ 10 x 100. National average = total score of all facilities ÷ number of facilities sampled. Rational use of medicines 1. % of medicines adequately labelled at public health facility dispensaries (Survey Forms 6 and 14) Purpose To assess quality of dispensing practice. If medicines are to be used properly, they should be labelled appropriately by the person dispensing them. Prerequisite An adequate label includes the name of the medicine, how muchis to be taken and the frequency of administration. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility dispensaries. See page 28 for sampling instructions. Process Interview patients leaving the dispensing area or leaving the facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Check if each medicine label conforms to all requirements for adequate labelling. Count a medicine as adequately labelled only if all requirements are met. Calculation % of medicines adequately labelled = total number of medicines adequately labelled ÷ total number of medicines dispensed x 100. National average = sum of % of medicines adequately labelled at all public health facility dispensaries ÷ number of facilities sampled. Annex 2 Level II Indicators - 77 - 2. % of patients knowing how to take medicines at public health facility dispensaries (Survey Forms 6 and 14) Purpose To assess if patients have adequate knowledge about how to take their medicines. Prerequisite Adequate knowledge includes knowing the appropriate dosage and duration of each medicine. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility dispensaries. See page 28 for sampling instructions. Process Interview patients leaving the dispensing area or leaving the facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Check if the patient knows both the appropriate dosage and duration of each medicine (i.e., how much, how often and for how long he or she should take each medicine). Count the patient as having adequate knowledge only if both criteria are met for all medicines dispensed to the patient. Calculation % of patients knowing how to take medicines = number of patients who know how to take medicines ÷ number of patients sampled x 100. National average = sum of % patients knowing how to take medicines at all public health facility dispensaries ÷ number of facilities sampled. 3. Average number of medicines prescribed in public health facilities (Survey Forms 6 and 7) Purpose To determine the prevalence of polypharmacy, which is one measure of unnecessary prescribing. Prerequisite Outpatient treatment records covering the past 12 months or, if unavailable, current treatment records sufficient to randomly select 30 outpatient encounters. Source of data Sample of 30 outpatient encounters (retrospective or prospective) at 30 public health facilities. See pages 27—28 for sampling instructions. Process Survey form6- Interview patients leaving the dispensing area or leaving the facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Survey fprm7- Request all available records for the past 12 months before beginning sampling. Consider only encounters for a single disease, complaint, or symptom.List the number of medicines given per encounter. Combination products are counted as one medicine. The same product prescribed consecutively in different forms (i.e. injection and tablet) should be counted as one medicine, however if the same product is prescribed simultaneously in different forms, each form should be counted separately. Calculation Average number of medicines per encounter = total number of medicines prescribed ÷ number of patient encounters reviewed. National average = sum of average number of medicines per encounter from all public health facilities ÷ number of facilities sampled. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors - 78 - Note: Survey Form 6 collects data from patient interviews and Survey Form 7 collects data from patient records. Results may therefore differ for Survey Forms 6 and 7. 4. % of patients prescribed antibiotics in public health facilities (Survey Form 7) Purpose To determine the prevalence of antibiotic prescribing, since over-prescribing of antibiotics is one common type of inappropriate medicine use. Prerequisite Understanding of which medicines should be counted as antibiotics. Definitions of medicines considered as antibiotics must be agreed upon at the national level. Antimicrobial agents are not always classified in an identical way. Indicators for antibiotic use can be sensitive to certain medicines, especially in places with high incidence of parasitic infections, such as malaria or tuberculosis. Medicines such as antiprotozoals and anthelminthics are also usually placed in a different category of antibiotics. How to classify topical antibiotics widely used in areas where trachoma, bacterial conjunctivitis and bacterial skin infection are common will also need to be considered. Source of data Sample of 30 outpatient encounters (retrospective or prospective) at 30 public health facilities. See pages 27—28 for sampling instructions. Process Request all available records for the past 12 months before beginning sampling. Determine encounters where at least one antibiotic has been prescribed. Calculation % of patients prescribed antibiotics = number of encounters in which one or more antibiotics is prescribed ÷ number of patient encounters reviewed x 100. National average = sum of % of patients prescribed antibiotics in all public health facilities ÷ number of facilities sampled. 5. % of patients prescribed injections in public health facilities (Survey Form 7) Purpose To determine the prevalence of injection use, since over-prescribing of injections is one common type of inappropriate medicine use. Source of data Sample of 30 outpatient encounters (retrospective or prospective) at 30 public health facilities. See pages 27—28 for sampling instructions. Process Request all available records for the past 12 months before beginning sampling. Determine encounters where an injection has been prescribed. Do not count immunizations and injectable contraceptives. Calculation % of patients prescribed injections = number of encounters in which one or more injection is prescribed ÷ number of patient encounters reviewed x 100 National average = sum of % of patients prescribed injections in all public health facilities ÷ number of facilities sampled. 6. % of prescribed medicines on the essential medicines list at public health facilities (Survey Form 7) Purpose To measure the degree to which prescribing practice conforms to the national essential medicines list (EML). The essential medicines concept is one of the main strategies being promoted in medicines policy. More and more countries Annex 2 Level II Indicators - 79 - are formulating national EMLs. For most, this should be the basis for all public medicines procurement and prescribing. Prerequisite A current national essential medicines list officially endorsed by the ministry of health. If there is no current officially endorsed EML, then this indicator should not be measured. Source of data Sample of 30 outpatient encounters (retrospective or prospective) at 30 public health facilities. See pages 27—28 for sampling instructions. Process If there is a current offici

View the publication