IMMPACT Toolkit- Perceptions of Quality of Care (PQOC)

Publication date: 2007

Toolkit A guide and tools for maternal mortality programme assessment MODULE 4, Tool 4 Perceptions of Quality of Care (PQOC) Version 2.0 PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools Immpact © 2007 University of Aberdeen List of Acronyms FGD focus group discussion HWIS Health Worker Incentive Survey NVivo analysis software PQOC Perceptions of Quality of Care TBA traditional birth attendant WHO World Health Organization Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools Immpact © 2007 University of Aberdeen Module 4, Tool 4 – PQOC: Table of Contents INTRODUCTION. 1 4(4).1 What is PQOC?. 1 4(4).2 Why use PQOC?. 1 4(4).3 Adaptation of the PQOC approach . 2 4(4).4 Limitations of using PQOC. 2 4(4).5 Overview of the approach. 2 4(4).6 Using PQOC. 3 Step 1: Design your study . 3 Step 2: Obtain permissions . 4 Step 3: Train the research team . 4 Step 4: Conduct pilot study. 5 Step 5: Formulate an analysis framework. 5 Step 6: Collect and analyse the data . 6 Step 7: Disseminate findings and recommendations . 7 4(4).7 Budget implications of using PQOC. 8 4(4).8 PQOC GHANA case study . 9 4(4).9 PQOC data collection instruments . 10 1. Summary table for PQOC Guide to test and identify training needs . 11 2. In-depth interview guide for users and non-users of maternity services. 13 3. Focus group discussion guide. 16 4. Provider In-depth Interview Guide . 18 5. Manager’s In-depth Interview Guide . 20 REFERENCES. 22 Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 1 Immpact © 2007 University of Aberdeen INTRODUCTION One of the strategies identified as crucial for saving pregnant women’s lives, and a good indicator of progress towards the reduction of maternal mortality, is skilled attendance at delivery, a key element of which is quality of care. Yet this potentially life-saving intervention has not been provided or taken up sufficiently in many countries to ensure adequate coverage. Factors related to both users and providers of health care have been identified as potential causes of this lack of uptake, but it is still necessary to study contextual factors contributing to the lack of use of skilled care at delivery in different settings in order to give decision-makers and other safe motherhood stakeholders the requisite knowledge to provide user-centred maternity services that will attract and maintain user interest at all stages of the reproductive cycle, including labour and delivery. This research tool draws on an application of qualitative methods in the evaluations carried out by Immpact in Burkina Faso, Ghana and Indonesia in 2005–06. It is designed to generate data that will shed light on the quality of maternity care provision from two sides: the ‘demand’ side (users of services and the broader community) and the ‘supply’ side (providers of care). On the demand side, PQOC seeks to understand health-care-seeking behaviours, women’s experiences of maternity care, and other factors likely to affect their use of skilled care at delivery, with the aim of providing evidence to inform the development of services that will be better able to facilitate increased community use of such care. On the supply side, PQOC seeks to understand maternity care practitioners’ views on their own conditions of work and on the maternity care policy under evaluation. The sample instrument given here is focused on views of a fee exemption policy for maternity care in Ghana, but could be adapted to explore other types of programmatic intervention. This tool provides step-by-step guidance in applying the PQOC approach. The qualitative methods that have been used are generic and the sample instruments can be adapted by users to their own context and specific requirements. Users of this tool should refer to more detailed guidance on the use of qualitative methods in Technical Annex B in Module 5, as well as the resources section in Technical Annex F, which also highlight where alternative methods could be used. 4(4).1 What is PQOC? PQOC is a research tool that uses a combination of qualitative data sources to explore the perceptions of the community and health care providers on the factors that may affect the quality and processes of maternal health care. PQOC includes: • Community orientation of the research team; • In-depth interviews; • Focus group discussions. 4(4).2 Why use PQOC? It will be necessary to collect qualitative data on users’ perceptions of services and on health care workers’ views of their work conditions and the care they provide in many safe motherhood evaluations. This data can illuminate the processes of maternity services implementation and utilization in practice ‘on the ground’, and can highlight important barriers to, and facilitators of, effective access and provision. For the evaluation of the fee exemption policy in Ghana, for example, it was important to understand whether local communities and health care practitioners knew about and understood the policy, and to know how it affected their interactions around childbirth care. Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 2 Immpact © 2007 University of Aberdeen 4(4).3 Adaptation of the PQOC approach The PQOC approach is relevant for the majority of safe motherhood programme evaluations in low resource contexts. Adaptation of the research instruments may, however, be required. PQOC studies were conducted for all three initial Immpact evaluations. Burkina Faso, Ghana and Indonesia have differing health systems and cultures. Research instruments were therefore adapted to these different settings (for example by making terminology local and relevant), and were also adapted to answer the different safe motherhood evaluation questions. In Ghana, the context was that of community and provider perceptions of factors affecting provision and use of skilled care at delivery in the light of the recently introduced free delivery policy. Therefore, specific questions on this fee exemption policy were added to the instruments. The Indonesian evaluation was concerned with a programmatic intervention to increase community midwifery practice, while in Burkina Faso, the evaluation focused on an initiative to increase coverage of skilled care through community mobilization, improvement of maternity facilities, training of staff and improvement of supplies and equipment. Technical Annex C in Module 5 contains general information on adaptation of Immpact tools. 4(4).4 Limitations of using PQOC The principal limitations of using PQOC are the time and budget implications of qualitative methods, discussed in section 4(4).7, and the need for an experienced qualitative researcher to adapt the instruments, to train the team of field workers, and to conduct the data analysis with input from the team. It should also be noted that PQOC is not designed to be a self-standing assessment of all aspects of quality of maternity care, but rather to be used in combination with other methods from the toolkit. Its role is to provide detailed explanatory data on the functioning of the particular intervention under evaluation. 4(4).5 Overview of the approach Three elements of data collection are used in PQOC: community orientation of the research team, in-depths interviews, and focus group discussions. Community orientation of the research team The PQOC study team should, if possible, live in the community during the data collection period and immerse themselves as much as possible in community activities. The purpose is to get to know community life and how priorities may be formed, through informal data collection. During the Immpact evaluations, the PQOC teams spent time chatting informally with community members, visited the markets and spent time observing antenatal clinics. They wrote up field notes about community norms and characteristics, and the relationships with the maternity centre, and they were able to use the insights that they had gained from this informal data collection to guide the analysis of the interviews and focus group discussions. In-depth interviews Key informants from the community The community orientation will have allowed the study team to identify key influential people in the community. These key informants are interviewed to obtain a general view of perceptions held by the community about factors affecting uptake of skilled care at delivery. Users and non-users of maternity services Interviews are usually needed both with women who have recently used the maternity services under investigation and with women who have been pregnant but did not use the service. The topics explored may include their normal health-care-seeking behaviours, their relationships with maternity centre staff and any traditional birth attendants, and their perceptions of factors that may affect use of skilled care at delivery. Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 3 Immpact © 2007 University of Aberdeen Questions may also probe other factors such as family circumstances, distance from the health facility, and access to transport that may affect choice of delivery care attendant. Maternity care providers Interviews with local practitioners and managers of maternity services can explore their perspectives on the safe motherhood intervention under evaluation and on general factors affecting provision of maternity care. They can also gather potentially important contextual information about health care providers’ work conditions and these may impact upon the ability of services to deliver the desired care. Such qualitative information can be used to assist in the design, adaptation or interpretation of the Health Worker Incentive Survey (HWIS) tool (see Tool 5 in this module), if this is also to be applied. Focus group discussions (FGDs) FGDs should be conducted with users, non-users and providers in separate groups. Whereas individual interviews are valuable in obtaining a range of individuals’ stories and views in some detail, focus groups can draw out the collective viewpoint. Focus groups can bring people together to discuss and build on each other’s accounts, and to compare and contrast their perspectives. 4(4).6 Using PQOC The application of PQOC involves the following seven steps. In each case, thought must be given to the context (the national, local, socio-economic and cultural setting, as well as the specific programme being evaluated), to allow the various methods outlined above to be adapted appropriately. Step 1: Design your study Selection of study sites This step may require discussions about the characteristics of communities that are desirable for the study, but selection will usually be linked to the design of the overall safe motherhood evaluation question. For example, in Ghana, two different study sites were selected with differential experience with the fee exemption policy. Usually, visits to the districts and communities will be required, accompanied by members of the district health team, to look at the selected villages and confirm that they meet the desired characteristics. These visits also offer an opportunity to meet key stakeholders in the communities to introduce and explain the research project. This process should involve partners and collaborators, where possible, and all those who are going to participate in fieldwork. For example, for collaborative work with governments and other stakeholders, representatives from these institutions should be offered the chance of participating in or contributing to at least the general content of the proposed study. Participants The number of participants and data collection events in qualitative studies is determined by the complexity of the subject being studied and the resources that are available for the study. You can make decisions about a likely sample size at the start of the study for resource and budget planning, but be aware that some modification may be necessary once preliminary data analysis has commenced and it becomes clearer what range of experiences needs to be captured. As a guide, the numbers of participants in the different elements of the PQOC study for the Ghana Immpact evaluation are presented in section 4(4).9. Participants in the demand side of the study should be community members as follows: • Users of maternity services, i.e. women of reproductive age (15–49 years) who are currently using maternity services, pregnant or postpartum; • Non-users of maternity services, i.e. women of reproductive age who may be pregnant or who delivered without using maternity services; Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 4 Immpact © 2007 University of Aberdeen • Key informants, i.e. people of some standing in the community, such as the village head, community leaders, chairpersons of community committees and organizations, and influential relatives such as the mother-in-law, grandmother and husband. It may be important to think about how you will reach and include socially marginalized groups such as the very poor in your demand side sample. On the provider side participants should include the following: • Midwives or other skilled birth attendants; • Midwife supervisors, as appropriate for the country, for example, matrons, medical assistants; • Key informants working at the health facilities, such as auxiliaries and orderlies who will throw more light on the daily activities of midwives and on potential constraints to service provision. The research team It is recommended that the research team comprises one lead researcher experienced in qualitative methods and two to three other experienced qualitative researchers. It is important that the researchers who are going to do your fieldwork have a thorough understanding of the aims of the larger overall evaluation and of the PQOC component within that. You should involve them in the decisions about the techniques to be used, including that of identification of participants. Your field researchers should also be involved in drafting the proposed question or topic guides for in-depth interviews and focus group discussions according to their level of experience. The make-up of the research team will depend on local circumstances such as the availability of experienced researchers in the fields of study but should ideally incorporate and represent different social and ethnic groups as relevant. Thought should be given to how the gender and social status of interviewers and FGD facilitators may affect the dynamics of the data collection. It is important that respondents feel comfortable with the researcher and able to talk openly to them. Step 2: Obtain permissions At this stage, the design of your study should be fairly clear and documented in a research protocol. The usual ethics committees and research clearance procedures should apply. You may also need to seek agreement from community leaders, as well as consent from the individual participants themselves (see Module 5, Annex B). It is clear that the study involves dealing with information that may be sensitive in nature. You should be careful to protect the identities of the participants as much as possible. All data should be anonymized and the findings presented in a collated manner so that individuals cannot be easily identified. In order to ensure that your study will have the highest impact, it is a good idea to inform high-level authorities that the study is taking place, and receive their support from an early stage. The Standards and Principles Checklist for qualitative research contained in Technical Annex E (Module 5) can be helpful here. Step 3: Train the research team Training needs can be identified by conducting a small test in one or two sites similar to the ones to be selected for the study. Section 4(4).9 containing PQOC data collection instruments includes a checklist to help evaluate training needs. During this training needs test, there should be a discussion meeting at the end of the day to evaluate the work conducted and give feedback. This can be used for any further testing activities. The researchers may need support during this time, such as information on open-ended questioning, ethical principles in research, recruitment of research participants, data analysis and interpretation. If you identify any limitations in capacity, you should conduct a training programme in the relevant topics. A training workshop should be led on principles of qualitative methods and on ethical issues in the conduct of fieldwork. Action-oriented learning approaches could be used, which adopt participatory training methods. For example, participants can suggest training topics based on their experience and an assessment of their prior Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 5 Immpact © 2007 University of Aberdeen pre-test work and areas that need improvement. They can also participate in mock focus group discussions and video-recorded role-plays to practise research skills. Following the role-plays, each participant evaluates their own performance and discusses ways of improving. This is followed by further feedback and comments from other participants and facilitators. Various different training methodologies can be used for capacity-building activities. A selection of training resources and reference material on qualitative research are contained in Technical Annex F in Module 5. Further capacity-building support may be required after an initial training course throughout the study process. Areas which often require further support include: • In-depth interview skills and techniques; • Evaluation of field performance; • Iterative data collection and analysis; • Facilitation of focus group discussions. Mechanisms for providing this support should also be discussed and agreed upon. During the training and pilot stages, all team members should work together and should each evaluate their own performance and receive comments from team members after each activity. During the main study, an experienced researcher should be available and accessible to support field staff as necessary while allowing them enough space and flexibility to conduct their own evaluation. Step 4: Conduct pilot study Conduct a pilot study to test the approaches and procedures you plan to use, based on findings from the initial exploratory work and community orientation activities; and using the in-depth interview and focus group discussion guides developed for this study (see section 4(4).9). A pilot study is a small study usually comprising two to three focus group discussions and five in-depth interviews conducted before the main study to test out the research methodology and identify any unanticipated effects, problems and further needs. This can be another opportunity to assess researcher performance and training needs. Step 5: Formulate an analysis framework There are several different ways of conducting an analysis of qualitative data from interviews and focus groups in health services research (Pope et al, 2000). Some draw heavily upon ‘grounded theory’ methods in which the researcher attempts to enter the analysis with no preconceptions and to ‘let the data talk’. Others take a more structured approach at the outset. In this case, the study is being conducted to answer some quite specific questions about how communities perceive and access maternity care and about health care practitioners’ perceptions and conditions of work, in order to complement and shed light on findings from other tools. You may therefore find it useful to construct an initial analysis framework. To do this, you should identify the questions you want to answer, what you think you already know from other sources, and any assumptions about the situation that you want to test with this data. For example, you may want to explore the validity of the assumptions that knowledge about the fee exemption policy is widespread in the community that you are studying, and that health care providers are aware of it and do implement it comprehensively. You may also want to explore assumptions that financial costs are a deterrent to seeking care in some families and that the reduction of costs by removing users fees would be likely make a real difference to utilization. And so on. You can use data collected in the pilot study to help you to develop an initial analysis framework. To do this, your team will need to conduct a line-by-line analysis of the first transcripts to identify themes. Analysis should generally take place alongside data collection so that new ideas and insights can be incorporated into subsequent interviews, and if necessary new themes can be incorporated into the analysis framework. The framework will thus become more refined as you progress with your analysis. Step 6 includes more on analysis of data. Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 6 Immpact © 2007 University of Aberdeen Step 6: Collect and analyse the data The community orientation component of PQOC does not require formal data collection, but it is useful if your team members write some notes about what they have observed that is relevant to your study purpose. You will need to arrange opportunities for the team to come together to share and discuss these insights with you and with each other. There are various options for sequencing of the in-depth interviews and focus groups. One useful approach is to hold a focus group and then to follow this up with in-depth interviews with some of its participants in order to pursue some issues in more detail. If you do this, however, you may need to think about who did not attend the FGD and try to approach them in another way. For example, there may be people who are unlikely to take part in focus groups – perhaps because they are away at work, or unable to leave the house, or are socially marginalized – and their experience of maternity may be important for you to capture. Alternatively, you may wish to conduct the in-depth interviews and the FGDs concurrently using different participants in each, and plan the timing around the availability of your interviewees. Technical Annex B in Module 5 gives more information on qualitative data collection. It is important to note that you should maintain careful organization of your data collection and to draw up and constantly review a recruitment plan to ensure that you have the range of views and experiences that you need to be able answer your research questions with some confidence. All interviews and FDGs should be conducted in the local language, and audio-taped with the permission of the interviewee(s). Later these can be transcribed verbatim before being translated and back-translated to check accuracy (if that is necessary for analysis). Professional translation of transcripts for analysis may be needed. If the senior researcher is fluent in the local language translation prior to analysis may not be necessary. Attention should be paid to quality assurance in the research at all stages – see the Standards and Principles Checklist in Technical Annex E in Module 5. For example, an audit trail would be one way of keeping track of the process and mechanisms for the analysis, or transcripts should be back-translated to assure quality. To code and analyse the transcripts of in-depth interviews and focus groups, Immpact used NVivo qualitative analysis software. It is equally possible, however, to analyse data manually. As mentioned earlier, after initial codes have been developed the data analysis process is an iterative one: codes will normally be updated and refined as themes become clarified and new issues emerge. A small example from a PQOC coding structure is shown in the following table. Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 7 Immpact © 2007 University of Aberdeen Table 4(4).1 (table1): Example of a coding structure used in a PQOC study Theme A: Quality of care Quality of care/Supplies and equipment Quality of care/Waiting time Quality of care/Characteristics of maternity ward Quality of care/Characteristics of maternity ward/Delivery positions Quality of care/Characteristics of maternity ward/Space Quality of care/Characteristics of maternity ward/Lack of privacy Quality of care/Midwife attitude Theme B: Costs Cost/Facility fees Cost/Facility fees/Delivery fees Cost/Facility fees/Other fees Cost/Transport Theme C: Awareness of fee delivery policy Awareness of policy/No Awareness of policy/Yes Theme D: Views about policy effect Views about policy effect/Community satisfaction Views about policy effect/Community satisfaction/Positive Views about policy effect/Community satisfaction/Negative During analysis, key themes may be identified as meriting further investigation. For example, in this case, cost emerged as a key factor affecting use of skilled delivery attendants, and so transcripts were reanalysed to explore in detail the frequency and nature of discussions around cost by different participants with different characteristics. Results may be produced in text format, collating and summarizing thematic findings, and using suitably anonymized case studies as illustration if appropriate. It may also be useful to portray findings diagrammatically. Figure 1 uses findings from the three PQOC studies conducted by Immpact in Burkina Faso, Ghana and Indonesia to construct a diagram of the barriers and facilitators for use of skilled care at delivery. Step 7: Disseminate findings and recommendations In order to disseminate your findings successfully, you should discuss local protocols for dissemination with any collaborators, target local, national and international safe motherhood stakeholders and qualitative methods groups for peer review and further development of methods. If PQOC is being applied as part of a broader evaluation on the quality of maternal health care, the findings from this arm of the evaluation should be integrated with other study results. More information on dissemination of results and integration of research findings is contained in Step 9 of Module 3. Immpact Toolkit: a guide and tools for maternal mortality programme assessment Possible factors affecting use of skilled care at delivery PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 8 Immpact © 2007 University of Aberdeen Figure 4(4).1 (fig1): Barriers and facilitators to use of skilled care at delivery 4(4).7 Budget implications of using PQOC The costs of PQOC data collection and analysis will depend upon the existing capacity for qualitative methods and the research training that is required. Where possible, qualified local experts can provide research-capacity- building, which not only reduces costs but is also culturally sensitive. The resource requirements for using PQOC will depend on various factors including: • Sample size; • Training requirements; • The range of methods selected. An example of resource requirements from an Immpact PQOC study in Ghana is summarized in table 2. Data collection for the study was conducted in two districts, each involving two maternity centres, with interviews and focus groups conducted in two villages per centre. This required data collection in a total of eight villages. Sustained use of maternity services Intervening Conditions Perceived Quality of Care Desire for well - being Immunizations Safe delivery Official documents Characteristics of maternity care Characteristics of staff Attitude Privacy Waiting times Transport/ Distance Culture Authority Possible factors affecting use of skilled care at delivery Attitude Authority Socio- economic status Sustained use of maternity services Intervening conditions Perceived Quality of Care Desire for well-being Immunizations Safe delivery Official documents Characteristics of maternity care Privacy Culture Transport/Characteristics of staff DistanceWaiting times Socio- economic status Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 9 Immpact © 2007 University of Aberdeen Table 4(4).2 (table2): PQOC resource requirements Quantity Time Supplies Photocopies of interview and FGD guides and training workshop materials Not applicable 1 experienced lead researcher 8 hours training and orientation 20 days 2–3 experienced qualitative researchers 8 hours training and orientation 20 days per person Personnel 3 data collectors 2–3 months Travel and communication Vehicle to travel to interview sites and to health facilities Travel reimbursement to study participants Phone calls, emails, postage Not applicable Building operation and maintenance Shared premises with other research Over a period of 6 months 4(4).8 PQOC GHANA case study This section outlines the PQOC research process as it was conducted in Ghana. Section 4(4).9 also contains a set of PQOC instruments used in Ghana (training assessment form, in-depth interview guides, focus group discussion guides). Ghana evaluation question What is the effectiveness and cost-effectiveness of the policy of universal fee exemption on delivery care in Ghana? Ghana PQOC questions • What are community views of the acceptability and standard of care provided in the context of free delivery care in Ghana? • What are community perceptions and health care providers’ views of the free delivery policy? • What are community views about factors affecting their utilization of skilled care in the context of the free delivery policy? • What are provider views and experiences of their work conditions and of factors affecting staff motivation to provide good quality of care under the free delivery care policy? As seen from the questions above, on the community side, PQOC in Ghana: • Investigated women’s perceptions and knowledge of free delivery policy and factors that change women’s care-seeking behaviour and how these factors could be related to the introduction of fee exemption; • Investigated, among other factors, past experience with delivery charges, experience of using the free delivery services and perceptions of quality of care before and during the free delivery policy. On the provider side, PQOC aimed to investigate: • Providers’ perceptions of the free delivery policy and factors that influence their provision of quality of care under the free delivery policy; • Providers’ experiences of working before and during the free delivery policy and perceptions of quality of care; • Providers’ experiences of implementing the policy and how it affected their motivation; • Providers’ views of their work conditions and working relationships. Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 10 Immpact © 2007 University of Aberdeen Study sites and participants In Ghana, the evaluation was carried out in two regions: Central and Volta. In the Central Region, the policy of free delivery had been in place for a number of years. Within this region, a district was chosen which had consistently received funds since beginning of the free delivery policy. In Volta Region, the free delivery policy had been in operation for just a few months, and a district was selected where the policy included free delivery with a trained traditional birth assistant. The aim was to better understand the factors affecting community use of maternity services in districts where service cost was not necessarily a barrier. Research was conducted in two maternity centres per district where women can self refer. In total, eight sites across the two districts were selected, including those with local hospitals, and those with and without a local health centre. In-depth interviews Purposive sampling based on targeting a range of women aged 15–49, users and non-users of services, distance from health facility and social standing, etc., was used to select about 30–40 participants for in-depth interviews with users, about 20 with providers and a separate group of about 10–15 key informants per region. Interviews lasted 30–60 minutes. Focus group discussions Purposive sampling was used to select participants for the focus groups. The criteria for selection aimed to create fairly homogenous groups (by age, gender, social standing etc) within which participants could interact easily. Six focus groups were conducted per region with eight to ten participants selected according to their background characteristics of age, parity or type of care currently sought, e.g. a group of pregnant mothers and a group of mothers with children less than six months old. 4(4).9 PQOC data collection instruments This section contains the following PQOC data collection instruments that are based on Immpact evaluations in Ghana: 1. Summary table for PQOC guide to test and identify training needs; 2. In-depth interview guide for users and non-users of maternity services; 3. Focus group discussion guide; 4. Provider in-depth interview guide; 5. Manager’s in-depth interview guide. Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 11 Immpact © 2007 University of Aberdeen 1. Summary table for PQOC Guide to test and identify training needs Using the table This table can be used to facilitate the training and pilot testing phases of PQOC and to identify training needs in order for the final study to proceed. The table contains crucial elements that must be in place for the fieldwork to be conducted successfully. The research team can review this list to see if anything needs to be modified for their specific context. Each researcher can indicate the ease or difficulty in achieving these skills or preparations by ticking the relevant box and then giving details of what actually happened or what needs to be done in the detailed comments section. At the end of each day during the pre-test and pilot test, the summary tables should be discussed and more notes added to the detailed comments, documenting action to either improve or follow up on the particular line- item. Aspect of study to assess Easy Difficult Detailed comments Availability of potential participants Accessibility of potential participants How study fits in daily activities of participants Desire of potential participants to participate Data collection Easy Difficult Detailed comments Venue chosen for interviews Privacy for interviews Ease of using method Accuracy of translation Clarity of questions Number of participants/informants Easy Difficult Detailed comments Selection criteria Time it takes to locate participants Particular study site chosen Recruitment Easy Difficult Detailed comments Methods of locating suitable participants Methods of obtaining informed consent Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 12 Immpact © 2007 University of Aberdeen Data processing Easy Difficult Detailed comments Tape recorders, transcribers Data storage Effectiveness of analysis Research team Easy Difficult Detailed comments Application of methods Output of individual members Team dynamics Need for training Work plan Easy Difficult Detailed comments Time management Time allocation for analysis Sequence of activities Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 13 Immpact © 2007 University of Aberdeen 2. In-depth interview guide for users and non-users of maternity services Based on the PQOC study conducted within the Immpact evaluation in Ghana. The following general questions will be asked to start discussions with users, but some of the questions may not be asked as the questioning will follow the flow of the interview. Please note that the interview starts after the interviewer has explained the study to the participant, the participant has indicated that s/he understands and is willing to participate and to have the interview tape- recorded, and a record of that consent has been obtained. During the initial discussion to explain the study, the interviewer and participant should have had the opportunity to introduce themselves and to make small talk to break the ice. PQOC instruments are designed to encourage participants to come up with issues to discuss. However, sometimes it may be difficult to get participants to talk spontaneously or you may want to hear more detail on a relevant topic, and prompts and probes can be used such as those outlined below. Greetings. As we explained at length and described in the information leaflets, we want to hear your views about delivery care in health services in your village since the introduction of the free delivery policy. I am interested in hearing about where you go for maternity services (name of maternity centre), who currently provides maternity services, for example, midwife, auxiliary midwife, village midwife and traditional midwife. Please also tell us your views about how they perform and which provider you think is the best and why. When you are ready please tell us what you think, just feel free to talk. Firstly we will try to just listen and not interrupt you, but at the end we may ask you some more questions about what you told us or about other things we want to know about. Your views are very important to us, so just feel free to talk… [Note to interviewer: Wait patiently and encourage with nodding and smiles and see what response you get. Follow up according to the response from participant. If the following were not addressed, you may use these questions and their follow up probes.] Please tell us about the free delivery care policy. Probes: Do you know about the policy? If so how did you get to know? I am interested in knowing of things have changed in any way since this policy came in. Did you use maternity services before the introduction of the policy? Probes: Have you used maternity services since introduction? What are your impressions of service provision under the policy in comparison to before? What is your opinion of the free delivery care policy in practice? Probes: What do you like most about the policy? What difference, if any, has it made to your ability or willingness to use maternity services? What, if anything, do you not like about the policy? So tell me a little more about where you go for health care. What health facility do you usually use? Probes: Why do you use that particular health facility? Do you have a choice of what facility to use? What do you think about services provided at your health facility? Is the care provided at your facility different from care at other facilities you know? How is it different? Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 14 Immpact © 2007 University of Aberdeen Apart from at the health facility, do you get maternity services elsewhere? Probes: If so, where? How do these services compare with those provided at your local maternity centre? Have you ever been attended by the TBA or know someone who has been attended by her/him? Probe: What do you think about services provided by the TBA? I’d like to know how is easy it is for you to get to the health facility. How far is your home from the health facility? Probes: How do you get to the health facility (e.g. walk, bus, taxi, motorbike etc.)? Do you have to pay for transport to get to the health facility? Who pays when you go to maternity services? How easy/difficult is it for you? Who makes decisions about where you should go for maternity services? Probes: Who makes decisions about whether you should go to maternity services? Can you go to maternity services without consulting anybody? Why/why not? What factors make it easy for you to go to maternity services? What factors make it difficult for you to go to maternity services? What could be done to make it easy for you to go to maternity services? If you were asked to choose a health centre anywhere in the country to attend (with free transport) what would be your first choice? Why? We are interested in learning from people’s experiences of going for health care, and what makes these good or bad experiences. First I’d like to ask you about a good experience. I’d like you to think back now over all the times you have seen a health worker. Can you remember a time when you went to see a health care provider and received exceptionally good care? If yes, tell me about that time. Probes: Who attended you? What made you go the health centre/hospital (what care were you seeking?) What happened? What made the experience good for you? If no, is there an occasion that you remember when excellent care was given to someone else – someone you know, maybe someone you accompanied or an incident you heard about? Tell me about that. Now I’d like you to think back about any time when you felt you had bad care. Can you recall such an occasion? Probes: Tell me about that bad experience. Who attended you? What made you go the health centre (what care were you seeking?) What happened? If you have not had a bad experience yourself, perhaps you know someone else who has, can you tell us something about that? Sometimes people are unable to go to the health centre and are attended by someone else instead. Has that ever been the case for you? Probes: Why were you unable to go to the health facility? Where did you go instead? Why? What happened? If you were to be in the same situation again, what would you do? Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 15 Immpact © 2007 University of Aberdeen Are you able to communicate well with your health care provider about your health or about childbirth problems? Probe: What things do you communicate about? And when you have a health problem or a childbirth problem in the family, whom do you tell first – apart from your family and friends? Let’s talk a little more about problems in pregnancy. If you have problems with your pregnancy during the first three months who is it that can help you? (Probe about cultural beliefs and practices about early pregnancy.) Probes: If you cannot get the help you need within the family what do you do? Are there any beliefs related to pregnancy and childbirth that affect women’s use of the maternity services provided at the health centre/clinic/hospital? I want you to imagine you have been chosen to talk to the government on behalf of the women in your community! What you have been asked to talk about is the provision of good maternity services for pregnant mothers. Please tell me what ideas you would suggest for improvement of the maternity services or the care that woman receive. Further topics If the following issues do not arise spontaneously during the answers to the above, you may want to raise them yourself. Note: Avoid asking direct questions as much as possible; rather, prompts should be in the form of ‘give us an example of …’, ‘what do you think of …’, ‘tell us more about …’. Satisfaction with specific aspects of childbirth care Waiting times Privacy during examination Information provided about tests and procedures Involvement in decision-making about care (opportunity to ask questions) Physical environment at the health facility Continuity of care (same person at the health facility, ability to build relationship) / use of same facility Respect for community norms (religious, cultural), Respect for community members Personal characteristics of the provider, e.g. age, language, behaviour Attitude of health staff towards those accompanying user to health facility Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 16 Immpact © 2007 University of Aberdeen 3. Focus group discussion guide More information on FGDs is available in Module 5, Technical Annex B. There are also further qualitative research resources contained in Technical Annex F. Topics For the focus group discussions with users and non-users of maternity services, you can use the same principles and topics as outlined in the in-depth interview guide above), but where specific issues have been identified as pertinent in this community, the FGD might be used to focus on these in detail and more specifically, for example: Who makes decisions about going to the health centre? Who pays for maternity service costs? Reasons for home deliveries Reasons for using village midwives and TBAs Transport to health centre Attitudes of different members of staff (who provides best care? what is best care?) Privacy during consultations Time for consultations Opportunity to ask questions Discussion of good and bad experiences Note-taking schedule for focus group discussions Note: it is best to have a note-taker (who observes) as well as a facilitator (who guides the discussion) Date and time of the discussion Seating plan Names and characteristics of participants Description of group dynamics e.g. who is participating and who is not participating? Who is dominating the discussion at any particular point? What is the level of interest in the topic? Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 17 Immpact © 2007 University of Aberdeen Notes on key statements from participants Emotions displayed during the discussions Any relevant discussions during break times Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 18 Immpact © 2007 University of Aberdeen 4. Provider In-depth Interview Guide The following general questions will be asked to start discussions with providers but some of the questions may not be asked as the questioning will follow the flow of the interview. Please note that the interview starts after the interviewer has explained the study to the participant, the participant has indicated that s/he understands and is willing to participate and to be tape-recorded, and a consent has been obtained and recorded. During the initial discussion to explain the study, the interviewer and participant should have had the opportunity to introduce themselves and to make small talk to break the ice. This study aims to learn about perceived barriers and facilitators to the provision of good quality maternity services under the free delivery policy. We would like to hear your views about the implementation of the policy and how it has impacted your conditions of work, what encourages you in your job, and what discourages you. Please give me your honest views about these matters. When you are ready you can start; we will try and just listen and at the end we may ask some questions to clarify some of the things you talked about. (If the following points are not spontaneously mentioned you may ask further questions as follows, as necessary.) First I’d like to ask you about the fee exemption policy for maternity care. When was the fee exemption policy implemented in your area? Probes: Were you already working here when it was implemented? What are your thoughts about this policy? For example, why was it implemented? Has it made any change as far as use of skilled care at delivery is concerned? How has it affected your work? Is your work easier? More difficult? Why? What do you like about the policy? What do you not like? If you were consulted in an evaluation of the fee exemption for delivery policy what issues would you raise? Probes: What would be your recommendations for care of women? What would be your recommendations for making skilled care at delivery more accessible and acceptable? What would be your recommendations concerning the providers of maternity care? How best to motivate them and keep them interested to provide the best care they can under the policy? Now I’d like to ask you a little about your own preferences about where you’d like to live and work. For example, if you had a completely free choice about where you could work, if were asked to choose a district and village in which to work, what would be your first choice? Probe: Why? And if you could choose any health centre in your current district to work in, which would be your first choice? Probe: Why? And if you were asked to choose a health centre anywhere in the country to work, which would be your first choice? Probe: Why? If you were consulted in order to redraft your job description and improve conditions of service, what aspects of your current job would you choose? Probes: Which aspects of your job would you prefer not to do? What new activities, if any would you add? What recommendations would you make for improving conditions of service? Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 19 Immpact © 2007 University of Aberdeen How do you feel about your accommodation at the moment? Probes: Who provides your accommodation? Probe on type and condition of house, distance from work, distance from the community, distance from facilities e.g. shops, bars, entertainment, distance from family and relatives. And how do you feel about your salary? Further topics The following are some additional topics that may be of interest. Support from superiors Relationship with the community Involvement in decision-making Opportunities for in-service training Whether or not living with or near close family Importance of choice of place of work Hours of work and workload Relationships with other colleagues Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 20 Immpact © 2007 University of Aberdeen 5. Manager’s In-depth Interview Guide Based on PQOC study within the Immpact evaluation in Ghana. Please note that the interview starts after the interviewer has explained the study to the participant, the participant has indicated that s/he understands and is willing to participate and to be tape-recorded, and a consent has been obtained and recorded. During the initial discussion to explain the study the interviewer and participant should have had the opportunity to introduce themselves and to make small talk to break the ice. We are interested to hear about factors that may affect your staff in providing good quality care under the free delivery policy. We want to know about the main factors affecting provision of services under the policy, and about the state of motivation of your staff. When you are ready please just talk about your views on this; first we will just listen and after we may ask you some questions to clarify or to ask about other things. How do you feel about the fee exemption policy generally? Are there any differences in services provided in different health centres under the policy? Probes: What are the differences? How do they affect staff motivation? If you were asked to choose one health centre in your district as a model, which one would you choose? Probe: Why? (What is special about that one?) How do you feel about the current usage of maternity services by the community? Probes: Has the fee exemption policy made any difference? Why? What do you think affects usage of maternity services? What are your thoughts about quality of care provided in health centres in your district? Are there any differences in quality of the care provided at different health centres in the district? Probes: What are the differences? So, in your view, what causes the differences? What are your views about the current state of motivation among maternity staff? Are there any differences in motivation in different health centres? Probe: What do you think causes those differences? What aspects of the free delivery policy do you like most? Probe: Why? What are your views about motivation of staff under the policy? Probes: You may ask also about provision of equipment, supplies, transport, community mobilization, staff housing. If you were asked to evaluate the free delivery policy what would you say? If you were asked to make recommendations to the Health Minister about how you might improve quality of services for women in your district, what would you suggest? If you were asked to make recommendations to the Health Minister about how you might improve motivation of maternity staff in your district, what would you suggest? Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 21 Immpact © 2007 University of Aberdeen Further topics Avoid asking direct questions as much as possible, rather prompts should be in the form of ‘give us an example of …’, ‘what do you think of …’, ‘tell us more about …’. The following are some additional topics that may be of interest. Workload Guidelines/protocols Supervision Adequate supplies and equipment to provide care according to the protocols Mechanisms for community feedback to the hospital management Choice of job Other comments Immpact Toolkit: a guide and tools for maternal mortality programme assessment PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC PQOC Module 4: Evaluation tools 4 : PQOC : 22 Immpact © 2007 University of Aberdeen REFERENCES AbouZahr C, Wardlaw T (2001) Maternal mortality at the end of a decade: signs of progress? Bull World Health Org, 79 (6), 561–8. De Bernis L, Sherratt DR, AbouZahr C, Van Lerberghe W (2003) Skilled attendants for pregnancy, childbirth and postnatal care. British Medical Bulletin Pregnancy: Reducing maternal death and disability, 67, 39–57. Elo I (1992) Utilization of maternal health services in Peru: the role of women's education. Health Transit Review, 2, 61. Family Care International (2002) Skilled care during childbirth: policy brief. Family Care International. Hunt L, Glantz N, Halperin D (2002) Childbirth care seeking behaviour in Chiapas. Health Care for Women International, 23, 98–118. Pope C, Ziebland S, Mays N (2000) Analysing qualitative data. BMJ, 320, 114–16. Ronsmans C, Etard JF, Walraven G, Hoj L, Dumont A, Bernis L, Kodio B (2003) Maternal mortality and access to obstetric services in West Africa. Trop Med Intl Health, 8, 940–8. Sundari T (1992) The untold story: how health care systems in developing countries contribute to maternal mortality, Int J Health Serv, 22, 513–28. WHO (1989) The role of the health services in preventing maternal deaths. In: Royston E, Armstrong S (Eds) Preventing maternal deaths. Geneva: World Health Organization, pp. 153–83. Yanagisawa S, Oum S, Wakai S (2006) Determinants of skilled birth attendance in rural Cambodia, Trop Med Intl Health, 11, 238–51. Note-taking schedule for focus group discussions

View the publication