- Review
- Open access
- Published:
Quality in crisis: a systematic review of the quality of health systems in humanitarian settings
Conflict and Health volume 15, Article number: 7 (2021)
Abstract
Background
There is a growing concern that the quality of health systems in humanitarian crises and the care they provide has received little attention. To help better understand current practice and research on health system quality, this paper aimed to examine the evidence on the quality of health systems in humanitarian settings.
Methods
This systematic review was based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol. The context of interest was populations affected by humanitarian crisis in low- and middle- income countries (LMICs). We included studies where the intervention of interest, health services for populations affected by crisis, was provided by the formal health system. Our outcome of interest was the quality of the health system. We included primary research studies, from a combination of information sources, published in English between January 2000 and January 2019 using quantitative and qualitative methods. We used the High Quality Health Systems Framework to analyze the included studies by quality domain and sub-domain.
Results
We identified 2285 articles through our search, of which 163 were eligible for full-text review, and 55 articles were eligible for inclusion in our systematic review. Poor diagnosis, inadequate patient referrals, and inappropriate treatment of illness were commonly cited barriers to quality care. There was a strong focus placed on the foundations of a health system with emphasis on the workforce and tools, but a limited focus on the health impacts of health systems. The review also suggests some barriers to high quality health systems that are specific to humanitarian settings such as language barriers for refugees in their host country, discontinued care for migrant populations with chronic conditions, and fears around provider safety.
Conclusion
The review highlights a large gap in the measurement of quality both at the point of care and at the health system level. There is a need for further work particularly on health system measurement strategies, accountability mechanisms, and patient-centered approaches in humanitarian settings.
Background
In 2015, the Sustainable Development Goals (SDGs) launched with the strong message of leaving no one behind [1]. These goals cannot be achieved without including the more than 70 million refugees, internally displaced persons (IDPs), and asylum-seekers [2]. The right to the highest attainable health extends to all individuals, regardless of their circumstance or legal status. International human rights treaties, most notably the International Covenant on Economic, Social and Cultural Rights, guarantee “the right of everyone to the enjoyment of the highest attainable standard of physical and mental health” [3].
There has been considerable progress in global health over the past 20 years driven by expanded access to public health services in low- and middle-income countries (LMICs) (e.g., clean water and sanitation) and health services (e.g., vaccination, antenatal care, and HIV treatment) [4]. These improvements have saved millions of lives primarily by averting deaths from infectious diseases [5]. However, strengthened strategies will be needed to tackle chronic and complex conditions and reach the health related SDGs. It has become clear that access alone is not enough without an accompanying focus on the quality of services being provided. High quality health systems are needed to improve health outcomes.
Humanitarian health activities commonly focus on immediate life-saving interventions. This often necessitates vertical programming in order to rapidly start and scale-up health services, particularly for historically key issues of preventing disease outbreak and reducing malnutrition. However, the challenge now is that crises are increasingly protracted, which requires longer-term and more coordinated and sustainable approaches. There are specific fears around the lack of continuity of care for conditions that require multiple visits to the health system, such as antenatal care (ANC) and postnatal care (PNC) and chronic conditions such as HIV/AIDS and diabetes [6]. This requires greater engagement in the broader health system and a focus on sustainable quality of care.
This review examined empirical evidence and provided an overview of how the literature defines and measures quality in crisis situations. Additionally, as a sub-aim, this review analyzed the methods used to measure quality (i.e., the quality of the quality measure), so that researchers can improve their work in this area. This review aimed to give direction to the humanitarian field on where future improvement strategies could be targeted. With the current emphasis in the global health community on quality of health systems this study is extremely relevant and timely.
Defining high quality health systems
Building on the past work on quality and the developments in the field of health systems, the Lancet Global Health Commission on High Quality Health Systems in the SDG Era (HQSS) defined high quality health systems as systems that “optimize health in a given context by consistently delivering care that improves or maintains health, being valued and trusted by all people, and responding to changing population needs” [7]. This included, but was not limited to, the quality of care provided at the point of contact between provider and user. The definition goes on to state that equity, efficiency, and resilience are values that underpin a high quality health system [7]. Here, equity implies that it is available and affordable to all people regardless of underlying social disadvantages. An efficient system aims to achieve the highest possible health improvement with the given resources and a resilient system responds to routine and catastrophic challenges.
The conceptual framework developed by the HQSS Commission (Fig. 1) has three main domains: processes of care (competent care and user experience), quality impacts (better health, confidence in system and economic benefit) and foundations (population, governance, platforms, workforce and tools) [7]. The HQSS Commission believed that health systems should primarily be assessed based on processes and impacts of care because research shows that poor health systems can operate even when all the adequate tools are present [8]. The foundations of the system included the population’s health needs and expectations, governance of the health and non-health sectors, accessible and organized platforms of care, skilled workforce in numbers, and tools such as medicine and data.
Health systems in crisis settings
Humanitarian crisis settings can vary widely in context, but are situations that involve widespread human suffering resulting from complex political, economic or social emergencies and natural hazards requiring large-scale provisions of aid. These include acute humanitarian crises that have a sudden onset and chronic, or protracted, humanitarian crises including forced displacement [9]. According to the Lancet Series on health in humanitarian crises, “protracted situations, often with additional acute emergencies, are becoming the new norm” [10]. Most refugees are not living in camps but rather in urban and rural areas and are not displaced but rather entrapped in conflict settings, such as those in Syria and Yemen. As a consequence, refugees and IDPs are at their highest number in over 50 years [11].
In many crisis situations, the health system goes through a period of degradation and fragmentation due to increasing violence and insecurity, weakening governance, and loss of resources. Reduced government activities create a void in services provided that is often filled by faith-based, private or informal providers [12]. Dozens of non-governmental organizations (NGOs) may be active in any of the main ongoing humanitarian crises and providing essential health services. These services tend to focus on primary health care for communicable disease control and management, nutrition, reproductive, maternal newborn and child health (RMNCH) and more recently mental health and NCDs. They are typically part of the UN Cluster System, with health care falling under the Health Cluster led by WHO, and sometimes in partnership with the local government [13]. They commonly follow the Sphere Standards and aim to support secondary and tertiary care facilities [14]. However, even within the same crisis different actors may pursue inconsistent and uncoordinated health strategies [15]. Assessing the performance of health actors in crises is challenging because of: insecurity and limited access; population mobility; short operational and funding time-periods (typically six-monthly cycles); rapidly developing health events; weak collection and sharing of routine health data; and limited monitoring, evaluation and research capacity [16].
This need has been magnified by the increasing number of conflicts in countries with a disease burden already heavily dominated by NCDs (e.g., Syria) [17]. To address these shortcomings there have been increasing efforts in the humanitarian sector to engage in accountability – particularly ensuring perspectives of affected populations are included [16]. In addition, despite the many epidemiological studies that have been conducted on humanitarian crises, there has been persistent concern over the quality of data for humanitarian crises [9, 18,19,20,21,22,23], and how data have been used to guide humanitarian health interventions and ultimately improve health outcomes [24,25,26,27,28]. To help better understand current practice and research on health system quality, this paper aims to examine the evidence on the quality of health systems in humanitarian settings.
Methods
We conducted a systematic review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol [29].
Eligibility criteria
The context of interest was populations affected by humanitarian crisis in low- and middle- income countries (LMICs). We defined the situation of humanitarian settings using the criteria specified by the Sphere Standards [14] and the income status by the World Bank’s 2019 classifications. We included contexts affected by war, terrorist attack, political violence or armed conflict. Refugee/Internally displaced person (IDP) camps or settlements were also included, as were refugees, IDPs, or conflict-affected people who are living in non-camp settings. This included protracted displacement crises. We excluded large epidemics or pandemics since they vary considerably by pathogen and context. We also excluded post-conflict/post-disaster settings, which we classified as more than 5 years after the formal end of an armed conflict (e.g., signing of a peace agreement) or natural disaster.
We included studies where the intervention of interest, health services for populations affected by crisis, was provided by the formal health system. We defined the formal health system to be care by a trained provider, including public, NGO and private facilities. We excluded studies that fell outside the formal health system, for example school-based malnutrition campaigns.
Our outcome of interest was descriptive- how these studies defined and measured quality in the formal health system. We used the definition of high-quality health systems presented by HQSS and detailed above. We excluded studies if they reported solely on coverage or access to health care but did not specifically focus on quality.
We included primary research studies published in English between January 2000 and January 2019 using quantitative methods (any experimental, quasi-experimental, or observational design). We included randomized controlled trials if the outcome of interest was a quality measure but excluded clinical trials that aimed to prove only efficacy. Qualitative methods (any design) were included.
Data sources and search terms
We used a combination of information sources to identify studies meeting the inclusion criteria: (1) electronic bibliographic databases for published studies, using a comprehensive search; (2) grey literature; and (3) the reference lists of studies included in the review. For the bibliographic databases, we searched published literature in PubMed, Embase, and Web of Science. For the grey literature, we searched the following databases and websites: WHO Global Health Library, UNHCR database, Reproductive Health Response in Crisis Consortium (RHRC), and the Inter-Agency Working Group on Reproductive Health in Crisis (IAWG). To approach literature saturation, we checked reference lists of included studies. We documented the information sources, including the name of each search, the date range searched and the search platform.
For the bibliographic database searching, we used Medical Subject Headings (MeSH) terms and key words from prior published literature. We constructed sets of search terms to capture four concepts: quality health system (which includes but is not limited to “quality of care”), focus on low- and middle-income countries, populations, and humanitarian settings (e.g., “quality” AND “LMIC” AND “population” AND “humanitarian setting”). The search strategy for PubMed is outlined in the Additional file 1: Appendix A. This strategy was applied to the other electronic databases and modified if necessary.
Screening process
We downloaded and saved all search results into reference management software (EndNote version X7) and screened using an abstract management software (Abstrackr). Prior to the screening process we created a detailed selection criteria worksheet, which can be found in the Additional file 2: Appendix B, which all reviewers (KJ and TL) built consensus around and used through the screening process. We then did a pilot round applying the selection criteria to a subset of abstracts and came together to discuss the process. Two reviewers screened the entire set of abstracts (KJ and TL). We separately applied the criteria to the entire data set and adjudicated conflicts through consensus building. Full text reviews were done for the final sample by one reviewer (KJ) and reviewed by a second (TL).
Data extraction and analysis
We applied the High Quality Health System Framework (Fig. 1) to help organize and analyze the data. This framework was chosen because it addresses health system quality more broadly, rather than just quality of care, is uniquely applicable to LMICs, and is easily adapted to humanitarian settings. Specifically, we extracted data related to the framework’s domains and sub-domains of quality health systems (Box 1).
We extracted data from the final eligible studies using a standardized form. The following variables were extracted: year of publication, title, study type, evaluated country, study setting, population type, number of participants, study description, quality domain (from framework, Box 1), quality sub-domain (from framework, Box 1), methodological quality, main findings/results, miscellaneous. A deductive approach was used to organize the analysis within the domains of the High Quality Health System Framework and then an inductive approach used to explore emerging themes from within the domains of the framework.
To assess the methodological quality of the included studies, we evaluated each study in six domains: selection bias, appropriateness of data collection, appropriateness of data analysis, generalizability, ethical considerations and clarity of the study’s methods [30]. This domain-based evaluation was chosen instead of a scale or checklist due to its ability to critically assess different domains separately, as suggested by The Cochrane Collaboration [31].
Results
We identified 2269 records through the database search and an additional 16 records through the other sources. After removal of duplicates and screening, 55 papers remained for full-review (the screening process and main reasons for exclusion are provided in Fig. 2).
Of the 55 papers included in the systematic review the majority of studies were published since 2015 (34; 62%). Multiple study designs were used: 17 (31%) were quantitative, 16 (30%) were qualitative, and another 13 (24%) used mixed quantitative and qualitative methods. Eleven (20%) took place in the WHO African Region, two (4%) in the European Region, eight (14%) in the South East Asia Region, 25 (45%) in the Eastern Mediterranean Region, two (4%) in the Western Pacific Region, and seven (13%) were multi-country studies that spanned multiple regions. The majority of studies took place in conflict-affected settings (45; 82%), eight (14%) in refugee settlements, and only two (4%) studies taking place in a post-natural disaster setting. About one third of the population type studied was refugees, IDPs, or conflict-affected cross-boarder migrants (17; 31%). Fifteen studies (27%) focused on women and/or children, and 11 (20%) focused solely on health care providers. Table 1 describes the characteristics of included studies.
We intentionally did not restrict the papers included by study design in order to gain insight from qualitative and mixed methods studies. The rigor of the studies was assessed and the majority (34, 62%) of the studies adequately addressed at least five of the six quality areas. Fifty-one (93%) studies used a sample that was appropriate to its research questions, 49 (89%) studies collected the data appropriately, 46 (84%) analyzed the data appropriately, 31 (56%) studies had results that were transferable by setting, 23 (42%) adequately addressed potential ethical issues, and 48 (87%) studies were clear in their overall approach.
We used the High Quality Health Systems Framework to analyze the included studies by quality domain and sub-domain. The majority of studies (46; 84%) addressed multiple quality domains simultaneously, with a strong emphasis on workforce (38; 69%), tools (26; 45%), and competent care (23; 44%). Table 2 depicts the numbers of addressing the domains and sub-domains.
Process of care
The majority of the studies included addressed process of care as a quality domain. A theme that emerged within those studies was that poor diagnosis and treatment was a major barrier health systems faced in conflict settings. Common manifestations of poor quality were incorrect diagnosis [37, 40, 44, 66, 69] and inappropriate treatment of illness [44, 46, 73, 81, 82]. Lainez and colleagues highlighted the issue of “competent care” in their study on the prevalence of respiratory symptoms in Afghanistan and found that there was a gap in diagnosis with 23.8% of patients with TB-suggestive symptoms going undiagnosed [37]. Another manifestation of the “competent care” sub-domain was necessary patient referrals [32, 34, 66, 69, 77]. For example, Elmusharaf et al. found that outcomes were better for pregnant women in South Sudan where there was no facility available rather than when the woman accessed a non-functioning facility, and the absence of a health care provider was better than the presence of a non-competent provider [66]. This finding was primarily influenced by inadequate referral systems (including late referrals to appropriate facilities and multiple referrals). In two studies the lack of continuity of care was expressed as a quality concern [35, 66]. However, when care was integrated (e.g., mental health services into primary health care) a positive outcome was seen [40, 76].
User experience, including feeling that the health staff was judgmental or discriminatory was cited as a major barrier to high quality care and impacted care seeking behavior [33, 36, 41, 47, 67]. For example, the study by Kruk and colleagues on population preferences for health care in Liberia showed that a patient’s choice of clinic was influenced by respectful treatment along with other factors such as provision of a thorough physical exam, availability of medicines, and government management [41]. Language barriers and the lack of clear communication also negatively impacted the patient-provider relationship in the studies examined [33, 45, 50, 67, 72, 75]. This was particularly true for refugees in host countries [33, 45, 50, 72].
Quality impacts
About one third of included studies addressed “quality impacts” as a quality domain. Nine focused on the patient’s confidence in the system, seven focused on better health, and three focused on economic benefit.
The patient’s confidence in the health system was most commonly measured as care satisfaction [35, 39, 47, 53, 62, 78, 79, 83]. The findings around satisfaction were mixed. Some studies reported that patients were dissatisfied with the low quality care they received [47, 78, 79], and Kibiribiri et al. found that refugees were more dissatisfied than the general population in South Africa [62]. However, other studies reported that patient satisfaction was high even where quality was poor [35, 39, 83]. In their study on perceptions and utilization of primary health care services in Iraq, Burnham and colleagues showed that high satisfaction corresponded with low expectations of the health system [39]. The patient’s perception of the low quality of care was a barrier to care uptake and retention in the health system [32, 67, 78]. A qualitative study by Hunter-Adams et al. on the language barriers between South African health care providers and conflict-affected cross-boarder migrants suggested that providing interpretive services could increase the patient’s confidence in the system and potentially increase preventative care visits [67].
In many studies, clinical outcomes, most commonly hospital mortality, were used to measure the quality of care provided by the health system [34, 43, 49, 54, 65, 75, 77]. For example, Auto et al. discussed the importance of hospital quality improvement strategies to start with a clear understanding of (child) mortality, its causes, and distribution.
The economic burden that poor quality care had both on the patient and the system was only addressed in three studies [84,85,86]. Chuah and colleagues used a qualitative approach to assess the health system responses to the health needs of refugees and asylum-seekers in Malaysia. One of the key findings was that healthcare financing was a major challenge in responding to refugee health issues and even with the discounted fee for refugee patients at public healthcare facilities the out-of-pocket expenditure for them was still too high [84]. Bertone et al. examined how performance based financing (PBF) can be adapted in fragile settings and found that providing free care to IDPs, even where free care was not an official government policy, successfully improved access by reducing financial burden [86]. Additionally, a study on health service resilience in Nigeria found that political instability had a direct impact on financial barriers, which through multiple pathways influenced utilization of health services [85]. This study found that drug subsidy schemes and programs offering free services “moderated the health impact of the disruption of livelihoods resulting from insurgency” [85].
Foundations
The majority of the studies included addressed the foundations of a health system. A common theme that emerged was an emphasis on the workforce and tools quality sub-domains. The evidence identified on workforce constraints focused on human resource shortages [34, 39, 57, 63, 80, 81, 84], low workforce moral [47, 75], and inadequate provider training [48, 51, 56, 58, 61, 71, 74] as the most cited causes of poor quality. A qualitative study in Afghanistan articulated the intense physical and mental pressures that medical staff face in the field and assured that sub-optimal care was unlikely deliberate but rather the result of “conflicting priorities, the workload, poor clinical skills and the struggle for survival” [56]. The physical resources needed for a functioning health system was often used in the studies as a quality measure, specifically limited access to medicines [32, 34, 38, 57, 60, 61, 79, 81, 84] and supplies [42, 48, 57,58,59,60, 63, 64, 70, 72, 79]. Mowafi and colleagues highlighted the severe material and human resource constraints that Syrian trauma hospitals operate under including the large amount of nonfunctioning diagnostic equipment (e.g., 23% broken X-ray machines) [63].
Other quality sub-domains were also addressed, though to a lesser degree. Within the “population” sub-domain, the cultural barriers patients faced during service delivery were raised in a number of studies [45, 53, 58, 67, 73, 79, 84, 85]. In a study on refugee and migrant women’s views of antenatal ultrasound, Rijken and colleagues suggested that “transient embarrassment or shame on exposing the abdomen (a part not normally exposed in public by local women in this culture)” was a potential barrier to receiving medical attention [45]. Community resources and cohesion, as described by Ager et al., was identified as a key driver of pathways of influence mitigating the impacts of the crisis. The study went on to detail how communities “pooled resources (knowledge, transport and finance) to enable physical and financial access to health facilities for those in need”. Communities also played a key role in mobilizing political will for quality care [85]. “Platform” barriers to quality care were included in six studies and addressed the number and distribution of facilities [39, 52, 57, 85] and the disorganized structure of service delivery [70, 81]. A study looking at the implementation of Afghanistan’s Basic Package of Health Services (BPHS) found that access to and utilization of primary health care services in rural areas increased dramatically because the number of BPHS facilities more than doubled [52]. Governance improvements, specifically political commitment and enhanced leadership, were highlighted as necessary ways to improve health system performance [34, 38, 52, 55, 61, 80, 84,85,86]. A study in Afghanistan by Anwari et al. suggested that improvements in stakeholder engagement, cultivating accountability, setting a shared strategic direction and stewarding resources responsibility were possible by implementing a people-centered governance approach [55].
Discussion
This systematic review, which included 55 studies, examined the evidence on the quality of health systems providing care in humanitarian settings. It was the first study to our knowledge that addressed this topic. The key findings suggest: poor diagnosis and inappropriate treatment of illness (including inadequate patient referrals) were commonly cited barriers to quality care; there was a limited focus on the health impacts that health systems have in the studies identified; and a strong focus was placed in these studies on the foundations of a health system with emphasis on the workforce and tools. The implications from these findings for future research include: expanding the definition of quality to include quality impacts, developing and validating quality measures suitable for crisis settings, and incorporating more diverse and rigorous study designs.
There are many contextual challenges highlighted in the identified literature to providing quality care in humanitarian settings, however, there are also findings from this review that are broadly in line with evidence from more stable LMICs. In terms of “process of care” the HQSS Commission also found that many LMICs struggle to consistently deliver high quality care and well-known, effective treatments are not consistently provided [7]. Many of the studies included in this review attempted to improve care competence with short in-service provider trainings. However, there are many studies that attest to the know-do gap (i.e., the gap between provider knowledge and clinical care provided) [87, 88], suggesting that these efforts may not have the long-term quality improvements they are aiming for. Further, the disrespectful care that many of the studies attested to is widespread throughout LMIC health systems. The HQSS Commission found that 1 in 3 patients experienced disrespectful care, short consultations, poor communication or long wait times [7].
The limited focus that the studies included in this review placed on “quality impacts” belies the primary goal of health systems, which is to improve or maintain health. The initial findings from this review suggest potentially lower levels of satisfaction in crisis settings than the general population. High satisfaction with health care is common across LMICs even where quality is poor, possibly due to low expectations, and the HQSS Commission warned that patient satisfaction as a measure of quality should be carefully interpreted. A lack of confidence in the health system can in turn hinder care uptake and retention, which is already fragile given the setting. Although causes of death and disease are multifactorial some conditions are highly dependent on quality of care and how well the health system is working, such as maternal and newborn deaths [89]. The emphasis that the studies in this review placed on measuring maternal, newborn, and child mortality is therefore in line with other LMIC health system quality indicators. High quality health systems generate many economic benefits, such as reducing premature mortality and reducing health system waste. The studies in this review, however, primarily forced on financial risk protection [85, 86].
The “foundations” of a health system are often the most cited and measured elements of quality [7, 90]. Workforce and facility constraints, specifically, are widespread through LMICs and the HQSS Commission found that 47% of improvement research was targeted towards these two sub-domains [7]. Though foundations are essential to health-care provision, prior studies have commented on the weak associations between input measures and care competence [8]. The HQSS Commission therefore recommended a shift in measurement away from foundations (or inputs) to what matters most to people: competent care, user experience, health outcomes, and confidence in the system [7]. This people-centered approach places the emphasis on the user of the health system and aims to create a setting in which people have agency over their own health and health-care decisions. This can be particularly challenging in the face of violence, displacement or forced migration. Strengthening health system quality has unique challenges in crisis situations, particularly when the majority of services are provided by NGOs and large gaps exist in critical services, and inequity in the distribution of those services. The studies in this review suggest that a key step is assessing the needs of the population in crisis, who are often vulnerable groups that face further marginalization due to context. Keeping equity and the underserved a priority from the beginning requires localizing health system improvements. The low emphasis that the studies placed on “population” needs suggest it is an area to focus for improvement. As many of the studies in this review point out changes towards a people-centered health system take strong local political commitment and leadership. In support of this finding, the WHO National Quality Policy and Strategy Handbook suggested that well-aligned policies and strategies be based on locally-accepted definitions of quality and national goals for improved outcomes [91].
The findings from this review are in line with the broader literature on humanitarian settings and highlight the limited evidence on health impacts of interventions and use of economic methods in humanitarian settings [92, 93]. There were many studies that fell outside the eligibility criteria of this review that assessed the role of user fees and performance incentives on quality and utilization rates that could offer lessons and areas for future study [94,95,96]. Additionally, there is a call to increase measurement in humanitarian settings and link that to a focus on accountability mechanisms [6, 16]. Different quality measurement techniques have been used in humanitarian contexts but an overarching theme was that consideration and adaption of design processes are needed to meet local circumstances [6, 97]. Process of care measures (i.e., what a provider does to maintain or improve health) have shown to play an important role in assessing care for vulnerable populations [98] and this review suggests they could be particularly useful to inform the users about the care they should expect to receive and increase demand for high quality. There are additional measurement constraints in crisis settings due to insecurity and other contextual challenges and resource constraints, which in many instances make more rigorous study designs, particularly experimental studies, operationally and ethically challenging [9, 99, 100]. There are, however, many valuable studies on health system resilience and rebuilding in post-conflict and fragile settings that could be useful in humanitarian contexts [101, 102]. Also, alternate designs have proven successful in humanitarian settings at showing changes in health outcomes over time, such as stepped wedge designs, greater use of longitudinal data and routine health services data, and collection of process data that can be a reliable proxy for health outcome data [9, 103, 104]. Additionally, due to the complexity of health systems research a more narrow approach that focuses on specific aspects of quality may be beneficial for future work as well as a stronger use of interdisciplinary research (e.g., social science, political science, epidemiology).
The findings from this study have many policy relevant implications and they point to the need for ‘macro-level’ system wide transformations. First, they highlight the need for new and improved quality measurements that move beyond the foundational aspects of quality. Adopting measures that focus on system competence (e.g., timeliness and continuity of care), user experience and health outcomes could potentially shape future quality improvement strategies. Second, they point to a need for initiatives that focus on improving accountability in humanitarian settings. Multipronged strategies that build partnerships across the system are needed that combine legal, performance and social accountability tools. This potentially involves legislating for vulnerable people’s right to quality health care, educating the population on their rights, creating strong regulations and standards, and enforcing mechanisms for remedy and redress. Finally, they suggest that there has been a failure to respond to key health system concepts such as patient-centered care. All people deserve to be treated with respect and dignity within the health system. Additionally, health workers need to receive the support they need to fulfill their professional duty even under the most dire circumstances. A high-quality people-centered health system should take into account the needs, experiences, and preferences of even the most vulnerable populations.
Limitations
We may not have captured all the data available on quality of care in humanitarian settings. In particular, we limited our review to English language studies, only used three electronic bibliographic databases, and though the entire first round was double screened only one screener did the final full text reviews. The absence of experimental designs is also a limitation considering they could show attribution of interventions to changes in health outcomes. However, this review included a large number of qualitative and mixed-methods studies, which provides a unique insight into user experience.
The two areas of methodological quality in which included studies were lacking were generalizability and ethical considerations. In terms of generalizability, most of the studies included in our review were on conflict-affected settings and the findings may not be applicable to other crises such as natural disasters. There are many ethical challenges when doing research in humanitarian crises considering the added vulnerability of the population.
Conclusion
There has been a growing interest on the quality of care health systems provide in humanitarian settings throughout the past two decades. However, a large gap still exists on studies that systematically measure quality both at the point of care and at the health system level. The findings from this review highlight key quality issues including incorrect diagnoses and treatments, low levels of confidence in the health system, and a disproportional emphasis on the health workforce and tools. The review also suggests some barriers to high quality health systems that are specific to humanitarian settings such as language barriers for refugees in their host country, discontinued care for migrant populations with chronic conditions, and fears around provider safety. Individuals, families, and communities in humanitarian crises have specific health needs that require an understanding of their culture, preferences, and health knowledge in order to be met. There is need to expand work on the topic, particularly for focusing on health system measurement strategies, accountability mechanisms, and patient-centered approaches in humanitarian settings.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Program UND. Sustainable development goals. Geneva: United Nations; 2015.
Agency UR. Global trends: forced displacement in 2018; 2019.
Assembly UG. International covenant on economic, social and cultural rights. United Nations, Treaty Series. 1966;993(3):2009–57.
Victora CG, Requejo JH, Barros AJ, et al. Countdown to 2015: a decade of tracking progress for maternal, newborn, and child survival. Lancet. 2016;387(10032):2049–59.
Murray CJ, Ortblad KF, Guinovart C, et al. Global, regional, and national incidence and mortality for HIV, tuberculosis, and malaria during 1990–2013: a systematic analysis for the global burden of disease study 2013. Lancet. 2014;384(9947):1005–70.
Kersten R, Bosse G, Dörner F, Slavuckij A, Fernandez G, Marx M. Too complicated for the field? Measuring quality of care in humanitarian aid settings. Glob Health Action. 2013;6(1):20311.
Kruk ME, Gage AD, Arsenault C, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Global Health. 2018;6(11):e1196-252.
Leslie HH, Sun Z, Kruk ME. Association between infrastructure and observed quality of care in 4 healthcare services: a cross-sectional study of 4,300 facilities in 8 countries. PLoS Med. 2017;14(12):e1002464.
Blanchet K, Ramesh A, Frison S, et al. Health in humanitarian crises 1 evidence on public health interventions in humanitarian crises. 2017.
Spiegel PB. The humanitarian system is not just broke, but broken: recommendations for future humanitarian action. Lancet. 2017.
UNHCR. Global trends: forced displacement in 2016. Geneva: UN High Commissioner for Refugees; 2017.
Hill PS, Pavignani E, Michael M, Murru M, Beesley ME. The “empty void” is a crowded space: health service provision at the margins of fragile and conflict affected states. Confl Heal. 2014;8(1):1–10.
Organization WH. Health cluster guide: a practical guide for country-level implementation of the health cluster: World Health Organization, 2009.
Association S. Sphere handbook: humanitarian charter and minimum standards in humanitarian response: PRACTICAL ACTION; 2018.
Shukor AR, Klazinga NS, Kringos DS. Primary care in an unstable security, humanitarian, economic and political context: the Kurdistan region of Iraq. BMC Health Serv Res. 2017;17(1):592.
Checchi F, Warsame A, Treacy-Wong V, Polonsky J, van Ommeren M, Prudhon C. Public health information in crisis-affected populations: a review of methods and their use for advocacy and action. Lancet. 2017.
Mokdad AH, Jaber S, Aziz MIA, et al. The state of health in the Arab world, 1990–2010: an analysis of the burden of diseases, injuries, and risk factors. Lancet. 2014;383(9914):309–20.
Boss LP, Toole MJ, Yip R. Assessments of mortality, morbidity, and nutritional status in Somalia during the 1991-1992 famine: recommendations for standardization of methods. Jama. 1994;272(5):371–6.
Garfield R. Studies on young child malnutrition in Iraq: problems and insights, 1990–1999. Nutr Rev. 2000;58(9):269–77.
Spiegel PB, Salama P, Maloney S, Van Der Veen A. Quality of malnutrition assessment surveys conducted during famine in Ethiopia. Jama. 2004;292(5):613–8.
Bostoen K, Bilukha OO, Fenn B, et al. Methods for health surveys in difficult settings: charting progress, moving forward. BioMed Central; 2007.
Prudhon C, Spiegel PB. A review of methodology and analysis of nutrition and mortality surveys conducted in humanitarian emergencies from October 1993 to April 2004. Emerg Themes Epidemiol. 2007;4(1):10.
Spiegel P, Le P. HIV behavioural surveillance surveys in conflict and post-conflict situations: a call for improvement. Global Public Health. 2006;1(2):147–56.
Salama P, Spiegel P, Talley L, Waldman R. Lessons learned from complex emergencies over past decade. Lancet. 2004;364(9447):1801–13.
Duffield A, Reid G, Shoham J, Walker D. Evidence base for interventions in complex emergencies. Lancet. 2005;365(9462):842–3.
Roberts L. Advances in monitoring have not translated into improvements in humanitarian health services. Prehospital Disaster Med. 2007;22(5):384–9.
Banatvala N, Zwi AB. Conflict and health: public health and humanitarian interventions: developing the evidence base. BMJ. 2000;321(7253):101.
Griekspoor A, Collins S. Raising standards in emergency relief: how useful are sphere minimum standards for humanitarian assistance? BMJ. 2001;323(7315):740.
LA Moher D, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.
Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ (Clinical research ed). 2008;337:a1035.
Higgins JPT GSe. Cochrane handbook for systematic reviews of interventions version 5.1.0 The Cochrane Collaboration 2011; (0).
Hafeez A, Riaz R, Shah SU, Pervaiz J, Southall D. Integrating health care for mothers and children in refugee camps and at district level. Bmj. 2004;328(7443):834–6.
Rutta E, Williams H, Mwansasu A, et al. Refugee perceptions of the quality of healthcare: findings from a participatory assessment in Ngara, Tanzania. Disasters. 2005;29(4):291–309.
Auto J, Nasi T, Ogaoga D, Kelly J, Duke T. Hospital services for children in the Solomon Islands: rebuilding after the civil conflict. J Paediatr Child Health. 2006;42(11):680–7.
Wickramasinghe W, Widanapathirana N, Vijayabandara G, Karunathilake I. Are Tsunami survivors satisfied with the provision and quality of healthcare they received? Asia Pac J Public Health. 2007;19(1_suppl):35–9.
Garang PG, Odoi RA, Kalyango JN. Adherence to antiretroviral therapy in conflict areas: a study among patients receiving treatment from Lacor hospital, Uganda. AIDS Patient Care STDs. 2009;23(9):743–7.
Lainez YB, Todd CS, Ahmadzai A, Doocy SC, Burnham G. Prevalence of respiratory symptoms and cases suspicious for tuberculosis among public health clinic patients in Afghanistan, 2005-2006: perspectives on recognition and referral of tuberculosis cases. Tropical Med Int Health. 2009;14(5):564–70.
Chan GJ, Parco KB, Sihombing ME, Tredwell SP, O'Rourke EJ. Improving health services to displaced persons in Aceh, Indonesia: a balanced scorecard. Bull World Health Organ. 2010;88(9):709–12.
Burnham G, Hoe C, Hung YW, et al. Perceptions and utilization of primary health care services in Iraq: findings from a national household survey. BMC Int Health Hum Rights. 2011;11(1):1–9.
Hijazi Z, Weissbecker I, Chammay R. The integration of mental health into primary health care in Lebanon. Intervention. 2011;9(3):265–78.
Krukz ME, Rockers PC, Varpilah ST, Macauley R. Population preferences for health Care in Liberia: insights for rebuilding a health system. Health Serv Res. 2011;46(6):2057–78.
Burnham G, Malik S, Al-Shibli ASD, et al. Understanding the impact of conflict on health services in Iraq: information from 401 Iraqi refugee doctors in Jordan. Int J Health Plann Manag. 2012;27(1):e51–64.
Hynes M, Sakani O, Spiegel P, Cornier N. A study of refugee maternal mortality in 10 countries, 2008-2010. Int Perspect Sex Reprod Health. 2012;38(4):205–13.
Lima J, Abeyasinghe RR, Fitzpatrick R, Fernando SD. Diagnosis and treatment of malaria by health care providers: findings from a post conflict district in Sri Lanka. Int Health. 2012;4(2):148–50.
Rijken MJ, Gilder ME, Thwin MM, et al. Refugee and Migrant Women’s Views of Antenatal Ultrasound on the Thai Burmese Border: A Mixed Methods Study. PLoS One. 2012;7(4):e34018.
Asgary R, Jacobson K. Comprehensive on-site medical and public health training for local medical practitioners in a refugee setting. Disaster Med Public Health Preparedness. 2013;7(1):82–8.
Rahmani Z, Brekke M. Antenatal and obstetric care in Afghanistan–a qualitative study among health care receivers and health care providers. BMC Health Serv Res. 2013;13(1):166.
Ahamadani FA, Louis H, Ugwi P, et al. Perinatal health care in a conflict-affected setting: evaluation of health-care services and newborn outcomes at a regional medical centre in Iraq. Eastern Mediterranean Health J. 2014;20(12):789–95.
Parr M, Dabu CP, Wai NS, et al. Clinical audit to enhance safe practice of skilled birth attendants for the fetus with nuchal cord: evidence from a refugee and migrant cohort. BMC Pregnancy Childbirth. 2014;14(1):1–9.
Shannon PJ. Refugees' advice to physicians: how to ask about mental health. Fam Pract. 2014;31(4):462–6.
Low S, Tun KT, Mhote NP, et al. Human resources for health: task shifting to promote basic health service delivery among internally displaced people in ethnic health program service areas in eastern Burma/Myanmar. Glob Health Action. 2014;7:24937.
Newbrander W, Ickx P, Feroz F, Stanekzai H. Afghanistan's basic package of health services: its development and effects on rebuilding the health system. Glob Public Health. 2014;9(sup1):S6–S28.
Svoronos T, Macauley RJ, Kruk ME. Can the health system deliver? Determinants of rural Liberians' confidence in health care. Health Policy Plan. 2015;30(7):823–9.
Alvarado O, Trelles M, Tayler-Smith K, et al. Orthopaedic surgery in natural disaster and conflict settings: how can quality care be ensured? Int Orthop. 2015;39(10):1901–8.
Anwari Z, Shukla M, Maseed BA, et al. Implementing people-centred health systems governance in 3 provinces and 11 districts of Afghanistan: a case study. Confl Heal. 2015;9(1):1–16.
Arnold R, van Teijlingen E, Ryan K, Holloway I. Understanding afghan healthcare providers: a qualitative study of the culture of care in a Kabul maternity hospital. Bjog-an Int J Obstet Gynaecol. 2015;122(2):260–7.
Casey SE, Chynoweth SK, Cornier N, Gallagher MC, Wheeler EE. Progress and gaps in reproductive health services in three humanitarian settings: mixed-methods case studies. Confl Heal. 2015;9(1):1–13.
Curry DW, Rattan J, Huang S, Noznesky E. Delivering high-quality family planning Services in Crisis-Affected Settings II: results. Global Health Sci Pract. 2015;3(1):25–33.
Ho LS, Labrecque G, Batonon I, Salsi V, Ratnayake R. Effects of a community scorecard on improving the local health system in eastern Democratic Republic of Congo: qualitative evidence using the most significant change technique. Confl Heal. 2015;9(1):1–11.
Hoogenboom G, Thwin MM, Velink K, et al. Quality of intrapartum care by skilled birth attendants in a refugee clinic on the Thai-Myanmar border: a survey using WHO safe motherhood needs assessment. BMC Pregnancy Childbirth. 2015;15:17.
Jones A, Howard N, Legido-Quigley H. Feasibility of health systems strengthening in South Sudan: a qualitative study of international practitioner perspectives. BMJ Open. 2015;5(12):e009296.
Kibiribiri ET, Moodley D, Groves AK, Sebitloane MH. Exploring disparities in prenatal care between refugees and local south African women. Int J Gynecol Obstet. 2016;132(2):151–5.
Mowafi H, Hariri M, Alnahhas H, et al. Results of a Nationwide capacity survey of hospitals providing trauma Care in war-Affected Syria. Jama Surg. 2016;151(9):815–22.
Savas N, Arslan E, Inandi T, et al. Syrian refugees in Hatay/Turkey and their influence on health care at the university hospital. Int J Clin Exp Med. 2016;9(9):18281–90.
Doocy S, Paik KE, Lyles E, et al. Guidelines and mHealth to Improve Quality of Hypertension and Type 2 Diabetes Care for Vulnerable Populations in Lebanon: Longitudinal Cohort Study. Jmir Mhealth Uhealth. 2017;5(10):e158.
Elmusharaf K, Byrne E, AbuAgla A, et al. Patterns and determinants of pathways to reach comprehensive emergency obstetric and neonatal care (CEmONC) in South Sudan: qualitative diagrammatic pathway analysis. BMC Pregnancy Childbirth. 2017;17(1):278.
Hunter-Adams J, Rother HA. A qualitative study of language barriers between south African health care providers and cross-border migrants. BMC Health Serv Res. 2017;17(1):97.
Hynes M, Meehan K, Meyers J, Mashukano Maneno L, Hulland E. Using a quality improvement approach to improve maternal and neonatal care in north Kivu, Democratic Republic of Congo. Reproductive health matters. 2017;25(51):140–50.
Mohammadi S, Gargari SS, Fallahian M, Kallestal C, Ziaei S, Essen B. Afghan migrants face more suboptimal care than natives: a maternal near-miss audit study at university hospitals in Tehran, Iran. BMC Pregnancy Childbirth. 2017;17(1):1–9.
Sami S, Kerber K, Kenyi S, et al. State of newborn care in South Sudan’s displacement camps: a descriptive study of facility-based deliveries. Reprod Health. 2017;14(1):1–12.
Sami S, Kerber K, Tomczyk B, et al. "you have to take action": changing knowledge and attitudes towards newborn care practices during crisis in South Sudan. Reproductive Health Matter. 2017;25(51):124–39.
Tanabe M, Myers A, Bhandari P, Cornier N, Doraiswamy S, Krause S. Family planning in refugee settings: findings and actions from a multi-country study. Confl Heal. 2017;11(1):1–12.
Tschirhart N, Thi SS, Swe LL, Nosten F, Foster AM. Treating the invisible: gaps and opportunities for enhanced TB control along the Thailand-Myanmar border. BMC Health Serv Res. 2017;17(1):1–11.
Yalahow A, Hassan M, Foster AM. Training reproductive health professionals in a post-conflict environment: exploring medical, nursing, and midwifery education in Mogadishu, Somalia. Reproductive Health Matters. 2017;25(51):114–23.
Bottcher B, Abu-El-Noor N, Aldabbour B, Naim FN, Aljeesh Y. Maternal mortality in theGaza strip: a look at causes and solutions. BMC Pregnancy Childbirth. 2018;18.
Echeverri C, Le Roy J, Worku B, Venteyoger P. Mental health capacity building in refugee primary health care settings in sub-Saharan Africa: impact, challenges and gaps. Global Mental Health. 2018;5.
Lagrou D, Zachariah R, Bissell K, et al. Provision of emergency obstetric care at secondary level in a conflict setting in a rural area of Afghanistan - is the hospital fulfilling its role? Confl Heal. 2018;12(1):1–9.
Mohamed-Ahmed R, Aziz MA, Walker R. Antenatal Care in Sudan: a qualitative study into accessibility and quality of maternal health Services in Khartoum. Int J Childbirth. 2018;8(2):77–86.
Mugo NS, Dibley MJ, Damundu EY, Alam A. "the system here isn't on patients' side"- perspectives of women and men on the barriers to accessing and utilizing maternal healthcare services in South Sudan. BMC Health Serv Res. 2018;18(1):1–8.
Sami S, Amsalu R, Dimiti A, et al. Understanding health systems to improve community and facility level newborn care among displaced populations in South Sudan: a mixed methods case study. BMC Pregnancy Childbirth. 2018;18(1):325.
Wanzira H, Muyinda R, Lochoro P, et al. Quality of care for children with acute malnutrition at health center level in Uganda: a cross sectional study in West Nile region during the refugee crisis. BMC Health Serv Res. 2018;18(1):561.
Roberts B, Makhashvili N, Javakhishvili J, et al. Mental health care utilisation among internally displaced persons in Ukraine: results from a nation-wide survey. Epidemiol Psychiatr Sci. 2019;28(1):100–11.
Amsalu R, Morris CN, Chukwumalu K, et al. Essential newborn care practice at four primary health facilities in conflict affected areas of Bossaso, Somalia: a cross-sectional study. Conflict Health. 2019;13:27.
Chuah FLH, Tan ST, Yeo J, Legido-Quigley H. Health system responses to the health needs of refugees and asylum-seekers in Malaysia: A qualitative study. Int J Environ Res Public Health. 2019;16(9) (no pagination)(1584).
Ager AK, Lembani M, Mohammed A, et al. Health service resilience in Yobe state, Nigeria in the context of the Boko haram insurgency: a systems dynamics analysis using group model building. Confl Heal. 2015;9(1):30.
Bertone MP, Jacobs E, Toonen J, Akwataghibe N, Witter S. Performance-based financing in three humanitarian settings: principles and pragmatism. Confl Heal. 2018;12(1):28.
Mohanan M, Vera-Hernández M, Das V, et al. The know-do gap in quality of health care for childhood diarrhea and pneumonia in rural India. JAMA Pediatr. 2015;169(4):349–57.
Leonard KL, Masatu MC. Professionalism and the know-do gap: exploring intrinsic motivation among health workers in Tanzania. Health Econ. 2010;19(12):1461–77.
Ronsmans C, Graham WJ. Group LMSSs. Maternal mortality: who, when, where, and why. Lancet. 2006;368(9542):1189–200.
Gergen J, Josephson E, Coe M, Ski S, Madhavan S, Bauhoff S. Quality of care in performance-based financing: how it is incorporated in 32 programs across 28 countries. Global Health. 2017;5(1):90–107.
Organization WH. Handbook for national quality policy and strategy: a practical approach for developing policy and strategy to improve quality of care. Geneva: World Health Organization; 2018.
Blanchet K, Ramesh A, Frison S, et al. Evidence on public health interventions in humanitarian crises. Lancet. 2017;390(10109):2287–96.
Makhani LA, Moran V, Sadique Z, Singh NS, Revill P, Roberts B. Examining the use of economic evaluations in health-related humanitarian programmes in low-and middle-income countries: a systematic review. Health Policy Plan. 2020;35(2):210–8.
Maini R, Van den Bergh R, van Griensven J, et al. Picking up the bill-improving health-care utilisation in the Democratic Republic of Congo through user fee subsidisation: a before and after study. BMC Health Serv Res. 2014;14(1):504.
Steinhardt LC, Rao KD, Hansen PM, Alam S, Peters DH. The effects of user fees on quality and utilization of primary health-care services in Afghanistan: a quasi-experimental health financing pilot study in a post-conflict setting. Int J Health Plann Manag. 2013;28(4):e280–e97.
Bonfrer I, Van de Poel E, Van Doorslaer E. The effects of performance incentives on the utilization and quality of maternal and child care in Burundi. Soc Sci Med. 2014;123:96–104.
Burnham GM, Rowley EA, Ovberedjo MO. Quality design: a planning methodology for the integration of refugee and local health services, West Nile, Uganda. Disasters. 2003;27(1):54–71.
Bilimoria KY. Facilitating quality improvement: pushing the pendulum back toward process measures. Jama. 2015;314(13):1333–4.
Sanson-Fisher RW, Bonevski B, Green LW, D’Este C. Limitations of the randomized controlled trial in evaluating population-based health interventions. Am J Prev Med. 2007;33(2):155–61.
Waldman RJ, Toole MJ. Where is the science in humanitarian health? Lancet. 2017;390(10109):2224–6.
Berendes S, Lako RL, Whitson D, Gould S, Valadez JJ. Assessing the quality of care in a new nation: South Sudan's first national health facility assessment. Tropical Med Int Health. 2014;19(10):1237–48.
Erismann S, Gürler S, Wieland V, et al. Addressing fragility through community-based health programmes: insights from two qualitative case study evaluations in South Sudan and Haiti. Health Res Policy Syst. 2019;17(1):20.
Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6(1):54.
Culbert H, Tu D, O’Brien DP, et al. HIV treatment in a conflict setting: outcomes and experiences from Bukavu, Democratic Republic of the Congo. PLoS Med. 2007;4(5):e129.
Acknowledgements
I would like to thank Thomas D’Aunno, Margaret E. Kruk, and Diana R. Silver for their continued support.
Funding
There was no funding source for this study.
Author information
Authors and Affiliations
Contributions
KJ and BR conceived and designed the review. KJ completed the search, KJ and TL screened the results, and KJ completed the data extraction. KJ wrote the first draft with subsequent input from TL and BR. The author(s) read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Jordan, K., Lewis, T.P. & Roberts, B. Quality in crisis: a systematic review of the quality of health systems in humanitarian settings. Confl Health 15, 7 (2021). https://doi.org/10.1186/s13031-021-00342-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13031-021-00342-z