Current HSPA approaches may make it difficult for policy-makers to prioritize action

The selection of indicators is often determined by the availability of data at the national and international levels, rather than by relevance and usefulness for health policy-makers. This can lead to comprehensive but cumbersome HSPA exercises which, however, may lack context, analysis or interpretation. While data availability is a prerequisite, this focus on relevance is crucial because extensive HSPA exercises often leave policy-makers with an overwhelming amount of information that does not help to identify priorities for health system strengthening and says nothing about what possible policy options might have the optimal impact on health system transformation. This is true even for well conducted health system performance assessments that collect high-quality data. Without clarity on the policy objectives or their implications for health system reform, policy-makers cannot identify the options for addressing them or monitor progress of the key performance issues. Establishing a set of key tracer indicators that speak to health system performance at a broad level, organizing them around the concrete policy questions and providing clear guidance on how to interpret them may help policy-makers “see the wood for the trees”.

Tracer indicators can support policy-makers to assess and address health systems performance

Health system tracer indicators are specific high-level metrics that are indicative of broader aspects of the health system’s functioning and can be used to “trace” (identify and assess) health system performance. They help policy-makers to pinpoint performance issues, understand them in the national context, identify areas for improvement, and monitor the impact of policies and interventions at the health system level, as well as to track progress over time. The idea of tracer indicators then is not just to make the volumes of data more manageable. They are an opportunity to shift the focus of performance assessment from very particular inputs and “micro” processes within a particular health system function to a broader perspective that looks at the system level and at how the health system addresses its intermediate objectives and final goals.

HSPA Frameworks allow policy-makers to see performance linkages between areas measured by indicators

The tracer indicators selected in this brief sit within existing HSPA frameworks, such as the WHO-Observatory global HSPA framework (Papanicolas et al., 2022) or the OECD renewed HSPA framework (OECD, 2024). These frameworks serve as blueprints that link the performance of various discrete health system functions to performance of the health system as a whole, as demonstrated in Health at a Glance (OECD, 2023a). Crucially, these HSPA frameworks can be used to track indicators across functions, subfunctions, assessment areas and health system goals through performance linkages, maintaining the policy-makers’ vantage point on wider health system performance issues.

The WHO-Observatory global HSPA framework and the OECD HSPA framework are aligned and complementary in that they build on common elements and either can be used to assess health systems performance in the context of current pressing health system and broader societal challenges (see Appendices 1 and 2). The former provides detailed assessment areas and highlights dynamics of performance pathways between health system functions and goals. The latter emphasizes the key policy elements, showing broader categories and the main relationships that can be zoomed in as needed. Common elements appear in both frameworks, with different levels of emphasis.

How does the WHO-Observatory global HSPA framework work?

The WHO has played a pivotal role in health system performance assessment for several decades. Notably, its landmark World Health Report 2000 (WHO, 2000) laid the foundation for a comprehensive framework that specifically linked health system inputs to the overarching goals of the health system, providing a clear methodology for their measurement. An integral element of the 2000 report was the incorporation of “responsiveness” as a fundamental health system goal, positioned alongside the traditional objectives of health improvement and financial protection, thereby bringing in not only “objective” metrics into performance thinking but also subjective and experiential aspects.

The WHO-Observatory global HSPA framework evolved upon this foundation, expanding and refining it two decades later (Papanicolas et al., 2022). In 2023 the WHO-Observatory framework has been renewed (Rajan et al., 2023); it signposts performance assessment also in light of pandemics, conflict, climate change and other recent major developments. This has resulted in a decided emphasis on environmental sustainability, digital health, data governance, people-centredness, and participation. A key novel element is the elaboration of societal goals, and the recognition of the health system’s contribution to them.

In this framework (see Fig. 1.1), each of the four health system functions from the 2000 report (governance, resource generation, financing, service delivery) has been delineated into subfunctions (see Table 1.1). In addition, the linkages between inputs (functions and subfunctions) and outcomes (health system goals) are fleshed out, distinguishing between intermediate objectives and final goals. The intermediate objectives are direct outcomes of the service delivery function (quality, with its subdimensions of effectiveness, safety, and user experience, and access) while the final goals (health improvement, people-centredness, financial protection) are achieved at the systems level principally through the intermediate objectives. Equity and efficiency traverse both intermediate objectives and final goals, depending on whether the analysis is at the service delivery or health system level. All of these health system goals contribute to societal well-being, described through the paradigm of sustainability, which considers a balance of economic, social and environmental domains in pursuit of improved well-being. ‘Economic’ relates to sustainable economic development, ‘social’ to social cohesion and ‘environment’ to environmental sustainability.

Figure 1.1

WHO-Observatory global HSPA framework. Source: Rajan et al (2023)

Table 1.1

Functions and subfunctions of the WHO-Observatory global HSPA framework.

Central to this framework remains its prioritization of people and patients through their inclusion in a governance subfunction (population and civil society participation), as an outcome of service delivery quality (user experience), and a final goal (people-centredness). The concept of people-centredness is seen as a twenty-first-century version of the responsiveness concept, i.e. a measure of how far the system is perceived as responsive to people’s and patients’ needs and views.

The renewed WHO-Observatory framework also places a nuanced focus on governance, an aspect often overlooked in indicator development and health system assessments. Governance is crucial because it offers policy-makers a potent lever from within the health system to go outside the system (depicted in a dotted line in Fig. 1.1), to collaborate across sectors, and to act on the health determinants. A nuanced view on governance can also be seen in the differentiation between governance linked to other functions (for example, health workforce planning, service delivery facility management) and system-level governance issues (human rights legislation, national health policies) (see Table 1.1).

Fig. 1.2 illustrates the application of the WHO-Observatory global HSPA framework and some of the performance pathways. Here, the novel element of assessment areas adds value by defining key characteristics which are essential for gauging the performance of the functions as well as their broader impact on overall health system performance. They are linked to each of the subfunctions and functions and are measurable through relevant indicators (see Appendix, Fig. A1.1). A policy question can thus be anchored to a concrete subfunction or an assessment area, with the framework guiding the user to root causes of the policy issue (within the functions) or towards the impact of the policy issue on system performance (objectives and goals). More importantly, visualizing the linkages helps to anchor the framework in a policy-maker’s day-to-day reality, helping to craft a viable policy solution.

How does the OECD renewed HSPA framework work?

For three decades, the OECD has helped countries to identify the key principles of high-performing health systems and to assess health system performance based on internationally comparable health indicators. This work has contributed to, and been guided by, the development of conceptual frameworks for health system performance developed by the OECD over time (Hurst & Jee-Hughes, 2001; Kelley & Hurst, 2006; Carinci et al., 2015). The original OECD framework was last updated in 2015, with a focus at the time on quality and outcomes. Since then, health systems have been called upon to become more centred around people’s needs and expectations, with enduring challenges pressing them on a daily basis. All of this has fostered a major shift in the broader health policy context, and expectations of what a health system performance assessment framework should measure have evolved substantially.

The OECD renewed HSPA framework is also in the final stages of adoption (Fig. 1.3). It places people at the centre of health systems and incorporates new key health system objectives such as sustainability (from both the economic and environmental perspectives) and resilience. It also stresses more clearly the interconnectedness and potential trade-offs across different health systems dimensions (such as balancing efficiency and equity or sustainability and resilience).

The renewed OECD HSPA framework focuses on the health system and does not serve as a conceptual model for determinants of health but acknowledges that the overall context plays a significant role in shaping the functions of all health systems, by either facilitating or restricting their performance.

In the framework, the outcomes of health systems correspond to the consequences of a health system’s activities, policies, and interventions on the health and well-being of the population. The framework places people’s health needs and preferences at the core of health systems and as such, people-centredness is regarded as an objective of health systems, as well as being instrumental to achieving other policy objectives. The section on health systems resources and characteristics covers the “structural” elements of health systems, i.e. the inputs necessary to enable the health system to function and the context in which it operates. It includes the following six pillars:

• expenditure and financing;
• workforce;
• technologies and pharmaceuticals;
• governance;
• data and digital; and
• knowledge and innovation.

The framework’s section on healthcare services and public health interventions includes all activities that fall under healthcare, such as curative care, long-term care, mental health care, and palliative care, while also including prevention and health promotion interventions, such as screening, vaccination and public health campaigns.

The renewed OECD framework also includes four “cross-cutting” dimensions of health system performance, namely efficiency and equity on one side, and sustainability and resilience on the other. The reason that these are cross-cutting is that they do not belong to one particular block in the framework but relate to them all.

Some relations between concepts are acknowledged in the renewed framework. For example, although different terminology is used, Donabedian’s model of structure, process, and outcomes (Donabedian, 1988) remains visible in the framework through the relationships between health system resources, characteristics, and policy (structure); healthcare services and public health interventions (process); and individual and population health (outcomes). Yet the framework remains high level. It shows the main elements in relation to each other at a higher level and is not intended to detail all possible conceptual relationships.

Figure 1.2

WHO-Observatory global HSPA framework (expanded version). Source: Rajan et al (2023)

Figure 1.3

2024 renewed OECD HSPA framework. Source: OECD (2024)

A policy brief as a proof of concept and a step towards a performance dashboard

There is scope to populate the HSPA frameworks with tracer indicators but this is a major undertaking that requires experts from the different strands to agree which indicators matter most and what they mean. As a proof of concept (see Box 1.4), this policy brief focuses on selected policy questions within the workforce and digital health subfunctions and the health system goal of people-centredness. These areas were chosen in light of the lessons from the COVID-19 pandemic (Sagan et al., 2021; OECD, 2023b), which highlighted the critical importance of the healthcare workforce, the system-level transformation brought by the rapid uptake of digital health, and the importance of people-centred systems in order to gain people’s trust. Access and quality were chosen to examine the high-level performance of service delivery and interaction of all health system functions and to illustrate how health system tracer indicators can be used to identify and map system-level performance and issues.

Box 1.4

Scope of this brief: a summary.

This policy brief has sought to identify key performance indicators that work as tracers for the healthcare workforce, digital health, people-centredness, and access and quality as outcomes of service delivery (primarily using literature review and WHO and OECD expert groups and working parties as described in Box 1.5). This document is in coherence with the programmes and outcomes of the WHO’s Tallinn Charter 15^th Anniversary Health Systems Conference: Trust and Transformation – Resilient and Sustainable Health Systems for the Future and the OECD’s Health Ministerial Meeting: Better Policies for More Resilient Health Systems. Its ambition is to support the future development of a HSPA dashboard of indicators that will offer policy-makers easy access to key information including guidance on interpretation, and a clear sense of health systems’ relative strengths and weaknesses (across the system and in comparison to other countries).

Box 1.5

Policy questions and key performance indicators: an approach and methods.

References

• Carinci F, et al. OECD Health Care Quality Indicators Expert Group. Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators. Int J Qual Health Care. 2015;27(2):137–46. Epub 2015 Mar 10. [PubMed: 25758443] [CrossRef]
• Donabedian A. The quality of care: How can it be assessed? JAMA. 1988;260(12):1743–8. [PubMed: 3045356]
• Hurst J, Jee-Hughes M. Performance Measurement and Performance Management in OECD Health Systems. Paris: OECD Publishing; 2001. OECD Labour Market and Social Policy Occasional Papers, No. 47. Available at: https://doi​.org/10.1787/788224073713 (accessed 5 December 2023)
• Kelley E, Hurst J. Health Care Quality Indicators Project: Conceptual Framework Paper. Paris: OECD Publishing; 2006. OECD Health Working Papers, No. 23. Available at: https://doi​.org/10.1787/440134737301 (accessed 5 December 2023)
• Kessner DM, Kalk CE, Singer J. Assessing health quality–the case for tracers. N Engl J Med. 1973;288(4):189–94. [PubMed: 4682231]
• Nolte E, Bain C, McKee M. Diabetes as a tracer condition in international benchmarking of health systems. Diabetes Care. 2006;29(5):1007–11. [PubMed: 16644629]
• OECD. Measuring Health Care, 1960–1983: Expenditure, Costs and Performance. Paris: OECD Publishing; 1985.
• OECD. Health at a Glance 2023: OECD Indicators. Paris: OECD Publishing; 2023a. Available at: https://doi​.org/10.1787/7a7afb35-en (accessed 5 December 2023)
• OECD. OECD Health Policy Studies. Paris: OECD Publishing; 2023b. Ready for the Next Crisis? Investing in Health System Resilience. Available at: https://doi​.org/10.1787/1e53cf80-en (accessed 5 December 2023)
• OECD. Rethinking health system performance assessment: a renewed OECD Framework. Paris: OECD Publishing; 2024.
• OECD/European Union. Health at a Glance: Europe 2022: State of Health in the EU Cycle. Paris: OECD Publishing; 2022. Available at: https://doi​.org/10.1787/507433b0-en (accessed 5 December 2023)
• Papanicolas I, Smith P. Health system performance comparison: an agenda for policy, information and research. Basingstoke: Open University Press; 2013. Available at: https:​//eurohealthobservatory​.who.int/publications​/m/health-system-performance-comparison-an-agenda-for-policy-information-and-research (accessed 5 December 2023)
• Papanicolas I, Rajan D, Karanikolos M, Soucat A, Figueras J. Health system performance assessment: a framework for policy analysis. Copenhagen: WHO Regional Office for Europe (on behalf of the European Observatory on Health Systems and Policies); 2022. Available at: https:​//eurohealthobservatory​.who.int/publications​/i/health-system-performance-assessment-a-framework-for-policy-analysis (accessed 5 December 2023) [PubMed: 37023239]
• Rajan D, et al. Health system performance assessment: renewing the global framework to guide post-pandemic policy-making. Copenhagen: WHO Regional Office for Europe (on behalf of the European Observatory on Health Systems and Policies); 2023.
• Rechel B, Maresso A, van Ginneken E. Health systems in transition: template for authors. Copenhagen: WHO Regional Office for Europe (on behalf of the European Observatory on Health Systems and Policies); 2019. Available at: https://iris​.who.int​/bitstream/handle/10665​/333262/HiT-template-2019-eng​.pdf?sequence=1 (accessed 5 December 2023)
• Sagan A, Webb E, Azzopardi-Muscat N, de la Mata I, McKee M, Figueras J. Health systems resilience during COVID-19: Lessons for building back better. Copenhagen: WHO Regional Office for Europe (on behalf of the European Observatory on Health Systems and Policies); 2021. Available at: https://iris​.who.int​/bitstream/handle/10665​/348493/9789289051873-eng​.pdf?sequence=2 (accessed 5 December 2023) [PubMed: 37023237]
• Smith P, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health systems improvement: experiences, challenges and prospects. Cambridge: Cambridge University Press; 2009. Available at: https:​//eurohealthobservatory​.who.int/publications​/m/performance-measurement-for-health-system-improvement-experiences-challenges-and-prospects (accessed 5 December 2023)
• UHC 2030 Technical Working Group. Health systems assessment technical working group. 2023. Available at: https://www​.uhc2030.org​/what-we-do/improving-collaboration​/technical-working-groups​/health-systems-assessment-technical-working-group/ (accessed 5 December 2023)
• WHO. World Health Report 2000: Health systems: improving performance. Geneva: World Health Organization; 2000. Available at: https://www​.who.int/publications​/i/item/924156198X (accessed 5 December 2023)
• WHO Regional Office for Europe. Health and care workforce in Europe: time to act. Copenhagen: Regional Office for Europe; 2022. Available at: https://www​.who.int/europe​/publications/i/item/9789289058339 (accessed 5 December 2023)
• WHO Regional Office for Europe. Digital Health in the WHO European Region: the ongoing journey to commitment and transformation. Copenhagen: WHO Regional Office for Europe; 2023. Available at: https://www​.who.int/europe​/publications/m​/item/digital-health-in-the-who-european-region-the-ongoing-journey-to-commitment-and-transformation (accessed 5 December 2023)