6Discussion and conclusions

HSPA is pivotal for policy-makers, as it provides a clear understanding of how the health system works and how it can be improved. However, HSPA often relies on a plethora of metrics that reduces the relevance for decision-making. This brief is a proof of concept for a policy-friendly HSPA dashboard. It tests how a subset of tracer indicators could be used to understand and address practical policy questions. The brief builds on the recently revised WHO-Observatory global HSPA framework and the OECD renewed HSPA framework (Rajan et al., 2023; OECD, 2024).

The brief reflects the agendas of both the Tallinn Charter 15^th Anniversary Health Systems Conference: Trust and Transformation – Resilient and Sustainable Health Systems for the Future and the OECD’s Health Ministerial Meeting: Better Policies for More Resilient Health Systems. It uses four health system policy entry points that are key for resilience – workforce, digital health, people-centredness, and service delivery. They contain a set of policy questions that have been used to test how key metrics can support decision-makers. The brief operationalizes both the WHO-Observatory and the OECD frameworks, and uses them to navigate through certain health system functions, subfunctions, assessment areas, and health system goals, and the causal links between them, to trace the causes of poor system performance and identify possible responses. It serves as the starting point for a development of a policy-friendly HSPA dashboard with carefully curated indicators informing and supporting policy action.

International work on HSPA has informed this brief. It draws on the overarching frameworks such as those of the WHO, the OECD, and the Observatory, as well as more “specific” frameworks that focus on parts of the health system or specific policy purposes. Health system assessment tools (for example, Rechel, Maresso & van Ginnekin, 2019) and efforts to collect and disseminate quantitative and qualitative information on health system performance (see OECD’s Health at a Glance and the European Observatory’s Health Systems in Transition series, OECD Health Statistics, WHO’s Health for All database, and the European Commission’s State of Health in the EU country health profiles) initiatives.

At the national level, HSPA processes have also generated key insights although they tend to encompass a wide array of indicators that are difficult to be collected at international level and result in reports that require extensive technical and analytical input. For example, the OECD has recently supported Czechia (OECD, 2023a) and Estonia (OECD, 2023b) in the development or updating of their national HSPA frameworks. Several other countries, including Belgium, Croatia and Ireland, have also recently updated their national HSPA frameworks. International HSPA work is not meant to replace such national processes, but rather to complement them. National HSPAs provide a nuanced, context-specific understanding of healthcare resources, needs, and outcomes. International HSPA work by its turn expands the field of vision to facilitate benchmarking, mutual learning, and cross-country collaboration for the identification of policy solutions that are effective to improve health system performance.

The brief recognizes the progress that has already been made in moving beyond measuring inputs (financial, physical, and human resources) to assessing health objectives and goals and takes a further step in linking indicators to health system functioning and policy options. Each of the four sections addresses a specific policy area. Rather than listing selected indicators, the sections are structured around practical policy questions. The workforce section, for example, concerns policies to ensure the delivery of high-quality and accessible health services and emphasizes health workforce shortages, skill-mix, and retention. The digital health section explores readiness, utilization, and digital health literacy, underscoring the importance of a focused assessment of the role of digital health in the health system. The people-centredness section addresses trust, participation, and enabling the co-production of healthcare because of the significance of health systems in meeting people’s diverse needs. The service delivery section tackles the broad question of healthcare accessibility and quality through high-level tracer indicators of avoidable mortality and unmet need, spotlighting outcomes. These are initial explorations but further investigation is needed to further understand causes of sub-optimal performance. Finally, the brief demonstrates how the notion of policy areas and corresponding tracer indicators can be operationalized through the WHO-Observatory and the OECD HSPA frameworks to become practical tools for mapping the causes of poor performance and identifying policy options.

The selection of tracer indicators is very much a work in progress. The authors consider various international and regional reports and datasets like the OECD’s Health at a Glance and Health Statistics, the WHO’s Health for All database, and others, identifying those with the best potential to reflect performance issues and provide a plausible narrative for the use and interpretation of these indicators. The brief is intended to initiate discussions on the best tracers for assessing specific health system functions and outcomes. It does not attempt to provide a definitive list of indicators. Instead, the examples, and the way their use is illustrated, move forward the understanding of how indicators for workforce, digital health, people-centredness, and service delivery can be mapped onto the WHO/Observatory and OECD frameworks and explores how they can be mobilized to address key policy issues. It shows how the use of these tracer indicators, in combination with the HSPA frameworks, can be an aid in assessing causes, revealing data gaps and limitations, identifying options, and monitoring change.

There is real scope for indicators to make a valuable contribution, as outlined above, but significant limitations have been identified and should be acknowledged, including issues of data availability, level of disaggregation, timeliness, comparability, and quality, particularly at the international level – as elucidated in the respective sections. In terms of data availability, the WHO European Region and the OECD have a number of key databases providing indicators on various aspects of health system performance. The OECD, the WHO Regional Office for Europe, and Eurostat cover different sets of countries and indicators, but work through the joint data collection questionnaire, with data typically sourced from the national sources, harmonized for cross-country comparisons and over time. Data disaggregation is limited in certain areas, such as the assessment of the various dimensions of health equity.

Timeliness is a particular concern for decision-makers. The data collection and harmonization process typically results in a two-year lag in data availability at the international level, which may extend even longer in certain areas. This is in contrast with the increasing policy needs for real-time data, as evidenced by the response to the COVID-19 pandemic (Sagan et al., 2021; OECD, 2023c). National data sources are necessary to address current health system challenges and monitor change, and they underpin international comparability and benchmarking, but the harmonization process takes time and depends on capacities of statistical teams in international organizations. Despite the overall high quality of data in Europe, relevant quality issues persist across the health sector. Even well regulated areas, such as death registration, face challenges, with some EU countries still reporting a substantial share (5% or more in 2020) of unspecific or ill-defined causes of death (European Commission, 2023).

A further limitation arises from the intrinsic nature of tracer indicators. They are reflections of specific health system areas, indicating whether there may be a systemic issue with the performance of that function. These indicators must be understood within the broader context of the status of the health system at present, but can also be followed over time. Moreover, insights gained from tracer indicators may need to be supplemented with more in-depth investigations into potential root causes of problems identified. It is crucial for policy-makers to fully comprehend these limitations to avoid any inadvertent misinterpretation.

Synthesizing the multitude of HSPA indicators into a smaller set with critical policy relevance is indeed to help decision-makers. The groundwork in this brief establishes an approach that has four complementary elements:

• Focusing on a targeted set of tracer indicators that reflect performance issues in different parts of the health system (supported by additional work to select and further develop appropriate tracers for health system functions and outcomes).
• Mapping indicators onto the WHO/Observatory and OECD HSPA frameworks, tracing links and causal pathways between functions, subfunctions, assessment areas, intermediate objectives, and final goals, making the performance dynamics of the entire health systems clear to policy-makers.
• Using policy-relevant questions as the way into assessment and framing performance and the metrics around practical concerns so that the focus is on the needs of policy-makers and can inform concrete policy actions.
• Providing a broader narrative for the meaning and scope of each indicator to prevent misinterpretation and to highlight gaps, limitations, and pathways for improved data collection and the development of new metrics.

This brief serves as a proof of concept and represents the first step in an evolutionary pathway towards policy-oriented HSPA: one that will facilitate health systems transformation and the achievement of their health and societal goals. There are four concrete lessons for health system leaders for future development in this field:

• Invest in addressing data issues: resources are needed in national systems and international organizations to enhance data collection and the resolution of ingrained data issues. Digitalization enables faster, more comprehensive, and higher-quality data collection but investment is crucial if systems are to develop capacity, and validate and deploy appropriate tools to facilitate efficient, timely, safe, and reliable information flows.
• Apply HSPA to practical policy questions: employing tracer indicators and mapping them onto the WHO/Observatory and OECD HSPA frameworks can assess system weaknesses and identify policy options. Expanding the methodology and its scope beyond the policy issues that are central to policy-makers to other health system areas will be a possible next step.
• Revise the existing health data body: a comprehensive review of existing health indicators would give a deeper understanding of definitions, use, and fitness for purpose, and identify data gaps. It would be particularly helpful to focus on the development of new indicators, not least to address the crucial data gaps identified in this brief. Reliable, routinely reported, and internationally comparable indicators on health workforce composition and migration, the health workers’ ability to provide digital health services, patients’ access to digital services, societal participation in health policy-making, and patients’ involvement in their own care all have huge policy relevance. Further work that goes beyond selecting from available indicators to explore innovative uses of existing indicators and to create or expand novel indicators would make tracer indicators more meaningful.
• Collaborate closely between organizations: collaboration among key international organizations and bodies working in this field, notably the WHO, the OECD, the EU and the European Observatory, to align methodologies and develop a shared dashboard of tracer indicators would support policy decision-making.