Box 1.5Policy questions and key performance indicators: an approach and methods
The brief’s identification of key performance metrics starts by establishing a policy question and then selecting a set of tracer indicators that provide system-level information to address that question.
The notion of using “markers” or tracers in health systems is not novel, however, but dates back to Kessner, Kalk & Singer (1973), who advocated using tracer conditions (frequent, well defined conditions with known epidemiology that have agreed pathways of appropriate care) to assess the quality of health services. The concept was taken forward by Nolte, Bain & McKee (2006), using diabetes as a tracer condition in international benchmarking of health systems with a diabetes mortality-to-incidence ratio seen as a simple way of differentiating quality of care for people with diabetes in different countries. There was a clear understanding that tracers could not give precise measures of the scale of a problem but their value as indicators of potential problems and as prompts for further investigation was recognized.
This brief builds on previous thinking and aims to identify some tracer indicators for the key areas of workforce, digital health and people-centredness, as well as access and quality as outcomes of service delivery. The choices of these areas and the elements explored under those headings were based on several factors, including: a) policy priorities presented at the Tallinn Conference and the OECD’s 23 January 2023 Meeting of Health Ministers (which in turn were defined in consultation with an extensive range of policy-makers and other stakeholders); b) practical experience of policy-makers and health system transformation during and post-pandemic; c) the insights offered by the international initiatives engaged in the development of the HSPA frameworks (including UHC 2030 Technical Working Group), which highlighted how connections work across functions, subfunctions, assessment areas, and intermediate and final goals; and d) consideration was given to the availability of comparable data, albeit working on the premise that lack of comparable data alone should not preclude inclusion of an indicator that is relevant (rather, this should serve as a call for improved data collection).
The tracer indicators were identified based on a HSPA literature review, including examination of indicators presented in the OECD’s Health Statistics database and Health at a Glance and the WHO’s Health for All database and reports on the workforce (WHO Regional Office for Europe, 2022) and digital health (WHO Regional Office for Europe, 2023), expert consensus within WHO and OECD working groups, as well as wider consultation. The selected areas and tracer indicators are not exhaustive and the indicators are intended to offer a glimpse at performance, monitoring at system level, and benchmarking where possible. They are not seen as a tool to address all the specifics of the processes within health system functioning but rather as a starting point for identifying key methodological principles and shortcomings, and to present domains that need further analysis.
The brief serves as a proof of concept and establishes the foundations for further work on populating the HSPA frameworks with tracer indicators and on policy dashboards.
