About this research group

We are active in international health systems research and in particular in health benefit package design, informed by health technology assessment (HTA).

Our activities have led to the development of ‘evidence-informed deliberative processes’ (EDPs), and its implementation in several countries, including Ghana, Indonesia, Iran, Kazachstan, Kyrgyzstan, Lebanon, Liberia, Pakistan, Rwanda, and Tanzania. We do this in close collaboration with ministries of health, other national authorities and local experts.

We also undertake studies to inform health policy decisions in specific areas, such as HIV/AIDS programming, maternal health, and scaling-up of surgery.

go to main content
Research Global health priorities

About this research group

We are active in international health systems research and in particular in health benefit package design, informed by health technology assessment (HTA).

read more

Research group leader

dr. Rob Baltussen

+31 (0)24 361 31 19
contact form

Aims

We support decision-makers in countries around the world in development and implemen-tation of legitimate processes for health policy making, in particular for health benefit package design and revision.

read more

Aims

We support decision-makers in countries around the world in the development and implementation of legitimate processes for health policy making, in particular for health benefit package design and revision.

Health benefit package design is an intrinsically complex and value-laden process, in which multiple criteria beyond cost-effectiveness are important. Stakeholders often justifiably disagree about the relative importance of these criteria. Yet, present theory and methods for priority setting do not reflect this, and are typically technocratic in nature. This risks to undermine the legitimacy of decisions.

We consider legitimacy, or perceived reasonableness of decisions, as the key challenge of priority setting in the coming decades. HTA organizations need to manoeuvre between the rapid development of new technologies and ever-expanding pressures on health budgets on the one hand, and strong stakeholder voices demanding access to these technologies on the other hand. HTA organizations are increasingly scrutinized on how they make decisions, including their processes and the evidence used.

We have developed evidence-informed deliberative processes (EDPs) for HTA-related organisations to improve the legitimacy of their benefit package design.

Guidance for HTA bodies

The use of evidence-informed deliberative processes.

Our products

Guides

  • The guide provides support to implement EDPs on the basis of practical questions structured around the HTA process.

    read more

    EDPs: a practical guide for HTA bodies

    The guide provides support to implement EDPs on the basis of practical questions structured around the HTA process (i.e.,  installing an advisory committee, defining decision criteria, selecting health technologies, assessment, appraisal, communication and appeal, and monitoring and evaluation).

    EDPs are presently employed by national health authorities in a range of low- and middle-income countries to design or revise their health their benefit packages. EDPs are recommended in the guidance on HTA of the World Health Organization.
     
    The guide can be downloaded here.

  • Making explicit choices on the path to UHC: the JLN health benefits package revision guide

    In recent decades, many LMICs have adopted a Health Benefits Package (HBP), which defines the coverage of services, the proportion of the costs that are covered, and who can receive these services under Universal Health Coverage. Once developed and implemented, a HBP should not be static as revisions ensure that the package is up-to-date and is delivered appropriately, and that available resources are used efficiently and wisely.

    The JLN Guide for Health Benefits Package Revision aims to support practitioners in low- and middle-income countries (LMICs) to revise their HBPs, while responding to changing disease burdens, fluctuating budgets, and the emergence of new services and health technologies, and to correct for implementation challenges.

    Co-produced by the JLN’s Efficiency collaborative, this Guide packages the experiences and expertise from 14 JLN member countries, including Bangladesh, Ethiopia, Ghana, India, Indonesia, Kenya, Laos, Malaysia, Mongolia, Nigeria, the Philippines, South Africa, Sudan and Vietnam.

    The guide can be downloaded here.

  • A distinct framework

    The EDP framework has a number of particular features, which distinguish it from other HTA frameworks:

    • EDPs consider HTA as a political and intrinsically complex undertaking, and has a strong emphasis on deliberation among relevant stakeholders. We have developed approaches that stimulate debate and deepen the argumentation. This is different from many other HTA frameworks which are often limited to evidence collection and synthesis.
    • The EDP framework combines existing theories and best HTA practices around the world – it is not proposing anything new. In terms of theory, EDPs are based on rational decision-making (as in multi-criteria decision analyses – MCDA) and fair decision-making (as in accountability for reasonableness – A4R).
    • The framework provides a practical stepwise approach that covers the whole HTA process (see Figure below). Other HTA frameworks typically concentrate on selected steps only, e.g. assessment.
    • EDPs take the decision-making context as the starting point, and offers specific advice depending on the level of HTA development. The EDP framework is also relevant for countries that have not (yet) established an HTA organization.  
    • EDPs can be considered as a practical application of the WHO ‘Fair choices’ framework by Norheim et al., in terms of the stepwise approach to achieve Universal Health Coverage (UHC).
    • WHO Headquarters recommends the use of EDPs for HTA organizations. See for example this document.
    • Our EDP guide is also recommended by other agencies; for example the International Decision Support Initiative (iDSI), a global network of health, policy and economic expertise, working to achieve Universal Health Coverage and the health Sustainable Development Goal (SDG 3). Our GHP group is part of the iDSI network.

    See the six steps of Evidence Informed Deliberative Processes in the Figure below.

EDP publications

Country work

  • We provide on-going support to the Ministry of Health on the institutionalisation of HTA in the country.

    read more

    EDPs in Ghana

    In Ghana, Gavin Surgey provides on-going support (through Access and Delivery Partnership/PATH) to the Ministry of Health on the institutionalisation of HTA in the country. Recent achievements include the development and approval of the country’s HTA process.

  • EDPs in Indonesia

    Together with researchers from Padjajaran University in Bandung, we have implemented EDPs to improve priority setting of HIV/AIDS health services to support the AIDS Commission in West Java province, Indonesia, in the development of its strategic plan for 2014-2018.

    Under the responsibility of the provincial AIDS Commission, an EDP was implemented to select priority interventions using six steps: (i) situational analysis; (ii) formation of a multistakeholder Consultation Panel; (iii) selection of criteria; (iv) identification and assessment of interventions' performance; (v) deliberation; and (vi) selection of funding and implementing institutions. An independent researcher conducted in-depth interviews (n = 21) with panel members to evaluate the process.

    The Consultation Panel included 23 stakeholders. They identified 50 interventions and these were evaluated against four criteria: impact on the epidemic, stigma reduction, cost-effectiveness and universal coverage. After a deliberative discussion, the Consultation Panel prioritised a combination of several treatment, prevention and mitigation interventions.

    The EDP improved both stakeholder involvement and the evidence base for the strategic planning process. EDPs fill an important gap which international guidelines and current tools for strategic planning in HIV control leave unaddressed.

    Publications:

  • EDPs in Iran

    With the support of the WHO country office in Tehran we are currently supporting the implementation of EDPs in Iran, in close collaboration with the High Council for Health Insurance (HCHI) of the Ministry of Health and Medical Education, to define the country’s Health Insurance Benefit Package, targeting some 30 conditions across six disease areas.

    Our activities include:

    • Support for the implementation of evidence-informed deliberative processes in the form of remote consultations with a local team of analysts;
    • Provide guidance on the application of decision criteria for periodic health benefits package revisions in Iran;
    • Provide guidance for the presentation of available evidence on interventions targeting specific diseases/conditions to local technical working groups for deliberation and appraisal;
    • Assist in developing and implementing an M&E framework for health insurance benefit package revision;
    • Provide guidance to institutionalise EDPs as part of periodic HIBP revision in Iran.

    Publication:

    Nouhi M, Baltussen R, Razavi SS, Bijlmakers L, Sahraian MA, Goudarzi Z, Farokhian P, Khedmati J, Jahangiri R, Olyaeemanesh A. The Use of Evidence-Informed Deliberative Processes for Health Insurance Benefit Package Revision in Iran. Int J Health Policy Manag. 2022;11(11):2719–26

  • EDPs in Kazakhstan

    In Kazakhstan we have been working with the Ministry of Health on employing EDPs to determine the State Guaranteed Benefit Package (SGBP) and the package of the Mandatory Social Health Insurance (MSHI), using HTA. The activities involved:

    • Undertaking a situational analysis based on desk research and surveying key stakeholders, including the Ministry of Health, Republican Center for Health Development and the Social Health Insurance Fund;
    • Formation of an advisory committee under the leadership of the Ministry of Health;
    • Definition of a relevant set of criteria for priority setting based on surveying of and a workshop with the advisory committee;
    • Selection and assessment of 25 interventions according to international standards;
    • Appraisal of the 25 interventions using quantitative MCDA and decision rules; and
    • Development of an implementation plan and M&E framework.

    Publication:

    Oortwijn W, Surgey G, Novakovic T, Baltussen R, Kosherbayeva L. The Use of Evidence-Informed Deliberative Processes for Health Benefit Package Design in Kazakhstan. Int J Environ Res Public Health. 2022;19(18):11412

  • EDPs in Lebanon

    We supported the National Social Security Fund (NSSF) to develop their HTA framework in the shape of a guidebook for HTA practitioners and health stakeholders who seek the reimbursement of their health technology by the NSSF. The initiative reflects the solid commitment of the NSSF to institutionalize HTA, and it constitutes a “stepping stone” for a longer journey.

    The practical guide explains the framework that will be adopted by the NSSF to assess the health technologies in question. It provides an overview of the HTA process based on the principles of EDPs, falling in line with the NSSF’s aspiration of optimizing the efficiency of the use of its limited resources, while ensuring equitable investment decisions.

  • EDPs in Liberia

    The Disease Control Priorities 3 (DCP3) Country Translation project is supporting selected low- and middle-income countries in setting priorities for essential packages of health services (EPHS), on the pathway to universal health coverage (UHC). In 2021, DCP3 initiated collaboration with the Liberia Ministry of Health (MoH) to provide technical assistance in revising the national EPHS. Radboudumc took responsibility for organising the prioritisation process of these revisions.

    The UHC EPHS prioritisation process, involving a wide array of stakeholders, led to the shortlisting of >100 high-priority interventions, using decision criteria that included cost-effectiveness of interventions, disease burden targeted, disability-adjusted life years (DALYs) averted, budget impact, financial risk protection, equity (targeting vulnerable population groups), and feasibility of implementation.

    The MoH now has the data, technical capacity, and the tools to adjust this package to fit any future policy and fiscal changes or address additional considerations that may emerge during implementation.

  • EDPs in Moldova

    Commissioned by the World Bank, Radboudumc support the government of Moldova to plan and implement the most suitable system for HTA, using EDPs as the guiding framework.

    This study falls under the Toward Universal Health Coverage Project which is a joint project implemented by the World Bank and financed by the Swiss Agency for Development and Cooperation (SDC). Our study run between November 2019 and December 2021 and had three main tasks: conducting a needs assessment in terms of stakeholders to involve in HTA development, identifying HTA capacity and skills, mapping HTA systems in European countries to draw lessons for Moldova, and then developing an implementation plan, including a roadmap for short- and long-term implementation.

  • EDPs in Pakistan

    In Pakistan we supported the implementation process of the Disease Control Priorities 3 (DCP3) project using an evidence-informed deliberative process. DCP3 responds to the increasing need of low- and lower-middle-income countries for technical guidance and support in benefit package design and in accelerating progress towards UHC.

    The EDP process was embedded in a broader institutional effort around health benefit packages in Pakistan, initiated by a joint WHO-EMRO and DCP3 secretariat mission visit to Pakistan in Jan 2019. The six steps of evidence-informed deliberative processes were operationalized for health service prioritization and UHC benefit package design, by supporting Pakistan’s Ministry of National Health Services, Regulation and Coordination with the necessary templates and guidance for the presentation of available evidence, stakeholder deliberation, appraisal and formulation of recommendations. A journal article describing the prioritization process will be published soon along with other papers that describe the defined service packages.

    Publications: under peer-review.

  • EDPs in Rwanda

    In Rwanda, we provide support to the Ministry of Health to revise the community-based health insurance (CBHI) benefit package. The project is implemented by iDSI in close collaboration with the School of Public Health of the University of Rwanda, the Rwanda Social Security Board, the World Health Organization, and Radboudumc. The project started in 2021 and evaluates cancer control strategies as a first cluster of conditions, with the aim to provide recommendations to the MoH on inclusion of cancer services in the CBHI cancer package. The evaluation of other clusters is planned afterwards. 

    The role of Radboudumc is to develop the decision-making process for benefit package design following the principles of EDPs. This involves leading on and contributing, through a series of online and onsite training activities, to the development of an appropriate governance structure, the establishment of an advisory committee, the selection of decision criteria and the organisation of appraisal meetings.

  • EDPs in Tanzania

    In Tanzania, Gavin Surgey provides on-going support (through ADP/PATH) to the Ministry of Health on the institutionalisation of HTA in the country and capacity building.

    Recent achievements include the development of their HTA process guidance EDPs as the guiding framework and, an analysis of selected medicines reimbursed by the National Health Insurance Service and ongoing capacity building for policy makers on HTA.

    Publication:

    • Surgey G, Mori AT, Baltussen R. Health technology assessment in Tanzania: capacity and experience of HTA committee members. Journal of Global Health Economics and Policy. 2022;2:e2022004. Doi: 10.52872/001c.33116

  • EDPs in Ukraine

    HTA is included in the Ukrainian Health Law fundamentals and became mandatory by 2020. SAFEMed has been supporting the Ministry of Health to integrate HTA into the decision-making ecosystem and continuous capacity development in HTA. We are collaborating with SAFEMed during 2022-2023 to:

    • create a model curriculum for an HTA master’s program, including EDPs, to be implemented in Ukraine;
    • review the capacity and requirements of the existing academic centres globally;
    • develop a list of minimum criteria for academic centres to deliver a successful HTA program;
    • perform a training needs assessment of users and doers of HTA via an online survey that captured level of experience and knowledge gaps;
    • develop a 20-weeks training program for users, doers and trainers based on the needs assessment; and
    • conduct and evaluate the online HTA training program for about 65 users, doers and trainers in Ukraine.

  • EDPs in United Arab Emirates

    We support Abu Dhabi’s Department of Health in the development and implementation of a roadmap for HTA, using EDPs. This roadmap, which was based on a situational analysis and developed in consultation with local policymakers, healthcare providers, payers, product makers and research institutions, was formally adopted at the end of 2022. It pays particular attention to creating appropriate structures and processes and to strengthening local capacity for improved and timely decision making regarding health insurance coverage. Financial support is being provided by Roche.

Other publications

  • Sorted on year and first author.

    read more

    Non-EDP publications

    2022

    • Gert Jan van der Wilt, Bart Bloemen, John Grin, Iñaki Gutierrez-Ibarluzea, Laura Sampietro-Colom, Pietro Refolo, Dario Sacchini, Bjørn Hofmann, Lars Sandman, Wija Oortwijn. Integrating empirical analysis and normative inquiry in health technology assessment: the values in doing assessments of health technologies approach. Int J Technol Assess Health Care 2022;38(1):e52. doi: 10.1017/S0266462321001768
    • Vassall, Anna; Sweeney, Sedona; Barasa, Edwine; Prinja, Shankar; Keogh-Brown, Marcus R; Tarp Jensen, Henning; Smith, Richard; Baltussen, Rob; M Eggo, Rosalind; Jit, Mark. Integrating economic and health evidence to inform Covid-19 policy in low- and middle- income countries. Wellcome Open Research 2022;5:272. doi: 10.12688/wellcomeopenres.16380.1
    • Gert Jan van der Wilt, Wija Oortwijn, on behalf of the VALIDATE-HTA Consortium (2022). Health technology assessment: A matter of facts and values. Int J Technol Assess Health Care 2022;38(1):e53:1-2. doi: 10.1017/S026646232200010100
    • Oortwijn W, Husereau D, Abelson J, Barasa E, Bayani D, Canuto V et al. Designing and Implementing Deliberative Processes for Health Technology Assessment – a good practices report of a joint HTAi/ISPOR task force. Value Health 2022 Jun;25(6):869-86. doi: 10.1016/j.jval.2022.03.018.
    • Oortwijn, W., & Sampietro-Colom, L. (Eds.). The VALIDATE handbook: an approach on the integration of values in doing assessments of health technologies. Version 2.0. DOI: 10.54195/CKHB1659.  - Radboud University Press, 2022.
    • Jip Janssen, Samuel Afari-Asiedu, Annelie Monnier, Martha Ali Abdulai, Theresa Tawiah, Heiman Wertheim, Rob Baltussen, Kwaku Poku Asante. Exploring the economic impact of inappropriate antibiotic use: the case of upper respiratory tract infections in Ghana. Antimicrob Resist Infect Control 2022;11:53. https://doi.org/10.1186/s13756-022-01096-w
    • John Grin, Bart Bloemen, Iñaki Gutierrez-Ibarluzea, Laura Sampietro-Colom, Pietro Refolo, Dario Sacchini, Bjørn Hofmann, Lars Sandman, Wija Oortwijn, Gert Jan van der Wilt. Learning and practicing more value-reflective, problem-setting health technology assessment: experiences and lessons from the VALIDATE project. Int J Technol Assess Health Care. 2022;38(1):e63. doi: 10.1017/S0266462322000204.
    • Pietro Refolo, Dario Sacchini, Bjørn Hofmann, Lars Sandman, Wija Oortwijn, Gert Jan van der Wilt, Bart Bloemen, John Grin, Iñaki Gutierrez-Ibarluzea, Laura Sampietro-Colom. Prioritization of COVID-19 vaccination: the added value of the “VALIDATE” approach. Health Policy. 2022;126(8):770-6.
    • O'Rourke B, Oortwijn W, Schuller T. Response to redefining health technology assessment: a comment on “The new definition of health technology assessment: a milestone in international collaboration”. Int J Technol Assess Health Care, 2022:38(1),e55:1–2. doi: 10.1017/S0266462322000344
    • Ifeanyichi M, Broekhuizen H, Adinan Juma, Kondo Chilonga, Edward Kataika, Jakub Gajewski, Ruairí Brugha, Leon Bijlmakers. Economic cost of providing district- and regional-level surgeries in Tanzania. Int H Health Pol Manag. 2022,11(7):1120-31. Doi 10.34172/ijhpm.2021.09
    • Judith Munthali, Chiara Pittalis, Leon Bijlmakers, John Kachimba, Mweene Cheelo, Ruairi Brugha, Jakub Gajewski. Challenges in introducing innovation. Barriers and enablers to the WHO Surgical Safety Checklist at the University Teaching Hospital in Lusaka, Zambia: a qualitative study. BMC Health Serv Res 2022;22:894. Doi:10.1186/s12913-022-08257-y
    • Zhang M, Gajewski G, Pittalis C, Shrime M, Broekhuizen H, Ifeanyichi M, Clarke M, Borgstein E, Lavy C, Drury G, Juma A, Mkandawire N, Mwapasa G, Kachimba J, Cheelo M, Mbambiko M, Chilonga K, Bijlmakers L, Brugha R. Surgical capacity, productivity and efficiency at the district level in sub-Saharan Africa – a three-country study. PLoS ONE 2012;17(11): e0278212. Doi:10.1371/journal.pone.0278212
    • Pittalis C, Brugha R, Bijlmakers L, Cunningham F, Mwapasa G, Clarke M, Broekhuizen H, Ifeanyichi MI, Borgstein E, Gajewski J. Using network and complexity theories to understand the functionality of referral systems for surgical patients in resource-limited settings, the case of Malawi. Int J Health Policy Manag. 2022;11(11):2502-13. Doi: 10.34172/ijhpm.2021.175
    • Broekhuizen H, Martilord Ifeanyichi, Gerald Mwapasa, Chiara Pittalis, Patrick Noah, Nyengo Mkwandawire, Erik Borgstein, Ruairi Brugha, Jakub Gajewski, Leon Bijlmakers. Improving access to surgery through surgical team mentoring – policy lessons from group model building with local stakeholders in Malawi. Int J Health Policy Manag. 2022;11(9):1744-55. Doi 10.34172/ijhpm.2021.78

    2021

    • Bloemen B, Jansen M, Rijke W, Oortwijn W, van der Wilt GJ. Mixed claims in Health Technology Assessment: The case of non-invasive prenatal testing. Soc Sci Med. 2021;270:113689 
    • Brian Godman, Andrew Hill, Steven Simoens, Gisbert Selke, Iva Selke Krulichová, Carolina Zampirolli Dias, Antony P Martin, Wija Oortwijn, et al. Potential approaches for the pricing of cancer medicines across Europe to enhance the sustainability of healthcare systems and the implications. Expert Rev Pharmacoecon  Outcomes Res. 2021;21(4):527-40.
    • Clarke M, Pittalis C, Borgstein E, Bijlmakers L, Cheelo M, Ifeanyichi M, Mwapasa G, Juma A, Drury G, Lavy C, Kachimba J, Broekhuizen H, Mkandawire N, Chilongo K, Brugha, R, Gajewski J. Surgical service monitoring and quality control systems at district hospitals in Malawi, Tanzania and Zambia – a mixed-methods study. BMJ Quality & Safety 2021;0-1-11. Doi 10.1136/bmjqs-2020-012751 
    • Ifeanyichi M, Broekhuizen H, Cheelo M, Juma A, Mwapasa G, Borgstein E, Kachimba J, Gajewski J, Brugha R, Pittalis C, Bijlmakers L. Surgical ambulance referrals in sub-Saharan Africa – Financial costs and coping strategies at district hospitals in Tanzania, Malawi and Zambia. BMC Health Services Research 2021;21(728).  Doi 10.1186/s12913-021-06709-5
    • Ifeanyichi M, Gajewski J, Baltussen R, Borgstein E, Kachimba J, Brugha R, Bijlmakers L. Covid-19 pandemic: Revisiting the case for a dedicated financing mechanism for surgical care in resource-poor countries – Viewpoint article. J Glob Health 2021;11:03090. Doi 10.7189/jogh.11.03090
    • Henk Broekhuizen, Martilord Ifeanyichi, Mweene Cheelo, Grace Drury, Chiara Pittalis, Etiënne Rouwette, Michael Mbambiko, John Kachimba, Ruairí Brugha, Jakub Gajewski, Leon Bijlmakers. Options for surgical mentoring: lessons from Zambia based on stakeholder consultation and systems science.   PLoS ONE 2021;16(9): e0257597. Doi 10.1371/journal.pone.0257597
    • Ifeanyichi M, Aune E, Shrime M, Gajewski J, Pittalis C, Kachimba J, Borgstein E, Brugha R, Baltussen R, Bijlmakers L: Financing of surgery and anaesthesia in Sub-Saharan Africa: A scoping review. BMJ Open 2021;11:e051617. Doi 10.1136/bmjopen-2021-051617
    • Gajewski J, Zhang M, Bijlmakers L, Pittalis C, Borgstein E, Mwapasa G, Kachimba J, Cheelo M, Waterman K, Brugha R. Rates of surgical deaths and infections at district hospitals in Malawi and Zambia, a prospective multi-centre cohort study. BMJ Open 2021(11):e049126. Doi  10.1136/bmjopen-2021-049126
    • Angelis A, Baltussen R, and Tervonen T. The Need for Novel Approaches in Assessing the Value of COVID-19 Vaccines. American Journal of Public Health 111, no. 2 (February 1, 2021): pp. 205-208.
    • Mueller D, Gutierrez-Ibarluzea I, Chiumente M, Oortwijn W. Towards a common understanding of competencies for HTA: Enhancing educational and training programs around the globe. International Journal of Technology Assessment in Health Care, 2021 37, e29, 1–10.

    2020

Connected to