Antimicrobial resistance (AMR) constitutes a serious risk to health worldwide with major economic impacts. Bacteria carrying resistance determinants can be present in humans, animals and the environment and spread resistance genes between them. This needs to be considered when designing integrated surveillance in AMR and antimicrobial use (AMU) surveillance programmes to identify emerging resistance genes and monitor trends of resistance, understand the AMR epidemiology, and plan and evaluate policies for AMU and AMR reduction. International organisations have called for collaboration across sectors and adoption of a “One Health” approach.
In order to maximize the effectiveness and efficiency of surveillance systems for AMU and AMR and to inform surveillance stakeholders on best practices for integrated surveillance, surveillance systems need to be regularly evaluated.
AMU/AMR surveillance evaluation goals can include, among others:
- Identify strengths and weaknesses, challenges and opportunities
- Contribute to identifying actionable recommendations
- Provide an external view of the current surveillance strategy
- Monitor the relevance of surveillance with regards to evolving issues
- Certify the functioning of surveillance to gain the trust of international partners or national citizens
- Evaluate the efficiency of the surveillance system
In the case of AMR, surveillance evaluation should be an integral part of the global efforts for its control. National action plans (NAP) for AMR contemplate the deployment of surveillance systems to monitor the plans’ functioning and impact, inform their improvement and resource allocation decisions. In the international context of AMR surveillance and control, evaluations also appear as a means for international benchmarking, for self-assessment and direction of national and international efforts.
Links to selected publications:
Aenishaenslin, Cécile, Barbara Häsler, André Ravel, Jane Parmley, Katharina Stärk, et David Buckeridge. « Evidence needed for antimicrobial resistance surveillance systems ». Bulletin of the World Health Organization 97, no 4 (1 avril 2019): 283‑89. https://doi.org/10.2471/BLT.18.218917.
Evaluation of integrated surveillance systems for AMU and AMR
Definition:
We use the term integrated surveillance of AMU and AMR in the context of One Health to refer to surveillance that is based on a systemic, cross-sectoral, multi-stakeholder perspective to inform mitigation decisions with the aim to keep antimicrobials effective for future generations.
Several integrated surveillance strategies already exist around the world with different approaches and components. For example, different collaborative strategies between relevant organizations and stakeholders can be undertaken at the country level. Moreover, integration can be increased by sampling from different animal species, production types, or different points along the food chain. Integration can also be increased by harmonizing sample and data analyses methods across sectors to enhance the comparability of the information produced; by centralizing surveillance operations related to data collection across one or more institutions; by integrating all surveillance information into a joint report; and even by increasing the multidisciplinary nature of the network that is reached by these dissemination products. An overview of the different dimensions of integration within surveillance systems for AMR and AMU can be found in Aenishaenslin et al. 2019.
The implementation of integrated surveillance for AMU and AMR is a central recommendation of the World Health Organization (WHO) Global action plan on antimicrobial resistance.
Why is evaluating integrated surveillance for AMR different from evaluating other surveillance programmes or systems?
A major goal of integrated AMU and AMR surveillance is to combine information from humans, animals and ecosystems to guide risk management decisions while using resources efficiently. Its systemic, cross-sectoral and multi-stakeholder nature necessitates a collaborative approach that is not present in conventional domain-specific surveillance systems. The barriers and facilitators to integration and collaboration necessitate the evaluation of governance and operational attributes, which complement more conventional surveillance evaluation attributes such as functional, performance or value attributes. The latter are well described in surveillance systems evaluation frameworks commonly used in animal health and public health.
Also, the evaluation needs and objectives for integrated surveillance systems may be different from evaluation objectives for a non-integrated (e.g. uni-sectoral) surveillance system. For example, the goal of the evaluation could target the evaluation of the collaboration of different stakeholders or of other points of integration in the system.
Here are examples of evaluation questions that are specific to integrated surveillance for AMU and AMR:
-
-
Does the integrated surveillance system produce data that are comparable between sectors?
-
What is the capacity of the integrated surveillance system to detect links between trends of and “hotspots” for AMR in different animal species (including animal-derived foods) and in human populations?
-
What is the capacity of the integrated surveillance system to detect links between AMU and AMR in different species?
-
How balanced is the contribution of different sectors to AMU and AMR surveillance in terms of funding and resources?
-
How are the resources and expertise shared between sectors in relation to the workload and the potential risks and benefits?
-
How well do the different sectors communicate the surveillance information to relevant stakeholders?
-
Is collaboration across stakeholders relevant and effective to achieve the integrated surveillance objective?
-
How is the surveillance information used to develop and assess the effectiveness of interventions?
-
What is the added value of implementing an integrated surveillance approach?
-
Which agency coordinates this integrated surveillance for AMU and AMR at the national level?
-
The figure below represents a conceptual model for an integrated surveillance system for AMU and AMR (adapted from Aenishaenslin et al., 2019). It shows the expected outputs and outcomes of such a system. With regards to this model, five evaluation levels could be considered by stakeholders who would like to perform an evaluation of the surveillance system:
LEVEL 1 – One Health integration
LEVEL 2 – Production of information and expertise
LEVEL 3 – Knowledge generation
LEVEL 4 – Decision-making
LEVEL 5 – Impacts
Currently, no tool covers all evaluation levels described in this model. By clicking on each level, it is possible to see the evaluation tools that may be relevant to explore if you are interested in a specific one or few evaluation levels presented here.
The 5 Evaluation Levels
Conceptual model for evaluation of integrated surveillance for AMU and AMR
Level 1 evaluation aims at evaluating OH integration across the different components of the surveillance system. This includes the evaluation of the collaboration between stakeholders and organisations, shared governance, as well as integration of different OH components from human, animal, plant and environmental health in surveillance activities (planning and design, data collection, analysis, interpretation and dissemination).
Evaluation tools that can be interesting to explore at this level are:
Level 2 evaluation aims to assess the capacity of the surveillance system to produce integrated information and expertise. Three main outputs can be evaluated at this level: the performance of the One Health surveillance team (the operational team), the performance of the One Health network of stakeholders that are united by the surveillance system, and the capacity of the system to produce up-to-date OH information on AMR.
Evaluation tools that can be interesting to explore at this level are:
Level 3 evaluation aims to evaluate how combining the OH information and its interpretation by the OH team and its OH network contributes to the generation of new and actionable knowledge on AMR at the human-animal-environment interface. Three aspects are particularly important at this stage: 1) an evaluation of how the integration in the system enhances the capacity of the system to detect trends and correlations in AMR circulating between humans, animals and the environment, 2) an evaluation of how the integration in the system enhances the capacity to explain trends and correlations in AMR circulating between humans, animals and the environment, and then lead to a better understanding of the epidemiology of this complex problem and 3) an evaluation of the capacity of the system to increase the awareness of relevant stakeholders (actionable knowledge).
Evaluation tools that can be interesting to explore at this level are:
Level 4 evaluation aims at evaluating how the knowledge generated by the integrated system is used for decision-making. This includes evaluating if and how the knowledge generated is translated into direct interventions (e.g., sanctions if higher levels of antimicrobial use are detected in a particular animal or human setting). It also involves two other types of decisions that should be evaluated: 1) if and how the system is able to influence policy changes (for example restricting the use of some types of antimicrobials in agriculture by public or private institutions), and 2) if and how the system is able to influence individual behaviours that are not necessarily linked to formal policy changes (eg. changes in prescribing practices by veterinarians).
Evaluation tools that can be interesting to explore at this level are:
Level 5 evaluation aims to evaluate the impacts of the decisions that can be attributed to OH integration in the surveillance system, including impacts on AMR reduction, and their consequences on human, animal and ecosystem health. It is also at this level that the economic impacts of the whole mitigation system can be evaluated.
Evaluation tools that can be interesting to explore at this level are: