How well do coverage surveys and programmatically reported mass drug administration coverage match? Results from 214 mass drug administration campaigns in 15 countries, 2008-2017
INTRODUCTION
Delivering preventive chemotherapy through mass drug administration (MDA) is a central approach in controlling or eliminating several neglected tropical diseases (NTDs). Treatment coverage, a primary indicator of MDA performance, can be measured through routinely reported programmatic data or population-based coverage evaluation surveys. Reported coverage is often the easiest and least expensive way to estimate coverage; however, it is prone to inaccuracies due to errors in data compilation and imprecise denominators, and in some cases measures treatments offered as opposed to treatments swallowed.
OBJECTIVE
Analyses presented here aimed to understand (1) how often coverage calculated using routinely reported data and survey data would lead programme managers to make the same programmatic decisions; (2) the magnitude and direction of the difference between these two estimates, and (3) whether there is meaningful variation by region, age group or country.
METHODS
We analysed and compared reported and surveyed treatment coverage data from 214 MDAs implemented between 2008 and 2017 in 15 countries in Africa, Asia and the Caribbean. Routinely reported treatment coverage was compiled using data reported by national NTD programmes to donors, either directly or via NTD implementing partners, following the implementation of a district-level MDA campaign; coverage was calculated by dividing the number of individuals treated by a population value, which is typically based on national census projections and occasionally community registers. Surveyed treatment coverage came from post-MDA community-based coverage evaluation surveys, which were conducted as per standardised WHO recommended methodology.
RESULTS
Coverage estimates using routine reporting and surveys gave the same result in terms of whether the minimum coverage threshold was reached in 72% of the MDAs surveyed in the Africa region and in 52% in the Asia region. The reported coverage value was within ±10 percentage points of the surveyed coverage value in 58/124 of the surveyed MDAs in the Africa region and 19/77 in the Asia region. Concordance between routinely reported and surveyed coverage estimates was 64% for the total population and 72% for school-age children. The study data showed variation across countries in the number of surveys conducted as well as the frequency with which there was concordance between the two coverage estimates.
CONCLUSIONS
Programme managers must grapple with making decisions based on imperfect information, balancing needs for accuracy with cost and available capacity. The study shows that for many of the MDAs surveyed, based on the concordance with respect to reaching the minimum coverage thresholds, the routinely reported data were accurate enough to make programmatic decisions. Where coverage surveys do show a need to improve accuracy of routinely reported results, NTD programme managers should use various tools and approaches to strengthen data quality in order to use data for decision-making to achieve NTD control and elimination goals.
Related studies
Alternative approaches to coverage surveys (Burkina Faso)
Compare coverage evaluation methods to identify a method that is statistically rigorous and feasible for programs. This study will focus on assessing MDA coverage for lymphatic filariasis by comparing the cost, time and feasibility of 3 different methods: the EPI approach (n=1768), LQAS design (n=95) and probability sampling alternatives (n=1768).
Primary Findings and Lessons Learned
Coverage surveys are an important tool for programs to evaluate their reporting systems and to determine whether effective MDA coverage has been achieved. However, for various reasons coverage surveys are seldom implemented. Some key challenges are: perceived technical difficulty, lack of resources, and lack of standardized guidance on how to conduct coverage surveys. This protocol seeks to address the 1st and 3rd points by comparing the feasibility of three different coverage survey methods (EPI approach, LQAS, and segmentation). This study was completed in 3 districts in Burkina Faso. All 3 districts found that their survey coverage was above the WHO target threshold (65% for LF). Furthermore, in all 3 cases the survey coverage validated (or nearly validated) the reported coverage. Taken together this suggests that the Burkina Faso program is working well. The feasibility results found all 3 methods to be very similar with regards to time, cost and perceived difficulty. Because only the segmentation approach results in a probability sample, this method was recommended by the M&E Working Group and ultimately approved by the STAG. Since the approval, significant work has been underway to create guidelines for conducting coverage surveys for preventive chemotherapy. An excel tool was created to improve the usability of the tool and online learning modules are currently in the works.
Compare coverage evaluation methods to identify a method that is statistically rigorous and feasible for programs. This study will focus on assessing MDA coverage for lymphatic filariasis by comparing the cost, time and feasibility of the EPI approach (n=1768), LQAS design (n=95) and probability sampling alternatives (n=1768).
Primary Findings and Lessons Learned
Coverage surveys are an important tool for programs to evaluate their reporting systems and to determine whether effective MDA coverage has been achieved. However, for various reasons coverage surveys are seldom implemented. Some key challenges are: perceived technical difficulty, lack of resources, and lack of standardized guidance on how to conduct coverage surveys. This protocol seeks to address the 1st and 3rd points by comparing the feasibility of three different coverage survey methods (EPI approach, LQAS, and segmentation). This study was completed in 3 districts in Burkina Faso. All 3 districts found that their survey coverage was above the WHO target threshold (65% for LF). Furthermore, in all 3 cases the survey coverage validated (or nearly validated) the reported coverage. Taken together this suggests that the Burkina Faso program is working well. The feasibility results found all 3 methods to be very similar with regards to time, cost and perceived difficulty. Because only the segmentation approach results in a probability sample, this method was recommended by the M&E Working Group and ultimately approved by the STAG. Since the approval, significant work has been underway to create guidelines for conducting coverage surveys for preventive chemotherapy. An excel tool was created to improve the usability of the tool and online learning modules are currently in the works.
Alternative approaches to coverage surveys (Uganda)
To compare coverage evaluation methods to identify a method that is statistically rigorous and feasible for programs. This study will focus on assessing MDA coverage for lymphatic filariasis by comparing the cost, time and feasibility of 3 different methods: the EPI approach (n=1768), LQAS design (n=95) and probability sampling alternatives (n=1768).
Primary Findings and Lessons Learned
Coverage surveys are an important tool for programs to evaluate their reporting systems and to determine whether effective MDA coverage has been achieved. However, for various reasons coverage surveys are seldom implemented. Some key challenges are: perceived technical difficulty, lack of resources, and lack of standardized guidance on how to conduct coverage surveys. This protocol seeks to address the 1st and 3rd points by comparing the feasibility of three different coverage survey methods (EPI approach, LQAS, and segmentation). This study was completed in 3 districts in Burkina Faso. All 3 districts found that their survey coverage was above the WHO target threshold (65% for LF). Furthermore, in all 3 cases the survey coverage validated (or nearly validated) the reported coverage. Taken together this suggests that the Burkina Faso program is working well. The feasibility results found all 3 methods to be very similar with regards to time, cost and perceived difficulty. Because only the segmentation approach results in a probability sample, this method was recommended by the M&E Working Group and ultimately approved by the STAG. Since the approval, significant work has been underway to create guidelines for conducting coverage surveys for preventive chemotherapy. An excel tool was created to improve the usability of the tool and online learning modules are currently in the works.