Quantifying Uncertainty in Intervention Effectiveness: An Application to Obstetric Fistula

Authors: Abigail Colson, Sweta Adhikari, Ambereen Sleemi, Ramanan Laxminarayan


Intervention effectiveness is a major input of evidence-informed priority setting in healthcare, but information on intervention effectiveness is generally lacking. This is particularly problematic in the context of poorly resourced health care settings where even efficacious interventions fail to translate into improvements in health. The few intervention effectiveness studies focus on the experience of single facilities and do not consider the impact of multiple factors that may affect health outcomes.

Methods and Findings:

We use the Cooke method of structured expert judgment to assess the effectiveness of surgery to correct obstetric fistula in a low-income setting. This method has been used to quantify uncertainty in the areas of engineering and environmental risk assessment when data are unavailable. Expert judgment is the basis for irreversible decisions with far reaching consequences, but new to the realm of public health. Under this method, experts quantify their uncertainty about rates of long-term disability in fistula patients following treatment in different contexts, but the information content drawn from their responses is statistically conditioned upon the accuracy and informativeness of their responses to a set of calibration questions. Through this method, we develop best estimates and uncertainty bounds for the rate of disability associated with each treatment scenario and setting.


Estimates developed using the Cooke method were statistically superior to those involving a simple averaging of expert responses. We find that the Cooke method is a viable approach to investigating the effectiveness of medical interventions where randomized controlled trials are not possible. Understanding the effectiveness of surgery performed at different types of facilities can guide program planning to increase access to fistula treatment.