| Session Title: Evaluating Health Improvement Collaboratives Using a Systems Thinking Developmental Evaluation Approach |
| Multipaper Session 786 to be held in Washington Room on Saturday, November 10, 12:10 PM to 1:40 PM |
| Sponsored by the Health Evaluation TIG |
| Chair(s): |
| Tessie Catsambas, EnCompass LLC, tcatsambas@encompassworld.com |
| Abstract: The USAID-funded Quality Assurance Project that aims to develop and apply methods for improving healthcare quality in developing countries. The session shows how a combination of internal and external developmental evaluation helped to document the emerging model of rapid collaborative improvement of healthcare quality in developing countries. Developed by the Institute for Healthcare Improvement in the mid-90s, the improvement collaborative is a major new approach for rapidly improving the quality and efficiency of healthcare. By applying systems thinking, the evaluation tested program assumptions, definitions, and mental models to facilitate learning throughout the evaluation. A -lessons learned week- brought together managers of 17 collaboratives from Russia, Rwanda, Niger, Uganda, Tanzania, Eritrea, Benin, Honduras, Nicaragua, Ecuador, and South Africa. Joint client/evaluator teams collected data in 6 countries where they visited health sites, government, and other stakeholders. The evaluation report explored the added value of collaboratives, factors for collaborative readiness and sustainability. |
| Applying the Collaboratives Model to Developing Country Settings |
| Jorge Hermida, University Research Company LLC, jhermida@urc-chs.com |
| Mary Gutmann, EnCompass LLC, mgutmann@encompassworld.com |
| The improvement collaborative, a major new approach for rapidly improving the quality and efficiency of healthcare, was developed by the Institute for Healthcare Improvement in the mid-90s. This model underwent its first adaptation by QAP when before it was introduced to developing country settings. Up to that point, collaboratives had been applied only in wealthier US and European countries with decentralized and privatized healthcare systems. By contrast, in developing countries, collaboratives were set up in the context of resource poor, centralized healthcare systems. As the project evolved, major lessons emerged for managers of individual collaboratives resulting in further adaptations of the model. Challenges, lessons and good practices were shared among collaborative managers resulting in further adaptations still. Key concepts include: demonstration and spread collaboratives, norms and standards, change package, learning sessions, measuring and reporting, coaching, PDSA cycles, and continuous quality improvement. |
| Investing in Collaboratives |
| Lynne Franco, University Research Company, LLC, lfranco@urc-chs.com |
| Strategically, the United States Agency for International Development decided to invest in collaboratives based on the work of the Quality Assurance Project over a 10-year period. Quality improvements that focused on creating specific healthcare improvements were unable to help the world tackle larger more universal and acute problems related to maternal and child health (HIV, TB, Malaria, and others). USAID made a strategic decision to support approaches such as collaboratives that promised rapid, evidence-based quality improvement. Lessons from the work of the Quality Assurance Project and this evaluation have important implications for developing country healthcare improvement. |
| Evaluating Collaboratives |
| Tessie Catsambas, EnCompass LLC, tcatsambas@encompassworld.com |
| An outside evaluation team was engaged to implement the evaluation of collaboratives early on. The evaluation team worked alongside implementers helping to document the emerging model, insights from implementation, changes in definitions and approaches, challenges, and breakthroughs. Many things evolved during project implementation e.g. the use of experts for agreeing on quality standards, coaching, spread strategies, nature of team-based improvement. System thinking was used to test assumptions with individual collaboratives managers and the group as a whole. In a lessons learned week, the evaluation team facilitated -depth conversations on achievements, challenges and lessons. To ensure that learning occurred within the project, evaluation team members paired up with collaboratives managers to conduct field visits. Collaborative managers visited countries where there were not working for independence, deepening collaboration and learning among project staff. The resulting evaluation report explored questions related to the added value of collaboratives, factors for collaborative readiness, and sustainability. |