PolicyLab announces the release of our latest Evidence to Action brief, Evaluation of Maternal and Child Home Visitation Programs: Lessons from Pennsylvania. This brief draws on lessons learned through PolicyLab's evaluation of the Pennsylvania Nurse-Family Partnership, and presents key concepts and recommendations for public health program evaluation.
As many maternal and child home visitation programs scale up under the Affordable Care Act, and as many policymakers are calling for greater connection between government funding decisions and public health program outcomes, the importance of evaluation to demonstrate program effectiveness has grown. Yet evaluating the effectiveness of these programs can be challenging. The complexity of program models, coupled with considerations of real-world program implementation, requires thoughtful and flexible evaluation strategies. Despite challenges, a well-planned evaluation and follow-up can significantly contribute to the impact of programs that serve an increasing number of families and communities.
This Evidence to Action brief, part of PolicyLab’s ongoing work in maternal and child home visitation program evaluation, highlights four key program evaluation concepts to help guide home visitation program administrators plan real-world evaluations.
Key Concept 1: Real-world evaluation results will reflect implementation environments with fewer supports, resources, and standardization in comparison to randomized trial environments.
Response for Evaluation: Set realistic target outcomes, knowing that results from studies conducted under experimental conditions will likely demonstrate larger effects relative to evaluation results achieved under real-world conditions.
Key Concept 2: Program performance is altered by the local context of the implementing site and the community it serves.
Response for Evaluation: Collect data at the site level, knowing that some sites will be more effective than others, depending upon local resources and baseline population differences. Learn from program outliers, both those that exceed expectations and those that underperform.
Key Concept 3: Program effectiveness increases over time following wide-scale implementation.
Response for Evaluation: Select evaluation benchmarks that account for expected changes in effectiveness over time, including the likely lag from implementation to effectiveness.
Key Concept 4: Engaging stakeholders enriches program evaluation.
Response for Evaluation: Include program stakeholders in the evaluation process to inform the interpretation of findings for targeted quality improvement.
Matone M, O'Reilly A, Luan X, Localio R, Rubin D. Home visitation program effectiveness and the influence of community behavioral norms: a propensity score matched analysis of prenatal smoking cessation. BMC Public Health. 2012;12(1):1016. doi: 10.1186/1471-2458-12-1016
Matone M, O’Reilly A, Luan X, Localio R, Rubin D. Emergency department visits and hospitalizations for injuries among infants and children following statewide implementation of a home visitation model. Maternal and Child Health Journal. Published online December 2, 2011. doi: 10.1007/s10995-011-0921-7
Rubin D, O’Reilly A, Luan X, Dai D, Localio R, Christian C. Variation in pregnancy outcomes following statewide implementation of a prenatal home visitation program. Archives of Pediatric and Adolescent Medicine. Published online November 1, 2010. doi:10.1001/archpediatrics.2010.221