Using pre- and post-test methods to measure program outcomes
Abstract
Pre- and post-tests document the services of a program, measure outcomes, and demonstrate success. Pre- and post-tests are useful not only for obtaining impact data for progress reports required by funders, but they also act as a thermostat, providing real-time feedback of program efforts, to help program directors decide whether or not to make changes in the implementation of activities throughout the program year. This effective practice, submitted in October 2002 by Donald J.Pratt with Project STAR, addresses the issues of how and why to use pre- and post-test methods to measure program outcomes.Issue
Selecting an appropriate approach to measuring program outcomes is part of overall program accountability and therefore a decision program directors face. Understanding how and why to use measurement instruments is part of sound evaluation practices.Action
According to researchers at Project STAR, using pre- and post-test instruments in the following ways enhances measurement results and overall program accountability:- Select the type of instrument that gathers the most direct information about the impact of the program. The more specific an instrument is to the activities performed, the more likely the evaluation will produce useful results.
- Consider who the respondents are, as well as the type and quantity of information needed. Pre and post tests work best when administered to the same respondents at the beginning and at the end of service, e.g., a student who will be tested at the beginning and end of the school year or participants who are tested at the beginning and end of a one day course.
- Employ quantitative or selection-type responses that can be easily obtained and compared during analysis. For example, pre and post tests work well for grades, test scores, scaled items, and yes/no questions because the numbers can be more easily obtained and compared during data analysis. Qualitative or open-ended questions, such as, "How can we improve our services? " or "What about our services did you like? " would be much more difficult to analyze and compare pre and post service.
- Avoid having the measurement instrument impose an unreasonable burden on program staff or participants. The difficulty an instrument presents depends both on the type and length of service people are receiving and on the level of skill of the people taking or administering the instrument. Even a short written survey may present an undue hardship for participants who have difficulties reading, while longer observations, interviews or phone surveys may not present a problem at all.
- Exercise care in tracking and organizing during data analysis, because final scores from the post test of one person need to be compared to the initial scores from the pre test of that exact same person to reach helpful conclusions.
Context
Pre- and post-tests help answer the question: "What changed from the beginning of the program to the end? " If program objectives include standards of success such as "Eighty percent of youth will increase in their knowledge of water conservation," then a pre- and post-test will provide the most concrete and direct evidence of impact because it provides a baseline, captured before service took place, that can be compared with post-service scores. Other methods such as post-only questions e.g., "What do you know about water conservation now that you didn't know before participating? " may not measure changes as accurately as pre and post tests, or may not give information on increases at all.
Project STAR is a Corporation for National and Community Service-sponsored technical assistance provider in evaluation and available to assist National Service programs with evaluation endeavors to assess the impact of their services. Project STAR is a project of Aguirre Division, JBS International, Inc., which provides high quality social issues research, evaluation and technical assistance for government, business and nonprofit sectors.
Outcome
When pre- and post-tests are utilized:- Impact data for progress reports required by funders is obtained.
- Immediate feedback of program efforts is gained.
- Overall program accountability is increased.
Posted On
October 2, 2002For More Information