Evaluating “cart-before-the-horse” programs
Until recently, the adage of “putting the cart before the horse” was commonplace when describing funding programs. It means organizations have put together programs—including experiments, classes, trials or any event that requires evaluation—quickly before funding spending limits kick in. When this happens, the good intentions of measuring the program’s impact and outcomes isn’t considered during the program implementation phase. If measurements are taken when the program ends, the organization has to take the time to step back to evaluate progress. Conducting an evaluation at the end of a program, instead of collecting it along the way, causes valuable information to be lost. This includes the loss of key people, such as participants, managers and others who can provide feedback during the program.
Even though the “cart-before-the-horse” method is not the best way to conduct program evaluation, it can still be done. Here are some simple strategies to help in collecting data after a program has ended:
- Bring together the key individuals who are responsible for the services provided to your target audience.
- Review the organization’s commitments to funders to identify areas that need to be measured.
- Review the organization’s vision, mission and goal statements.
- Determine priorities and develop plans for evaluating program components.
- Based on prioritized plans, decide how extensive you want your evaluation to be.
- Develop an evaluation plan or hire a consultant to develop the plan for your program/services.
The evaluation may be as simple as reviewing training logs and attendance sheets, or it can be as complex as developing evaluation tools to measure the short or long-term changes in knowledge, attitude and behavior of the individuals who have benefited from your program.
The evaluation component that your program may have lost putting the “cart before the horse” is the opportunity to collect baseline data. This means you won’t be able to show the changes that have happened to your participants since they began your program. This could be a problem if your program is of the “one-shot” variety.
However, if your program accepts new participants on a continual basis, then you’ll have a chance to create the tools and collect the baseline and follow-up data from the newcomers. In addition, if your program has the capability to follow clients over time, you could create tools to measure the “sustained” changes in your clients.
In the end, one thing to remember is when planning new programs is to involve all team/program players as implementation begins. Use their input to develop the evaluation components in the early stages of your program. This way the tools, protocols and trained staff are prepared to measure the results of your well-planned program.
