Choosing an Evaluator and Developing a Scope of Services
Being a program evaluator is a lot like being an explorer, you never know what or who you will find until you get to your destination. Before your journey begins it is important to find the best possible guide/program evaluator.
When selecting an evaluator/guide, program planners often assume that they should look for someone with strong technical skills. This is only half of the story. It is important that the evaluator also possess the following skills or qualities:
- The person you select as an internal or external evaluator needs good communication skills (otherwise, you risk a finished evaluation project that will not be useful and may damage your program). Please see the tip sheet How to Do An Internal or External Evaluation for more information.
- You should be comfortable with the person you select as an evaluator, especially if you need to work with them for several years of the project's life. One generally keeps the same evaluator throughout the life of the project.
- The evaluator should be familiar with the type of program and population you served.
- Your evaluator should believe in what the agency is trying to accomplish.
- The evaluator should have a history of community involvement if your program works with the community.
- Your evaluator should possess passion and strongly believes in and is excited about what you are doing!
Before making a final selection of an evaluator, ask to see samples of previous reports or research completed by the candidate. Look for reports that you can understand. They should be written in a clear and concise manner. Ask for references and information about other evaluation projects that they have completed.
Once you have tentatively identified an internal or external evaluator, the next step should be to work with your evaluator to develop a Scope of Services. A detailed (usually two to five pages) Scope of Services will serve as a framework for the evaluation. The scope should include the following components:
- Program background: This should include a comprehensive program description, population served and specific services and activities offered.
- Focus of the planned evaluation: The focus should let readers know which program parts will be studied in detail. There should be a description of the procedures to be used in the evaluation process.
- Deliverables: This section should include a description of surveys, etc. to be carried out, and reports and summaries to be provided by the evaluators. These deliverables should be specific and concrete.
- Budget: A detailed budget should be included. The amount budgeted for evaluation often ranges between 5% and 10% of the program budget.
- Timeline: A working timeline should be included that describes the duration of the evaluation process and date deliverables is due.
Once the Scope of Services is agreed upon, then the real work begins! The evaluator or evaluation team begins to organize existing program data, collect new data, design logic models, and report on ongoing investigations of the program and its successes and challenges. The evaluators may look at Best Practices (please see Comparing Your Program Against the Best: Using "Best Practice" in Evaluation or Community Wide Indicators). You also will want to be thinking about how you will use the evaluation findings (please see Now We Have Some Findings: How Do We Use Them to Improve the Program) and about ways that you want the evaluation findings reported to you and your team so that the information will be meaningful (please see It's Not Enough to Collect the Data: Presenting Evaluation Findings So That They Will Make A Difference).
by Mary France