Looking for Other Kinds of Existing Data: Using Community-wide Indicators in an Evaluation
When conducting an evaluation of a program, community wide indicators can be useful tools. More and more program are turning to community indicators to assess current conditions and eventually whether desired conditions or improvements have been achieved as a consequence of the various programs in the city (for more information on current and desired conditions and their relations to programs, please see "Understanding the Overall Program: Logic Models Make it Easier to Run Toward the Goal.")
What are community wide indicators you ask? Community indicators are statistics that tell us about the social, economic or environmental pulse of the community. For example, if you are evaluating a program whose aims are to increase adult literacy, you could look for community statistics of adult literacy rates. You may find that you want your evaluator to include community-wide indicators in an evaluation of your program. Often funders are interested in these indicators (both in showing the need for a program and showing that the program's impact).
Where to start in finding indicators?
Where can you find community wide indicators? Much data about communities is available from Federal, State and Local agencies. For example, if you want to find out about a community's rates of poverty, you can look at the U.S. census data at your local library. If you have access to the Internet, you can locate much information about a community. Many agencies that provide statistics about communities such as the Department of Education, the Department of Mental Health have web pages.
How can indicators be helpful?
How can community wide indicators help with an evaluation? Community wide indicators can be one source of information about the progress that a program is making towards achieving its stated goals. Let's say that you are evaluating a program that helps keep youth out of gangs. In addition to gathering information directly from the program participants who might say that they haven't joined any gangs since taking part in the programs activities, you may also want to look at indicators of gang activity in that community through local law enforcement agencies. This will give you another source of information.
Community wide indicators can also help you measure whether a program has had a positive impact within a specific population group or upon the larger community. For example, suppose you are evaluating the "Jobs for Peace Program" that assists Vietnam Veterans secure jobs in their local communities. You decide to look at statistics from various sources such as the Department of Employment and Training, or other federal/state agencies that provides statistics about Vietnam Veterans. After looking at several employment indicators, you find that the program you are evaluating could have contributed to a decline in the unemployment rate of Vietnam Veterans in the community. This approach is one way of determining that the program you are evaluating is in fact making progress toward its goals.
What are the limitations of community wide indicators?
Now that I have told you all the great things you can with community indicators it us time to tell you what the limitations are!
Community wide indicators are not always easy to locate. Sometimes you have to hunt down the information you are looking for. Or you may have to go to more than one place to find the data you are looking for.
Some community wide data may not be relevant to the program you are evaluating. For example, you are evaluating a program that provides transportation to the elderly. You look for community wide data that points to the number of elderly who do not have cars, instead you find statistics that tell you the number of elderly people who are in nursing homes!
Community wide data is not always the most up to date. Oftentimes you will encounter data that is several years old, as is the case with U.S. Census indicators.
Community wide data may have been gathered for only one point in time. The data can tell you about the need for a program, for example, but will not tell you if the program has led to any change.
Changes in community indicators may not be due to your program. If other programs are operating or if other things have changed in the community, these factors can have led to the declines rather than your program being responsible.
by Beata Murrell