Monitoring Community Development

by WANEP
(Freetown, Sierra Leone )

Visitor Question: How do you monitor and evaluate a strategy for community development?

Editors Reply: Thank you for a great question. We think that both monitoring and evaluation should be based on numbers to the extent possible. Even highly subjective traits of individuals, such as job satisfaction or identification with a cultural or ethnic group, need to be reduced to a numerical scale to produce meaningful monitoring and evaluation.

We just posted an overview article on evaluation on this site (www.agoodcommunity.org/community-development-evaluation.html), so here we will confine our comments to community development strategy monitoring.

It is important to set up any strategy or program with the data points to be monitored determined in advance. Otherwise we are likely to bias the results by consciously or unconsciously choosing to monitor aspects of the strategy that yield positive results.

Before you begin, determine what is worth monitoring and how those things will be measured.

First, consider the goals of the community development strategy. You should not claim you have a strategy if you do not know what your goals are, or for that matter, the more specific objectives of the programs you design to address those goals.

Second, tentatively select between one and three pieces of data that relate to each objective. In the real world it is highly unlikely that you will have the staffing and budget to monitor every variable that you consider relevant to the performance of your community development strategy. You will have to make choices, but start your exploration with as many as three measures for each objective, and then narrow the list based on data availability, reliability, and expense to gather and process.

If an objective is one-dimensional, that choice might be easy. For instance, if your objective is that each adult in your community reads at an eighth grade level, then your data point is the reading level of every adult resident.

Immediately you see that even when you have only one variable that is relevant to your objective, actually obtaining the data may be a major challenge. If there is a reliable national census or private data-gathering agency, the task may be relatively simple, but often data must be gathered from a variety of sources and the correct values to be used for some data must be estimated.

In our example, providing you cannot test every adult, what pieces of relevant data do you have? Are they sufficiently related to the reading level of the entire population of adult, or just some sub-group of adults? How do you know?

If you are not satisfied that you can collect data that are reliable enough to make the effort worthwhile, you now face the possibility that you must simply omit that objective from your monitoring project, or you must choose a not-so-obvious type of available data that research shows to be highly related to reading level. Maybe in your community, that would be the number of years of formal education. If data on this substitute variable is much more available, as it would be in the U.S., then you can decide to use that, or you can decide that you don’t trust the correlation between adult reading level and number of years spent in school, and look elsewhere.

Third, collect the data, carefully documenting any reason you suspect the accuracy or relevance of the data. Remember that since you ask about monitoring of a strategy, it is necessary and appropriate to have a baseline level for comparison if possible. If you have data for a year or two--or a decade or two--before the adoption of your community development strategy, collect and analyze that data as well.

We should say a few words about translating qualitative factors into data. Let’s suppose that one of your objectives is to improve housing quality in your village. In this example, you would have to agree on ways to quantitative ways to measure housing quality, such as awarding one point each for indoor plumbing, electricity, vermin-proof floors, roof not sagging, roof not leaking, walls without holes, or whatever is important in your culture and setting. Then you may not have the budget to survey every dwelling, but maybe you have young people who can look over every tenth housing unit.

Or let’s say that your objective is to increase job satisfaction among merchants in the market. Now you must ask those merchants some questions, which may include directly rating their job satisfaction on a scale of 1 to 10, or you might ask more indirect questions, such as whether they would like to be doing the same work five years from now, or whether they would change jobs if there were another opportunity offering the same income. If you are clever, you may figure out how to monitor behavior relevant to job satisfaction, rather than basing your monitoring on self-reports.

Fourth, analyze what the data are telling you about the strength of your interventions as part of the community development strategy. Your computer cannot fully accomplish this task. You must apply your own intelligence to what else, other than your community development strategy, may be leading to the results you believe you are seeing.

The particular difficulty of monitoring and evaluation in the community development field, compared to monitoring the achievement of school children for instance, is that community development results often are spread over a number of years, and that positive results tend to be clustered together in time, followed by long periods of absorption and regrouping.

Incidentally, for more about formal community development evaluation, see our evaluation page.

Monitoring must occur frequently enough that the results can lead to strategy or program modifications, but not so frequently that little else is achieved other than monitoring. Likewise the data collected during monitoring must be analyzed and then summarized into usable results often enough that the results actually improve program performance.

Join GOOD COMMUNITY PLUS, which provides you monthly with short features or tips about timely topics for neighborhoods, towns and cities, community organizations, and rural or small town environments. Unsubscribe any time. Give it a try.