SELECTING A CONTROL GROUP IN EVALUATIONS/ RANDOMIZED CONTROL TRIALS

There has been debate in various quarters of the Research, Monitoring and Evaluation field over the importance of control groups particularly when planning or conducting evaluation. For many project managers, the process of planning both mid-term and end-term evaluations remains a challenge. The big question for them always is whether to use a control group and subsequently, how to choose a suitable one. Let’s take a look at what a control group is, the need for one, and how to select an appropriate group.

What is a control group and why is it important?

When rolling out an intervention targeted at any particular group, there are set objectives that need to be accomplished. These are usually the expected outcomes of the project. A control group is usually a group that is homogenous to the targeted group, but which does not benefit from the intervention. A control group serves to establish that the reported outcomes of an intervention can indeed be attributed to that intervention and not external factors. Indeed, without a control group the impact of an intervention may be misreported. Most of the time, without a control group, the impact of a project is overestimated, sometimes it’s underestimated.

Take for example a project on condom distribution whose objective is to improve HIV and AIDS prevalence. The project is targeted at Location A with homogeneous characteristics and HIV prevalence of 25%. If the project decides to target distribution of condoms at Sub-location A1 and A3 but not A2, then the sub location A2 may be considered as a control group.  If at the end of the intervention A1 and A3 report a decline in prevalence of HIV to about 20% while there is also a decline in HIV prevalence in A2 to 23%, then the impact of condom distribution on reducing HIV prevalence in that location would be 3% and not 5% as would be wrongly interpreted if there was no control group. If however, sub-location A2 reported an increase in prevalence to 26%, then the impact of the project would be 6%.

NB: An important point to note is that a control group must be identified before the intervention is conducted.

The need for a control group/ when to select a control group

The need to use a control group in evaluation depends on first and foremost, the type of intervention that is being rolled out. For some interventions, it may be possible to have a control group while for others, it simply isn’t. This is especially true for community targeted interventions versus household targeted interventions. For household targeted interventions, it may be possible to have a control group while in community targeted interventions, it’s difficult to establish a control group since everyone is expected to benefit. Take for example, a household cash transfer project to promote food security. In this case, it’s possible to have a control group since the households that do not receive cash transfers may be used as a control group. However, in the case of an advocacy project targeted at a community, it may be difficult to establish a control group since you cannot restrict the flow of information to a particular group.

The second consideration for using a control group is the ethical consideration. In some cases, it is not ethical to exclude participants from an intervention programme. This is especially so when life is at stake. A perfect example is in humanitarian emergencies. When offering humanitarian assistance, saving life is the most important consideration. It would be thus unethical to decide to exclude one group from an intervention for the sake of measuring impact.

Other considerations to make before deciding whether to use control groups include the feasibility of establishing homogeneity, costs involved vis-à-vis the budget, donor requirements, the scope of the intervention in terms of geographical stretch among others. At the end of the day it is upon the manager, based on organizational needs, to consider if the intended intervention requires using a control group.

Selecting a control group

Now that we understand what a control group is and the need for it, just as important to understand is how to select a control group. We will use the cash transfer project to understand this process. In this project we will attempt to establish whether household cash transfer to food insecure households, impacts on their food security. To understand more on cash transfer click here or here.

The first consideration to make when selecting a control group is to ensure that it is homogenous to the population for which the intervention is targeted. For our case, once we have identified the area where we wish to make the intervention, the next step is to establish a list of the food insecure households, since they are the intended beneficiaries of the project. This becomes the targeted population (let’s take 20,000 for our case).

Next is to select a sample which is to be targeted through monthly cash transfers, also known as the intervention group. This is especially in the case where all the identified households cannot be adequately met by the intervention. If the project targets 2000 households, then this becomes the intervention group. The intervention group needs to be randomly selected to increase the validity and reliability of the evaluation findings

Third is to select the control group. This group also needs to be selected randomly. However, they will not receive any form of intervention in form of cash transfers. No standard practice exists for calculating the size of the control group but common sense dictates that it needs to be proportionate to the sample. The rule is, the bigger the size, the better.

When conducting evaluation, the same tool is used for both the intervention group and the control group to establish the changes in both groups. The impact of the cash transfer project would therefore be the change in prevalence of food insecurity of the control group subtracted from the change in prevalence of food insecurity of the intervention group.