What is random assignment?

It is widely accepted that the most reliable estimates of a program’s impacts are produced through social experiments, the defining design feature of which is random assignment.

The technique of random assignment is a powerful tool for determining the effectiveness of new policy ideas. To know what difference a program makes you have to know what people would have done on their own without the program.

In evaluating a welfare-to-work program, for example, we must recognize that people leave the welfare rolls all the time through their own efforts and with the assistance of existing programs and services. In isolation, information on outcomes always overstates program achievements because all positive developments are counted as program accomplishments, such an analysis cannot identify the extent to which any of the outcomes actually result from the program rather than simply representing what people would normally do on their own. Policy-makers need to know the difference that the program makes. It is this difference that measures the program’s impact — the change in an outcome that results only from the program.

By identifying a comparison group that closely resembles those who take part in the program, we can determine what people would do on their own, regardless of the economic environment or other factors. The best way of creating a comparison group is by means of random assignment. We start with a group of people, all of whom would meet the selection criteria of the new program, and then randomly decide whether each person will be assigned to the group that will be eligible for the program or to a group that will not be eligible. Those assigned to the latter group provide the comparison for evaluation purposes. When random assignment is used they are called a “control group.”

In general, random assignment is the most effective way of ensuring that program and control group members are virtually identical in all respects, such as their employability, education, and past welfare history. More importantly, it ensures that the two groups are virtually identical in unmeasured (and perhaps unmeasureable) characteristics, for example, intelligence and motivation. When the experiences of the program and control groups are compared over time, we can be confident that any differences that we see between the two groups can be attributed to the program.

Not only does random assignment produce the best possible comparison group for measuring impacts, but selecting people randomly is also often the fairest way of allocating scarce places in the program. Rather than a “squeaky wheel” approach (which favours the most vocal) or a “first-come, first-served” approach (which favours those who can most easily make arrangements for child care or transportation) or “creaming” (selecting the people that program administrators perceive to be the “best” candidates), random assignment offers everyone in the specified target group an equal chance of being selected for the program.

The rigorousness of SRDC’s research means that policy-makers and practitioners can have confidence in the findings. As a result, time and energy can be spent determining the policy implications of the research rather than debating whether the findings are plausible.