Field Experiments in Economics: Volume 10

There are several ways to define words. One is to ascertain the formal definition by looking it up in the dictionary. Another is to identify what it is that you want the word-label to differentiate.

If we are to examine the role of “controls” in different experimental settings, it is appropriate that the word be defined carefully. The Oxford English Dictionary (Second Edition) defines the verb “control” in the following manner: “To exercise restraint or direction upon the free action of; to hold sway over, exercise power or authority over; to dominate, command.” So the word means something more active and interventionist than is suggested by it’s colloquial clinical usage. Control can include such mundane things as ensuring sterile equipment in a chemistry lab, to restrain the free flow of germs and unwanted particles that might contaminate some test.

The results of standard lab experiments have long been questioned because of the convenience samples of subjects they typically employ and the abstract nature of the lab settings. These two characteristics of experimental economics, it is argued, are the key factors that endanger the external validity of experiments.

Researchers have tried to address these issues by bringing the lab to non-traditional subjects including participants in remote locations, and/or by moving the setting of experiments closer to reality by using real goods and/or settings that are not stripped of context.

While field experiments might help experimental economists to increase the external validity of their investigations, these potential benefits might come at costs that can be considerable. Specifically, going into the field can dramatically increase the demands on, and challenges to, experimental control. This is particularly true for experiments in small-scale societies in remote locations on which I focus in this article.

We discuss the following three themes on the use of field experiments to study economic development: (1) We summarize the arguments for and against using experiments to gather behavioral data in the field; (2) We argue and illustrate that field experiments can provide data on behavior that can be used in subsequent analyses of the effect of behavioral social capital on economic outcomes; and (3) We illustrate that field experiments can be used as a development tool on their own to teach communities about incentives and strategic interaction.

We design experiments to jointly elicit risk and time preferences for the adult Danish population. The experimental procedures build on laboratory experiments that have used traditional subject pools. The field experiments utilize field sampling designs that we developed, and procedures that were chosen to be relatively transparent in the field with non-standard subject pools. Our overall design was also intended to be a general template for such field experiments in other countries. We examine the characterization of risk over a wider domain for each subject than previous experiments, allowing more precise estimates of risk attitudes. We also examine individual discount rates over six time horizons, as the first stage in a panel experiment in which we revisit subjects to test consistency and stability of responses over time. Risk and time preferences are heterogeneous, varying by observable individual characteristics. On a methodological level, we implement a refinement of existing procedures which elicits much more precise estimates, and also mitigates framing effects.

We explore the predictive capacity of short-horizon time preference decisions for long-horizon investment decisions. We use experimental evidence from a sample of Canadian working poor. Each subject made a set of decisions trading off present and future amounts of money. Decisions involved both short and long time horizons, with stakes ranging up to 600 dollars. Short horizon preference decisions do well in predicting the long-horizon investment decisions. These short horizon questions are much less expensive to administer but yield much higher estimated discount rates. We find no evidence that the present-biased preference measures generated from the short-horizon time preference decisions indicate any bias in long-term investment decisions. We also show that individuals are heterogeneous with respect to discount rates generated by short-horizon time preference decisions and long-horizon time preference decisions.

To investigate the external validity of Ultimatum and Dictator game behavior we conduct experiments in field settings with naturally occurring variation in “social framing.” Our participants are students at Middlebury College, non-traditional students at Kansas City Kansas Community College (KCKCC), and employees at a Kansas City distribution center. Ultimatum game offers are ordered: KCKCC > employee > Middlebury. In the Dictator game employees are more generous than students in either location. Workers behaved distinctly from both student groups in that their allocations do not decrease between games, an effect we attribute to the social framing of the workplace.

This paper demonstrates how economic field experiments may offer researchers a method to quickly assess policy outcomes that otherwise are difficult to measure. We compare lottery winners to losers of a privately run educational voucher program to measure the program’s effect on confidence. We measure confidence on academic ability using protocols developed to assess the educational program. We find that confidence does not differ robustly between winners and losers. Among non African-Americans, however, winners were significantly less overconfident than losers in predicting their academic achievement test scores. We also find older children are significantly more confident in their abilities.

Field experiments have raised important issues of interpretation of bargaining behavior. There is evidence that bargaining behavior appears to vary across groups of populations, such as nationality, ethnicity and sex. Differences have been observed with respect to initial behavior and with respect to the adjustment pattern over time. Often, such behavioral differences are referred to as cultural, although the delineation of the cultural group has been confined to one or other observable characteristic in isolation. We show that this way of characterizing cultural differences is overly simplistic: at best, it leads to unreliable claims; at worst, it leads to erroneous conclusions. We reconsider the evidence provided by previous experiments using ultimatum game rules, and undertake new experiments that expand the controls for demographics. The lesson from our demonstration is that the task of designing experiments for the field offers many challenges if one wants to define and control for cultural impacts, but that field experiments also offer potential for providing new insights into these issues.